This is an open access article distributed under the terms of the Creative Commons Attribution License (The concept of attitudes has always been one of social psychology's most central research topics (Vogel & Wänke, 2016). Attitudes not only predict future behavior (Ajzen, 1991), but they also influence the formation and cohesion of social groups (Bohner & Dickel, 2011). Therefore, it is important to understand the environmental regularities leading to the formation of attitudes, as well as the underlying cognitive processes. In that regard, one prominent and robust finding is that people change their attitudes towards people and objects based on their mere co-occurrence with positive or negative objects in the environment. This change is known as the evaluative conditioning (EC) effect (for reviews, see Hofmann et al., 2010; Moran et al., 2023), defined as the change in the liking of a conditioned stimulus (CS) due to its pairing with positive or negative unconditioned stimuli (USs; De Houwer, 2007). For example, encountering an unknown stranger (CS) in the company of a good friend (positive US) may be enough to make the stranger more likable.
EC has received much attention in social psychology because stimulus pairings are arguably frequent in everyday life (Alves et al., 2020). EC effects have also been found across various domains, leading to attitude change toward brands (Ingendahl, Vogel, et al., 2023), health behaviors (Tello et al., 2018), and individuals and groups (Alves et al., 2020; Hütter et al., 2012). Therefore, it is important to understand when, how, and why stimulus pairings lead to changes in stimulus evaluations.
In the present research, we delve into the cognitive processes underlying attitude formation via EC. We extend prominent memory-based process explanations (Gast, 2018; Stahl & Aust, 2018) by connecting them with research on metacognitive experiences. Specifically, we investigate whether the subjective ease of retrieving a stimulus pairing from memory influences the size and potentially even the direction of the EC effect at the stage of information integration (i.e., the CS evaluation). Doing so not only furthers our understanding of the processes underlying EC but also reveals potential boundary conditions of the role of memory in the EC effect. In the following sections, we will first discuss the role of memory in EC, focusing on the retrieval stage, before we elaborate on the ease-of-retrieval effect.
Memory and Evaluative Conditioning
One central question in EC research has always been to what extent EC effects depend on the memory of the stimulus pairings. Whereas some earlier findings and theories on EC indicated that EC effects can occur without explicit memory (Jones et al., 2009; Olson & Fazio, 2006; Walther & Nagengast, 2006), the last decade of EC research has accumulated strong evidence for the importance of explicit memory in EC. A large body of research demonstrates that explicit memory of the stimulus pairings is one of the strongest moderators for EC effects (Bar-Anan et al., 2010; Hofmann et al., 2010; Ingendahl, Ingendahl, et al., 2024; Ingendahl, Woitzel, Propheter, et al., 2023; Mierop et al., 2017; Pleyers et al., 2007; Stahl et al., 2009, 2024). Accordingly, current process theories on EC emphasize the role of memory in EC (Gast, 2018; Hütter, 2022; Stahl & Aust, 2018). Most of these models postulate that EC effects require the formation of memory traces when encountering a stimulus pairing, the maintenance of this memory trace over time (i.e., storage), and later (partial) retrieval of the pairing upon future encounters with the CS to influence the CS’s evaluation.
Typically, research on the role of memory retrieval in EC focuses on memory content. Memory content in EC is defined as either identity memory (i.e., memory for the individual US that was paired with the CS) or valence memory (i.e., memory for the US valence; Pleyers et al., 2007; Stahl et al., 2009), with the latter being considered more important for the EC effect (Stahl et al., 2009).
The main question in previous memory research in EC has been to what extent EC effects can emerge without explicitly remembering the contingencies (e.g., Hütter et al., 2012; Pleyers et al., 2007). Other research has investigated which specific memory content can be memorized better or worse (Alves et al., 2020) or which conditions boost encoding, maintenance, and retrieval of the memory content in EC (Gast et al., 2016; Richter & Gast, 2017). For example, EC effects increase from distributed learning (Richter & Gast, 2017) but decrease from a larger time interval between learning and retrieval (Hütter et al., 2012). Notably, memory in EC is also regularly used as a tool to inform theorizing in EC research. Memory measures have been implemented to gain a better understanding of basic learning principles in EC (Alves et al., 2020; Gawronski & Walther, 2012; Reichmann et al., 2023).
Although the explicit consideration of memory processes has been useful for theorizing as it helped in prominent theoretical debates such as dual- versus single-process theories in EC research (Gawronski & Bodenhausen, 2018; Hütter, 2022; Stahl & Aust, 2018), we argue that the current focus on memory content in previous EC research is yet incomplete. Specifically, although the stage of memory-based information integration is considered in memory-based EC theories (e.g., Stahl & Aust, 2018), current memory research in EC does not fully account for humans’ active contribution during the retrieval process (for some exceptions, see Hütter & De Houwer, 2017; Ingendahl, Ingendahl, et al., 2024; Stahl et al., 2024). Similar to recent studies emphasizing people’s active role during the learning phase in EC (Hütter et al., 2022), we aim to take people’s active engagement at the retrieval stage into account.
We argue that humans do not merely passively retrieve stored information but actively monitor their retrieval processes during their evaluative judgments. Importantly, people use such metacognitive experiences as an additional input source in their judgments (Schwarz, 2004), which may add to the information people assess directly from memory. Consequently, previous research has overlooked an essential component of the role of memory retrieval in EC that is a direct consequence of content retrieval and informs the information integration stage (i.e., CS evaluations).
Retrieval in Evaluative Conditioning
Retrieval is “the act of accessing information from memory” (Roediger & Abel, 2022, p. 708). Although much memory research in EC has focused more directly on the stimulus encoding stage (e.g., in terms of stimulus presentation time), the retrieval stage is of equal importance for theorizing in EC. For instance, it has been argued that properties of the response format for the dependent variable’s assessment (direct vs. indirect measurement) can influence information retrieval and information integration which in turn affects the EC effect (Stahl & Aust, 2018).
In this project, we focus on ease of retrieval as a metacognitive feeling that occurs at the retrieval stage. Ease of retrieval describes the subjective difficulty of remembering that accompanies the retrieval process. According to research on metacognitive feelings, judgments are influenced by both the declarative content of relevant information that comes to mind and the subjective experience of ease or difficulty with which this evidence is retrieved (Schwarz, 2004). Importantly, declarative and experiential information can sometimes indicate opposite judgments: Whereas positive evaluations should increase with the number of positive pieces of evidence (e.g., positive attributes of a product), it becomes more difficult to retrieve positive attributes as the number of required attributes increases. This metacognitive experience of difficult retrieval can function as a negative piece of evidence itself (i.e., difficult retrieval signals that there are only a few positive pieces of evidence) and can, therefore, decrease liking (Weingarten & Hutchinson, 2018).
In a seminal study, Schwarz et al. (1991) asked participants to recall twelve or six examples of assertive or unassertive behavior from the past. Recalling twelve examples should feel difficult, whereas recalling six examples should feel easy. Crucially, participants rated themselves as less assertive after retrieving twelve examples compared to six examples of assertive behavior. Likewise, they rated themselves as more assertive after retrieving twelve examples compared to six examples of unassertive behavior. The ease-of-retrieval effect can explain these results: Participants use the ease of retrieving the information as a judgmental cue, with more difficult retrieval experiences implying less extreme values on the respective judgment dimension.
Such ease-of-retrieval effects have been shown not only in judgments about self-rated traits (Schwarz et al., 1991) but also for other judgment types such as evaluations of political actions (Ruder & Bless, 2003), subjective likelihood (Sanna et al., 2002), or observable behaviors (Stephen & Pham, 2008). They have also been found across different content domains like consumer (Wänke et al., 1997), political (Haddock, 2002), or health psychology (Raghubir & Menon, 1998; for reviews, see Wänke, 2013; Weingarten & Hutchinson, 2018). This research has relied on the few-vs.-many manipulation as used by Schwarz et al. (1991). A meta-analysis found a small-to-medium effect size of r = .25 in this paradigm (Weingarten & Hutchinson, 2018). Most importantly, ease-of-retrieval effects apply especially to evaluative judgments (e.g., Pocheptsova et al., 2010; Wänke et al., 1997)—and might thus also be relevant to EC effects.
In fact, several arguments suggest that ease-of-retrieval effects are likely to emerge in EC as well. First, in contrast to earlier process explanations of EC (Jones et al., 2009), current theories consider EC as an effect arising due to a memory-based judgment process (Gast, 2018; Stahl & Aust, 2018), where evaluative information from memory is retrieved and integrated into an evaluative response. Thus, EC effects are based on a similar construction process as in standard ease-of-retrieval experiments, where a judgment (e.g., assertiveness) is formed based on information retrieved from memory (e.g., autobiographical instances of assertive behavior). The only missing link is whether people are equally sensitive to experiential information in the construction process for EC as for previous ease-of-retrieval judgment domains. Second, recent evidence on EC suggests that people internally monitor this construction process (Ingendahl, Ingendahl, et al., 2024; Stahl et al., 2024). In fact, EC effects might even require that memory retrieval is consciously accessible to the individual (Stahl et al., 2024). Third, other sources of metacognitive feelings than memory retrieval have been shown to modulate EC, primarily, the perceptual ease of processing a stimulus (Landwehr & Eckmann, 2020; Vogel et al., 2021). Thus, it is likely that ease from memory retrieval contributes to EC effects as well.
For EC, ease-of-retrieval effects would have one central implication: Retrieving a stimulus pairing successfully from memory does not guarantee an EC effect in the direction of the US’ valence. For example, if retrieving a positive US from memory feels difficult, then this metacognitive experience might lead to a more negative evaluation of the CS. Thus, despite the successful retrieval of a positive or negative US, the EC effect can be opposite to the valence of the retrieved US. As one potential consequence, the EC effect might reverse in situations where retrieving the stimulus pairings is difficult. As another potential consequence, the correlation between successful retrieval of positive/negative content and the EC effect underestimates how memory processes contribute to the EC effect. In other words, memory processes might play a much larger role in EC than previously inferred from the relationship between memory accuracy and the strength of EC (Hofmann et al., 2010).
The Present Research
In the present research, we advance EC research by investigating whether the subjective ease of retrieving memory content influences EC effects or if it is the content alone that contributes to EC effects. As such, our research adds a new and more comprehensive perspective on the role of retrieval in EC.
We tested the role of ease-of-retrieval in EC with two experiments. We designed one experiment closer to previous ease-of-retrieval paradigms (for reviews, see Wänke, 2013; Weingarten & Hutchinson, 2018) and one experiment closer to previous EC research (see Moran et al., 2023). In both experiments, we followed established EC methodology (see Moran et al., 2023) and paired neutral CSs with different positive or negative USs. In both experiments, we followed the standard ease-of-retrieval methodology (Weingarten & Hutchinson, 2018) and asked some participants to retrieve a few of the USs and other participants to retrieve many of the USs. We explain the specific predictions derived from EC and ease-of-retrieval research within the respective study sections.
Experiment 1: The Influence of Ease of Retrieval in a Typical Ease-of-Retrieval Ecology
Experiment 1 investigated the influence of ease of retrieval in an EC paradigm adapted to typical ease-of-retrieval studies with a few-vs.-many paradigm (Weingarten & Hutchinson, 2018). Each CS was paired with an equal amount of positive and negative USs, thereby having no objective contingency with valence. This allowed a pure test of memory retrieval processes in EC, which can be induced by instructing participants to recall exclusively positive or negative US(s), similar to previous ease-of-retrieval research. We visualize the predictions of the ease-of-retrieval account in Figure 1.
Figure 1
Predictions Based on Ease of Retrieval
If people rely on the ease of retrieval when evaluating a CS, then experiencing difficulty due to retrieving many USs should lead to CS evaluations opposite to the valence of the retrieved USs. Thus, based on ease-of-retrieval research, retrieving a few positive USs should lead to more favorable CS evaluations than retrieving many positive USs. Retrieving a few negative USs, on the other hand, should lead to more negative CS evaluations than retrieving many negative USs (Figure 1a). However, these predictions are idealized. If the ease-of-retrieval manipulation is weaker than theoretically predicted (see also the General Discussion), then difficult retrieval might only attenuate but not reverse the effect of retrieval valence, leading to a pattern as in Figure 1b. Nevertheless, this pattern would be consistent with ease-of-retrieval effects.
Method
Experiment 1 was preregistered on aspredicted.org (see Ingendahl et al., 2024a).
Design
Experiment 1 had a 2 (retrieved US valence: positive USs vs. negative USs) × 2 (ease of retrieval: high vs. low) mixed design with repeated measures on the first factor.
Sample
Based on a power analysis with G*Power (Faul et al., 2007), we calculated a required sample size of N = 120 participants to detect a medium-sized (Weingarten & Hutchinson, 2018) between-within interaction effect of f = .15 with a power of 1-β = .90 and an alpha error of 5%. To compensate for potential exclusions (see below), we decided to sample N = 130 German native speakers from Prolific Academic. Our final sample size was N = 131 (58 female, 70 male, 3 diverse, Mage = 38.11).
Procedure
We adapted the basic structure of our experiment from Ingendahl and Vogel (2023). After giving informed consent, participants were exposed to an evaluative conditioning procedure where neutral brands were shown together with positive and negative words. Afterward, we applied the ease-of-retrieval manipulation and assessed evaluations of the CSs. Finally, we probed participants’ memory in a separate memory test. In the following, we will explain each task step by step.
At the beginning of the conditioning procedure, we told participants that they should look at a slideshow of brand logos presented together with words. On the following slides, four neutral brand logos (CSs) were presented together with positive and negative words (USs). In Experiment 1, each brand was paired with four different positive and four different negative words, thereby being objectively not contingent on valence. Each pairing was repeated three times for 2500ms, leading to a conditioning phase of 96 trials presented in a random order. As an attention check during the conditioning phase, we included three trials where participants had to press a button within 10 seconds. We preregistered excluding participants who did not correctly respond to more than one of these attention checks, which were none.
After the conditioning procedure, we told participants that they should now freely recall the words that were shown together with the brands. On the next four slides, we applied the ease-of-retrieval manipulation and assessed the CS evaluations. Half of the participants were asked for each CS to recall one US from the conditioning phase (high ease-of-retrieval condition). In contrast, the other half were asked to recall four USs from the conditioning phase (low ease-of-retrieval condition). Crucially, for two of the four CSs, we asked participants to recall only positive US(s), and for the other two CSs, we asked them to recall only negative US(s). For each of the four retrieval trials, the CS appeared in the middle of the screen. Below, participants in the high ease-of-retrieval condition were tasked with: “List one positive [negative] word this brand has been shown with. Important: Try to recall only one of the positive [negative] words!”, presented together with a single open text box. The task in the low ease-of-retrieval condition was: “Which positive [negative] words has this brand been shown with? Try to recall all of them. Important: Try to recall only the positive [negative] words!”, which was presented together with four open text boxes. After each recall trial, participants were again presented with the CS and the question: “How would you evaluate this brand?”, together with a single-item scale ranging from 1 (very negative) to 7 (very positive).
After all recall trials and CS ratings, participants judged their subjective ease of retrieval. Specifically, we asked participants, “How easy was the memory task for you? (i.e., retrieving one word [four words])” with responses ranging from 1 (very difficult) to 7 (very easy). This question served as a manipulation check and for a follow-up analysis (see below), indicating to what extent retrieval was indeed easier when retrieving one versus four pieces of information.
Finally, participants also completed a recognition memory test. Here, participants had to identify all words paired with the brand logo. This test served as a fair memory test because the first memory test required participants to retrieve a specific subset of the words. In this second memory test, each CS was presented on a single screen, together with a matrix of eight US words (four correct, four lure USs, two for each valence level). Participants clicked on checkboxes for each word, whether it was shown with the CS or not.
Last, participants provided demographic information (gender and age) and information about their response behavior (self-reported note-taking, open comments). Following the preregistration, we excluded one participant who reported having taken notes during the learning task, leading to a final sample size of N = 130. The procedure received ethics approval from the first author’s university’s ethics commission.
Materials
For the USs, we selected affective words from the BAWL-R (Võ et al., 2009). We selected 30 positive (M = 2.45, SD = 0.16) and 30 negative (M = -2.54, SD = 0.21) nouns from the database. The word sets did not differ in the number of phonemes, letters, syllables, frequency, or imaginability (all p’s > .135). The detailed stimulus set is provided on the OSF (see Ingendahl et al., 2023). As CSs, we used four neutrally valenced fictitious brand logos adopted from and pretested in previous EC research (Ingendahl, Vogel, et al., 2024).
Results
Data Preparation
To ensure that participants responded to the ease-of-retrieval task as intended, we coded whether responses in the open text box were meaningful responses (i.e., words from our stimulus set). We preregistered to exclude all participants who did not enter any meaningful word in the retrieval task, which were none.
Manipulation Check (Preregistered)
We first compared the two ease-of-retrieval conditions in the subjective ease of the memory test with a one-tailed between-subjects t-test. As expected, participants in the high ease-of-retrieval condition judged the memory test as easier, Mhigh = 2.69, SDhigh = 1.40, than participants in the low ease-of-retrieval condition, Mlow = 1.55, SDlow = 0.76, t(128) = 5.72, p < .001, d = 1.00.
Main Analysis (Preregistered)
We present the mean CS evaluations in Figure 2a. Detailed descriptive statistics are provided on the OSF.
Figure 2
Mean CS Evaluations in Experiment 1 Depending on Experimentally Induced Ease (a), Perceived Ease (b), Recognition Memory (c), and Perceived Ease When Controlling for Recognition Memory (d)
Note. Error bars and shaded areas represent 95% Confidence Intervals.
We analyzed the CS evaluations with a 2 (retrieved US valence: positive USs vs. negative USs) × 2 (ease of retrieval: high vs. low) mixed ANOVA with the afex package (Singmann et al., 2024). There was a significant main effect of retrieved US valence, F(1, 128) = 91.43, p < .001, = .417, such that CS evaluations were more positive when positive compared to negative USs had to be retrieved. There was no significant main effect of the ease-of-retrieval manipulation, F(1, 128) = 0.02, p = .880, <.001. Crucially, although descriptively in the expected direction, there was no significant interaction, F(1, 128) = 1.70, p = .195, = .013, showing no evidence of an ease-of-retrieval effect induced by the ease-of-retrieval manipulation.
Correlative Analyses (Preregistered Exploratory Analyses)
In addition to the pure experimental test, we preregistered a correlational analysis to test for potential ease-of-retrieval effects. Here, we took the perceived ease from the manipulation check as a numeric predictor instead of the experimental few-vs-many manipulation. Thus, we conducted a multilevel analysis using the z-standardized perceived ease, the retrieved US valence coded with -1 (negative) and +1 (positive), and their interaction to predict the CS evaluations. The analysis was run with the lme4 package (Bates et al., 2015), using random intercepts for the participants. The results are shown in Table 1 and Figure 2b.
Table 1
Multilevel Regression Results of Experiment 1
| Predictor | Perceived Ease | Recognition Memory | Perceived Ease + Recognition Memory | ||||||
|---|---|---|---|---|---|---|---|---|---|
| b | t | p | b | t | p | b | t | p | |
| (Intercept) | 3.53 | 51.77 | < .001 | 3.53 | 51.88 | < .001 | 3.53 | 51.68 | < .001 |
| Retrieval Valence | 0.57 | 10.22 | < .001 | 0.57 | 10.07 | < .001 | 0.57 | 10.23 | < .001 |
| Perceived Ease | -0.02 | -0.29 | .772 | -0.01 | -0.18 | .860 | |||
| Retrieval Valence x Perceived Ease | 0.19 | 3.45 | .001 | 0.20 | 3.61 | < .001 | |||
| Recognition Memory | -0.05 | -0.78 | .436 | -0.05 | -0.74 | .459 | |||
| Recognition Memory x Retrieval Valence | -0.04 | -0.73 | .465 | -0.07 | -1.27 | .204 | |||
Similar to the main analysis, there was a significant main effect of retrieval valence and no significant main effect of subjective ease. Crucially, there was a significant interaction of retrieval valence and subjectively perceived ease of retrieval, such that the effect of retrieved US valence was indeed more pronounced for higher subjectively perceived ease (see Figure 2b).
As we were not only interested in the influence of ease-of-retrieval itself but also its contribution beyond the retrieved memory content, we conducted a second multilevel analysis with the recognition memory measure as a moderator. Overall, participants’ corrected hit rate (hit rate – false alarm rate) was mediocre, M = .25, SD = .23. In our second model, we included the retrieval valence coded with -1 (negative) and +1 (positive), recognition memory as the z-standardized corrected hit rate1 , and the interaction of both predictors as fixed effects. The results are shown in Table 1 and Figure 2c. Apart from the main effect of retrieval valence, no effects were significant.
In a final model, we added perceived ease and recognition memory as predictors together with their interactions with retrieval valence. This model served as a test of whether the effect of perceived ease could be simply due to memory (i.e., successful retrieval also feels easy). This model showed similar results as the one without recognition memory (see Figure 2d). Thus, there was a significant moderation by perceived ease, even when controlling for memory.
Discussion
Experiment 1 shows inconsistent evidence regarding ease-of-retrieval effects in EC. Although the traditional few-vs.-many manipulation successfully induced differences in ease-of-retrieval, it did not induce ease-of-retrieval effects in EC. However, the perceived ease was indeed associated with stronger effects of retrieved US valence. This association was still significant after controlling for memory. We will discuss this result pattern in detail in the General Discussion.
Experiment 2: The Influence of Ease of Retrieval in a Typical EC Learning Ecology
In Experiment 2, each CS was paired exclusively with positive or negative USs, being more similar to traditional EC paradigms. Again, we manipulated the ease-of-retrieval by following the established ease-of-retrieval methodology and asking participants to retrieve either a few (easy) or many (difficult) of the USs paired with each CS (Weingarten & Hutchinson, 2018). As the CSs were paired exclusively with positive or negative USs, the information that is retrieved is exclusively positive or negative, even if retrieval feels very difficult. Therefore, we deemed it more likely that the feeling of difficult retrieval leads to an attenuation but not a reversal of the EC effect, as shown in Figure 1b2 .
Method
Experiment 2 was preregistered on aspredicted.org (see Ingendahl et al., 2024b).
Design
We used a 2 (US valence: positive vs. negative) × 2 (ease of retrieval: high vs. low) mixed design with repeated measures on the first factor.
Sample
Based on a power analysis with G*Power (Faul et al., 2007), we required N = 266 participants to detect a between-within interaction effect of f = .1 with a power of 1-β = .90 and an alpha error of 5%. We expected a smaller effect size in this experiment due to the ordinal interaction (see Figure 1b), which should be approximately 25–50% of the basic EC effect of d = 0.52 (Hofmann et al., 2010; Sommet et al., 2023). We preregistered recruiting a sample of N = 270 German native speakers from Prolific Academic. Due to an experimental error (i.e., an outdated study link), the first participants needed to be excluded, leading to the final sample size N = 261 (101 female, 156 male, 4 diverse, Mage = 35.03).
Procedure and Materials
The procedure and materials of Experiment 2 were as in Experiment 1, except that there were actual contingencies between CSs and US valence. That is, two CSs were paired with six positive, and two CSs with six negative US words. Each pairing was shown three times and the 72 trials were presented in a random order. In contrast to Experiment 1, we asked participants to recall either one or all six USs, depending on the between-participants manipulation, but we did not ask specifically for recalling positive or negative USs. For the low ease-of-retrieval condition, the task was: “List one word that was shown with this brand name!”, presented together with a single text box. For the high ease-of-retrieval condition, the task was: “Which words were shown with this brand name? Try to recall all of them!”, presented with six text boxes. The rest of the study was identical to Experiment 1, except for the recognition test. Here, each CS was presented with a matrix of twelve US words (three correct words, nine incorrect words, six per US valence). The procedure received ethics approval from the first author’s university’s ethics commission.
Results
Data Preparation
Again, we coded whether responses in the open text box were meaningful responses (i.e., words from our stimulus set). We preregistered to exclude all participants who did not enter any meaningful word in the retrieval task, which were none.
Manipulation Check (Preregistered)
As expected, participants in the high ease-of-retrieval condition judged the memory test as easier, Mhigh = 4.32, SDhigh = 1.86, than participants in the low ease-of-retrieval condition, Mlow = 1.88, SDlow = 1.04, t(251) = 12.76, p < .001 (one-tailed), d = 1.61.
Main Analysis (Preregistered)
We present the mean CS evaluations in Figure 2b. Detailed descriptive statistics are again provided on the OSF.
We analyzed the CS evaluations with a 2 (US valence: positive vs. negative) × 2 (ease of retrieval: high vs. low) mixed ANOVA with the afex package (Singmann et al., 2024). There was a significant main effect of US valence, F(1, 251) = 575.01, p < .001, = .696, such that CSs paired with positive USs were evaluated more positively than those paired with negative USs. There was no main effect of ease-of-retrieval, F(1, 251) = 0.07, p = .786, < .001. Crucially, there was also no significant interaction, F(1, 251) = 0.02, p = .888, < .001, providing no evidence of ease-of-retrieval effects.
Correlative Analyses (Preregistered Explorative Analyses)
As in Experiment 1, we conducted a correlative analysis with the perceived ease as a predictor instead of the experimental few-vs.-many manipulation, using the same multilevel model. The results are shown in Table 2 and Figure 3b. Similar to the main analysis, there was a significant main effect of US valence, but also a significant main effect of subjectively perceived ease. Crucially, as in Experiment 1, there was a significant interaction between US valence and perceived ease, such that the effect of US valence was indeed more pronounced for higher subjectively perceived ease (see Figure 3b).
Table 2
Multilevel Regression Results of Experiment 2
| Predictor | Perceived Ease | Recognition Memory | Perceived Ease + Recognition Memory | ||||||
|---|---|---|---|---|---|---|---|---|---|
| b | t | p | b | t | p | b | t | p | |
| (Intercept) | 3.86 | 124.62 | < .001 | 3.86 | 124.72 | < .001 | 3.86 | 124.87 | < .001 |
| US Valence | 1.69 | 24.46 | < .001 | 1.69 | 25.64 | < .001 | 1.69 | 25.79 | < .001 |
| Perceived Ease | 0.07 | 2.19 | .029 | 0.05 | 1.68 | .094 | |||
| Perceived Ease x US Valence | 0.22 | 3.11 | .002 | 0.13 | 1.96 | .051 | |||
| Recognition Memory | 0.08 | 2.46 | .014 | 0.06 | 2.02 | .044 | |||
| Recognition Memory x US Valence | 0.39 | 5.96 | < .001 | 0.36 | 5.39 | < .001 | |||
Figure 3
Mean CS Evaluations in Experiment 2 Depending on Experimentally Induced Ease (a), Perceived Ease (b), Recognition Memory (c), and Perceived Ease When Controlling for Recognition Memory (d)
Note. Error bars and shaded areas represent 95% Confidence Intervals.
We also conducted the same analysis with recognition memory (M = 0.47, SD = 0.25) as in Experiment 1. The results are shown in Table 2 and Figure 3c. In contrast to Experiment 1, there was a main effect of recognition memory, with more positive CS evaluations for participants with better memory. However, there was also a significant interaction, such that the EC effect was stronger for participants with better memory.
In a final model, we again added both memory and perceived ease as predictors. Whereas the interaction with memory remained significant, the interaction with perceived ease was above the preregistered alpha level. Thus, the effect of perceived ease was not incremental to what could be explained by differences in memory.
Discussion
Experiment 2 showed similar results as Experiment 1 in an EC paradigm with actual contingencies with valence. First, there was a strong EC effect. Second, retrieving a few USs was perceived to be easier than retrieving many USs. Third, the manipulation of ease-of-retrieval did not moderate the EC effect. However, the subjectively perceived ease-of-retrieval did moderate the EC effect. We will discuss these findings in detail below in the General Discussion.
General Discussion
Evaluative Conditioning (EC) is the change in stimulus evaluations due to the stimuli’s pairing with other positive/negative stimuli and a key effect in attitude formation. Current process explanations on EC emphasize the role of memory processes, including the retrieval of US valence when giving an evaluative judgment. However, previous research has neglected that metacognitive experiences such as the ease of retrieving the US’s valence might contribute to EC effects. To close this gap in the literature, we applied the established few-vs.-many manipulation from ease-of-retrieval research (Schwarz et al., 1991; Weingarten & Hutchinson, 2018) to an EC paradigm.
Our two experiments showed a similar result pattern. Although participants experienced retrieving many USs as more difficult than retrieving a few USs, this manipulation did not affect the influence of (retrieved) US valence on CS evaluations. In other words, we found no evidence favoring and even evidence against3 ease-of-retrieval effects in EC when applying the established few-vs.-many manipulation. However, we found that the perceived ease-of-retrieval was associated with stronger effects of (retrieved) US valence. In Experiment 1, we also found that this association was incremental to what could be explained by memory. In the following, we will discuss these findings in relation to our main research question and the cognitive processes underlying EC.
Are There Ease-Of-Retrieval Effects in EC?
Our result pattern is challenging to interpret. On the one hand, we took an established manipulation from ease-of-retrieval research (Weingarten & Hutchinson, 2018), which successfully induced strong differences in the perceived ease of retrieval. Yet, this manipulation did not induce any ease-of-retrieval effects on EC. On the other hand, subjectively perceived ease itself had a significant moderation effect in both experiments, with stronger EC effects at higher perceived ease. We believe that there are two interpretations of this pattern—a conservative and a more lenient one.
A conservative interpretation of our findings would be that we did not obtain ease-of-retrieval effects in EC using established and internally valid methods. As the few-vs.-many manipulation has proven successful in various ease-of-retrieval studies (Weingarten & Hutchinson, 2018), and also induced strong differences in subjective ease in our studies, we have no reason to suspect that the manipulation was insufficient. Although we found an association between the perceived ease and the EC effect, it remains a mere correlation, leaving open several alternative explanations. First, perceived ease is naturally correlated with successful memory retrieval. Experiment 2 (but not Experiment 1), for example, shows that the association with perceived ease is no longer significant when keeping memory statistically constant. Second, the correlation with perceived ease was assessed on the person-level instead of the stimulus-level, which is prone to various alternative explanations by third variables. For example, those participants who report higher ease and also show stronger EC effects might be those participants who pay more attention to the task (Field & Moore, 2005), have more intense affective reactions towards the USs (Ingendahl & Vogel, 2023), are more likely to comply with the experimental tasks (Ingendahl, Woitzel, & Alves, 2023), or who are more confident in their memory and their stimulus evaluations (Tormala & Rucker, 2007). Likewise, reverse causality is possible, such that if participants expressed more extreme CS evaluations, they inferred that the memory test was probably rather easy. Therefore, one should be cautious about interpreting the association between perceived ease and EC effects.
A more lenient interpretation of our findings might be that ease-of-retrieval does contribute to EC effects—it is just not the ease originating from remembering few or many USs, but from other, perhaps more incidental factors. For example, for some participants, the randomly sampled CSs and USs might create a better fit, leading to a sense of easy retrieval and a more pronounced EC effect. This interpretation is supported by recent research on the role of metacognition in EC (Ingendahl, Ingendahl, et al., 2024), which found that being convinced that one will remember a stimulus pairing was associated with a stronger EC effect, incremental to whether participants could actually remember a pairing. In these studies, the stronger EC effect was explained by factors unrelated to memory, such as the experienced fit between CS and US or how easily a stimulus pairing could be processed. Whereas these previous studies focused on metacognitive experiences during the learning stage, our research shows that subjective feelings at the retrieval stage might be associated with the strength of the EC effect as well. However, similar to Ingendahl, Ingendahl, et al. (2024), it remains unclear whether this association with metacognitive experiences is causal or can be explained by various third variables. Also, this more lenient interpretation of our findings has two core problems: First, it leaves open how people are capable of distinguishing between multiple sources of ease-of-retrieval and discounting the ease-of-retrieval induced by one particular source (i.e., the experimental manipulation). Second, it leaves open why people usually rely on ease induced by recalling few versus many pieces of information in their judgments (Weingarten & Hutchinson, 2018), but not in the context of EC.
Thus, the question arises as to why the established few-vs.-many paradigm did not induce ease-of-retrieval effects in our EC experiments, given that such effects have been found in various other domains with evaluative judgments as an outcome. One conceptual difference between EC and previous domains of ease-of-retrieval effects is that in the EC stimulus ecology, the amount of retrievable information is more restricted by the number of presented stimuli, which might limit how informative ease-of-retrieval experiences are when forming a judgment (Schwarz, 2004). Classical ease-of-retrieval studies let participants generate arguments in a specific judgment domain, such as arguments in favor or against an educational policy (e.g., Ruder & Bless, 2003). Here, the information that can be retrieved is, in principle, infinite, and thus, people might consider the retrieval experiences as informative regarding how good/bad the educational policy is. In EC paradigms, however, the amount of information that can be retrieved is restricted (e.g., a CS was paired with twelve USs). Thus, people might consider the ease-of-retrieval experience uninformative for assessing the amount of valenced information that could be retrieved. This would imply that people rely on ease-of-retrieval experiences in EC only when the number of stimuli is difficult to monitor. Given that EC procedures often follow a one-to-one mapping (i.e., a CS is paired with only one US), this argumentation would also imply that ease-of-retrieval effects are unlikely to occur in many EC studies.
Irrespective of whether one favors the conservative or lenient interpretation of our findings, one critical implication of our results is that differences in ease-of-retrieval are unlikely to reverse EC effects. Previous ease-of-retrieval studies (e.g., Ruder & Bless, 2003; Schwarz et al., 1991) have shown result patterns where difficult retrieval of information can reverse the effect of the valence of that particular information. For example, difficult retrieval of positive information might eventually lead to more negative evaluations compared to a neutral baseline. Such reversals would have critical implications for previous memory-based EC accounts (Gast, 2018; Stahl & Aust, 2018), which presume that the successful retrieval of US valence is necessary for EC effects, but neglect the influence of metacognitive experiences during the retrieval. Our findings are in line with these accounts, as there is—if at all—a small attenuation of the EC even when retrieval feels difficult. This might also explain why US valence memory substantially or even fully explained when people show or do not show an EC effect in previous EC studies (e.g., Bar-Anan et al., 2010; Hütter et al., 2012; Pleyers et al., 2007; Stahl et al., 2009). Even if retrieval feels extremely difficult, the EC effect still operates in the same direction as the retrieved US’s valence. Yet, in this regard, our studies also showed some unexpected findings, which brings us to the limitations of our research.
Limitations and Directions for Future Research
Our experiments are compromised by some limitations. First, we obtained a significant moderation by memory in Experiment 2, but not in Experiment 1. However, in Experiment 1, each CS was shown with equally as many positive and negative USs. Thus, recognizing the correct stimuli was conceptually unrelated to the valence of the to-be-retrieved stimuli, limiting the moderating influence of memory. To test this explanation, we conducted an additional analysis where we coded recognition memory as whether participants recognized USs in line with the retrieval valence manipulation (e.g., recognized positive USs for the positive retrieval valence condition). Consistent with this explanation, these analyses indeed showed a moderation by memory, but also an additional moderation by perceived ease. We provide these analyses on the OSF.
Second, we deliberately chose to stay as near as possible to the established few-vs.-many manipulation from ease-of-retrieval research. Yet, this manipulation only induces ease/difficulty of retrieval through the amount of information to be retrieved. Future research might investigate other potential sources of ease at the retrieval stage, such as the concreteness/abstractness of the US materials or the thematic fit between CS and US, which may influence EC.
Third, we obtained rather strong EC effects in our paradigm. One could speculate that this might have left less “room” for the perhaps more subtle cue of retrieval ease. However, previous research has identified comparable meta-analytic effect sizes of EC (d = .52; Hofmann et al., 2010) and ease-of-retrieval (r = .25 and thus, d = .52; Weingarten & Hutchinson, 2018). Thus, based on the anticipated effect size of both phenomena, there is no indication that ease-of-retrieval effects are entirely overshadowed by EC. Likewise, although EC effects were strong in our paradigm, they were not uniform but left “room” for two significant moderations – one by memory, and one by the measured ease-of-retrieval. Thus, although our results clearly show a strong influence of the retrieved content on evaluative judgments, we believe that this was not the reason why we did not obtain ease-of-retrieval effects in the few-vs.-many paradigm.
Instead, ease-of-retrieval research also points to several boundary conditions that could potentially explain why we found no ease-of-retrieval effects in an EC context, which might be picked up in future research. First, ease-of-retrieval effects depend on people’s naïve theory of what the subjective feeling of ease implies (Schwarz et al., 1991; Wänke et al., 1995). Likewise, ease-of-retrieval effects can also be diminished if participants attribute the feeling of easy/difficult recall to causes irrelevant to the judgment at hand (Schwarz et al., 1991). We did not control for these factors in our paradigm. Thus, future research might consider either measuring or manipulating people’s naïve beliefs of what the experience of easy/difficult recall signals in an EC paradigm (e.g., typicality; Vogel et al., 2021). In addition, people are less likely to rely on their ease-of-retrieval if they have a high cognitive capacity (Greifeneder & Bless, 2007) or if they are in an abstract thinking mindset (Tsai & Thomas, 2011)—both structural aspects that we did not control for in our paradigm. Also, ease-of-retrieval effects seem to operate best under medium levels of retrieval difficulty (Tybout et al., 2005). Although our second experiment indeed achieved medium levels of retrieval difficulty, as indicated by the manipulation check, participants in our first experiment overall found the memory task quite difficult, which might have diminished ease-of-retrieval effects. Thus, overall, previous ease-of-retrieval research offers various starting points4 to explore whether, under specific conditions, ease-of-retrieval effects might also be obtained in an EC paradigm.
Conclusion
In two experiments, we did not find evidence for ease-of-retrieval effects in evaluative conditioning in response to an established ease-of retrieval manipulation. Overall, these experiments suggest that ease-of-retrieval effects do not contribute to EC or might only be small in size. As such, the presented research is a valuable extension to the current perspective on retrieval effects in EC.