Automatic and strategic effects in the guidance of attention by working memory representations

doi:10.1016/j.actpsy.2010.06.012

Acta Psychol (Amst). Author manuscript; available in PMC 2012 June 1.

Published in final edited form as:

Acta Psychol (Amst). 2011 June; 137(2): 217–225.

Published online 2010 July 18. doi: 10.1016/j.actpsy.2010.06.012

PMCID: PMC2991492

NIHMSID: NIHMS224249

Automatic and strategic effects in the guidance of attention by working memory representations

Nancy B. Carlisle and Geoffrey F. Woodman

Vanderbilt University, Vanderbilt Vision Research Center, Vanderbilt Center for Cognitive and Integrative Neuroscience

Correspondence from the Editor/Publisher should be sent to: Geoffrey F. Woodman, Department of Psychology, Vanderbilt University, PMB 407817, 2301 Vanderbilt Place, Nashville, TN 37240-7817, Email: geoffrey.f.woodman/at/vanderbilt.edu, Phone: (615) 322-0049, Fax: (615) 343-8449

The publisher's final edited version of this article is available at Acta Psychol (Amst)

See other articles in PMC that cite the published article.

Publisher's Disclaimer

Abstract

Theories of visual attention suggest that working memory representations automatically guide attention toward memory-matching objects. Some empirical tests of this prediction have produced results consistent with working memory automatically guiding attention. However, others have shown that individuals can strategically control whether working memory representations guide visual attention. Previous studies have not independently measured automatic and strategic contributions to the interactions between working memory and attention. In this study, we used a classic manipulation of the probability of valid, neutral, and invalid cues to tease apart the nature of such interactions. This framework utilizes measures of reaction time (RT) to quantify the costs and benefits of attending to memory-matching items and infer the relative magnitudes of automatic and strategic effects. We found both costs and benefits even when the memory-matching item was no more likely to be the target than other items, indicating an automatic component of attentional guidance. However, the costs and benefits essentially doubled as the probability of a trial with a valid cue increased from 20% to 80%, demonstrating a potent strategic effect. We also show that the instructions given to participants led to a significant change in guidance distinct from the actual probability of events during the experiment. Together, these findings demonstrate that the influence of working memory representations on attention is driven by both automatic and strategic interactions.

Keywords: attention, working memory, cuing, automaticity, strategic control, PsychINFO classification 2346

1. Introduction

To direct our limited-capacity perceptual attention to the important objects in the complex scenes of daily life, we must have internal representations of the objects we wish to locate. For example, when we drive near a school we scan our visual field for hazards such as children crossing the street. Numerous theories of visual attention propose that we implement top-down control of visual attention by actively maintaining a target template in working memory (Bundesen, 1990; Desimone & Duncan, 1995; Duncan, 1996; Duncan & Humphreys, 1989). According to such theories, holding a representation of a child in working memory would automatically guide our limited-capacity mechanisms of perceptual attention to children while driving in a school zone, even without the goal of looking for children per se.

The biased competition theory of visual attention (Desimone & Duncan, 1995) provides a concise explanation for how working memory representations enable top-down control. According to biased competition, an attentional template representation in working memory increases the activation of cells selective for the template’s features by feeding back to lower-level areas in the brain that perform perceptual analysis. Then, when the system is presented with a complex scene containing multiple competing inputs, this top-down bias increases the probability that the neural representation of the template-matching object will be attended. This theoretical account of top-down attentional control is attractive for its simplicity, but tests of the proposal have shown mixed results.

The typical method researchers have used to test the hypothesis that working memory representations guide attention required the participants to hold a representation in working memory while performing a visual search task (Downing, 2000; Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006; Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005; Soto, Hodsoll, Rotshtein, & Humphreys, 2008). The researchers reasoned that if working memory guides attention, then attention should be directed toward memory-matching items during visual search, even if these items are distractors in the search array (i.e., non-target items). Some studies have found evidence that items in the visual field matching working memory representations do attract attention to themselves (e.g., Downing, 2000; Olivers et al., 2006; Soto et al., 2005). This evidence has motivated the conclusion that representations in working memory automatically guide attention (e.g., Soto et al., 2008). However, other studies have found no attentional preference for items matching working memory representations or find that attention is directed away from the memory-matching items when doing so could improve search performance (Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006; Peters, Goebel, & Roelfsema, 2009; Soto & Humphreys, 2008; Woodman & Luck, 2007). These studies lead some researchers to conclude that the influence of working memory representations on perceptual attention is governed by flexible, higher-level strategies (Downing & Dodds, 2004; Woodman & Luck, 2007). Like the debate over the automaticity of attentional selection during different spatial cuing paradigms (e.g., Jonides, 1981; Yantis & Jonides, 1990), the current debate about attentional guidance by working memory representations is framed in terms of automaticity versus voluntary control. It is important for us to note that no single methodological difference can distinguish the studies that conclude working memory automatically guides attention from those that support conclusions of strategic, voluntary control (Olivers, 2009; Soto et al., 2008). The variability of findings from such experiments leads us to hypothesize that multiple factors are playing out during the participants’ performance in these dual-task paradigms.

Han and Kim (2009) proposed an account that sought to reconcile the variety of results described above. They proposed that there is an automatic guidance of perceptual attention to select memory-matching items in the visual field, but that cognitive control can override this guidance given sufficient time for this control to be implemented. In their account, if perceptual processing of a visual search array can be accomplished quickly (reaction times, RTs, on the order of 1100 ms in their study), then cognitive control will not be able to counteract the effects of automatic guidance to memory-matching items. However, if perceptual processing takes a sufficient amount of time (RTs of approximately 1700 ms in their study), then cognitive control can be used to counteract the default attentional shift toward the working memory matching item. Although this hypothesis provides a succinct account of previous results, Han and Kim (2009) had to rely upon observing qualitatively different patterns of results across experiments with different methods. Thus, a goal of the present study was to determine whether the proposed automatic and strategic interactions between working memory representations and perceptual selection could be measured simultaneously.

The basic dual-task paradigm used in these previous studies (i.e., performing visual search during a working memory task) has proven inadequate for definitely resolving the debate over whether guidance of attention by working memory representations is automatic or strategic. What extensions of this paradigm could help us distinguish between automatic and strategic effects? We turned to classic studies of attention and automaticity to answer this question (Posner, 1978, 1980; Posner & Snyder, 1975). Posner and Snyder (1975) partialled out the influence of automatic and strategic processes during priming paradigms by manipulating the probability of valid, neutral, and invalid cues. They proposed that cues which match the identity of subsequent letters (valid cues) would lead to RT benefits (i.e., faster RTs) compared to cues that gave no information about the upcoming letters that participants were required to discriminate (neutral cues). In contrast, cues that were different than the subsequent stimuli (invalid cues) would lead to RT costs compared to neutral cues (i.e., slower RTs). Posner and Snyder always presented 50% neutral cues, but manipulated the probabilities of the valid and invalid cues. In the high probability condition, 80% of non-neutral trials were validly cued, and 20% were invalidly cued. In the medium probability condition, 50% of non-neutral trials were validly cued and 50% were invalidly cued. In the low probability condition, 20% of non-neutral trials were validly cued and 80% were invalidly cued. According to their logic, automatic processes should lead to costs or benefits even when the cue was not likely to be valid. In contrast, strategic influences would be measured by an increase in costs or benefits as cue validity increased. Thus, this methodological framework allows us to independently estimate the contribution of automatic and strategic processes during a single experimental paradigm.

Posner and Snyder (1975) applied the logic to their findings from the probability manipulation in the following way. They found that all probability conditions led to a benefit, with only a small increase in the size of the benefit as the probability of valid cues increased. In contrast, costs were not present in the low-probability condition, and only appeared in the medium and high probability conditions. Based on this pattern of effects, the authors concluded that priming led to an automatic benefit, but costs depended on the participant’s strategic use of the cue information.

The classic logical framework and manipulations of Posner and Snyder (1975) are particularly relevant to the current debate over the role of strategic and automatic processes in the guidance of attention by working memory. When the experiments of Posner and Snyder (1975) are viewed through the lens of the debate regarding the role of working memory, it seems possible that the priming effects they found were in fact due to storage of the prime stimuli in working memory.

One existing study suggests that probability manipulations can change the way that working memory representations guide attention. Woodman & Luck (2007) showed that attention was not influenced by the information stored in working memory when no valid trials were presented (i.e., 0%), but attention was directed to memory-matching stimuli in the search array when valid trials were possible but unlikely (i.e., 16.7%). Thus, we believe that the current debate can be informed by utilizing the methods pioneered in classic studies to distinguish contributions from automatic versus voluntarily controlled processes.

2. Experiment 1

In Experiment 1, we examined the automatic and strategic components of the guidance of attention by working memory representations across three probability levels. We applied the classic probability manipulation used by Posner and colleagues (Posner, 1978, 1980; Posner & Snyder, 1975) to the typical dual-task paradigm used in studies of working memory guidance of attention. In the task illustrated in Figure 1, participants were first shown a colored square. They needed to remember the color until the end of the trial so that they could perform a change-detection task. Then, participants were shown a visual search array and pressed one of two keys to report whether a square with a gap up or down was present among the four objects in the array. The trial concluded with the presentation of a memory-test item. Participants pressed one of two keys to indicate whether the test item matched the memory item shown at the beginning of the trial.

Figure 1

Example trial with the memory-matching item as a distractor. Participants made a speeded button-press response to the target shape in the search array and an unspeeded response to the memory-test item. During the entire trial participants performed a (more ...)

There were three trial types distinguished by the relationship between the item that observers had to represent in working memory and the items in the search array. On neutral trials, the visual search array did not have an item that matched the memory item. On invalid trials the visual search array contained a non-target item that matched the color of the memory item. Finally, on valid trials the search target matched the color of the memory item. Across participants, we manipulated the likelihood of valid and invalid trials in the search array. Based on the logic of Posner and Snyder (1975), if attention is automatically directed to the memory match, we expected to find significant costs, benefits, or both even in the condition with few valid trials. In addition, if individuals strategically use knowledge of the likelihood of a memory-target match to guide attention, then costs, benefits, or both should increase as valid-trial probability increases.

In the assessment of working memory guidance of attention, early RTs have been emphasized as the best assessment of automatic effects. For example, Soto and colleagues (2005) based their claims of involuntary effects of working memory items on visual attention on their assessment of the earliest 10% of RTs. In the same vein, Han and Kim (2009) propose automatic attentional orienting to memory matching items occurs initially and strategic cognitive control only comes online at some duration after the presentation of the visual search array. According to this logic, automatic effects of guidance may only be apparent in easy perceptual tasks where responses occur before cognitive control comes online. A strict application of their account would predict that our study should find no evidence of strategic effects when RTs are short than approximately 1700 ms. However, a more conservative prediction derived from the account of Han and Kim (2009) would be that strategic effects may emerge in later RTs, but should not be present in the early RTs.

To test the more detailed predictions about the dynamics of strategic cognitive control and to examine how the pattern of RTs changed across the three probability conditions, we also computed vincentized cumulative RT distributions of valid, neutral, and invalid trials in each probability condition (Ratcliff, 1979). If attention is oriented to memory-matching items automatically and is subsequently overridden when strategic cognitive control comes online, then we should observe that the earliest RTs evidence an obligatory shift of attention to the memory-matching item with the later RTs showing evidence of strategic influences across the probability conditions. Thus, we tested for the presence of automatic and strategic effects on search performance by examining the effects of the probability manipulation on mean RT as well as across the RT distributions.

2.1 Method

2.1.1 Participants

Thirty-six participants were recruited through the Vanderbilt University participant pool for course credit or payment. All reported having normal color vision and normal or corrected-to-normal acuity. After obtaining informed consent, the participants were randomly assigned to one of the three probability conditions (20%, 50%, or 80% memory-target match trials, as described in detail below). Two participants were replaced because they failed to respond on more than 25% of trials, and one additional subject was replaced due to search accuracy more than 3 standard deviations below the group mean.

2.1.2 Stimuli and Apparatus

Stimuli were presented using Psychtoolbox for Matlab (Brainard, 1997). Each participant was seated approximately 57 cm from the monitor. The stimuli were drawn from a set of five easily discriminable colors: red (x = .602, y = .336, 33 cd/m²), green (x = .282, y = .586, 104 cd/m²), yellow (x = .399, y = .504, 119 cd/m²), blue (x = .148, y = .078, 17 cd/m²), and magenta (x = .289, y = .151, 42 cd/m²). The fixation point in the center of the screen subtended approximately 0.36° X 0.36° of visual angle and was drawn in white (<120 cd/m²). The memory item on each trial was a filled square (0.90° X 0.90°) presented centered 1.35° above the central fixation cross. Each search array was composed of four Landolt squares (0.90° X 0.90°, 0.18° line thickness, with a gap on one side of approximately 0.36°). The Landolt squares were presented on an imaginary circle with a radius of 7.16° from the center of the fixation cross. The search target had a gap on its top or bottom, and the distractors had gaps on their left or right. Articulatory suppression stimuli were four letters or digits presented embedded in an instructional sentence before each block, drawn in white (<120 cd/m²) with each character presented within an area of 0.65° X 0.65°.

Each trial began with a 500-ms presentation of the fixation cross, followed by a 500-ms presentation of the memory-sample item. After a period of 500 ms in which only the fixation cross was visible, the search array was presented for 3000 ms. Participants searched for a target with a gap on its top (50% of trials) or a gap on its bottom (50% of trials). Participants made a speeded response to the location of the target gap by pressing one button on a gamepad when the gap was on the top of the square and a different button when the target gap was on the bottom. The offset of the search array was followed by a 500 ms interval in which only the fixation cross was visible. Next, the memory-test item was presented in the same location as the memory-sample item for 2000 ms. The memory test was identical to the memory sample on half of all trials and was a different color on the other half of trials. Participants responded with one button if the memory-test item was the same color as the memory-sample color and another button if the color had changed (separate buttons were used for search and memory task responses). Whether the memory item changed was not correlated with the type of visual search array presented (i.e., neutral, invalid, or valid trial) and all factors were randomized across the experimental session.

2.1.3 Design and procedure

Each participant completed 8 practice trials before completing 8 blocks of 30 trials for a total of 240 experimental trials. There were three between-subjects conditions, across which we manipulated the probability of the different trial types. In all three conditions, 50% of all trials were neutral, in which no memory-matching item was present in the search array. For the 20% memory-target match condition, the memory item was the same color as the target on 20% of the non-neutral trials (total trials: 10% valid, 40% invalid, 50% neutral). Of the non-neutral trials for the 50% memory-target match condition, the memory match was the same color as the target on 50% of trials (total trials: 25% valid, 25% invalid, 50% neutral). In the 80% memory-target match condition, the memory-match was the same color as the target on 80% of the non-neutral trials (total trials: 40% valid, 10% invalid, 50% neutral). Each participant was informed about the assigned probability of valid, invalid, and neutral trials before beginning the practice block.

At the beginning of the experiment, a string of letters or digits (i.e., “a,b,c,d”, “1,2,3,4”, “6,7,8,9”, or “w,x,y,z”) was presented until the participant made a button press response to begin the experiment. After each block of trials, participants were given a 15-second break during which they were presented with a new set of articulatory suppression instructions. Participants were required to repeat these letters or digits aloud at a rate of 3–4 per second throughout each trial of the experiment. Their verbal responses were recorded for offline analysis to verify that they complied with these instructions.

The neutral condition was used as a baseline to assess the costs of the memory-matching item appearing as a distractor and the benefits of the memory-matching item as a search target for each participant. Statistical analyses of search RT were performed on all three trial types (valid, invalid, and neutral) and the derived cost and benefit measures.¹ Trials in which there was no memory response or no search response (3.5% of trials) were excluded from all analysis. Additionally, trials with an incorrect memory response (3.5% trials) were excluded from analysis of search RTs, as were trials with a RT below 200 ms or 3 standard deviations above each subject’s mean (1% of trials). All p-values were Greenhouse-Geisser corrected due to violations of the assumption of sphericity.

To examine the RT distributions, we computed vincentized cumulative RT distributions of valid, neutral, and invalid trials in each probability condition. We first computed distributions for each condition for each subject, and then created a group mean distribution by averaging each quantile within each group and condition. This procedure ensures that the shape of the mean distribution is not a distortion of the shape of the individual distributions (Ratcliff, 1979).

2.2 Results

2.2.1 Search Task Performance

Across probability conditions, search RTs were fastest when the target matched the memory item (M = 848 ms), slower on neutral trials in which there was not an item in the search array that matched the color of the memory item (M = 1132 ms), and slowest when the memory-matching item was a distractor in the search array (M = 1240 ms). As the mean RTs in Table 1 show, the overall search RTs were similar across the validity probability condition. An ANOVA with the within-subjects factor of trial type (valid, invalid, or neutral) and the between-subjects factor of probability condition (20% versus 50% versus 80% memory-target match) yielded a significant main effect of trial type (F (2, 66) = 255.11, MSE = 5784, p < .001) due to the pattern of RTs above. We also found a significant interaction of trial type X probability condition (F (4, 66) = 10.21, MSE = 5784, p < .001) due to trial types having the largest effects in the 80% condition, followed by the 50% condition, and the smallest influence in the 20% condition. Search response accuracy was above 99% correct across trial types in all conditions (F < 1.0).

Table 1

Mean responses times (rounded to the nearest millisecond) for each array type and probability condition in Experiment 1 and Experiment 2.

We next examined the derived cost and benefit measures to directly assess the automatic and strategic contributions to the guidance of attention by working memory. Following the classic logical framework described above, we expected costs or benefits in the 20% condition to be significantly larger than zero if working memory representations automatically influence visual attention. We found a significant cost (M = 69 ms; t (11) = 3.54; p < .01), as well as a significant benefit (M = −200 ms; t (11) = 6.54; p < .0001). These show automatic costs and benefits of attending to the memory-matching items.

If there are strategic influences on the guidance of attention by working memory, then we expected the costs or benefits to increase with increasing probability of memory-target match. As shown in Figure 2, the magnitude of both the costs and benefits increased as the probability of memory-target match increased. In the 20% condition, the cost was 69 ms, in the 50% condition the cost was 108 ms, and in the 80% condition the cost was 147 ms. Benefits also increased (i.e., became more negative) as the probability of a memory-target match increased, from a mean of −200 ms in the 20% condition, to −260 ms in the 50% condition, to −392 ms in the 80% condition. These observations were supported by the results of an ANOVA with the within-subject factor of memory-match influence (costs versus benefits) and the between-subjects factor of probability condition (20% versus 50% versus 80% memory-target match). This ANOVA yielded a main effect of memory-match influence (F (1, 33) = 321.50, MSE = 8602, p < .001) due to costs being positive values and benefits being negative values, but no main effect of probability condition (F (2, 33) = 2.49, MSE = 8904, p = .10). The modulation of the positive costs and negative benefits with increasing memory-target match resulted in an interaction of memory-match influence X probability condition (F (2, 33) = 12.88, MSE =8602, p < .001).

Figure 2

The derived RT costs and benefits from Experiment 1. The RTs are shown in terms of costs (invalid trial mean RT minus neutral trial mean RT) and benefits (valid trial mean RT minus neutral trial mean RT). Error bars for the costs and benefits represent (more ...)

Our pair-wise planned comparisons of the influence of memory-target match probability on the costs and benefits showed that the cost in the 80% condition was significantly larger than the 20% condition (t(22) = 2.74, p < .05). The benefit in the 80% condition was significantly larger than the 50% condition (t(22) = 2.82, p < .01) and larger than the 20% condition (t(22) = 4.24, p < .001). No other pair-wise comparisons were significant. The pairwise comparisons confirm that both costs and benefits increase with increasing probability of memory-target match trials, supporting the hypothesis that potent mechanisms of strategic control influence working memory guidance of attention.

If strategic effects only come online later in the trials and are present in the slower RTs, then we should find similar effects at fast RTs (e.g., the first 10% of trials) for each condition, with group differences only appearing later. To test this, we examined the RT distributions for each trial type in each probability condition. The results of this analysis suggest that the explanation of the mean RT effects based on differences only in the tails of the RT distributions is incorrect. Instead, we found strategic effects due to the probability manipulation even in earliest decile of the RT distribution. Wilcoxon rank order tests on the costs and benefits of the 1^st quantile of RT data showed the costs in the earliest quantile were larger for the 80% condition (243 ms) compared to the 50% condition (120 ms; Z = 3.52; p < .001) and the 20% condition (77 ms; Z = 3.72; p < .001). Similarly, the benefits for the earliest quantile were larger for the 80% condition (198 ms) than the 50% condition (124 ms; Z = 2.92; p < .01) and the 20% condition (67ms; Z = 4.01; p < .001). In addition, the benefits for the 50% condition were larger than the benefits for the 20% condition (Z = 2.40; p < .05). These findings show that the strategic effects of condition were present even in the earliest RTs in the distributions.²

2.2.2 Memory Task Performance

Figure 4 shows memory accuracy across the three probability conditions. Memory-task accuracy was high, averaging 96% correct across all conditions and trial types. Specifically, memory accuracy was 95% correct when the memory-matching item was a distractor, 96% correct when the memory item was not present, and was best when the memory item was the target (97% correct). This resulted in a significant effect of trial type (F (2,66) = 8.21, MSE = 7.46, p < .01). The trial type had the largest effect on the 80% probability group then the 50% probability group and had little influence on the 20% probability group, leading to an interaction of trial type X probability condition (F (4,66) = 4.35, MSE=7.46, p < .01).

Figure 4

Memory-task accuracy from Experiment 1. Error bars represent 95% within-subjects confidence intervals.

2.3 Discussion

In Experiment 1, we made two primary observations. First, in the 20% condition, in which a match between the memory item and the target was unlikely, we found both significant costs and benefits on search RT from a match between an object in the search array and the memory representation. Using the classic interpretative framework of Posner and Snyder (1975), this provides evidence for an automatic orienting of attention to memory-matching objects in the search array. Second, we found that the magnitude of both the costs and benefits increased across the conditions with increasing probability of a memory-target match. This effect of probability structure led to a large change in the measured costs and benefits, with the costs and benefits approximately doubling between the 20% memory-target match group and the 80% group. This pattern of results indicates that potent strategic influences modulate the size of the working memory guidance effects relative to the magnitude of the automatic costs and benefits.

There are two factors that may have contributed to the scaling of costs and benefits with increasing probability of a match between the memory item and the search target. First, it is possible that the effects were due to the instructions about the probability structure. In other words, participants conformed their task performance to the explicit instructions causing the top-down effects we observed during Experiment 1. Second, the actual probability structure during the task may have caused the sensitivity to probability that we observed in Experiment 1. That is, the probability that the memory item and the target were the same color may have driven the effects. These alternative explanations are not mutually exclusive, but contrast top-down versus bottom-up sources of the large strategic effects upon attentional guidance by working memory. That is, it is possible that both of these factors contributed to the pattern of effects in Experiment 1. In Experiment 2, we sought to distinguish between these explanations for the large scaling of the costs and benefits observed in Experiment 1 across the probability conditions.

3. Experiment 2

To determine the source of the effects we observed in Experiment 1, we held the actual probability of the different trial types constant and gave different instructions to the participants regarding the probability structure of the experimental design. If the instructions in Experiment 1 influence the probability scaling of the costs and benefits on search RT, then providing different instructions about the probability structure to the participants should induce changes in costs and benefits. In contrast, if the actual probability of the different trial types drove the effects in Experiment 1, then our instructional manipulation should not modulate the costs or benefits. Finally, it is possible that either the RT costs or benefits may be more sensitive to top-down sources of information (i.e., the task instructions). If this were the case, then we would expect the instructional manipulations to dissociate the cost and benefit measures of attention due to a memory match. This would be similar to the findings of Posner and Snyder (1975) that benefits were automatic but that that costs were dependent on observers strategically using the information that was provided.

3.1 Method

All methods in Experiment 2 were identical to Experiment 1, except as follows.

3.1.1 Participants

A new group of 34 participants from Vanderbilt University participated in this study for course credit after giving informed consent. These participants were randomly assigned to one of two possible instructional conditions (80% and 20% memory-match instructions, described below). Participants were replaced if they failed a manipulation check at the end of the experiment in which they had to recall the probability structure they were told during the instructions. This led to the replacement of 9 participants. Two additional participants were replaced because they failed to respond on more than 25% of trials and one was replaced because memory accuracy was more than three standard deviations below the group average.

3.1.2 Design and procedure

The participants in the 80% instruction condition were instructed that when the memory-matching item was present in the array, it would be the search target on 80% of trials. The participants in the 20% instruction condition were instructed that when the memory-matching item was present in the array, it would be the search target on 20% of trials. The actual probability structure presented to both instruction conditions was identical to the 50% memory-match condition in Experiment 1 (total trials: 50% neutral trials, 25% valid trials, and 25% invalid trials). Thus, each participant was given false instructions about the probability structure before beginning the practice block. At the end of the experimental session, the participants were asked two questions. First, they were asked to indicate the proportion of trials the memory-matching item would be the search target that we told them in the instructions. Any participant that failed the manipulation check by incorrectly reporting the instructed proportion of trials where the memory color would match the search target was replaced. Second, they were asked to indicate what they thought the actual proportion of trials that the memory-matching item was the search target when it was in the array.

3.2 Results

3.2.1 Search Task Performance

Search RTs were fastest when the memory-matching item was the target (M = 830 ms) followed by when there was no memory-matching item in the array (M = 1138 ms), and was slowest when the memory-matching item was a distractor (M = 1274 ms; see also Table 1). Mean search RTs were 1034 ms for the 20% instruction condition and 1126 ms for the 80% instruction condition. The ANOVA (within-subjects factor of trial type (valid, invalid, or neutral) and between-subjects factor of instruction condition (20% versus 80%)) yielded significant main effects of trial type (F (2, 64) = 382.05, MSE = 4599, p < .001), as described above, and a significant interaction of trial type X probability condition (F (2, 64) = 4.669, MSE = 4599, p < .05) due to the larger effects of trial type in the 80% than the 20% condition. The main effect of probability instruction failed to reach significance (p = .07). Search Accuracy was above 98% correct across all trial types and conditions (p > .30).

To further examine the influence of instruction on search RT, we next contrasted the costs and benefits derived from the two conditions. As illustrated in Figure 5, the 20% instruction condition showed a 124 ms mean cost and the 80% instruction condition showed a 149 ms cost. The RT benefits were 71 ms smaller for the 20% instruction condition (M = −272 ms) than the 80% instruction condition (M = −343 ms). Entering these data into an ANOVA yielded a main effect of memory-match influence (F (1, 32) = 440.73, MSE = 7598, p < .001) and an interaction of memory-match influence X instruction condition (F (1, 32) = 5.26, MSE = 7598, p < .05). The main effect of instruction condition failed to reach significance (p = .18).

Figure 5

The derived RT costs and benefits from Experiment 2. The RTs are shown in terms of costs (invalid trial mean RT minus neutral trial mean RT) and benefits (valid trial mean RT minus neutral trial mean RT). Error bars for the costs and benefits represent (more ...)

To directly assess the influence of instruction on search performance, we next examined the costs and benefits separately. We found that there was no effect of instruction on costs (p = .20). However, instructions that the memory-match would be more likely to be the target led to increased benefits (F (1, 32) = 4.72, MSE = 9180, p < .05). Thus, the analyses of costs and benefits indicate that benefits were more sensitive to strategic control governed by the instructions alone than the costs, which more closely tracked the objective probability of trial types.

To directly compare the magnitude of the RT costs and benefits between Experiment 1 and 2 we performed an additional analysis using the within-subjects factor of memory-match influence (costs or benefits) and the between-subjects factors of probability (20% versus 80%) and experiment (Experiment 1 versus 2). As expected, we found a main effect of memory-match influence (F (1,54) = 644.4; MSE = 7845, p < .001), and an interaction of memory-match influence and condition (F (1,54) = 30.0, MSE = 7845, p < .001). The 71 ms increase in benefits and 25 increase in costs across the 20% and 80% instruction conditions in Experiment 2 was smaller than the 192 ms increase in benefits and the 78 increase in costs from 20% to 80% conditions in Experiment 1. This pattern created a significant interaction of memory-match influence X condition X experiment (F (1,54) = 6.598, MSE = 7845, p <.05).³

As in Experiment 1, we also examined the RT distributions to determine if instructions would lead to a shift in the distribution or to a change in the slope of the distribution as would be predicted if cognitive control comes online during the processing of the search stimuli (Han & Kim, 2009). The distributions in Figure 6 show that we again observed a shift in the entire distribution. Similar to results of Experiment 1, Wilcoxon rank tests found that the costs in earliest quantile were larger for the 80% condition (188 ms) compared to the 20% condition (136 ms; Z = 2.02; p < .05) and benefits for the earliest quantile were larger for the 80% condition (165 ms) than the 20% condition (116 ms; Z = 1.96; p < .05). These findings indicate that the RT effects of manipulating the instructions influenced the speed of even the fasted responses and not simply the long tail of the distribution as expected based on previous proposals about the timing of strategic effects.

Figure 6

Reaction time distributions from Experiment 2. Data points represent vincentized values averaging the responses of all subjects in a condition at each quantile. Solid lines are responses when the memory-matching item was the target, dashed lines are the (more ...)

3.2.2 Memory Task Performance

Figure 7 shows that memory accuracy was high across trial types and conditions (M = 96% correct). As in Experiment 1, performance was influenced by the type of search array participants viewed on each trial (F (2, 64) = 4.60, MSE = 8.96, p < .05). Performance was best when they had just searched an array where the memory item as the target (i.e., valid trials, M = 97% correct) compared to when the memory item was not present (i.e., neutral trials, M = 95% correct) and when the memory item was a distractor (i.e., invalid trials, M = 95% correct). The pattern of data for the 80% instruction condition and 20% instruction condition matched the pattern found in Experiment 1. Participants in the 80% instruction condition were most accurate on the memory task on valid trials (97% correct), followed by neutral trials (94% correct) and invalid arrays (94% correct), while the 20% instruction condition observers showed similar performance in all 3 conditions (correct performance on 97% of valid trials, 97% neutral trials, and 96% invalid trials). However, the interaction of instruction and trial type did not reach significance (p = .40).

Figure 7

Memory-task accuracy from Experiment 2. Error bars represent 95% within-subjects confidence intervals.

3.3 Discussion

In Experiment 2, the effects of instruction were accompanied by effects due to the objective probability of the different trial types when compared to Experiment 1. However, we found that the magnitude of the RT benefit was influenced by instruction alone, even when both groups experienced the same objective probability of the different trial types. Despite the difference in the size of the benefit effect across Experiment 1 and 2, the increase indicates that top-down influences by themselves can alter the way information in working memory is used to guide attention. In contrast, the costs in Experiment 2 were not significantly increased by instruction alone. We discuss possible explanations for this difference in top-down control on costs and benefits in the General Discussion. In summary, our findings from Experiment 2, which show influences of both the actual probability and the expectations due to instructions, converge with those of Experiment 1 in demonstrating roles for both automatic and strategic influences on how attention is deployed.

4. General Discussion

In this study, we used the classic framework of Posner and Snyder (1975) to examine the automatic and strategic components of the guidance of attention by working memory representations. In Experiment 1, we found automatic effects of working memory representations on the guidance of visual attention. We also found that strategy could double the size of the effects. In Experiment 2, we found that benefits were scaled by instruction, while costs were not affected by instructions. These findings provide insights into the automatic and strategic interactions of working memory representations and the deployment of attention within a single paradigm.

Overall, we found that the size of the benefits from the memory-matching item appearing in the array as the search target were more than twice the size of the costs due to the memory-matching item appearing as a distractor. This is a critical departure from several of the classic studies of spatial attention. In the context of spatial cuing paradigms, Posner (1978; 1980) found that costs and benefits of cuing spatial attention were of similar magnitude. The study of Posner and Synder (1975) on priming is an important exception, in that they found significant RT benefits in the absence of observable costs (i.e., with low cue validity probabilities).

The findings of benefits on RT by Posner and Synder (1975) are typically interpreted as providing evidence that a priming stimulus automatically enhances the activity of perceptual detectors for that stimulus. The costs, which became evident in their study with higher valid trial probabilities, were proposed to be due to strategically controlled inhibition of the activity of detectors for the other possible targets that are not cued by the prime. The asymmetry that we observed in the present study could be interpreted in a similar vein. Specifically, our observation that RT benefits are more than twice the size of the costs might suggest that there is asymmetric enhancement of features matching those in memory relative to the magnitude of suppression of features other than those stored in working memory. The findings from Experiment 2 provide a useful piece of additional evidence in that they show that the benefits are also more sensitive to the instructional manipulations than the costs. This further supports the idea that the mechanisms underlying the positive and negative RT effects of attentional guidance from working memory are, at least to some degree, independent.

The asymmetric benefits and costs we observed could be due to distinct mechanisms that perform target enhancement versus distractor suppression (Eriksen & Hoffman, 1974). This theoretical proposal has gained strong support from event-related potential (ERP) studies. Specifically, a body of work indicates that the P1 component of the ERP waveform indexes a mechanism involved in distractor suppression whereas attentional modulations of the N1 component measures a mechanism of target enhancement (Luck, 1995). More recently, ERP experiments have suggested that dissociable mechanisms of target enhancement and distractor suppression are a part of other ERP measures of attentional selection (Hickey, Di Lollo, & McDonald, 2009). However, we believe that this perspective of directly relating costs to the suppression of features and benefits to the enhancement of features needs to be treated with caution in the dual-task paradigms used to study interactions between working memory and attention. In the present experiments, the neutral trials involved presenting a memory item that could have, but did not appear in the array, unlike the truly neutral cue in the classic priming experiments of Posner and Snyder (i.e., a “+” instead of a letter). Thus, it is likely that even during our neutral trials, and those in previous studies (Soto et al., 2005), any mechanism that serves to suppress non-memory-matching features was operative (e.g., red and green would be suppressed when the memory item was blue). We believe that future research using neuroscientific markers is needed to provide definitive evidence regarding how working memory representations influence mechanisms of suppression versus enhancement.

For both costs and benefits, we found that strategy could double the size of the effects due to a match between the working memory representation and an item in the search array. This means that strategic use of the representations in working memory has large modulatory effects on the attention mechanisms enabling the performance of visual search. Moreover, our experimental design may have actually underestimated the contribution due to strategic factors. Although we used the 20% valid condition as a measure of purely automatic effects, performance in this condition could also be influenced by strategy. Previous studies have shown qualitative changes in how the contents of working memory guide attention when the likelihood of memory-target matches goes from 0 to 16.7% of trials (Woodman & Luck, 2007). Although the memory color was just as likely to be paired with the search target shape as the other colors in the array, our observers may have used the strategy of beginning search for the target shape at the location of the memory-matching color. Indeed, a number of participants spontaneously commented during the debriefing period that they found it easier to start their search with the memory-color matching item.

We also found that the strategic effects were just as evident in the earliest tail of the RT distribution as in later RTs. This is contrary to the account of Han and Kim (2009) that voluntary cognitive control takes a considerable amount of time to implement following the onset of a visual scene. Our failure to find a shift in the state of cognitive control across time cannot be explained by timing differences between our study and that of Han and Kim (2009). The presentation duration of the search array and the ISI between memory item and search array used in the present experiments were nearly identical to the timing used in their study. In addition, our RTs were in line with those of the easy perceptual task in Experiment 2 of their paper. Han and Kim (2009) explained the significant influence of working memory on attention in this easy perceptual task by stating that it was impossible to implement cognitive control before the responses were made. However, we found that both probability and instructional manipulations resulted in shifts of the entire RT distributions due to strategic control, including very early RTs. Therefore, our current findings are inconsistent with explanation based on the timing of cognitive control proposed by Han and Kim (2009).

A number of previous studies have not found an effect of working memory representations on visual attention (Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006; Peters et al., 2009; Soto & Humphreys, 2008; Woodman & Luck, 2007). How can we reconcile these findings with those presented in the current paper? In the current paper, we sought to measure automatic effects and determined if strategic control of attention could modulate the size of those automatic effects. In essence, both the strategic and automatic effects were working toward creating an influence of working memory representations on RT measures of visual attention. In contrast, these previous studies have pitted strategic effects against automatic effects by ensuring that no valid trials were included (Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006; Peters et al., 2009; Soto & Humphreys, 2008; Woodman & Luck, 2007). Our finding that the actual probability also has an effect from Experiment 2, may help explain why the previous studies that included valid trials concluded that working memory guides attention. If it is the case that the strategic effects are at least equivalent in size to the automatic effects, as we found in our study, then when automatic effects guide attention toward the memory-matching item and strategic effects guide attention away, these competing effects will cancel each other out.

Before we conclude, let us consider the larger implications of our findings. First, the original goal of studies examining the influence of working memory representations on the deployment of attention was to test theories of attention (Bundesen, 1990; Bundesen, Habekost, & Kyllingsbaek, 2005; Desimone & Duncan, 1995; Duncan, 1996; Duncan & Humphreys, 1989). Accounts that propose feature specific attentional weights (Bundesen, 1990; Bundesen, Habekost, & Kyllingsbaek, 2005) can easily be modified to account for our results by assuming stronger weightings for the features of the memory representations with increasing valid-trial probability. However, according to the theory of biased competition (Desimone & Duncan, 1995), holding a template in working memory is sufficient to successfully bias perceptual mechanisms to select task-relevant items in complex scenes. That is, the target template functions as a mechanism of top-down control without intervention by higher-level executive control mechanisms (in contrast see Miller & Cohen, 2001). In our present study, we used visual search arrays composed of items that were similar in bottom-up salience. Therefore we only measured the influence of top-down factors. We found that the automatic effect was doubled by strategic contributions in this situation where bottom-up effects do not conflict with the top-down biases. In everyday life, it is likely that the task-relevant inputs will not be the most salient items in the visual field. In these cases, strategic effects may be essential for directing attention to the task-relevant items. If this is the case, then biased competition is missing the crucial strategic component of attentional control. Thus, we argue that the biased competition theory should be extended to account for the large strategic effects that may swamp the automatic guidance of perceptual attention mechanisms to memory-matching inputs.

Figure 3

Reaction time distributions from Experiment 1. Data points represent vincentized values averaging the responses of all subjects in a condition at each quantile. Solid lines are responses when the memory- matching item was the target, dashed lines are (more ...)

Acknowledgements

We would like to thank Gordon Logan and Trisha Van Zandt for guidance with the analyses of the RT distributions. We would also like to thank Donald Bruce Ross for help with data collection. This work was supported by grants from NSF (BCS-0957072) and NEI (RO1-EY019882).

Footnotes

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

¹To remove trials with outlying search RTs we identified trials where the RT was more than 3 standard deviations from the mean for each participant within each condition were trimmed (Van Selst & Jolicoeur, 1994).

²The classic recommendation for creating individual cumulative distributions functions suggests a minimum of 10 observations per cell per subject (Ratcliff, 1979). Given the nature of the probability manipulation, we were not able to achieve this number of observations for all cells for each subject in each group (i.e. the 20% group only had 24 valid trials). Therefore, these results should be viewed with some caution. However, examination of the pooled group data leads to similar results. Costs and benefits of the 1st quantile of the pooled group data are very similar to the means calculated from individual distributions (20% group 78 ms costs and 93 ms benefits, 50% group 126 ms costs and 127 ms benefits, 80% group 217 ms costs and 195 ms benefits). Observation of the first correct pooled group responses for valid and invalid trials is telling. The first correct invalid response for the 20% group was made before the first correct valid response, whereas the first correct invalid response for the 50% group was made after 9.6% of valid responses, and the first correct invalid response for the 80% group was made after 58.3% of all correct valid responses. Consistent with these group findings, more recent work suggests that the 10-observation threshold is not particularly special because lower sample sizes do not create systematic biases in the observed distributions provided the data are not so sparse such that large portions of the cumulative distributions functions are missed (Van Zandt, 2000).

³An alternative explanation for the influence of the probability manipulation on costs and benefits is inter-trial effects. Increasing the probability of valid trials also increases the probability of N-1 valid trials, which has the potential of increasing the costs and benefits. To address this alternative explanation, in separate analyses, we examined the influence of N-1 array type on costs and benefits. We found no main effect of N-1 array type or interaction of array type and condition (ps > .10), indicating that this alternative explanation cannot explain the influence of probability.

References

Brainard DH. The Psychophysics Toolbox. Spatial Vision. 1997;10:433–436. [PubMed]
Bundesen C. A theory of visual attention. Psychological Review. 1990;97:523–547. [PubMed]
Bundesen C, Habekost T, Kyllingsbaek S. A neural theory of visual attention: Bridging cognition and neurophysiology. Psychological Review. 2005;112:291–328. [PubMed]
Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience. 1995;18:193–222. [PubMed]
Downing PE. Interactions between visual working memory and selective attention. Psychological Science. 2000;11:467–473. [PubMed]
Downing PE, Dodds CM. Competition in visual working memory for control of search. Visual Cognition. 2004;11:689–703.
Duncan J. Cooperating brain systems in selective perception and action. In: Inui T, McClelland JL, editors. Attention and performance XVI: Information integration in perception and communication. Cambridge, MA: MIT Press; 1996. pp. 549–578.
Duncan J, Humphreys GW. Visual search and stimulus similarity. Psychological Review. 1989;96(3):433–458. [PubMed]
Eriksen CW, Hoffman JE. Selective attention: Noise suppression or signal enhancement? Bulletin of the Psychonomic Society. 1974;4:587–589.
Han SW, Kim M-S. Do the contents of working memory capture attention? Yes, but cognitive control matters. Journal of Experimental Psychology: Human Perception and Performance. 2009;35:1292–1302. [PubMed]
Hickey C, Di Lollo V, McDonald JJ. Electrophysiological indices of target and distractor processing in visual search. Journal of Cognitive Neuroscience. 2009;21:760–775. [PubMed]
Houtkamp R, Roelfsema PR. The effect of items in working memory on the deployment of attention and the eyes during visual search. Journal of Experimental Psychology: Human Perception and Performance. 2006;32:423–442. [PubMed]
Jonides J. Voluntary versus automatic control over the mind's eye's movement. In: Long JB, Baddeley AD, editors. Attention and Performance IX. Hillsdale, New Jersey: Erlbaum; 1981. pp. 187–203.
Luck SJ. Multiple mechanisms of visual-spatial attention: Recent evidence from human electrophysiology. Behavioural Brain Research. 1995;71:113–123. [PubMed]
Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Annual Review of Neuroscience. 2001;24:167–202. [PubMed]
Olivers CNL. What drives memory-driven attentional capture? The effects of memory type, display type and search type. Journal of Experimental Psychology: Human Perception and Performance. 2009;35:1275–1291. [PubMed]
Olivers CNL, Meijer F, Theeuwes J. Feature-based memory-driven attentional capture: Visual working memory contents affects visual attention. Journal of Experimental Psychology: Human Perception and Performance. 2006;32:1243–1265. [PubMed]
Peters JC, Goebel R, Roelfsema PR. Remembered but unused: The Accessory items in working memory that do not guide attention. Journal of Cognitive Neuroscience. 2009;21:1081–1091. [PubMed]
Posner MI. Chronometric Explorations of Mind. Hillsdale, New Jersey: Erlbaum; 1978.
Posner MI. Orienting of attention. Quarterly Journal of Experimental Psychology. 1980;32:3–25. [PubMed]
Posner MI, Snyder CR. Facilitation and inhibition in the processing of signals. In: Rabbitt PM, Dornic S, editors. Attention and Performance. Vol. 5. San Diego, CA: Academic Press; 1975. pp. 669–682.
Ratcliff R. Group reaction time distribution and an analysis of distribution statistics. Psychological Bulletin. 1979;86(3):446–461. [PubMed]
Soto D, Heinke D, Humphreys GW, Blanco MJ. Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance. 2005;31(2):248–261. [PubMed]
Soto D, Hodsoll J, Rotshtein P, Humphreys GW. Automatic guidance of attention from working memory. Trends in Cognitive Sciences. 2008;12:342–348. [PubMed]
Soto D, Humphreys GW. Stressing the mind: The effect of cognitive load and articulatory suppression on attentional guidance from working memory. Perception & Psychophysics. 2008;70(5):924–934. [PubMed]
Van Selst M, Jolicoeur P. A solution to the effect of sample size on outlier elimination. Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology. 1994;47A(3):631–650.
Van Zandt T. How to fit a response time distribution. Psychonomic Bulletin & Review. 2000;7:424–465. [PubMed]
Woodman GF, Luck SJ. Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of Experimental Psychology: Human Perception and Performance. 2007;33:363–377. [PMC free article] [PubMed]
Yantis S, Jonides J. Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal of Experimental Psychology: Human Perception and Performance. 1990;16:121–134. [PubMed]