Experiment 1: ERP correlates of Selecting and Tracking moving Objects
On each trial, subjects were presented a bilateral array containing six squares in each hemifield (see ). For the first 500ms of each trial (cue period), the objects were stationary with a subset of the items in a given hemifield drawn in red (the targets) and the remaining items drawn in black (the distractors). Green items appeared at the start of each trial on the unattended side. These items were photometrically isoluminant and equal in number to the red target items on each trial. Half of the subjects tracked red items while the others tracked green. After 500ms, the red and green items changed to black and all of the objects began to move amongst each other in random directions within the hemifield for 2 seconds; at that point, the items stopped moving and one item turned red. Subjects were instructed to attentionally track the targets and pressed one of two buttons to indicate whether the final red item was one of the targets or not. We time-locked the ERPs to the onset of the cue array and recorded throughout the duration of the trial so that we could observe both the transient selection of the targets during the cue period as well as the sustained attention response during the tracking period. In Experiment 1, we asked subjects to track one, two, or three targets on each trial so that we could determine whether the activity was modulated by the number of tracked items.
200ms following the onset of the cue array, we observed a transient negative-going wave over the hemisphere that was contralateral to the attended hemifield. This activity was followed by a larger and sustained contralateral negative wave that began shortly after the tracking period started and persisted throughout the course of the trial until the test was presented. As shown in , the amplitude of both of these waves was strongly modulated by the number of target items; increasing the number of targets resulted in substantial increases in amplitude (3 targets > 2 targets > 1 target; all p's < .01). Moreover, the amplitude of this activity was highly sensitive to whether or not the subject performed the tracking task correctly: both waves showing large, significant decreases in amplitude on error trials relative to correct trials (both p's < .01). This indicates that both waves reflect processes that are necessary antecedents to correct tracking performance. shows the distribution of these waves across each of the lateral recording sites. The transient activity during the selection phase was primarily centered over posterior electrodes with a maximum over lateral occipital electrodes (OL/OR). During this selection period there was no significant lateralized activity observed over frontal electrodes (F < 1). The sustained activity during the tracking period was more broadly distributed over the posterior electrode sites with a maximum over posterior parietal electrodes (PO3/PO4). This activity was also observed over frontal electrode sites (F3/F4), though the contralateral effect at these sites was not significantly modulated by the number of tracked targets (F=2.3, p >.10).
The transient wave during the cue period appears to be the N2pc wave which, as described in the Introduction, has previously been shown to reflect the selection of targets amongst distractors in visual search tasks (Luck et al., 1997
; Hopf et al., 2000
; Woodman and Luck, 2003
). By contrast, the large sustained wave during tracking appears to be the contralateral delay activity (CDA) that we and others have shown reflects the number of active object representations held in visual short term memory (VSTM) (e.g., Vogel and Machizawa, 2004
). Together, the N2pc and CDA waves appear to index two critical components of attentional tracking: the initial selection of the target objects during the cue period (N2pc); and sustained attention towards the target representations as they move about the hemifield (CDA). Although the N2pc and the CDA were both modulated by the number of targets, we found that these two waves have distinct scalp distributions yielding a highly significant electrode position by time window (200−300ms vs 800−1200ms) interaction (p<.01, See Methods): with the N2pc showing a more ventral distribution than the more dorsal CDA. This finding supports a previous demonstration of distinct scalp distributions for these two components in the context of a working memory task (McCollough et al., 2007
). Together, these results suggest that while there appears to be a tight coupling between object selection and sustained attention towards the targets, they may reflect the output of distinct cortical areas.
Experiment 2: Spatial extent of attention or number of objects?
Although the amplitude of both the N2pc and CDA in the first experiment increased as a function of the number of targets, it is possible that this increase is simply due to the required spatial extent of the target area rather than reflecting the increasing number of targets selected and tracked during the trial. That is, as the number of target items increases, there is also potential for a corresponding increase in the area of the attentional window or “spotlight” that encompasses the targets and this may be what caused the increases in amplitude in the first experiment (e.g., Eriksen and St. James, 1986
; Hillyard et al., 1998
). To test this alternative, in the second experiment we directly manipulated the amount of area required to track the targets. Subjects tracked two or three targets that either encompassed a large area or a small area within the hemifield. We found that while the amplitudes of both the N2pc and CDA were again significantly modulated by the number of targets (both p's < .01), there was no significant effect of area on amplitude for either wave (both F's < 1; see ). We did however find a significant effect of area on behavioral tracking performance, where performance in the small area conditions was significantly poorer (~10%) than in the large area conditions (p < .01). These results are consistent with previous studies that have shown that displays with a high density of items result in more difficult tracking and poorer performance (e.g., Intriligator and Cavanagh, 2001)
. It also helps to confirm that our manipulation of area was substantial enough to observe a significant behavioral effect. Indeed, the lack of an amplitude modulation by area also argues against the hypothesis that the amount of general effort or difficulty required to track more targets is the cause of the observed increase in amplitude. That is, despite the small area condition being significantly more difficult than the large area condition, there was no concomitant rise in amplitude for either the N2pc or the CDA. Nonetheless, it is important to note that the apparent dissociation between behavioral performance and CDA amplitude in this experiment may be due to a limitation of our measure. In particular, it is possible that poorer behavioral performance in the small area condition is due to the subjects inadvertently tracking distractor items that were mistaken, or swapped, for target items during the course of the trial due to the closer proximity of targets and distractors. This scenario would lead to a decrease in behavioral performance because the wrong items were being tracked. However, it would predict no change in CDA amplitude because the same total number of items are being tracked on the trial. Specifically, the limitation of this component is that it provides an index of the number of objects currently being tracked irrespective of whether or not they are targets.
Fig. 3 (A) Behavioral performance in Experiment 2 showing significant main effects of both area and number of items. (B) Mean amplitude of CDA activity in Experiment 2. While there was a significant main effect of number of targets, area had no significant effect (more ...)
Experiment 3: Sensitivity to behavioral tracking limitations
The results of the first two experiments are consistent with the proposal that the amplitude of both the N2pc and the CDA reflects the number of targets being selected or tracked, respectively. However, to strengthen this claim it is necessary to demonstrate that this activity is indeed sensitive to the known behavioral performance limitations associated with attentional tracking. Therefore, in the third experiment we measured these two waves under a task condition that is likely to exceed the subject's tracking capacity so that we could determine whether this activity is sensitive to these performance limitations. Indeed, this has been a significant limitation of previous neuroimaging studies examining tracking-related load effects because they have not tested whether the observed activity continues to increase when the number of targets exceeds capacity. In addition, by examining a wider range of target array sizes, we can begin to examine whether these two types of activity are sensitive to differences across individuals in tracking ability. In this experiment, subjects tracked one, three, or five targets on each trial. In this experiment, all trials contained 10 items so that 50% of the items were distractors when subjects tracked 5 items. We divided subjects into high capacity and low capacity groups on the basis of a median split of their behavioral tracking capacity (see Methods). shows the N2pc and CDA waves for each target array size for the high and low capacity groups. As can be seen in the figure, both groups showed an increase in amplitude for both the N2pc and the CDA from one to three targets (low capacity: both p's < .05; high capacity: both p's < .001). However, the two groups diverged greatly when tracking five items. The amplitude for the high capacity group when tracking five items remained equivalent to that of tracking three items (N2pc: F<1; CDA: p > .15). Thus, when given more items than they could track, the high capacity subjects appeared to be able to continue to track their limit of objects (i.e., ~3 items). However, for the low capacity group, the track five amplitude decreased significantly below the three item level and was equivalent to that of tracking a single item (N2pc: p < .001; CDA: p < .05). While the precise cause of this amplitude decrease is currently unclear, it does appear to reflect a consistent pattern across all subjects dependent upon their specific tracking capacity. That is, there was a significant negative correlation between an individual's tracking capacity and the amount of decrease between three targets and five targets (r = −.60 N2pc; r = −.56 CDA; both p's < .01), such that as tracking capacity increased the amount of amplitude drop decreased. In summary, the results of Experiment 3 provide further evidence that the amplitude increases of the N2pc and CDA are the consequence of the number of items that are currently being selected or tracked. In particular, these results demonstrate that the amplitude is not simply driven by the amount of cognitive load required to perform the task because the amplitude of each component reached an asymptotic limit at roughly 3 items, even though the amount of cognitive load continued to increase when the subjects attempted to track 5 items. Thus, the properties of these neural mechanisms appear to be finely sensitive to the known capacity limitations associated with attentional tracking.
Fig. 4 ERP difference waves for correct trials in Experiment 3 divided between high capacity (A) and low capacity individuals (B) on the basis of a median split of tracking performance. Mean amplitude (in microvolts) of the N2pc (C) and the CDA (D) for the high (more ...)
Experiment 4: Predicting individual differences in tracking capacity
The results of Experiment 3 indicate that the amplitude of both the N2pc and the CDA are highly sensitive to the tracking capacity limitations that constrain performance in this task because it reaches a limit at tracking three targets and is also finely attuned to individual differences in tracking capacity. However, this sensitivity to individual differences was not restricted to the response to supracapacity target arrays, but was also observed in the size of the increase in amplitude from one target to three targets. This resulted in a highly significant interaction between group (high vs low) and number of targets (1 vs 3) (N2pc: p < .001; CDA: p < .01), with a larger increase from one to three targets for the high capacity group than for the low capacity group. The smaller difference in amplitude between one and three targets for the low capacity group suggests that the one-target arrays consumed a larger proportion of available capacity than for the high capacity group, resulting in a smaller increase to three items. Paired t-tests support this assertion because the difference between the high and low groups was not significant in the track 1 condition (p's > .15) but the difference between these two groups was highly significant in the track 3 condition (N2pc: p < .005; CDA: p < .01).
We tested the robustness of this relationship by running an additional group of subjects in the one and three target conditions and combining this data with all of the subjects from the previous experiments so that we could have a large sample (N=63). shows the amplitude of both waves for tracking one or three targets divided between high capacity and low capacity subjects. From the figure, there are two apparent differences between the high and low capacity groups: first, the high capacity group tends to have overall larger amplitudes for each wave; and second, the high capacity group shows a larger rise in amplitude from 1 to 3 items than the low capacity group. This pattern of effects was confirmed in an Analysis of Variance (ANOVA), yielding significant main effects of group (both p's < .05) and number of targets (both p's < .001), as well as a significant interaction between group and number of targets (p < .01). Although high capacity subjects tend to have higher overall amplitudes (irrespective of number of targets), this factor is only a fairly weak to moderate predictor of an individual's tracking capacity (N2pc: r=.22, p <.10; CDA: r=.31, p <.05). By contrast, we found that the rise in amplitude from one target to three targets was a much stronger predictor of an individual's tracking capacity (N2pc: r=.70, p < .001; CDA: r =.48; p < .001). Importantly, these strong correlations persisted even when we partialled out the relationship between overall amplitude and tracking capacity (partial r's = .68 and .41 for N2pc and CDA, respectively). Thus, it appears that it is the amount of differentiation in amplitude between increasing numbers of targets that may be most predictive of an individual's tracking capacity. We also found that the rise in N2pc amplitude from one to three targets was strongly correlated with the rise of the CDA (r = .72, p < .001) which further indicates that there is a tight coupling between these measures of object selection and sustained attention. However, because of this strong relationship, we also calculated partial correlations for both the N2pc and CDA effects (i.e., rise from 1 to 3 targets) so that we could measure each wave's unique contribution to predicting tracking capacity. Although the N2pc effect remained a strong predictor of tracking capacity when the contribution of the CDA effect was removed (partial r = .59, p < .001), the CDA effect was only a weak predictor of tracking capacity when the N2pc effect was removed (partial r = .09; ns). Importantly, these effects were not simply due to more variability in the CDA than the N2pc. Measurements of the reliability of each component revealed that both components were highly stable within subjects, and that the CDA actually had a higher reliability than the N2pc (Cronbach's alpha = 0.74 for the N2pc; 0.94 for the CDA). Consequently, these results demonstrate that while neural indices of both target selection (N2pc) and sustained attention (CDA) can serve as strong neurophysiological predictors of attentional tracking capacity, it is the selection process that explains most of the unique variance in tracking capacity across individuals.
Fig. 5 (A, B) ERP difference waves for high and low capacity subjects in Experiment 4. (C, D) Mean amplitudes of the N2pc and CDA waves across high and low capacity groups. There was a significant interaction between group (high/low) and number of objects for (more ...)
Experiment 5: Limiting factor for tracking capacity: selection or tracking?
Our observation that how efficiently an individual initially selects the target items strongly predicts their overall tracking capacity is somewhat surprising because selection occurs well before tracking (i.e., motion onset) even begins. In this regard, one could argue that there must always be a strong relationship between selection and tracking performance because subjects can track only the targets that were appropriately selected in the first place. However, there are likely to be many processes that contribute to an individual's overall tracking capacity depending upon the specific nature of the tracking task that is being used to estimate capacity (vanMarle and Scholl, 2003
; Oksama and Hyönä, 2004
; Alvarez et al., 2005
; Liu et al., 2005
; Pylyshyn and Annan, 2006
). Indeed, our behavioral estimate of tracking capacity may actually load heavily on the selection stage because the subjects were required to hold fixation while selecting a subset of targets amongst distractors within a single hemifield. Moreover, it is possible that there is a somewhat weaker contribution of sustained attention activity in our behavioral measure because our tracking period is relatively short (i.e., 1.5 seconds) compared to previous studies that tend to use longer periods of tracking (e.g., 8−10 seconds).
In the final experiment we tested whether these two neural predictors of tracking capacity would be sensitive to a change in the relative contributions of selection and sustained attention by assessing each component's (i.e., N2pc and CDA) ability to predict an individual's tracking capacity in a “whole field” tracking task with a longer duration. More specifically, subjects were tested in two separate sessions. In a behavior-only session, subjects were asked to track 3, 4, or 5 target items amongst distractors that were spread across the entire visual field (“whole-field”) and they tracked these items for 8 seconds. In a separate ERP session, subjects performed a single hemifield tracking task that was identical to that used in Experiment 4. We estimated each subject's “whole field” tracking capacity on the basis of performance in the behavior-only session, and used this estimate as a predictor of his or her N2pc and CDA effects that were measured in the single hemifield ERP tracking task. In a “whole field” tracking situation, the difficulty of target selection should be reduced because the subjects could freely view and select the targets across the entire display. In contrast, the difficulty of sustained attention should be raised because of the substantial increase in how long the targets needed to be tracked continuously. Consequently, we would expect that the N2pc effect should now become a weaker predictor of “whole field” tracking capacity; simultaneously, we expect that the CDA should become a stronger predictor of tracking capacity as the limiting factor in task performance shifts from selection to sustained attention. As shown in , we observed that while the correlation between the N2pc difference effect and whole field tracking capacity was considerably weaker than we observed previously (r= .31, p < .07), the CDA difference became a much stronger predictor of tracking performance (r= .72; p <.001). Again, the N2pc and CDA effects were strongly correlated (r = .52, p < .05). Moreover, when we partialled out the contribution of the N2pc effect, the relationship between the CDA effect and tracking capacity remained strong (partial r = .69; p < .01); Conversely, the N2pc was no longer predictive of tracking capacity when the CDA effect contribution was removed (partial r = .10; ns). Thus, in this “whole field” tracking context, it is our index of sustained attention that explains most of the unique variance in attentional tracking capacity across individuals.
Correlations between an individual's Whole Field tracking capacity and the rise in amplitude from 1 to 3 targets for the N2pc (A) and the CDA (B). Tracking capacity was estimated by averaging behavioral performance across all set sizes (3, 4 and 5).