Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Neurosci. Author manuscript; available in PMC 2008 October 20.
Published in final edited form as:
PMCID: PMC2570324

Neural measures of individual differences in selecting and tracking multiple moving objects

Monitoring Editor: Dr. Fitzpatrick and Dr. Kastner


Attention can be divided so that multiple objects can be tracked simultaneously as they move amongst distractors. Although attentional tracking is known to be highly limited such that most individuals can track only about four objects simultaneously, the neurophysiological mechanisms that underlie this capacity limitation have not been established. Here, we provide electrophysiological measures in humans of the initial selection and sustained attention processes that facilitate attentional tracking. Each measure was modulated by the number of objects the subject was tracking and was highly sensitive to each individual's specific tracking capacity. Consequently, these measures provide strong neurophysiological predictors of an individual's attentional tracking capacity. Moreover, by manipulating the difficulty of these two phases of the task we observe that the limiting factor underlying tracking capacity can flexibly shift between these two attentional mechanisms depending upon the requirements of the task.

Keywords: Attention, event-related potentials, capacity limits, individual differences, cognitive, correlated variability

Common tasks such as driving a car in traffic are dependent upon our ability to simultaneously attend multiple objects as they move about in the visual field. This ability is known to be highly limited such that most individuals can track only about four moving objects simultaneously (Pylyshyn and Storm, 1988; Scholl et al., 2001; Cavanagh and Alvarez, 2005). Multiple object tracking is thought to require at least two components of visual attention: a transient selection process that initially determines which items will be tracked, and a sustained process that keeps an updated representation of each object as it moves amongst identical distractors (Yantis, 1992; Alvarez and Cavanagh, 2005; Pylyshyn and Annan, 2006). Although previous imaging and neurophysiological studies have reported neural correlates of both selection (Woodman and Luck, 1999; Buschman and Miller, 2007) and tracking (Culham et al., 1998; Culham et al., 2001; Jovicich et al., 2001) these two mechanisms have typically been studied in isolation. Moreover, the relationship between these types of activity and the capacity limitations that constrain attentional tracking has not been demonstrated. As a result, it is still unclear whether capacity limits in tracking are due to limitations of initially selecting multiple targets amongst distractors, sustaining attention to the moving targets, or some combination of these two factors.

Recent neuroimaging studies of attentional tracking have reported that the Intraparietal sulcus (IPS) and the Superior Frontal Sulcus (SFS) show significant load-dependent activations, such that as the number of items tracked increased, BOLD activation in these areas also increases (Culham et al., 1998; Culham et al., 2001; Jovicich et al., 2001). However, it is still currently ambiguous what these load-dependent activations actually reflect. For example, increases in tracking load are necessarily accompanied by increases in task-general processes such as effort and arousal, which makes it difficult to determine whether the increasing cortical activity is the result of more attended object representations or simply due to the subject expending more effort when tracking more items. A further ambiguity of these studies regards which component of attention underlies these load effects: is it driven by the initial selection of the targets, or does the activity reflect the sustained attention to the items as they move about the visual field? Because the previous neuroimaging studies of tracking all used FMRI, the poor temporal resolution of the technique makes it difficult to disentangle the quick sequence of attentional events in this task.

In the current study, we sought to establish distinct electrophysiological measures of target selection and sustained attention during a tracking task as a means of determining which of these components of attention is the principal limiting factor in tracking performance. To do this, we recorded event-related potentials (ERPs) from subjects while they performed a multiple object tracking task (MOT) in which they were presented a bilateral array of objects and were instructed to attend a subset of objects in a single hemifield. The advantage of this bilateral stimulus design is that it allows us to isolate the lateralized effects of attention from the bilateral perceptual response evoked by the onset and motion of the stimuli in the display. There are several candidate ERP components that have been observed in lateralized attention tasks that may play a role in both the selection and sustained attention to the moving targets in a tracking task. In terms of initially selecting the targets, we expect to observe an N2pc component, which is a transient contralateral negative wave appearing at approximately 200ms post-stimulus over posterior electrode sites (Luck and Hillyard, 1994; Eimer, 1996). This component has been shown to reflect the selection of targets amongst distractors in visual search tasks and has been localized to generators in extrastriate cortex, including V4 and posterior portions of inferior temporal cortex (Luck et al., 1997; Hopf et al., 2000; Hopf et al., 2002; Hopf et al., 2006). Moreover, it appears to be functionally equivalent to another component labeled the “early directing attention negativity” (EDAN) (Harter et al., 1989; Van Velzen and Eimer, 2003). At more frontal electrode sites, it is also possible that we would observe an ADAN (anterior directing attention negativity), which is a transient negative wave (350−500ms) that is thought to reflect control signals in prefrontal cortex involved in orienting attention towards the general location of an upcoming target (Harter et al., 1989; Nobre et al., 2000; Simpson et al., 2006) .

In terms of sustained attention during tracking, there are two known lateralized components that have been shown to be sensitive to the orienting of attention towards a single hemifield. The first component, the LDAP (late directing attention positivity; e.g., Hopf and Mangun, 2000), is highly similar to the ADAN but appears over more posterior and temporal electrode sites and has a positive voltage. Like the ADAN, it also appears to reflect the orienting of spatial attention towards a hemifield following a centrally presented spatial cue in anticipation of an upcoming target. Though it is a sustained wave, this component does not appear to reflect attentional processing of the targets per se because it is not sensitive to the task demands imposed by the targets, and typically has expired prior to target onset (Hopf and Mangun, 2000). By contrast, the second component, the CDA (contralateral delay activity) appears to be a good candidate for sustained attention to targets during tracking because it has been shown to be finely sensitive to the number of objects that are currently being maintained in visual working memory as well as being sensitive to the capacity limits of this system (Vogel and Machizawa, 2004; Vogel et al., 2005; Jolicoeur et al., 2006; McCollough et al., 2007; Mazza et al, 2007; Woodman & Vogel, 2008). This component is a sustained negative wave over posterior contralateral electrode sites, and likely stems from a source in the lateral intraparietal sulcus in the parietal cortex (Todd and Marois, 2004, 2005; Xu and Chun, 2006).

While the precise role of visual working memory during attentional tracking tasks is currently unclear (see e.g., Fougnie and Marois, 2006), there are at least two lines of evidence that suggest that similar mechanisms likely underlie the performance of each type of task. First, there is considerable evidence that maintaining object information in visual working memory requires sustained spatial attention to the locations of the remembered items (e.g., Awh et al., 2000). Second, the capacity of attentional tracking (~4 items) is highly similar to the capacity of visual working memory (3−4 items) and at least one study has found evidence that an individual's memory capacity positively predicts his or her tracking capacity (Oksama and Hyönä, 2004). On the basis of these previous findings, it appears highly plausible that similar capacity-limited mechanisms underlie performance of both tasks, and thus we expected that the CDA component would be observed while subjects sustained attention upon the moving targets during the tracking task.

Materials and Methods


Neurologically normal participants (12 In Experiment 1, 15 in Experiment 2, 18 in Experiment 3, 33 in Experiment 4, and 18 in Experiment 5; Age range 18−31) from the Eugene, Oregon community gave informed consent according to procedures approved by the University of Oregon institutional review board.

Stimulus Displays and Procedure

All stimulus arrays were presented in regions subtending 5.1 × 6.0 degrees (or 4.2 × 4.9 in the small area condition of Experiment 2) that were centered 3.2 degrees to the right or left of a central fixation cross (see Figure 1a). The inner boundary of each movement area was lateralized 0.5 degrees of visual angle to the left or right of fixation to minimize the impact of small movements of eye position. Each trial began with stationary squares that subtended .38 × .38 degrees of visual angle and were displayed in both left and right regions for 500ms. A subset of the squares was red in one hemifield, and green in the other; the remaining items were black. In each experiment, half of the subjects were asked to track red squares while the other half were instructed to track the green squares. In Experiments 1, 2, and 4 there were 8 total squares in each hemifield while Experiments 3 and 5 had 10 items to insure that at least 50% of the boxes were distractors in each trial. After 500 ms, targets (red and green items) changed to black and all items began to move for 1500ms. When motion stopped, one square was drawn in red in one hemifield and another became green in the opposite hemifield which initiated a 2000ms response window for the participant. The probed square was one of the original targets on 50% of trials and was a randomly selected distractor within the hemifield on the remaining trials. Each participant completed 240 trials per condition in the first experiment, 200 in the second experiment, 160 in the third experiment, and 224 in the final two experiments.

Fig. 1
(A) ERP multiple object tracking task. Participants tracked either red or green boxes while maintaining central fixation. In each experiment, the number of total objects (including distractors) was held constant while the number of target boxes varied ...

Motion Parameters

In Experiments 1 and 2, the direction of motion varied randomly and the boxes bounced off the border of the viewing area, but not off of each other (brief occlusion possible). The speed of motion varied from .25 to 1.86 degrees of visual angle/second with an average of about 1 degree per second. Motion trajectory was linear and changed at random intervals or when the object made contact with (invisible) outer barrier of the viewing area. Several of these parameters were modified slightly in Experiments 3 and 4. In particular, the size of the squares was increased to .7 degrees and the squares bounced off (no occlusion) of each other when they made contact. Furthermore, the average speed in these experiments was increased to 1.58 degrees/ second. These changes made no observable difference in the ERP data or behavioral performance between experiments.

In Experiment 5, in a separate behavior-only session participants were asked to track 3,4 or 5 objects that were distributed across both hemifields for 10 seconds following a 500ms cue that was identical to the cue in previous experiments. In the ERP session of this experiment, subjects performed a tracking task that was identical to that used in Experiment 4.

Measuring Tracking Capacity

We used Scholl's (2001) formula to derive the effective number of objects tracked: M = n(2P-1). Where M is effective number of objects tracked, n is number of targets and P is the empirically observed proportion of correct answers.

Electrophysiological recording and analysis

Event-related potentials (ERPs) were recorded in each experiment using our standard recording and analysis procedures, including rejection of trials contaminated by blocking, blinks or large (>1 degree) eye-movements (see (Vogel et al., 1998; McCollough et al., 2007). We recorded from 22 tin electrodes mounted in an elastic cap (Electrocap International) using the International 10/20 System. 10/20 sites F3, FZ, F4, T3, C3, CZ, C4, T4, P3, PZ, P4, T5, T6, O1 and O2 were used along with 5 nonstandard sites: OL midway between T5 and O1; OR midway between T6 and O2; PO3 midway between P3 and OL; PO4 midway between P4 and OR; POz midway between PO3 and PO4. All sites were recoded with a left-mastoid reference, and the data were re-referenced offline to the algebraic average of the left and right mastoids. Horizontal electrooculogram (EOG) was recorded from electrodes placed approximately 1 cm to the left and right of the external canthi of each eye to measure horizontal eye movements. To detect blinks, vertical EOG was recorded from an electrode mounted beneath the left eye and referenced to the left mastoid. Subjects with trial rejection rates >25% were excluded from the sample.

Contralateral waveforms were computed by averaging the activity recorded over the right hemisphere when subjects tracked items in the array at the left side of screen. Contralateral tracking activity was measured at posterior parietal, lateral occipital, posterior temporal, parietal and occipital electrode sites as the difference in mean amplitude between the ipsilateral and contralateral waveforms. We used two measurement windows: 200−300ms after the onset of the items for the N2pc analyses, and 800−1200ms (300−700ms after motion onset) for the tracking analyses. Differences in scalp topography were tested by normalizing the data for each component following the procedure described by McCarthy & Wood (1985) and testing for the interaction between electrode position and time window (i.e., 200−300ms vs 800−1200ms). The EEG and EOG were amplified with a SA Instrumentation amplifier with a bandpass of 0.01−80Hz and were digitized at 250 Hz in LabView 6.1 running on a Macintosh.

Eye Movements

Any trials containing either a blink or eye-movement were excluded from further analysis. The horizontal electrooculogram (EOG) for Experiment 1 is plotted as a function of the cued hemifield in Figure 2b. There was a small but significant tendency for eye position to drift towards the attended side during the latter half of the trial (p<.05). Though, the magnitude of this deviation was not influenced by the number of targets being tracked, nor was it related to tracking performance. Further, this deviation from fixation was quite small: the mean amplitude of this EOG activity was 2.5 μV, which corresponds to an eye-movement of less than 0.16 degrees of visual angle from the fixation point (Hillyard and Galambos, 1970). Given that the area that the boxes moved within was lateralized by a minimum of more than 0.5 degrees from fixation, it is unlikely that these small drifts in fixation affected our data.

Fig. 2
(A) Contralateral and ipsilateral activity in response to the three tracking loads in Experiment 1 across all frontal, parietal, and occipital electrodes. Waveforms were time-locked to the initial appearance of targets and motion began at 500ms. (B) Grand-averaged ...


Experiment 1: ERP correlates of Selecting and Tracking moving Objects

On each trial, subjects were presented a bilateral array containing six squares in each hemifield (see Figure 1A). For the first 500ms of each trial (cue period), the objects were stationary with a subset of the items in a given hemifield drawn in red (the targets) and the remaining items drawn in black (the distractors). Green items appeared at the start of each trial on the unattended side. These items were photometrically isoluminant and equal in number to the red target items on each trial. Half of the subjects tracked red items while the others tracked green. After 500ms, the red and green items changed to black and all of the objects began to move amongst each other in random directions within the hemifield for 2 seconds; at that point, the items stopped moving and one item turned red. Subjects were instructed to attentionally track the targets and pressed one of two buttons to indicate whether the final red item was one of the targets or not. We time-locked the ERPs to the onset of the cue array and recorded throughout the duration of the trial so that we could observe both the transient selection of the targets during the cue period as well as the sustained attention response during the tracking period. In Experiment 1, we asked subjects to track one, two, or three targets on each trial so that we could determine whether the activity was modulated by the number of tracked items.

200ms following the onset of the cue array, we observed a transient negative-going wave over the hemisphere that was contralateral to the attended hemifield. This activity was followed by a larger and sustained contralateral negative wave that began shortly after the tracking period started and persisted throughout the course of the trial until the test was presented. As shown in Figure 1B, the amplitude of both of these waves was strongly modulated by the number of target items; increasing the number of targets resulted in substantial increases in amplitude (3 targets > 2 targets > 1 target; all p's < .01). Moreover, the amplitude of this activity was highly sensitive to whether or not the subject performed the tracking task correctly: both waves showing large, significant decreases in amplitude on error trials relative to correct trials (both p's < .01). This indicates that both waves reflect processes that are necessary antecedents to correct tracking performance. Figure 2a shows the distribution of these waves across each of the lateral recording sites. The transient activity during the selection phase was primarily centered over posterior electrodes with a maximum over lateral occipital electrodes (OL/OR). During this selection period there was no significant lateralized activity observed over frontal electrodes (F < 1). The sustained activity during the tracking period was more broadly distributed over the posterior electrode sites with a maximum over posterior parietal electrodes (PO3/PO4). This activity was also observed over frontal electrode sites (F3/F4), though the contralateral effect at these sites was not significantly modulated by the number of tracked targets (F=2.3, p >.10).

The transient wave during the cue period appears to be the N2pc wave which, as described in the Introduction, has previously been shown to reflect the selection of targets amongst distractors in visual search tasks (Luck et al., 1997; Hopf et al., 2000; Woodman and Luck, 2003). By contrast, the large sustained wave during tracking appears to be the contralateral delay activity (CDA) that we and others have shown reflects the number of active object representations held in visual short term memory (VSTM) (e.g., Vogel and Machizawa, 2004). Together, the N2pc and CDA waves appear to index two critical components of attentional tracking: the initial selection of the target objects during the cue period (N2pc); and sustained attention towards the target representations as they move about the hemifield (CDA). Although the N2pc and the CDA were both modulated by the number of targets, we found that these two waves have distinct scalp distributions yielding a highly significant electrode position by time window (200−300ms vs 800−1200ms) interaction (p<.01, See Methods): with the N2pc showing a more ventral distribution than the more dorsal CDA. This finding supports a previous demonstration of distinct scalp distributions for these two components in the context of a working memory task (McCollough et al., 2007). Together, these results suggest that while there appears to be a tight coupling between object selection and sustained attention towards the targets, they may reflect the output of distinct cortical areas.

Experiment 2: Spatial extent of attention or number of objects?

Although the amplitude of both the N2pc and CDA in the first experiment increased as a function of the number of targets, it is possible that this increase is simply due to the required spatial extent of the target area rather than reflecting the increasing number of targets selected and tracked during the trial. That is, as the number of target items increases, there is also potential for a corresponding increase in the area of the attentional window or “spotlight” that encompasses the targets and this may be what caused the increases in amplitude in the first experiment (e.g., Eriksen and St. James, 1986; Hillyard et al., 1998). To test this alternative, in the second experiment we directly manipulated the amount of area required to track the targets. Subjects tracked two or three targets that either encompassed a large area or a small area within the hemifield. We found that while the amplitudes of both the N2pc and CDA were again significantly modulated by the number of targets (both p's < .01), there was no significant effect of area on amplitude for either wave (both F's < 1; see Figure 3). We did however find a significant effect of area on behavioral tracking performance, where performance in the small area conditions was significantly poorer (~10%) than in the large area conditions (p < .01). These results are consistent with previous studies that have shown that displays with a high density of items result in more difficult tracking and poorer performance (e.g., Intriligator and Cavanagh, 2001). It also helps to confirm that our manipulation of area was substantial enough to observe a significant behavioral effect. Indeed, the lack of an amplitude modulation by area also argues against the hypothesis that the amount of general effort or difficulty required to track more targets is the cause of the observed increase in amplitude. That is, despite the small area condition being significantly more difficult than the large area condition, there was no concomitant rise in amplitude for either the N2pc or the CDA. Nonetheless, it is important to note that the apparent dissociation between behavioral performance and CDA amplitude in this experiment may be due to a limitation of our measure. In particular, it is possible that poorer behavioral performance in the small area condition is due to the subjects inadvertently tracking distractor items that were mistaken, or swapped, for target items during the course of the trial due to the closer proximity of targets and distractors. This scenario would lead to a decrease in behavioral performance because the wrong items were being tracked. However, it would predict no change in CDA amplitude because the same total number of items are being tracked on the trial. Specifically, the limitation of this component is that it provides an index of the number of objects currently being tracked irrespective of whether or not they are targets.

Fig. 3
(A) Behavioral performance in Experiment 2 showing significant main effects of both area and number of items. (B) Mean amplitude of CDA activity in Experiment 2. While there was a significant main effect of number of targets, area had no significant effect ...

Experiment 3: Sensitivity to behavioral tracking limitations

The results of the first two experiments are consistent with the proposal that the amplitude of both the N2pc and the CDA reflects the number of targets being selected or tracked, respectively. However, to strengthen this claim it is necessary to demonstrate that this activity is indeed sensitive to the known behavioral performance limitations associated with attentional tracking. Therefore, in the third experiment we measured these two waves under a task condition that is likely to exceed the subject's tracking capacity so that we could determine whether this activity is sensitive to these performance limitations. Indeed, this has been a significant limitation of previous neuroimaging studies examining tracking-related load effects because they have not tested whether the observed activity continues to increase when the number of targets exceeds capacity. In addition, by examining a wider range of target array sizes, we can begin to examine whether these two types of activity are sensitive to differences across individuals in tracking ability. In this experiment, subjects tracked one, three, or five targets on each trial. In this experiment, all trials contained 10 items so that 50% of the items were distractors when subjects tracked 5 items. We divided subjects into high capacity and low capacity groups on the basis of a median split of their behavioral tracking capacity (see Methods). Figure 4 shows the N2pc and CDA waves for each target array size for the high and low capacity groups. As can be seen in the figure, both groups showed an increase in amplitude for both the N2pc and the CDA from one to three targets (low capacity: both p's < .05; high capacity: both p's < .001). However, the two groups diverged greatly when tracking five items. The amplitude for the high capacity group when tracking five items remained equivalent to that of tracking three items (N2pc: F<1; CDA: p > .15). Thus, when given more items than they could track, the high capacity subjects appeared to be able to continue to track their limit of objects (i.e., ~3 items). However, for the low capacity group, the track five amplitude decreased significantly below the three item level and was equivalent to that of tracking a single item (N2pc: p < .001; CDA: p < .05). While the precise cause of this amplitude decrease is currently unclear, it does appear to reflect a consistent pattern across all subjects dependent upon their specific tracking capacity. That is, there was a significant negative correlation between an individual's tracking capacity and the amount of decrease between three targets and five targets (r = −.60 N2pc; r = −.56 CDA; both p's < .01), such that as tracking capacity increased the amount of amplitude drop decreased. In summary, the results of Experiment 3 provide further evidence that the amplitude increases of the N2pc and CDA are the consequence of the number of items that are currently being selected or tracked. In particular, these results demonstrate that the amplitude is not simply driven by the amount of cognitive load required to perform the task because the amplitude of each component reached an asymptotic limit at roughly 3 items, even though the amount of cognitive load continued to increase when the subjects attempted to track 5 items. Thus, the properties of these neural mechanisms appear to be finely sensitive to the known capacity limitations associated with attentional tracking.

Fig. 4
ERP difference waves for correct trials in Experiment 3 divided between high capacity (A) and low capacity individuals (B) on the basis of a median split of tracking performance. Mean amplitude (in microvolts) of the N2pc (C) and the CDA (D) for the high ...

Experiment 4: Predicting individual differences in tracking capacity

The results of Experiment 3 indicate that the amplitude of both the N2pc and the CDA are highly sensitive to the tracking capacity limitations that constrain performance in this task because it reaches a limit at tracking three targets and is also finely attuned to individual differences in tracking capacity. However, this sensitivity to individual differences was not restricted to the response to supracapacity target arrays, but was also observed in the size of the increase in amplitude from one target to three targets. This resulted in a highly significant interaction between group (high vs low) and number of targets (1 vs 3) (N2pc: p < .001; CDA: p < .01), with a larger increase from one to three targets for the high capacity group than for the low capacity group. The smaller difference in amplitude between one and three targets for the low capacity group suggests that the one-target arrays consumed a larger proportion of available capacity than for the high capacity group, resulting in a smaller increase to three items. Paired t-tests support this assertion because the difference between the high and low groups was not significant in the track 1 condition (p's > .15) but the difference between these two groups was highly significant in the track 3 condition (N2pc: p < .005; CDA: p < .01).

We tested the robustness of this relationship by running an additional group of subjects in the one and three target conditions and combining this data with all of the subjects from the previous experiments so that we could have a large sample (N=63). Figure 5 shows the amplitude of both waves for tracking one or three targets divided between high capacity and low capacity subjects. From the figure, there are two apparent differences between the high and low capacity groups: first, the high capacity group tends to have overall larger amplitudes for each wave; and second, the high capacity group shows a larger rise in amplitude from 1 to 3 items than the low capacity group. This pattern of effects was confirmed in an Analysis of Variance (ANOVA), yielding significant main effects of group (both p's < .05) and number of targets (both p's < .001), as well as a significant interaction between group and number of targets (p < .01). Although high capacity subjects tend to have higher overall amplitudes (irrespective of number of targets), this factor is only a fairly weak to moderate predictor of an individual's tracking capacity (N2pc: r=.22, p <.10; CDA: r=.31, p <.05). By contrast, we found that the rise in amplitude from one target to three targets was a much stronger predictor of an individual's tracking capacity (N2pc: r=.70, p < .001; CDA: r =.48; p < .001). Importantly, these strong correlations persisted even when we partialled out the relationship between overall amplitude and tracking capacity (partial r's = .68 and .41 for N2pc and CDA, respectively). Thus, it appears that it is the amount of differentiation in amplitude between increasing numbers of targets that may be most predictive of an individual's tracking capacity. We also found that the rise in N2pc amplitude from one to three targets was strongly correlated with the rise of the CDA (r = .72, p < .001) which further indicates that there is a tight coupling between these measures of object selection and sustained attention. However, because of this strong relationship, we also calculated partial correlations for both the N2pc and CDA effects (i.e., rise from 1 to 3 targets) so that we could measure each wave's unique contribution to predicting tracking capacity. Although the N2pc effect remained a strong predictor of tracking capacity when the contribution of the CDA effect was removed (partial r = .59, p < .001), the CDA effect was only a weak predictor of tracking capacity when the N2pc effect was removed (partial r = .09; ns). Importantly, these effects were not simply due to more variability in the CDA than the N2pc. Measurements of the reliability of each component revealed that both components were highly stable within subjects, and that the CDA actually had a higher reliability than the N2pc (Cronbach's alpha = 0.74 for the N2pc; 0.94 for the CDA). Consequently, these results demonstrate that while neural indices of both target selection (N2pc) and sustained attention (CDA) can serve as strong neurophysiological predictors of attentional tracking capacity, it is the selection process that explains most of the unique variance in tracking capacity across individuals.

Fig. 5
(A, B) ERP difference waves for high and low capacity subjects in Experiment 4. (C, D) Mean amplitudes of the N2pc and CDA waves across high and low capacity groups. There was a significant interaction between group (high/low) and number of objects for ...

Experiment 5: Limiting factor for tracking capacity: selection or tracking?

Our observation that how efficiently an individual initially selects the target items strongly predicts their overall tracking capacity is somewhat surprising because selection occurs well before tracking (i.e., motion onset) even begins. In this regard, one could argue that there must always be a strong relationship between selection and tracking performance because subjects can track only the targets that were appropriately selected in the first place. However, there are likely to be many processes that contribute to an individual's overall tracking capacity depending upon the specific nature of the tracking task that is being used to estimate capacity (vanMarle and Scholl, 2003; Oksama and Hyönä, 2004; Alvarez et al., 2005; Liu et al., 2005; Pylyshyn and Annan, 2006). Indeed, our behavioral estimate of tracking capacity may actually load heavily on the selection stage because the subjects were required to hold fixation while selecting a subset of targets amongst distractors within a single hemifield. Moreover, it is possible that there is a somewhat weaker contribution of sustained attention activity in our behavioral measure because our tracking period is relatively short (i.e., 1.5 seconds) compared to previous studies that tend to use longer periods of tracking (e.g., 8−10 seconds).

In the final experiment we tested whether these two neural predictors of tracking capacity would be sensitive to a change in the relative contributions of selection and sustained attention by assessing each component's (i.e., N2pc and CDA) ability to predict an individual's tracking capacity in a “whole field” tracking task with a longer duration. More specifically, subjects were tested in two separate sessions. In a behavior-only session, subjects were asked to track 3, 4, or 5 target items amongst distractors that were spread across the entire visual field (“whole-field”) and they tracked these items for 8 seconds. In a separate ERP session, subjects performed a single hemifield tracking task that was identical to that used in Experiment 4. We estimated each subject's “whole field” tracking capacity on the basis of performance in the behavior-only session, and used this estimate as a predictor of his or her N2pc and CDA effects that were measured in the single hemifield ERP tracking task. In a “whole field” tracking situation, the difficulty of target selection should be reduced because the subjects could freely view and select the targets across the entire display. In contrast, the difficulty of sustained attention should be raised because of the substantial increase in how long the targets needed to be tracked continuously. Consequently, we would expect that the N2pc effect should now become a weaker predictor of “whole field” tracking capacity; simultaneously, we expect that the CDA should become a stronger predictor of tracking capacity as the limiting factor in task performance shifts from selection to sustained attention. As shown in Figure 6, we observed that while the correlation between the N2pc difference effect and whole field tracking capacity was considerably weaker than we observed previously (r= .31, p < .07), the CDA difference became a much stronger predictor of tracking performance (r= .72; p <.001). Again, the N2pc and CDA effects were strongly correlated (r = .52, p < .05). Moreover, when we partialled out the contribution of the N2pc effect, the relationship between the CDA effect and tracking capacity remained strong (partial r = .69; p < .01); Conversely, the N2pc was no longer predictive of tracking capacity when the CDA effect contribution was removed (partial r = .10; ns). Thus, in this “whole field” tracking context, it is our index of sustained attention that explains most of the unique variance in attentional tracking capacity across individuals.

Correlations between an individual's Whole Field tracking capacity and the rise in amplitude from 1 to 3 targets for the N2pc (A) and the CDA (B). Tracking capacity was estimated by averaging behavioral performance across all set sizes (3, 4 and 5).


Overall, these results indicate that we have isolated neural measures of the target selection and sustained attention processes that underlie our limited ability to track multiple moving objects. Indeed, by measuring the amplitudes of the N2pc and CDA waves we could determine how many targets were being selected or tracked during a trial as well as being highly sensitive to a given subject's specific tracking capacity. Moreover, these two neural measures allow us to finely index what the primary limiting factors for performance are on a given measure of tracking capacity. Under difficult selection conditions, variability in the N2pc effect strongly predicts tracking performance. Whereas, when selection is less taxing but the targets must be tracked for longer durations, it is the variability in the CDA that strongly predicts tracking performance. However, under both situations, we found that it was the amount of separation in amplitude between different numbers of targets (i.e., rise from 1 to 3 targets) that was the primary predictor of tracking ability. Consequently, these results suggest that individual differences in tracking performance may be primarily determined by how efficiently the visual system can individuate the targets from one another as well as from the distractors (Sears and Pylyshyn, 2000; Intriligator and Cavanagh, 2001; Ogawa et al., 2002; Vogel et al., 2005; Suganuma and Yokosawa, 2006).

The results of this series of experiments also have significant implications regarding the neural systems that underlie the attentional mechanisms involved in selecting and tracking moving objects. For example, we have found that the amplitude of the N2pc provides a reliable index of the number of targets being selected, but strongly follows the limits of attentional tracking capacity. Considering that the N2pc is thought to be generated in V4 and posterior portions of inferior temporal cortex (e.g., Hopf et al., 2006), these results suggest that selective attention effects in these regions may show similar sensitivity to capacity limits. This is consistent with the viewpoint that attention effects in these regions may reflect processes that help to individuate targets from distractors (e.g., Desimone and Duncan, 1995; Kastner and Ungerleider, 2000). Moreover, recent work has suggested that the attentional “spotlight” can be split to two noncontiguous locations simultaneously without also being allocated to the intervening space (Awh and Pashler, 2000; Müller and Hübner, 2002; Müller et al., 2003), and that areas of extra-striate cortex show distinct focal activation patterns under split-attention conditions (McMains and Somers, 2004, 2005). Thus, it is plausible that similar attentional mechanisms underlie our current N2pc target selection effects and these demonstrations of split attentional foci. If this is the case, we would predict that the attentional capacity of the observer would impose an upper limit on the number of locations that could be simultaneously selected.

The response of the CDA during tracking also allows provides implications regarding the underlying neural mechanisms involved in sustaining attention towards targets. The primary candidate neural source for the CDA is the IPS, which stems from the fact that previous work has shown that this area was modulated by the number of items that are being tracked (e.g., Jovicich et al., 2001), as well as the finding that this region shows highly similar patterns of BOLD activation during working memory load manipulations. Like the CDA, the IPS also reaches asymptotic activity levels for memory loads of approximately 3 items, and is sensitive to individual differences in working memory capacity (Todd and Marois, 2005). Thus, the finding that the CDA shows parallel responses during attentional tracking and visual working memory tasks suggests that cells in the IPS may actually facilitate the processing of both tasks. In this regard, the IPS may reflect a smart, but limited-capacity pointer system that helps keep individuated representations of objects actively maintained in working memory tasks and spatially updated in attentional tracking tasks. While the current results are highly consistent with the interpretation that similar neural mechanisms may underlie the capacity limits of both types of tasks, they are still insufficient to resolve this particular question because we have not directly compared the neural activity during visual working memory and attentional tracking tasks in the same subjects. However, the present results appear to provide an experimental approach for addressing this question in the future.


Our limited ability to divide attention so that we may keep track of multiple moving objects is a central limitation within cognition, and is thought to underlie our performance of a wide assortment of common tasks. Moreover, an individual's tracking capacity has been shown to be positively related to performance on a broad range of high-level cognitive functions, including measures of fluid and intelligence (Oksama and Hyönä, 2004). The present results demonstrate strong and robust neurophysiological predictors of individual differences in attentional tracking capacity. Thus, they provide an initial link between this fundamental cognitive limitation and the two primary stages of neural activity that facilitate attentional tracking.


The research reported here was supported by grants from the National Science Foundation, USA and the National Institute of Mental Health, USA to EKV.


  • Alvarez GA, Cavanagh P. Independent resources for attentional tracking in the left and right visual hemifields. Psychological Science. 2005;16:637–643. [PubMed]
  • Alvarez GA, Horowitz TS, Arsenio HC, DiMase JS, Wolfe JM. Do multielement visual tracking and visual search draw continuously on the same visual attention resources? Journal of Experimental Psychology-Human Perception and Performance. 2005;31:643–667. [PubMed]
  • Awh E, Pashler H. Evidence for split attentional foci. Journal of Experimental Psychology: Human Perception and Performance. 2000;26:834–846. [PubMed]
  • Awh E, Anllo-Vento L, Hillyard S. The role of spatial selective attention in working memory for locations: Evidence from event-related potentials. Journal of Cognitive Neuroscience. 2000;12:840–847. [PubMed]
  • Buschman TJ, Miller EK. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science. 2007;315:1860–1862. [PubMed]
  • Cavanagh P, Alvarez GA. Tracking multiple targets with multifocal attention. Trends in Cognitive Sciences. 2005;9(7):349–354. Jul 2005. [PubMed]
  • Culham JC, Cavanagh P, Kanwisher NG. Attention response functions: Characterizing brain areas using fMRI activation during parametric variations of attentional load. Neuron. 2001;32:737–745. [PubMed]
  • Culham JC, Brandt SA, Cavanagh P, Kanwisher NG, Dale AM, Tootell RBH. Cortical fMRI activation produced by attentive tracking of moving targets. Journal of Neurophysiology. 1998;80:2657–2670. [PubMed]
  • Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience. 1995;18:193–222. [PubMed]
  • Eimer M. The N2pc component as an indicator of attentional selectivity. Electroencephalography and clinical Neurophysiology. 1996;99:225–234. [PubMed]
  • Eriksen CW, St. James JD. Visual attention within and around the field of focal attention: A zoom lens model. Perception & Psychophysics. 1986;40:225–240. [PubMed]
  • Fougnie D, Marois R. Distinct Capacity Limits for Attention and Working Memory: Evidence From Attentive Tracking and Visual Working Memory Paradigms. Psychological Science. 2006;17(6):526–534. Jun 2006. [PubMed]
  • Harter MR, Miller SL, Price NJ, LaLonde ME, Keyes AL. Neural processes involved in directing attention. Journal of Cognitive Neuroscience. 1989;1:223–237. [PubMed]
  • Hillyard SA, Galambos R. Eye movement artifact in the CNV. Electroencephalography and Clinical Neurophysiology. 1970;28:173–182. [PubMed]
  • Hillyard SA, Vogel EK, Luck SJ. Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society: Biological Sciences. 1998;353:1257–1270. [PMC free article] [PubMed]
  • Hopf J-M, Mangun GR. Shifting visual attention in space: An electrophysiological analysis using high spatial resolution mapping. Clinical Neurophysiology. 2000;111:1241–1257. [PubMed]
  • Hopf J-M, Luck SJ, Girelli M, Hagner T, Mangun GR, Scheich H, Heinze HJ. Neural sources of focused attention in visual search. Cerebral Cortex. 2000;10:1233–1241. [PubMed]
  • Hopf J-M, Boelmans K, Schoenfeld AM, Heinze HJ, Luck SJ. How does attention attenuate target-distractor interference in vision? Evidence from magnetoencephalographic recordings. Cognitive Brain Research. 2002;15:17–29. [PubMed]
  • Hopf JM, Luck SJ, Boelmans K, Schoenfeld AM, Boehler CN, Rieger J, Heinze H. The neural site of attention mtaches the spatial scale of perception. Journal of Neuroscience. 2006;26:3532–3540. [PubMed]
  • Intriligator J, Cavanagh P. The spatial resolution of visual attention. Cognitive Psychology. 2001;43:171–216. [PubMed]
  • Jolicoeur P, Sessa P, Dell'Acqua R, Robitaille N. On the control of visual spatial attention: evidence from human electrophysiology. Psychological Research-Psychologische Forschung. 2006;70:414–424. [PubMed]
  • Jovicich J, Peters RJ, Koch C, Braun J, Chang L, Ernst T. Brain areas specific for attentional load in a motion-tracking task. Journal of Cognitive Neuroscience. 2001;13:1048–1058. [PubMed]
  • Kastner S, Ungerleider LG. Mechanisms of visual attention in the human cortex. Annual Review of Neuroscience. 2000;23:315–341. [PubMed]
  • Liu G, Austen EL, Booth KS, Fisher BD, Argue R, Rempel MI, Enns JT. Multiple-Object Tracking Is Based on Scene, Not Retinal, Coordinates. Journal of Experimental Psychology: Human Perception and Performance. 2005;31(2):235–247. Apr 2005. [PubMed]
  • Luck SJ, Hillyard SA. Spatial filtering during visual search: Evidence from human electrophysiology. Journal of Experimental Psychology: Human Perception and Performance. 1994;20:1000–1014. [PubMed]
  • Luck SJ, Girelli M, McDermott MT, Ford MA. Bridging the gap between monkey neurophysiology and human perception: An ambiguity resolution theory of visual selective attention. Cognitive Psychology. 1997;33:64–87. [PubMed]
  • Mazza V, Turatto M, Umilta C, Eimer M. Attentional selection and identification of visual objects are reflected by distinct electrophysiological responses. Experimental Brain Research. 2007;181:531–536. [PMC free article] [PubMed]
  • McCarthy G, Wood CC. Scalp distributions of event-related potentials: An ambiguity associated with analysis of variance models. Electroencephalography and Clinical Neurophysiology. 1985;62:203–208. [PubMed]
  • McCollough AW, Machizawa MG, Vogel EK. Electrophysiological measures of maintaining representations in visual working memory. Cortex. 2007;43:77–94. [PubMed]
  • McMains SA, Somers DC. Multiple spotlights of attentional selectdion in human visual cortex. Neuron. 2004;42:677–686. [PubMed]
  • McMains SA, Somers DC. Processing efficiency of divided spatial attentoin mechanisms in human visual cortex. Journal of Neuroscience. 2005;25:9444–9448. [PubMed]
  • Müller MM, Hübner R. Can the spotlight of attention be shaped like a donut? Evidence from steady-state visual evoked potentials. Psychological Science. 2002;13:119–124. [PubMed]
  • Müller MM, Malinowski P, Gruber T, Hillyard SA. Sustained division of the attentional spotlight. Nature. 2003;424:309–312. [PubMed]
  • Nobre AC, Sebestyen GN, Miniussi C. The dynmamics of shifting visuospatial attention revearled by event-related potentials. Neuropsychologia. 2000;38:964–974. [PubMed]
  • Ogawa H, Takeda Y, Yagi A. Inhibitory tagging on randomly moving objects. Psychological Science. 2002;13:125–129. [PubMed]
  • Oksama L, Hyönä J. Is multiple object tracking carried out automatically by an early vision mechanism independent of higher-order cognition? An individual difference approach. Visual Cognition. 2004;11:631–671.
  • Pylyshyn ZW, Storm RW. Tracking multiple independent targets: Evidence for a parallel tracking mechanism. Spatial Vision. 1988;3(3):179–197. 1988. [PubMed]
  • Pylyshyn ZW, Annan V. Dynamics of target selection in Multiple Object Tracking (MOT). Spatial Vision. 2006;19:485–504. [PubMed]
  • Scholl BJ, Pylyshyn ZW, Feldman J. What is a visual object? Evidence from target merging in multiple object tracking. Cognition. 2001;80(1−2):159–177. Jun 2001. [PubMed]
  • Sears CR, Pylyshyn ZW. Multiple object tracking and attentional processing. Canadian Journal of Experimental Psychology. 2000;54(1):1–14. Mar 2000. [PubMed]
  • Simpson GV, Dale CL, Luks TL, Miller WL, Ritter W, Foxe JF. Rapid targeting followed by sustained deployment of visual spatial attention. Neuroreport. 2006;17:1595–1599. [PubMed]
  • Suganuma M, Yokosawa K. Grouping and trajectory storage in multiple object tracking: Impairments due to common item motions. Perception. 2006;35(4):483–495. 2006. [PubMed]
  • Todd JJ, Marois R. Capacity limit of visual short-term memory in human posterior parietal cortex. Nature. 2004;428:751–754. [PubMed]
  • Todd JJ, Marois R. Posterior parietal cortex activity predicts individual differences in visual short-term memory capacity. Cognitive Affective & Behavioral Neuroscience. 2005;5:144–155. [PubMed]
  • Van Velzen J, Eimer M. Early posterior ERP components do not reflect the control of attentional shifts toward expected peripheral events. Psychophysiology. 2003;40:827–831. [PubMed]
  • vanMarle K, Scholl BJ. Attentive tracking of objects versus substances. Psychological Science. 2003;14(5):498–504. Sep 2003. [PubMed]
  • Vogel EK, Machizawa M. Neural activity predicts individual differences in visual working memory capacity. Nature. 2004;428:748–751. [PubMed]
  • Vogel EK, Luck SJ, Shapiro KL. Electrophysiological evidence for a postperceptual locus of suppression during the attentional blink. Journal of Experimental Psychology: Human Perception and Performance. 1998;24:1656–1674. [PubMed]
  • Vogel EK, McCollough AW, Machizawa MG. Neural measures reveal individual differences in controlling access to working memory. Nature. 2005;438:500–503. [PubMed]
  • Woodman GF, Luck SJ. Electrophysiological measurement of rapid shifts of attention during visual search. Nature. 1999;400:867–869. [PubMed]
  • Woodman GF, Luck SJ. Serial deployment of attention during visual search. Journal of Experimental Psychology-Human Perception and Performance. 2003;29:121–138. [PubMed]
  • Woodman G, Vogel EK. Selective storage and maintenance of an object's features in visual working memory. Psychonomic Bulletin & Review. 2008;15:223–229. [PubMed]
  • Xu YD, Chun MM. Dissociable neural mechanisms supporting visual short-term memory for objects. Nature. 2006;440:91–95. [PubMed]
  • Yantis S. Multi-Element visual tracking - attention and perceptual organization. Cognitive Psychology. 1992;24:295–340. [PubMed]