Maintenance of object representations across temporary gaps in space and time might comprise a combination of lower and higher perceptual and cognitive mechanisms. We used an object tracking task and functional neuroimaging to begin to clarify these mechanisms, in part by distinguishing them from mechanisms involved in tracking fully visible objects and from mechanisms involved in estimating a simple temporal gap. Results from our behavioral task demonstrated that observers’ were highly engaged in the tracking task, and judgments of target arrival time were reliably different across the three conditions suggesting different strategies may have been employed. When the target was fully visible, judgments of target arrival time tended to be late, implying a strategy in which observers tracked the visible motion and initiated a button press as the target reached the appropriate location. When the target was occluded, judgments of target arrival time tended to be early, suggesting that observers anticipated its arrival, and did not wait to press when it reappeared, which would have led to a longer latency. This suggests that observers maintained a representation of the spatiotemporal information during occlusion and interpolated the invisible motion of the target. When the target shrank and expanded, judgments of target arrival time were close to the actual time of reappearance. Our intuitions of a time-keeping strategy were supported anecdotally: Several observers reported that they employed a “counting” strategy in which they simply tried to time the reemergence from learning the timing and counting to themselves after the target imploded. These reaction time differences suggest that the three tasks evoked a different cognitive state, mental operation, or strategy to perform the different tasks as accurately as possible.
The Unoccluded > Occluded
and Unoccluded > Shrinking
contrasts yielded increased activation in extrastriate (occipital, inferior temporal and posterior parietal) visual cortical areas. This is not surprising given that in unoccluded trials observers tracked a continuously visible target object moving in a constant trajectory. Increased activation in these regions of extrastriate cortex has been previously reported during conditions of attentive tracking relative to passive viewing of multiple moving objects (Culham, Brandt, Cavanagh, Kanwisher, Dale & Tootell, 1998
). The Occluded > Unoccluded
contrast yielded increased activation in precentral sulcus, inferior parietal lobule, temporal cortex, and prefrontal cortical regions along the dorsal medial wall. Notably, almost identical foci in the precentral sulcus and inferior parietal lobule show an attentive tracking load effect (i.e., activation increases with the number of moving items that are simultaneously tracked) (Culham 2001, neuron). Moreover, these same areas also activate during visual working memory (Courtney et al., 1997) and sustained visual attention (Serences & Yantis, in press cereb cortex). The Occluded > Shrinking
contrast yielded increased bilateral activation in lateral occipital cortex (LOC) that may have been evoked by the continued representation of the object. Consistent with this interpretation, several other neuroimaging studies also report increased neural activity in LOC during tasks of form perception and object recognition (Lerner, Hendler & Malach, 2002
; Malach, Reppas, Benson, Kwong, Jiang, Kennedy, Ledden, Brady, Rosen & Tootell, 1995
), illusory contours (Ffytche & Zeki, 1996
; Hirsch, DeLaPaz, Relkin, Victor, Kim, Li, Borden, Rubin & Shapley, 1995
), and perceptual completion of static surfaces (Mendola, Dale, Fischl, Liu & Tootell, 1999
; Stanley & Rubin, 2003
). In all comparisons against Unoccluded
, we found increased activation in the pre-SMA. Past studies have found increased activation in this area when and internal representation of time must be maintained (e.g., estimating a length of time or the timing of a motor response)
Maintaining active representations of objects through occlusion is likely accomplished by a combination of mechanisms such as perceptual completion (Nakayama, He & Shimojo, 1995
), selective attention (Awh, Jonides & Reuter-Lorenz, 1998
; Scholl, 2001
), and visual working memory (Pasternak & Greenlee, 2005
). Moreover, mechanisms supporting inferred motion and trajectory extrapolation (Assad & Maunsell, 1995
; Barborica & Ferrera, 2003
) and preparatory oculomotor behaviors (Curtis, 2006
; Curtis & D’Esposito, 2006a
; Curtis, Rao & D’Esposito, 2004
) constantly update the visual system with the moving target’s changing location in space. These mechanisms may work in concert to maintain a representation of the moving object in space and time despite perceptual interference such as occlusion.
Selective attention may serve as a crucial higher order mechanism that facilitates representations of dynamic object occlusion in visual working memory. Increased responses in posterior parietal cortex have been associated with maintaining the locus of visuospatial attention in working memory (Todd & Marois, 2004
), and the overall magnitude of posterior parietal activation may be an indicator of observers’ visual working memory capacity (Todd & Marois, 2005
). In addition, several frontal, posterior parietal, and temporal cortical areas show evidence of persistent activity during delay periods when observers maintain a representation of an object or its position in working memory (Curtis & D’Esposito, 2006b
). Therefore, the activations associated with attentive tracking through occlusion that we report here may reflect some of the same mechanisms that support maintenance of an object’s spatiotemporal information in visual working memory.
A recent study compared human parietal cortex activation during an occlusion task, in which the object simply blinked out of existence (Olson, Gatenby, Leung, Skudlarski & Gore, 2003
). The authors found that a portion of the posterior parietal cortex, bilaterally, showed a greater response during occlusion. We report here that what appears to be the same portion of the parietal cortex showed greater activation during occlusion than shrinking. This activation may reflect the activity of neurons in posterior parietal cortex that are motion sensitive and increase their rates of firing during a period when a moving object is briefly occluded (Assad & Maunsell, 1995
). Therefore, the posterior parietal cortex may be involved in processing spatiotemporal properties, including the inferred motion, of objects. Moreover, we extend the Olson et al. results by demonstrating that the posterior parietal cortex is only one part of a larger neural network that additionally includes LOC, superior temporal, superior frontal, and premotor cortices, that together are involved in tracking objects through occlusion. Further studies are necessary to tease apart the relative contributions of these areas in object tracking. [This is a good place to put a sentence or two about what might be special about maintaining “spatiotemporal” representations above spatial + temporal representations.
These data provide evidence of separate mechanisms involved in maintaining an object representation during covert tracking under conditions of full visibility versus two kinds of temporary concealment. The evidence points to different networks of cortical regions supporting the cognitive mechanisms (i.e., form and motion perception, spatial attention, and visual working memory) that we propose to be involved in object tracking. More importantly, we used functional neuroimaging techniques to isolate the cognitive operations supporting the active maintenance of an object’s spatiotemporal representation and ultimately the mental state of tracking an object continuously through occlusion.