The aim of the present study was to test long-standing views about the relationship between the short-term retention of information and sustained delay-period activity. Using an information-based analysis approach with fMRI data collected during a delayed-recognition task for visual motion, we tested two hypotheses: first, that sustained, elevated delay-period activity carries stimulus-specific information; and second, that stimulus information can be encoded in distributed patterns of subthreshold activity. To test the first hypothesis we trained pattern classifiers with BOLD signal from frontal and parietal areas that showed sustained, elevated delay-period activity. We failed to find evidence that these voxels carried stimulus-specific information during the delay period. To test the second hypothesis, we applied the same procedure to BOLD data from posterior regions that showed robust responses to visual stimuli, but no elevated delay-period activity. The classifiers were successfully able to decode the remembered direction throughout the delay period, providing strong evidence in support of this hypothesis.
The first finding can be seen as a failure to support an enduring assumption in cognitive neuroscience, albeit one that is increasingly being called into question (Curtis and D’Esposito, 2003
; Lebedev et al., 2004
; Curtis and Lee, 2010
; Lewis-Peacock and Postle, 2012
). Although on its own it might be qualified as a null result, there are several factors that must influence its interpretation. Most saliently, it is paired with a positive result using the same method, and derived from statistically “subthreshold” voxels located in areas that are active during the perception of the to-be-remembered information. Empirical evidence thus shows that this method is sensitive. Indeed, although there remains some controversy about the physiological and representational factors that underlie the patterns of activity that correspond to different brain states (Freeman et al., 2011
; Thompson et al., 2011
), we are not familiar with any suggestion that there may exist brain states to which MVPA is less sensitive than traditional analysis of activation levels of individual voxels or groups of voxels. To the contrary, the near-consensus view is that MVPA methods are more sensitive than traditional activation-based analyses (Kriegeskorte et al., 2006
; Norman et al., 2006
; Haynes, 2011
; Lewis-Peacock and Postle, 2012
; Jimura and Poldrack, 2012
Further, although we cannot rule out the possibility that stimulus information might be represented in frontoparietal cortex at either a spatial scale that is too fine to be detected with our fMRI methods, or perhaps via a signal to which BOLD is relatively insensitive (e.g., low-frequency oscillations in local field potentials), we did demonstrate that this is not a limitation for the decoding of trial-specific task instruction
-related information. From this perspective our results are consistent with, for example, the finding from monkeys that PFC and posterior parietal cortex (PPC) represent the category
to which a stimulus belongs (Freedman and Assad, 2006
; Swaminathan and Freedman, 2012
). It is also worthy of note that although MVPA has been applied successfully to sensory processing in topographically organized cortex (e.g., the decoding of orientation (Harrison and Tong, 2009
; Serences et al., 2009
)), it has also been successfully applied to “higher level” processing in polymodal cortex. Thus, for example, MVPA has demonstrated contextual reinstatement during episodic memory retrieval (Polyn et al., 2005
), the recognition of individual faces (Kriegeskorte et al., 2007
), and neural correlates of free choice (Soon et al., 2008
), all entailing the decoding of information from polymodal temporal, parietal, and/or frontal cortex.
Consistent with our preferred interpretation of the null findings in frontal cortex are several factors. First, there are the results from extracellular recording in monkeys performing a similar task with similar stimuli, where no evidence for direction-selective persistent activity was found in the PFC throughout the delay period (Zaksas and Pasternak, 2006
; Hussar and Pasternak, 2012
). Second, a similar pattern to the MVPA results that we describe here has been reported for STM for four categories of visual objects (Linden et al., 2012
) and for complex artificial visual stimuli (Christophel et al., Revision under review). Third, the fact that STM can be intact despite lesions of PFC (D’Esposito and Postle, 1999
; Tsujimoto and Postle, 2012
) is consistent with the failure to find physiological evidence for STM representations in this region.
The frontoparietal network that has been a focus of this study is known to support the endogenous control of attention (Corbetta and Shulman, 2002
; Beck and Kastner, 2009
; Noudoost et al., 2010). Interestingly, one account of WM storage is that it is supported by this same top-down mechanism (e.g., Armstrong et al., 2009
; Curtis and D’Esposito, 2003
; Postle, 2011
). From this perspective, the sustained delay-period activity observed in this study may correspond to a control signal that does not vary with stimulus identity. Future work would need to reconcile this possibility with the finding that multivariate patterns of frontoparietal activity do discriminate between directions of motion during a sustained attention task (Liu et al., 2011
). In addition to specifically memory-related functions, many other functions might be supported by sustained delay-period activity of frontal and parietal regions. Because the frontal and parietal activity observed in the present study () resembles activity that has been reported in countless prior neuroimaging studies (Curtis and D’Esposito, 2003
), it may well be that it does not correspond to a stimulus-specific or even task-specific function. More general demands that many cognitive tasks (including working memory tasks) have in common include decision making (Curtis and Lee, 2010
), prioritizing certain task-relevant representations and/or processes over others (Miller and Cohen, 2001
), monitoring the environment to control the processing of potentially interfering exogenous events (Chao and Knight, 1998
; Postle, 2005
), actively representing a “behavioral set” (Woolgar et al., 2011
), and monitoring behavior so as to prevent prepotent responses (Knight and D’Esposito, 2003
), including perseverative responses (Milner, 1963
; Tsujimoto and Postle, 2012
). (Note that although the behavioral set account might be consistent with the successful decoding of cue identity in frontal and parietal regions, this explanation does not generalize to the first portion of the delay period.) This is, of course, an incomplete list.
One important question for future study is the nature of the mental codes with which subjects represent motion information across the delay period. In the monkey, a psychophysical study using backward masking provided evidence that the initial memory trace is perceptually based, retaining a high-fidelity representation of the sample (including such trial-irrelevant information as the local velocity of individual dots in the random-dot motion stimulus). This representation only endured few hundred milliseconds into the delay period, however, perhaps because, in this study, the animals could predict the major features of the impending memory probe (Zaksas et al., 2001
). Although the BOLD signal did not afford high temporal resolution in the present study, results with classifiers trained on different portions of the trial suggested that the mnemonic representation is relatively stable. We cannot know with certainty, however, whether this representation was primarily perceptual, motoric, or categorical in nature, or perhaps some combination of these. Our working assumption is that the mnemonic representation of direction was perceptually based, because it is from visual regions that we were able to recover stimulus direction information. Had subjects used, for example, a covert eye-movement strategy, we would have expected to have been able to decode stimulus information from frontal and parietal regions (Ikkai and Curtis, 2011
). The same reasoning makes us skeptical that subjects depended on a verbal strategy for remembering either direction or speed. We did not, however, monitor eye movements, nor take steps to discourage covert speech.
The results presented here highlight the differing conclusions that can be drawn from activation- vs. information-based analyses of the same data set. In so doing, they raise questions about the longstanding belief that information retained during working memory is stored via sustained delay-period activity, preferentially in frontal and parietal cortex. Instead, the memory trace may be represented in patterns of “subthreshold” levels of activity distributed across regions of low-level sensory cortex.