|Home | About | Journals | Submit | Contact Us | Français|
For effective decision making, individuals must be able to form subjective values from many types of information. Yet, the neural mechanisms that underlie potential differences in value computation across different decision scenarios are incompletely understood. Here, we used functional magnetic resonance imaging (fMRI), in conjunction with the machine learning technique of support vector machines (SVM), to identify brain regions that contain unique local information associated with different types of valuation. We used a combinatoric approach that evaluated the unique contributions of different brain regions to model generalization strength. Local voxel patterns in left posterior parietal cortex contained unique information differentiating probabilistic and intertemporal valuation, a result that was not accessible using standard fMRI analyses. We conclude that the early valuation phases for these reward types differ on a fine spatial scale, suggesting the existence of computational topographies along the value construction pathway.
Understanding the mechanisms that underlie the construction of value is a central goal of neuroeconomic research (Camerer et al., 2005; Glimcher and Rustichini, 2004; Rangel et al., 2008). In the current study, we investigated the neural systems underlying the valuation of economic outcomes through the application of alternative analytic techniques to functional magnetic resonance imaging (fMRI) data.
Recent work has delineated neural systems that underlie valuation processes, both in human (Hare et al., 2008; Kable and Glimcher, 2007; Plassmann et al., 2007) and nonhuman primates (Padoa-Schioppa and Assad, 2006; Roesch and Olson, 2004), as summarized in comprehensive reviews (Montague and Berns, 2002; Padoa-Schioppa, 2007; Wallis, 2007). These and earlier works (Delgado et al., 2000; Knutson et al., 2000; Schultz et al., 1997) have focused on the ventral striatum (VSTR) and medial prefrontal cortex (MPFC)/orbitofrontal cortex (OFC), based on demonstrations that these regions encode anticipated value and chosen value, respectively. Together, these regions may convert a broad range of rewards into a common currency that expedites choice, with OFC ultimately playing the central role in comparison (Montague and Berns, 2002).
Although there is mounting evidence to support the role of OFC in representing subjective value and guiding economic choice, relatively little is known about how other regions may support the construction of value based on different underlying information (Rangel et al., 2008). For example, parietal cortex has been discussed in some models of value computation, specifically with regard to evaluating and integrating the value of potential actions (Glimcher et al., 2005; Sugrue et al., 2004). Even if OFC represents the value of choice options, across a wide range of information, there may still be processing differences associated with different forms of valuation, most likely in earlier regions that support particular computations necessary for some decision variables, but not others. These differences may not be reflected in the overall metabolic demand within a given region, and thus could be inaccessible via standard neuroimaging analysis methods. To identify local information associated with the construction of value, we used statistical analyses from the emerging field of machine learning.
Machine learning refers to a set of statistical learning methods, a subset of which are commonly called “pattern classifiers”, which uses the characteristics of known examples to develop feature sets that can classify new examples into categories (Vapnik and Lerner, 1963). Although machine learning algorithms represent a relatively new approach to fMRI data analysis, their use is increasing rapidly (Haynes and Rees, 2006; Norman et al., 2006; O’Toole et al., 2007). Support vector machines (SVM) underlie the most common approaches (Cortes and Vapnik, 1995), although other techniques have been applied (Friston et al., 2008; Haynes and Rees, 2005; Mitchell et al., 2004). Given a binary classification problem, an SVM finds the maximum-separating hyperplane for a pattern, while minimizing the upper bound on the generalization error of the classifier. Unlike univariate analyses that evaluate the overall change in activation of a cluster of voxels, pattern classification employs a multivariate approach that includes information derived from the joint changes in activation across voxels. The most common application has been to decode various feature representations and demonstrate the existence of topographies in visual cortex (Cox and Savoy, 2003; Haynes and Rees, 2005; Kamitani and Tong, 2005; Kay et al., 2008). Yet, pattern classification has also shown potential for evaluating covert and subjective experiences: lie detection (Davatzikos et al., 2005), prediction of conscious decisions about emotional faces (Pessoa and Padmala, 2005), attending to specific features (Serences and Boynton, 2007), hidden intentions (Haynes et al., 2007), free will (Soon et al., 2008) and basic choices (Hampton and O’Doherty, 2007). Here, by comparing the valuations of different types of rewards, we aimed to provide new information about the value construction process.
Two common types of valuations involve outcomes that only occur with some probability or after some time delay. A vast body of theoretical and behavioral research has provided descriptive and prescriptive theories of decision making in probabilistic and intertemporal settings (Frederick et al., 2002; Starmer, 2000). Many behavioral phenomena are similar across these two domains, leading to theoretical models of intertemporal and probabilistic choice that account for these commonalities (Green and Myerson, 2004; Prelec and Loewenstein, 1991). Each has, at least independently, been a target of neuroscience research. For neuroimaging studies of uncertainty (Critchley et al., 2001; Hsu et al., 2005; Huettel et al., 2005; Huettel et al., 2006; Knutson and Peterson, 2005; Kuhnen and Knutson, 2005; Paulus et al., 2003; Preuschoff et al., 2006), most have implicated insular cortex (INS), posterior parietal cortex (PPAR), inferior prefrontal cortex (IPFC), and OFC in processes used to evaluate and choose between uncertain prospects. There is a comparatively smaller collection of neuroimaging studies of intertemporal choice (Hariri et al., 2006; Kable and Glimcher, 2007; McClure et al., 2007; McClure et al., 2004; Tanaka et al., 2004; Weber and Huettel, 2008). These studies point to a variety of regions: VSTR, MPFC, medial orbitofrontal cortex (MOFC), anterior cingulate cortex (ACC), posterior cingulate cortex (PCC), PPAR, and dorsolateral prefrontal cortex (DLPFC). When considered as a whole, these studies have identified a set of candidate regions that may support value construction.
We report here functional neuroimaging data collected while participants valued probabilistic and intertemporal rewards. Whereas traditional machine learning analyses are concerned with the absolute predictive power of a model, our analyses sought to identify the specific brain regions that – when added to a classifier – contributed most to relative changes in predictive power. The localization of this unique information provides a robust measure for the contributions of neural patterns to brain states. Specifically, our results point to potential neural topographies involving explicit intertemporal or probabilistic value computations.
Thirteen (four male) healthy participants (mean: 23 ± 5 years, range: 18–35) participated in an experimental session involving collection of both behavioral and fMRI data. Two of these individuals were dropped from the sample prior to data analyses: one for excessive head motion and one due to a data collection error, leaving a final sample of eleven participants. All participants were prescreened to exclude individuals with prior psychiatric or neurological illness. Participants gave written informed consent as part of a protocol approved by the Institutional Review Board of Duke University Medical Center.
Each trial began with the presentation of a probabilistic or intertemporal economic prize (Fig. 1, presented in grayscale). For the probabilistic trials, outcome probabilities ranged continuously from 0.25 to 0.75 and prize values ranged from $20 to $75. Both the prize and its probability were drawn pseudo-randomly from uniform distributions, constrained so that expected value ranged from $15 to $50. The probability of winning the prize was represented by the green area of the rectangle, which was always on top. The alternate outcome was $0 on all trials. For the intertemporal trials, the period until delivery ranged from one to sixteen weeks, and the prize ranged from $15 to $50 (integer values only). The delay and prize were both drawn randomly from uniform distributions. Here, length of delay was represented by the gray portion of the rectangle, meaning the green area represented the fraction of the sixteen weeks that the participant would not have to wait. Thus, our two types of trials were matched as closely as possible in their visual representations.
In a pre-scanning session, participants faced 12 trials of each type (24 total). All gambles were presented in pseudo-random order, with no more than two of the same type appearing consecutively. Each gamble was initially shown for 7 s, then participants were given unlimited time to type their subjective value (i.e., certainty or immediacy equivalent) for that prize. This initial session served two purposes. First, it familiarized the participants with the task before the later scanner session. Second, it allowed us to calculate preliminary estimates of risk- and delay-sensitivity that guided selection of the potential response values within the fMRI session. Participants completed a second, short, practice run during the collection of the anatomical images within the scanner.
In the scanner session, participants completed eight runs, each with 24 valuation trials. The timing was the same as the practice run: prizes were visible for 7 s, followed by a 4 s response window. Eight potential values were presented for each prize, and participants selected the value closest to their subjective rating by pressing one of eight buttons. The potential response values were presented in ascending or descending order (randomized across trials), with the predicted response for each participant randomly placed between button three and button six. Once a button was pressed, a blue rectangle was displayed around the selected value (Fig. 1) for a brief interval (0.2 s). Trials were separated by a varying interval of 0.2 s to 0.5 s. The experiment was presented using the Psychophysics Toolbox (Brainard, 1997) in MATLAB (MathWorks).
Participants were compensated entirely with Amazon.com gift cards. They were guaranteed at least $30 in gift cards, although it was possible that a fraction (including all) of their payment could be delayed by up to 16 weeks. Non-cash payment was chosen to ensure accurate delayed payment (all payments were sent via an email confirmation) and to maintain a single currency throughout the experiment (no cash amount was paid; the guaranteed amount was built into minimum values for the intertemporal and probabilistic prizes). Participants were informed of all of these conditions prior to agreeing to participate.
Payment was based on two randomly-chosen trials from the scanner session. First, the participants rolled an eight-sided die twice, once for a probabilistic trial and once for an intertemporal trial, to determine the two runs from which a computer would randomly select the trials for payment. Once the trials were chosen, we used a Becker-DeGroot-Marschak procedure (Becker et al., 1964) to determine payment. For each trial, the computer randomly drew an integer from a uniform distribution of values between zero and the expected value of the chosen trial’s gamble. If the random integer was less than the stated subjective value for that trial, then the gamble was carried out. If it was greater than the participant’s value equivalent, they were given a gift certificate (payable immediately) equivalent to the integer value, in dollars. Average gift-certificate payout was $55 ± $22.
We acquired fMRI data on a General Electric 3.0 Tesla MRI Scanner with a multi-channel (eight-coil) parallel imaging system. Images sensitive to blood-oxygenation-level-dependent (BOLD) contrast were acquired using a gradient echo-planar imaging (EPI) sequence (TR = 2000 ms; TE = 27 ms; FOV = 256 mm; voxel size = 4*4*4 mm) at 30 axial slices parallel to the line connecting the anterior and posterior commissures. For each participant we collected 8 runs consisting of 182 images (except for one participant with 178), with an initial saturation buffer of ten images. Whole-brain high-resolution T1-weighted structural scans with voxel size 1*1*2 mm were acquired from each participant for normalizing and coregistering data.
Functional images were corrected for motion using MCFLIRT (Motion Correction using FMRIB’s Linear Image Registration Tool) (Jenkinson et al., 2002), were slice-time corrected, and were normalized into a standard stereotaxic space (Montreal Neurological Institute, MNI) using FSL 4.0 (Smith et al., 2004), maintaining a resolution of 4*4*4-mm for pattern classification. We note that data were left unsmoothed to preserve local voxel information.
We selected anatomically distinct regions of interest (ROIs) based on recent fMRI studies of probabilistic or intertemporal outcome environments (Huettel et al., 2005; Huettel et al., 2006; McClure et al., 2007) and on the existing literature on executive function and decision making (Hampton and O’Doherty, 2007). Each ROI was made up of several voxel spheres centered on various (x, y, z) MNI coordinates. ROI masks were constructed using the Wake Forest University PickAtlas 2.4.
All behavioral analysis was carried out in MATLAB. Participants’ discounting rate was estimated using the standard hyperbolic function (Frederick et al., 2002),
where V is the participant’s reported value for prize Z with delay W in weeks. The discounting rate is then k. A perfectly delay-neutral individual would have k = 0. For the probabilistic trials, we assume the subjective value V of a monetary prize Z can be modeled as a simple power function (Huettel et al., 2006),
where a determines whether a participant is risk-averse (a < 1), risk-neutral (a = 1), or risk-seeking (a > 1). Given our probabilistic prizes, we also assumed participants’ preferences have an expected utility representation.
Following preprocessing, we identified an estimate of mean signal intensity within every voxel for every trial, defined as the mean signal across three consecutive volumes within the valuation period (lagged by 5 s following valuation onset). The goal was to capture BOLD signal that corresponded to the early phases of the valuation part of the task, specifically avoiding activity associated with the response. We then used the signal in specific voxels to construct pattern classifiers from four sets of feature data: ROI global, ROI local, searchlight global, and searchlight local (Fig. 2). The ROI global signal was calculated by taking the mean signal across all N voxels in the ROI. The ROI local signal was obtained by subtracting the ROI global signal from each individual voxel’s signal, resulting in an N-dimensional pattern vector for each trial pair, where N depends upon the ROI or combination of ROIs. For both of these types of ROI data, we used SVM to determine whether local information decoded trials of intertemporal valuation versus probabilistic valuation. In all cases, the classifier was a linear SVM using a radial basis set. A grid search was used to optimize the tradeoffs among error, margin, and the radial basis function used.
In order to check the robustness of our ROI analyses and to reduce the size of our feature spaces used to train the classifiers, our third and fourth sets (Fig. 2) used searchlight feature sets (Kriegeskorte et al., 2006). These datasets are also more spatially specific; we might be able to identify individual voxels that are driving ROI local or ROI global performance. For every voxel in each ROI, we constructed a searchlight corresponding to a spherical cluster of radius 12 mm that contained M voxels (up to 123, depending on the distance between that voxel and the ROI boundary). The searchlight global classifier was defined as the mean signal of all voxels in the searchlight, while the searchlight local classifier was obtained (as for the larger ROIs) by subtracting the searchlight global signal from the activation of every voxel in the ROI.
Prediction performance was judged based on five-fold cross-validation (CV), a common method for evaluating the reliability of a pattern classifier (Mitchell et al., 2004). We repeatedly selected four-fifths of the trials to train the SVM, with the remaining portion of trials used as the test set of trial pairs. This process was repeated five times, and the average performance across the five tests was the SVM’s CV rate. For any SVM, a high CV is indicative of a flexible learning algorithm that is more likely to accurately classify additional data. Classification was performed using the LIBSVM software (http://www.csie.ntu.edu.tw/~cjlin/libsvm). For statistical testing of individual ROIs and searchlights, we tested CV against performance of a chance classifier (50%). All claims of significance refer to p < 0.05, corrected for multiple comparisons using False Discovery Rate (FDR), where the correction is over the number of classifiers being compared (e.g., for single ROIs, there were 13). As an additional test for our classifier accuracy, we also used LIBSVM to generate receiver operating characteristic (ROC) curves, which can describe the relationship between the fraction of correct predictions for one trial-type (e.g. risk) and the fraction of incorrect predictions for the other trial-type (e.g. delay), for each participant and ROI.
For the ROI local classifiers, we also constructed classifiers using all possible combinations of two, three, and four ROIs. This resulted in 1092 SVMs for each participant. To compare the performance of these classifiers to each other, we took their CV rates and ranked them (within each participant) from 1 to 1092. This process allowed us to control for differences across participants in the mean performance of all classifiers (i.e., overall better classification across all ROIs). The average ranks of each possible ROI combination across participants were then used for paired-difference rank-order tests. For statistical testing of individual ROI contributions to combinations, we tested increases or decreases against no change (0%). Images of CV performance were generated using MRICron (Rorden et al., 2007).
Analysis using a general linear model (GLM) was carried out using FEAT (FMRI Expert Analysis Tool) 5.92 in FSL 4.0. First, MELODIC 3.0 (Beckmann and Smith, 2004) was used to identify scanner artifacts, and an automated and unbiased process was implemented to remove them from each run for all participants. Motion correction was done using MCFLIRT, non-brain removal using BET, the high pass filter cutoff was set to 100, and the data were spatially smoothed with a Gaussian kernel of 8 mm. Registration to MNI space was carried out using FLIRT.
The model was constructed with four explanatory variables: two for intertemporal trials and two for probabilistic trials. Within each valuation type, one explanatory variable represented the valuation period, and a second represented the response period. Each of the two valuation regressors was modeled for the full 7 s valuation interval, time-locked to the onset of the presentation of the gamble or delayed prize, and convolved with the canonical double-gamma hemodynamic response function. Regressors were constructed similarly for the response period, although that interval was modeled for its 4 s interval. Initial multiple regressions were conducted on each of the eight individual runs for our eleven participants. Data were combined across runs using a fixed-effect analysis, and data were combined across participants using random-effects analysis via FSL’s FLAME algorithm. These third-level analyses used the standard cluster significance threshold of p = 0.05, corrected for multiple comparisons using FDR across the whole brain. We report both whole-brain analyses and analyses masked by our a priori ROIs (i.e., with a smaller number of voxels and thus fewer statistical comparisons).
Participants successfully indicated their subjective value on nearly all trials, with ten of eleven hitting an average of 99.6% of 192 trials; the remaining participant missed 16%. (We note that all reported analyses were robust to elimination of this participant, or of any other participant, and that this subject’s preferences did not differ from those of other participants when responses were made). Across all trials, there were no systematic biases toward responding with one hand. Responses were balanced across the eight possible values for each trial, with the most responses occurring near the middle values, reflecting our pre-estimation procedure. There was no significant difference in number of responses between the two trial types for any of the eleven participants. Mean response time (RT) for the intertemporal condition (1.8 ± 0.5 s) was significantly faster (paired difference, p < 0.01) than that for the probabilistic condition (2.1 ± 0.4 s). This may reflect a greater difficulty in locating the chosen value on the reporting screen, not necessarily an increased time spent valuing the gamble, given that participants were allocated seven seconds to complete valuation before the response screen was even displayed. Using RT as a one-dimensional classification rule, we found an average CV of 64.1% across participants. This indicates that RT contains some valuation-relevant information (no relevant information would have place the classifier CV at 50%), providing a useful baseline for performance.
We also obtained measures of delay and risk preferences for each of our participants (Table S1, Supplementary Material) using the data from the eight scanning runs. Individual preferences were estimated using Equations 1 and 2. Our participants ranged from delay-neutral to quite delay-averse, with all participants’ behavior fit well by a hyperbolic discounting function (median r2 = 0.97; Fig. S1, Supplementary Material). Participants were risk-neutral to slightly risk-seeking, with their choices generally well fit by a power model for risk (median r2 = 0.84; Fig. S1, Supplementary Material). Risk and delay preferences were not correlated across participants [r (9) = 0.07, p = 0.84].
Our initial analyses evaluated the degree to which overall activation in ROIs predicted the type of valuation: probabilistic or intertemporal. All thirteen of our a priori chosen ROIs performed significantly above chance when the pattern classifier used local information; i.e., the pattern of increases and decreases across voxels from the average ROI signal (Fig. 3). The top performing local signal region was the left posterior parietal cortex (LPPAR; CV of 76.2%, p < 0.001). In contrast, none performed significantly above chance when the pattern classifier used our control feature space, global information; i.e., the average signal across all voxels in the ROI. For average signal, the highest was in PCC (54.0%); after correcting for multiple comparisons, this effect was not significantly different from chance. For all regions, the local signal encoded significantly more information than the global signal, with the greatest difference again found in LPPAR (local signal CV was 24.8% greater than global signal CV, p < 0.001).
Our ROC curves for the ROI local classifiers provide further supporting evidence that LPPAR contains unique valuation information. Using the area under the curve (AUC) as our measure of classifier performance (Fawcett, 2006) across all possible thresholds, we found that the average AUC for LPPAR was 0.79 (p < 0.001), the highest of the 13 ROIs. Further, this measure of classifier accuracy led to similar conclusions as CV performance about the relative information carried by the different ROIs (Fig. 3, circles).
A concern intrinsic to fMRI data is that this differential classifier performance could depend on functional signal-to-noise ratio (fSNR), which could itself vary across regions based on scanner properties (e.g., distance from receiver coils in the phased array). We determined the fSNR for every ROI and found that fSNR differences explained less than 6% of all variance across ROIs. This indicates that differences in fMRI data sensitivity can only account for a minimal portion of our variance in predictive power across regions.
To evaluate the unique local information contained within each region, we combined data from all combinations of one, two, three, and four regions, resulting in a total of 1092 classifiers for each participant. By using rankings of the ROI combinations to perform our statistical tests (see Materials and Methods), we can assess how these combinations perform with respect to each other, while controlling for interindividual differences in absolute predictive power. Across participants, the best-performing pair was LPPAR-RPPAR, with an average CV of 76.1%. The triple combination of LPPAR-PCC-RPPAR (average CV of 77.7%) was the best-performing of all possible combinations. This group was significantly better (p < 0.05) than all other combinations of three or four ROIs that did not contain two of those three regions. Interestingly, this combination of three is significantly better at predicting valuation type than either PCC or RPPAR on their own (p < 0.01), but not significantly better than LPPAR by itself (p = 0.32). Finally, the top two combinations of four ROIs were DSTR-LPPAR-PCC-RPPAR (76.6%) and INS-LPPAR-PCC-RPPAR (76.3%); note that neither of these outperformed the best combination of three ROIs.
Our results showed that local information in LPPAR was a better predictor of the form of valuation than local information from any other region. LPPAR by itself significantly outperformed all other combinations (p < 0.05) that did not contain at least one of LPPAR, PCC, or RPPAR, and was not significantly outperformed by any combination of regions. Moreover, when considering all of the 1092 possible classifiers, LPPAR consistently does better than PCC (p < 0.02) and RPPAR (p < 0.001). Thus, our data indicated that LPPAR provided the greatest amount of unique information differentiating probabilistic and intertemporal valuations.
To illustrate the independent contributions of the chosen ROIs, Fig. 4A shows increases or decreases in CV of a specific ROI when one of the other twelve ROIs was paired with it. For example, when LPPAR was paired with AMYG (lower left of Fig. 4A) the resulting classifier increased CV by 15% when compared to AMYG alone. Yet, as shown in the upper-right corner, the combined ROIs actually performed worse (i.e., a reduction in CV by 5%) than LPPAR alone. Fig. 4B collapses across rows of Fig. 4A to demonstrate that, on average, LPPAR (average 9.3%, p < 0.001), PCC (6.6%, p < 0.001), RPPAR (5.9%, p < 0.001), and DLPFC (5.5%, p < 0.003) all added predictive power to the other twelve ROIs. When collapsing across columns for combinations of two ROIs (Fig. 4C), the CV rates of many of those regions were either unaffected or improved by the addition of a second ROI. The exception is LPPAR, which evinced a significant decrease in CV when paired with any another ROI (−2.9%, p < 0.001). That is, the addition of other ROIs to LPPAR consistently decreases the new classifier’s CV. No other ROI exhibited similar decreases when paired with other regions.
These results held even when considering larger combinations of three or four ROIs (Fig. S2, Supplementary Material). The addition of LPPAR to other combinations of ROIs still consistently improved the overall predictive power, whereas the additions of other ROIs to combinations that contained LPPAR tended to reduce predictive power. Thus, data from these combinations of ROIs converges upon a robust conclusion: voxels within LPPAR contained unique information that differentiates probabilistic from intertemporal valuation.
While ROI-based approaches have many advantages for fMRI research (e.g., reducing the number of statistical tests), they risk misidentifying functional regions, especially if the voxels carrying meaningful signal are included within models that incorporate a large number of noisier voxels. As a check on our primary ROI results, we can assess the predictive power associated with sub-regions within our ROIs: we created pattern classifiers from smaller searchlights (Fig. 2B), defined as ~100-voxel spheres surrounding each voxel in the ROI (Haynes et al., 2007; Kriegeskorte et al., 2006). The process for constructing the global and local classifiers was analogous to the process for each ROI. We tested the predictive power present in a given searchlight using the same five-fold CV procedure as with the ROIs.
Searchlight results are summarized in Table 1. The only region containing searchlights with predictive accuracy greater than 70% was LPPAR (Fig. 5A), within which more than 23% of all voxels passed that level (maximum searchlight CV of 78.6%). We note that this performance, which reflects contributions from a small set of contiguous voxels in one brain region, was roughly equivalent to the best combination of local information across multiple ROIs. For every ROI, the mean global signal classifier performance fell between chance and ~55%, although each ROI had at least one local searchlight and one global searchlight with a CV significantly above chance. Moreover, the greatest difference between local and global searchlights (~39%) is again found in LPPAR (Fig. 5B) indicating that LPPAR contains the greatest amount of local information. These findings are in line with the ROI results (Fig. 3 and Fig. 4).
Also interesting was the observation that the peak global searchlight in MPFC (66.7%) was higher than the peak local searchlight in MPFC (64.9%), indicating that local information made minimal contributions (if any) compared to the average signal within that region.
As a check on our pattern classification data, we used standard fMRI analysis methods based on the GLM to identify activation clusters that differentiated probabilistic and intertemporal valuation (local maxima for all contrasts are listed in Table S2, Supplementary Material). For valuation of intertemporal delays (D), we found significant activation in both ACC and medial frontal cortex. On trials involving probabilistic risk (R), significant activation was also found in ACC, as well as in the superior frontal gyrus, and along with portions of LPPAR, with a local maximum in the superior parietal lobule. Importantly, the CV for the local searchlight about that local maximum was 57.7%, indicating a discrepancy between the coordinates of maximal global and local information in LPPAR. We also identified regions that were significantly more active for intertemporal valuations compared to probabilistic valuations (D>R) or for probabilistic compared to intertemporal (R>D). For D>R, significant differences were found in MOFC and several other cortical regions. The only cortical regions that proved significant for the R>D contrast were regions of visual cortex.
As an additional check for our ROIs, we repeated our GLM analyses using our ROIs as a volume mask before thresholding and correcting for multiple comparisons (Table S3, Supplementary Material). We found the same ACC activation for both intertemporal and probabilistic valuations. Both valuation processes also recruited left IPFC, anterior INS, and bilateral PPAR. For D>R, ACC, bilateral DLPFC, MOFC, MPFC, and PCC all were significant. Importantly, none of the ROIs were significant for R>D, and PPAR did not distinguish either trial type from the other. Thus, we conclude that the use of pattern classification methods here provided novel information about the neural systems differentiating forms of economic valuation; i.e., information that was not available within standard fMRI analysis methods.
Convergent evidence from several forms of machine learning analyses indicated that unique information about value construction was provided by the pattern of local activation within LPPAR. The most-predictive voxels in LPPAR were located in the more posterior part of the superior parietal lobule, along the intraparietal sulcus, within the left hemisphere. These local computational maps were not evident when examining the results of standard fMRI analysis, perhaps because we used a simple valuation task that controlled for several possible confounds, including value range, motor preparation, and visual features. Our interpretation is that while the valuation processes of different rewards and sets of information may involve a similar system of brain regions, there is differential recruitment of local circuitry within at least one of those regions, posterior parietal cortex (PPAR).
Our standard fMRI analyses revealed brain regions typically implicated in decision making under uncertainty: aINS, lateral PFC, and PPAR, among others (Table S3, Supplementary Material). This result supports the inference that participants were treating the probabilistic and intertemporal valuations as meaningful decisions. For both valuation processes, we also observed significant dorsal ACC activation, which may be relevant to task and choice representation (Rushworth et al., 2007). Yet, when we examined the regression contrast between probabilistic and intertemporal valuation, no significant differences were observed in PPAR. Our pattern classification data, however, led us to a different conclusion: the local pattern of voxel activation within PPAR provides unique and highly predictive information about the form of valuation being attempted by the individual.
A recent study used an ascending search of ROI combinations to try to identify an optimal classifier in an executive function task (Hampton and O’Doherty, 2007). Our interest in the present study was not to correctly classify the largest percentage of our data, but rather to identify unique information that can form the basis of robust models of computation. We used a full combinatoric approach that developed classifiers from all combinations of up to four regions. This allowed measurement of the unique information within a given region that could contribute to a general model of value construction. We also used increases or decreases in cross-validation, rather than model prediction, as our measure of performance. This second step is crucial; regions that lead to increases in cross-validation reveal useful information and serve to construct a more robust model. Based on examination of all combinations of two, three, and four ROIs, we found that LPPAR consistently added the most information, regardless of combination size. Even though PCC and RPPAR also increased the predictive power of ROI combinations, these improvements disappeared when LPPAR was already present in the classifier, as is demonstrated in Fig. 4A. This conclusion was further supported by our searchlight analysis (Fig. 5 and Table 1). So, local information encoded in other brain regions was redundant with the information encoded in LPPAR, which further suggests that there are elements of value construction unique to PPAR.
That pattern classification identifies local patterns associated with distinct forms of valuation is by itself unsurprising. Indeed, neural differences between the processing of probabilistic and intertemporal decisions have been recently identified (Weber and Huettel, 2008). More critical is our finding that LPPAR contains unique local information that is not present elsewhere within the decision making system. One potential interpretation of this result is that it reflects a computational topography within that specific region, similar to topographies demonstrated using fMRI pattern classification for features (e.g., orientation) within the primary visual cortex (Haynes and Rees, 2005; Kamitani and Tong, 2005). Consistent with this speculation, lateral intraparietal areas have been postulated to provide maps of salience (Gottlieb, 2007) and subjective value (Glimcher et al., 2005). Alternatively, local patterns could reflect separate projections to other regions where other computations can be performed. We consider these possibilities in more detail in the following section.
Prior research has implicated parietal cortex, particularly the lateral intraparietal area, in the construction of value (Glimcher et al., 2005; Sugrue et al., 2005). This region may reflect an early stage in processing, one well before distinct classes of rewards are represented within a common currency (Montague and Berns, 2002). Despite the common assumption of fMRI analyses that the function of an ROI is homogenous, our searchlight analyses provided strong evidence for fine-scale heterogeneity within PPAR. We found that voxels within a 12 mm radius differentiated intertemporal and probabilistic trials; it was striking that these two valuations evoked different and reproducible patterns of activation within highly localized regions of PPAR.
Just as stimulus features like orientation (Haynes and Rees, 2005; Kamitani and Tong, 2005) and object category (Shinkareva et al., 2008) are represented by spatial topographies within specific regions, there may be a similar topography in parietal cortex based on the sorts of computations (e.g., distinct mathematical operations) necessary for economic valuations. Such spatial proximity could provide efficiencies for valuation of complex stimuli that require different subsets of local computations, as frequently would be the case for real-world decision stimuli (e.g., purchasing tradeoffs). These potential computational efficiencies may be described by two non-exclusive hypotheses. First, both forms of valuation, even if other aspects may differ, could require a common computation, and thus spatial proximity may reflect a need for common operations used to compute risk and delay. Second, spatial proximity could facilitate comparisons between these different valuation types (and perhaps others) within PPAR as may be required in many real-world decisions but not in the present study.
Given that our task design contained separate valuation and response phases, we can be confident that the observed effects in PPAR do not confound decision and motor preparation processes. Equally important is that our task differs considerably from those of previous fMRI studies using pattern classification. As examples, prior studies have asked participants to add or subtract (Haynes et al., 2007), to switch or stay in an executive function task (Hampton and O’Doherty, 2007), or to be honest or deceitful (Davatzikos et al., 2005). Our task, for comparison, presents participants with a serious of prizes and asks them to evaluate each one independently, not to choose between prizes. This is a more introspective procedure and hence provides novel information about subjective valuation, instead of distinguishing between types of choices themselves.
Thus, we conclude that PPAR plays a key role in the valuation process via preliminary information representation and integration prior to, but still necessary for, computing economic value. Because this PPAR information is likely upstream of other regions that ultimately encode subjective value in the context of choice (Padoa-Schioppa, 2007; Rangel et al., 2008), future models of both value computation and choice should account for what computational output PPAR might provide VSTR and OFC. While our task was not designed to parse out specific stages of value construction (e.g. probability and reward magnitude), some exploratory classification results indicated that many of our ROIs, including PCC, contain local patterns that can be used to discern high-probability from low-probability trials and short-delay from long-delay trials. This within-valuation class analysis provides further evidence of computational topographies that would play a role in early parts of reward presentation and value computation.
A common concern for neuroimaging experiments is that observed effects could reflect differences in degree (i.e., magnitude of neuronal activity), but not kind (i.e., population or pattern of neuronal activity). Considering our experiment, if participants were relatively delay-neutral (as they were in our study) and used a simple heuristic for those trials, probabilistic valuations could more strongly recruit brain regions involved in calculation and perhaps lead to more activation (in general) throughout cortex. Subregions of PPAR are widely-demonstrated to be involved in numerical computation and the representation of numerical information, particularly along the intraparietal sulcus (Cantlon et al., 2006; Dehaene et al., 2003) as well as approximate calculation (Piazza et al., 2004). If we interpret response time differences as the result of calculation time differences, then our data support this hypothesis, as both valuation types recruited PPAR within the GLM analyses (Table S3, Supplementary Material). However, there is evidence arguing against this possibility. Our searchlight data demonstrated that local patterns of voxel activation reliably discriminated the two forms of valuation, even after the global activation level across that searchlight was removed. We note, however, that other regions, including visual cortex, were significantly more active on probabilistic trials than on intertemporal trials (Table S2 and Fig. S3, Supplementary Material), consistent with overall computational or arousal effects in those regions.
Our posited role for PPAR in value construction could also reflect a cognitive process specific to our experimental design, but not general to economic valuation. Left parietal cortex (including the intraparietal sulcus) has been found to increase in activation when mediating representations of potential responses (Bunge et al., 2002), and PPAR activation scales with representational difficulty (Anderson et al., 2005). Our robust ACC activation in both valuation types is consistent with the task switching literature (Liston et al., 2006) which also frequently implicates PPAR as sensitive to stimulus representation conflict. Also, some of our participants demonstrated “aversive” or “neutral” preferences on one trial type (intertemporal) and “seeking” preferences on another trial type (probabilistic). A possible heuristic-based explanation might also apply here: a difference in how participants treat the two types of prizes, not in how they value them, could potentially lead to neural differences that were decoded by our classifiers. Importantly, we note that LPPAR remained the best discriminator of economic valuation even if we examined only the delay-averse participants or only the non risk-seeking participants (Fig. S4, Supplementary Material). Our results were also robust to removal of any one participants from the analyses. Finally, we note that LPPAR was either the most or second-most predictive region in every participant within the sample.
Our findings indicate several avenues for further investigations of the mechanisms – both psychological and neural – that underlie value construction. Many of our participants exhibited risk-neutral to risk-seeking behavior on probabilistic decision making trials. While such preferences are uncommon, we note that risk-seeking behavior has been sometimes reported both in neuroscience studies (Kuhnen and Knutson, 2005) and in behavioral research, particularly when stakes are small (Weber and Chapman, 2005). Moreover, our presentation-style was novel (visual representations were nearly identical across both trial-types) and our payments were entirely in Amazon.com dollars. One or both of these facets of our design may have changed the relative valuation (or set aspiration levels) across prizes. Economics frequently explores the effects of fungibility on value (Thaler, 1990), but further research would be required to evaluate the effects of prize modality upon risk preferences. We note that changes in the relative valuation of the prizes, even seemingly maladaptive changes to risk-seeking behavior, do not obviate conclusions drawn about the differential valuation of probability and delay.
Our study also raises questions about the existence of computational topographies involving components of subjective value. Future modifications of our task and methods could validate or refute our claims of differences in the early phases of value construction. Consider a task that involves choosing between two rewards, one probabilistic and one intertemporal. Under the first alternative above, similar activity in PPAR would be expected regardless of whether a participant faces a mixed-type trial or a same-type trial, as there is only a common computation to perform (another region, however, could show differences in activation associated with the comparison process). But, under the second alternative, PPAR activation patterns would discriminate whether a participant was choosing between the above two rewards versus two rewards of the same type, providing evidence of the comparison computation. The use of connectivity analyses (Buchel and Friston, 1997; Friston et al., 2003) or a more precise temporal classification process (Mourao-Miranda et al., 2007; Soon et al., 2008) could provide important information about differences in the input to and output from PPAR associated with valuation type. A more comprehensive understanding of the role of PPAR in valuation will then be informative for understanding and modeling the general valuation pathway (Rangel et al., 2008).
Using pattern classification, we demonstrated that local voxel information provides novel and independent evidence for neural regions of interest associated with economic valuation. We developed simple measures of predictive power, based on combinatoric measures, that identify the unique information provided by specific regions. We used convergent analyses to reach our primary objective: gaining insight into the neural processing of reward valuation. Notably, posterior parietal cortex (and, secondarily, posterior cingulate cortex) encoded information about whether participants were making an intertemporal valuation or a probabilistic valuation, and much of this information is not available from analyses of the mean activation across voxels. These results suggest a potential computational topography in posterior parietal cortex, a possibility worthy of future research.
Table S1. Summary of Participant Preferences.
Listed are the individual estimates of risk- and delay-preferences based on Equations (1) and (2). Both equations provided a good fit for all participants. All but one of the participants demonstrated mild risk-seeking preferences (a > 1). Most participants demonstrated mild to moderate delay-averse preferences (k > 0). The ordering of participants is the same as in Fig. S1.
Table S2. Summary of Results for GLM.
Significant clusters in delay, risk, and their difference, using a whole brain analysis. An analysis limited to each ROI is shown in Table S3. A standard cluster correction threshold of z > 2.3 and a cluster significance threshold of p= 0.05 were used, corrected for multiple comparisons across the whole brain. Region abbreviations: Cuneus (CUN), Lateral Occipital Gyrus (LOG), Lingual Gyrus (LING), Thalamus (THAL), Superior Colliculus (SC), Thalamus (THAL), Medulla (MED), Precentral Gyrus (preCG), Supramarginal Gyrus (SMG), Postcentral Gyrus (postCG), Gyrus Rectus (GREC), Superior Frontal Gyrus (SFG), Middle Temporal Gyrus (MTG), Angular Gyrus (AG), and Inferior Temporal Gyrus (ITG).
Table S3. Standard fMRI Analysis for Areas that Distinguish Delay and Risk.
Local maxima and significance values for regions that were significantly active for either delay or risk. Local maxima for areas that were significantly more active for delay than risk or for risk than delay are also listed. We ran third-level analyses using our thirteen ROI masks before thresholding to determine if any of them were significantly activated by either type of valuation. A standard cluster significance threshold of p = 0.05 were used, corrected for multiple comparisons.
Fig. S1. Participant Preference Behavior.
Value responses of all participants, plotted in order of their delay preferences, from most delay-averse (participant 1) to most delay-neutral (participant 11). The median response for each participant is the red line, the 25th and 75th quartiles are represented by the box. The error bars are drawn from the median, with length equal to 1.5 times the interquartile range. Outlier trials are plotted in red. (A) Subjective values for intertemporal prize trials, expressed as a ratio of subjective value over prize. (B) Subjective values for probabilistic prize trials, expressed as a ratio of subjective value over expected value.
Fig. S2. Increases in Cross-Validation for each Region of Interest in Combinations of Three and Four.
Average increase in CV accuracy (y-axis) from the addition of one ROI (x-axis) to combinations of two, creating combinations of three (A), and combinations of three, creating combinations of four (B). The relative ordering of ROIs from Fig. 4B, which reflected combinations of two ROIs, is maintained for larger combinations. Error bars indicate standard error of the mean.
Fig. S3. Expansion of Searchlight Analyses to the Entire Brain.
While we used a comprehensive list of regions commonly implicated in decision making and other executive processes, there remains a chance that other regions also encode information that would allow us to differentiate the two tasks. To account for this possibility, we also ran a searchlight for the entire brain using an analysis similar to that shown in Fig. 5. We found that in addition to LPPAR, there are portions of occipital cortex (OCC) containing CV rates well above 70% (A), as well as large differences between local signal searchlight performance and average signal searchlight performance (B).
Fig. S4. Cross-Validation Performance for Individual Regions of Interest with Subpopulations Removed.
To show that our results are not influenced by the specific risk- or delay-preferences of our participants, we repeated our analyses using the ROI local information after removing different subpopulations: our five most delay-neutral participants (A) and the three most risk-seeking participants (B). The overall performance and ordering of regions was strikingly similar, in both replications, to that from the entire sample (cf. Fig. 3), demonstrating that our effects were not driven by the preferences of particular subpopulations of our sample. Error bars indicate s.e.m.
We thank Ben Hayden for his comments on this manuscript. This research was supported by the US National Institute of Mental Health (NIMH-70685) and by the US National Institute of Neurological Disease and Stroke (NINDS-41328). The authors have no competing financial interests.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.