The neural correlates of subjective value during intertemporal choice

doi:10.1038/nn2007

Nat Neurosci. Author manuscript; available in PMC 2010 March 25.

Published in final edited form as:

Nat Neurosci. 2007 December; 10(12): 1625–1633.

Published online 2007 November 4. doi: 10.1038/nn2007

PMCID: PMC2845395

NIHMSID: NIHMS180727

The neural correlates of subjective value during intertemporal choice

Joseph W Kable and Paul W Glimcher

Center for Neural Science, New York University, 4 Washington Place, Room 809, New York, New York 10003, USA

Correspondence should be addressed to J.W.K. (Email: kable/at/cns.nyu.edu) or P.W.G. (Email: glimcher/at/cns.nyu.edu)

The publisher's final edited version of this article is available at Nat Neurosci

See other articles in PMC that cite the published article.

Abstract

Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions—particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex—tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

Since its inception, perceptual psychophysics has rested on the notion that the objective intensity of a stimulus is distinct from the subjective intensity that guides behavior¹^,². If, for example, the number of photons emitted by a light source is doubled, the perceived intensity only increases by a factor of about 0.3 (ref. ³). Since the eighteenth century, economics has rested on a similar observation⁴. If, for example, the gain earned for an action is doubled, people behave as though the desirability of this action has been increased by a much smaller fraction. Economic theory calls the functions that relate objective values and subjective desirabilities ‘preference functions’, and, as in psychophysics, the form of these perceptual functions is inferred from a person’s observed behavior.

Although the notion that choices reflect subjective desirability is central to nearly all economic theories of decision-making, studies of the neurobiological basis of decision-making have usually related neural activity to more objective measures⁵^–¹⁶ (but see refs. ¹⁷^–²⁵ for exceptions). Thus, although perceptual neurobiologists have accumulated evidence that the nervous system encodes subjective sensory percepts, rather than objective reality³^,²⁶^–²⁹, there is less evidence that subjective valuations, or preference functions, are encoded in the brain.

Decisions about monetary gains that will be available after different delays clearly illustrate the role of preference functions. An individual possessing a $20 check that can be cashed in one week might trade that check for as little as $18 in cash, available immediately. If the check was payable in one month, she might accept as little as $15 now in exchange. The subjective value of a $20 gain declines as the imposed delay to its receipt increases, a phenomenon known as temporal discounting³⁰^–³⁴. This decline in subjective value can differ across individuals—a second person might accept as little as $10 immediately in lieu of $20 in a month. Therefore, the objective amount and delay associated with each gain alone cannot predict choice; an idiosyncratic function that relates delay to subjective value is required. This person-specific function, which describes the decline in subjective value with increasing delays, is called a discount function. Whereas temporal discounting has been widely studied behaviorally, there have only been a few investigations of the neural mechanisms of inter-temporal choice in humans²²^,³⁵.

One of the most fruitful approaches for relating neural activity to behavior (including behavior that depends on subjective experiences) has been psychometric-neurometric comparisons²⁶^,²⁷. This method tests whether a particular, externally quantifiable variable influences both psychophysical and neurobiological measurements in a similar manner. One influential example of this approach found a close match between monkeys’ perceptual decisions about motion direction and predictions from simultaneously recorded neural activity in motion area MT²⁹. This approach, though, need not be restricted to directly observable phenomena such as subjects’ choices; it can also be used to relate indirectly revealed perceptual experiences or mental states to neural activity³^,²⁶. Indeed, the earliest psychometric-neurometric experiments showed that the subjective intensity of a somatosensory stimulus and the activity of somatosensory nerves were both similar power functions of objective stimulus intensity²⁸. Used in this manner, a psychometric-neurometric comparison does more than predict behavior — it tests a specific relationship, or linking proposition³⁶, between psychological and physiological states. It tests the hypothesis that the rate of neural activity (or some other candidate code) is linearly related to, and thus can serve as a representation of, the magnitude of a subjective mental experience.

Here we adapt the psychometric-neurometric technique to determine whether any brain areas encode the subjective value of delayed monetary rewards to individual human decision-makers. Subjects made repeated choices between immediate and delayed rewards while we simultaneously measured neural activity using functional magnetic resonance imaging (fMRI). We found a clear match between the subjective preferences of our subjects and neural activity in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex. These data provide unambiguous evidence that idiosyncratic preference functions are part of the neural mechanism for choice, as opposed to being a purely descriptive (‘as if’) theoretical construct, an idea that has been widely proposed in economics³⁷^–³⁹. From an economic perspective, these findings indicate the existence of a neural valuation process that is strikingly similar to the representations of subjective values that are employed in revealed preference theories. From a neurobiological perspective, these findings implicate particular neural structures in the subjective valuation of outcomes during decision-making.

RESULTS

Psychometric results

On each trial, subjects chose between a fixed immediate reward of $20 and a larger delayed reward that varied randomly from trial to trial (Fig. 1). The value of the larger reward varied from $20.25 to $110, and the delay varied from 6 h to 180 d. Before fMRI, subjects completed three preliminary behavioral sessions. On the basis of the choices in these behavioral sessions, we estimated a discount function for each subject (Fig. 2a–i). Note that interpreting Figure 2g–i as pure discount functions relies on the assumption that subjective value increases linearly with the objective amount of money. Violations of this assumption would not alter the fact that these curves reflect subjective value, but would mean that they are better described as discounted utility functions as they incorporate effects of amount as well as delay. These individual preference curves, which express how a subjective quantity (subjective value) changes as a function of an objective variable (delay and monetary amount), are directly analogous to sensory psychophysical functions²^,³. Consistent with previous findings in a range of species³⁰^–³⁴, the discount curves for all subjects were well–characterized by a hyperbolic function (median R² from scanning sessions = 0.95, range = 0.84–0.98):

(1)

where SV is subjective value (expressed here as a fraction of immediate value), D is delay (in d) and k is a subject-specific constant. The best fitting k parameter (the discount rate) varied widely across subjects. For our most patient subject (k = 0.0005, Fig. 2a,d,g), an immediate gain of $20 was less preferable than a gain of $21 in a month, while for our most impulsive subject (k = 0.1189, Fig. 2c,f,i), an immediate gain of $20 was more preferable than a gain of $68 in a month. Ten subjects (out of twelve) showed stable discount rates across sessions, even though sessions occurred over a 1–6-month time span (Supplementary Fig. 1 online). Only these ten subjects, with relatively stable functions over the tested range, were included in the fMRI experiment.

Figure 1

Intertemporal choice task. The sequence of events within a trial is shown. On each trial, subjects chose between an immediate and a delayed reward. The immediate reward was the same ($20) on every trial and was never presented visually. A red dot signaled (more ...)

Figure 2

Subject-specific discount functions. (a–c) Choice data from three subjects during a single scanning session. Points are shaded according to the imposed delay to the delayed reward, and denote the fraction of times the subject chose the delayed (more ...)

For comparison, we also fit each subject’s discount curve with a single exponential function:

(2)

as well as a sum of two exponential functions:

(3)

where SV is subjective value and D is delay as in equation (1), and c, β and δ are subject-specific constants. Consistent with the literature, the hyperbolic function equation (1) described the discount curves better than the single parameter exponential function equation (2) (Supplementary Fig. 2 online). The sum of two exponentials is one formulation of an economic model that explains hyperbolic-like discounting by the combined action of two systems³²^,³⁵: one (β) that discounts more steeply than the person’s resulting behavior, and a second (δ) that discounts less steeply than the person’s resulting behavior. Not surprisingly, as this functional form can account for hyperbolic-like discounting, the sum of exponentials function fit the data better than the single parameter exponential, and as well as the single parameter hyperbolic function (Supplementary Fig. 2). Although other forms of the discount function have been proposed³²^–³⁴, for simplicity, we considered only these three widely used forms.

Neurometric results

The ten subjects with stable discount functions completed the same choice task as was used in the preliminary behavioral sessions during fMRI. The behaviorally derived preference curves, which estimated how subjective value declined with delay for each individual, were used to find neural activity that correlated with subjective value. The subjective value of the delayed reward on each trial was estimated by multiplying the objective amount of the reward by the discount fraction (estimated behaviorally) for that delay. This inferred subjective value was then used as a regressor for brain activity. As the immediate reward was constant, this analysis finds areas that track the sum of the subjective values of both rewards, the difference in subjective value of the two rewards, or the ratio of subjective values of the two rewards.

Early in the trial, there was a significant correlation between neural activity and subject-specific subjective value (random effects group analysis, voxel-wise P < 0.001, spatial extent > 100 mm³) in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex (Fig. 3a, Supplementary Table 1 online). The timing of this effect (6–10 s into the trial) indicated that it arose from neuronal activity during the visual presentation of the delayed reward option, which we confirmed in a second model that assumed a hemodynamic response function (Supplementary Table 2 online). We did not find any significant effects of subjective value during other portions of the trial.

Figure 3

Group analysis showing areas in which activity is correlated with subjective value. (a) Areas in which neural activity was correlated with subjective value (during the 6–10-s window) in a random-effects group analysis, overlayed on the mean normalized (more ...)

A region in which neural activity is correlated with the subjective value of a delayed reward should show increased activity when the objective amount of this reward increases, decreased activity when the delay to this reward increases, and increased activity when the delayed reward is chosen because it is more valuable. This is exactly what we observed in these three regions (Fig. 3b–d, Supplementary Fig. 3 online and Supplementary Tables 3–5 online). Subjective value accounted for activity in these regions better than did the objective reward characteristics or the subjects’ choices: both the strength (peak z-score) and spatial extent (cluster size in mm³) of the correlation with subjective value were larger than those of these other variables (Fig. 3a–d and Supplementary Tables 1–5). Of these other variables, only delay-to-reward reached significance, and only in the medial prefrontal cortex. Furthermore, taking all voxels identified by any of these four variables (subjective value, monetary amount, delay and choice), subjective value showed a stronger effect in pair-wise comparisons with these other variables in 100% of voxels in the ventral striatum, at least 85% of voxels in the posterior cingulate cortex, and at least 62% of voxels in the medial prefrontal cortex. Idiosyncratic subjective value also accounted for activity in these regions better than did assuming a single fixed discount rate for all subjects (Fig. 3e and Supplementary Table 6 online). The effect of value, assuming a single fixed discount rate, only reached significance in the medial prefrontal cortex, and the strength and spatial extent of the correlation with subject-specific subjective value was larger in both the ventral striatum and the posterior cingulate cortex. Finally, these effects of subjective value cannot be explained by choice difficulty or corresponding attentional demands. Using two different indices of choice difficulty, we found no significant effects in the ventral striatum, medial prefrontal cortex or posterior cingulate cortex (Supplementary Tables 7 and 8 online).

We also found significant effects of subjective value in single-subject analyses, for subjects across the entire spectrum of discount rates. Despite marked differences in the discount functions of the five example subjects shown in Fig. 4a–e, ranging from patient (k = 0.0042) to impulsive (k = 0.1189), activity in parts of the ventral striatum, medial prefrontal cortex and posterior cingulate cortex were correlated with subjective value in each subject (Fig. 4f–j). These same regions thus exhibited a different pattern of activity across subjects, with each subject’s idiosyncratic pattern of brain activity being predicted by that subject’s idiosyncratic preferences (Fig. 5a–f, Supplementary Fig. 4 online). As a function of delay, both the subjective value of delayed gains and the neural activity associated with those gains decreased in a similar manner, whether subjects were more impulsive and showed steeper decreases in both domains (Fig. 5c,f) or were less impulsive and showed more gradual decreases in both domains (Fig. 5a,d).

Figure 4

Single-subject analyses showing areas in which activity correlated with subjective value. Data from five subjects are shown to illustrate that subjective value effects were evident for subjects spanning the entire range of behavioral discount rates. ( (more ...)

Figure 5

Single-subject time courses and neural discount functions. (a–c) Data from three subjects (HM, see Fig. 4a,f; RA, see Fig. 4c,h; and CH, see Fig. 4e,j) are shown. Data were averaged over all voxels that showed a correlation between activity and (more ...)

To quantify the degree to which these regions show a different pattern of activity across subjects, we averaged activity from all voxels that showed a subjective value effect in each of the three regions in each subject, and then regressed neural activity on each trial (summed over the 6–10-s window) separately against the imposed delay and objective amount associated with the variable delayed reward. If neural activity tracks subjective value across subjects, then the ratio of these two regression coefficients (delay over amount, reversing the sign on the negative delay coefficient) should be high for steep discounters and low for shallow discounters, as delay has a stronger effect on subjective value for more impulsive discounters. Consistent with this notion, this ratio increased as the subject’s discount rate increased (Fig. 6a; slope_VS ± standard error = 0.77 ± 0.19, t-test, P = 0.004; slope_MPFC = 0.90 ± 0.21, P = 0.003; slope_PCC = 0.99 ± 0.31, P = 0.02; slope_ALL = 0.87 ± 0.12, P < 10⁻⁷; VS, ventral striatum; MPFC, medial prefrontal cortex; PCC, posterior cingulate cortex).

Figure 6

Psychometric-neurometric comparisons. (a,d) A measure of the neural effect of delay is plotted against the subject’s behavioral discount rate for both (a) ROIs defined on the basis of the subjective value regression and (d) value ROIs defined (more ...)

The preceding analysis demonstrates the correspondence between neural activity and discount rate across subjects without making strong assumptions about the functional form of the discount function. For a more precise test, we assumed that the neural discount function has a hyperbolic form and estimated the neural discount rate for these areas, by refitting the regression model to the averaged time courses from the correlated regions while allowing the discount rate parameter (k) to vary. A psychometric-neurometric match requires both that the neural discount rate increases with the subject’s behavioral discount rate and that there is no difference between the two on average. By contrast, if activity in these regions were best explained by the objective amount (delay) of the delayed reward, then the neural discount rate should be consistently smaller (greater) than the behavioral discount rate. If activity were best explained by a single fixed discount rate across all subjects, then the neural discount rate should show no correlation with the behavioral discount rate. Consistent with a psychometric-neurometric match, the neural discount rate in these three regions increased with the subject’s behavioral discount rate (Fig. 6b, slope_VS ± standard error = 0.58 ± 0.23, t-test, P = 0.04; slope_MPFC = 0.75 ± 0.28, P = 0.03; slope_PCC = 0.80 ± 0.37, P = 0.06; slope_ALL = 0.69 ± 0.16, P = 0.0002), and the difference between the neural discount rate and the subject’s behavioral discount rate was centered on zero (Fig. 6c; median_VS = −0.0035, Wilcoxon signed rank test versus zero, P = 0.30; median_MPFC = −0.0008, P = 1; median_PCC = −0.0006, P = 1; median_ALL = −0.0009, P = 0.67).

As the previous analyses were conducted in regions-of-interest (ROIs) that were defined by the subjective value regression, they merely confirm that the regions identified exhibit a psychometric-neurometric match. It would be even more compelling if such a match existed in value-sensitive voxels in these regions that were selected in an unbiased manner. We defined value-sensitive ROIs in these areas in a manner that was unbiased with respect to the neural discount rate (see Methods). In these ROIs, the negative effect of delay, relative to the positive effect of amount, again increased with the subject’s behavioral discount rate (Fig. 6d; slope_VS ± standard error = 0.58 ± 0.36, t-test, P = 0.15; slope_MPFC = 0.86 ± 0.36, P = 0.046; slope_PCC = 0.90 ± 0.46, P = 0.10; slope_ALL = 0.78 ± 0.21, P = 0.0009). In addition, consistent with a psychometric-neurometric match, the neural discount rate in these ROIs increased with the subject’s behavioral discount rate (Fig. 6e; slope_VS ± standard error = 0.51 ± 0.48, t-test, P = 0.32; slope_MPFC = 0.73 ± 0.67, P = 0.31; slope_PCC = 0.89 ± 0.66, P = 0.23; slope_ALL = 0.70 ± 0.31, P = 0.03), and the difference between the neural and behavioral discount rates was centered on zero (Fig. 6f; median_VS = −0.0039, Wilcoxon signed rank test versus zero, P = 0.50; median_MPFC = −0.0008, P = 0.95; median_PCC = −0.0006, P = 0.94; median_ALL = −0.0009, P = 0.93; see Fig. 7a–d for a single-subject example).

Figure 7

Single-subject example of unbiased value ROIs and resulting neural discount functions. (a) ROIs in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex are shown for one subject (HM, same subject shown in Fig. 4a,f and Fig. 5a,d (more ...)

Finally, to evaluate an alternative hypothesis about the role of these regions in intertemporal choice³²^,³⁵, we repeated the above analyses using the β (steep exponential) and δ (shallow exponential) value functions from the sum of exponentials discount function fit to each subject’s behavior. Perhaps not surprisingly, as the β and δ value functions are correlated to some degree with subjective value, the brain regions in which activity is correlated with these three value functions are largely overlapping (Supplementary Fig. 5 online, Supplementary Tables 9–11 online). A psychometric-neurometric comparison, though, provides a more precise test of which value function best accounts for neural activity. As β and δ are exponential functions, we estimated the neural discount rates using a single exponential (rather than hyperbolic) function, and compared this rate to the behavioral discount rate estimated using a single exponential and the exponential β and δ discount rates estimated behaviorally. In both sets of ROIs, the difference between the neural discount rate and the behavioral discount rate was again centered on zero (Fig. 8a,b; subjective value ROIs, median = −0.0006, Wilcoxon signed rank test versus zero, P = 0.91; unbiased value ROIs, median = −0.0006, P = 0.77). By contrast, the difference between the neural discount rate and β is centered to the left of zero, indicating that the theoretically defined β term discounts more steeply than neural activity in these areas (Fig. 8c,d; subjective value ROIs, median = −0.0081, P = 0.001; unbiased value ROIs, median = −0.0088, P = 0.23), while the difference between the neural discount rate and δ is centered to the right of zero, indicating that the theoretically defined δ term discounts less steeply than neural activity (Fig. 8e,f, subjective value ROIs, median = 0.0024, P < 0.0001; unbiased value ROIs, median = 0.0016, P = 0.003). Thus, neural activity in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex tracks the subjective value of rewards as determined from behavior, rather than tracking a theoretically defined component of value that is more impulsive (β) or more patient (δ) than the person’s behavior.

Figure 8

Neural activity tracks subjective value, and not a more impulsive (β) or more patient (δ) estimate of value. (a,b) Difference between the neural (single exponential) discount rate and the behavioral (single exponential) discount rate, (more ...)

DISCUSSION

Revealed preference theories in economics posit that decision-makers behave as though different options have different subjective values⁴^,³⁷^–³⁹. Here we have shown that neural activity in several brain regions—particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex—tracks the revealed subjective value of delayed monetary rewards. Activity in these regions increases as the objective amount of a reward increases and decreases as the imposed delay to a reward increases. Furthermore, across subjects, the neural tradeoffs between amount and delay that are captured by the neurometric discount functions match the behavioral tradeoffs between these variables that are captured by the psychometric discount functions. These results indicate that choosing between immediate and delayed monetary rewards involves, at a physical level, comparing neurally encoded subjective values.

Our suggestion that the ventral striatum, medial prefrontal cortex and posterior cingulate cortex help to value immediate and delayed outcomes is consistent with a large body of existing evidence. The medial prefrontal and posterior cingulate cortices are part of a ‘baseline’ cortical system that is important for behavior mediated by internal goals⁴⁰^,⁴¹. Furthermore, all three regions show increased activity after subjects receive a reward or immediately before an expected reward. This reward-related activity correlates with reward size and probability⁶^–¹⁶, and is greater for more preferred rewards than for less preferred rewards¹⁷^,¹⁸^,²⁰^,²¹. In reinforcement learning tasks, activity in the medial prefrontal cortex is correlated with the predicted reward value estimated from behavioral models²²^,²⁴. In choice tasks, these three areas show greater activity during choices that involve large gains than during choices that involve small gains¹³^,²³^,⁴²^,⁴³, as well as greater activity during choices that involve small losses than during choices that involve large losses⁴⁴. Our results extend these findings in two ways. First, we show that the ventral striatum, medial prefrontal and posterior cingulate cortex also participate in the subjective valuation of delayed rewards, rather than just of immediate gains and losses. Second, by quantitatively estimating both behavioral and neural discount functions, we show that activity in these regions precisely tracks the subjective value of possible rewards during choice — in other words, these regions exhibit a psychometric-neurometric match.

Our findings falsify a hypothesis regarding the neurobiological basis of intertemporal choice³⁵. In ref. ³⁵, the authors hypothesized that these same three regions — the ventral striatum, medial prefrontal cortex and posterior cingulate cortex — form an impulsive neural system that exclusively or primarily values immediate rewards. Drawing an analogy to a valuable economic model³², they suggested that hyperbolic-like discounting resulted from the action of this impulsive (β) system working with a more patient (δ) system, based in the lateral prefrontal cortex and posterior parietal cortex. This conclusion was based principally on the finding that these three areas showed greater activity for choices that involved an immediate reward than for choices that involved only delayed rewards. However, this empirical finding, which our current results do not challenge, is also compatible with the hypothesis that activity in these regions represents the subjective value of rewards at all delays, as the subjective value of immediate rewards is greater than that of delayed rewards. Our observation that activity in these regions varies when only the delayed reward changes falsifies the hypothesis that these regions exclusively value immediate rewards. Furthermore, our finding that neural activity in these regions tracks changes in the subjective value of the delayed reward (determined by subject-specific behavioral discount functions) shows that these regions do not even primarily value immediate rewards, as the value implied by neural activity is not more impulsive than the person’s behavior, as the β–δ hypothesis requires.

The fact that the ventral striatum, medial prefrontal cortex and posterior cingulate cortex encode the subjective value of delayed monetary rewards, whether those rewards are selected by pressing a button or by taking no action, indicates that these regions are involved in the valuation of reward outcomes rather than in movement control. If this is correct, then these areas might provide input to structures that encode the subjective value of particular actions, such as the movement-related areas of the posterior parietal cortex⁵^,¹⁹^,⁴⁵^,⁴⁶. Consistent with this notion, activity in the parietal cortex was correlated with subjective value in one analysis (Supplementary Table 2). Future studies should more closely examine the interaction between outcome (or goods-based²³) and action values, as well as the process by which value signals affect choice. The three regions we have focused on might carry out different roles in this process, and such investigations might also shed light on the function of other regions (such as the temporal-parietal junction and insula) that were identified in some of our analyses.

A few issues suggest some caution when interpreting our results. First, unlike ref. ³⁵, we did not include an experimental condition that involved choices between two delayed rewards. Future studies should include such choices to verify that these regions do not encode subjective value in a categorically different manner when an immediate reward is not available. By varying both rewards, such studies could also address how activity in these regions reflects the combination of subjective values from multiple rewards presented simultaneously. In this experiment, as one option was always constant, areas that implemented different combination rules (addition, subtraction, division) or that encoded the subjective value of the chosen option would all show similar patterns of activity in our analyses. In addition, we only scanned individuals with stable discount functions. It remains unknown what activity in these regions looks like for individuals who show a different kind of choice behavior — for example, those who follow a simple heuristic or cut-off rule.

In the current study we adapted the method of psychometric-neurometric comparisons, which has been widely used in perceptual neurophysiology, for use with fMRI. Although this approach is related to previous model-based fMRI studies²²^,²⁴^,²⁵, few studies have directly estimated neurometric functions from fMRI data. Our results indicate that such psychometric-neurometric comparisons should prove fruitful in studying the neural mechanisms of decision-making, and hold promise for contributing to both neuroscience and economics. For neuroscience, this method can link the function of different neural systems with the hidden processes that are posited in quantitative models of decision-making. For economics, this method provides preliminary evidence that the physical mechanism that underlies inter-temporal choice is strikingly similar to the most general class of revealed preference models, resting on a continuous underlying scale for subjective value. Although this similarity does not prove or disprove theoretical models that make no predictions about the physical mechanisms of choice, it does underline the conviction of many that theoretical models resting on a demonstrable physiological foundation will probably prove more robust than those that lack such a foundation.

METHODS

Subjects

Twelve paid volunteers participated in the experiment (7 women, 5 men, all right-handed, mean age = 21.7 years). Ten (6 women, 4 men, mean age = 21.2 years) participated in both behavioral and scanning sessions. The remaining two participated only in behavioral sessions, as their intertemporal preferences were not stable. One of these subject’s choices reversed more than once as the delayed amount increased, precluding estimation of an indifference point; the other had significantly different discount rates in their last two behavioral sessions. All participants gave informed consent in accordance with the procedures of the University Committee on Activities Involving Human Subjects of New York University.

Task

On each trial, subjects chose between a smaller amount of money paid immediately and a larger amount paid at a later time (Fig. 1). The smaller immediate amount was $20 on all trials. To simplify the display, only the variable delayed option was presented on each trial. This option was constructed using one of six delays (6 h–180 d) and one of six amounts ($20.25–$110). Thus there were 36 unique choices in each session. Each of these was presented four times, for a total of 144 trials. Delays were the same for every subject, but changed across sessions; amounts were chosen individually for each subject on the basis of their behavior in previous sessions (and thus also varied across sessions) to ensure that an approximately equal number of immediate and delayed options were chosen. Subjects were not told how options would change across sessions, and we did not observe any systematic shifts in subject’s choices in response to these changes (Supplementary Fig. 1). Subjects indicated their choices by releasing a button. For half of each session, subjects released the button to select the fixed immediate reward, and for the other half subjects released the button to select the variable delayed reward. A button release was used, rather than a button press, to keep subjects alert on trials in which their choice did not require a response. All subjects participated in three 1-h behavioral sessions, and ten participated in one or two 2-h scanning sessions. Sessions were completed over 1–6 months, with consecutive sessions separated by at least 3 d. As people’s decisions involving real gains might differ from those involving hypothetical gains⁴⁷^–⁴⁹, subjects were paid according to four randomly selected trials per session (except for the first behavioral session, which involved only hypothetical choices), using commercially available pre-paid debit cards (see Supplementary Methods online for details).

Imaging

Imaging data were collected with a Siemens Allegra 3T head-only scanner equipped with a head coil from Nova Medical. T2*-weighted functional images were collected using an EPI sequence (TR = 2 s, TE = 30 ms, 35 axial slices acquired in ascending interleaved order, 3 × 3 × 3 mm, 64 × 64 matrix in a 192-mm field of view). Each scan consisted of 236 images. The first two images were discarded to avoid T1 saturation effects. There were eighteen choice trials during each scan; each trial lasted 14 s with a 12-s inter-trial interval (Fig. 1). Each subject completed 8–16 scans over 1–2 sessions, with most subjects (n = 7) completing eight scans in one session. High-resolution, T1-weighted anatomical images were also collected using an MPRAGE sequence (TR = 2.5 s, TE = 3.93 ms, TI = 900 ms, flip angle = 8°, 144 sagittal slices, 1 × 1 × 1 mm, 256 × 256 matrix in a 256-mm field of view).

Data analysis

Functional imaging data were analyzed using BrainVoyager QX (Brain Innovation), with additional analyses performed in MATLAB (Math-Works). Functional images were sinc-interpolated in time to adjust for staggered slice acquisition, corrected for any head movement by realigning all volumes to the first volume of the scanning session using six-parameter rigid-body transformations, and de-trended and high-pass filtered (cutoff of 3 cycles per scan, or 0.0064 Hz) to remove low-frequency drift in the fMRI signal. Images were then co-registered with each subject’s high-resolution anatomical scan, rotated into the AC-PC plane, and normalized into Talairach space using piecewise affine Talairach grid scaling. All spatial transformations of the functional data used trilinear interpolation. For group-level random effects analyses only, data were also spatially smoothed with a gaussian kernel of 8 mm (full-width half-maximum).

Single-subject analyses were performed using multiple linear regression, estimated with ordinary least-squares. As we did not want to assume a priori where in a trial subjective value might have effects, we used a variant of a ‘finite-impulse response’ or ‘deconvolution’ model. This model included twelve covariates that fit the mean activity, across all trials, for each of the first twelve time points in a trial, and twelve covariates that fit the deviations from the mean at each time point that were correlated with the subjective value of the delayed reward across trials. Additional models (Supplementary Tables 3–11) replaced the subjective value terms with an alternative variable (see Supplementary Methods).

We performed group random-effects analyses using the summary statistics approach, which tests whether the mean effect at each voxel is significantly different from zero across subjects. On the basis of previous reports²²^,³⁵, we hypothesized a priori that the ventral striatum, medial prefrontal cortex and posterior cingulate cortex would contribute to valuation during intertemporal choice. Accordingly, we considered any activation in these regions significant if it exceeded P < 0.001 (uncorrected), spatial extent > 100 mm³. For display purposes only, contrast maps were interpolated to 1 × 1 × 1 mm, and thresholded at P < 0.005 (uncorrected), spatial extent > 100 mm³.

We defined anatomical ROIs in individual subjects in the medial prefrontal cortex, posterior cingulate cortex and ventral striatum. Voxels were selected within each ROI according to their functional responsiveness in single-subject analyses. For the subjective value ROIs, all voxels were selected that exhibited a subjective value effect at time points 4–6 of P < 0.05 (uncorrected). For the unbiased value ROIs, all voxels were selected that exhibited greater activity (time points 4–6, P < 0.05, uncorrected) for either the largest amount compared to the smallest amount of the delayed reward across all delays, or the shortest delay compared to the longest delay to the delayed reward across all amounts. ROIs where no voxels met these functional criteria (n = 3/60) were omitted from further analyses.

Activity was averaged across all selected voxels within an ROI and summed over time points 4–6 for each trial. The resulting data were regressed separately against the objective amount of the larger delayed reward or the imposed delay to the larger delayed reward, and also fit with a nonlinear version of the subjective value regression to estimate the neural discount rate. For the regression analyses in Figure 6, both the ratio of the delay and amount regression coefficients and the neural discount rates were log transformed because of their skewed distribution. Because the log transformation only permits positive values, any negative values were replaced with the smallest observed positive value. All negative values were close to zero, so constraining values to be positive in this manner only reduced the range of ratios by less than 2.5% and the range of neural discount rates by less than 1%. ROIs for which the amount regression coefficient was negative (n = 2/57) or where no discount rate accounted for a significant amount of the variance in neural activity (n = 5/57) were omitted from further analyses. Fits to the ROI data and the regression analyses in Figure 6 both used robust regression to minimize the effect of outliers. Robust regression was performed in MATLAB, which uses an iteratively re-weighted least-squares algorithm with the weights at each iteration determined by applying a bi-square function to the residuals from the previous iteration. Greater detail regarding the statistical analyses of the fMRI data are provided in the Supplementary Methods.

Supplementary Material

Supplementary Data

Click here to view.^{(694K, pdf)}

Acknowledgments

We thank A. Bisin, A. Caplin, M. Grantner, D. Heeger and I. Levy for their comments on earlier versions of this paper. This work was supported by grants from the National Institutes of Health (F32-MH75544 to J.W.K. and R01-NS054775 to P.W.G.), the McDonnell Foundation (to P.W.G.) and the Seaver Foundation.

Footnotes

Note: Supplementary information is available on the Nature Neuroscience website.

Published online at http://www.nature.com/natureneuroscience

Reprints and permissions information is available online at http://npg.nature.com/reprintsandpermissions

References

1. Weber EH. De Pulsu, Resorptione, Audita et Tactu. Annotationes Anatomicae et Physiologicae. CF Koehler; Lipsiae: 1834.

2. Fechner GT. Elemente der Psychophysik. Breitkopf & Hartel; Leipzig: 1860.

3. Stevens S. Neural events and the psychophysical law. Science. 1970;170:1043–1050. [PubMed]

4. Bernoulli D. The Works. Birkhauser; Boston: 1982.

5. Platt ML, Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature. 1999;400:233–238. [PubMed]

6. Abler B, Walter H, Erk S, Kammerer H, Spitzer M. Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage. 2006;31:790–795. [PubMed]

7. Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol. 2003;89:2823–2838. [PubMed]

8. Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol. 2000;84:3072–3077. [PubMed]

9. McCoy AN, Crowley JC, Haghighian G, Dean HL, Platt ML. Saccade reward signals in posterior cingulate cortex. Neuron. 2003;40:1031–1040. [PubMed]

10. O’Doherty J, Critchley H, Deichmann R, Dolan RJ. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci. 2003;23:7931–7939. [PubMed]

11. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron. 2001;30:619–639. [PubMed]

12. Nieuwenhuis S, et al. Activity in human reward-sensitive brain areas is strongly context dependent. Neuroimage. 2005;25:1302–1309. [PubMed]

13. Rogers RD, et al. Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biol Psychiatry. 2004;55:594–602. [PubMed]

14. Kuhnen CM, Knutson B. The neural basis of financial risk taking. Neuron. 2005;47:763–770. [PubMed]

15. Knutson B, Taylor J, Kaufman M, Peterson R, Glover G. Distributed neural representation of expected value. J Neurosci. 2005;25:4806–4812. [PubMed]

16. Bowman EM, Aigner TG, Richmond BJ. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol. 1996;75:1061–1073. [PubMed]

17. Cromwell HC, Hassani OK, Schultz W. Relative reward processing in primate striatum. Exp Brain Res. 2005;162:520–525. [PubMed]

18. McCoy AN, Platt ML. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci. 2005;8:1220–1227. [PubMed]

19. Dorris MC, Glimcher PW. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron. 2004;44:365–378. [PubMed]

20. McClure SM, et al. Neural correlates of behavioral preference for culturally familiar drinks. Neuron. 2004;44:379–387. [PubMed]

21. Gottfried JA, O’Doherty J, Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 2003;301:1104–1107. [PubMed]

22. Tanaka SC, et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci. 2004;7:887–893. [PubMed]

23. Padoa-Schioppa C, Assad JA. Neurons in the orbitofrontal cortex encode economic value. Nature. 2006;441:223–226. [PMC free article] [PubMed]

24. Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ. Cortical substrates for exploratory decisions in humans. Nature. 2006;441:876–879. [PMC free article] [PubMed]

25. O’Doherty JP, Buchanan TW, Seymour B, Dolan RJ. Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron. 2006;49:157–166. [PubMed]

26. Johnson KO, Hsiao SS, Yoshioka T. Neural coding and the basic law of psychophysics. Neuroscientist. 2002;8:111–121. [PMC free article] [PubMed]

27. Parker AJ, Newsome WT. Sense and the single neuron: probing the physiology of perception. Annu Rev Neurosci. 1998;21:227–277. [PubMed]

28. Werner G, Mountcastle VB. Quantitative relations between mechanical stimuli to the skin and neural responses evoked by them. In: Kenshalo DR, editor. The Skin Senses. Charles C Thomas Publisher; Springfield, Illinois: 1968. pp. 112–138.

29. Newsome WT, Britten KH, Movshon JA. Neuronal correlates of a perceptual decision. Nature. 1989;341:52–54. [PubMed]

30. Frederick S, Loewenstein G, O’Donoghue T. Time discounting and time preference: a critical review. J Econ Lit. 2002;40:351–401.

31. Mazur JE. An adjusting procedure for studying delayed reinforcement. In: Commons ML, Mazur JE, Nevin JA, Rachlin H, editors. Quantitative Analysis of Behavior: The Effects of Delay and Intervening Events on Reinforcement Value. Lawrence Erlbaum Associates, Publishers; Hillsdale, New Jersey: 1987. pp. 55–73.

32. Laibson D. Golden eggs and hyperbolic discounting. Q J Econ. 1997;12:443–477.

33. Green L, Myerson J. A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull. 2004;130:769–792. [PMC free article] [PubMed]

34. Benhabib J, Bisin A, Schotter A. Working Paper. New York University Department of Economics; New York: 2006. Present-bias, quasi-hyperbolic discounting and fixed costs. < http://www.econ.nyu.edu/user/benhabib/pshype0106.pdf>.

35. McClure SM, Laibson DI, Loewenstein G, Cohen JD. Separate neural systems value immediate and delayed monetary rewards. Science. 2004;306:503–507. [PubMed]

36. Teller DY, Pugh EN. Linking propositions in color vision. In: Mollon JD, Sharpe LT, editors. Colour Vision: Physiology and Psychophysics. Academic Press; New York: 1983. pp. 577–589.

37. von Neumann J, Morgenstern O. The Theory of Games and Economic Behavior. Princeton University Press; Princeton, New Jersey: 1944.

38. Samuelson PA. Foundations of Economic Analysis. Harvard University Press; Cambridge, Massachusetts: 1947.

39. Gul F, Pesendorfer W. Working Paper. Princeton University Department of Economics; Princeton, New Jersey: 2005. The case for mindless economics. < http://www.princeton.edu/~pesendor/mindless.pdf>.

40. Gusnard DA, Raichle ME. Searching for a baseline: functional imaging and the resting human brain. Nat Rev Neurosci. 2001;2:685–694. [PubMed]

41. Zysset S, Huber O, Ferstl E, von Cramon DY. The anterior frontomedian cortex and evaluative judgment: an fMRI study. Neuroimage. 2002;15:983–991. [PubMed]

42. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. Neural systems responding to degrees of uncertainty in human decision-making. Science. 2005;310:1680–1683. [PubMed]

43. Paulus MP, Frank LR. Anterior cingulate activity modulates nonlinear decision weight function of uncertain prospects. Neuroimage. 2006;30:668–677. [PubMed]

44. Tom SM, Fox CR, Trepel C, Poldrack RA. The neural basis of loss aversion in decision-making under risk. Science. 2007;315:515–518. [PubMed]

45. Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science. 2004;304:1782–1787. [PubMed]

46. Shadlen MN, Newsome WT. Motion perception: seeing and deciding. Proc Natl Acad Sci USA. 1996;93:628–633. [PubMed]

47. Collier M, Williams MB. Eliciting individual discount rates. Exp Econ. 1999;2:107–127.

48. Camerer CF, Hogarth RM. The effects of financial incentives in experiments: a review and capital-labor-production framework. J Risk Uncertain. 1999;19:7–42.

49. Smith VL, Walker JM. Monetary rewards and decision cost in experimental economics. Econ Inq. 1993;31:245–261.