On each trial, subjects chose between a fixed immediate reward of $20 and a larger delayed reward that varied randomly from trial to trial (). The value of the larger reward varied from $20.25 to $110, and the delay varied from 6 h to 180 d. Before fMRI, subjects completed three preliminary behavioral sessions. On the basis of the choices in these behavioral sessions, we estimated a discount function for each subject (). Note that interpreting as pure discount functions relies on the assumption that subjective value increases linearly with the objective amount of money. Violations of this assumption would not alter the fact that these curves reflect subjective value, but would mean that they are better described as discounted utility functions as they incorporate effects of amount as well as delay. These individual preference curves, which express how a subjective quantity (subjective value) changes as a function of an objective variable (delay and monetary amount), are directly analogous to sensory psychophysical functions2,3
. Consistent with previous findings in a range of species30–34
, the discount curves for all subjects were well–characterized by a hyperbolic function (median R2
from scanning sessions = 0.95, range = 0.84–0.98):
is subjective value (expressed here as a fraction of immediate value), D
is delay (in d) and k
is a subject-specific constant. The best fitting k
parameter (the discount rate) varied widely across subjects. For our most patient subject (k
= 0.0005, ), an immediate gain of $20 was less preferable than a gain of $21 in a month, while for our most impulsive subject (k
= 0.1189, ), an immediate gain of $20 was more preferable than a gain of $68 in a month. Ten subjects (out of twelve) showed stable discount rates across sessions, even though sessions occurred over a 1–6-month time span (Supplementary Fig. 1
online). Only these ten subjects, with relatively stable functions over the tested range, were included in the fMRI experiment.
Figure 1 Intertemporal choice task. The sequence of events within a trial is shown. On each trial, subjects chose between an immediate and a delayed reward. The immediate reward was the same ($20) on every trial and was never presented visually. A red dot signaled (more ...)
Figure 2 Subject-specific discount functions. (a–c) Choice data from three subjects during a single scanning session. Points are shaded according to the imposed delay to the delayed reward, and denote the fraction of times the subject chose the delayed (more ...)
For comparison, we also fit each subject’s discount curve with a single exponential function:
as well as a sum of two exponential functions:
is subjective value and D
is delay as in equation (1)
, and c
are subject-specific constants. Consistent with the literature, the hyperbolic function equation (1)
described the discount curves better than the single parameter exponential function equation (2)
(Supplementary Fig. 2
online). The sum of two exponentials is one formulation of an economic model that explains hyperbolic-like discounting by the combined action of two systems32,35
: one (β
) that discounts more steeply than the person’s resulting behavior, and a second (δ
) that discounts less steeply than the person’s resulting behavior. Not surprisingly, as this functional form can account for hyperbolic-like discounting, the sum of exponentials function fit the data better than the single parameter exponential, and as well as the single parameter hyperbolic function (Supplementary Fig. 2
). Although other forms of the discount function have been proposed32–34
, for simplicity, we considered only these three widely used forms.
The ten subjects with stable discount functions completed the same choice task as was used in the preliminary behavioral sessions during fMRI. The behaviorally derived preference curves, which estimated how subjective value declined with delay for each individual, were used to find neural activity that correlated with subjective value. The subjective value of the delayed reward on each trial was estimated by multiplying the objective amount of the reward by the discount fraction (estimated behaviorally) for that delay. This inferred subjective value was then used as a regressor for brain activity. As the immediate reward was constant, this analysis finds areas that track the sum of the subjective values of both rewards, the difference in subjective value of the two rewards, or the ratio of subjective values of the two rewards.
Early in the trial, there was a significant correlation between neural activity and subject-specific subjective value (random effects group analysis, voxel-wise P
< 0.001, spatial extent > 100 mm3
) in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex (, Supplementary Table 1
online). The timing of this effect (6–10 s into the trial) indicated that it arose from neuronal activity during the visual presentation of the delayed reward option, which we confirmed in a second model that assumed a hemodynamic response function (Supplementary Table 2
online). We did not find any significant effects of subjective value during other portions of the trial.
Figure 3 Group analysis showing areas in which activity is correlated with subjective value. (a) Areas in which neural activity was correlated with subjective value (during the 6–10-s window) in a random-effects group analysis, overlayed on the mean normalized (more ...)
A region in which neural activity is correlated with the subjective value of a delayed reward should show increased activity when the objective amount of this reward increases, decreased activity when the delay to this reward increases, and increased activity when the delayed reward is chosen because it is more valuable. This is exactly what we observed in these three regions (, Supplementary Fig. 3
online and Supplementary Tables 3–5
online). Subjective value accounted for activity in these regions better than did the objective reward characteristics or the subjects’ choices: both the strength (peak z
-score) and spatial extent (cluster size in mm3
) of the correlation with subjective value were larger than those of these other variables ( and Supplementary Tables 1–5
). Of these other variables, only delay-to-reward reached significance, and only in the medial prefrontal cortex. Furthermore, taking all voxels identified by any of these four variables (subjective value, monetary amount, delay and choice), subjective value showed a stronger effect in pair-wise comparisons with these other variables in 100% of voxels in the ventral striatum, at least 85% of voxels in the posterior cingulate cortex, and at least 62% of voxels in the medial prefrontal cortex. Idiosyncratic subjective value also accounted for activity in these regions better than did assuming a single fixed discount rate for all subjects ( and Supplementary Table 6
online). The effect of value, assuming a single fixed discount rate, only reached significance in the medial prefrontal cortex, and the strength and spatial extent of the correlation with subject-specific subjective value was larger in both the ventral striatum and the posterior cingulate cortex. Finally, these effects of subjective value cannot be explained by choice difficulty or corresponding attentional demands. Using two different indices of choice difficulty, we found no significant effects in the ventral striatum, medial prefrontal cortex or posterior cingulate cortex (Supplementary Tables 7 and 8
We also found significant effects of subjective value in single-subject analyses, for subjects across the entire spectrum of discount rates. Despite marked differences in the discount functions of the five example subjects shown in , ranging from patient (k
= 0.0042) to impulsive (k
= 0.1189), activity in parts of the ventral striatum, medial prefrontal cortex and posterior cingulate cortex were correlated with subjective value in each subject (). These same regions thus exhibited a different pattern of activity across subjects, with each subject’s idiosyncratic pattern of brain activity being predicted by that subject’s idiosyncratic preferences (, Supplementary Fig. 4
online). As a function of delay, both the subjective value of delayed gains and the neural activity associated with those gains decreased in a similar manner, whether subjects were more impulsive and showed steeper decreases in both domains () or were less impulsive and showed more gradual decreases in both domains ().
Figure 4 Single-subject analyses showing areas in which activity correlated with subjective value. Data from five subjects are shown to illustrate that subjective value effects were evident for subjects spanning the entire range of behavioral discount rates. ( (more ...)
Figure 5 Single-subject time courses and neural discount functions. (a–c) Data from three subjects (HM, see ; RA, see ; and CH, see ) are shown. Data were averaged over all voxels that showed a correlation between activity and (more ...)
To quantify the degree to which these regions show a different pattern of activity across subjects, we averaged activity from all voxels that showed a subjective value effect in each of the three regions in each subject, and then regressed neural activity on each trial (summed over the 6–10-s window) separately against the imposed delay and objective amount associated with the variable delayed reward. If neural activity tracks subjective value across subjects, then the ratio of these two regression coefficients (delay over amount, reversing the sign on the negative delay coefficient) should be high for steep discounters and low for shallow discounters, as delay has a stronger effect on subjective value for more impulsive discounters. Consistent with this notion, this ratio increased as the subject’s discount rate increased (; slopeVS ± standard error = 0.77 ± 0.19, t-test, P = 0.004; slopeMPFC = 0.90 ± 0.21, P = 0.003; slopePCC = 0.99 ± 0.31, P = 0.02; slopeALL = 0.87 ± 0.12, P < 10−7; VS, ventral striatum; MPFC, medial prefrontal cortex; PCC, posterior cingulate cortex).
Figure 6 Psychometric-neurometric comparisons. (a,d) A measure of the neural effect of delay is plotted against the subject’s behavioral discount rate for both (a) ROIs defined on the basis of the subjective value regression and (d) value ROIs defined (more ...)
The preceding analysis demonstrates the correspondence between neural activity and discount rate across subjects without making strong assumptions about the functional form of the discount function. For a more precise test, we assumed that the neural discount function has a hyperbolic form and estimated the neural discount rate for these areas, by refitting the regression model to the averaged time courses from the correlated regions while allowing the discount rate parameter (k) to vary. A psychometric-neurometric match requires both that the neural discount rate increases with the subject’s behavioral discount rate and that there is no difference between the two on average. By contrast, if activity in these regions were best explained by the objective amount (delay) of the delayed reward, then the neural discount rate should be consistently smaller (greater) than the behavioral discount rate. If activity were best explained by a single fixed discount rate across all subjects, then the neural discount rate should show no correlation with the behavioral discount rate. Consistent with a psychometric-neurometric match, the neural discount rate in these three regions increased with the subject’s behavioral discount rate (, slopeVS ± standard error = 0.58 ± 0.23, t-test, P = 0.04; slopeMPFC = 0.75 ± 0.28, P = 0.03; slopePCC = 0.80 ± 0.37, P = 0.06; slopeALL = 0.69 ± 0.16, P = 0.0002), and the difference between the neural discount rate and the subject’s behavioral discount rate was centered on zero (; medianVS = −0.0035, Wilcoxon signed rank test versus zero, P = 0.30; medianMPFC = −0.0008, P = 1; medianPCC = −0.0006, P = 1; medianALL = −0.0009, P = 0.67).
As the previous analyses were conducted in regions-of-interest (ROIs) that were defined by the subjective value regression, they merely confirm that the regions identified exhibit a psychometric-neurometric match. It would be even more compelling if such a match existed in value-sensitive voxels in these regions that were selected in an unbiased manner. We defined value-sensitive ROIs in these areas in a manner that was unbiased with respect to the neural discount rate (see Methods). In these ROIs, the negative effect of delay, relative to the positive effect of amount, again increased with the subject’s behavioral discount rate (; slopeVS ± standard error = 0.58 ± 0.36, t-test, P = 0.15; slopeMPFC = 0.86 ± 0.36, P = 0.046; slopePCC = 0.90 ± 0.46, P = 0.10; slopeALL = 0.78 ± 0.21, P = 0.0009). In addition, consistent with a psychometric-neurometric match, the neural discount rate in these ROIs increased with the subject’s behavioral discount rate (; slopeVS ± standard error = 0.51 ± 0.48, t-test, P = 0.32; slopeMPFC = 0.73 ± 0.67, P = 0.31; slopePCC = 0.89 ± 0.66, P = 0.23; slopeALL = 0.70 ± 0.31, P = 0.03), and the difference between the neural and behavioral discount rates was centered on zero (; medianVS = −0.0039, Wilcoxon signed rank test versus zero, P = 0.50; medianMPFC = −0.0008, P = 0.95; medianPCC = −0.0006, P = 0.94; medianALL = −0.0009, P = 0.93; see for a single-subject example).
Figure 7 Single-subject example of unbiased value ROIs and resulting neural discount functions. (a) ROIs in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex are shown for one subject (HM, same subject shown in and (more ...)
Finally, to evaluate an alternative hypothesis about the role of these regions in intertemporal choice32,35
, we repeated the above analyses using the β
(steep exponential) and δ
(shallow exponential) value functions from the sum of exponentials discount function fit to each subject’s behavior. Perhaps not surprisingly, as the β
value functions are correlated to some degree with subjective value, the brain regions in which activity is correlated with these three value functions are largely overlapping (Supplementary Fig. 5
online, Supplementary Tables 9–11
online). A psychometric-neurometric comparison, though, provides a more precise test of which value function best accounts for neural activity. As β
are exponential functions, we estimated the neural discount rates using a single exponential (rather than hyperbolic) function, and compared this rate to the behavioral discount rate estimated using a single exponential and the exponential β
discount rates estimated behaviorally. In both sets of ROIs, the difference between the neural discount rate and the behavioral discount rate was again centered on zero (; subjective value ROIs, median = −0.0006, Wilcoxon signed rank test versus zero, P
= 0.91; unbiased value ROIs, median = −0.0006, P
= 0.77). By contrast, the difference between the neural discount rate and β
is centered to the left of zero, indicating that the theoretically defined β
term discounts more steeply than neural activity in these areas (; subjective value ROIs, median = −0.0081, P
= 0.001; unbiased value ROIs, median = −0.0088, P
= 0.23), while the difference between the neural discount rate and δ
is centered to the right of zero, indicating that the theoretically defined δ
term discounts less steeply than neural activity (, subjective value ROIs, median = 0.0024, P
< 0.0001; unbiased value ROIs, median = 0.0016, P
= 0.003). Thus, neural activity in the ventral striatum, medial prefrontal cortex and posterior cingulate cortex tracks the subjective value of rewards as determined from behavior, rather than tracking a theoretically defined component of value that is more impulsive (β
) or more patient (δ
) than the person’s behavior.
Figure 8 Neural activity tracks subjective value, and not a more impulsive (β) or more patient (δ) estimate of value. (a,b) Difference between the neural (single exponential) discount rate and the behavioral (single exponential) discount rate, (more ...)