In summary, our data provide both direct behavioral and neurobiological support for marginal utility theory in the context of a choice model that incorporates temporal discounting. Furthermore, our results suggest that the dorsal striatum may act as a site of convergence of these two systems – so as to construct the discounted utility that plays an important role in subsequent choice.
The striatum has been identified in previous studies of temporal discounting in both animals (
Cardinal et al., 2001;
Cardinal et al., 2004;
Roesch et al.,, 2007;
Kobayashi and Schultz, 2008), and humans (
McClure et al., 2004;
Tanaka et al., 2004;
Tanaka et al., 2007;
Kable and Glimcher 2007;
Wittmann et al., 2008;
Ballard and Knutson, 2008;
Luhmann et al., 2008) and less directly, in marginal utility (
Tobler et al., 2007). In humans, activity has been shown to correlate with preferences for immediate options (
McClure et al., 2004), and for discounted magnitude across both immediate and delayed options over both short (
Tanaka et al., 2004) and long timescales (
Kable and Glimcher, 2007). However, the exact nature of this signal has been unclear including whether it merely reports on value calculations, or their prediction errors, performed elsewhere (Tanaka et al.,
2004,
2007;
Luhmann et al., 2008). For instance, the well-recognised role of this region in reinforcement learning (
Robbins et al., 1989;
O’Doherty et al., 2004,
Seymour et al., 2004;
Haruno and Kawato, 2006) does not necessarily speak to a role in constructing value and choice. However, previous data are consistent with distinct roles in encoding delay (
Tanaka et al., 2007) and marginal utility (
Tobler et al., 2007). The data presented here advance these insights and support a broader and more sophisticated role for this region than previously thought, wherein choices are determined by an integration of distinct determinants of value.
The exact nature of the representation of temporal discount remains unclear (
Wittmann and Paulus, 2008). Superficially, the diminished utility associated with increasing time has strong parallels to probability discounting, and indeed some theoretical accounts of temporal discounting propose just this: that uncertainty, for instance through unexpected occurrences that might interfere with reward delivery, accumulates with time (
Stevenson, 1986). However, recent neurophysiological evidence suggests that uncertainty and temporal discount factors may be, at least in part, distinct (Luhman et al., 2008). Furthermore, that the BOLD activity correlates with a single parametric regressor does not in itself imply that it is driven by a single neural determinant, since distinct psychological processes (such as the utility of anticipation or anxiety, Lowenstein, 1987;
Wu, 1999) and neurochemical processes (such as 5HT and dopamine;
Tanaka 2007,
Roesch et al., 2007) may make independent contributions.
From a behavioral and economic perspective, neglecting non-linear utility has the potential to confound inferences about discounting since any model could over-estimate the discount rate to account for marginal utility effects (e.g.
Andersen et al., 2008). A similar argument could apply to the neurophysiological data. This has particular relevance for understanding personality characteristics such as impulsivity. The term impulsive is a general description of a diverse group of behaviours with distinct features (likely dependent on distinct neural processes), which are encompassed by a general theme of behavior in the absence of adequate foreseight (e.g.,
Evenden, 1999;
Winstanley et al., 2006). These include motor/behavioural impulsiveness, the inability to withhold a prepotent behavioural response, and reflection impulsiveness, a failure to slow down (or ‘hold your horses’ (Frank et al., 2008)) in response to decision-conflict, to properly consider options. Another feature, choice/temporal impulsiveness, is often defined as the propensity to choose small short-term gains in preference to larger delayed gains (or larger delayed losses in preference to smaller immediate losses) (e.g. (
Herrnstein, 1981;
Mazur, 1987;
Logue, 1988;
Evenden, 1999;
Ainslie, 2001;
Cardinal et al., 2003;
Green and Myerson, 2004). Traditionally, the psychological basis of impulsive choice has rested on the discount rate parameter, such that those with a higher rate are described as impulsive and those with a low rate as self-controlled (
Herrnstein, 1981;
Mazur, 1987;
Logue, 1988;
Evenden, 1999;
Ainslie, 2001;
Cardinal et al., 2003;
Green and Myerson, 2004). However, the data presented here illustrate that impulsivity and self-control are also determined by the concavity of an individual’s utility function and that these two parameters are independent of one another. Specifically, the more concave the function, the faster marginal utility diminishes and the more impulsive the individual. This is because a concave utility function diminishes the value of the larger reward relative to the smaller reward, making it less attractive. A corollary of this is that subjects who are more impulsive (as a result of a more concave utility function) may be more risk-averse, since the concavity of the utility function is also a key determinant of choice under uncertainty (
von Neumann and Morgenstern, 1947;
Kahneman and Tversky, 1979;
Pindyck and Rubinfeld, 2004).
We conclude that impulsivity in choice (temporal impulsiveness) should not solely be defined by
K. Moreover,
K and
r in our view should be kept separate as there is no theoretical reason why the discounting of time and scaling of magnitude (two different features of preferences) should influence each other. Although it has been suggested that such a correlation may exist (
Anderhub et al., 2001), we did not observe it in our results, and previous attempts to find a correlation by simultaneously administering risk preference (to estimate
r) and intertemporal choice (to estimate
K) tasks have been mixed (
Ahlbrecht and Weber, 1997;
Anderhub et al., 2001). In our view it is perfectly possible that a person with a high discount rate but a close to linear utility function is as behaviorally impulsive as a counterpart with a low discount rate but a more concave function – although both parameters will correlate with impulsiveness, individually. Future studies of impulsive choice should therefore consider these determinants and other – including top-down, inhibitory control mechanisms – when hypothesizing about the underlying cause of a change in intertemporal choice behavior across experimental conditions. These considerations have an important bearing on studies of psychopathologies where impulsive choice is a quintessential clinical feature, such as drug addiction (
Cardinal et al., 2003;
Bickel et al., 2007) and attention-deficit hyperactivity disorder (ADHD) (
Sagvolden and Segeant, 1998;
Winstanley et al., 2006), particularly since dysfunction of the striatum is implicated in both conditions.
One caveat relating to the fMRI data is that when comparing different valuation models we found that the hyperbolic discounting of utility, was the best model for describing the behavioral data. However, due to constraints in the design of the study we were not able to use the fMRI data to make such inferences regarding the accuracy of the different models. The regressors used to analyse the imaging data were created only from the model we proposed, which was also selected by the AIC analysis; and so we caution that these results may not be independent of the model used (e.g. exponential vs. hyperbolic). We anticipate further studies which aim to assess the validity of these models using fMRI data.
One of the useful aspects of the model is the ability to calculate utility functions from intertemporal choices. Previous methods to construct utility functions have mostly used risk preference tasks such as simple gambles. These studies suggest that the average utility function derived from risk-preference tasks is magnitude to the power of 0.88 (
Tversky and Kahneman, 1992). This value leads to a slightly more concave utility function than that observed in our task. This discrepancy may have arisen from natural variance of the population or the range of magnitudes used in the task to characterise the function (1-100 pound in our study vs. a larger hypothetical or smaller real range of amounts, used in other studies). It is also likely that the realistic nature of our study (real amounts paid with real delays) may lead to differences from previous estimates, where, for the most part, hypothetical choices were made. Finally, further studies should address whether utility estimates derived from intertemporal choices differ from those derived from gambles.
Finally, our results bear relevance to a related, but distinct personality trait – that of decisiveness. When people have to make choices between similarly valued options, decision-conflict can occur. Decision-conflict often leads to a slowing down of responses and increase in activity of conflict areas such as the ACC (e.g.
Botvinick et al., 2004;
Cohen et al., 2005; Kennerly et al., 2006;
Botvinick, 2007;
Pochon et al., 2008). Whilst this phenomenon is relatively well studied in lower level, perceptual and motor decision-making tasks, it is less well characterized in higher level tasks (
Pochon et al., 2008). We show that decision-conflict occurs in intertemporal choice, and that it can be caused by choosing between similarly valued options but also options that are far apart in time (independent of difference in value). Furthermore, we demonstrated that ACC activity in response to conflict correlated with the degree to which individual subjects were slowed down by choice difficulty. This suggests that the psychological trait of decisiveness may be predicted by or relate to an individual’s degree of ACC activity.
The publisher's final edited version of this article is available free at 
