|Home | About | Journals | Submit | Contact Us | Français|
While making economic choices, individuals assign subjective values to the available options. Values computed in different behavioral conditions, however, can vary substantially. The same person might choose some times between goods worth a few dollars, and other times between goods worth thousands of dollars, or more. How does the brain system that computes values—the “valuation system”—handle this large variability? Here we show that the representation of value in the orbitofrontal cortex (OFC)—an area implicated in value assignment during economic choice—adapts to the behavioral condition of choice and, more specifically, to the range of values available in any given condition. In the experiments, monkeys chose between different juices and their choice patterns provided a measure of subjective value. Value ranges were varied from session to session and, in each session, OFC neurons encoded values in a linear way. Across the population, the neuronal sensitivity (defined as the change in neuronal activity elicited by the increase in one value unit) was inversely proportional to the value range. Conversely, the neuronal activity range did not depend on the value range. This phenomenon of range adaptation complements that of menu invariance observed in a previous study. Indeed, the activity of each neuron adapts to the range values it encodes, but does not depend on other available goods. Our results thus suggest that the representation of value in the OFC is at one time instantiative of preference transitivity (menu invariance) and computationally efficient (range adaptation).
Multiple lines of evidence implicate the orbitofrontal cortex (OFC) in value assignment underlying economic choice. Neural activity in this area is generally elevated when subjects make choices compared to when choices are made for them (Arana et al., 2003; Chaudhry et al., 2009). Moreover, blood oxygenation correlates with assigned values in a variety of choice tasks (Plassmann et al., 2007; Tom et al., 2007; Valentin et al., 2007; Hare et al., 2009; Venkatraman et al., 2009). Individual OFC neurons encode the subjective value monkeys assign to different juices when they choose between them (Padoa-Schioppa and Assad, 2006). Interestingly, neuronal activity in this area is also affected by the effort exerted by the animal to obtain the juice, by the probability with which the juice is delivered (Kennerley et al., 2009), and by the duration of a delay intervening before juice delivery (Roesch and Olson, 2005). Finally, OFC lesions specifically disrupt choice behavior (Gallagher et al., 1999; Rahman et al., 1999; Fellows and Farah, 2007; Machado and Bachevalier, 2007). Current evidence thus suggests that economic choices may be based on values represented in the OFC.
A broad and fundamental question is whether and how this representation of value depends on the behavioral context of choice. Notably, the behavioral context can vary in multiple ways. For example, a person in a wine store might serially compare different pairs of bottles—she might first compare a Nebbiolo to a Bordeaux, and later compare the same Nebbiolo to a Zinfandel. Changes in the alternative to a given option, which typically occur moment-to-moment, are referred to as changes of menu. However, the context of choice can also change in other ways. For example, while visiting northern Italy, the same person might choose between buying a 12-bottle case of that Nebbiolo and spending one more night in Venice. Broad changes such as from a wine store to a vacation, where value ranges vary, are referred to as changes of condition. In previous work, we found that the representation of value in the OFC is invariant for changes of menu (Padoa-Schioppa and Assad, 2008). Here we investigated whether and how this representation of value depends on the behavioral condition.
Several reasons led us to hypothesize that the representation of value in the OFC might be subject to neuronal adaptation. First, an early observation of Tremblay and Schultz, considered in the light of our previous results, suggests an adapting encoding of value (Tremblay and Schultz, 1999; Padoa-Schioppa and Assad, 2008). Second, from a computational perspective, a valuation system endowed with neuronal adaptation would present clear advantages. Indeed, while values faced in different conditions can vary by orders of magnitude, any neuronal representation of value is ultimately subject to physiological limitations on neuronal firing rates. These limitations imply that choices cannot be ubiquitously accurate (i.e., consistent) unless the valuation system effectively adapts to the range of values available in any given condition. On this basis, we predicted an inverse relationship between the sensitivity of value-encoding neurons and the range of values available in any particular condition. Our results confirmed this hypothesis.
For this study, we pooled and re-analyzed two data sets originally collected to examine the neuronal representation of economic value (Exp.1, 931 cells (Padoa-Schioppa and Assad, 2006)) and its independence from the menu (Exp.2, 557 cells (Padoa-Schioppa and Assad, 2008)). In all the experiments, monkeys chose in every trial between 2 juices labeled A and B, with A preferred. In Exp.1, two juices (A and B) were used in each session. Offers were represented by sets of colored squares on a computer monitor, and monkeys indicated their choice through eye movements (Fig.1a). In any given session, the amounts of the two juices varied from trial to trial, and the behavior of the monkey presented a quality/quantity trade-off. When offered equal amounts of the two juices, monkeys would consistently choose juice A. However, when juice B was offered in sufficiently large amounts, the monkeys would choose it. The “relative value” of the two juices was inferred from the indifference point—the quantity ratio for which the monkey would choose either juice equally often. For example, if the monkey was indifferent between 1A and 3B, we inferred that the value of 1A was equal to the value of 3B (relative value = 3). Trials were divided into “trial types” based on the offer type and the choice. For example, a monkey facing the offer type 4B:1A could choose either 1A or 4B, corresponding to the two trial types (4B:1A, 1A) and (4B:1A, 4B). Neuronal firing rates were examined in seven time windows: 0.5s pre-offer, 0.5s post-offer, late delay (0.5–1.0s after the offer), 0.5s pre-go, reaction time (from ‘go’ to saccade), 0.5s pre-juice, and 0.5s post-juice. A “neuronal response” was defined as the activity of one neuron in one time window as a function of the trial type. In Exp.2, we used identical procedures for behavioral control and neuronal recordings except that three juices (A, B and C) were used in each session (Padoa-Schioppa and Assad, 2008). Trials with the three juice pairs (A:B, B:C and A:C) were randomly interleaved. For the present study, we separated the three sets of trials and we re-labeled, for each juice pair, the two juices as A and B (A preferred). Thus each neuron recorded in Exp.2 provided three separate neuronal responses for each time window (separate sets of trials). Data from Exp.2 were then analyzed with the same procedures used for Exp.1
The analysis performed here builds on previous results showing that the population of neurons in the OFC encodes three variables—offer value (the value of one of the two juices, Fig.1bc), chosen value (the value of the chosen juice, Fig.1d) and taste (a binary variable indicating the identity of the chosen juice but not its quantity). Only neuronal responses significantly modulated by the trial type (ANOVA, p<0.001) were included in the analysis. For each response, we performed a linear regression on each variable. A variable was said to “explain” the response if the regression slope differed significantly from zero (p<0.05). If a given response was explained by more than one variable, the encoded variable was identified as the one with the highest R2. The encoding of value was linear in the sense that adding a quadratic value term to the regression generally failed to significantly improve the fit (Padoa-Schioppa and Assad, 2006).
The present study focused on neuronal responses encoding the offer value or the chosen value. For each response, “slope” and “intercept” were obtained from the linear regression onto the encoded variable. The “value range” was defined as the difference between the maximum value and the minimum value available in that session, the “activity range” was defined as the product of the slope by the value range, and the “baseline activity” was defined as the activity corresponding to the minimum value available in that session. Neuronal responses recorded in different time windows likely differ for their functional significance. However, the present results held true in each time window. We thus describe our findings pooling responses from different time windows, although we also report the main results obtained for individual time windows.
Each neuron in this data set (main data set) was recorded in one session with one value range. However, sizable groups of neurons were recorded with different value ranges. To examine how the encoding depended on the value range, we thus proceeded with a population-based approach. We divided the data set in “sub-populations” of neuronal responses recorded with different value ranges. We then considered one encoding parameter (slope, activity range, etc.), analyzed it for each sub-population, and compared the results across sub-populations (e.g., Fig.4d). This can be viewed as a repeated-measure procedure performed at the level of the neuronal population.
Unless otherwise indicated, we always express values in conventional units uV, defined as uA for offer value A, uB for offer value B, and uB for chosen value. Units uA (uB) are defined as equal to the value of one drop of juice A (juice B). Value ranges were thus “quantized” for offer value responses, while they could assume any numerical value for chosen value responses (see x-axes in Fig.4a-c). Sub-populations were naturally defined for offer value responses. For chosen value responses, we divided value ranges in bins of 1-2 uB width and we computed the average value range for each bin. A similar procedure was used to analyze neuronal responses in non-conventional value units (e.g., Fig.8).
Population analyses of the main data set indicated that the encoding of value in the OFC adapts to the range of values available in the behavioral condition. To confirm that adaptation takes place at the level of individual neurons, we collected an additional 111 cells from the OFC of one monkey. The experimental procedures were similar to those previously described (Padoa-Schioppa and Assad, 2006), with minor differences. Briefly, all surgical procedures were as previously described. The behavioral task was controlled by a custom-written software (Asaad and Eskandar, 2008) based on Matlab (PC version R2007b, MathWorks). Tungsten electrodes were advanced using a motorized micro-drive as previously described. Neuronal signals were amplified and filtered (Neuralynx; low-frequency cutoff 300 Hz, high-frequency cutoff 5 kHz), recorded at 25 kHz by a dedicated processor (Power 1401, CED) and saved to disk for off-line processing. Recordings were located in the same region examined in previous studies, tentatively identified as area 13m. Clustering procedures and preliminary data analyses were all as previously described. All experimental procedures conformed to the NIH guidelines and with the regulations of the Washington University School of Medicine.
The behavioral task was identical to that of Exp.1 (Fig.1a). However, each session included two blocks of trials, with small/large value range. For both offer value and chosen value responses, large value ranges were roughly twice as large as small value ranges. Each trial block included 200-300 trials. Data from the two trial blocks were first analyzed separately. For each neuron, time window and trial block, we determined whether the activity was modulated by the trial type (ANOVA, p<0.01) and, if so, whether it was explained by any variable (offer value A, offer value B, chosen value, taste; regression slope different from zero, p<0.05). Each linear regression provided an R2 and we conventionally set R2=0 for variables that did not explain the response. For each neuron, we then identified the encoded variable as the one that best explained responses across time windows and across trial blocks (highest total R2). To compare across trial blocks, we imposed that neuronal responses encode the same variable in both trial blocks. These criteria identified 19 offer value responses and 17 chosen value responses.
Our main data set included 937 responses encoding the offer value (Fig.1bc) and 817 responses encoding the chosen value (Fig.1d). We previously found that the encoding of value is linear in our experiments (Padoa-Schioppa and Assad, 2006). In other words, there is a linear relationship between the firing rate ϕ of a neuron encoding the offer value or the chosen value and the encoded value V, such that:
where parameters c0 and c1 represent, respectively, the intercept and the slope of the encoding.
To study how the representation of value depends on the behavioral condition, we first examined the regression slope. We considered separately neuronal responses encoding offer value A, offer value B and chosen value. For each response we computed the numerical value of the regression slope, and we plotted the results obtained for the entire population in a histogram (Fig.2a-c). This analysis revealed several points. First, for each encoded variable, neuronal responses can have either positive or negative slopes. Second, for each variable, the two distributions obtained for the subsets of responses with positive slopes and rectified negative slopes were statistically indistinguishable (all p>0.15, Wilcoxon test). We thus combined for each variable the two subsets of responses by computing the slope’s absolute value (Fig.2d-f). Interestingly, the distributions obtained for the three variables were different. In particular, the slope distribution for offer value A responses was significantly broader than that for offer value B responses (p<10-10, Wilcoxon test). This result may seem somewhat puzzling. In principle, neurons encoding offer value A and offer value B would seem to be “of the same type” as their responses encode the value of one of the juices available in the session. Labels ‘A’ and ‘B’ only indicate which one of the two juices is preferred, and it is not clear why this should affect the slope of the encoding. How can we explain the difference in slope distribution?
One possible explanation follows from the observation that, in the experiments, the range of juice B offered to the monkeys was typically larger than the range of juice A. Indeed, in all the sessions, the minimum quantity offered for each juice was always 0 drops, whereas the maximum quantity varied. Since monkeys preferred juice A to juice B, we generally offered larger quantities of juice B to ensure that the animals would choose juice B for some offer types. For example, in one representative session (Fig.1b), the quantity of juice A offered varied between 0 drops and 3 drops (value range = 3 units), while the quantity of juice B offered varied between 0 drops and 6 drops (value range = 6 units). (See also Figs.1c-d.)
To illustrate how the difference in value range may relate to the difference in regression slope, we developed a qualitative model of neuronal adaptation (Fig.3a). The model describes, for example, the activity of one neuron encoding the variable offer value X recorded in different sessions, in which juice X is offered in different value ranges. The fundamental assumption of the model is that the activity range of the neuron remains unchanged in different sessions. In any given session, the neuron adapts to the range of values of X offered to the monkey in such a way that the activity range of the neuron (Δϕ, y-axis) corresponds to the value range (ΔV, x-axis) offered in that session. The key prediction of this adaptation model is that the slope of the encoding should be proportional to the inverse value range (Fig.3b-c). In other words, according to this model, the following relationship should hold true: slope 1/ΔV (where ‘’ stands for ‘is proportional to’). In principle, the same model of neuronal adaptation might also describe the activity of neurons encoding the chosen value, leading to the same predicted relationship between slope and value range.
Each neuron in our main data set was recorded in one session with one value range. To test the predictions of the adaptation model, we thus performed a population analysis. We studied the distribution of regression slopes as a function of the value range (Fig.4a-c). For each of the three encoded variables (offer value A, offer value B and chosen value), we observe that regression slopes decrease as a function of the value range. This phenomenon is particularly clear when the mean slope is plotted against the value range (Fig.4d). As predicted by the adaptation model, the relationship between the mean slope and the inverse value range appears roughly linear (Fig.4e). For a statistical test, we performed a linear fit of the mean slope onto the inverse value range: slope = a0 + a1*1/ΔV. The result obtained for the coefficient a0 does not differ from zero (95% confidence interval). In other words, the measured relationship between the mean slope and the inverse value range is statistically indistinguishable from the predicted relationship slop 1/ΔV. Interestingly, this result does not depend on the particular units used to express values (scale invariance, see below).
Figures 4de suggest that offer value A and offer value B responses are indeed of the same type and that the difference in slope distribution (Fig.2) was due to the difference in value range. To confirm this point, we restricted the comparison of slope distributions to responses recorded with the same value range. In this case, we did not find any significant difference between offer value A and offer value B (analysis performed for ΔV = 3 and ΔV = 4, both p>0.5, Wilcoxon test). We thus pooled all offer value responses in subsequent analyses.
A close match between the activity of neurons in the OFC and the predictions of the adaptation model can also be observed examining directly the firing rate of individual neurons as a function of the encoded value. We first describe the results obtained for the population of 937 responses encoding the offer value (Fig.5). For each response, we subtracted the baseline activity corresponding to the minimum value available in that session. We then rectified neuronal responses with negative regression slope and plotted the resulting firing rate as a function of the encoded value (Fig.5a). Different colors in the figure label sub-populations of neuronal responses recorded with different value ranges. A qualitative inspection suggests that the various sub-populations have similar distribution of maximum firing rates. This point is especially clear when we average firing rates separately for each sub-population (Fig.5b). The emerging picture well matches that of the adaptation model (Fig.3a).
To further test the relationship between the neuronal firing rate and the range of values available in any behavioral condition, we defined for each neuronal response the activity range Δϕ as the product between the regression slope and the value range. Thus Eq.1 can be re-written as:
where Δϕ = c1 * ΔV is the activity range, ϕ0 = c0 + c1V0 is the baseline activity, and V0 is the minimum value available in that session. We then analyzed the distribution of activity ranges at the population level (Fig.5c). Multiple statistical analyses found that the distributions of activity ranges obtained for different sub-populations of responses did not depend on the value range (p>0.13, Kruskal-Wallis test; p>0.7, correlation analysis).
Thus for offer value responses, the distribution of neuronal activity ranges remains constant and does not depend on the range of values available in the behavioral condition. One concern might be whether this result is due to saturating neuronal activity or ceiling effects. The analysis of the average firing rate (Fig.5b), however, rules out this possibility. Indeed, the average firing rates recorded for different value ranges (indicated with different colors in Fig.5b) are well separated throughout the value spectrum, and the measure obtained for any particular value (e.g., V = 2 in Fig 5b) varies orderly depending on the value range. This indicates that the linear relationship between regression slope and inverse value range or, equivalently, the fact that the neuronal activity range does not depend on the value range are genuinely due to neuronal adaptation.
We repeated these same analyses for the population of 817 responses encoding the chosen value and we obtained very similar results (Fig.6). A qualitative inspection of individual responses suggests that different sub-populations recorded with different value ranges have similar distributions of maximum firing rates (Fig.6a). Computing the average firing rates for different sub-populations (Fig.6b) provides a picture resembling that of the adaptation model (Fig.3a). Importantly, average firing rates are separated throughout the value spectrum. Finally, statistical tests indicate that the activity range Δϕ is independent of the value range ΔV (p>0.05, Kruskal-Wallis test; p>0.4, correlation analysis; Fig.6c).
These results were obtained pooling responses from different time windows. However, analyses of data from individual time windows provided a very similar picture (Supplement, Fig.S1). In particular, we confirmed for each time window and for each encoded variable (offer value and chosen value) that the activity range Δϕ was independent of the value range ΔV (all p>0.05, Kruskal-Wallis test; all p>0.1, correlation analysis). In another control, we restricted the analysis to the neuronal population previously shown to be menu invariant (Padoa-Schioppa and Assad, 2008). The results of this analysis were very similar to those obtained for the entire population, indicating that menu invariance and range adaptation are complementary phenomena (Supplement, Fig.S2; see Discussion).
In conclusion, the activity of value-encoding neurons in the OFC appears to adapt to the range of values available in any behavioral condition—a result that holds true both for offer value neurons and for chosen value neurons. Thus the encoding of value in this area seems well described by Eq.2, where the activity range (Δϕ) is independent of the value range (ΔV). Additional recordings confirmed this result at the level of individual neurons (see below).
One important question is whether the encoding of value in the OFC presents other forms of neuronal adaptation. Referring to Eq.2, the question is whether the baseline activity (ϕ0) depends on the value range and whether the baseline activity and/or the activity range (Δϕ) depend on other variables that describe the behavioral condition. We examined several hypotheses.
First, we tested whether the activity range of neurons encoding the offer value of one particular juice depended on whether that juice was preferred or non-preferred. In our experiments, any given juice (e.g., cranberry juice) could be the preferred juice in some sessions (e.g., when it was offered against peppermint tea) and the non-preferred juice in other sessions (e.g., when it was offered against grape juice). We thus analyzed the distribution of activity ranges recorded for neurons encoding the offer value of cranberry juice in sessions in which cranberry juice was preferred (juice A) or non-preferred (juice B). As illustrated (Fig.7a), the two distributions were very similar and statistically indistinguishable (p>0.2, Wilcoxon test). We conducted this same analysis for the 6 different juices for which we had at least 5 responses for each preference status and we obtained consistent results. The distribution of activity ranges recorded for offer value neurons did not depend on whether the encoded juice was preferred or non-preferred (all p>0.05, Wilcoxon test).
Second, we tested whether the activity range of value-encoding neurons depended on the relative value the monkey assigned to the juices at the time of recording. Given two juices (e.g., apple juice and peppermint tea), preferences were usually consistent across sessions (monkeys preferred apple juice). However, the relative value of the two juices could vary from session to session depending, for example, on the thirst of the animal. For this analysis we thus considered all the neuronal responses encoding the chosen value recorded in sessions in which apple juice was offered against peppermint tea (Fig.7b). A linear fit of the activity range onto the relative value did not reveal any significant correlation between the two variables (95% confidence interval). We repeated this analysis across the population as follows. For chosen value responses, we considered in each instance the set of neuronal responses recorded with one particular pair of juices. For offer value responses, we considered in each instance the set of responses encoding the value of one particular juice recorded in sessions in which that juice was offered against another specific juice. We obtained a total of 32 instances (we restricted this analysis to instances with at least 20 data points). The results were rather consistent. In 29 out of 32 instances (91%) a linear fit failed to reveal any significant correlation between the activity range and the relative value (95% confidence interval).
Third, we examined whether the baseline activity (ϕ0 in Eq.2) is also subject to adaptation. We performed a series of analyses similar to those described above for the activity range. In particular, we investigated whether the baseline activity depends on the value range, on the juice preference, or on the relative value. In summary, we did not find any evidence of adaptation. These results are detailed in the Supplement (Fig.S3-S5).
Finally, we examined whether either the activity range or the baseline activity depend on the “other” value range (e.g., whether the activity of neurons encoding the offer value A depends on the range of values of B). We did not find any systematic relationship (Supplement, Fig.S6).
One interesting implication of Eq.2 is that the encoding of value in the OFC is “scale invariant”. In other words, the relationship between the neuronal firing rate ϕ and the value V should not depend on the particular units used to measure value (if the same units are used for V and ΔV). Consequently, the results presented above should also not depend on the value units. We tested this specific prediction as follows. Usually, we express values in conventional units uV corresponding to uA for offer value A, uB for offer value B and uB for chosen value. However, the behavioral choice pattern establishes in each session a relationship between uA and uB—the relative value equals uA/uB—such that any quantity of either juice can in fact be measured in either units. Thus we can test, for example, whether the relationship between the regression slope and the value range (slope 1/ΔV) is indeed independent of the value units.
For offer value and chosen value responses, we plotted the mean slope against the inverse value range, expressing all values in uA or, separately, all values in uB (Fig.8). The two sets of data points lie essentially on the same line. Indeed, an analysis of covariance did not find any significant difference between the 4 fitted lines (p>0.1). Note that each data point in Fig.8 represents the mean regression slope (y-axis) measured for the sub-population of responses recorded with a given value range (x-axis). By changing value units, we essentially re-shuffled responses on the x-axis. That the relationship between mean slope and value range was unaffected by this procedure implies that the relationship slope 1/ΔV indeed holds true, and that the results obtained in conventional units (Fig.4e) were not accidental.
In addition to providing a useful control, scale invariance is also interesting from a theoretical point of view. It implies that there is no intrinsic unit for economic value at the neuronal level, at least in the OFC.
The results presented in previous sections were based on population analyses. To confirm that individual cells undergo range adaptation, we collected an additional 19 offer value responses and 17 chosen value responses in sessions divided in two trial blocks, with small and large value range (small ΔV block and large ΔV block). The activity of one neuron encoding the chosen value is illustrated (Fig.9a). Consistent with adaptation, the slope of the encoding was shallower in the large ΔV block compared to the small ΔV block. This was typically the case for these neurons. Indeed, for 34 out of 36 responses the regression slope was shallower in the large ΔV block compared to the small ΔV block, a very unlikely result if regression slopes varied randomly (Fig.9b; p<10-9, binomial test). In contrast, the activity range did not systematically depend on the value range (Fig.9c; p>0.1, binomial test).
For the response illustrated in Fig.9a, it can be noted that the firing rate corresponding to chosen value = 6 uB was lower in the large ΔV block than in the small ΔV block, suggesting that the difference in slope was indeed due to neuronal adaptation as opposed to ceiling effects. To verify that this was true across the population, we considered for each response the firing rate recorded for the maximum value available in both trial blocks. We rectified responses with negative encoding and we compared the firing rate across trial blocks. For a significant majority of responses, the firing rate was lower in the large ΔV block compared to the small ΔV block (27/36 responses; p<0.001, binomial test). This result is consistent with adaptation but not with ceiling effects.
One interesting question concerns the timing of neuronal adaptation. In our experiments, the value range was not explicitly indicated, but it remained constant for a large number of trials. Since different trial types were randomly interleaved, monkeys learned the value range by performing multiple trials at the beginning of each session. Hence, neuronal adaptation presumably took place gradually over the course of multiple trials. However, recent results from other laboratories show that if the value range is explicitly updated on a trial-by-trial basis, at least partial neuronal adaptation can be observed on a short time scale (Elliott et al., 2008; Kobayashi et al., 2009). We thus examined whether trial-by-trial adaptation also occurred in our experiments. Specifically, we examined whether, in any given trial, neuronal responses encoding the value available in that trial were partly modulated by the value available in the previous trial.
Neuronal adaptation leads to a specific prediction. Consider a neuron encoding the chosen value (Figure 10a, red). There is a linear relationship between the firing rate and the value chosen by the monkey in the current trial. Now consider trials such that chosen value = 4. If the cell undergoes adaptation, then the activity in the current trial should be slightly elevated if the chosen value in the previous trial was lower than 4. Conversely, the activity in the current trial should be slightly depressed if the chosen value in the previous trial was higher than 4. To test this prediction, we divided trials in two groups, depending on whether the chosen value in the current trial (trial n) was higher or lower than the chosen value in the previous trial (trial n-1). Indicating with V(k) the value chosen in trial k, the two groups of trials were thus defined by V(n)>V(n-1) and V(n)<V(n-1), respectively. Consistent with neuronal adaptation, the activity for trials V(n)>V(n-1) was slightly higher than the activity for trials V(n)<V(n-1) (Fig.10a).
To quantify this effect, we computed for each trial type the difference in firing rate between trials V(n)>V(n-1) and trials V(n)<V(n-1), normalized by the firing rate obtained pooling all trials. We then defined δ as the average normalized difference across trial types. In essence, δ represents the percent modulation of trial n-1 on the activity measured on trial n. For the response illustrated (Fig.10a), we obtained δ=14%. We thus computed δ for each response in the main data set. For responses with negative encoding, we inverted the sign of δ. Thus for every response, trial-by-trial adaptation would result in δ>0. For both offer value and chosen value responses, δ varied substantially across the population (Fig.10b). However, for both variables there was a significant majority of responses with δ>0 (both p<10-10, binomial test). Moreover, mean(δ) was significantly greater than zero (both p<10-9, t-test). Quantitatively, we found that mean(δ) was ~6% for both variables. To test the reliability of this result, we repeated the analysis for trial n-2, trial n-3, etc. (i.e., we divided trials depending on whether the current value was higher/lower than the value 2 trials before, 3 trials before, etc.). As expected, the effect faded rapidly with the trial distance—mean(δ) was ~6% for trial n-1, ~2% for trial n-2, and indistinguishable from zero for earlier trials (Fig.10c). For a control, we repeated the analysis for trial n+1 (i.e., we divided trials depending on whether the current value was higher/lower than the value in the next trial). We did not expect any such dependence because neurons cannot adapt to future events. Indeed, mean(δ) was statistically indistinguishable from zero (Fig.10c).
In conclusion, partial neuronal adaptation occurred on the time scale of individual trials, as the value available in any one trial modulated neuronal responses in the following trial. Notably, this effect is in the direction opposite to that expected if the value available in any one trial elicited long neuronal responses that outlast the trial end (Simmons and Richmond, 2008).
Values processed in different behavioral conditions can vary by orders of magnitude. This variability is analogous to that faced by the visual system, where various mechanisms of neuronal adaptation ensure that contrast sensitivity remains high independently of the range of luminance (Laughlin, 1989; Clifford et al., 2007). Our present results show that an analogous process of neuronal adaptation takes place in the valuation system. In particular, we found that OFC neurons encode value in a linear way and that their neuronal sensitivity (i.e., the slope of the encoding) is inversely proportional to the range of values available in any given condition. Conversely, the distribution of activity ranges does not depend on the value range. Interestingly, similar phenomena of neuronal adaptation were also observed in dopamine neurons (Tobler et al., 2005) and in the anterior cingulate cortex (Sallet et al., 2007).
The present results complement our previous findings. In another study, we showed that the encoding of value in the OFC is menu invariant. In other words, the response of a neuron encoding the value of one good (a particular juice in the case of offer value neurons, or the chosen juice in the case of chosen value neurons) does not depend on the type and value of the other good available at the same time (i.e., the non-encoded good) (Padoa-Schioppa and Assad, 2008). Here we showed that the encoding of value in the OFC is range adapting. In other words, the response of a neuron encoding the value of one good (a particular juice in the case of offer value neurons, or the chosen juice in the case of chosen value neurons) adapts to the range of values available for that good in the behavioral condition (i.e., the distribution of encoded values). In essence, both offer value and chosen value neurons adapt to the range of values they encode, while they do not depend on the goods or values that they do not encode. The two phenomena—menu invariance and range adaptation—are thus complementary (indeed, they were observed in the same neuronal population). Taken together, they indicate that the encoding of value in the OFC is at one time instantiative of preference transitivity (menu invariance) and computationally efficient (range adaptation).
One important issue relates to the time scale of range adaptation. As in the study of Tremblay and Schultz (1999) (see below), in our experiments value ranges remained constant for a large number of trials. In contrast, menu invariance was demonstrated by randomly interleaving trials with different juice pairs. This might suggest that range adaptation is an intrinsically slow process, whereas menu invariance is a property of the short time scale (Padoa-Schioppa and Assad, 2008). However, our current analysis and other reports (Elliott et al., 2008; Kobayashi et al., 2009) indicate that at least partial adaptation can take place on the time scale of individual trials. In fact, menu invariance is fully consistent with range adaptation occurring on a short time scale. Consider for example a neuron encoding the offer value of apple juice. The encoding undergoes adaptation in the sense that the neuronal activity elicited by a particular quantity of apple juice depends on the range of apple juice available in that session. At the same time, the encoding is menu invariant in the sense that the neuronal activity elicited by a particular quantity of apple juice does not depend on the type and quantity of the other juice available. Thus not only is the encoding of value both menu invariant and range adapting. Menu invariance holds true independently of the time course of range adaptation. Importantly, this understanding captures the essence of menu invariance and its fundamental link with preference transitivity (Grace, 1993; Tversky and Simonson, 1993; Shafir, 1994; Padoa-Schioppa and Assad, 2008).
The present results shed new light on the early observation of Tremblay and Schultz (1999). In their study, monkeys were delivered one of three possible juices (A, B and C, in decreasing order of preference). Trials were blocked, and only two juices were used in each block. A subset of OFC neurons responded to juice A but not to juice B in ‘A:B’ blocks, and to juice B but not to juice C in ‘B:C’ blocks. Our results suggest that these might have been chosen value neurons whose activity encoded the value received by the monkey in any given trial, and whose encoding adapted to the range of values received within each block. In ‘A:B’ blocks, any such neuron would respond with maximum firing rate when the monkey received juice A and with minimum firing rate when the monkey received juice B. In ‘B:C’ blocks, the same neuron would respond with maximum firing rate when the monkey received juice B and with minimum firing rate when the monkey received juice C. In this view, neurons in the OFC encode value (not relative preference), but in a range-adapting way. One interesting question is whether range adaptation also explains similar observations in other brain regions (Hosokawa et al., 2007).
While the evidence for neuronal adaptation appears robust, important questions remain. First, the exact mathematical definition of ‘value range’ remains to some extent undetermined. In this study, we defined the value range as the difference between the maximum value and the minimum value available in the behavioral condition. However, for offer value responses, the minimum value was always equal to zero in our experiments. Thus we cannot rule out that offer value neurons might adapt to the maximum encoded value per se—an alternative definition of ‘value range’. This distinction was theoretically possible for chosen value neurons, but in practice our data did not disambiguate between the two hypotheses. Thus more experiments are necessary to conclusively define the ‘value range’. Second, our results demonstrate that the activity range of OFC neurons represents different value ranges in different conditions—a “quantitative adaptation”. In addition, it is possible that OFC neurons undergo a “qualitative adaptation” when choices are made in different behavioral settings. For example, a neuron encoding the offer value of apple juice in sessions in which apple juice is available might encode the offer value of another juice in sessions where apple juice is not available. Given the relatively large number (10-12) of juices used in our experiments, and the potentially much larger number of goods available in nature, this putative mechanism of flexible encoding seems a credible hypothesis. Future work will investigate this intriguing issue.
An increasing body of work links the OFC to the computation of subjective value underlying economic choice. Imaging studies show that neural activity in the OFC and ventromedial prefrontal cortex correlates with value in choices between food and money (Plassmann et al., 2007; Hare et al., 2008), risky choices (Rolls et al., 2008; Venkatraman et al., 2009), intertemporal choices (Kable and Glimcher, 2007; Hare et al., 2009), choices that involve gains and losses (Tom et al., 2007; De Martino et al., 2009), and in food devaluation paradigms (Valentin et al., 2007). At the single-cell level, our work shows that individual neurons in the OFC encode the offer value and the chosen value, two variables seemingly necessary for economic choice (Padoa-Schioppa and Assad, 2006). Moreover, neuronal activity in this area is affected by a variety of variables relevant to choice behavior, including food-specific satiety (Rolls et al., 1989; Pritchard et al., 2008), exerted effort, probability of juice delivery (Kennerley et al., 2009; Kennerley and Wallis, 2009), and time delays (Roesch and Olson, 2005). Single cell recordings in rats provide corroborating evidence (Roesch et al., 2006; van Duuren et al., 2007; Kepecs et al., 2008), although there are notable differences between rodents and primates (Feierstein et al., 2006; Zald, 2006; Wise, 2008). Last but not least, numerous studies demonstrate that OFC lesions specifically impair choice behavior (Bechara et al., 1996; Pasquier and Petit, 1997; Gallagher et al., 1999; Rahman et al., 1999; Fellows and Farah, 2007; Koenigs and Tranel, 2007; Machado and Bachevalier, 2007; Baxter et al., 2009). This accumulated evidence suggests that economic choices may be based on values represented in the OFC (Fellows, 2007; Padoa-Schioppa, 2007; Wallis, 2007). The present study sheds further light onto the nature of this computation, indicating that OFC neurons represent value in an adapting and thus computationally efficient way. More generally, it supports the understanding that fundamental principles first documented in sensory and motor systems, such as neuronal adaptation, also govern neurobiological processes underlying more complex behaviors, such as economic choice.
The author thanks W. Schultz for extended discussions and for sharing unpublished results from his laboratory. The author also thanks J. Assad and L. Snyder for helpful comments on an earlier version of the manuscript, and W. Asaad for providing software for behavioral control. Analysis of the main data set was started at Harvard University and completed at Washington University, where we collected additional data and prepared the manuscript. This work was supported by a Pathway to Independence Award from the National Institute of Mental Health (grant number R00-MH080852). The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.