PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Neurosci. Author manuscript; available in PMC 2010 December 1.
Published in final edited form as:
PMCID: PMC2942083
NIHMSID: NIHMS216040

Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys

Abstract

The value of events that predict future rewards, thereby driving behavior, is sensitive to information arising from external (environmental) and internal factors. The ventral prefrontal cortex, an anatomically heterogeneous area, has information related to this value. We designed experiments to compare the contribution of two distinct subregions, orbital and ventromedial, of the ventral prefrontal cortex to the encoding of internal and external factors controlling the perceived motivational value. We recorded the activity of single neurons in both regions in monkeys while manipulating internal and external factors that should affect the perceived value of task events. Neurons in both regions encoded the value of task events, with orbitofrontal neurons being more sensitive to external factors such as visual cues and ventromedial neurons being more sensitive to internal factors such as satiety. Thus, the orbitofrontal cortex emphasizes signals for evaluating environment-centered, externally-driven motivational processes whereas ventromedial prefrontal cortex emphasizes signals more suited for subject-centered, internally-driven motivational processes.

Keywords: motivation, emotion, reward, orbitofrontal cortex, ventromedial prefrontal cortex, primate

Introduction

Motivation, that is, what causes an organism to act, is regulated by information arising from internal and external factors (B. W. Balleine and A. Dickinson, 1998; K. C. Berridge, 2004; T. Minamimoto et al., 2009). Internal, subject-related information is related to primary needs such as hunger or thirst, but also to spontaneously initiated cognitive processes. For instance, this is the case both when we go shopping simply because we are hungry, and when we plan to go to the store at convenient times to maintain a sufficient food storage. The information about external, environment-related factors, arises from sensory cues in the external world, for instance when we purchase the snack that just appeared in an advertisement.

Normal motivation relies on the ventral prefrontal cortex, a heterogenous area (A. R. Damasio, 1994; R. N. Cardinal et al., 2002; S. M. Cox et al., 2005; K. J. Ressler and H. S. Mayberg, 2007; J. D. Wallis, 2007; K. C. Berridge and M. L. Kringelbach, 2008). Based on cytoarchitachtonics and connectivity, ventral prefrontal cortex appears to be segregated into two differentiable circuits: medial (ventromedial prefrontal cortex, VMPFC), and orbital (orbitofrontal cortex, OFC). VMPFC is heavily interconnected with limbic and autonomic structures, and OFC is heavily interconnected with sensory areas (D. Ongur and J. L. Price, 2000). Behavioral and functional imaging results suggest that the anatomical segregation is accompanied by functional differences (B. W. Balleine and A. Dickinson, 1998; G. Egan et al., 2003; J. A. Gottfried et al., 2003; M. L. Kringelbach et al., 2003; J. W. Kable and P. W. Glimcher, 2007; S. B. Ostlund and B. W. Balleine, 2007; M. F. Rushworth et al., 2007; T. E. Behrens et al., 2008; J. Glascher et al., 2009; T. A. Hare et al., 2009). We hypothesized that value information that regulates motivational intensity is evaluated differently in OFC and VMPFC, with OFC neurons emphasizing information about external factors, and VMPFC neurons emphasizing information about internal factors.

We compared the activity of single OFC and VMPFC neurons in behaving monkeys while manipulating external and internal factors controlling the perceived value of task events. We assessed the perceived value of task events by measuring the intensity of 2 behavioral responses: an operant response (bar release) and an appetitive Pavlovian response (lipping) (S. Bouret and B. J. Richmond, 2009). We measured neuronal activity in trials where behavior was guided by visual cues (external factor) or self-initiated (internal factor). We also monitored the influence of satiety, a key internal factor influencing motivation, on behavior and neuronal activity. The neuronal activity in both regions is closely related to the value of task events. The neurons in OFC are more sensitive to external, environment-related information (visual cues), whereas the neurons in VMPFC neurons are more sensitive to internal, subject-related information (self-initiated behavior and satiety).

Methods

Animals

Two male rhesus monkeys, D (9.5 kg) and T (6.5 kg) were used. The experimental procedures followed the NIH Guide for the Care and Use of Laboratory Animals, and were approved by the NIMH Animal Care and Use Committee.

Behavior

Each monkey squatted in a primate chair positioned in front of a monitor on which visual stimuli were displayed. A touch sensitive bar was mounted on the chair at the level of the monkey’s hands. Liquid rewards were delivered from a tube positioned with care between the monkey’s lips but away from the teeth. With this placement of the reward tube the monkeys did not need to protrude their tongue to receive rewards. The tube was equipped with a force transducer to monitor the movement of the lips (referred to as ‘lipping’, as opposed to licking which we reserve for the situation in which tongue protrusion is needed) (S. Bouret and B. J. Richmond, 2009). Before each experiment, the amplitude of the signal evoked by delivering of a drop of water through the spout was checked to ensure that it matched the observed lipping response.

Monkeys were trained to perform the task depicted on Fig. 1. Cued Active trials: Both monkeys had experience in operant tasks involving a sequential color discrimination task, in which they were rewarded for detecting when a target, consisting of a small dot, changed from red to green. Each trial began when the monkey touched the bar. One of three visual cues appeared, followed five hundred milliseconds later by a red target (wait signal) in the center of the cue. After a random interval of 500–1500 ms, the target turned green (go signal). If the monkey released the touch-bar 200–800 ms after the green target appeared, the target turned blue (feedback signal), and a liquid reward was delivered 400–600 ms later. In Cued Active trials the reward sizes of 1, 2 or 4 drops of liquid were related to the cues. If the monkey released the bar before the go signal appeared, or after the go signal disappeared, an error was registered. No explicit punishment was given for an error in either condition, but the monkey had to perform a correct trial to move on in the task. That is, the monkey had to repeat the same trial with a given reward size until the trial was completed correctly. Performance of the operant bar release response was quantified by measuring reaction times and error rates.

Figure 1
Experimental design

Once monkeys adjusted their operant performance as a function of reward-predicting cues (1–2 days), they were exposed to Cued-Passive trials. In these passive trials, monkeys still had to touch the bar to initiate a trial but once the cue had appeared on the screen, releasing or touching the bar had no effect. Two seconds after cue onset, the blue point also used as a feedback signal in Cued-Active trials was presented and water was delivered 400–600 ms later as a reward. The 2 seconds delay between cue onset and feedback signal was chosen to match the average interval between these 2 events in Cued Active trials. After 2–3 days of training with Passive trials alone, monkeys had virtually stopped releasing the bar in Cued-Passive trials. Monkeys were then exposed to a block version of the Cued trials for another 2–3 days: blocks of approximately 100 trials of each category (Cued-Active or Cued-Passive) alternated without interruption or explicit signaling. In the final version, the 6 trial types with a combination of reward size (1,2 or 4 drops) and action contingency (Active or Passive) alternated randomly. Monkeys were trained for a week in this final version before we started electrophysiological recordings.

Self-Initiated Trials

Animals were placed in the same environment as before except that the background of the screen was changed to a large green rectangle. Monkeys rapidly (1 day) learned to hold and release the bar in order to get the reward without any conditioned cue signaling reward size or timing of actions. To facilitate comparison with Cued trials, a blue point was also used as a feedback signal upon bar release and reward was delivered within 400–600 ms. Before the neurophysiological recordings were taken, monkeys were trained for a week with alternating blocks of different reward sizes (1, 2 or 4 drops). Each block comprised approximately 50–70 trials with a given reward size and blocks alternated randomly and abruptly, without explicit signaling.

‘Satiation’ procedure

This procedure was conducted in separate sessions. After a neuron had been isolated for recording and the monkey had completed about 120–160 Cued trials, we interrupted the task and delivered ~ 100 cc of water through the spout. We then resumed the task for as long as the monkey would work or for an equivalent number of trials as collected before the ‘free’ water delivery.

Electrophysiology

After initial behavioral training, an MR image at 1.5 T was obtained to determine the placement of the recording well. Then, a sterile surgical procedure was carried out under general isoflurane anaesthesia in a fully equipped and staffed surgical suite to place the recording well and head fixation post. The well was positioned at the level of the genu of the corpus callosum, with an angle of approximately 20 degrees in the coronal plane (Fig. 2).

Figure 2
Localization of recording sites using MRI

Electrophysiological recordings were made with tungsten microelectrodes (FHC or Microprobe, impedance: 1.5 MΩ). The electrode was positioned using a stereotaxic plastic insert with holes 1mm apart in a rectangular grid (Crist Instruments, 6-YJD-j1). The electrode was inserted through a guide tube. After several recording sessions, MR scans were obtained with the electrode at one of the recording sites; the position of the recording sites was reconstructed based on relative position in the stereotaxic plastic insert and on the alternation of white and grey matter based on electrophysiological criteria during recording sessions.

Data analysis

Lipping behavior

The lipping signal was monitored continuously and digitized at 1 kHz (Fig 3A). For each trial, the latency of lipping responses after cue appearance and feedback (blue spot) signals was defined as the first of 3 successive windows in which the signal displayed a consistent increase in voltage of at least 100 mV from a reference epoch of 250 ms taken right before the event of interest (cue or feedback).

Figure 3
Lipping behavior

Single unit activity

All data analyses were performed in the R statistical computing environment (Team RDC 2004). The data were first screened using a sliding window procedure. For each neuron, we counted spikes in a 300 ms test window that was moved in 20 ms increments around the onset of the cue (from −400 to + 1300 ms), around the feedback signal (from −500 to + 1300 ms) and around reward delivery (from −800 to + 1200 ms). At each point, a 2 way ANOVA was completed with spike count as the dependant variable. The two factors were Reward Size (3 levels: 1, 2, 4 drops) and Action (2 levels: Active, Passive). At each time point, we measured the encoding of information about a given factor (% variance explained) for each neuron, as well as the percentage of neurons showing a significant effect (p<0.05, corrected for multiple comparisons using False Discovery Rate (FDR, ‘p.adjust’ function in R). In self-initiated trials, we used the same approach to study the encoding of Reward Size around the feedback signal and reward delivery.

Using this screening procedure, we found the epochs that showed a peak in encoding (i.e. variance explained), and focused the analysis on these epochs (n=5 and 3 in Cued and Self-Initiated trials, respectively; see Result section). In each epoch of the Cued trials, spike counts were compared across conditions using a 2 way ANOVA, with Reward Size (again, 1, 2, 4 drops) and Action (Active vs Passive) as factors. We defined responding neurons as those with a significant effect (p<0.05) of either factor or their interaction. In Self-Initiated trials, we used a 1 way ANOVA to quantify responses to the Reward Size factor.

Response latency was defined as the beginning of the first of 3 successive windows showing a significant effect in the sliding window analysis (p<0.05). We also measured the time of maximum variance explained after cue onset and around the feedback. It was defined for each neuron as the start time of the window for which the variance explained by a given effect was maximal, whether or not there was a significant response. We considered a 1000 ms cue period starting at cue onset and a 800 ms feedback period centered on the onset of the feedback signal.

To determine the influence of ‘progression through a session’ on neuronal activity, we measured the proportion of responding neurons in each of the 5 epochs in Cued trials (3 epochs in Self-Initiated trials). The means of the proportions of responding neurons across all the epochs of a trial were compared using ANOVA with ‘brain region’ as one factor (2 levels: OFC and VMPFC) and ‘progression through a session’ as another factor (3 levels: beginning, middle and end). Neurons were sorted according to the order in which they were recorded in a session: first (beginning), intermediate (half way through) and last (last complete recording before the monkey stopped). Neurons that did not belong to any of these 3 categories were not included in this analysis.

To quantify the effect of the active ‘satiation’ procedure (giving the monkey ~ 100 cc of water) on individual neuronal responses, we measured spike counts in epochs where a significant response was detected before the animal was given the bolus of free water. We carried out a 3-way ANOVA with satiety as the first factor (2 levels, before and after the bolus delivery), the second being Reward Size (3 levels, 1, 2 and 4 drops) and the third being Action (Active or Passive trials). Responses displaying either a main effect of satiety or an interaction between satiety and either of the other 2 factors were analyzed using a post-hoc Tukey HSD test. The effect of satiation was then classified as ‘increase’ when the encoding of a factor increased, i.e. when a given factor accounted for significantly more variance after than before the satiation, ‘decrease’ when the encoding of a factor decreased, or ‘change when the type of response changed, e.g. if the neuron was encoding Reward Size before and Action after the satiation.

Results

Experimental design

The stimulus-reward and action-reward contingencies were manipulated using 3 different trial types: Cued-Active, Cued-Passive and Self-Initiated trials (Fig. 1). In any one trial of each of these trial types, the amount of reward could be 1, 2, or 4 drops of fluid. In Cued trials, the cue appearing at the beginning of each trial indicated both the amount of fluid reward that would be delivered and whether the trial was Active or Passive. In Cued-Active trials, the monkeys had to perform an operant bar release response when a red point turned green. A feedback signal (blue point) replaced the green point immediately after each correct response. In Cued-Passive trials, a cue also appeared at the beginning of each trial, and the feedback signal appeared 2 seconds later, independently of the monkey’s behavior (2 seconds is the average interval between cue and feedback onset in Active trials). Cued-Active and Cued-Passive trials were randomly interleaved during a session. In Self-Initiated trials the monkeys only had to touch and release a bar; there was no visual cue at the beginning of a trial but the feedback signal appeared immediately after bar release. Self-initiated trials were presented in randomly alternating blocks, each with a constant reward size (1, 2 or 4 drops). After approximately 60 trials, the reward size was changed abruptly. In all trials, Cued or Self-initiated, the reward was delivered approximately 500 ms after the feedback signal.

Behavior

We trained 2 monkeys (T and D). To measure the value of task events in all task conditions, we monitored a Pavlovian appetitive lipping reaction to the cues and the feedback signal (Fig. 3A). The percentage of trials with lipping responses to cue appearance increased with reward size but was indistinguishable between Cued-Active and Cued-Passive trials (Fig. 3B, left; 2-way ANOVA: significant effect of Reward Size factor (monkey D: F(2)=26, p<10−10 ; monkey T: F(2)=36, p<10−10); no effect of Action factor (Cued-Active vs Cue-Passive, D: F(1)=1.8, p=0.2 ; T: F(1)=3.5, p=0.06)). This indicates that the perceived value of cues depended upon expected reward size, and not upon whether an action would be needed to obtain the reward. In contrast, lipping at the feedback was stronger in Cued-Active than in Cued-Passive trials, and there was relatively little effect of the Reward Size factor (Fig. 3B, center; 2-way ANOVA: Action: D: F(1)=91, p<10−10 ; T: F(1)=5.6, p=0.01; Reward Size: (D: F(2)=11, p=2 × 10−5 ; T: F(2)=1.9, p=0.1). This indicates that the value of the feedback in Cued trials depended much more upon the way in which the trial was completed (Active or Passive) than on the size of the expected reward. In Self-Initiated trials, lipping at the feedback increased significantly with reward size (Fig. 3B, right; ANOVA for Reward Size: D: F(2)=19, p=4 ×10−7; T: F(2)=46, p<10−10). Thus, in the Self-Initiated trials, where there is no Action factor, the value of the feedback is strongly related to Reward Size.

We also assessed perceived value by monitoring an operant response, bar release (Fig. 4), known to be driven by incentive motivation in similar tasks (S. Bouret and B. J. Richmond, 2009; T. Minamimoto et al., 2009). In Cued-Active trials, error rates decreased significantly with increasing reward sizes (ANOVA, D: F(2)=33, p<10−10 ; T: F(2)=8, p=4 × 10−4). In Cued-Passive trials, the monkeys virtually never released the bar. In Self-Initiated trials, release intervals decreased with increasing reward sizes (ANOVA: D: F(2)=33, p<10−10 ; T: F(2)=8, p=4 × 10−4). Thus, the incentive influence of value on operant actions increases with expected reward size in both Self-Initiated and Cued-Active trials, but not in Cued-Passive trials.

Figure 4
Bar Release Behavior

Electrophysiology

We recorded 167 and 188 neurons from the ventral prefrontal cortex of monkey T and D, respectively. All neurons encountered along the track were included in the analysis, as long as the units were well isolated using a time-voltage threshold discrimination criterion. The activity profiles were similar in the 2 animals so the neuronal data were pooled. We reconstructed the locations of the neurons using MRI (Fig. 2). In Cued trials, 112 and 121 neurons were recorded from OFC and VMPFC, respectively. In Self-Initiated trials, 70 and 74 neurons were recorded from OFC and VMPFC, respectively. Based on a visual inspection, neuronal activity in both regions was affected by both the Reward Size (1, 2 or 4 drops) and the Action (Passive vs Active trials) factors in Cued trials (Fig. 5A–D) and by the Reward Size factor in Self-Initiated trials (Fig. 5E–F). We used a screening procedure using ANOVA in sliding windows (with a repeated measure correction, see methods) to identify epochs with a strong encoding of Reward Size and /or Action. There were 5 epochs in Cued trials (‘Cue’, from 0 to 450 ms after cue onset; ‘Wait’, from 500 to 950 ms after cue onset; ‘Pre-Feedback’, from 450 to 0 ms before the feedback; ‘Feedback’, from 0 to 450 ms after the feedback and ‘Reward’, from 0 to 450 ms after reward delivery). There were 3 epochs in Self Initiated trials (Pre-Feedback, Feedback and Reward). In each epoch, we identified responding neurons using a 2 way ANOVA (Reward Size x Action) in Cued trials and a 1 way ANOVA (Reward Size) in Self-Initiated trials (see methods).

Figure 5
Examples of single unit activity

Neurons in OFC and VMPFC encode the perceived value of task events

We inferred that neuronal activity encoding the value of task events should follow the same pattern as the lipping behavior (strong effect of Reward Size at cue onset, strong effect of Action at the feedback in Cued trials and of Reward Size in Self-Initiated trials, Fig. 3). We compared the effects of Reward Size and Action factors on spike counts for each neuron using 2-way ANOVAs in successive 300 ms windows moved in 20 ms steps (sliding window analysis). At cue onset, the encoding of the Reward Size factor engaged a larger proportion of neurons and accounted for more variance than the encoding of the Action factor (Fig. 6 and Fig. 7, left panels). The encoding of Action became more prominent during the course of a trial, with a sharp increase in the proportion of neurons encoding this factor at the feedback (Fig. 6, center panels and Fig. 7C). In Self-Initiated trials, the information about Reward Size arises from the structure of the task (block design) rather than from visual stimuli. A large proportion of neurons encoded this factor (Fig. 6, right panels and Fig. 7E). Thus, neuronal activity in both areas followed the same pattern as lipping responses to cues and feedback signals, in line with the idea that neurons in ventral prefrontal cortex encode the value of these events.

Figure 6
Dynamic encoding of Action and Reward Size in OFC and VMPFC
Figure 7
Percentage of responses and mean variance at cue onset and feedback in OFC and VMPFC

Neuronal activity in these areas was not related to the overt behavior in any simple way that we could identify. We looked for correlation between neuronal activity (firing rates at the cue and around the feedback) and several measures of behavior (reaction time, lipping at the cues and lipping at the feedback) on a trial-by-trial basis. The number of neurons displaying a significant correlation between firing rate and either reaction time or lipping did not reach significance (i.e., the number of neurons displaying a significant correlation remained lower than the number expected by chance for a sample of this size). Neuronal activity was not related to the physical properties of the stimuli either. In Cued trials, 8 neurons were tested with 2 cue sets with which monkeys were equally familiar, and response patterns were indistinguishable between the 2 cue sets, showing that the responses depended on their associations with the predicted outcomes. Thus, the activity of these neurons is not simply encoding basic motor or sensory processes.

Value-related activity differs between OFC and VMPFC

In Cued trials neurons in both regions were more sensitive to Reward Size at cue onset and more sensitive to Action at the feedback. Nonetheless, response patterns in OFC and VMPFC were different. In OFC, the proportion of neurons encoding Reward Size was greater than the proportion of neurons encoding Action or the interaction between these 2 factors (Fig 8A, top). In VMPFC, the overall proportions of neurons responding to Reward Size and Action were similar, and they were both greater than the proportion of neurons encoding the interaction between the 2 factors (Fig. 8A, bottom). In addition, the encoding of the Action Factor around the feedback differed between OFC and VMPFC: In VMPFC, the increase in proportion of neurons encoding Action occurred before the feedback (Fig. 8A, bottom) whereas in OFC, the encoding of Action peaked after the feedback (fig 8A, top). In Self-Initiated trials, the encoding of Reward Size was indistinguishable across the 3 epochs (chi-squared p > 0.05, Figure 8B).

Figure 8
Proportion of responding neurons across epochs of a trial

We also examined the timing of responses to the cues and the feedback signal in Cued trials by measuring response latencies (when the encoding of an effect started, fig 9A) and the time of maximum variance explained (when the encoding of an effect peaked, fig 9B). At cue onset the median response latency for Reward Size was significantly shorter in OFC (median=60 ms, IQR= 0–140 ms) than in VMPFC (median=160 ms, IQR= 40–260 ms; Wilcoxon: p=0.04, fig 9A, left). The time at which selectivity peaked (time of maximum variance explained) was shorter for the Reward Size than for the Action factor, but there was no difference between the 2 regions (Fig. 9B, left). Thus, at the cue, the encoding of Reward Size began earlier in OFC than in VMPFC but the time at which the effect peaked was indistinguishable between the 2 areas. At the feedback, response latencies for Reward Size, Action or their interaction were indistinguishable in OFC. In VMPFC, response latencies to Action were significantly shorter than responses to Reward Size or to the interaction between these 2 factors (fig 9A, right). In addition, in VMPFC, the encoding of Action (% variance explained) peaked earlier (at the feedback) than the encoding of Reward Size (after the feedback) whereas in OFC, the encoding of Reward Size peaked before that of Action (Fig. 8B, right).

Figure 9
Latencies of responses to Action, Reward Size and their interaction

In short, at cue onset, the encoding of Reward Size is more prominent and arises earlier in OFC than in VMPFC. At the feedback in Cued trials, neurons become more sensitive to the Action factor and the transition begins earlier in VMPFC (before the feedback) than in OFC (after the feedback signal).

Ventral prefrontal neurons do not encode the incentive influence of value on operant actions

We reasoned that to encode the incentive influence of value on operant actions, i.e. the amount of energy invested in goal-directed behavior, neuronal activity should follow the same pattern as the bar release responses. That is, firing should be affected by Reward Size in Cued-Active but not in Cued-Passive trials. We searched neurons displaying that specific pattern among neurons displaying a significant interaction between Reward Size and Action. Less than five percent of all the neurons showed this specific pattern across the 5 epochs of a trial (means: 2.8 ± 1% in OFC and 1.4 ± 1% in VMPFC). Thus, the activity of ventral prefrontal neurons does not reflect the incentive effect of event value on operant actions.

Differential influence of internal and external factors on OFC and VMPFC activity

To assess the relative influence of information about external and internal factors, we compared neuronal activity between Cued and Self-Initiated trials. We reasoned that in Cued trials, the value of events was determined mostly based on external information (visual stimuli) whereas in Self-Initiated trials, value depended more upon internal knowledge. A direct comparison between the 2 regions showed that the Reward Size factor was predominantly encoded in VMPFC neurons during Self-Initiated trials and predominantly encoded in OFC during Cued trials (Fig. 10; Fig. 2-way ANOVA on percentages of neurons encoding the factor Reward Size: significant effect of region F(1)= 116, p=1.6 × 10−7; no effect of trial type (F(1)=2, p=0.2 and significant interaction F(1,12)=77, p=1.5 × 10−6). Thus, VMPFC neurons are more heavily involved when monkeys spontaneously engage in reward-directed behavior, whereas OFC neurons are more heavily involved when motivational value relies on information provided by visual stimuli.

Figure 10
Encoding of Reward Size across trial types and brain regions

To assess the influence of satiety, a critical internal factor affecting motivational values, we examined changes in behavior and neuronal activity as monkeys accumulated water during the course of a session (T. Minamimoto et al., 2009). Monkeys displayed a progressive decrease in lipping responses and bar release performance as they progressed through a session (Fig. 11A, insets), showing that the value of task events decreases as monkeys accumulate water. In Self-Initiated trials, in both VMPFC and in OFC, the proportion of selective neurons (i.e., showing a significant discrimination) decreased over the course of a session (Fig. 11A, left; 2-way ANOVA, significant effect of progression: F(2)=4.5, p=0.03; significant effect of region: F(1)=13, p=3.5 × 10−3 and no interaction: F(2,12)=0.9, p=0.4). In Cued trials, the proportion of selective neurons decreased in VMPFC but not in OFC, where the proportions of selective neurons across the 3 periods of a session were indistinguishable (Fig. 11A, right; 2-way ANOVA, no main effect of progression: F(2)=0.8, p=0.4; significant effect of regions: F(1)=54, p=1.4 × 10−7 and significant interaction: F(2,24)= 3.5, p=0.04). To examine whether the decreases could be at least partly due to satiety (fatigue would be another obvious contributor to this effect), we recorded another set of neurons (n=14 & 12 in VMPFC and OFC, respectively) while the monkeys were given a large bolus of water (~50% of their usual daily intake) early in the session (before they had completed 200 trials). There was an immediate, significant decrease in performance (average error rate increased from 1.8% ±0.4 to 13% ± 2; t(1)=5.3, p=2 × 10−6). There was also a significant decrease in selectivity (i.e in the amount of variance explained by Reward Size and Action factors or their interaction, 3-way ANOVA) for 8/9 selective VMPFC neurons, whereas the proportion of responding neurons was unchanged in OFC (Fig. 11B). This confirms that the effects observed during the course of a session could at least in part be due to satiety. Thus, VMPFC responses appear to depend upon the amount of fluid received up to the time of the recording. In OFC, the proportion of selective neurons was affected by satiety in Self-Initiated trials, but not in Cued-trials, suggesting that when an external stimulus is present its influence is powerful enough to mask any effect related to an internal factor.

Figure 11
Response modulation with progression in a session and satiety

Discussion

Neuronal activity in both OFC and VMPFC is closely related to the perceived value of task events. OFC neurons emphasize value information arising from visual stimuli, whereas VMPFC neurons emphasize value information arising from intrinsic knowledge or satiety levels. The differences in neuronal activity between OFC and VMPFC provides physiological support for the hypothesis, originally based on anatomy, that OFC and VMPFC play different roles in calculating motivational values.

Our data are compatible with previous reports in monkeys and rats showing that OFC neurons are very sensitive to the value of sensory cues, with little influence of the type of operant response animals must perform in order to get the reward (G. Schoenbaum et al., 1998; L. Tremblay and W. Schultz, 2000; M. R. Roesch and C. R. Olson, 2004; C. Padoa-Schioppa and J. A. Assad, 2006; J. D. Wallis, 2007; S. W. Kennerley and J. D. Wallis, 2009). This is in line with anatomical data showing a strong interactions between OFC and sensory cortices (D. Ongur and J. L. Price, 2000). After the feedback, OFC neurons became more sensitive to the Action factor (Cued Active vs Passive trials), in line with a recent report showing that OFC neurons encode the behavioral response at the time of the feedback (S. Tsujimoto et al., 2009). Here, we show that OFC neurons encode information about the perceived value of both cues and feedback signals, measured using a Pavlovian response. This procedure also revealed that the OFC neurons encode the value of task events in a non-choice situation, even when no movement (or absence of movement) is explicitly required to obtain the reward.

VMPFC neurons are significantly less involved in externally-driven motivational processes, but heavily engaged when value information arises from internal factors such as spontaneous initiation of actions and thirst. This is in line with anatomical data showing a strong interactions between VMPFC, limbic areas and brainstem nuclei involved in autonomic regulation (D. Ongur and J. L. Price, 2000). These data are consistent with functional imaging studies showing that in humans BOLD signals in VMPFC correlate with the feeling of thirst, subjective decision value or ‘self-relatedness’ (D. A. Gusnard et al., 2001; I. E. de Araujo et al., 2003; G. Egan et al., 2003; J. W. Kable and P. W. Glimcher, 2007; V. V. Valentin et al., 2007; T. E. Behrens et al., 2008; G. Northoff and J. Panksepp, 2008; J. Glascher et al., 2009; T. A. Hare et al., 2009). This is also compatible with earlier experiments showing that electrical stimulation of the VMPFC can elicit drinking in sated monkeys (B. W. Robinson and M. Mishkin, 1968).

The value of cues arises mainly from their association with different reward sizes (R. N. Cardinal et al., 2002; K. C. Berridge, 2004). In line with the idea that OFC neurons are more sensitive to external information about event value, the encoding of reward size is stronger and appears earlier in OFC than in VMPFC. The value of the feedback signal depends much more upon the Action factor, i.e upon whether completing the trial required a bar release. This effect could be mediated by inputs from structures controlling the action and/or structures monitoring the movement. In any case, the event defined by the appearance of the feedback signal is more valuable to the animal when it follows an operant response. In line with the idea that VMPFC neurons are more sensitive to internal information about event value, the encoding of the Action factor increased earlier in VMPFC (at the feedback) than in OFC (after the feedback).

In contrast to measuring the value of task events and their associated outcomes as is often done using choice paradigms (L. Tremblay and W. Schultz, 2000; C. Padoa-Schioppa and J. A. Assad, 2006; J. W. Kable and P. W. Glimcher, 2007; J. Glascher et al., 2009; S. W. Kennerley and J. D. Wallis, 2009), here we measure the perceived value of task events using the intensity of an operant bar release and a Pavlovian lipping response. The comparison of these responses in passive and active trials allowed us to distinguish activity related to the value of events in all conditions (measured using lipping) from activity more specifically related to operant, goal directed behavior (B. W. Balleine and A. Dickinson, 1998; R. N. Cardinal et al., 2002; K. C. Berridge, 2004; K. C. Berridge and M. L. Kringelbach, 2008; T. Minamimoto et al., 2009). Although we cannot ask animals about the subjective aspects of these processes, Pavlovian conditioning is widely used as a reflection of emotional processes in animals (R. N. Cardinal et al., 2002; K. C. Berridge, 2004; K. C. Berridge and M. L. Kringelbach, 2008; S. Bouret and B. J. Richmond, 2009). It is reasonable to think that lipping occurs when an event has a positive (hedonic) affective value. The lipping patterns that we observed are consistent with this interpretation. Neuronal activity in both OFC and VMPFC was closely related to the pattern of lipping responses, but firing did not to merely encode motor aspects of lipping. Thus, these data support the idea that the neuronal responses in these 2 regions of the ventral prefrontal cortex are related to hedonic value of task events (R. N. Cardinal et al., 2002; M. L. Kringelbach et al., 2003; S. M. Cox et al., 2005; J. W. Kable and P. W. Glimcher, 2007; T. E. Behrens et al., 2008; K. C. Berridge and M. L. Kringelbach, 2008).

These two areas, OFC and VMPFC, are thought to be involved in assessing information about outcome values during decision processes (A. R. Damasio, 1994; A. Izquierdo et al., 2004; M. F. Rushworth et al., 2007; J. D. Wallis, 2007). However, our data indicate that relatively few neurons directly encode the incentive influence of the reward on the operant bar release response. In other words, ventral prefrontal neurons do not seem to carry information directly relevant to the modulation of goal-directed actions as a function of the expected reward value. The roles that ventral prefrontal areas play in operant aspects of motivation and decision-making could be exerted via their projections to other structures such as ventral striatum, ventral pallidum, premotor cortex or anterior cingulate cortex, where information about value would be integrated with motor information to drive reward-directed actions (M. Shidara and B. J. Richmond, 2002; K. Matsumoto et al., 2003; C. Amiez et al., 2006; S. W. Kennerley and J. D. Wallis, 2009).

We propose that OFC and VMPFC have different roles in motivation: they both seem sensitive to the perceived value of events, with VMPFC critical for subject-centered, internally-driven motivational processes whereas OFC is critical for environment-centered, externally-driven motivational processes.

Acknowledgments

We are grateful to Janine Simmons, Andrew Clark, John Wittig Jr., Narihisa Matsumoto, Walter Lerchner and Mortimer Mishkin for their helpful comments on this work. This work was supported by the Intramural Research Program of the National Institute of Mental Health.

Bibliography

  • Amiez C, Joseph JP, Procyk E. Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex. 2006;16:1040–1055. [PMC free article] [PubMed]
  • Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. [PubMed]
  • Behrens TE, Hunt LT, Woolrich MW, Rushworth MF. Associative learning of social value. Nature. 2008;456:245–249. [PMC free article] [PubMed]
  • Berridge KC. Motivation concepts in behavioral neuroscience. Physiol Behav. 2004;81:179–209. [PubMed]
  • Berridge KC, Kringelbach ML. Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology (Berl) 2008;199:457–480. [PMC free article] [PubMed]
  • Bouret S, Richmond BJ. Relation of locus coeruleus neurons in monkeys to Pavlovian and operant behaviors. J Neurophysiol. 2009;101:898–911. [PubMed]
  • Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002;26:321–352. [PubMed]
  • Cox SM, Andrade A, Johnsrude IS. Learning to like: a role for human orbitofrontal cortex in conditioned reward. J Neurosci. 2005;25:2733–2740. [PubMed]
  • Damasio AR. Descartes' error : emotion, reason, and the human brain. New York: G.P. Putnam; 1994.
  • de Araujo IE, Kringelbach ML, Rolls ET, McGlone F. Human cortical responses to water in the mouth, and the effects of thirst. J Neurophysiol. 2003;90:1865–1876. [PubMed]
  • Egan G, Silk T, Zamarripa F, Williams J, Federico P, Cunnington R, Carabott L, Blair-West J, Shade R, McKinley M, Farrell M, Lancaster J, Jackson G, Fox P, Denton D. Neural correlates of the emergence of consciousness of thirst. Proc Natl Acad Sci U S A. 2003;100:15241–15246. [PubMed]
  • Glascher J, Hampton AN, O'Doherty JP. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2009;19:483–495. [PMC free article] [PubMed]
  • Gottfried JA, O'Doherty J, Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 2003;301:1104–1107. [PubMed]
  • Gusnard DA, Akbudak E, Shulman GL, Raichle ME. Medial prefrontal cortex and self-referential mental activity: relation to a default mode of brain function. Proc Natl Acad Sci U S A. 2001;98:4259–4264. [PubMed]
  • Hare TA, Camerer CF, Rangel A. Self-control in decision-making involves modulation of the vmPFC valuation system. Science. 2009;324:646–648. [PubMed]
  • Izquierdo A, Suda RK, Murray EA. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J Neurosci. 2004;24:7540–7548. [PubMed]
  • Kable JW, Glimcher PW. The neural correlates of subjective value during intertemporal choice. Nat Neurosci. 2007;10:1625–1633. [PMC free article] [PubMed]
  • Kennerley SW, Wallis JD. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci. 2009;29:2061–2073. [PMC free article] [PubMed]
  • Kringelbach ML, O'Doherty J, Rolls ET, Andrews C. Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cereb Cortex. 2003;13:1064–1071. [PubMed]
  • Matsumoto K, Suzuki W, Tanaka K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science. 2003;301:229–232. [PubMed]
  • Minamimoto T, La Camera G, Richmond BJ. Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys. J Neurophysiol. 2009;101:437–447. [PubMed]
  • Northoff G, Panksepp J. The trans-species concept of self and the subcortical-cortical midline system. Trends Cogn Sci. 2008;12:259–264. [PubMed]
  • Ongur D, Price JL. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb Cortex. 2000;10:206–219. [PubMed]
  • Ostlund SB, Balleine BW. The contribution of orbitofrontal cortex to action selection. Ann N Y Acad Sci. 2007;1121:174–192. [PubMed]
  • Padoa-Schioppa C, Assad JA. Neurons in the orbitofrontal cortex encode economic value. Nature. 2006;441:223–226. [PMC free article] [PubMed]
  • Ressler KJ, Mayberg HS. Targeting abnormal neural circuits in mood and anxiety disorders: from the laboratory to the clinic. Nat Neurosci. 2007;10:1116–1124. [PMC free article] [PubMed]
  • Robinson BW, Mishkin M. Alimentary responses to forebrain stimulation in monkeys. Exp Brain Res. 1968;4:330–366. [PubMed]
  • Roesch MR, Olson CR. Neuronal activity related to reward value and motivation in primate frontal cortex. Science. 2004;304:307–310. [PubMed]
  • Rushworth MF, Behrens TE, Rudebeck PH, Walton ME. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn Sci. 2007;11:168–176. [PubMed]
  • Schoenbaum G, Chiba AA, Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci. 1998;1:155–159. [PubMed]
  • Shidara M, Richmond BJ. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science. 2002;296:1709–1711. [PubMed]
  • Tremblay L, Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol. 2000;83:1864–1876. [PubMed]
  • Tsujimoto S, Genovesio A, Wise SP. Monkey orbitofrontal cortex encodes response choices near feedback time. J Neurosci. 2009;29:2569–2574. [PMC free article] [PubMed]
  • Valentin VV, Dickinson A, O'Doherty JP. Determining the neural substrates of goal-directed learning in the human brain. J Neurosci. 2007;27:4019–4026. [PubMed]
  • Wallis JD. Orbitofrontal cortex and its contribution to decision-making. Annu Rev Neurosci. 2007;30:31–56. [PubMed]