|Home | About | Journals | Submit | Contact Us | Français|
Pavlovian cues for rewards become endowed with incentive salience, guiding ‘wanting’ to their learned reward. Usually, cues are ‘wanted’ only if their rewards have ever been ‘liked’, but here we show that mesocorticolimbic systems can re-compute ‘wanting’ de novo by integrating novel physiological signals with a cue’s pre-existing associations to an outcome that lacked hedonic value. That is, a cue’s incentive salience can be re-computed adaptively. We demonstrate that this re-computation is encoded in neural signals coursing through the ventral pallidum. Ventral pallidum neurons do not ordinarily fire vigorously to a cue that predicts the previously ‘disliked’ taste of intense salt, although they do fire to a cue that predicts the taste of previously ‘liked’ sucrose. Yet we show that neural firing rises dramatically to the salt cue immediately and selectively when that cue is encountered in a never before experienced state of physiological salt depletion. Crucially, robust neural firing to the salt cue occurred the first time it was encountered in the new depletion state (in cue-only extinction trials), even before its associated intense saltiness has ever been tasted as positively ‘liked’ (salt taste had always been ‘disliked’ before). The amplification of incentive salience did not require additional learning about the cue or the newly positive salt taste. Thus dynamic re-computation of cue-triggered ‘wanting’ signals can occur in real time at the moment of cue re-encounter by combining previously-learned Pavlovian associations with novel physiological information about a current state of specific appetite.
When attributed with incentive salience, learned reward cues (Pavlovian conditioned stimuli; CSs) can trigger ‘wanting’ for their reward (unconditioned stimulus; UCS) and become attractive “motivational magnets” (Toates, 1986;Robinson and Berridge, 1993;Dickinson and Balleine, 2002b;Mahler and Berridge, 2009) Incentive salience at a given moment depends on current physiological state as well as on learned associative values of a cue. For example, a cue for ice cream may have greater motivational pull (‘wanting’) when the perceiver is hungry or hot.
Natural appetites (e.g., thirst, hunger, salt-appetite) and some drug states (e.g., intoxication priming, long-term sensitization) potentiate the intensity of cue-triggered ‘wanting’ for relevant rewards (e.g., hunger potentiates food cues) (Toates, 1986;Berridge, 2001;Tindell et al., 2005;Mahler and Berridge, 2009). Interestingly, in some new physiological states, enhanced ’wanting’ may occur the first time a relevant CS is re-encountered even before the reward UCS itself is experienced in the new state (Toates, 1986;Robinson and Berridge, 2003;Berridge, 2004). Salt appetite provides a good example (Cannon, 1932;Richter, 1956). When rats are sodium depleted, they will press a salt-associated lever or approach or consume a bitter/sour CS previously associated with intense salt (NaCl). They may do this even if the actual salt UCS is absent and has never yet been re-tasted as ‘liked’ (Krieckhaus and Wolf, 1968;Fudim, 1978;Berridge and Schulkin, 1989;Schulkin, 1991), and even if they remain incapable of more cognitive act-outcome inferences before re-tasting (Dickinson, 1986;Dickinson and Dawson, 1987).
According to the incentive salience hypothesis, this anticipatory motivational revaluation occurs because the Pavlovian CS has acquired incentive-relevant features of its UCS, allowing the CS to interact with physiological states as would its UCS (Bindra, 1978;Toates, 1986;Dickinson and Balleine, 2002b;Berridge, 2004). In other words, CS incentive salience is a computational product that dynamically combines learned CS-UCS associations with current relevant physiological states (Robinson and Berridge, 1993;Berridge, 2001;Zhang et al., 2005;Berridge, 2007;Zhang et al., 2009). Modulation of incentive salience adaptively guides motivated behavior to appropriate rewards. However, when the underlying brain circuitry goes awry, such as in addiction, excessive ‘wanting’ and compulsive pursuit of rewards may result (Everitt and Wolf, 2002;Robinson and Berridge, 2003).
The neural basis for the dynamic re-computation of incentive salience is still unknown. Here we focus on mesocorticolimbic signals that pass through the ventral pallidum (VP), a critical circuit in reward and addiction (Lim et al., 2004;Tang et al., 2005;Kalivas and Volkow, 2005;Zahm, 2006;Heimer and Van Hoesen, 2006;Pessiglione et al., 2007;Mickiewicz et al., 2008;Smith et al., 2009). Ventral pallidum firing patterns encode the incentive salience of Pavlovian cues that predict hedonic rewards, and the modulation of reward impact by physiological states (Tindell et al., 2004;Tindell et al., 2005;Tindell et al., 2006).
Our question here was: can dynamic re-computation of incentive salience be detected in ventral pallidal neural activity correlated to cue presentation? We show that it can.
To test if signals for dynamically re-computed incentive salience are carried in VP circuits, we recorded VP neural responses and behavioral affective reactions to cues and tastes in rats that had learned Pavlovian associations between auditory predictive cues (CSs) and oral infusions of fluids containing either sucrose or intense NaCl (UCSs). One auditory cue (CSsalt) predicted a taste infusion directly into the mouth of a ‘disliked’ intense salt UCS (1.5 M NaCl: an aversive concentration 3 times higher than seawater or an order of magnitude above isotonic). Another auditory tone (CSsucrose) predicted infusion of a ‘liked’ 0.5 M sucrose UCS. A third control tone (CS−) predicted no UCS (Figure 1). Recording took place on two separate days: once in normal physiological homeostasis, and 24 hrs later in a novel sodium-depleted state of salt-appetite (induced overnight). Neural activity was recorded on one day in normal homeostasis and on another day after sodium depletion, using multiwire electrodes implanted into the posterior portion of the VP. The posterior VP was targeted because previous studies showed it contains a 1-mm3 ‘hedonic hotspot’ where mu opioid stimulation causes increase in ‘wanting’ and ‘liking’ for food rewards, and where neuronal firing codes the motivational value of CSs that predict reward, as well as the hedonic impact of UCS taste rewards (Tindell et al., 2004;Tindell et al., 2005;Smith and Berridge, 2005;Tindell et al., 2006;Smith and Berridge, 2007;Smith et al., 2007). Rats had never before experienced a salt appetite prior to the depletion test.
To ensure that we were isolating the dynamic motivational value of the cues alone on a given test day (uncontaminated by any accompanying unconditioned reward), each recording session began with extinction trials (CSs presented with no reward UCSs; “CS only” trials; Figure 1), No UCS stimulus fluid was delivered to the rat during extinction trials. Subsequently, a second series of trials with actual tastes (“CS plus UCS” trials; Figure 1) was presented on the same days to confirm the increase in palatability of NaCl taste by assessing behavioral affective orofacial reactions during VP recordings.
The University Committee on the Use and Care of Animals approved all experimental methods. Male Sprague-Dawley rats (300–450 g; N = 18) were maintained on an ad libitum sodium-free diet (Purina) and distilled water. Rats also had free access to a separate drinking tube containing 0.5 M NaCl solution (except for the 24-hour period when sodium depletion was being induced). Detailed surgical procedures have been described (Tindell et al., 2006). Briefly, animals were anesthetized with a ketamine/xylazine (100 mg/kg/10 mg/kg) for sterile, stereotaxic surgery. Bilateral intraoral cannulae for taste infusions were implanted along with multiwire (50 µm tungsten) electrodes into the posterior ventral pallidum (range of recording sites: AP: −0.11 to −1.53; ML: 1.8 to 3.3; DV: 6.8 to 7.7). This anatomic region encompassed the hedonic hotspot of posterior VP identified by Smith and Berridge (Smith and Berridge, 2005;Tindell et al., 2006;Smith and Berridge, 2007). Anchoring bone screws in the skull served as ground reference electrodes. Animals were allowed 7 days to recover before training and testing began.
Training and testing was conducted in a 25 cm cylinder with a glass floor with a mirror underneath for video recording orofacial taste reactivity and other behavior. A laboratory computer program controlled cue (conditioned stimulus, ‘CS’) and reward (unconditioned stimulus, ‘UCS’) presentations. Rats received 5 training sessions in which they learned discriminative associations between three distinct auditory cues (Figure 1). Two 5-sec cues predicted oral infusions that began at their offset. A third cue (CS−) predicted nothing. The sucrose or salt taste infusion (0.1 ml per infusion at 0.02 ml/sec) lasted for 5 sec. The ‘CSsalt cue’ predicted a concentrated NaCl infusion (1.5 M). A different ‘CSsucrose cue’ predicted sucrose infusion (0.5 M concentration). For each rat, CS assignments were counterbalanced between a 400 Hz, 0.75 sec on/off pulsed tone; a 3800 Hz, continuous tone; and a 8000 Hz, 0.1 sec on/off pulsed tone. Ten pairs each of sucrose cue/infusion and salt cue/infusion were presented randomly (Figure 1) on a variable intertrial interval with a mean value of 1 minute. Each session began and ended with presentation of 5 CS− cues under the same intertrial interval schedule in order to bracket the block of 20 CS+ (10 CSsalt and CSsucrose in random order). There was no gap between the CS+ and CS− blocks other than the 1 min intertrial interval. The bracket design presented CS+ blocks and CS− blocks in a balanced ABBA pattern (CS−, random CSsalt or CSsucrose, CS−) to rule out gradual drift in neural firing over the trial.
Neural activity was recorded in two sessions spaced 24 hours apart. On Day 1, rats were in normal homeostasis; on Day 2, rats were in sodium appetite. A syringe containing sucrose or NaCl solution was connected to the intra-oral delivery tube so as to create a dead-space (extending over about 2 cm of tubing) visible as an air bubble in front of an advancing UCS solution. This bubble, which could be seen clearly through the translucent delivery tube, marked the onset point of the advancing UCS solution. During extinction testing, the position of this bubble space was monitored carefully to be sure that no solution would enter the rat’s mouth until later trials when the pump was turned on. The bubble blocked fluid delivery until the beginning of the reinforced CS-UCS session when the syringe pump was activated to advance the solution. As in training, each test day began with 5 CS− presentations (Figure 1). Those were followed by the extinction block of CS+ alone trials (unaccompanied by UCS). A block of CS-UCS reinforced trials followed, followed by another block of CS− trials (so that CS− presentations bracketed all CS+ and UCS presentations, occurring both before and after them). A 10 min period was imposed between each block.
We emphasize that the CS+ extinction block was the most important test block for assessing dynamic re-computations: it contained 20 CS+ trials in random order (10 CSsalt and 10 CSsucrose), in which CS+s occurred alone without accompanying UCS taste infusions (“CS only” trials; Figure 1). Thus crucially, in extinction tests, rats did not taste any UCS. Instead, rats had only associative information based on previous CS-UCS experiences of CSsalt obtained from pairings on previous days. On those previous days animals were in normal physiological state and thus the association with NaCl tastes had been aversive (i.e., ‘disliked’). In the second phase of each test day actual taste rewards followed the appropriate cues (“CS plus UCS”). CSsalt and CSsucrose trials were intermixed randomly in both phases (10 each; Figure 1, bottom).
Sodium appetite was induced by a combined regimen of the diuretic furosemide (to promote sodium loss and stimulate angiotensin II production), the mineralocorticoid hormone deoxycortisone acetate (to mimic aldosterone elevation), and a sodium-free diet (to prevent sodium replacement). Rats received an s.c. injection of 7.5 mg/kg furosemide and 5 mg/kg deoxycortisone acetate immediately after testing on Day 1 (Flynn et al., 2005), followed 2 hours later by an additional injection of 7.5 mg/kg furosemide (Tamura and Norgren, 1997). NaCl was removed from the home cage after the first injection and not replaced for the next 24 hours. Sodium-free chow (Purina) and distilled water were maintained. We previously found this regimen induces a robust salt appetite over 24 hours that is sufficiently strong to cause ’disliked’ 1.0 M NaCl taste NaCl taste to become positively ‘liked’ and elevate NaCl-elicited VP firing (Tindell et al., 2006). Salt appetite induction was subsequently verified after neural recording by measuring consumption of 0.5 M NaCl solution in the home cage, which was returned after testing on Day 2, and which also served to restore normal sodium balance. A 2000% increase in 24-hr intake of 0.5 M NaCl (mean intake: Day 1 = 1 ml; Day 2 = 22 ml; t15=9.27, p<0.001) confirmed the salt appetite and enhancement of NaCl ‘wanting’.
Taste reactivity (TR) responses to CS and to UCS were evaluated offline in a slow motion, frame-by-frame video analysis (Berridge, 2000;Tindell et al., 2006). Experimenters blind to condition scored hedonic ‘liking’, aversive ‘disliking’, and neutral orofacial movements elicited by CS cues (5 sec) and UCS tastes (5 sec infusion, followed by an additional 5-sec period to capture after-reactions). Hedonic responses included rhythmic midline tongue protrusions, lateral tongue protrusions, and paw licking. Aversive responses included gapes, headshakes, chin rubs, and forelimb flails. Neutral responses included rhythmic mouth movements and grooming (Grill et al., 1992). Behaviors that typically occur in discrete events, including lateral tongue protrusions, gapes, headshakes, chin rubs, and forelimb flails were counted each time they occurred. Behaviors that occur in continuous bouts (rhythmic tongue protrusions, paw licking, rhythmic mouth movements, grooming) were timed in seconds.
Hedonic/aversive reactions were summed into affective categories to form an overall positive hedonic ‘liking’ score and a negative aversive ‘disliking’ score. Other behaviors, including orientation toward CS cues (rearing, approach, head turns), grooming movements (mouth, limbs) were totaled separately. Mixed-design ANOVAs (day=between subject, taste=within subject) and Bonferroni-corrected post-hoc tests were performed to evaluate effects of the cues (CSsalt, CSsucrose, CS−), UCS tastes (salt, sucrose), and physiological condition (baseline-replete, depletion, re-repletion) on taste reactivity and other behavioral responses. Latencies to the first unconditioned response after infusion onset were also compared across tastes and test sessions using ANOVA.
Neural units were separated from each other and background noise with Offline Sorter (Plexon Inc., Dallas TX). Unit activity was analyzed with NeuroExplorer (Nex Technologies, Littleton MA). Single unit adherence was verified by distinct spike waveforms and clear refractory periods in an autocorrelation histogram. Cross-correlation analyses were performed to ensure that all neurons were counted only once in the analyses. We did not presume that the same individual units were recorded on separate days, but instead compared data as though different units were recorded on different days, because of the impossibility of being certain of neuronal identity across sessions. We adapted this conservative group design approach for all analyses described below.
Firing changes in response to CS cues were computed for each trial in a bin 500 msec wide beginning at the onset of the CS cue. For response to tastes, a 5 sec bin began at the onset of taste infusion period (or temporally matched period in extinction trials that had no-infusion). Firing rate changes in each trial were normalized by dividing by baseline rate (average across trials of 5 sec period before cue). A neuron was defined as responsive to a cue or taste if the firing rate across trials during the examined bin was significantly different (p<0.05 in Bonferroni-corrected paired t-tests) from the average firing rate during background (5 sec before infusion).
To assess firing rate effects, we averaged the normalized firing rate for each responsive unit across trials (except when comparing trial effects explicitly), and compared response magnitudes across stimuli and test days using mixed-measures ANOVA (day/state was treated as a between-subjects factor because of uncertainty of whether the same neurons were recorded on different days; stimuli comparisons on the same day were treated as a within-subject factor); post hoc comparisons were conducted by Bonferroni-corrected tests.
For population comparisons, numbers of neurons activated by the cues and/or tastes were tallied for a binomial analysis and compared across extinction vs. normal trials and physiological conditions using ANOVAs.
After recordings were completed, rats were given an overdose of pentobarbital, and perfused transcardially with saline and formaldehyde. Brains were removed, sliced in 40 µm coronal sections, and stained with cresyl violet. Slices were examined under a microscope to verify electrode placement in VP, and electrode sites were mapped onto a computerized brain atlas (Paxinos and Watson, 2007).
Most VP neurons responded phasically to auditory CS cues, rising within 100 msec of tone onset and decaying nearly back to baseline by 500 msec, although the physical tone continued another 4.5 sec (Figure 2). Even in extinction presentations of a CS by itself, 52% of neurons (66/128) were activated by a CS during the first phase of each test. Many neurons were responsive to more than one cue (41%; Figure 3). When cues were followed and reinforced with actual UCS tastes, in the second phase of each test day, the proportion of responsive neurons increased to 79% (97/123). VP responses to UCS tastes were slower and more prolonged than to CS tones, tending to last for much of the 5-sec duration of the UCS oral infusion. Excitation in firing rate was the predominant response to both CS and UCS stimuli.
Previous studies indicated that at least two neural coding schemes in ventral pallidum may carry information about reward events: firing rates in individual neurons (i.e., rate coding), and recruitment of more participant neurons to enlarge the information-carrying network (i.e., population coding) (Tindell et al., 2004;Tindell et al., 2006). We describe population coding data for all stimuli and trials first, and then rate coding for the same stimuli and trials.
Even though no tastes were delivered in extinction trials (‘CS only’), VP neurons responded discriminatively to CS cues (46% of units in homeostasis and 53% in salt-depleted conditions; main effect of cue across all trials F2,127=10.145, p<0.001; Table 1). In normal homeostasis, the cue that previously predicted sucrose (CSsucrose) activated three times as many neurons as CSsalt (42% vs. 14%; posthoc p<0.05. df=127). Sodium depletion dramatically changed this relationship by selectively doubling the number of CSsalt-responsive neurons in extinction trials from 14% on Day 1 to 31% on Day 2 (posthoc p<0.05 df=127). Only the salt cue population expanded in the novel state. The sucrose cue continued to be a robust stimulus on Day 2 during the sodium depletion test, activating 42% of VP neurons. That proportion was not significantly changed from the proportion observed during CSsucrose extinction trials in normal homeostasis on the previous day (no overall interaction: F2,127=1.869, p=0.156, posthoc p>0.05, df=127). The control cue (CS−), which predicted no taste, activated 21% of VP neurons on Day 1 and 18% on Day 2. CS− populations were smaller than the CSsucrose population on the normal day, and smaller than both CSsucrose and CSsalt populations on the sodium-depleted day (each posthoc cue comparison p<0.05 df=127). Thus, the sizes of VP populations activated by cues coded the incentive value of each CS appropriate to the rats’ physiological state on that day. It is important to note that all oral infusions of NaCl taste had been unpalatable on previous presentations in the normal state, yet the population to the CSsalt still doubled in the depletion state during its first presentations in extinction.
In the second phase of testing on each day, UCS infusions followed their CS cues, as in training, to allow assessment of VP coding of sucrose and NaCl tastes in relation to their predictors. Most neurons were activated by both cues and tastes (67% across both test days, N=83/123; main effect: F4,122=12.367, p<0.001).
The size of the population responsive to tastes of salt versus sucrose UCS depended on the physiological state of the day (interaction: F4,122=8.256, p<0.0001). During normal homeostasis on the Day 1 test, sucrose infusions (31/63 [49%]) activated more than twice as many VP neurons as NaCl infusions (13/63 [21%]; posthoc p=0.001 df=62, Figure 3). In contrast, during the sodium depletion test on the next day, the VP population activated by NaCl infusions more than doubled from the previous normal day (32/60 [53%]), and was actually 50% larger than the sucrose population (19/60 [32%]; posthoc p=0.02 df=60, Figure 3). This change replicates our previous report that VP population size codes salt palatability (Tindell et al., 2006), and reflected both an increase in the proportion of neurons responsive to salt tastes on the sodium depleted test day (posthoc p<0.0001 df=122), and a simultaneous decrease in the proportion of neurons responsive to sucrose tastes (posthoc p<0.05 df=122). Thus, the ventral pallidum population code represented the conversion of concentrated salt solution from a bad taste to a good taste.
The most dramatic evidence for dynamic re-computation of CS incentive salience came from firing rates of responsive VP neurons, confirming that a VP rate code sensitively represents the incentive salience of reward cues (overall main effect on model, F4,127=4.848, p=0.011).
On the first test day when rats were in normal homeostasis, sucrose cues (CSsucrose) evoked brief elevations of firing within 50 to 100 msec after onset of a CS+ tone (Figure 2). Peak values were reached by 250 msec and rates declined toward baseline by 500 msec (Figure 2, ,44 and and5).5). In some cells firing rates decreased with CSsucrose tone onset. On average though, the initial response was a marked increase that reached nearly 40% above background (posthoc p=0.020; df =25, Figure 4a, 4c). In contrast, the salt cue (CSsalt) evoked activity in only 23% of cells on Day 1 normal homeostasis (Table 1). As with sucrose tones both excitation and inhibitions were observed; however with CSsalt responses, they essentially cancelled each other out so that on average there was no net change in firing rate compared to pre-cue baseline (posthoc p=0.812 df=25, Figure 4a, 4c). Rates to the CS− also never significantly changed from baseline. Thus, on Day 1 average firing rates to CSsalt and CS− were equally low (posthoc p=0.367 df=25). Firing rates to CSsucrose, on the other hand, were higher than either CSsalt (posthoc p=0.018 df=25) or CS− (posthoc p=0.020 df=25).
The CSsucrose and CSsalt rate pattern changed with the induction of the novel sodium depletion state on the second test day. This change was highlighted by a dramatic and selective initial burst of activity to the CSsalt. The firing rates of this response to the salt cue on the salt appetite day almost doubled from the previous day in normal homeostasis (Figure 4c; posthoc on day p<0.0001 df=63). CSsalt elicited firing rose to 170% above baseline within 250 msec, and then fell to baseline levels by 500 msec after tone onset. This phasic response to salt cues resembled the CSsucrose peaks, which did not change with salt depletion. On Day 2 firing rates to CSsalt did not differ from CSsucrose (posthoc p=0.220 df=63, Figure 4a; 4b; 4c). Activity evoked by the CS− control tone in sodium depleted state was similar to normal homeostasis except for a slight but significant increase in rate during the first 500 ms (posthoc p<0.05; df=63, Figure 4d). This change may indicate some response generalization between the CS+s and CS− tones.
The elevation in firing to CSsalt was evident even on the first “CS only” trials in extinction testing. For example, on the very first extinction trial after depletion, firing to the CSsalt was markedly enhanced (154%) above level seen for the first presentation in homeostasis the day before (Figure 5). Firing rate was higher on each of the 10 CSsalt trials in depletion state than in normal homeostasis, though there was a partial decline in normalized firing from trial 1 to trial 10 on both days (Figure 5, ANOVA across trial F9,136=2.507, p=0.011). The immediate and strong response to CSsalt in the first extinction trials demonstrates that the learned Pavlovian association between CSsalt and its UCS salt taste could be used to re-compute a revised representation for this cue even before UCS salt had been tasted in the new appetite state.
The firing changes evoked by CS tones sometimes persisted at lower plateau levels for 4 to 5 sec throughout the entire CS presentation after the initial phasic 0.5 sec response peak, and those prolonged changes could occur as either excitatory plateaus or inhibitory troughs. For example, the CSsucrose elicited a prolonged excitatory plateau lasting roughly 5-sec on both test days, at a level which was statistically elevated above baseline on the sodium depleted day (p<0.0001, df=38) and marginally elevated on the normal homeostatic test day (p=0.052, df=25). By contrast, on the normal homeostatic test day the CSsalt elicited an opposite inhibitory trough in prolonged firing that fell statistically below baseline during most of the 5-sec tone; gray shaded region Figure 4a, p=0.007, df=25), but did not do so on the sodium depleted day (i.e. no inhibitory 4.5 sec trough after the phasic 0.5 sec excitatory peak to CSsalt; p=0.055; df=38, Figure 4b).
In extinction tests (“CS Only”), the moment a taste UCS would have occurred during training represents a UCS omission at 5 sec after CS+ onset, a situation that permits the possibility of a negative prediction error as has been reported for mesolimbic dopamine neurons (Schultz, 2006). However, we did not observe a true prediction error in VP firing on extinction trials. Although there was a small initial decrease after CSsucrose offset, rates at the moment of UCS omission did not differ from the precue baseline. After a CS+ that elicited a strong firing peak (CSsucrose on both days; CSsalt on depletion days), firing rates declined from their CS+ peak levels toward baseline at the moment of UCS omission (p<0.005; df = 25 or 38), but never dipped below the pre-cue baseline.
On CS-UCS reinforced trials, the firing rates to salt UCS were enhanced by sodium depletion, as well as to CSsalt, whereas firing to sucrose UCS infusions and to its CSsucrose remained equally high on both days.
The UCS tastes elicited firing rates proportional to their hedonic palatability in each physiological state. During normal homeostasis on the first test day, sucrose taste infusions elicited faster VP firing rates than NaCl taste infusions (Tindell et al., 2006) (posthoc on taste from above, p=0.004 df=25, Figure 6). Sucrose UCS elevations exceeded 50% above baseline within the first 1 sec of oral infusion, and remained approximately 25% above baseline for most of the 5 sec duration of sucrose infusion. Salt UCS infusion did not elicit significant changes from baseline in firing when rats were in a normal homeostatic state on the first day.
After sodium depletion on the second day, this UCS rate pattern reversed so that NaCl taste infusions actually elicited slightly higher firing rates than sucrose taste, similar to our previous report (Tindell et al., 2006) (day/taste interaction: F2,127=18.946, p<0.0001; posthoc p<0.0001 df=38, Figure 6). The reversal in UCS firing rates involved both a major increase to NaCl taste (posthoc p=0.001 df=63), and a minor decrease to sucrose taste (posthoc p<0.0001 df=63). During sodium depletion, salt taste elicited firing peaks of over 60% above pre-cue baseline within 2 sec, and remained 20% to 40% above baseline level for most of the 5-sec infusion (as well as above sucrose UCS levels) (Figure 6). Thus sodium depletion specifically reorganized rate coding of UCSs in a manner that reflected the new relative hedonic values of sucrose and salt tastes, similar to population coding. Firing to CSs followed the pattern described above for extinction tests.
Sodium depletion produced no changes in absolute baseline firing rates (measured in the absence of CS or UCS stimuli: 5-sec periods prior to presentations of CS cues or UCS taste infusions (F1,297=1.006, p=0.317 for extinction phases; F1,296=0.361, p=0.548 for reinforced phases). Baselines within a trial similarly remained stable from before to after UCS taste infusions (CS Only: F2,297=0.047, p=0.954; CS Plus UCS: F2,296=0.417, p=0.660). In short, VP baselines were quite stable in this experiment, and none of the CS or UCS effects described above could be ascribed to changes in baseline firing.
The relative hedonic impacts of taste UCS stimuli was confirmed by video analyses of affective orofacial behavioral reactions. Sucrose taste infusions always elicited a high number of positive hedonic affective reactions, during both homeostasis and depletion states (e.g., rhythmic or lateral tongue protrusions, paw licking). By contrast, NaCl infusions elicited mostly negative aversive reactions (e.g., gapes, headshakes, forelimb flails) on the first test day when rats were in normal homeostasis. After sodium depletion, however, the NaCl valence reversed to predominantly positive reactions (interactions: Hedonic: F2,30=5.991, p=0.006, df =2; Aversive: F2,30=39.982, p<0.0001, Figure 7). On the salt appetite day NaCl elicited as many positive hedonic reactions as sucrose infusions (Day 2 hedonic post hoc p=0.372,df=15). Sodium depletion caused both the number of positive hedonic reactions to NaCl taste to dramatically rise (p=0.002 df = 31; Figure 7a), and the number of negative aversive reactions to fall to near zero (p<0.0001 df =30; Figure 7b). This pattern confirmed that intensely salty taste switched in palatability from relatively ‘disliked’ to ‘liked’ as a consequence of the sodium-depleted physiological state, whereas sucrose remained constantly ‘liked’. Generally, affective reactions were not elicited in significant numbers by CS tones without UCS tastes.
To ascertain whether VP firing reflected changes in stimulus processing or motor reactions, we assessed neuronal activation during movements identified by video. VP neuronal activity was not strongly associated to any spontaneous movements such as head turns, forelimb movements or mouth or tongue movements, confirming that VP firing does not primarily code movements per se (Tindell et al., 2004;Tindell et al., 2006). For example, only 2% of VP units that responded to CS cues were activated during spontaneous head turn movements scored outside of a CS. Even during cues or UCS stimuli, VP firing typically occurred 500 msec or more before any elicited movement, which may be too early for a movement-coding signal. Further, neural responses to CS stimuli were equally strong regardless of whether any observable movements occurred or not. Thus we conclude that the intensity of VP firing to stimuli did not tend to reflect movement parameters, but rather most likely reflected the motivational value features of CS tones and UCS tastes as described above.
We observed no elicitation of hedonic ‘liking’ reactions when CSs tones were presented alone. Still, we leave open the question of whether the increase in CS ‘wanting’ was matched by any increase in CS ‘liking’ or hedonic impact (Fudim, 1978;Toates, 1986;Berridge and Schulkin, 1989). Lack of orofacial ‘liking’ reactions helped prevent motor confounds in interpreting VP firing, and so was useful here for isolating cue-triggered ‘wanting’ signals. However, pure auditory tones are not the best CSs for evoking conditioned hedonic reactions, and it remains possible that future studies employing CSs with oral or gustatory components might yet reveal CS ‘liking’ in the future (Delamater et al., 1986;Holland et al., 2008).
Our results demonstrate that neural signals within ventral pallidum (VP) circuits encode incentive salience for a reward-predicting cue (CS), by patterns of neural firing in the reward-related anatomical hotspot of posterior VP (Smith et al., 2009). They also demonstrate that representations of incentive salience can be dynamically re-computed at the moment of cue re-encounter if the incentive value of the cue is physiologically altered. Here, we provoked re-computation that inverted the incentive value of a specific cue (CSsalt). CSsalt normally predicted the taste of aversive salt solution (UCS); however, by inducing sodium depletion, a physiological state that was novel to the animal, the incentive value of the salt-predicting cue was selectively enhanced. When the salt cue was suddenly re-encountered in this new salt-appetite state, neurons fired within 100-msec to CSsalt as vigorously as they ever did to the sucrose cue (CSsucrose), nearly doubling firing rate on average during a 250-msec initial peak. This ability to specific re-computation of incentive value could potentially guide behavior adaptively toward ‘wanted’ cues for a reward that would be highly ‘liked’ at that moment.
The most important feature of our results was to demonstrate that re-computation of the cue’s incentive salience was expressed by VP neurons even before the rats had ever tasted its associated salty UCS as positively ‘liked’. The rats had not yet re-encountered the NaCl taste in the new appetite state when extinction tests exposed them to the CSsalt (salt cue) presented alone. Based solely on an accumulated cache of valenced associations between the salt cue and its previously unpleasant taste, therefore, VP neurons should not have exhibited firing peaks to CSsalt. The change in VP firing reveals a dynamic revision of CS incentive value in the appetite state, which directly reversed the valence of previous associations. Our results provide the first direct evidence for a neural mechanism capable of such dynamic motivational transformations.
To understand why this transformation occurred and was predicted by our hypothesis, it is useful to consider how incentive salience is thought to relate to a Pavlovian CS (Robinson and Berridge, 1993;Berridge, 2001;Berridge, 2007;Robinson and Flagel, 2009;Mahler and Berridge, 2009). The incentive salience of a previously-learned CS is actively attributed at the moment of cue re-encounter by mesocorticolimbic circuits that respond to the CS, which funnel through the VP. However, cached Pavlovian associations provide only half the input to the calculation of incentive salience. The re-computation also draws on current states of physiological appetite and satiety, and corresponding neurobiological states of mesolimbic brain systems that normally process the incentive value of relevant rewards. That combination dynamically determines the incentive salience attributed to a particular cue at its moment of re-encounter. Because the Pavlovian CS associatively takes on incentive-relevant properties of its UCS, the CS can often interact with relevant physiological states as its UCS would. As a consequence, a CS can be re-valued by some physiological states, such as specific appetites, even when the state is completely novel and the UCS is absent (Krieckhaus and Wolf, 1968;Fudim, 1978;Toates, 1986;Berridge, 2001).
Applied to neuronal signals, a novel state may transform cue-evoked neural representations of saltiness and endow the cued signal with a motivation value appropriate to what its UCS would carry at the moment. This can directly revalue a relevant CS – even before the UCS is re-encountered in the new state – making the cue a more potent attractor that is able to pull behavior and guide approach, or to trigger increased ‘wanting’ of its UCS goal. Such pre-emptive neural coding could therefore adaptively contribute to smart and rapid guidance of behavior toward appropriate rewards in appetite states.
In more formal computational terms, an implication of our results is that the incentive salience of a Pavlovian cue may equal its previously learned value if and only if relevant physiological states are the same during learning as in subsequent tests. This conclusion is different from perspectives that equate incentive value to cached accumulations of previous learned values, such as the popular temporal difference model of reward learning (prediction error model):
If motivation simply always equaled the accumulated cache of prior rewards, then the incentive salience value of a cue would simply sum all previously learned values (rt) (McClure et al., 2003;Daw et al., 2005;Redish et al., 2008). But Pavlovian motivation is not so stable (Niv et al., 2006;Niv, 2007). In particular, the dynamic reversal of CS incentive salience shown here requires something more. To accomplish that reversal, Zhang et al. recently proposed a model that more accurately re-computes incentive salience at the moment of cue re-encounter (Zhang et al., 2009). That dynamic model explicitly incorporates a physiological factor (κ) reflecting current hypothalamic and mesocorticolimbic states: Thus, a current physiological state can interact with previous associations of a CS with relevant reward (rt) to transform its cue-triggered incentive value (V). The interaction can be encapsulated roughly as (rt, κ). The κ factor allows the incentive salience of an associated CS to be adjusted as physiological states emerge or disappear (e.g., hungers, satiety, drug intoxication, mesolimbic sensitization, etc.). Here, the specific reversal of ‘wanting’ triggered by the salt cue from negative to positive could be expressed via an additive interaction for (rt, κ), according to the model of Zhang et al. (2009):
Cue-triggered “wanting’ is thus transformed in valence and intensity by a relevant state factor. ‘Wanting’ is also focused in an appropriate direction (e.g., toward salt), because the κ factor is specific to its own particular CS and UCS reward combinations. Thus, a specific appetite for salt need not alter incentive computations for sucrose CSs, nor a caloric hunger state alter the value of salt CSs. Similarly, drug addicts may excessively ‘want’ drugs most of all, whereas binge eaters particularly ‘want’ food.
We note that other reward-related learning processes (e.g., cognitive expectations and act-outcome representations or simpler associative S-R habits) also operate alongside Pavlovian-guided incentive salience, using different rules and different brain systems. Some, such as cached habits or cognitive-based instrumental learning, may require actual re-tasting of a revalued UCS to change (Dickinson and Dawson, 1987;Berridge, 2001;Dayan and Balleine, 2002;Dickinson and Balleine, 2002a). By contrast, the CS incentive salience dynamically changed here in advance of UCS re-tasting.
Signals in the VP reflect circuits from other mesocorticolimbic components, including orbitofrontal and insular cortex, ventral tegmentum, and nucleus accumbens (Groenewegen et al., 1993;Kalivas and Volkow, 2005;Zahm, 2006). Salt appetite signals, such as angiotensin II and aldosterone, activate the subfornical organ, extended amygdala, and brainstem sites to recruit limbic circuits of motivation (Johnson et al., 1999;Krause and Sakai, 2007). Recruitment includes enhancement of dopamine and opioid neurotransmission in the nucleus accumbens shell and in neostriatum (e.g., reduced dopamine transporter binding and increased enkephalin mRNA expression), which might help modulate ‘wanting’ and ‘liking’ encoded in the firing of VP neurons (Lucas et al., 2003).
A powerful way for drugs of abuse and addiction to usurp brain circuitry of natural rewards would be to hijack VP-related circuits that evolved to dynamically modulate the motivation value of natural incentives. Although salt appetite is relatively unique as a specific appetite, the principles demonstrated here are shared by a more general class of ‘wanting’ re-computations. The class includes several other natural appetites (e.g., food hunger, water thirst) and also addiction-related states (e.g., mesolimbic sensitization; drug intoxication) (Berridge, 2001;Robinson and Berridge, 2003;Tindell et al., 2005;Zhang et al., 2009).
The incentive-sensitization theory of drug addiction posits that drug sensitization usurps mesocorticolimbic circuits to dynamically amplify the attribution of incentive salience to drug cues, producing excessive cue-triggered ‘wanting’ to take drugs, just as biological sodium hunger here amplified the value of a salt cue (Robinson and Berridge, 2003;Tindell et al., 2005). In support of this convergence, we note that we previously reported drug sensitization to produce an incentive salience enhancement for a reward cue that was similar to the salt cue enhancement shown here, as a dynamic integration of (rt, κ) (Tindell et al., 2005). It also seems interesting that repeated sodium depletions may produce cross-sensitization to psychostimulant drugs (Bernstein, 2003;Clark and Bernstein, 2006). Thus natural salt appetite and drug sensitization and addiction may share underlying mesolimbic mechanisms. However, we also note a difference in that natural salt appetite enhances hedonic impact (‘liking’) as well as incentive salience (‘wanting’) for a relevant reward, whereas drug sensitization may enhance only ‘wanting’ for targeted rewards (Robinson and Berridge, 2003;Tindell et al., 2005). Either may be sufficient to pull animals in a state of sodium deficiency toward a salt lick or pull addicts relentlessly toward drug-related cues and their addictive targets.
In conclusion, VP signals encode the current incentive salience of appropriate reward cues, in ways that can dynamically reverse a CS cue from ‘unwanted’ to ‘wanted’. Firing patterns of VP-related circuits may in this way adaptively modulate motivation to guide behavior toward relevant goals. As a downside, this may also create vulnerabilities to addiction to drugs that usurp the VP-related capacity for dynamic revaluation to cause excessive ‘wanting’ for particular rewards.
The authors would like to acknowledge the assistance of Jonathan Lee for computational data analysis and manuscript preparation, and Drew Thodeson and Nadia Siddiqui for experimental assistance. We also acknowledge our colleague Jun Zhang, who developed the computational model described here. The research was supported by NIH grants, DA015188, DA017752, and MH63649 (J.W.A. and K.C.B.), National Science Foundation Fellowship (A.J.T.), and NIH training grant DC00011 (K.S.S.).