PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Neurosci. Author manuscript; available in PMC 2011 July 5.
Published in final edited form as:
PMCID: PMC3078720
NIHMSID: NIHMS261998

Extracellular Dopamine Levels in Striatal Subregions Track Shifts in Motivation and Response Cost During Instrumental Conditioning

Abstract

Tonic dopamine (DA) signaling is widely regarded as playing a central role in effort-based decision-making and in the motivational control of instrumental performance. The current study used microdialysis to monitor changes in extracellular DA levels across subregions of the nucleus accumbens and dorsal striatum of rats as they lever pressed for food reward on a probabilistic schedule of reinforcement, a procedure that ensured they would experience variation in the amount of effort needed to earn rewards across tests. Each rat was given three tests. Rats were hungry for the first and last test, but were sated on food before the middle test, allowing us to assess the effects of a downshift in motivational state on task performance and conditioning-induced DA efflux. During hungry tests, DA levels rose in both the shell and core of the accumbens and, to a lesser degree, in both the medial and lateral divisions of the dorsal striatum. Interestingly, changes in DA efflux across hungry tests in the accumbens core were negatively correlated with changes in the effort required to obtain rewards. We also found that – across regions – the DA response to instrumental conditioning was attenuated when rats were sated prior to testing. Furthermore, the effect of satiety on DA efflux in the accumbens shell was positively correlated with its effect on task performance. Altogether, the results indicate that tonic DA contributes to the control of instrumental performance by conveying information about the costs and benefits of responding to different striatal subregions.

Keywords: microdialysis, nucleus accumbens, dopamine, reward, operant, striatum

Striatal dopamine (DA) signaling is known to be involved in effort-based decision-making and in regulating the vigor with which instrumental actions are performed (Salamone et al., 2007; Niv et al., 2007; Walton et al., 2006). Microdialysis studies have found that nucleus accumbens DA levels rise as rats lever press for food rewards, with a few studies finding a positive correlation between the magnitude of conditioning-induced DA efflux and response rate (McCullough et al., 1993; Sokolowski et al., 1998; Cousins et al., 1999), suggesting that DA may provide a response-activating function. However, in apparent contrast to these findings, studies using fast-scan cyclic voltammetry have found evidence that DA may encode the cost of responding, with high-effort actions being accompanied by equal or smaller elevations in DA than low-effort actions (Gan et al., 2009; Day et al., 2010).

In free operant performance, response rate is modulated by motivational state. Rats will, for example, press a lever that delivers food pellets more rapidly when they are hungry than when they are sated. Not surprisingly, striatal DA has been assigned an important role in mediating the effects of primary motivation on instrumental performance (Niv et al., 2007; Willner et al., 1988), with some researchers suggesting that its involvement may be limited to the motivational support provided by stimuli paired with reward (Dickinson et al., 2000). If so, then conditioning-induced DA efflux should be modulated by motivational state. While there is some support for this claim (Wilson et al., 1995; Ahn & Phillips, 1999), existing studies have used passive feeding tasks, which may not engage the same motivational processes as instrumental conditioning and provide no information about the relationship between DA efflux and instrumental performance variables.

Although the dorsal striatum receives strong DAergic projections and has been implicated in the control of instrumental performance (Balleine et al., 2009; Yin et al., 2008), little is known about the role of dorsal striatal DA signaling in instrumental control. We used microdialysis to monitor changes in striatal DA levels in rats lever pressing for food rewards, targeting sites in both the nucleus accumbens (core or shell) and dorsal striatum (lateral or medial). Rats received three test sessions on separate days using an ABA design. The first and last tests were conducted when the rats were hungry, allowing us to characterize conditioning-induced DA efflux in these areas under a state of high food motivation. Prior to the second test, rats were fed to satiety so that we could determine if the DA response to instrumental conditioning was modulated by motivational state. This design also allowed us to determine if changes in DA efflux across tests varied with changes in either the rate of lever pressing or number of rewards earned per session. Furthermore, our use of a probabilistic (random ratio) reinforcement schedule ensured that the rats would experience variability in the number of responses (or effort) required to earn rewards across tests, allowing us to explore the relationship between conditioning-evoked DA efflux and response cost.

Methods

All procedures were approved by UCLA’s Animal Research Committee and were performed in accordance with National Institute of Health Guide for the Care and Use of Laboratory Animals.

Subjects and surgery

Male Long-Evans rats (approximately 90 days old at the beginning of the experiment; Harlan Laboratories, Indianapolis, IN) were housed singly in a temperature- and humidity-controlled vivarium. Experiments were conducted during the light phase of a 12 hr light/dark schedule. Rats were deeply anesthetized with isoflurane inhalation and implanted unilaterally (left or right hemisphere, counterbalanced across placements) with a guide cannula (CMA 11, Stockholm, Sweden) lowered into the core (AP: +1.2, ML: + or −1.8, DV: −6.5) or shell (AP: +1.7, ML: + or −0.8, DV: −6.5) of the nucleus accumbens, or the dorsolateral (DLS)(AP: +0.7, ML: + or −3.6, DV: −4.0) or dorsomedial striatum (DMS) (AP: −0.4, ML: + or −2.6, DV: −3.5) (all coordinates in mm relative to bregma). Dental cement was used to form a head cap, which sealed the wound and anchored the guide cannula and a tethering screw to three stainless steel screws placed in the skull. Rats were given 5–7d to recover before behavioral training. During the recovery period, they were placed on food restriction regimen (~12g of home chow per rat per day) to maintain them at approximately 85% their free-feeding bodyweight throughout training and testing.

Instrumental training

All behavioral training and testing sessions were conducted in identical Med Associates chambers (East Fairfield, VT). Rats were given a 180-min session of magazine training, during which grain-based food pellets (45 mg, Bio-serv, Frenchtown, NJ) were delivered on a 5-min random-time (RT) schedule into a recessed food magazine. A 2-sec clicker stimulus (10 Hz; ~75dB) accompanied pellet deliveries during all training and testing sessions. They were then given 7 d of instrumental conditioning. Each session began with the insertion of a retractable lever located to the left of the food magazine. Rats were allowed to earn 50 pellets on a continuous reinforcement schedule in session 1. Reinforcement was shifted to a probabilistic, random ratio (RR) schedule for the rest of training and testing, with the probability of reward given lever press performance decreasing over sessions: a RR-5 schedule [p(reward/press) = 0.20] was used in sessions 2–4, and a RR-10 schedule [p(reward/press) = 0.10] was used in sessions 5–7. Rats were allowed to earn up to 30 pellets in each of sessions 2–7. Each training session began after an initial context exposure period (60 min in session 1 and 90 min in sessions 2–7). The rats were tethered to a liquid swivel during sessions 6–7 to habituate them to this experience. The houselight (3 W, 24 V) was illuminated as soon as the rats were placed in the chamber and was extinguished once all of the available outcomes had been earned.

Microdialysis procedures and behavioral testing

After the last session of instrumental training, rats were implanted with a single CMA 11 microdialysis probe (length 2 mm, o.d. 0.24, molecular cut-off 6 kDA, CMA), attached to a dual channel liquid swivel (Instech Laboratories, Plymouth Meeting, MA) with FEP tubing (i.d. 0.12 mm, CMA). Connections were made using short lengths of PVC tubing. The rats were then individually housed in transparent Perspex cylindrical containers (height: 38 cm, diameter: 27 cm; subsequently referred to as home cages) in the vivarium for the remainder of the experiment when not undergoing dialysis testing. Rats were given their daily allotment of food after probe implantation. The microdialysis probe was continuously perfused with artificial cerebrospinal fluid (aCSF; 125 mM NaCl, 2.5 mM KCl, 0.9 mM NaH2PO4, 5 mM Na2HPO4, 1.2 mM CaCl2, 1 mM MgCl2, pH 7.4) for three days. A flow rate of 2.0 μl/min was used during behavioral testing. The flow rate was reduced to 0.3 μl/min between tests. On the day after probe implantation, rats were given the first of three test (conditioning) sessions, which were conducted at the normal time of training over consecutive days (i.e., at intervals of approximately 24 hours). For each session, rats were placed in the operant chamber, with the liquid swivel mounted on a balance arm attached to the side wall. After a 2h context habituation period, the lever was inserted into the chamber and rats were allowed to lever press for food pellets on a RR-10 schedule for 30 min. No limit was placed on the number of outcomes that could be earned. The lever was then retracted and the houselight was extinguished. The rats were left in the chamber for an additional 30 min before returning them to the vivarium. The three test sessions were identical, with one important exception: rats were hungry for Tests 1 and 3, but were sated on food for Test 2. Specifically, rats were given their daily allotment of food in the home cage shortly after Test 1 was complete. On the following day, rats were given 12g of home chow 2h before they placed in the chamber for Test 2. Any uneaten food was removed from the home cage at this time. Since no additional food was provided after Test 2, the rats entered test 3 having been deprived of food for approximately 23 h.

Dialysate sampling and electrochemical detection

Dialysate was collected into refrigerated (8 °C) tubes containing 0.6 μl of 12.5 mM perchloric acid/250 μM EDTA at 3-min intervals (6 μl per sample) before (3 samples), during (10 samples), and after (10 samples) each test session. Collection times were adjusted to account for the estimated time lag in flow from the probe membrane to the FEP tubing outlet (about 9 min). High-performance liquid chromatography with electrochemical detection (Antec Leyden, Leiden, The Netherlands) was used to determine the dialysate concentrations of DA, as previously described (Murphy & Maidment, 1999). The mobile phase (17% Acetonitrile, 0.68 mM sodium dodecyl sulfate, 10 μM EDTA, 35 mM phosphate buffer, ph 5.5) was pumped (Shimadzu model LC-10 AD) at a rate of 0.2 ml/min through a 2 × 100 mm column (2 μm particle size, TSK-GEL Super ODS, Tosoh Biosciences, Tokyo, Japan). The working electrode was maintained at +450 mV. Standard concentrations of DA were analyzed with each set of test samples. For each test, dopamine levels were normalized as a percentage of the average of the three pre-session (baseline) samples.

Histology

After the experiment, rats were given an overdose of sodium pentobarbital (100 mg/kg, i.p.). Their brains were removed, post-fixed and cryoprotected in a 30% sucrose-formalin solution, and cut into 50 μm coronal sections across the brain region being targeted. Sections were mounted on glass slides, stained with cresyl violet and analyzed under a light microscope to determine probe placement.

Results

Probe placement

Figure 1 presents the individual microdialysis probe placements for all rats. In general, the probes were successfully implanted in the targeted brain area, either the accumbens core (n = 6), accumbens shell (n = 7), DLS (n = 7), or DMS (n = 8).

Figure 1
Schematic representation of microdialysis probe placements for rats in Groups Core (n = 6; black), Shell (n = 7; gray), DLS (n = 7; white), and DMS (n = 8; horizontal stripes). Numbers represent distance (mm) of coronal section from bregma.

Acquisition of lever pressing

As can be seen in Figure 2, rats from the four probe placement conditions acquired comparable rates of instrumental performance. An ANOVA using surgical group (core, shell, DMS, and DLS) and session (1–7) as factors found a significant main effect of session (F6,144 = 99.72; p < 0.001), but found no effect of surgical group (F3,24 = 1.16; p = 0.35) or session by group interaction (F18,144 = 0.97; p = 0.50).

Figure 2
Acquisition of lever pressing. Mean (± SEM) lever presses performed per minute for rats in Groups Core, Shell, DLS and DMS.

Baseline dialysate DA levels

Mean DA levels during pre-testing baseline periods were 0.83 nM (SEM ± 0.15) for the core, 0.73 nM (SEM ± 0.13) for the shell, 1.46 nM (SEM ± 0.17) for the DLS, and 0.96 nM (SEM ± 0.14) for the DMS. Analysis of these data revealed a significant main effect of probe placement (F3,26 = 4.72; p = 0.01). Bonferroni post-hoc tests found that basal DA levels were significantly higher in the DLS than in core (p = 0.05) or shell (p < 0.01). No other differences reached significance, indicating that DA levels in the DMS were intermediate between those observed in the DLS and nucleus accumbens.

Conditioning-induced DA efflux in the nucleus accumbens

Tests 1 and 3 allowed us to measure changes in extracellular DA concentrations as rats lever pressed for food reward in a state of high food motivation. Although task performance was comparable across these two tests, a mixed ANOVA using probe placement (core and shell) and test (1 and 3) as factors found that rats with accumbens probe placements responded at a marginally higher rate in Test 3 than in Test 1 (F1,11 = 3.41; p 0.092), which may have resulted from the additional conditioning given during the test sessions. No effect of placement (F1,11 = 0.24; p = 0.634) or placement by test interaction (F1,11 = 0.20; p = 0.667) was detected. Analysis of the number of rewards earned during these tests found no effect of test (F1,11 = 0.24; p = 0.632) or placement (F1,11 = 0.06; p = 0.808), and found no test by probe interaction (F1,11 = 0.11; p = 0.742).

Instrumental conditioning resulted in an elevation of accumbal DA during the session, which returned to near baseline levels after the session ended (see Figure 3). These data were analyzed using a mixed ANOVA with probe placement, test, and time (23 × 3-min samples) serving as factors. The ANOVA detected a significant effect of time (F22,242 = 11.37; p < 0.001), confirming that DA levels changed over the course of the session. There was no effect of test (F1,11 = 0.94; p = 0.354) or probe placement (F1,11 = 0.26; p = 0.619). Nor was there a significant test by group (F1,11 = 0.01; p = 0.929), test by time (F22,242 = 0.83; p = 0.682), or test by time by group interaction (F22,242 = 0.75; p = 0.785).

Figure 3
DA efflux in the core (left panel) and shell (right panel) of the nucleus accumbens of hungry rats lever pressing for food pellets. Black lines represent mean (± SEM) DA concentrations (expressed as percentage of baseline; i.e., average for bins ...

Although the magnitude of conditioning-induced DA efflux in the accumbens was similar for the two hungry tests when the data were averaged across animals, our multi-test experimental design allowed us to explore the relationship between conditioning-induced DA efflux and important experimental variables using an approach that minimizes between-subjects variability (e.g., probe function). Specifically, we examined if individual differences in the average DA response during instrumental conditioning (average of bins 1–10 for each subject during test sessions) between hungry tests were correlated with differences in either the number of lever presses performed or the number of pellets earned between the two sessions. Furthermore, since reinforcement was delivered in a probabilistic manner according to a RR-10 schedule, rats experienced variability in the number of responses needed to obtain reward (i.e., total number of presses performed per session/total number of rewards obtained per session) between the two tests, allowing us to investigate the relationship between DA and the effort (or response cost) required to gain each outcome. For each variable, Test 1 values were subtracted from Test 3 values to produce a difference score. Table 1 presents the mean (SEM) and range of difference scores for each of these variables. We found that changes in DA efflux in the nucleus accumbens (n = 13) between Tests 1 and 3 were not significantly correlated (Pearsons, two-tailed) with changes in response rate (r = −0.049; p = 0.874) or changes in the number of rewards earned (r = 0.432; p = 0.140) between the two tests. Therefore, our data do not support the view that accumbal DA mediates a simple response-activating influence over instrumental performance, or else DA levels should have varied with response vigor. Nor does DA appear to simply track the number of actions performed or number of pellets earned. However, changes in DA were found to be negatively correlated with changes in the number of presses required to obtain reward (r = −0.580; p = 0.038; see Figure 4), such that the DA response during conditioning dropped as the work requirement increased, and vice versa. This correlation was significant when the data from rats with accumbens core placements were separately analyzed (n = 6; r = −0.851; p = 0.032), indicating that the modulation of conditioning-induced DA efflux by response requirement in this structure is particularly strong. Although no correlation was found for rats with shell placements (n = 7; r = −0.567; p = 0.184), this lack of effect should be interpreted with caution, particularly given the significant correlation found when both probe placement groups were included in the analysis.

Figure 4
Scatter plot showing the relationship between changes in conditioning-induced DA efflux and response requirement (i.e., the mean number of lever presses required to earn each reward) across hungry tests (Test 3 – Test 1) for rats in Groups Shell ...
Table 1
The mean difference in conditioning-induced DA efflux (average of bins 1–10), total lever presses performed, total rewards earned, and ratio of responses per reward (i.e., average response requirement) between the two Hungry Tests (Test 3 – ...

To assess the effects of general satiety on tonic DA signaling and instrumental performance, we contrasted data collected during tests conducted when the rats were hungry (average of Tests 1 and 3) with the data from Test 2, which was conducted after the rats had been sated for 2h on home chow. As shown in Figure 5, this satiety induction procedure was generally effective in reducing the rate of lever press performance (main effect of motivational state: F1,11 = 25.79; p < 0.001). No effect of probe placement (F1,11 = 0.65; p = 0.438) or state by placement interaction (F1,11 = 0.93; p = 0.355) was detected. An ANOVA performed on the number of rewards earned during hungry and sated tests found that rats earned significantly fewer rewards when sated than when hungry (F1,11 = 28.22; p < 0.001). No effect of placement (F1,11 = 0.41; p = 0.536) or state by placement interaction (F1,11 = 0.99; p = 0.340) was detected.

Figure 5
Effect of satiety on conditioning-induced DA efflux in the core (left panel) and shell (right panel) of the nucleus accumbens. Black lines represent mean (± SEM) DA concentrations (expressed as percentage of baseline; i.e., average for bins −3, ...

In general, the effects of instrumental conditioning on accumbal DA efflux also appeared to be sensitive to downshift in motivational state (see Figure 5). A mixed ANOVA performed on these data detected a marginally significant main effect of motivational state (F1,11 = 4.24; p = 0.064), a significant effect of time (F22,242 = 12.92; p < 0.001) and, more importantly, a significant state by time interaction (F22,242 = 1.78; p = 0.020), indicating that the time course of conditioning-induced DA efflux was affected by the rats’ motivational state at test, with rats exhibiting an attenuated DA response when they were sated. There was no main effect of probe placement (F1,11 = 0.14; p = 0.718), nor was there any interaction between placement and state (F1,11 = 0.04; p = 0.843), placement and time (F22,242 = 0.62; p = 0.910), or between these three variables (F22,242 = 0.98; p = 0.49).

The correlational analysis described above indicated that accumbal DA does not exert a direct response-activating effect on instrumental performance, or track the number of responses performed or rewards earned. However, if accumbal DA is involved in the motivational control of instrumental performance, one might expect the effects of satiety on DA efflux to vary with the behavioral effects of satiety. To assess this possibility, we examined whether individual changes in the average DA response during instrumental conditioning (average of bins 1–10 for each subject) between Sated (Test 2) and Hungry tests (data averaged across Tests 1 and 3) correlated with differences in the rate of responding or number of pellets earned between these conditions. For each variable, the mean value for the Hungry tests was subtracted from the value for the Sated test to generate a difference score; therefore, a negative difference score indicates that the Sated value was lower than the Hungry value. The mean difference scores (SEM) and the range scores are presented in Table 2. When the data were collapsed across sites, a marginally significant correlation was detected between changes in accumbal DA efflux and changes in response rate (r = 0.514; p = 0.072). Although a similar relationship was apparent between changes in DA and changes in the number of rewards earned, this correlation also failed reach the conventional threshold for significance (r = 0.455; p = 0.118). Further analysis limited to accumbens shell placements, however, found significant positive correlations for both response rate (r = 0.827; p = 0.022; see Figure 6a) and rewards earned (r = 0.778; p = 0.039; see Figure 6b), which were highly correlated with one another (r = 0.973; p < 0. 001) since rewards were delivered according to a ratio schedule. Changes in core DA efflux were not correlated with either variable (response rate: r = −0.241; p = 0.646, rewards earned: r = −0.243; p = 0.643). Thus it appears that, at least for the accumbens shell, the suppressive effects of satiety on conditioning-induced mesolimbic DA efflux vary with its behavioral effects.

Figure 6
Scatter plot showing the relationship between the effect of satiety (Sated Test – Hungry Test) on conditioning-induced DA efflux and its effects on lever press performance (Panel A) and on the number of rewards earned (Panel B) for rats in Groups ...
Table 2
The mean difference in conditioning-induced DA efflux (average of bins 1–10), total lever presses performed, and total rewards earned between the Sated and Hungry Tests (Test 2 – average of Tests 1 and 3). SEMs are presented in parentheses. ...

Conditioning-induced DA efflux in the dorsal striatum

A mixed ANOVA using probe placement (DLS, DMS) and test (1 and 3) as factors found that rats with dorsal striatal probe placements responded at similar rates across the two hungry tests (F1,13 = 0.87; p = 0.368). No effect of placement (F1,13 = 0.20; p = 0.662) or placement by test interaction was detected (F1,13 = 0.26; p = 0.621). Analysis of the number of rewards earned in Tests 1 and 3 found no effect of test (F1,13 = 1.03; p = 0.328) or placement (F1,13 = 0.16; p = 0.692), and found no interaction between these factors (F1,13 = 1.16; p = 0.300). Although somewhat more modest than in the nucleus accumbens, dorsal striatal DA levels also increased as hungry rats lever pressed for food reward during Tests 1 and 3 (see Figure 7), an effect that was observed in both lateral and medial regions. A mixed ANOVA using probe placement group, test, and time as factors detected a significant main effect of time (F22,286 = 2.83; p < 0.001), confirming that DA levels increased during these test sessions. There was no effect of test (F1,13 = 0.07; p = 0.791) or probe placement (F1,13 = 0.48; p = 0.502), nor was there any significant interactions between group and test (F1,13 = 0.02; p = 0.905), group and time (F22,286 = 1.16; p = 0.286), or between the combination of these three variables (F22,286 = 0.81; p = 0.720).

Figure 7
DA efflux in the DLS (left panel) and DMS (right panel) of hungry rats lever pressing for food pellets. Black lines represent mean (± SEM) DA concentrations (expressed as percentage of baseline; i.e., average for bins −3, −2, and ...

Although dorsal striatal DA levels significantly increased during tests conducted when rats were hungry, this effect did not appear to be associated with any of the behavioral variables being recorded. Changes in the DA response (n = 15, collapsed across placements) to instrumental conditioning between Tests 1 and 3 were not correlated with changes in response rate (r = 0.077; p = 0.785), number of rewards earned (r = 0.112; p = 0.691), or response requirement (r = 0.220; p = 0.431).

Figure 8 shows the effects of satiety on lever pressing and conditioning-induced DA efflux in the dorsal striatum. Rats with dorsal striatal placements also responded at a significantly lower rate when sated than when hungry (F1,13 = 8.43; p = 0. 012). There was no effect of probe placement (F1,13 = 0.87; p = 0.369) and no state by placement interaction (F1,13 = 0.73; p = 0.407). Analysis of the number of rewards earned in hungry and sated tests found a main effect of motivational state (F1,13 = 10.02; p = 0.007). No effect of placement (F1,13 = 0.26; p = 0.622) or interaction between state and placement (F1,13 = 0.06; p = 0.816) was detected. Although DA levels in the dorsal striatum appeared to increase slightly at the beginning of the sated test session, these changes were much less persistent than those observed when the rats were tested hungry (average of Tests 1 and 3). This interpretation was supported by the statistical analysis, which although failing to detect a significant overall effect of motivational state (F1,13 = 1.89; p = 0.192), did find a significant state by time interaction (F22,286 = 1.62; p = 0.041). There was no effect of group (F1,13 = 0.89; p = 0.362), nor was there any significant interactions between group and state (F1,13 = 0.001; p = 0.975), group and time (F22,286 = 1.25; p = 0.206), or between group, state and time (F22,286 = 0.51; p = 0.969). Although it appears that conditioning-related DA efflux in the dorsal striatum was modulated by motivational state, unlike in the accumbens, subject-specific changes in DA efflux between Hungry and Sated tests were not significantly correlated (n = 15; collapsed across placements) with changes in response rate (r = −0.108; p = 0.702) or rewards earned (r = 0.011; p = 0.969).

Figure 8
Effect of satiety on conditioning-induced DA efflux in the DLS (left panel) and DMS (right panel) of the nucleus accumbens. Black lines represent mean (± SEM) DA concentrations (expressed as percentage of baseline; i.e., average for bins −3, ...

Discussion

The current study replicates previous reports of instrumental conditioning-induced DA efflux in the nucleus accumbens and finds evidence of a similar, if more modest, effect in the dorsal striatum. Importantly, however, we found that task-related accumbal DA efflux varied with the work requirement rather than tracking reward receipt or behavioral output. Across the striatum, conditioning-induced DA efflux was attenuated when rats were sated on food prior to testing, an effect that correlated with task performance in rats with shell placements.

DA and instrumental conditioning

Striatal DA signaling appears to be involved in several aspects of instrumental control. DA plays a central role in incentive motivation (Beninger & Phillips, 1981; Berridge & Robinson, 1993; Dickinson et al., 2000). The response-invigorating effects that reward-paired cues have on instrumental performance are blocked by treatments that disrupt accumbal DA transmission (Corbit et al., 2007; Lex & Hauber, 2008; Marschall & Hauber, 2006). Furthermore, microdialysis studies have found cue-evoked DA efflux in the absence of any instrumental response requirement (Cheng et al., 2003), suggesting that DA efflux observed during instrumental tasks may be generated by Pavlovian conditioning occurring in the background (cf. Ahn & Phillips, 2007; Cheng & Feenstra, 2006). However, DA signaling also plays a role in effort-based decision-making (Salamone et al., 2007; Walton et al., 2006). Behavior maintained on high ratio schedules is more sensitive to disruption by DA depletion than behavior maintained on low ratio or interval schedules of reinforcement, which require less effort (Salamone et al., 2007). This contribution of DA signaling to instrumental control appears to be gated by primary motivation, as DA antagonists are more effective in suppressing performance in hungry than in sated rats (Dickinson et al., 2000) and appear to mimic the effects of satiety on instrumental performance (Willner et al., 1988).

A recent computational model of instrumental performance (Niv et al., 2007) proposes that tonic DA levels in the accumbens provide a running estimate of the net rate of reward, integrating the various costs and benefits of responding in the current environment. DA is assumed to play an indirect role in controlling response vigor by signaling what one should expect to lose by failing to act (i.e., the cost of sloth). According to this theory, the magnitude of DA efflux during instrumental conditioning should decrease if the effort needed to obtain reward increases, or if the value of reward decreases. An alternative approach to modeling the role of DA in decision-making (Phillips et al., 2009) proposes that DA release signals the expected value of an available reward, allowing this information to be contrasted with an independent estimate of the effort needed to obtain reward when deciding whether or not an action should be selected. Note that this account predicts that DA levels should be unaffected by manipulations of response cost.

In the current experiment, rewards were delivered according to a probabilistic schedule, such that average number of lever presses (or effort) required to obtain reward varied randomly across tests. We found that changes in conditioning-evoked accumbal DA efflux between tests were negatively correlated with changes in the amount of effort needed to obtain reward, a relationship that was particularly striking for the accumbens core. This finding indicates that information about response cost is conveyed to the accumbens via tonic DA signaling, consistent with predictions of Niv et al. (2007). Interestingly, this result mirrors recent findings from fast-scan cyclic voltammetry studies showing that phasic accumbal DA release to discriminative cues encodes the effort needed to obtain reward (Gan et al., 2009; Day et al., 2010). However, it should be noted that the single-action, free-operant procedure used here differs considerably from the two-action, discrete-trial procedures used in these other studies. The similarity between these findings should also be considered in light of the existing literature indicating that tonic and phasic DA release represent distinct channels of neurotransmission and that phasic DA signaling does not significantly contribute to the changes in DA concentrations that are measured with microdialysis (Floresco et al., 2003). It therefore seems likely that tonic and phasic DA independently convey information about response costs (and benefits), perhaps mediating fundamentally different neural and psychological functions.

Our failure to find a significant correlation between changes in response rate and accumbal DA efflux across hungry tests suggests that tonic DA does not simply track the number of rewards received or responses performed. Nor do these data support the view that DA mediates a simple response-activating function. There is, however, some support for this latter view. There have been multiple reports that conditioning-induced DA efflux in the ventral striatum positively correlates with response rate (McCullough et al., 1993; Sokolowski et al., 1998; Cousins et al., 1999). Our finding of a positive correlation between changes in Shell DA levels and response rate across hungry and sated tests could also be taken as support for this view. However, other studies have failed to find evidence of a relationship between response rate and DA efflux (Cousins et al., 1996; Ahn & Phillips, 2007). Interestingly, when higher ratio schedules are used, the correlation between DA and lever pressing can become hyperbolic, with rats responding at intermediate and high rates exhibiting equivalent levels of DA efflux (Salamone et al., 1994). Such mixed results suggest that the relationship between tonic DA signaling and lever press performance is complex and is likely to depend on the nature of the task and/or the schedule of reinforcement being used.

Dorsal Striatum

The finding that DA levels in the dorsal striatum increased during instrumental conditioning is perhaps not surprising given the massive DAergic projections to this region and given recent findings implicating it in instrumental action selection (Balleine et al., 2009; Yin et al., 2007). As with the nucleus accumbens, treatments that disrupt dorsal striatal DA transmission tend to suppress instrumental performance (Amalric & Koob, 1987; Salamone et al., 1993). It has also been reported that rats lever pressing for food exhibit increased DA metabolism in dorsal striatum (Church et al., 1986). It is therefore noteworthy that a recent study monitoring DA levels in the dorsal striatum during instrumental conditioning found no evidence of conditioning-induced DA efflux (Ahn & Phillips, 2007). However, the area being targeted in that study was considerably more anterior and ventral than either of the two dorsal striatal areas targeted in the current experiment, which were selected because of their established roles in goal-directed (DMS; Yin et al., 2005) and habitual (DLS; Yin et al., 2004) performance.

Conditioning-induced DA efflux in the dorsal striatum was of a substantially smaller magnitude than that observed in the nucleus accumbens. Consistent with this finding, it has recently been reported that food-evoked increases in DA are smaller in the dorsal striatum than in the nucleus accumbens (Zhang et al., 2009). It is also noteworthy that, unlike in the accumbens, dorsal striatal DA efflux was not significantly correlated with any of our behavioral variables. Such findings indicate that there is functional heterogeneity in striatal DA signaling, with accumbal – but not dorsal striatal – DA playing a central role in the motivational control of instrumental performance (Yin et al., 2007). However, dorsoventral differences in DA transporter density (Marshall et al., 1990) and DA re-uptake (Stamford et al., 1988; Garris et al., 1994) suggests that the relatively weak dorsal striatal DA response that we observed may reflect the rapid clearance of extracellular DA in this area rather than a regional difference in DA release.

Downshift in motivational state

Our results indicate that conditioning-related tonic DA signaling is modulated by primary motivation. Specifically, sated rats showed an attenuated DA response to instrumental conditioning, an effect that was observed in both the nucleus accumbens and dorsal striatum. This finding is generally consistent with previous reports that satiety treatments tend to decrease the accumbal DA efflux observed during and in anticipation of feeding (Wilson et al., 1995; Ahn & Phillips, 1999). However, these earlier studies used passive feeding procedures and therefore could not address questions about the relationship between DA and instrumental performance. In the current study, we found that the effect of satiety on conditioning-evoked DA efflux in the accumbens shell was correlated with its effects on lever pressing and on the number of rewards earned. Although our use of a ratio schedule of reinforcement resulted in a strong positive correlation between response rate and reward delivery, thus preventing us from determining whether one of these variables was preferentially associated with DA signaling, the current findings indicate that tonic DA signaling in the shell may play a particularly important role in the motivational control of instrumental performance.

Conclusion

The current study found broad but heteregeneous instrumental conditioning-related tonic DA signaling across sub-regions of the striatum. Although we found little evidence that tonic DA directly controlled behavioral output, it did appear to track shifts in primary motivation and response cost, variables that influence the vigor with which instrumental actions are performed. These results may inform future research investigating the role of tonic DA signaling in addiction and other disorders of behavioral control.

Acknowledgments

This work was supported by grant DA09359 and DA05010 from NIDA to NTM, AA18014 from NIAAA to BWB, and training fellowship T32 MH17140 to SBO.

References

  • Ahn S, Phillips AG. Dopaminergic correlates of sensory-specific satiety in the medial prefrontal cortex and nucleus accumbens of the rat. J Neurosci. 1999;19:29. [PubMed]
  • Ahn S, Phillips AG. Dopamine efflux in the nucleus accumbens during within-session extinction, outcome-dependent, and habit-based instrumental responding for food reward. Psychopharmacology (Berl) 2007;191:641–651. [PMC free article] [PubMed]
  • Amalric M, Koob GF. Depletion of dopamine in the caudate nucleus but not in nucleus accumbens impairs reaction-time performance in rats. J Neurosci. 1987;7:2129. [PubMed]
  • Balleine BW, Liljeholm M, Ostlund SB. The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res. 2009;199:43–52. [PubMed]
  • Beninger RJ, Phillips AG. The effects of pimozide during pairing on the transfer of classical conditioning to an operant discrimination. Pharmacology Biochemistry and Behavior. 1981;14:101–105. [PubMed]
  • Cheng JJ, de Bruin JPC, Feenstra MGP. Dopamine efflux in nucleus accumbens shell and core in response to appetitive classical conditioning. Eur J Neurosci. 2003;18:1306–1314. [PubMed]
  • Cheng J, Feenstra MG. Individual differences in dopamine efflux in nucleus accumbens shell and core during instrumental learning. Learning & Memory. 2006;13:168–177. [PubMed]
  • Church WH, Justice JB, Neill DB. Detecting behaviorally relevant changes in extracellular dopamine with microdialysis. Brain Res. 1987;412:397–399. [PubMed]
  • Church WH, Sabol KE, Justice JB, Neill DB. Striatal dopamine activity and unilateral barpressing in rats. Pharmacol Biochem Behav. 1986;25:865–871. [PubMed]
  • Corbit LH, Janak PH, Balleine BW. General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area. Eur J Neurosci. 2007;26:3141–3149. [PubMed]
  • Cousins MS, Trevitt J, Atherton A, Salamone JD. Different behavioral functions of dopamine in the nucleus accumbens and ventrolateral striatum: a microdialysis and behavioral investigation. Neuroscience. 1999;91:925–934. [PubMed]
  • Day JJ, Jones JL, Wightman RM, Carelli RM. Phasic Nucleus Accumbens Dopamine Release Encodes Effort-and Delay-Related Costs. Biological Psychiatry 2010 [PMC free article] [PubMed]
  • Dickinson A, Smith J, Mirenowicz J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci. 2000;114:468–483. [PubMed]
  • Floresco SB, West AR, Ash B, Moore H, Grace AA. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat Neurosci. 2003;6:968–973. [PubMed]
  • Gan JO, Walton ME, Phillips PE. Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat Neurosci. 2009;13:25–27. [PMC free article] [PubMed]
  • Garris PA, Ciolkowski EL, Wightman RM. Heterogeneity of evoked dopamine overflow within the striatal and striatoamygdaloid regions. Neuroscience. 1994;59:417–427. [PubMed]
  • Lex A, Hauber W. Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. Learning & Memory. 2008;15:483. [PubMed]
  • Marshall JF, O’Dell SJ, Navarrete R, Rosenstein AJ. Dopamine high-affinity transport site topography in rat brain: major differences between dorsal and ventral striatum. Neuroscience. 1990;37:11–21. [PubMed]
  • McCullough L, Cousins M, Salamone J. The role of nucleus accumbens dopamine in responding on a continuous reinforcement operant schedule: A neurochemical and behavioral study. Pharmacology Biochemistry and Behavior. 1993;46:581–586. [PubMed]
  • Murschall A, Hauber W. Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance. Learning & Memory. 2006;13:123. [PubMed]
  • Niv Y, Daw ND, Joel D, Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 2007;191:507–520. [PubMed]
  • Phillips PE, Walton ME, Jhou TC. Calculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine. Psychopharmacology. 2007;191:483–495. [PubMed]
  • Robinson TE, Berridge KC. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Research Reviews. 1993;18:247–291. [PubMed]
  • Salamone JD, Cousins MS, McCullough LD, Carriero DL, Berkowitz RJ. Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not free food consumption. Pharmacol Biochem Behav. 1994;49:25–31. [PubMed]
  • Salamone JD, Kurth PA, McCullough LD, Sokolowski JD, Cousins MS. The role of brain dopamine in response initiation: effects of haloperidol and regionally specific dopamine depletions on the local rate of instrumental responding. Brain Res. 1993;628:218–226. [PubMed]
  • Salamone JD, Correa M, Farrar A, Mingote SM. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology. 2007;191:461–482. [PubMed]
  • Salamone JD, Correa M, Mingote SM, Weber SM. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Current Opinion in Pharmacology. 2005;5:34–41. [PubMed]
  • Sokolowski JD, Conlan AN, Salamone JD. A microdialysis study of nucleus accumbens core and shell dopamine during operant responding in the rat. Neuroscience. 1998;86:1001–1009. [PubMed]
  • Walton ME, Kennerley SW, Bannerman DM, Phillips PEM, Rushworth MFS. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw. 2006;19:1302–1314. [PMC free article] [PubMed]
  • Willner P, Chawla K, Sampson D, Sophokleous S, Muscat R. Tests of functional equivalence between pimozide pretreatment, extinction and free feeding. Psychopharmacology. 1988;95:423–426. [PubMed]
  • Wilson C, Nomikos GG, Collu M, Fibiger HC. Dopaminergic correlates of motivated behavior: importance of drive. J Neurosci. 1995;15:5169–5178. [PubMed]
  • Yin HH, Ostlund SB, Balleine BW. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. The Eur J Neurosci. 2008;28:1437. [PMC free article] [PubMed]
  • Yin HH, Knowlton BJ, Balleine BW. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci. 2004;19:181–189. [PubMed]
  • Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci. 2005;22:513–523. [PubMed]
  • Zhang L, Doyon WM, Clark JJ, Phillips PEM, Dani JA. Controls of Tonic and Phasic Dopamine Transmission in the Dorsal and Ventral Striatum. Molecular Pharmacology. 2009;76:396–404. [PubMed]