PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Neuroscientist. Author manuscript; available in PMC 2011 July 6.
Published in final edited form as:
PMCID: PMC3130622
NIHMSID: NIHMS265739

The Nucleus Accumbens and Pavlovian Reward Learning

Abstract

The ability to form associations between predictive environmental events and rewarding outcomes is a fundamental aspect of learned behavior. This apparently simple ability likely requires complex neural processing evolved to identify, seek, and utilize natural rewards and redirect these activities based on updated sensory information. Emerging evidence from both animal and human research suggests that this type of processing is mediated in part by the nucleus accumbens and a closely associated network of brain structures. The nucleus accumbens is required for a number of reward-related behaviors, and processes specific information about reward availability, value, and context. Additionally, this structure is critical for the acquisition and expression of most Pavlovian stimulus-reward relationships, and cues that predict rewards produce robust changes in neural activity in the nucleus accumbens. While processing within the nucleus accumbens may enable or promote Pavlovian reward learning in natural situations, it has also been implicated in aspects of human drug addiction, including the ability of drug-paired cues to control behavior. This article will provide a critical review of the existing animal and human literature concerning the role of the NAc in Pavlovian learning with non-drug rewards and consider some clinical implications of these findings.

Keywords: Learning, Reward, Nucleus Accumbens, Drug Addiction, Conditioning

Species survival and propagation requires that individual organisms learn about their surroundings and continually adjust behavior accordingly. One elementary yet biologically critical form of learning involves the connection of positive outcomes with predictive cues. This ability enables organisms to track, locate, and secure food and necessary materials in demanding environments, revealing obvious survival value. Moreover, such Pavlovian learning is often the background for both normal and maladaptive human behaviors. Thus, understanding reward-related Pavlovian learning could shed light on a variety of human activities, including drug taking, food seeking, social attachment, and sexual behavior.

Recent endeavors have sought to identify and deconstruct the neurobiological correlates of Pavlovian learning, and there is now little doubt that a distributed network of brain nuclei mediates the ability to form associations between rewarding events and their predictors. The nucleus accumbens (NAc) is central to this network, and ongoing research focuses on how this area processes natural and drug rewards as well as related behaviors. This review will discuss the behavioral importance of Pavlovian learning (with particular attention to natural rewards), examine the involvement of the nucleus accumbens in this process, and consider how these factors may contribute to drug abuse and addiction.

Pavlovian reward learning

Historically, reward-related learning has been divided into stimulus-outcome (classical or Pavlovian) and action-outcome (operant or instrumental) branches. In the context of Pavlovian learning, biologically relevant outcomes such as food, water, and sexual stimuli are labeled unconditioned stimuli (US) because they are able to evoke innate or unconditioned responses (UR) such as salivation, approach, and consumption (Pavlov 1927). Pavlovian conditioning procedures involve the pairing of a neutral sensory stimulus, termed the conditioned stimulus (CS) with a US in a temporally contingent manner. Learning occurs as the previously neutral stimulus obtains predictive value for the coming reward based on repeated pairings of the CS and US. Eventually, this novel cue is able to evoke a response that is often topographically similar to that produced by the US itself. The learned response that the CS elicits is called the conditioned response (CR).

The existence of an Pavlovian association is typically inferred from the presence or absence of a CR. Therefore, complete understanding of the neurobiological basis of Pavlovian learning requires not only advanced cellular and pharmacological technologies, but also reliable behavioral techniques that can measure the acquisition and maintenance of an association. For example, stimulus-reward learning in animals is often quantified using an “autoshaping” or “sign-tracking” design, in which a CS predicts the availability of a natural reward such as food (US). Initially, food delivery produces orienting and approach responses that are followed by consumption. With repeated CS-US pairings, the CS itself begins to elicit highly selective approach responses even though reward delivery is independent of any specific behavior (Bussey and others 1997; Robbins and Everitt 2002). After extended conditioning, approach responses are observed nearly every time the reward-predictive CS is presented to an animal, indicating a strong association between this cue and the future reward. Interestingly, conditioned approach behaviors have been observed in a variety of species, including rats, monkeys, pigeons, and humans (Brown and Jenkins 1968; Sidman and Fletcher 1968; Wilcove and Miller 1974; Bussey and others 1997), and track diverse rewards such as heat, food, water, intracranial stimulation, intravenous cocaine, and copulation (Peterson and others 1972; Jenkins and Moore 1973; Wasserman 1973; Burns and Domjan 1996, 2001; Uslaner and others 2006). Thus, although directed approach behaviors are unlike traditional Pavlovian visceral or glandular responses, they provide a convenient measure of associative reward learning that is highly similar across species.

In real life, organisms use environmental cues to update expectancies and allocate behavioral resources in a way that maximizes value and minimizes energy expenditure. Therefore, Pavlovian relationships may be embedded within virtually all operant circumstances. For example, general contextual stimuli (e.g., a place where rewards are consumed) may come to be explicitly associated with reward delivery and operate as conditioned stimuli. In other cases, conditioned stimuli may even acquire some of the motivational properties of primary goals like food and sex through Pavlovian learning. In turn, such stimuli could act as conditioned reinforcers, with the ability to strengthen and redirect behavior on their own in an operant manner. For humans, money is perhaps the best example of this phenomenon. Although government currency possesses no innate biological importance, it is consistently paired with items that do have motivational significance. This pairing allows money to serve as a predictor for future rewards, but also as a powerful conditioned reinforcer.

From an ecological standpoint, reward-related Pavlovian learning may provide several adaptive advantages for an organism in a rapidly changing environment. Successful identification and consumption of unconditioned stimuli such as food involves physical contact that engages proximal receptors of taste, olfaction, and somatosensation. However, because these sensory modalities are not equipped to identify stimuli in a larger environmental field, they may not make up a complete set of tools for efficient foraging behavior. Through Pavlovian learning, visual and auditory information can be incorporated in the foraging experience and utilized to detect and predict available rewards. Moreover, the ability of stimuli to act as conditioned reinforcers means that goal-directed behaviors can be maintained for longer periods of time in the absence of a primary reward. Importantly, each of these functions may increase the chances that an organism will secure necessary materials and meet energy needs.

The nucleus accumbens: A limbic-motor interface

The ability to form and use Pavlovian associations requires the existence and cooperation of brain circuits equipped to integrate sensory and motivational information and alter motor output. The nucleus accumbens has received much attention in this respect. In the rat, the NAc is comprised of two primary subregions, the core and shell, which differ with respect to their anatomic organization and functional properties (Zahm and Brog 1992; Zahm and Heimer 1993; Zahm 1999, 2000; Voorn and others 2004). As illustrated in Figure 1, the core and shell of the rodent NAc receive afferent projections from a variety of cortical and subcortical structures including the basolateral amygdala (Zahm and Brog 1992; Brog and others 1993; Wright and others 1996), the prefrontal cortex (McGeorge and Faull 1989; Zahm and Brog 1992; Brog and others 1993), the subiculum of the hippocampus (Groenewegen and others 1987; Groenewegen and others 1991; Zahm and Brog 1992; Brog and others 1993), and the ventral tegmental area (Zahm and Brog 1992). Importantly, afferent projections to the NAc are not homogeneously distributed across the core and shell (Groenewegen and others 1987; McGeorge and Faull 1989; Groenewegen and others 1991; Zahm and Brog 1992; Brog and others 1993; Heimer and others 1995; Heimer and others 1997). For example, Brog and co-workers (1993) showed that a number of cortical afferents of the shell and core originate in separate areas (e.g., the infralimbic and posterior piriform cortices to the medial shell versus the dorsal prelimbic and anterior cingulate to the core).

Figure 1
General overview of key nucleus accumbens afferent and efferent projections. See text for more information on the anatomic organization of the nucleus accumbens. Note that placement of arrows does not necessarily indicate degree or precise location of ...

Likewise, the efferent projections from the NAc differ between the core and shell subregions in the rat (Heimer and others 1991; Zahm and Brog 1992; Zahm and Heimer 1993; Zahm 1999). That is, the core parallels basal ganglia circuitry sending outputs through the ventral pallidum (dorsolateral district), subthalamic nucleus and substantia nigra. These outputs in turn project via the motor thalamus to premotor cortical areas. In contrast, the shell projects preferentially to subcortical limbic regions including the lateral hypothalamus, ventral pallidum (ventromedial district) and VTA (Zahm 1999). Interestingly, recent findings show direct interconnections between core and shell neurons providing anatomic evidence that these NAc subregions do not function completely independently, but instead comprise interacting neuronal networks (van Dongen and others 2005).

Examination of the local organization of the striatum has revealed that over 90% of neurons in the neostriatum (caudate-putamen) are of the medium-spiny type, and that these medium spiny neurons comprise the majority of projections from the striatum (Groves 1983). Other cell types include, among others, the aspinous cholinergic interneurons and medium-size aspinous GABAergic interneurons (Groves 1983; Groenewegen and others 1991). Investigations have confirmed that the NAc is comprised of similar neuronal cell types (Voorn and others 1989; Groenewegen and others 1991), although there are morphological differences between core and shell neurons (Meredith and others 1992). Moreover, a heterogeneous distribution within the NAc subregions also exists with respect to markers for substances such as calbindin and enkephalin, among others (Voorn and others 1989; Groenewegen and others 1991; Zahm 1999).

Given the anatomic arrangement of the NAc, it was proposed by Mogenson (1987) and elaborated upon by others (Everitt and Robbins 1992; Pennartz and others 1994; Ikemoto and Panksepp 1999) that the NAc functions as a site for the integration of limbic information related to memory, drive and motivation, and the generation of goal-directed motor behaviors (termed ‘limbic-motor integration’). Within this model, the neurotransmitter dopamine plays a key role in this process by functioning to modulate or ‘gate’ the transfer of these neural signals and thereby influence goal-directed behaviors (Cepeda and Levine 1998). That is, dopamine functions as a neuromodulator, influencing the activation of NAc cells by specific afferents including the hippocampus, basolateral amygdala and prefrontal cortex.

Consistent with this view is the observation that NAc afferents make convergent synaptic contacts onto medium spiny neurons. Studies using immunocytochemistry in conjunction with electron microscopy showed that hippocampal and dopaminergic inputs make synaptic connections with the same NAc neuron (Totterdell and Smith 1989; Sesack and Pickel 1990). Likewise, Van Bockstaele and Pickel (1993) reported that 5-HT terminals were in direct contact with dopaminergic axons in both the core and shell of the rat. In addition, a convergence of inputs from the medial prefrontal cortex and the ventral subiculum on NAc neurons has recently been identified (French and Totterdell 2002) as well as the BLA and ventral subiculum (French and Totterdell 2003). These findings indicate that NAc afferents are capable of influencing NAc cell firing in behaving animals (Pennartz and others 1994; O'Donnell and Grace 1995; Carr and Sesack 2000; Pinto and Sesack 2000), consistent with a role of this structure in ‘limbic-motor’ integration (Mogenson 1987).

Reward processing in the nucleus accumbens

In order for Pavlovian learning to successfully modify reward-related behaviors, brain systems must first be able to process information about the identity and value of unconditioned stimuli that act as rewards. Further, once a reward is obtained, motor systems must appropriately redirect behavior to gain maximal utility from the reward. The application of in vivo electrophysiological recording techniques to the study of reward processing in the NAc has revealed a variety of changes in NAc activity during goal-directed behaviors. For example, a subset of NAc neurons exhibit phasic yet time-locked alterations in firing rate when animals make operant responses to obtain rewards such as water and food (Apicella and others 1991; Carelli and Deadwyler 1994, 1997; Carelli and others 2000; Hassani and others 2001; Nicola and others 2004). However, these patterns of cellular activity are not homogenous. In fact, some NAc cells display enhanced activation before a lever press, while the activity of other neurons may increase or decrease immediately after the lever press (Carelli and Deadwyler 1994, 1997; Carelli 2002). Thus, NAc neurons seemingly process remarkably different types of reward-related information, which could reflect the dual role of this structure in both reward seeking and reward consumption (Nicola and others 2004).

Electrophysiological studies typically investigate NAc reward processing using operant (action-outcome) tasks, making it difficult to distinguish NAc activity specific to rewards from activity related to reward seeking behaviors. However, a few recent studies have controlled for or circumvented this complication to assess reward-specific NAc activity. In one study, NAc cellular activity was monitored while naive rats received experimenter-controlled intra-oral infusions of rewarding sucrose (Roitman and others 2005a). Consistent with other reports (Nicola and others 2004; Taha and Fields 2006), the predominant response of NAc neurons to sucrose infusions was a decrease in activity (Fig. 2). As is evident in Figure 2, the same neurons exhibited opposite responses when an aversive quinine solution was delivered intra-orally. One hypothesis suggests that inhibitions observed during reward delivery occur among GABA-containing NAc neurons that project to important motor areas such as the ventral pallidum (VP). Through the dis-inhibition of target neurons, such a change in activity could provide a gating signal for reward-related behaviors such as consumption (Nicola and others 2004; Roitman and others 2005a; Taha and Fields 2006). In support of this hypothesis, a recent study found that individual VP neurons exhibit increases in firing rate during consumption of a rewarding sucrose solution (Tindell and others 2006). Notably, a separate subset of NAc neurons exhibit increases in activity when sucrose rewards are delivered (Taha and Fields 2005). However, the magnitude of activation varies based on the concentration of sucrose, indicating that these neurons encode the palatability of a food reward instead of reward delivery or consumption. Interestingly, not all inhibitory and excitatory NAc responses observed during the delivery of primary rewards are fixed or unconditional. Rather, a subgroup of NAc neurons exhibit differential responses based on the relative context of reward delivery, including the availability of more and/or less preferred rewards (Wheeler and others 2005).

Figure 2
A single NAc neuron showing opposite firing patterns during intraoral infusion of rewarding sucrose (left) versus bitter, aversive quinine (right). After an initial excitatory response, the same NAc neuron displayed an inhibition in firing rate during ...

Another approach to the study of NAc reward processing involves the use of pharmacological techniques to activate or inhibit specific neurotransmitter systems within the NAc. Using this method, multiple reports implicated glutamate, GABA, and opioid neurotransmission in key aspects of reward processing, and specifically in feeding behavior. Both GABA agonism and glutamate antagonism in the NAc produce increases in food consumption, further indicating that neuronal inhibition in this structure may play an important role in the initiation or maintenance of feeding behavior (Kelley 2004). Intra-NAc μ-opioid agonists have also been shown to boost food intake, while animals receiving μ-opioid antagonists exhibit attenuated consumption (Kelley and others 1996; Pecina and Berridge 2000). Interestingly, manipulations that increase food intake are most effective in the shell of the NAc, indicative of a functional division between NAc subregions. In addition, a spatially restricted area within the medial NAc shell has been specifically implicated in the ability of opioid agonists to alter hedonic reactions to both rewarding and aversive stimuli (Pecina and Berridge 2005). Thus, some categories of reward-related information may be processed by distinct neurotransmitter systems in functionally isolated regions of the NAc.

The midbrain dopaminergic projection to the NAc is critical for a number of reward-related behaviors, and several decades of experimental research have attempted to elucidate the precise functional role of this connection. Early support for the involvement of the mesocorticolimbic dopamine pathway in reward processing came from several studies demonstrating that the blockade of dopamine receptors produced a decrease in goal-directed behavior for food and other rewards (Wise and others 1978b; Wise and others 1978a; Wise and others 1992). Interestingly, although animals that received dopamine antagonists still worked to obtain rewards, responding decreased as a function of time, similar to what would be expected if rewards were removed altogether (Fouriezos and Wise 1976; Wise and others 1978b). These findings initially led to the suggestion that dopamine release in the NAc mediates the hedonic or “pleasure” aspects of rewarding stimuli, and, in turn, that both natural and drug rewards could be defined by this common path of activation (Wise and Bozarth 1985). However, this hypothesis has been questioned on a number of grounds. For example, dopamine antagonism in the NAc does not impair orofacial movements characteristic of reward “liking” (Pecina and others 1997), indicating that the hedonic value of a stimulus is not based on NAc dopamine transmission. Moreover, proper NAc dopamine function is also required for tasks that are motivated by aversion rather than by rewards (Blackburn and others 1992; Salamone 1994). Finally, NAc dopamine depletion disrupts behavioral performance when large amounts of effort are required to obtain rewards, but has little effect on easy tasks (Aberman and Salamone 1999). Taken together, these and other findings support a larger role for NAc dopamine beyond simple hedonic pleasure (Blackburn and others 1987; Salamone and others 1991; Schultz and others 1993; Hollerman and Schultz 1998; Waelti and others 2001; Salamone and others 2002; Pecina and others 2003).

Since the original “anhedonia” hypothesis, a number of new and/or revised theories have been developed to explain the function of NAc dopamine in reward processing (Blackburn and others 1992; Ikemoto and Panksepp 1999; Schultz 2001; Di Chiara 2002; Ungless 2004; Wise 2004; Salamone and others 2005). For example, one theory posits that NAc dopamine mediates the incentive salience possessed by rewards and other important stimuli (Berridge and Robinson 1998). According to this theory, the principal function of mesocorticolimbic dopamine is to promote the motivational aspects of goal-directed behavior. Another theory regards NAc dopamine as a more general system that enables an organism to respond to novel events, discern the meaning of environmental stimuli, and generate approach or withdrawal responses to rewarding and aversive stimuli (Ikemoto and Panksepp 1999). Still more hypotheses have proposed that dopamine release in the NAc is critical to the behavioral process of reinforcement or reward itself (Wise 2004), or is required to overcome larger response costs to obtain rewards (Salamone and others 2003).

Although the precise role of dopamine in reward processing is presently under much debate, new findings and technological advances have contributed greatly to our understanding of this issue. While microdialysis investigations have long reported increases in NAc dopamine levels during goal-directed behaviors and/or receipt of rewards (Di Chiara 2002), these investigations lack the temporal resolution necessary to associate dopamine with precise (real-time) behavioral observations. Recently, the ability to measure dopamine release on a physiologically and behaviorally relevant timescale has led to a focus on rapid NAc dopamine release events (Garris and others 1999; Phillips and others 2003; Robinson and others 2003). Using an electrochemical technique that permitted sub-second detection of dopamine, Roitman and colleagues (2004) demonstrated that operant responses for a sucrose reward were associated with brief but robust increases in NAc dopamine concentration. Similar dopamine signals have also been observed in male rats during exposure to and approach towards receptive females (Robinson and others 2001), suggesting that phasic changes in dopamine release in the NAc may dynamically modulate a variety of reward-directed behaviors. Furthermore, preliminary results indicate that subsecond increases in NAc dopamine concentration are promoted by primary rewards but not aversive stimuli, and that this response is innate (Roitman and others 2005b). Future studies will continue to examine the exact nature of dopamine release in the NAc and the role of fast dopamine signals in reward processing.

The nucleus accumbens and conditioned stimuli

In addition to its role in processing information about unconditioned rewards, emerging evidence from animal research indicates that the NAc is also critically involved in the formation of learned Pavlovian associations between these rewards and exteroceptive stimuli. However, because NAc neurons are not believed to process primary sensory information, this function likely requires that individual NAc circuits undergo dynamic modification during stimulus-reward learning. Two intriguing studies have recently indicated that this may be the case. Setlow and colleagues (2003) paired olfactory cues with rewarding sucrose in a go-no go task while monitoring the activity of neurons in the ventral striatum (including the NAc). Initially, delivery of olfactory cues produced a change in activity among very few neurons. However, as animals learned to associate olfactory stimuli with sucrose delivery, those cues began to evoke time-locked phasic responses in a number of neurons. In another study that employed a strictly Pavlovian design (Roitman and others 2005a), NAc neurons developed responses to reward-predictive audiovisual cues on the first day that these stimuli were paired. Thus, although the majority of individual NAc neurons do not exhibit innate phasic responses to environmental stimuli, such responses quickly emerge as animals come to associate those stimuli with impending outcomes.

The ability of conditioned stimuli to elicit changes in NAc cell activity may only increase as stimulus-reward associations become stronger. In one experiment, rats were repeatedly exposed to a CS that was always followed by a sucrose reward as well as a control stimulus that was not paired with a reward (Day and others 2006). Across several conditioning sessions, rats gradually developed selective conditioned approach responses towards the reward predictive cue, but not towards the unpaired cue. Consistent with another recent study that employed a similar paradigm (Wan and Peoples 2006), a majority (75%) of NAc neurons exhibited marked changes (increases and decreases) in firing rate during presentation of the reward-paired CS in well-conditioned rats. Of these cells, roughly half responded with a prolonged inhibition, while the other half were activated by the presence of the cue, again suggesting that individual neurons within the NAc may operate as a part of microcircuits with distinct functional responsibilities (Carelli and Wightman 2004). A characteristic excitatory NAc neuron is shown in Figure 3. Although this neuron displayed no change in activity during unpaired stimulus trials, presentations of the reward-paired cue produced a significant increase in firing rate. Moreover, the excitation produced by the CS was greater than that evoked by the reward itself. It has been suggested such excitations among NAc neurons may originate from glutamatergic inputs from cortical and limbic structures that compete for access to motor resources through striatal circuits (Pennartz and others 1994). Through this mechanism, higher-order processing centers could gain direct influence over motor areas and promote behavioral responses to conditioned stimuli and other important cues.

Figure 3
A single NAc neuron showing a characteristic increase in firing rate within seconds of the cue (lever extension) paired with sucrose (indicated by blue line under top PEH) but not the unpaired cue (extension of second lever never paired with sucrose delivery, ...

Support for the involvement of the NAc in stimulus-reward learning also comes from single-unit recordings of midbrain dopamine neurons. In primates, a majority of these neurons exhibit brief increases in activity when rewards are delivered unexpectedly (Mirenowicz and Schultz 1994; Hollerman and Schultz 1998). However, if rewards are fully predicted by a CS, they no longer evoke activation among dopamine neurons. Instead, conditioned stimuli alone elicit increases in dopamine burst firing that varies in magnitude based on the likelihood of reward delivery as well as the value of the expected reward (Schultz and others 1997; Fiorillo and others 2003; Tobler and others 2005). A current hypothesis based on these observations proposes that dopamine neurons may provide a “prediction error” signal consistent with contemporary reward learning theories that are directly applicable to Pavlovian contingencies (Schultz and others 1997; Sutton and Barto 1998). According to this hypothesis, phasic activation of dopamine neurons signal unexpected reward delivery because this produces an error in ongoing predictions about reward availability. Likewise, as conditioned stimuli become valid reward predictors, reward delivery does not constitute a violation of expectancy and therefore does not produce phasic dopamine cell firing. At the cellular level, phasic dopamine signals in the NAc may facilitate synaptic modification (Calabresi and others 2000a), enabling NAc neurons to incorporate new information. With respect to Pavlovian learning, such plasticity could help organisms identify cues that predict rewards and update evaluation of those cues based on actual outcomes.

Importantly, actual dopamine release during the presentation of conditioned stimuli may not be identical across all terminal regions. Using microdialysis to determine extracellular dopamine levels in the NAc core and shell, Bassareo and Di Chiara (1997, 1999) observed that while food rewards preferentially evoked increases in dopamine concentration in the NAc shell, conditioned stimuli paired with those rewards only elicited dopamine release in the NAc core. Furthermore, dopamine release in response to conditioned stimuli paired with cocaine rewards occurs selectively in the NAc core as well (Ito and others 2000). Based on these findings, it has been tentatively suggested that dopamine transmission in the NAc core specifically mediates associative learning processes, whereas dopamine increases in the NAc shell reflect primary reinforcement (Di Chiara 2002).

The functional role of the NAc and its dopaminergic innervation during Pavlovian conditioning has been explored extensively using site-specific lesions and pharmacological manipulations. These studies have also identified distinctions between NAc core and shell subregions. Parkinson and colleagues (1999) used an autoshaping paradigm to train rats to associate the presence of a previously neutral stimulus with the delivery of a food reward. Selective lesions were then made to either the core or shell of the NAc, and rats underwent additional pairing sessions in which conditioned approach responses towards the reward-paired cue were monitored. Lesions to the NAc core (but not shell) significantly impaired the expression of these approach responses, indicating that CS-US associations were disrupted. Similarly, dopamine antagonism or depletion in the NAc core also produces a profound impairment in the ability of animals to learn and express conditioned approach responses (Di Ciano and others 2001; Parkinson and others 2002). By comparison, NMDA antagonism disrupts conditioned responses only during acquisition, whereas AMPA antagonism preferentially impairs the expression of Pavlovian approaches (Di Ciano and others 2001). Taken together, these findings suggest that the reliance of conditioned approach responses on an intact NAc core reflects the contributions of dopamine and glutamate transmission within this structure.

The role of dopamine in associative reward learning may be selectively mediated by specific receptor subtypes within the NAc. Dopamine D1 and D2 receptors oppositely modulate the same intracellular cascade, and D1 receptor antagonism inhibits long term potentiation of striatal synapses (Calabresi and others 2000b; Kerr and Wickens 2001). Consistent with the distinct cellular effects attributed to these receptors, Eyny and Horvitz (2003) reported that selective D1 and D2 antagonists also differentially affect stimulus-reward learning. In this study, the systemic blockade of D1 receptors produced a reduction in conditioned approaches towards reward-paired stimuli, while D2 antagonists actually promoted the expression of learned associations (Eyny and Horvitz 2003). Intra-NAc D1 receptor antagonism immediately after Pavlovian conditioning also blocks the performance of subsequent conditioned approach responses in an autoshaping task, indicating that D1 receptors in the NAc may play a vital role in the consolidation of learned stimulus-reward associations (Dalley and others 2005).

In addition to the dopaminergic projection from the VTA, a number of other structures may contribute specific information to the NAc during associative learning (Robbins and Everitt 2002). For example, excitotoxic lesions to the anterior cingulate cortex (ACC) impair the acquisition and performance of Pavlovian approach responses towards conditioned stimuli (Cardinal and others 2002). However, in contrast to NAc core lesions, ACC lesions do not abolish approach responses, but rather increase that likelihood that animals will approach non-predictive cues (Bussey and others 1997). One potential explanation for this effect is that the ACC rapidly acquires the ability to discriminate between stimuli and then “teaches” this discrimination to other regions, such as the NAc (Cardinal and others 2002; Robbins and Everitt 2002). In agreement with this view, disconnection lesions between the NAc core and ACC also impair the expression of learned associations (Parkinson and others 2000). Importantly, other brain structures may also contribute to stimulus-reward learning in a NAc-independent manner. Indeed, a number of studies have indicated that a separate neural circuit consisting of the central nucleus of the amygdala, substantia nigra pars compacta, and dorsolateral striatum mediates the learning and expression of conditioned orienting responses elicited by cues that predict favorable outcomes as well as the potentiation of feeding by conditioned stimuli (Han and others 1997; Lee and others 2005; El-Amamy and Holland 2006).

Human brain imaging studies during reward learning tasks have confirmed and extended experimental findings that implicate the NAc in associative learning (Knutson and Cooper 2005). A number of investigations using fMRI techniques to assess blood oxygenation have reported increased activity in the ventral striatum during exposure to rewards ranging from water to money to sexual stimuli (McClure and others 2004). Consistent with the animal literature, reward prediction is a key feature in this pattern of activation. Berns and others (2001) found that unpredicted delivery of a rewarding juice substance to a volunteer’s mouth evoked a significantly greater change in activity of the ventral striatum than when rewards were delivered in a predictable fashion. Moreover, when rewards are predicted by a discrete conditioned stimulus, this CS itself can evoke a change in activity in the ventral striatum (McClure and others 2003; Ramnani and others 2004). Notably, the ventral striatum seems to encode such deviations from reward prediction in both passive (Pavlovian) and active (operant) tasks, whereas the dorsal striatum is only activated by prediction errors that occur in an operant situation (O'Doherty and others 2004). Thus, the ventral striatum and the NAc may have a wider role in linking stimuli with outcomes in a number of experimental and real-life situations. In keeping with this idea, event-related fMRI techniques have recently been employed to examine information processing in these areas in relation to social interactions, gambling, and cognition (Knutson and Cooper 2005).

Relevance to drug addiction

Addiction to abused substances such as cocaine is typically characterized by cycles of compulsive drug use followed by periods of abstinence and resumption of drug consumption (relapse). In human addicts, it has been well documented that cues associated with prior drug use (such as places where the drug was consumed, people with whom the drug was taken or drug paraphernalia) are strong elicitors of drug craving and this craving is one of the leading causes of relapse (Gawin 1991; O'Brien and others 1998; Volkow and others 2006). Importantly, neural systems identified as critical for Pavlovain learning involving natural rewards (Figure 1) have been shown to be critically involved in the addiction process. Thus, Pavlovian learning that is beneficial to an organism under normal circumstances becomes maladaptive in the drug addicted individual.

A number of studies conducted in human cocaine addicts using neuroimaging techniques such as PET or fMRI support this view. In these studies, cocaine addicted individuals are typically placed in a brain scanner and shown videotapes of either neutral settings (e.g., nature scene) or cocaine-associated events (e.g., portraying subjects smoking cocaine). Cocaine craving (as assessed with questionnaires) is determined under each condition. Using this approach, several researchers have reported activation in specific brain regions linked with cocaine-craving during presentation of cocaine-associated cues including portions of the prefrontal cortex (e.g., anterior cingulate and orbitofrontal cortex; Breiter and others 1997; Childress and others 1999; Volkow and others 1999; Wexler and others 2001), limbic regions (Childress and others 1999) and more recently the dorsal striatum (Volkow and others 2006). These changes are typically correlated with the degree of self-reported drug craving and are not observed during presentation of neutral stimuli. Thus, in human addicts, stimuli associated with cocaine can evoke strong craving for the drug and activation of brain systems involved in natural reward-related behaviors.

Animal studies have provided additional insight into neural mechanisms underlying Pavlovian conditioning involving abused substances (Everitt and Wolf 2002; Robbins and Everitt 2002). It should be noted that in animal models of drug addiction Pavlovian contingencies are typically embedded within operant tasks. For example, a key animal model of human drug addiction is the drug self-administration procedure in which animals are trained to make an operant response (e.g., press a lever) for intravenous infusion of a drug such as cocaine. Typically, cocaine infusion is paired with a discrete audiovisual stimulus (CS). Studies show that presentation of a cocaine-associated CS alone can increase cell firing in the NAc (Carelli 2000) and BLA (Carelli and others 2003) and evoke rapid dopamine release in the NAc (Phillips and others 2003). This latter finding is illustrated in figure 4. In this case, an electrochemical technique (fast scan cyclic voltammetry) was used to measure dopamine release with subsecond temporal resolution (i.e., in 'real time') during CS presentation. As can be seen in Figure 4, dopamine was rapidly released during presentation of the audiovisual CS linked with cocaine infusion, but only in animals with a history of cocaine self-administration. Importantly, presentation of cocaine-associated conditioned stimuli can serve as powerful elicitors of relapse of drug taking following removal of drug availability in animal models of cocaine addiction (Shaham and others 2003).

Figure 4
Rapid dopamine release in the core of the NAc, as measured with fast scan cyclic voltammetry, is observed during presentation of an audiovisual CS associated with cocaine infusion during self-administration sessions. Top, dopamine concentration significantly ...

In general, the same brain regions involved in natural reward processing are activated by abused substances (Figure 1). However, electrophysiological studies employing multiple schedules of reinforcement (i.e., animals responded on one lever for cocaine and a second lever for water or food within the same behavioral session) revealed that distinct populations of NAc neurons differentially process information about goal-directed behaviors for cocaine versus natural reward (Carelli and others 2000; Carelli and Wondolowski 2003). Moreover, conditioned stimuli associated with cocaine self-administration only activate NAc neurons that process information about goal-directed behaviors for cocaine and not water reward (Carelli and Ijames 2001). Collectively, these findings reveal a complex neural system in which discrete microcircuits exist (at least at the level of the NAc) that selectively process information about drug versus ‘natural’ reward and conditioned stimuli associated with them (Carelli and Wightman 2004).

Concluding remarks

The ability of animals to successfully secure food and other rewards from their environment may appear simple in nature, but in fact involves a rather complex level of processing. The organism must remember where the reward is located in the environment and how to get to there, must be motivated to respond for it, and must complete the appropriate set of behaviors to attain and consume the substance. This skill is enhanced by the capacity to form associations between predictive cues and their outcomes - the basis of Pavlovian learning. Research involving animal and human subjects implicate that the NAc and interconnected brain nuclei play a critical role in this phenomenon. The NAc processes information about unconditioned as well as conditioned aspects of Pavlovian learning that can be used to ultimately guide behaviors toward rewards in their environment. This fundamental form of learning affords obvious survival value but can also have devastating negative consequences when such associations are linked with maladaptive circumstances, such as those involving abused substances.

Acknowledgments

This work was supported by grants from the National Institute of Drug Abuse R01 DA14339 and DA017318 to RMC and F31 DA021979 to JJD.

Contributor Information

Jeremy J. Day, Department of Psychology, The University of North Carolina at Chapel Hill, CB# 3270, Davie Hall, Chapel Hill, NC 27599-3270, Phone: 919-962-0419, jjday/at/email.unc.edu.

Regina M. Carelli, Department of Psychology, The University of North Carolina at Chapel Hill, CB# 3270, Davie Hall, Chapel Hill, NC 27599-3270, Phone: 919-962-8775, FAX: 919-962-2537, rcarelli/at/unc.edu.

References

  • Aberman JE, Salamone JD. Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement. Neuroscience. 1999;92:545–552. [PubMed]
  • Apicella P, Ljungberg T, Scarnati E, Schultz W. Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res. 1991;85:491–500. [PubMed]
  • Bassareo V, Di Chiara G. Differential influence of associative and nonassociative learning mechanisms on the responsiveness of prefrontal and accumbal dopamine transmission to food stimuli in rats fed ad libitum. J Neurosci. 1997;17:851–861. [PubMed]
  • Bassareo V, Di Chiara G. Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments. Neuroscience. 1999;89:637–641. [PubMed]
  • Berns GS, McClure SM, Pagnoni G, Montague PR. Predictability modulates human brain response to reward. J Neurosci. 2001;21:2793–2798. [PubMed]
  • Berridge KC, Robinson TE. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev. 1998;28:309–369. [PubMed]
  • Blackburn JR, Phillips AG, Fibiger HC. Dopamine and preparatory behavior: I. Effects of pimozide. Behav Neurosci. 1987;101:352–360. [PubMed]
  • Blackburn JR, Pfaus JG, Phillips AG. Dopamine functions in appetitive and defensive behaviours. Prog Neurobiol. 1992;39:247–279. [PubMed]
  • Breiter HC, Gollub RL, Weisskoff RM, Kennedy DN, Makris N, Berke JD, et al. Acute effects of cocaine on human brain activity and emotion. Neuron. 1997;19:591–611. [PubMed]
  • Brog JS, Salyapongse A, Deutch AY, Zahm DS. The patterns of afferent innervation of the core and shell in the "accumbens" part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J Comp Neurol. 1993;338:255–278. [PubMed]
  • Brown PL, Jenkins HM. Auto-shaping of the pigeon's key peck. J Exp Anal Behav. 1968;11:1–8. [PMC free article] [PubMed]
  • Burns M, Domjan M. Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica) J Exp Psychol Anim Behav Process. 1996;22:297–306. [PubMed]
  • Burns M, Domjan M. Topography of spatially directed conditioned responding: effects of context and trial duration. J Exp Psychol Anim Behav Process. 2001;27:269–278. [PubMed]
  • Bussey TJ, Everitt BJ, Robbins TW. Dissociable effects of cingulate and medial frontal cortex lesions on stimulus-reward learning using a novel Pavlovian autoshaping procedure for the rat: implications for the neurobiology of emotion. Behav Neurosci. 1997;111:908–919. [PubMed]
  • Calabresi P, Centonze D, Gubellini P, Marfia GA, Pisani A, Sancesario G, et al. Synaptic transmission in the striatum: from plasticity to neurodegeneration. Prog Neurobiol. 2000a;61:231–265. [PubMed]
  • Calabresi P, Gubellini P, Centonze D, Picconi B, Bernardi G, Chergui K, et al. Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. J Neurosci. 2000b;20:8443–8451. [PubMed]
  • Cardinal RN, Parkinson JA, Lachenal G, Halkerston KM, Rudarakanchana N, Hall J, et al. Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav Neurosci. 2002;116:553–567. [PubMed]
  • Carelli RM. Activation of accumbens cell firing by stimuli associated with cocaine delivery during self-administration. Synapse. 2000;35:238–242. [PubMed]
  • Carelli RM. Nucleus accumbens cell firing during goal-directed behaviors for cocaine vs. 'natural' reinforcement. Physiol Behav. 2002;76:379–387. [PubMed]
  • Carelli RM, Deadwyler SA. A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J Neurosci. 1994;14:7735–7746. [PubMed]
  • Carelli RM, Deadwyler SA. Cellular mechanisms underlying reinforcement-related processing in the nucleus accumbens: electrophysiological studies in behaving animals. Pharmacol Biochem Behav. 1997;57:495–504. [PubMed]
  • Carelli RM, Ijames SG. Selective activation of accumbens neurons by cocaine-associated stimuli during a water/cocaine multiple schedule. Brain Res. 2001;907:156–161. [PubMed]
  • Carelli RM, Wondolowski J. Selective encoding of cocaine versus natural rewards by nucleus accumbens neurons is not related to chronic drug exposure. J Neurosci. 2003;23:11214–11223. [PubMed]
  • Carelli RM, Wightman RM. Functional microcircuitry in the accumbens underlying drug addiction: insights from real-time signaling during behavior. Curr Opin Neurobiol. 2004;14:763–768. [PubMed]
  • Carelli RM, Ijames SG, Crumling AJ. Evidence that separate neural circuits in the nucleus accumbens encode cocaine versus "natural" (water and food) reward. J Neurosci. 2000;20:4255–4266. [PubMed]
  • Carelli RM, Williams JG, Hollander JA. Basolateral amygdala neurons encode cocaine self-administration and cocaine-associated cues. J Neurosci. 2003;23:8204–8211. [PubMed]
  • Carr DB, Sesack SR. Dopamine terminals synapse on callosal projection neurons in the rat prefrontal cortex. J Comp Neurol. 2000;425:275–283. [PubMed]
  • Cepeda C, Levine MS. Dopamine and N-methyl-D-aspartate receptor interactions in the neostriatum. Dev Neurosci. 1998;20:1–18. [PubMed]
  • Childress AR, Mozley PD, McElgin W, Fitzgerald J, Reivich M, O'Brien CP. Limbic activation during cue-induced cocaine craving. Am J Psychiatry. 1999;156:11–18. [PMC free article] [PubMed]
  • Dalley JW, Laane K, Theobald DE, Armstrong HC, Corlett PR, Chudasama Y, et al. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc Natl Acad Sci U S A. 2005;102:6189–6194. [PubMed]
  • Day JJ, Wheeler RA, Roitman MF, Carelli RM. Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm. Eur J Neurosci. 2006;23:1341–1351. [PubMed]
  • Di Chiara G. Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res. 2002;137:75–114. [PubMed]
  • Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J Neurosci. 2001;21:9471–9477. [PubMed]
  • El-Amamy H, Holland PC. Substantia nigra pars compacta is critical to both the acquisition and expression of learned orienting of rats. Eur J Neurosci. 2006;24:270–276. [PubMed]
  • Everitt BJ, Robbins TW. The amygdala: Neurobiological aspects of emotion, memory, and mental dysfunction. New York: Wiley-Liss; 1992. Amygdala-ventral striatal interactions in reward-related processes; pp. 401–429.
  • Everitt BJ, Wolf ME. Psychomotor stimulant addiction: a neural systems perspective. J Neurosci. 2002;22:3312–3320. [PubMed]
  • Eyny YS, Horvitz JC. Opposing roles of D1 and D2 receptors in appetitive conditioning. J Neurosci. 2003;23:1584–1587. [PubMed]
  • Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. [PubMed]
  • Fouriezos G, Wise RA. Pimozide-induced extinction of intracranial self-stimulation: response patterns rule out motor or performance deficits. Brain Res. 1976;103:377–380. [PubMed]
  • French SJ, Totterdell S. Hippocampal and prefrontal cortical inputs monosynaptically converge with individual projection neurons of the nucleus accumbens. J Comp Neurol. 2002;446:151–165. [PubMed]
  • French SJ, Totterdell S. Individual nucleus accumbens-projection neurons receive both basolateral amygdala and ventral subicular afferents in rats. Neuroscience. 2003;119:19–31. [PubMed]
  • Garris PA, Kilpatrick M, Bunin MA, Michael D, Walker QD, Wightman RM. Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature. 1999;398:67–69. [PubMed]
  • Gawin FH. Cocaine addiction: psychology and neurophysiology. Science. 1991;251:1580–1586. [PubMed]
  • Groenewegen HJ, Vermeulen-Van der Zee E, te Kortschot A, Witter MP. Organization of the projections from the subiculum to the ventral striatum in the rat. A study using anterograde transport of Phaseolus vulgaris leucoagglutinin. Neuroscience. 1987;23:103–120. [PubMed]
  • Groenewegen HJ, Berendse HW, Meredith GE, Haber SN, Voorn P, Walters JG, et al. Functional anatomy of the ventral, limbic system-innervated striatum. In: Willner P, Scheel-Kruger J, editors. The mesolimbic dopamine system: From motivation to action. New York: John Wiley; 1991. pp. 19–59.
  • Groves PM. A theory of the functional organization of the neostriatum and the neostriatal control of voluntary movement. Brain Res. 1983;286:109–1232. [PubMed]
  • Han JS, McMahan RW, Holland P, Gallagher M. The role of an amygdalo-nigrostriatal pathway in associative learning. J Neurosci. 1997;17:3913–3919. [PubMed]
  • Hassani OK, Cromwell HC, Schultz W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol. 2001;85:2477–2489. [PubMed]
  • Heimer L, Zahm DS, Alheid GF. Basal ganglia. In: Paxinos G, editor. The rat nervous system. 2nd Edition. San Diego: Academic Press; 1995. pp. 579–628.
  • Heimer L, Zahm DS, Churchill L, Kalivas PW, Wohltmann C. Specificity in the projection patterns of accumbal core and shell in the rat. Neuroscience. 1991;41:89–125. [PubMed]
  • Heimer L, Alheid GF, de Olmos JS, Groenewegen HJ, Haber SN, Harlan RE, et al. The accumbens: beyond the core-shell dichotomy. J Neuropsychiatry Clin Neurosci. 1997;9:354–381. [PubMed]
  • Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci. 1998;1:304–309. [PubMed]
  • Ikemoto S, Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Brain Res Rev. 1999;31:6–41. [PubMed]
  • Ito R, Dalley JW, Howes SR, Robbins TW, Everitt BJ. Dissociation in conditioned dopamine release in the nucleus accumbens core and shell in response to cocaine cues and during cocaine-seeking behavior in rats. J Neurosci. 2000;20:7489–7495. [PubMed]
  • Jenkins HM, Moore BR. The form of the auto-shaped response with food or water reinforcers. J Exp Anal Behav. 1973;20:163–181. [PMC free article] [PubMed]
  • Kelley AE. Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev. 2004;27:765–776. [PubMed]
  • Kelley AE, Bless EP, Swanson CJ. Investigation of the effects of opiate antagonists infused into the nucleus accumbens on feeding and sucrose drinking in rats. J Pharmacol Exp Ther. 1996;278:1499–1507. [PubMed]
  • Kerr JN, Wickens JR. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J Neurophysiol. 2001;85:117–124. [PubMed]
  • Knutson B, Cooper JC. Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol. 2005;18:411–417. [PubMed]
  • Lee HJ, Groshek F, Petrovich GD, Cantalini JP, Gallagher M, Holland PC. Role of amygdalo-nigral circuitry in conditioning of a visual stimulus paired with food. J Neurosci. 2005;25:3881–3888. [PMC free article] [PubMed]
  • McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron. 2003;38:339–346. [PubMed]
  • McClure SM, York MK, Montague PR. The neural substrates of reward processing in humans: the modern role of FMRI. Neuroscientist. 2004;10:260–268. [PubMed]
  • McGeorge AJ, Faull RL. The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience. 1989;29:503–537. [PubMed]
  • Meredith GE, Agolia R, Arts MP, Groenewegen HJ, Zahm DS. Morphological differences between projection neurons of the core and shell in the nucleus accumbens of the rat. Neuroscience. 1992;50:149–162. [PubMed]
  • Mirenowicz J, Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol. 1994;72:1024–1027. [PubMed]
  • Mogenson GJ. Limbic-motor integration. Progress in Psychobiology and Physiological Psychology. 1987;12:117–169.
  • Nicola SM, Yun IA, Wakabayashi KT, Fields HL. Firing of nucleus accumbens neurons during the consummatory phase of a discriminative stimulus task depends on previous reward predictive cues. J Neurophysiol. 2004;91:1866–1882. [PubMed]
  • O'Brien CP, Childress AR, Ehrman R, Robbins SJ. Conditioning factors in drug abuse: can they explain compulsion? J Psychopharmacol. 1998;12:15–22. [PubMed]
  • O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–454. [PubMed]
  • O'Donnell P, Grace AA. Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J Neurosci. 1995;15:3622–3639. [PubMed]
  • Parkinson JA, Willoughby PJ, Robbins TW, Everitt BJ. Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. Behav Neurosci. 2000;114:42–63. [PubMed]
  • Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine. J Neurosci. 1999;19:2401–2411. [PubMed]
  • Parkinson JA, Dalley JW, Cardinal RN, Bamford A, Fehnert B, Lachenal G, et al. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav Brain Res. 2002;137:149–163. [PubMed]
  • Pavlov IP. Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. Oxford: Oxford University Press; 1927.
  • Pecina S, Berridge KC. Opioid site in nucleus accumbens shell mediates eating and hedonic 'liking' for food: map based on microinjection Fos plumes. Brain Res. 2000;863:71–86. [PubMed]
  • Pecina S, Berridge KC. Hedonic hot spot in nucleus accumbens shell: where do mu-opioids cause increased hedonic impact of sweetness? J Neurosci. 2005;25:11777–11786. [PubMed]
  • Pecina S, Berridge KC, Parker LA. Pimozide does not shift palatability: separation of anhedonia from sensorimotor suppression by taste reactivity. Pharmacol Biochem Behav. 1997;58:801–811. [PubMed]
  • Pecina S, Cagniard B, Berridge KC, Aldridge JW, Zhuang X. Hyperdopaminergic mutant mice have higher "wanting" but not "liking" for sweet rewards. J Neurosci. 2003;23:9395–9402. [PubMed]
  • Pennartz CM, Groenewegen HJ, Lopes da Silva FH. The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological and anatomical data. Prog Neurobiol. 1994;42:719–761. [PubMed]
  • Peterson GB, Ackil JE, Frommer GP, Hearst E. Conditioned approach and contact behavior towards signals for food and brain-stimulation reinforcement. Science. 1972;177:1009–1011. [PubMed]
  • Phillips PE, Stuber GD, Heien ML, Wightman RM, Carelli RM. Subsecond dopamine release promotes cocaine seeking. Nature. 2003;422:614–618. [PubMed]
  • Pinto A, Sesack SR. Limited collateralization of neurons in the rat prefrontal cortex that project to the nucleus accumbens. Neuroscience. 2000;97:635–642. [PubMed]
  • Ramnani N, Elliott R, Athwal BS, Passingham RE. Prediction error for free monetary reward in the human prefrontal cortex. Neuroimage. 2004;23:777–786. [PubMed]
  • Robbins TW, Everitt BJ. Limbic-striatal memory systems and drug addiction. Neurobiol Learn Mem. 2002;78:625–636. [PubMed]
  • Robinson DL, Venton BJ, Heien ML, Wightman RM. Detecting subsecond dopamine release with fast-scan cyclic voltammetry in vivo. Clin Chem. 2003;49:1763–1773. [PubMed]
  • Robinson DL, Phillips PE, Budygin EA, Trafton BJ, Garris PA, Wightman RM. Sub-second changes in accumbal dopamine during sexual behavior in male rats. Neuroreport. 2001;12:2549–2552. [PubMed]
  • Roitman MF, Wheeler RA, Carelli RM. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron. 2005a;45:587–597. [PubMed]
  • Roitman MF, Stuber GD, Phillips PE, Wightman RM, Carelli RM. Dopamine operates as a subsecond modulator of food seeking. J Neurosci. 2004;24:1265–1271. [PubMed]
  • Roitman MF, Stuber GD, Heien ML, Wightman RM, Carelli RM. 2005 Abstract Viewer/Itinerary Planner. Washington, DC: Society for Neuroscience; 2005b. Subsecond dopamine release is selectively evoked by rewarding and not aversive taste stimuli. Online.
  • Salamone JD. The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res. 1994;61:117–133. [PubMed]
  • Salamone JD, Correa M, Mingote S, Weber SM. Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. J Pharmacol Exp Ther. 2003;305:1–8. [PubMed]
  • Salamone JD, Correa M, Mingote SM, Weber SM. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr Opin Pharmacol. 2005;5:34–41. [PubMed]
  • Salamone JD, Arizzi MN, Sandoval MD, Cervone KM, Aberman JE. Dopamine antagonists alter response allocation but do not suppress appetite for food in rats: contrast between the effects of SKF 83566, raclopride, and fenfluramine on a concurrent choice task. Psychopharmacology (Berl) 2002;160:371–380. [PubMed]
  • Salamone JD, Steinpreis RE, McCullough LD, Smith P, Grebel D, Mahan K. Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology (Berl) 1991;104:515–521. [PubMed]
  • Schultz W. Reward signaling by dopamine neurons. Neuroscientist. 2001;7:293–302. [PubMed]
  • Schultz W, Apicella P, Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci. 1993;13:900–913. [PubMed]
  • Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. [PubMed]
  • Sesack SR, Pickel VM. In the rat medial nucleus accumbens, hippocampal and catecholaminergic terminals converge on spiny neurons and are in apposition to each other. Brain Res. 1990;527:266–279. [PubMed]
  • Setlow B, Schoenbaum G, Gallagher M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron. 2003;38:625–636. [PubMed]
  • Shaham Y, Shalev U, Lu L, De Wit H, Stewart J. The reinstatement model of drug relapse: history, methodology and major findings. Psychopharmacology (Berl) 2003;168:3–20. [PubMed]
  • Sidman M, Fletcher FG. A demonstration of auto-shaping with monkeys. J Exp Anal Behav. 1968;11:307–309. [PMC free article] [PubMed]
  • Sutton RS, Barto AG. Reinforcement Learning. Cambridge, MA: MIT Press; 1998.
  • Taha SA, Fields HL. Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens. J Neurosci. 2005;25:1193–1202. [PubMed]
  • Taha SA, Fields HL. Inhibitions of nucleus accumbens neurons encode a gating signal for reward-directed behavior. J Neurosci. 2006;26:217–222. [PubMed]
  • Tindell AJ, Smith KS, Pecina S, Berridge KC, Aldridge JW. Ventral pallidum firing codes hedonic reward: when a bad taste turns good. J Neurophysiol. 2006 [PubMed]
  • Tobler PN, Fiorillo CD, Schultz W. Adaptive coding of reward value by dopamine neurons. Science. 2005;307:1642–1645. [PubMed]
  • Totterdell S, Smith AD. Convergence of hippocampal and dopaminergic input onto identified neurons in the nucleus accumbens of the rat. J Chem Neuroanat. 1989;2:285–298. [PubMed]
  • Ungless MA. Dopamine: the salient issue. Trends Neurosci. 2004;27:702–706. [PubMed]
  • Uslaner JM, Acerbo MJ, Jones SA, Robinson TE. The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine. Behav Brain Res. 2006;169:320–324. [PubMed]
  • Van Bockstaele EJ, Pickel VM. Ultrastructure of serotonin-immunoreactive terminals in the core and shell of the rat nucleus accumbens: cellular substrates for interactions with catecholamine afferents. J Comp Neurol. 1993;334:603–617. [PubMed]
  • van Dongen YC, Deniau JM, Pennartz CM, Galis-de Graaf Y, Voorn P, Thierry AM, et al. Anatomical evidence for direct connections between the shell and core subregions of the rat nucleus accumbens. Neuroscience. 2005;136:1049–1071. [PubMed]
  • Volkow ND, Wang GJ, Fowler JS, Hitzemann R, Angrist B, Gatley SJ, et al. Association of methylphenidate-induced craving with changes in right striato-orbitofrontal metabolism in cocaine abusers: implications in addiction. Am J Psychiatry. 1999;156:19–26. [PubMed]
  • Volkow ND, Wang GJ, Telang F, Fowler JS, Logan J, Childress AR, et al. Cocaine cues and dopamine in dorsal striatum: mechanism of craving in cocaine addiction. J Neurosci. 2006;26:6583–6588. [PubMed]
  • Voorn P, Gerfen CR, Groenewegen HJ. Compartmental organization of the ventral striatum of the rat: immunohistochemical distribution of enkephalin, substance P, dopamine, and calcium-binding protein. J Comp Neurol. 1989;289:189–201. [PubMed]
  • Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 2004;27:468–474. [PubMed]
  • Waelti P, Dickinson A, Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature. 2001;412:43–48. [PubMed]
  • Wan X, Peoples LL. Firing patterns of accumbal neurons during a pavlovian-conditioned approach task. J Neurophysiol. 2006;96:652–660. [PubMed]
  • Wasserman EA. Pavlovian conditioning with heat reinforcement produces stimulus-directed pecking in chicks. Science. 1973;181:875–877. [PubMed]
  • Wexler BE, Gottschalk CH, Fulbright RK, Prohovnik I, Lacadie CM, Rounsaville BJ, et al. Functional magnetic resonance imaging of cocaine craving. Am J Psychiatry. 2001;158:86–95. [PubMed]
  • Wheeler RA, Roitman MF, Grigson PS, Carelli RM. Single neurons in the nucleus accumbens track relative reward. International Journal of Comparative Psychology. 2005;18:320–332.
  • Wilcove WG, Miller JC. CS-USC presentations and a lever: human autoshaping. J Exp Psychol. 1974;103:868–877. [PubMed]
  • Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5:483–494. [PubMed]
  • Wise RA, Bozarth MA. Brain mechanisms of drug reward and euphoria. Psychiatr Med. 1985;3:445–460. [PubMed]
  • Wise RA, Spindler J, Legault L. Major attenuation of food reward with performance-sparing doses of pimozide in the rat. Can J Psychol. 1978a;32:77–85. [PubMed]
  • Wise RA, Spindler J, deWit H, Gerberg GJ. Neuroleptic-induced "anhedonia" in rats: pimozide blocks reward quality of food. Science. 1978b;201:262–264. [PubMed]
  • Wise RA, Bauco P, Carlezon WA, Jr, Trojniar W. Self-stimulation and drug reward mechanisms. Ann N Y Acad Sci. 1992;654:192–198. [PubMed]
  • Wright CI, Beijer AV, Groenewegen HJ. Basal amygdaloid complex afferents to the rat nucleus accumbens are compartmentally organized. J Neurosci. 1996;16:1877–1893. [PubMed]
  • Zahm DS. Functional-anatomical implications of the nucleus accumbens core and shell subterritories. Ann N Y Acad Sci. 1999;877:113–128. [PubMed]
  • Zahm DS. An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci Biobehav Rev. 2000;24:85–105. [PubMed]
  • Zahm DS, Brog JS. On the significance of subterritories in the "accumbens" part of the rat ventral striatum. Neuroscience. 1992;50:751–767. [PubMed]
  • Zahm DS, Heimer L. Specificity in the efferent projections of the nucleus accumbens in the rat: comparison of the rostral pole projection patterns with those of the core and shell. J Comp Neurol. 1993;327:220–232. [PubMed]