|Home | About | Journals | Submit | Contact Us | Français|
Electrical stimulation of the medial forebrain bundle can reward arbitrary acts or motivate biologically primitive, species-typical behaviors like feeding or copulation. The sub-systems involved in these behaviors are only partially characterized, but they appear to trans-synaptically activate the mesocorticolimbic dopamine system. Basal function of the dopamine system is essential for arousal and motor function; phasic activation of this system is rewarding and can potentiate the effectiveness of reward-predictors that guide learned behaviors. This system is phasically activated by most drugs of abuse and such activation contributes to the habit-forming actions of these drugs.
The finding that rats return to the portions of the environment where they have received electrical stimulation of certain brain regions (Olds and Milner, 1954; Olds, 1956) has led to a half-century of intensive investigation of what has come to be known as “brain reward circuitry.” This electrical stimulation establishes instrumental response habits such as lever-pressing or alley-running much as do natural reinforcers like food for hungry animals (Beninger et al., 1977). The sites at which brain stimulation is reinforcing are varied and widespread, and we have only modest knowledge of the degree to which there is connectivity between them (Phillips, 1984; Wise and Bozarth, 1984). Just as electrical stimulation of the brain can be reinforcing, so can chemical stimulation by intracranial microinjections of neurotransmitters and drugs (Ikemoto and Wise, 2004). For the most part, the chemical trigger zones currently identified with the reinforcing effects of drugs, particularly drugs of abuse, have been linked to reward circuitry first identified by electrical stimulation studies.
The circuit elements most strongly implicated in reward function are the mesolimbic, mesocortical, and nigrostriatal dopamine systems. Pharmacological blockade of these systems (more correctly identified as sub-systems, inasmuch as they have a common embryologic origin and largely parallel projections) blocks or attenuates the reinforcing effects of hypothalamic brain stimulation, several drugs of abuse, and food or water for hungry or thirsty animals. In the present paper I review the evidence linking dopamine circuitry to reward function, indicate the afferents and efferents of the dopamine system that are also linked to reward function, and discuss the relationship of brain reward circuitry to the motivation of reward-seeking behaviors.
The most extensively studied substrate of reward function is embedded among the dozens of fiber systems (Veening et al., 1982) that comprise the hypothalamic portion of the medial forebrain bundle (MFB). The rewarding effects of MFB stimulation are attenuated by performance-sparing doses of dopamine antagonists (Fouriezos et al., 1978; Fouriezos and Wise, 1976; Franklin, 1978; Franklin and McCoy, 1979; Gallistel et al., 1982; Gallistel and Freyd, 1987; Gallistel and Karras, 1984; Zarevics and Setler, 1979) and are enhanced by indirect dopamine agonists such as cocaine and amphetamine and by drugs that stimulate or disinhibit the forebrain dopamine systems such as nicotine or morphine (Wise, 1996).
The sensitivity to such drugs and the fact that the anatomical dispersion of the reward system corresponds with the dorsal and ventral boundaries of the dopaminergic cell groups of the ventral tegmental area and substantia nigra (Corbett and Wise, 1980; Wise, 1981) initially suggested that MFB brain stimulation was rewarding because it directly activated ascending dopaminergic projections. Against this possibility was the fact that the optimal stimulation frequencies for activating the dopamine system are much lower than the optimal frequencies for supporting the behavior (Wise, 1978). Subsequent parametric studies have confirmed that, because of their high threshold (Yeomans et al., 1988), few dopamine fibers are directly activated by the stimulation in most self-stimulation studies. Under the usual conditions of testing, the directly activated fiber population is characterized by short refractory periods and fast conduction velocities typical of large myelinated fibers and inconsistent with the small unmyelinated dopaminergic fibers of the MFB (Gallistel et al., 1981). Indeed, rewarding MFB stimulation appears to depend on at least two sub-populations of directly activated fibers; a cholinergic population with ultra-short (0.5-0.6msec) refractory periods and a second population, unidentified as to neurotransmitter or neurotransmitters, with somewhat longer (0.7-2.5msec) refractory periods (Gratton and Wise, 1985). Collision tests suggest that the bulk of the directly activated fibers descend the medial forebrain bundle (Bielajew and Shizgal, 1986) to the level of the ventral tegmental area (VTA) and substantia nigra and perhaps beyond (Boye and Rompré, 1996).
While MFB reward sites between the lateral hypothalamus and ventral tegmental area appear to involve the same fiber system, neither the origin nor the termination of that system is clearly established. While it was once thought that a single MFB system traversed the length of the hypothalamus, the fibers connecting the lateral preoptic area and the lateral hypothalamus appear to have more varied conduction velocities than those connecting the lateral hypothalamus with the ventral tegmental area (Bielejew et al., 2001). Moreover, while there is connectivity between lateral preoptic and lateral hypothalamic reward sites and between lateral hypothalamic and ventral tegmental reward sites, connectivity between lateral preoptic and ventral tegmental sites has been difficult to document (Bielajew et al., 2000). And while it was once suggested that the termination was at the dopaminergic cell bodies of the ventral tegmental area and substantia nigra (Wise, 1980; Yeomans, 1982), a more recent hypothesis is that the bulk of the directly activated MFB reward fibers project through the ventral tegmental area and substantia nigra to the cholinergic neurons of the pedunculopontine or laterodorsal tegmental nucleus, from which the reward signal is relayed back to the ventral tegmental area (Rada et al., 2000; Yeomans et al., 1993; Yeomans et al., 2000). Note that unless there is antidromic activation of this cholinergic pathway it cannot account for the cholinergic portion of the directly-activated MFB fibers.
Attempts to establish connectivity between the reward circuitry of the MFB and rewarding stimulation sites in the rest of the brain have been limited; the greatest effort has focused on the possible relationship of prefrontal cortex reward sites with MFB reward circuitry. While stimulation of the dopamine-innervated portions of the frontal cortex of the rat is rewarding (Routtenberg and Sloan, 1972), the directly activated elements of mPFC self-stimulation have longer refractory periods (Schenk and Shizgal, 1982) and different strength-duration profiles (Schenk and Shizgal, 1985) than the directly activated elements of MFB self-stimulation; thus if the two are connected the connection is trans-synaptic. Moreover, mPFC self-stimulation, unlike MFB self-stimulation, is relatively insensitive to amphetamine, changes in current intensity, and food deprivation (Goodall and Carey, 1975; Robertson et al., 1981). Finally, large lesions of the MFB have failed to affect mPFC self-stimulation (Corbett et al., 1982). Rather, mPFC self-stimulation is disrupted by bilateral knife cuts of efferents from the mPFC to the sulcal prefrontal cortex (Corbett et al., 1982). Nonetheless, rewarding mPFC stimulation activates the mesolimbic dopamine system and such activation and medial prefrontal self-stimulation are blocked by perfusion of the excitatory amino antagonist kynurenic acid into the VTA (You et al., 1998). Thus there appears to be some degree of connectivity between MFB and mPFC reward sites.
Another approach to characterizing reward circuitry in the brain involves using intracranial microinjections of drugs rather than electrical stimulation to reward an arbitrary behavior such as lever-pressing. Early studies of this type by Olds (Olds and Olds, 1958) were discontinued because of the finding that physico-chemical rather than the neurotransmitter properties of the injected agents (calcium chelation in this case) could be rewarding (Olds et al., 1961). Subsequent studies of carbachol- and angiotensin- induced drinking revealed that injected substances often flow up the shaft of the injection cannula, particularly if it penetrates the lateral ventricle, to act at sites distal to the injection site (Simpson and Routtenberg, 1972; Johnson and Epstein, 1975). Failures to control for these artifacts flaw many central injection studies. Nonetheless, by using antagonists or inactive enantiomers to confirm that the effects of a drug are receptor-mediated, and by confirming a local site of action by comparing with adjacent (particularly dorsal) control injections, several investigators have identified drug reward sites in intracranial drug self-administration studies.
Several MFB-related reward sites have been identified. Mu and delta opioids are self-administered into the VTA (Bozarth and Wise, 1981; Welzl et al., 1989; Devine and Wise, 1994; Zangen et al., 2002); these rewarding effects of mu opioids appear to be localized to the caudal VTA (Zangen et al., 2002). The rewarding effects of mu opioids in the VTA are thought to be due to disinhibition: mu opioids inhibit nearby GABAergic neurons that normally hold the dopamine under inhibitory control (Johnson and North, 1992). GABAA agonists (Ikemoto et al., 1998) and the cholinergic agonist carbachol (Ikemoto and Wise, 2002) are also self-adminstered more readily into the caudal than the rostral VTA; the best sites for carbachol are in a thin band just dorsal to the interpeduncular nucleus (Ikemoto and Wise, 2002). GABAA agonists, like opioids, are presumed to be rewarding because they disinhibit the dopamine system by inhibiting its GABAergic neighbors; self-administration of GABAA agonists into the posterior VTA are antagonized by GABAA antagonists in the posterior VTA (Ikemoto et al., 1998). Carbachol is thought to be rewarding because it activates the dopamine system directly (Blaha et al., 1996); VTA carbachol self-administration is attenuated by the D1 dopamine antagonist SCH 23390 (Ikemoto and Wise, 2002). GABAAantagonists, on the other hand, are self-administered preferentially into the anterior VTA (Ikemoto et al., 1997), where they appear to diffuse to act at the supramammillary nucleus (Ikemoto, 2005); the rewarding effects of anterior VTA and supramammillary nucleus injections of GABAA antagonists are blocked by anterior VTA and supramammillary injections of GABAA agonists (Ikemoto, 2005; Ikemoto et al., 1997). GABAA agonists and antagonists injected into the middle of the VTA can each be rewarding in the conditioned place preference paradigm (Laviolette and van der Kooy, 2001), where rate of diffusion from the injection site is not so critical.
Other interesting drug reward sites include the nucleus accumbens, where amphetamine (Hoebel et al., 1983), cocaine (Carlezon and Wise, 1996; Ikemoto, 2003), nomifensine (Carlezon et al., 1995), morphine (Olds, 1982), phencyclidine (Carlezon and Wise, 1996), and met-enkephalin (Goeders et al., 1984) are self-administered, and the mPFC (Goeders et al., 1986; Goeders and Smith, 1986) and olfactory tubercle (Ikemoto, 2003), where cocaine is self-administered. Nucleus accumbens amphetamine is presumably rewarding because it causes local release of dopamine; this reward appears to be antagonized by either D1 or D2 dopamine antagonists or by their combination (Phillips et al., 1994). Phencyclidine, which also blocks the dopamine transporter (Gerhardt et al., 1987), is rewarding primarily because it blocks NMDA-type glutamate receptors (Carlezon and Wise, 1996); the dopamine transporter has much weaker affinity than the NMDA receptor for phencyclidine (Chaudieu et al., 1989; Ohmori et al., 1992), and other, selective, NMDA antagonists are also self-administered into this region (Carlezon and Wise, 1996). The effects of the dopamine uptake blocker nomifensine are blocked by a D2 dopamine antagonist while the effects of NMDA antagonists are not (Carlezon and Wise, 1996). Enkephalin is presumed to be rewarding because of actions on medium spiny output neurons.
Dopamine is the most strongly implicated neurotransmitter in reward function (Wise, 2004; Wise and Rompré, 1989). Most addictive drugs elevate brain dopamine (Di Chiara and Imperato, 1988), and the psychomotor stimulants amphetamine (Yokel and Wise, 1975), cocaine (de Wit and Wise, 1977), and nicotine (Corrigall et al., 1992) are rewarding because they do so. Brain dopamine does not seem to be necessary for all reward; dopamine-deficient mutant mice, if medicated with caffeine, have been shown capable of learning a maze task for food (Robinson et al., 2005). Nor is the role of dopamine limited to reward: dopaminergic neurons respond to painful (Becerra et al., 2001; Franklin, 1989), stressful (Roth et al., 1988) and novel (Dommett et al., 2005) stimuli as well as to prototypical rewards. Nonetheless, food, brain stimulation, and psychomotor stimulant rewards are not normally effective in animals with impaired dopamine function.
The bulk of the pharmacological evidence for a role of dopamine in reward function comes from studies in which animals are trained normally in an instrumental task and then tested under the influence of a dopamine antagonist. In such studies, normally rewarding events fail to maintain established response habits. Numerous procedures have been designed to discriminate between animals that are no longer rewarded for responding and animals that are no longer capable of responding, and it is clear that performance impairment cannot explain the deficits in instrumental behavior that can be caused by moderate doses of dopamine antagonists (Yokel and Wise, 1975; Fouriezos and Wise, 1976; Wise et al., 1978a; Wise et al., 1978b; Fouriezos et al., 1978; Franklin, 1978; Franklin and McCoy, 1979; Zarevics and Setler, 1979; Wise, 1982; Gallistel et al., 1982;Beninger and Freedman, 1982; Gallistel and Karras, 1984; Geary and Smith, 1985; Ettenberg and Camp, 1986a; Ettenberg and Camp, 1986b; Gallistel and Freyd, 1987; McFarland and Ettenberg, 1995). Thus forebrain dopamine systems (Wise, 1980; Yeomans, 1982; Yeomans et al., 1993) and their GABAergic efferents (Olds, 1982; Carlezon and Wise, 1996; Laviolette and van der Kooy, 2001) are viewed as the final common links in brain reward circuitry.
The nucleus accumbens is the dopamine terminal field most strongly implicated in reward function. The rewarding effects of intra-accumbens amphetamine are antagonized by intra-accumbens microinjections of dopamine antagonists (Phillips et al., 1994), thus implicating dopamine receptors on nucleus accumbens medium spiny neurons in reward function. Similarly intracranial self-administration of NMDA antagonists—not blocked by dopamine antagonists—implicates nucleus accumbens neurons, presumably medium spiny output neurons, in reward function (Carlezon and Wise, 1996). Enkephalin self-administration into nucleus accumbens is also thought to target medium spiny neurons (Goeders et al., 1984). However, nucleus accumbens is not the only site for dopamine-associated reward function. Local blockade of intracranial cocaine self-administration implicates dopamine receptors in mPFC (Goeders et al., 1986) and olfactory tubercle (Ikemoto, 2003) as well. Moreover, VTA microinjections of a D1 dopamine antagonist appear to attenuate the rewarding effect of intravenous cocaine; the mechanism of this effect is not fully understood but dendritically released dopamine is implicated (Ranaldi and Wise, 2001). Thus a variety of efferents of the mesocorticolimbic dopamine system are implicated in reward function.
Several afferents to the dopamine system are also implicated. Nicotinic cholinergic receptors in the VTA appear to be important for nicotine's rewarding actions. Intravenous nicotine self-administration is attenuated by lesions of VTA dopamine neurons or by intraventricular treatment with the nicotinic antagonist chlorisondamine (Corrigall et al., 1992). Chlorisondamine is a nicotinic channel blocker that is taken up by dopaminergic neurons and concentrated in dopaminergic cell bodies for months (El-Bizri et al., 1995). It blocks nicotinic actions on the dopamine system for 10 weeks or longer and blocks the ability of nicotine to potentiate lateral hypothalamic brain stimulation reward for at least three weeks (Wise et al., 1998); this action of nicotine is thought to be mediated at the level of the dopamine cell body or dendrites. This interpretation is consistent with the finding that the rewarding effects of VTA carbachol are attenuated by co-infusion of the nicotinic antagonist dihydro-β-erythroidine (Ikemoto and Wise, 2002). Muscarinic input to VTA is also implicated in reward function (Yeomans et al., 1993). Muscarinic M5 receptors on VTA dopamine neurons are thought to be involved (Yeomans et al., 2000). Thus nicotinic and muscarinic receptor activation in the VTA each contributes to reward function (Yeomans and Baptista, 1997).
The rewarding action of posterior VTA muscimol is blocked by the GABAA antagonist picrotoxin (Ikemoto et al., 1998), implicating GABAA receptors in this region in reward function. Mu and delta opioid reward in the VTA (Devine and Wise, 1994) are proportional to their ability to activate the mesolimbic dopamine system (Devine et al., 1993). Few delta opioid receptors are expressed in the VTA. Nonetheless, delta opioids are rewarding there and are not rewarding—and are not effective in activating the dopamine system—when injected into the adjacent interpeduncular nucleus, where delta opioid receptor expression is dense (Devine et al., 1993). While the access of delta opioids to the dopamine system remains to be understood, it is thought that mu opioids are rewarding because they inhibit the same GABAergic neurons as are inhibited by GABAA agonists in the VTA, disinhibiting the VTA dopamine system in parallel to the disinhibition of nigral dopamine neurons known to result from inhibition of nearby GABAergic neurons (Johnson and North, 1992; Tepper et al., 1995).
Recently we have found that drug-seeking behavior can be reinstated by footshock stress and by local infusion of corticotropin-releasing factor (CRF) into the VTA (Wang et al. 2004). While it has no such effect in cocaine-naïve animals, CRF causes glutamate and dopamine release in the VTA of cocaine-experienced animals. Footshock stress causes equal CRF release in the VTA of cocaine-naïve and cocaine-experienced animals; thus it appears that some aspect of cocaine exposure—and perhaps other stressors—gives neuronally released CRF access to brain reward circuitry.
Finally, drug-seeking (Carroll and Meisch, 1984) and brain stimulation reward (Carr and Simon, 1984) are also enhanced by the stress of food restriction. These enhancements are blocked by leptin (Fulton et al., 2000; Shalev et al., 2001), though the site or mechanism of interaction of leptin with reward circuitry is not yet known.
Stimulation of the same MFB sites can both reward the instrumental performance it follows and motivate reward-seeking and consummatory responses that it precedes (Margules and Olds, 1962). Again, lateral hypothalamic sites have been most frequently studied for stimulation-induced motivation of species-typical acts like feeding (Wise, 1974), sexual behavior (Roberts et al., 1967) or predatory attack (MacDonnell and Flynn, 1966). Again, however, stimulation at other sites along portions of the MFB can also be effective (Berntson, 1973; Gratton and Wise, 1988a; Gratton and Wise, 1988b). The directly activated fiber systems for stimulation-induced feeding have the same refractory periods and conduction velocities as those for stimulation-induced reward (Gratton and Wise, 1988a; Gratton and Wise, 1988b). In addition, food restriction and the adipocyte hormone leptin each modulate not only food consumption but also drug-seeking (Shalev et al., 2001) and brain stimulation reward (Fulton et al., 2004; Fulton et al., 2000). Thus the same MFB circuitry appears to play a role in both the initiation of reward-seeking behavior (motivation) and the stamping in of response habits by the earned reward. The fact that MFB brain stimulation can not only reinforce the responses it follows but also energize (“prime”) the responses it precedes (Gallistel, 1966; Gallistel et al., 1974) partially explains the preference of the brain stimulation specialist for the term “reward” over the term “reinforce” (Wise, 1989)
While lateral hypothalamic electrical stimulation can induce dopamine-dependent feeding in sated animals (Phillips and Nikaido, 1975), it appears that this system plays a stronger role in reinforcement than in motivation. Indeed, it has been argued that the stimulation motivates feeding only after animals have learned, through trial and error, that the stimulation makes food reinforcing even under conditions of satiety (Coons et al., 1965; Mendelson, 1966). Thresholds for stimulation-induced feeding decrease with experience (Wise, 1968), but even in experienced animals, it takes stronger levels of stimulation to motivate consummatory responding than to reinforce behavior. With current intensity at the same level, stimulation-induced reward requires longer stimulation trains (Ball, 1970) and somewhat higher stimulation frequencies (Gratton and Wise, 1988b) than does stimulation-induced feeding. This is consistent with the fact that free-feeding is more difficult to disrupt with neuroleptics (Wise and Raptis, 1986) or with dopamine-depleting lesions (Aberman and Salamone, 1999) than is food-rewarded lever-pressing (Aberman and Salamone, 1999; Wise et al., 1978a).
In assessing the relative contributions of reinforcing and motivational effects of dopamine, it must be remembered that severe depletion of forebrain dopamine (Stricker and Zigmond, 1985; Robinson et al., 2005) or large lateral hypothalamic lesions (Teitelbaum and Epstein, 1962) cause profound deficits in arousal and motor function. Thus discussions of the motivational effects of the dopaminergic component of motivational circuitry properly revolve around studies involving lesions or doses of dopamine antagonists that leave arousal and motor function relatively intact or preparations where arousal is provided by some experimental manipulation (Robinson et al., 2005). Even in the case of moderate levels of dopamine blockade, however, it has recently been suggested that moderate doses of dopamine antagonists interfere with the motivation to seek a reward rather than with the effectiveness of the reward once earned, that, in lay language, animals with impaired dopamine function no longer want the reward but still like it when they happen to get it (Berridge and Robinson, 1998).
This view is falsified by a number of experiments showing that the deterioration of motivation that develops in animals tested under conditions of moderate dopamine receptor blockade is a consequence of the devalued effectiveness of the reward in this condition. Most telling is the fact that dopamine antagonists disrupt reward-seeking behavior only after they disrupt the reinforcing effects of the reward. This is most evident in studies in which approach and consummatory responses are assessed in discrete trials (Fouriezos et al., 1978; McFarland and Ettenberg, 1995). For example, in a task where animals traversed a runway for a single intravenous heroin injection each day, running speed was normal on the day haloperidol was given, but was slow on the following day. Thus haloperidol disrupted the reinforcing effect of heroin but not the response habit or the response-eliciting effectiveness of the runway cues in trained animals that had never had haloperidol before (McFarland and Ettenberg, 1995); similar effects are seen with food reward (McFarland and Ettenberg, 1998. The ability of dopamine antagonists to impair free-feeding (Wise and Raptis, 1986) or lever-pressing for food (Wise et al., 1978a) increases progressively, both within and across test days, as the animal experiences the food under the antagonist. Thus the devaluation of reward by the antagonist is remembered and the memory of reward under dopamine blockade is what degrades subsequent motivation. Neuroleptics degrade the remembrance of value of rewards past before they degrade the acceptance of rewards present.
Thus the importance of the dopamine system and its afferents for motivation is complex. Motivation is strongly influenced by situational cues in the case of well-learned habitual behavior. For example, sated animals that were trained to respond for food under conditions of hunger will continue to respond normally for food and to eat it, for a considerable period, when subsequently tested when sated (Kimble, 1951; Koch and Daniel, 1945). Indeed, animals trained to work for food will continue to work for it despite the availability of free food (Jensen, 1963; Morgan, 1974). Thus motivation of response habits is strongly influenced not only by internal states but also by external stimuli. The immediate motivating effects of external reward-predictive stimuli appear to be largely dopamine-independent so long as the arousal and motor function are not impaired and so long as the animal has not yet had experience with the reward in the dopamine-antagonized condition (McFarland and Ettenberg, 1995; McFarland and Ettenberg, 1998). Reward-predictive stimuli begin to lose their effectiveness, however, as the animal gains experience with the reward under conditions of dopamine blockade (Fouriezos et al., 1978; Fouriezos and Wise, 1976; McFarland and Ettenberg, 1995; Wise and Raptis, 1986; Wise et al., 1978a).
Although dopamine receptor activation is not a necessary condition for the motivation induced by reward-predictive environmental stimuli, activation of dopamine receptors can augment the effectiveness of such cues. For example, rewarding brain stimulation (Gallistel, 1969) or dopamine agonists such as amphetamine and cocaine (Pickens and Harris, 1968) are routinely used to “prime” or “reinstate” responding in unresponsive animals that have access to these rewards. Indeed, dopamine itself (Cornish and Kalivas, 2000) or the dopamine agonist bromocriptine (Wise et al., 1990) can reinstate responding in animals that have ceased lever-pressing because the behavior is no longer rewarded. Similarly, amphetamine injection into nucleus accumbens can increase the lever-pressing for sucrose that is triggered by cues that have been associated with sucrose outside the context of the instrumental task (Wyvell and Berridge, 2000). Thus while minimal levels of dopamine seem necessary for performance capability (Levitt and Teitelbaum, 1975; Stricker and Zigmond, 1985; Ungerstedt, 1971; Robinson et al., 2005) phasic dopamine release does not seem critical for cue-induced motivation in well-trained animals. Nonetheless, activation of dopamine receptors through activation of brain reward circuitry can augment motivational arousal of remembered response habits.
A motivational substrate is embedded within the medial forebrain bundle. It involves descending and ascending fibers and it plays important roles in the stamping in of response habits by various rewards and in establishing motivational significance for environmental stimuli. Once established, these stimuli are sufficient, in a normal animal, to trigger response habits that can survive non-reinforcement for a considerable period. This substrate is important not only for establishing arbitrary response habits, but also for the motivation of a variety of species-typical, biologically primitive behaviors such as feeding, drinking, mating, territorial marking, and predatory attack.
I thank Satoshi Ikemoto, Yavin Shaham, and Bin Wang for critical reading and helpful comments.