The early focus of much experimental drug addiction research was to understand the reinforcing, or ‘rewarding’, effects of abused drugs; this has led to great advances in defining the primary molecular targets of addictive drugs as well as, more recently, the adaptations in these targets that develop with chronic drug self-administration (Nestler 2004; Koob & Le Moal 2005). However, it has been appreciated for some time that the molecular and neurochemical correlates of acute and chronic drug administration must be interpreted in behavioural and cognitive terms if the psychological processes and neurobiological mechanisms determining human drug addiction are to be specified. Therefore, we and others have increasingly come to view drug addiction as the endpoint of a series of transitions from initial drug use—when a drug is voluntarily taken because it has reinforcing, often hedonic, effects—through the loss of control over this behaviour, such that it becomes habitual and ultimately compulsive. We have recently reviewed the evidence that these transitions depend upon interactions between Pavlovian and instrumental learning processes (Everitt & Robbins 2005). Furthermore, we have hypothesized that the ‘switch’ from voluntary drug use to habitual and progressively compulsive drug use represents a transition at the neural level from prefrontal cortical to striatal control over drug-seeking and drug-taking behaviours, as well as a progression from ventral to more dorsal domains of the striatum, mediated at least in part by its stratified dopaminergic innervation (Everitt & Robbins 2005).

These observations indicate that the BLA and AcbC may function together as nodes within a limbic cortical–ventral striatopallidal system that underlies the acquisition of drug seeking. This notion is further supported by the observation that disconnecting these structures by unilateral pharmacological blockade of dopamine and AMPA receptors in the BLA and AcbC, respectively, on opposite sides of the brain also greatly diminished cocaine seeking (Di Ciano & Everitt 2004). Thus, the acquisition and early performance of cocaine seeking that is maintained over protracted periods of time by the contingent presentation of drug-associated conditioned reinforcers depends upon the integrity of the AcbC and its afferent input from the BLA. It can be assumed that, at this stage, drug seeking is under the control of instrumental response–outcome contingencies in which animals respond in a voluntary and goal-directed way for intravenous cocaine infusions. Indeed, we have clear evidence that this is so from studies using a ‘seeking–taking’ chained schedule in which animals perform a drug-seeking response in the initial link of the chain, which then gives access to a drug-taking response in the second link, performance of which delivers cocaine (Olmstead et al. 2001). After a limited experience with the drug, cocaine seeking was shown to be a goal-directed action in that its performance was sensitive to devaluation of the drug-taking link (Olmstead et al. 2001). This devaluation effect demonstrates that cocaine seeking at early stages of acquisition and performance is mediated by the knowledge of the contingency between the seeking response and its outcome.

This response–outcome process can be contrasted with a second, stimulus–response (S–R) instrumental process in which seeking behaviour is a response habit triggered and maintained by drug-associated stimuli (Everitt et al. 2001). Tiffany (1990), O'Brien & McClellan (1996) and ourselves (Robbins & Everitt 1999; Everitt et al. 2001; Everitt & Robbins 2005) have all advanced the hypothesis that drug addiction encompasses changes in the associative structure underlying drug seeking such that it becomes ‘automatic’ or habitual. We have additionally hypothesized that the progressive engagement of dorsal striatal mechanisms underlies this transition (Everitt & Robbins 2005), based upon evidence that the dorsolateral (DL) striatum mediates S–R habit learning—evidence that has been considerably strengthened through the recent studies of instrumental responding for food and its resistance to reinforcer devaluation, a canonical test of the development of S–R habits (Yin et al. 2004, 2006). What then is the evidence that the dorsal striatum mediates the performance of well-established drug-seeking habits that depend initially upon ventral striatal mechanisms during acquisition? Correlative evidence came from in vivo microdialysis measurement of extracellular dopamine in rats that had attained over a two-month period stable responding under a second-order schedule of cocaine seeking. While self-administered cocaine increased dopamine release in the AcbS, AcbC and caudate–putamen, extracellular dopamine was increased selectively in the AcbC in response to unexpected (i.e. non-response contingent) presentations of a cocaine-associated stimulus. However, during a prolonged period of cocaine seeking maintained by contingent presentations of the same cocaine-associated CS, dopamine release was increased only in the dorsal striatum, not in the AcbC or AcbS (Ito et al. 2000, 2002). Furthermore, dopamine release in the dorsal striatum was subsequently shown to be causally important for the maintenance of drug seeking, since it was greatly reduced by dopamine receptor blockade at this site, whereas this treatment was without effect in the AcbC (Vanderschuren et al. 2005). This apparent shift from ventral to dorsal striatal control over drug seeking provides support for the hypothesis that it is under S–R, or ‘habit’, control owing to accumulated evidence implicating the dorsal striatum in habit learning (Yin et al. 2004, 2006). Moreover, the fact that the second-order schedule we have principally used is of a fixed-interval type encourages this view, since interval schedules are known to result in the more rapid development of S–R habits through the weaker relationship between response and outcome that obtains, when compared with ratio schedules (Dickinson 1985). In addition, studies with orally self-administered cocaine and alcohol have shown the more rapid development of habitual drug seeking compared with the seeking of a natural sweet reward (Dickinson et al. 2002, Miles et al. 2003).

These observations supporting the increasing importance of the dorsal striatum in well-established, or habitual, drug seeking raise the issue of how, in neural terms, such a shift in the locus of control from ventral to dorsal striatum might occur. The observations of Haber et al. (2000) in primates and, more recently, Ikemoto (2007) in rats provide a possible neuroanatomical basis. They showed that ventral tiers of the striatal complex regulate the dopaminergic innervation of more dorsal tiers through so-called ‘spiralling’ connections with the midbrain (figure 1a). Thus, the AcbS projects to dopamine neurons in the ventral tegmental area that innervate not only the shell but also the more dorsally situated AcbC. Neurons in the AcbC innervate dopamine neurons projecting to both the AcbC and the immediately dorsal regions of the dorsomedial caudate–putamen and so on, in a serially cascading pattern ultimately to encompass more lateral parts of the dorsal striatum—the site at which dopamine release is increased during habitual drug seeking and where dopamine receptor antagonist infusions impair this behaviour.

We tested the hypothesis that the serial cascade of striato-nigro-striatal connectivity underlies this progressively greater control over well-established cocaine-seeking behaviour by the DL striatum using a novel ventral–dorsal striatal ‘disconnection’ (figure 1b). Thus, the AcbC was selectively lesioned on one side of the brain and combined with dopamine receptor blockade in the contralateral DL striatum, thereby functionally disconnecting serial interactions between these ventral and dorsal striatal domains bilaterally (Belin & Everitt 2008). This disconnection greatly and selectively decreased cocaine seeking in rats tested some weeks after stable responding had been attained under a second-order schedule of reinforcement (figure 1c). Two important additional observations underlined the specificity of the ventral–dorsal striatal disconnection effect. (i) The same animals were trained to perform a novel chain-pulling response for sucrose under a fixed ratio 1 schedule of reinforcement and again underwent the disconnection manipulation, or bilateral dorsal striatal dopamine receptor antagonist infusions (in non-lesioned rats), immediately after acquisition when the behaviour was under response–outcome control; neither manipulation had any effect (Belin & Everitt 2008). (ii) In separate groups of animals, either bilateral dorsal striatal dopamine receptor blockade or AcbC-dorsal striatal disconnection was performed at a much earlier stage of acquisition of the cocaine seeking, second-order schedule when responding was under ratio, rather than interval, control and when response–outcome mechanisms dominate performance. Again, neither manipulation had any effect on cocaine seeking (D. Belin & B. J. Everitt 2008, unpublished observations).

Taken together, the above results clearly indicate the devolution of control over drug seeking from ventral to dorsal striatum and also strongly suggest that this shift is progressive. Other data also support the notion of this shift. For example, using autoradiographic methods, Porrino and colleagues showed the development of neuroadaptations in D_2/3 dopamine receptors and other neurochemical, or metabolic, markers in the dorsal striatum following chronic, but not acute, cocaine self-administration by monkeys (Letchworth et al. 2001; Porrino et al. 2004). At earlier stages of training, these adaptations were largely restricted to the more ventral, nucleus accumbens region. The DL striatum has also been shown to be involved in ‘relapse’ to a cocaine-seeking habit, since neural inhibition induced by γ-aminobutyric acid receptor agonist infusion into this area, but not into the AcbS or AcbC, prevented the reinstatement of cocaine seeking after protracted withdrawal (Fuchs et al. 2006; See et al. 2007). Moreover, the presentation of drug cues to human cocaine addicts both induced drug craving (that has been shown to be correlated with the activation of the amygdala and limbic prefrontal cortical areas; Grant et al. 1996; Childress et al. 1999; Garavan et al. 2000) and also marked the activation of the dorsal striatum (Garavan et al. 2000; Volkow et al. 2006). These observations therefore strongly indicate a link between limbic cortical mechanisms and the engagement of the dorsal striatum in long-term drug abusers exposed to drug cues, whereas the results of our ventral/dorsal striatal disconnection experiments reveal that this recruitment is mediated by antecedent limbic cortex-dependent activity in the AcbC and its regulation of dorsal striatal dopaminergic projections.

It seems likely that the ventral to dorsal striatum shift is not specific to drug seeking, but would apply equally to the control over instrumental responding for natural reinforcers under appropriate conditions. Indeed, lesion or inactivation of the AcbC, dorsomedial or DL striatum in rats responding for ingestive reinforcers does not globally impair instrumental behaviour, but instead has major effects that depend upon the response–outcome or S–R associative structure underlying the behaviour. Lesions or N-methyl-d-aspartate receptor blockade of the AcbC (Kelley et al. 1997; Corbit et al. 2001) or dorsomedial striatum (Yin et al. 2004, 2005), impair instrumental behaviour under response–outcome control, but actually enhance the development of S–R habits in which responding persists after reinforcer devaluation (Yin et al. 2004). By contrast, DL striatal lesions, inactivation or dopamine denervation return previously habitual responding to response–outcome control, reinstating sensitivity to reinforcer devaluation (Yin et al. 2004; Faure et al. 2005) or action–outcome contingency degradation (Yin et al. 2006). These observations emphasize that response–outcome and S–R learning mechanisms are probably engaged not serially, but in parallel, with DL striatum-dependent S–R mechanisms eventually dominating the control over behaviour.

However, it is possible that the shift from ventral to dorsal striatal control occurs more rapidly in animals seeking drugs owing to the effects of these agents themselves on the plasticity mechanisms involved, particularly in the case of psychomotor stimulants which so powerfully increase dopamine transmission. Thus, an amphetamine sensitization treatment regimen leads to the more rapid instantiation of habit learning in animals responding for food (Nelson & Killcross 2006). Moreover, animals that had escalated their cocaine intake showed, when subsequently challenged with cocaine, a marked enhancement of stereotyped behavioural responses that have long been known to depend upon the dorsal, rather than the ventral, striatal dopaminergic innervation (Ferrario et al. 2005). Finally, Schoenbaum and colleagues have demonstrated a shift in the balance of associative encoding from ventral to dorsal striatum correlated with concomitant enhancement of cue-evoked neuronal firing in the dorsal striatum (Takahashi et al. 2007). This finding resonates with the observation of drug-associated CS-induced activation of the dorsal striatum in human cocaine abusers (Garavan et al. 2000; Volkow et al. 2006). Therefore, the unique properties of drugs as reinforcers, especially stimulant drugs, might accelerate, or more effectively consolidate, the development of drug seeking as a S–R habit (Everitt et al. 2001).

When allowed to self-administer cocaine intravenously, the impulsive rats showed a marked escalation of their cocaine intake compared with non-impulsive controls. They did not acquire cocaine self-administration more rapidly, but responded at a much higher rate for their cocaine infusions than did non-impulsive subjects (Dalley et al. 2007). This is in marked contrast to rats showing high locomotor responses to novelty (a ‘sensation-seeking’ phenotype), which self-administer cocaine at low doses that do not sustain self-administration in those rats that show low locomotor responses to novelty (Piazza et al. 1989). Thus, high impulsivity predicts the tendency to escalate cocaine intake, which is reminiscent of one of the diagnostic characteristics of substance dependence in DSM-IV which describes the taking of drugs in larger quantities than intended as a core symptom.

Highly impulsive animals were investigated neurobiologically using positron emission tomography to measure binding in the striatum of the selective, high affinity D_2/3 dopamine receptor antagonist [¹⁸F]fallypride (Mukherjee et al. 1999). Impulsive animals showed markedly reduced fallypride binding within the ventral, but not the dorsal striatum. Moreover, the reduced D_2/3 dopamine receptor availability in the ventral striatum was correlated with impulsivity on the 5-CSRTT (Dalley et al. 2007). Thus, low D_2/3 dopamine receptors in the ventral striatum, encompassing the nucleus AcbC and shell regions, are correlated both with impulsivity and the marked propensity to escalate cocaine intake. This observation, taken together with the role of the AcbS in the reinforcing and stimulant effects of cocaine, as well as involvement of the AcbC in the acquisition of drug-seeking behaviour (Ito et al. 2004) and the ability to tolerate delays to reinforcement (Cardinal et al. 2001), indicates the importance of the ventral striatum in the neural mechanisms underlying the propensity to seek and work for cocaine over extended periods of time. Nader and colleagues, in studies of socially housed monkeys, have also demonstrated the significant relationship between D₂ dopamine receptors in the striatum and cocaine self-administration (Czoty et al. 2004; Nader et al. 2006); these studies are reviewed comprehensively by Nader et al. (2008).

In attempting to model drug addiction in animals, we have tried to capture the compulsive quality of drug seeking by measuring its persistence despite negative or aversive outcomes, as in the DSM-IV. In developing such behavioural procedures, we have shown that compulsive drug seeking only emerges following an extended, or chronic, history of cocaine taking (Deroche-Gamonet et al. 2004; Vanderschuren & Everitt 2004; Pelloux et al. 2007). In the study by Deroche-Gamonet et al. (2004), three addiction-like behavioural criteria were measured in rats, namely (i) increased motivation to take the drug, (ii) inability to refrain from drug seeking, and (iii) maintained drug use despite aversive consequences. After approximately 40 days of cocaine self-administration, but not at earlier times, some 17% of subjects developed these addiction-like criteria showing increased break points under a progressive ratio of cocaine reinforcement, persistent responding during signalled periods of drug unavailability and, perhaps most importantly, persistence of the instrumental nose-poke response for cocaine even when it was punished by mild footshock. In the study by Pelloux et al. (2007), rats were trained on the seeking–taking chained schedule with intermittent punishment of the seeking response (i.e. on 50% of the seeking bouts, randomly occurring) to achieve suppression of drug seeking, or abstinence, as described previously. In this study too, whereas all rats suppressed their cocaine seeking after a limited history of cocaine self-administration, after an extended history 17–20% of subjects were completely resistant to punishment, continuing to seek and take drugs despite the ongoing, daily experience of the negative outcome. The proportion of rats compulsively seeking drugs, then, was similar both to that in the Belin study and to the addiction-vulnerable subgroup of human subjects, often estimated to be less than 20% of the population that initially use drugs (Anthony et al. 1994).

However, the origins of this propensity to seek cocaine compulsively have not been established. We hypothesized that impulsivity, which we have shown to be associated with low D_2/3 dopamine receptor availability in the ventral striatum and to predict the escalation of cocaine intake (Dalley et al. 2007), might also confer a vulnerability to develop compulsive drug seeking and addiction following extended access to cocaine. Rats were screened both for impulsivity in the 5-CSSRT and also for the sensation-seeking phenotype of high locomotor responsiveness to novelty (HR rats) which has earlier been suggested to be an addiction vulnerability phenotype (Piazza et al. 1989). The resultant groups were then studied in the acquisition of cocaine self-administration, and for the emergence of the three addiction-like behavioural criteria identified by Deroche-Gamonet et al. (2004), but particularly persistent responding for the drug in the face of punishment. As expected, and as reported previously (Piazza et al. 1989), the HR rats more readily acquired cocaine self-administration and showed an upward shift in the cocaine dose–response curve as compared both with rats with low responses to novelty and also high impulsivity (Belin et al. 2008). We also confirmed our earlier finding that impulsivity is not associated with more rapid acquisition of cocaine self-administration, but instead with the escalation of cocaine intake. However, and in marked contrast, it was high impulsivity, but not high reactivity to novelty, that predicted the switch (Leshner 1997) from controlled to compulsive cocaine taking (Belin et al. 2008). Highly impulsive rats displayed higher addiction scores and much greater resistance to punishment than rats with high or low responses to novelty or low impulsivity (electronic supplementary material, figure S1). In fact, highly impulsive rats did not differ from rats showing the three addiction-like behavioural criteria in any of their addiction-like behaviours after the extended period of cocaine self-administration. Therefore, it seems, perhaps counter-intuitively, that the propensity to acquire cocaine self-administration when first encountering the drug and the vulnerability to develop compulsive cocaine intake (addiction), depend upon distinct and seemingly orthogonal behavioural characteristics—novelty/sensation-seeking versus impulsivity, respectively—each of which might have a genetic or environmental basis. The results of this study also provide experimental evidence that high levels of impulsivity can antedate the onset of compulsive drug use, thereby emphasizing the importance of pre-existing impulsivity seen in individuals addicted to drugs (Jentsch & Taylor 1999; Dom et al. 2006).

Now that we have established models of compulsive drug seeking and addiction, it will be possible to investigate not only predisposing factors, such as impulsivity, but also the underlying neurobiological mechanisms. There are several current views about the origins of compulsion within the brain, which are often thought of as being, but in reality are not, mutually exclusive. The neuroadaptations occurring during behavioural sensitization to stimulant drugs have been argued to underlie an extreme incentive motivational state of drug ‘wanting’ (Robinson & Berridge 1993). According to this view, addicts experience this state especially when exposed to drug-associated cues, which leads to over-activation of the sensitized dopaminergic innervation of the nucleus accumbens, in which plasticity-associated structural changes in dendritic spines have also been observed (Ferrario et al. 2005). This hypothesis is discussed in detail by Robinson & Berridge (2008). One interpretation of compulsive drug seeking, then, is that it is a behavioural manifestation of this potentiated motivational state, which has been demonstrated in some studies as an increased break point under progressive ratio schedules of reinforcement (for a review, see Vezina 2004). However, as noted above, a sensitization treatment regimen with amphetamine also leads to the more rapid instantiation of S–R habits (Nelson & Killcross 2006) and it is not easy at the behavioural level to differentiate between an increased tendency to repeat drug-seeking responses elicited and maintained by drug-associated stimuli—the ‘must do!’ of compulsive habits discussed above—from an increased desire for a drug, which might also seem counter-intuitive given the development of tolerance to its rewarding or reinforcing effects.

Perhaps more directly related to the notion of drug seeking as a compulsive habit, however, is the observation of reductions in dopamine D₂ receptors in the dorsal striatum in abstinent alcoholics, cocaine, heroin and methamphetamine addicts (Volkow & Wise 2005) and also following chronic, but not acute, cocaine self-administration in monkeys (Moore et al. 1998; Nader et al. 2002). The consequences of this change in striatal dopamine D₂ receptors for the plasticity underlying instrumental learning and performance is unclear, but an intriguing putative sequential mechanism within the striatum and involving its dopaminergic innervation might be considered. Thus, the early vulnerability to escalate cocaine intake seen in impulsive rats is predicted by low D_2/3 dopamine receptor levels in the ventral, but not the dorsal striatum (Dalley et al. 2007). However, this escalated intake may lead to more rapid neuroadaptations, including downregulated D₂ dopamine receptors, in the dorsal striatum and mediated in part by aberrant engagement of the spiralling striato-nigro-striatal circuitry. This would lead to more rapid consolidation of drug-seeking habits that are difficult to relinquish, despite negative outcomes and are more readily reinstated after abstinence following exposure to response-eliciting drug-associated stimuli.

An alternative account may be provided by the impact of negative reinforcement, as has been suggested to underlie obsessive–compulsive disorder, whereby drug-seeking habits are maintained by the motivation to alleviate or avoid (self-medicate) the negative emotional state and dysregulation resulting from tolerance to, and withdrawal from, drugs taken in increasing amounts over time (Koob & Le Moal 2001). These counter-adaptations are prevalent in the central and extended amygdala and their motivational impact on drug self-administration is described in detail by Koob & Le Moal (2008). These mechanisms are not of course mutually exclusive. Addiction to drugs may reflect a combination of increased incentive motivation mediated by the upregulation of ventral striatal dopamine transmission, by ‘hyper-consolidated’ habit learning mediated by upregulated dorsal striatum, dopamine-dependent mechanisms and the drive engendered by negative emotional states in extra-striatal networks.

However, we and others have also hypothesized an additional neurobiological mechanism perhaps arising in part as the direct or indirect consequence of toxic drug effects. This mechanism may be implicated in a shift in balance of behavioural control processes from the prefrontal cortex to the striatum, thereby promoting compulsive habitual behaviour. There are abundant data suggesting prefrontal cortical, especially orbitofrontal (OFC), dysfunction in addicts, which are also increasingly supported by experimental studies in animals (Schoenbaum et al. 2006; Everitt et al. 2007; Olausson et al. 2007) and humans (see Garavan et al. 2008). Thus, in cocaine and methamphetamine abusers, reduced activity of the OFC that correlates with reduced D_2/3 dopamine receptors in the striatum (Volkow et al. 2001), and reduced grey matter volume in this region (Matochik et al. 2003) have been reported. There are also growing numbers of reports of impaired behavioural and cognitive functions, including poor behavioural adjustment (Bechara 2005) and impaired probabilistic reversal learning in cocaine abusers (Ersche et al. 2008), possibly due to reduced inhibitory control. Deficits have been reported in decision-making cognition on computerized versions of a gambling task, when stimulant abusers chose the most favourable option less frequently than control subjects and chose significantly against the odds in risky conditions, suggesting difficulties in estimating outcome probabilities (Rogers et al. 1999; Ersche et al. 2005). Similar changes in behaviour are seen in individuals with OFC damage (Rogers et al. 1999) and this has encouraged the view that chronic drug taking may actually be a causal factor in inducing such prefrontal cortex-dependent deficits. But suboptimal prefrontal cortical, including OFC and anterior cingulate cortex, function (Volkow & Fowler 2000; Kaufman et al. 2003; Hester & Garavan 2004) may also represent a pre-existing vulnerability trait that results in poor decisions and/or a lack of sensitivity to the consequences of such decisions, and hence drug abuse leading to addiction.

Experimental studies primarily involving psychostimulant treatment of rats and monkeys even after brief periods of exposure have supported the view that disrupted OFC function may indeed be a consequence of toxic drug effects during an addict's history of drug abuse (Jentsch & Taylor 1999; Schoenbaum et al. 2006). Short term, usually experimenter-, and not self-, administered cocaine or amphetamine enhanced the development of impulsivity (Jentsch & Taylor 1999; Roesch et al. 2007). Reversal learning was impaired by cocaine treatment in monkeys (Jentsch et al. 2002) and rats (Schoenbaum et al. 2004). Rats having self-administered and then been withdrawn from cocaine exhibited both increased extinction responding and a marked deficit in reversal learning during withdrawal (Calu et al. 2007). Schoenbaum and colleagues have emphasized the similarity between OFC lesions and these apparently long-lasting effects of relatively short-term treatment with cocaine, and also showed that the deficit in reversal learning is reflected in a change in the properties of OFC neurons, which do not develop appropriate responses to cues predicting outcomes (Stalnaker et al. 2006).

It is remarkable that even brief periods of drug exposure, whether experimenter administered or self-administered, can result in enduring changes in behaviour indicative of OFC dysfunction. However, in the great majority of imaging and neuropsychological investigations of addicts, there has been an exceptionally long history of drug abuse and often poly-drug abuse. These drug-addicted individuals must represent, therefore, a relatively small proportion of the much larger number of individuals in a population who have abused drugs over varying periods of time, but who have not made the transition to an addicted state as characterized by compulsive drug use. Thus, it would seem unlikely that experimental groups of rats receiving the fairly modest exposure to stimulant drugs in the experiments described above would in any sense fulfil the criteria for addiction, yet the changes in behaviour indicative of OFC dysfunction are seen in the entire population of treated experimental animals. It will be important, therefore, to investigate neurobiologically those models that capture chronic drug self-administration (Dalley et al. 2005a,b) and the compulsive drug seeking that develops in vulnerable sub-populations of rats if we are to understand the mechanisms underlying the interaction between predisposing behavioural traits and chronic drug exposure in the development of drug addiction.