PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Behav Pharmacol. Author manuscript; available in PMC 2010 September 1.
Published in final edited form as:
PMCID: PMC2874684
NIHMSID: NIHMS202796

Effects of chronic administration of drugs of abuse on impulsive choice (delay discounting) in animal models

Abstract

Drug addicted individuals demonstrate high levels of impulsive choice, characterized by preference for small immediate over larger but delayed rewards. Although the causal relationship between chronic drug use and elevated impulsive choice in humans has been unclear, a small but growing body of literature over the past decade has shown that chronic drug administration in animal models can cause increases in impulsive choice, suggesting that a similar causal relationship may exist in human drug users. This article reviews this literature, with a particular focus on the effects of chronic cocaine administration, which have been most thoroughly characterized. The potential mechanisms of these effects are described in terms of drug-induced neural alterations in ventral striatal and prefrontal cortical brain systems. Some implications of this research for pharmacological treatment of drug-induced increases in impulsive choice are discussed, along with suggestions for future research in this area.

Keywords: impulsive choice, impulsivity, decision making, delay discounting, intertemporal choice, addiction, drug abuse, cocaine

Drug addiction is associated with a range of neurobehavioral deficits, including high levels of impulsivity (Jentsch and Taylor, 1999; Bickel and Marsch, 2001; Monterosso et al., 2001; de Wit and Richards, 2003; Bechara, 2005). Despite the strong associations between elevated impulsivity and addiction, however, the cause and effect relationships between the two have been difficult to elucidate (Perry and Carroll, 2008; de Wit, 2009). Prospective studies in both humans and animal subjects suggest that pre-existing impulsive traits may predispose individuals to drug use (Tarter et al., 2003; Garavan and Stout, 2005; Perry et al., 2005, 2008; Dalley et al., 2007; Belin et al., 2008; Diergaarde et al., 2008). Other studies in animals show that chronic drug use may cause increases in impulsivity that in some cases can last well beyond the cessation of drug use. The purpose of this paper is to review work in this latter category, with a focus on animal studies in which cause and effect relationships can be clearly defined. Importantly, impulsivity is not a unitary construct, but rather may be fractionated into several behaviorally and neurobiologically distinct components. This paper will focus solely on impulsive choice (preference for a smaller sooner over a larger delayed reward, also termed “cognitive” impulsivity or impulsive decision-making), as opposed to impulsive action (or inability to withhold a pre-potent motor response (Evenden, 1999; Olmstead, 2006; Winstanley et al., 2006a)). Note that the fractionation of “impulsivity” into several components could be considered to lessen the utility of the term for research purposes, as different forms of impulsivity may seem to bear little relation with one another. However, given that these different components of impulsivity co-occur in human psychopathological conditions, and that the goal of much animal research on impulsivity is to model such conditions, we believe that the term still retains some utility, and thus the term “impulsive choice” will be retained throughout the review.

Drugs of abuse and impulsive choice

High levels of impulsive choice in drug users are evident on personality inventories, as well as on laboratory tests such as intertemporal choice (delay discounting) tasks (Bickel and Marsch, 2001; de Wit and Richards, 2003; Kirby and Petry, 2004; Heil et al., 2006; Reynolds, 2006). Delay discounting refers to the fact that rewards decrease in subjective value (i.e. - are discounted) if their delivery is delayed. Such discounting is observed in a range of species, including humans, in tasks in which subjects must choose between two alternatives – one which produces a small reward, and another which produces a large reward. When the delay to the two rewards is equal, subjects invariably choose the large reward. However, as the delay to the large reward is increased relative to the delay to the small reward, subjects begin to switch their preference to the smaller, more immediate reward, demonstrating that the value of the large reward has been reduced, or “discounted” relative to the small reward (Rachlin and Green, 1972; Ainslie, 1975; Mazur, 1987; Logue, 1988; Evenden and Ryan, 1996; Woolverton and Anderson, 2006). All subjects will discount future rewards in a delay-dependent manner to some degree; however, discounting of delayed rewards is considerably greater in drug users compared to non-users (Bickel and Marsch, 2001; Kirby and Petry, 2004; Reynolds, 2006). Importantly, abnormal preference for small immediate vs. large delayed rewards can also be caused by changes in sensitivity to reward magnitude (Ho et al., 1999). However, with few exceptions (Roesch et al., 2007) the effects of chronic drug administration specifically on reward sensitivity in decision-making tasks have not been examined.

It is well established that acute administration of a wide range of drugs of abuse can modulate impulsive choice in both human and animal subjects (although these effects have tended to be more robust in animal studies). Briefly, depressants such as ethanol and morphine tend to increase impulsive choice (i.e. – promote selection of small immediate over large delayed rewards), whereas the effects of stimulants such as caffeine, nicotine, and amphetamine have been more mixed, with reports of different effects on impulsive choice depending on the drug, task procedures (e.g. – the presence or absence of a reward-predictive cue during the delay period), and even baseline levels of impulsive choice (Cardinal et al., 2000; de Wit et al., 2002; Dallery and Locey, 2005; Pitts and McKinney, 2005; Olmstead et al., 2006; Winstanley et al., 2007; Barbelivien et al., 2008; Diller et al., 2008a; Pattij and Vanderschuren, 2008; Perry and Carroll, 2008; Stanis et al., 2008b). In cases of acute drug-induced increases in impulsive choice, such alterations have the potential to impair regulation of drug intake, rendering further immediate drug use more likely (de Wit, 2009). However, drug addiction (characterized by a desire to seek out and use drugs despite adverse consequences) can well outlast the period of acute intoxication and withdrawal. If prior drug use can increase impulsive choice even at long withdrawal time points, then such alterations could contribute to the addiction process by rendering individuals more likely to choose the immediate rewards of further drug use over the delayed but ultimately more beneficial rewards (such as health, employment, and family) of abstinence (Jentsch and Taylor, 1999; Bickel and Marsch, 2001; Monterosso et al., 2001; Bechara, 2005; Kalivas and Volkow, 2005; Olmstead, 2006), and could account in part for the disturbingly high relapse rates that follow even the best available addiction therapies (O'Brien, 2005). In the sections below, we review evidence that chronic exposure to stimulant drugs of abuse (particularly cocaine) can cause lasting alterations in impulsive choice in animal models, and discuss potential mechanisms of and treatment for such alterations.

Effects of chronic stimulant administration on impulsive choice

A number of studies have investigated the effects of chronic stimulant exposure on impulsive choice in rats, using intertemporal choice tasks that involve decisions between smaller sooner and larger delayed rewards. In one of the earliest of these studies, chronic administration of a high dose of methamphetamine (4 mg/kg daily for two weeks, 22 h prior to each test session) caused an increase in impulsive choice (i.e. – elevated preference for the smaller sooner reward) using a within-subjects design (Richards et al., 1999), an effect similar to that observed in human methamphetamine users (Monterosso et al., 2007; Hoffman et al., 2008). However, the extent to which this effect persisted following cessation of methamphetamine administration was not reported. The effects of chronic administration of a high dose of caffeine (30 mg/kg daily for at least 15 sessions, immediately prior to each test session) stand in contrast to those of chronic methamphetamine, in that chronic caffeine decreased impulsive choice (Diller et al., 2008b). However, because acute caffeine administration was also shown to decrease impulsive choice in the same study, it is not clear to what extent decreased impulsive choice during chronic administration was due to acute effects of caffeine.

The effects of chronic nicotine (0.3 or 1 mg/kg daily for over 65 sessions, immediately prior to each test session) were similar to those of methamphetamine in that treatment increased impulsive choice (Dallery and Locey, 2005). Importantly, the effects of chronic nicotine in this study were not solely due to acute effects of the drug (which also increased impulsive choice), as rats continued to display increased impulsive choice for at least a month after nicotine cessation, followed by a gradual return to pre-drug baseline. The fact that a regimen of chronic nicotine continued to promote choice of smaller sooner over larger delayed rewards well after the drug was removed suggests that nicotine-induced alterations in impulsive choice could contribute to relapse (e.g. – by promoting choice of smoking over the delayed but ultimately larger rewards of abstinence). In addition, the fact that impulsive choice eventually returned to pre-drug baseline, following nicotine cessation, accords with findings in human subjects that although levels of impulsive choice are higher in current smokers than in non-smokers, levels in ex-smokers are similar to those in non-smokers (Bickel et al., 1999; Mitchell, 1999).

A recent study by Stanis et al. (2008a) examined the effects on impulsive choice of a chronic regimen of D-amphetamine administration (3 mg/kg, i.p., every other day for 20 days, three weeks prior to the start of testing). In contrast to nicotine, there was no effect of chronic amphetamine on impulsive choice on an intertemporal choice task after amphetamine cessation, despite the fact that amphetamine-exposed rats showed robust psychomotor sensitization to amphetamine both before and after testing in the task. The absence of an effect of amphetamine in this study was somewhat surprising, given that this amphetamine regimen was shown to produce cognitive deficits in other behavioral tasks in rats (Belcher et al., 2005; Briand et al., 2005); however, these data are consistent with studies in humans in which amphetamine users display fewer cognitive deficits than chronic users of other drugs (particularly cocaine) (Ersche et al., 2008). In addition, the use of a reward-predictive cue during the delay prior to large reward delivery (illumination of a light above the large reward choice lever) may have altered the effects of amphetamine on task performance (Cardinal et al., 2000).

Effects of chronic cocaine administration on impulsive choice

Cocaine has been the most well studied drug of abuse in terms of its chronic effects on impulsive choice in animals, a focus that accords well with the human literature in which cocaine users show reliably greater impulsive choice compared to non-users (Coffey et al., 2003; Kirby and Petry, 2004; Bornovalova et al., 2005; Heil et al., 2006). In the earliest such study of which we are aware, Logue et al. (1992) used a within-subjects design to examine the effects of daily cocaine injections (15 mg/kg, i.p.) on impulsive choice using an adjusting delay procedure in rats, in which for each choice, the delay to the large reward was modified based on the rat's previous choices. Each rat received at least 10 cocaine injections, administered immediately prior to each test session, and behavior was assessed across a block of 5 sessions during which the mean adjusted delay (i.e. – the delay to the large reward that produced preference equivalent to that for the small reward) was stable. Cocaine caused an increase in impulsive choice (a decrease in the mean adjusted delay), relative to pre-cocaine baseline sessions in which rats received saline vehicle injections. However, because rats were given cocaine immediately prior to each test session, it is not clear whether this increased impulsive choice was due to acute or chronic effects of the drug. An additional feature of this study was that impulsive choice was shown to return to baseline (pre-cocaine) levels after cessation of cocaine injections, suggesting that there were no long-lasting effects of chronic cocaine administration. However, behavioral measures for the sessions immediately following cocaine cessation were not provided (only values for “stable” behavior – at least 10 sessions post-cocaine – were reported), and thus it is not clear whether cocaine withdrawal caused any temporary alterations in impulsive choice.

Paine et al. (2003) examined the effects of chronic cocaine administration on impulsive choice in rats using the fixed delays procedure developed by Evenden and Ryan (1996), in which each test session consists of five blocks of ten trials of choices between a small immediate and a large delayed reward, and in which the delay to the large reward increases across blocks (0, 10, 20, 40, 60 s) (Evenden and Ryan, 1996; Cardinal et al., 2000). Following pretraining on the task, rats received daily injections of cocaine (15 mg/kg, i.p., 3 times/day) or saline vehicle for 14 days. Behavioral testing continued during this period of drug administration, but took place prior to each day's injections, so as to avoid acute effects of the drug on behavior. Cocaine caused a small increase in impulsive choice (decreased choice of the large delayed reward in cocaine exposed rats compared to saline controls) that was statistically significant during the 7th and 8th day of cocaine administration. However, this effect appeared transient, as impulsive choice in cocaine exposed rats was no different from that in saline controls by the 14th day of cocaine administration, suggesting that, at least during the period of drug administration, there were no lasting effects of cocaine.

Winstanley et al. (2007) employed an experimental design similar to that of Paine et al. (2003), in that following pretraining on the fixed delay task, they examined the effects of 3 weeks of daily injections of cocaine (15 mg/kg, i.p., 2 times/day) or saline vehicle on performance during this same time period (the injections were timed such that performance would not be disrupted by acute effects of the drug). In contrast to Paine et al. (2003) however, they found no effects of chronic cocaine administration on impulsive choice, either during the period of cocaine injections or during two weeks of drug-free testing thereafter. The reasons for this difference are not clear, although the lower cumulative dose used by Winstanley et al. (30 vs. 45 mg/kg/day) could account for their failure to find effects of cocaine on impulsive choice.

Two other recent studies have taken a somewhat different approach to examining the effects of chronic cocaine on impulsive choice, in that testing took place only at long time points (at least 3 weeks) following cocaine administration (Roesch et al., 2007; Simon et al., 2007a). This approach was motivated by observations that some of the consequences of chronic psychostimulant exposure (e.g. - psychomotor sensitization and cue-induced reinstatement) undergo an “incubation” period, in that they may become more robust with the passage of time (Paulson and Robinson, 1995; Lu et al., 2004), as well as studies of human cocaine abusers showing that increased impulsive choice can last well into abstinence (Heil et al., 2006). In both of these studies, rats were given daily injections of cocaine (30 mg/kg, i.p., 1 time/day) or saline vehicle for 14 days, followed by at least a 3-week withdrawal period during which the rats were undisturbed (this same cocaine administration and withdrawal regimen was found previously to produce deficits in reversal learning, reinforcer devaluation, water maze learning, and extinction of Pavlovian fear conditioning (Schoenbaum et al., 2004; Schoenbaum and Setlow, 2005; Burke et al., 2006; Mendez et al., 2008).

Following the withdrawal period, Roesch et al. (2007) trained the rats in a novel two-choice task in which reward magnitude and delay to reward delivery were manipulated independently. Both manipulations caused changes in choice preference (e.g. – an increase in the delay to reward delivery associated with one choice caused an increase in preference for the immediately available reward associated with the other choice), but cocaine-exposed rats changed their choice preference more rapidly than controls, suggesting that prior cocaine administration caused an increase in sensitivity to both reward delay and reward magnitude. How such increases in delay and reward sensitivity might affect reward preference in an intertemporal choice task may depend on the test parameters (delay duration and reward magnitude) used; however, these data are consistent with the idea that cocaine administration increases impulsive decision making (Ho et al., 1999; Roesch et al., 2007).

In the study by Simon et al. (2007a), impulsive choice was assessed after the three-week withdrawal period with the fixed delays procedure used by other investigators (Evenden and Ryan, 1996; Cardinal et al., 2000). Rats given the two week cocaine administration regimen described above exhibited decreased preference for the large delayed reward (i.e. – increased impulsive choice). This effect was evident 3 months after cocaine cessation (due to the long duration of training in the fixed delays procedure), suggesting that chronic cocaine-induced alterations in impulsive choice can be relatively stable. This long time point also suggests a possible reason for the small or negative effects of chronic cocaine observed in previous studies, in that there may be an “incubation period” for cocaine-induced increases in impulsive choice, such that they do not become robustly evident until relatively late withdrawal time points (Grimm et al., 2001). However, an alternative explanation for these differences is that, unlike rats in the Paine et al. and Winstanley et al. studies described above, rats in the Simon et al. study were not pretrained on the delay discounting task prior to cocaine administration. Pretraining may modulate the impact of neurobiological manipulations on delay discounting (Mobini et al., 2002; Winstanley et al., 2004), and thus this difference could account for the different results across these experiments.

Despite the success of the studies of Roesch et al. (2007) and Simon et al. (2007a) in modeling the elevated impulsive choice observed in human cocaine users, an important consideration regarding these findings is the fact that they used experimenter-administered rather than self-administered cocaine. There is ample evidence that the route of administration can have a large impact on neurobiological consequences of cocaine administration that could potentially influence impulsive choice (McFarland et al., 2003; Kalivas et al., 2009). Although there is some limited evidence from animal studies that experimenter- and self-administered cocaine are equivalent in terms of their long-term effects on cognition (Schoenbaum et al., 2004; Calu et al., 2007), the fact remains that self-administration is the best available animal model of patterns of human drug intake. With this issue in mind, we recently examined the effects on impulsive choice of a two-week period of intravenous cocaine self-administration, in rats with a daily intake of at least 30 mg/kg/day, followed by a three-week withdrawal period (Mendez et al., in preparation). As in the Simon et al. study, rats self-administering cocaine showed increased impulsive choice, compared to yoked saline controls, three months after cocaine cessation, suggesting that the dose of cocaine and/or duration of withdrawal is more important than the method of drug delivery in determining its long-term effects on impulsive choice. Interestingly, this same cocaine self-administration regimen did not appear to affect reward preference in a “probabilistic discounting” task (Mendez et al., in preparation), in which rats chose between a small guaranteed reward and a large reward for which the probability of delivery varied across blocks within a session (Cardinal and Howes, 2005; Floresco et al., 2008). Although these data come with the caveat that the rats were tested at a longer withdrawal time point (four months) than rats in the intertemporal choice task, they are consistent with the idea that delay and probabilistic decision making are mediated by distinct (though similar) mechanisms, and that they can be affected independently by behavioral and neurobiological manipulations (Cardinal, 2006; Weber and Huettel, 2008; Adriani et al., 2009; Simon et al., 2009b) – see Rachlin et al. (1991), Green et al. (1999), and Kheramin et al. (2003) for further discussion of these issues.

Neurobiological substrates of drug-induced alterations in impulsive choice

Neurobiological experiments in animals and neuroimaging studies in humans have implicated a network of limbic-striatal brain structures in regulation of decision making and impulsive choice (McClure et al., 2004; Cardinal, 2006; Winstanley et al., 2006a; Bezzina et al., 2008). This evidence is perhaps strongest for the ventral striatum (VS) and its dopaminergic innervation from the ventral tegmental area, and for the prefrontal cortex (PFC). Experimentally-induced lesions within both of these systems produce alterations (either increases or decreases) in impulsive choice (Cardinal et al., 2001; Kheramin et al., 2002; Mobini et al., 2002; Winstanley et al., 2004; Rudebeck et al., 2006; Bezzina et al., 2007), and impulsive choice behavior is correlated with neural activity as assessed in human imaging studies, and with electrophysiological and neurochemical activity in animal studies (McClure et al., 2004; Roesch et al., 2006; Winstanley et al., 2006b). Functional imaging studies in humans have been perhaps most revealing, suggesting that PFC (especially lateral PFC) is particularly important for choices of delayed rewards, whereas VS and its dopaminergic afferents (along with some portions of medial PFC) are particularly important for choices of immediate rewards. Indeed, it has been suggested that it is activity within these two brain systems that determines the balance between choices of delayed vs. immediate rewards, and that alterations in their relative influence could bias behavior toward increased preference for immediate over delayed rewards (Jentsch and Taylor, 1999; McClure et al., 2004; Bechara, 2005; Bickel et al., 2007). This idea is of particular relevance to the issue of drug-induced alterations in impulsive choice, as both PFC and VS/ventral tegmental area undergo robust structural and functional plasticity following chronic drug (particularly stimulant) use, suggesting that drug-induced alterations in these systems may account in part for the effects of chronic drug administration on impulsive choice (Nestler, 2004; Robinson and Kolb, 2004; Kalivas and Volkow, 2005).

Ventral striatal system

Chronic administration of stimulant drugs of abuse can cause lasting increases in VS activity (particularly in the form of enhanced dopamine release) in response to rewards and reward-predictive cues, as well as increases in reward-directed behavior itself (Harmer and Phillips, 1998; Robinson and Berridge, 2001; Olausson et al., 2003; Tindell et al., 2005; Nordquist et al., 2007; Roesch et al., 2007; Simon et al., 2009a; Mendez et al., 2009). Functional brain imaging studies in humans have linked increased VS activity in response to rewards with greater preference for immediate over delayed rewards (i.e. – increased impulsive choice), suggesting that greater reward-induced VS activation (reward “hyper-reactivity”) may contribute to increases in impulsive choice behavior (Hariri et al., 2006). Such findings led Berridge (2007) to propose that enhanced VS activity (and the associated enhancement in reward motivation) resulting from chronic stimulant exposure may increase impulsive choice by preferentially enhancing the value of immediate rewards relative to delayed rewards. This hypothesis links increases in impulsive choice to the concept of psychomotor sensitization, and suggests that such sensitization, induced by chronic drug exposure, could account (at least in part) for increased impulsive choice. Indeed, cocaine administration regimens that cause lasting increases in impulsive choice also produce psychomotor sensitization (Schoenbaum et al., 2004; Schoenbaum and Setlow, 2005; Roesch et al., 2007; Simon et al., 2007a). It is also in accord with a recent finding that the same cocaine administration regimen that causes increased impulsive choice in rats (Simon et al., 2007a) also causes a lasting increase in firing rates of VS neurons in rats performing a go/no-go odor discrimination task (Takahashi et al., 2007).

Arguing against this idea, however, other data show that increased impulsive choice is not an obligatory consequence of robust psychomotor sensitization (see the discussion above of the effects of chronic amphetamine administration: Stanis et al., 2008a), suggesting a more complex role for altered VS activity in drug-induced increases in impulsive choice. In addition, because of the wide variety of chronic drug administration regimens studied, it largely remains to be determined how drug regimens that increase impulsive choice affect neurobiological markers in VS at those same time points (Takahashi et al., 2007). Further complicating the role of drug-induced alterations in VS (particularly activity increases) in increased impulsive choice is the fact that VS lesions also increase impulsive choice, implying that the view that impulsive choice varies directly with VS activity may not be correct in its simplest form (Cardinal et al., 2001; Pothuizen et al., 2005; Bezzina et al., 2007). These gaps and apparent contradictions highlight a need for further work in this area to elucidate relationships between chronic drug-induced alterations in VS and impulsive choice. They also suggest that drug-induced alterations in other brain areas may be more closely related to increased impulsive choice.

Prefrontal cortical system

As described above, activity in PFC has been linked to “self-controlled”, or non-impulsive choices (i.e. – choice of delayed over immediate rewards). Neural activity in some areas of PFC predicts choices of delayed over immediate rewards (McClure et al., 2004), and brain lesion experiments in rats have focused especially on the orbitofrontal cortex (OFC), as this area of PFC has been particularly implicated in decision-making and reward valuation processes (Bechara et al., 2000). Lesions of OFC (but not medial prefrontal cortex) in rats can increase impulsive choice in delay discounting tasks (Cardinal et al., 2001; Mobini et al., 2002), although such lesions can also have the opposite effect (Winstanley et al., 2004). Interestingly, recent evidence suggests that this apparent discrepancy may be due in part to differential regulation of impulsive choice by different OFC subregions, in that lateral OFC lesions may increase impulsive choice whereas medial OFC lesions decrease impulsive choice (Mar et al., 2008), a finding which may correspond to the preferential activation of lateral vs. medial OFC in human subjects during choice of delayed vs. immediate rewards, respectively (McClure et al., 2004) (note that this discrepancy could also stem from differential effects of OFC lesions on sensitivity to reward delay and magnitude: Kheramin et al., 2002).

Individuals with a history of drug (particularly cocaine) use show reduced structural and functional integrity in OFC (Volkow et al., 1993; Bechara et al., 2001; Bolla et al., 2003). Further evidence for possible OFC involvement in chronic drug-induced increases in impulsive choice comes from studies in rats in which chronic cocaine (either experimenter- or self-administered) causes lasting deficits in reversal learning, a signature characteristic of OFC dysfunction (Schoenbaum et al., 2002, 2004; Clark et al., 2004; Stalnaker et al., 2006; Calu et al., 2007). Unfortunately, beyond neuropsychological and imaging data in drug-exposed rats and humans, there is little information available as to the specific drug-induced alterations in OFC that might be linked to increased impulsive choice. However, in one of the first studies to focus particularly on OFC, Winstanley et al. (2007) used a gene array analysis to show that chronic cocaine administration (15 mg/kg, i.p., 2 times/day for 21 days) alters expression of a wide range of genes, including several involved in cortical excitability. In particular, they observed up-regulation of several genes involved in inhibiting cortical activity (perhaps as a compensatory mechanism to counteract chronic cocaine-induced over-excitation), and suggested that such increased inhibition could account for the decreased functional activity of OFC observed in cocaine users (Dom et al., 2005) (although notably, there were no effects of this cocaine administration regimen on impulsive choice in this study).

Increased impulsive choice as a consequence of either cocaine administration or outright OFC damage has been characterized in part as a “hypersensitivity to delays” (Mobini et al., 2002; Roesch et al., 2007). However, an alternative (and/or complementary) perspective is that increased preference for immediate over delayed rewards results from deficits in working memory (Hinson et al., 2003; Shamosh et al., 2008). Simon et al. (2007a) observed that cocaine-exposed rats tested in a delay discounting task spent less time than saline-exposed controls performing anticipatory nosepoke responses into the food trough during the delay period after selection of the large reward. These data could reflect an inability to access information about future outcomes during the delay period, which would be expected to reduce the subjective value of the large reward and promote choice of the smaller, non-delayed reward. Indeed, deficits in attention and working memory after extended drug administration have been observed in both humans and rodents (Hoffman et al., 2006; Verdejo-Garcia et al., 2006; Briand et al., 2008; George et al., 2008). This executive dysfunction may be linked to OFC, as this structure has been implicated in the representation of incentive information in the absence of cues or reinforcers (Schoenbaum and Setlow, 2001).

Implications for pharmacological treatment of drug-induced alterations in impulsive choice

Since the inception of research on delay discounting in drug users, it has been suggested that the elevated impulsive choice observed in these individuals could promote continued drug use, and that targeting this form of impulsivity might serve to reduce drug use and decrease the likelihood of relapse (Perry and Carroll, 2008). Current pharmacological treatments for impulsivity (as a symptom of other conditions such as ADHD) include stimulant medications such as amphetamine and methylphenidate (Dalley et al., 2008). There are obvious concerns about the potential for abuse and diversion of such drugs in patients with a history of drug abuse (Vocci, 2008). However, aside from these concerns, there is some evidence from animal models that such drugs might actually be ineffective, at least in individuals with a history of cocaine use.

Winstanley et al. (2007) examined the effects of acute cocaine administration in rats tested on an intertemporal choice task, using the fixed delays procedure. Acute cocaine caused a decrease in impulsive choice in previously drug-naïve rats. In rats with a history of chronic cocaine administration, however, (15 mg/kg, i.p., 2 times/day, × 3 weeks), acute cocaine had no effect, suggesting that chronic cocaine rendered the rats tolerant to the cognitive effects of acute cocaine administration (Winstanley et al., 2007). The ability of prior chronic cocaine administration to prevent the decreased impulsive choice caused by acute cocaine was reproduced by viral vector-induced overexpression of the constituitively-active transcription factor delta-FosB in OFC. As increased delta-FosB expression in OFC can be caused by chronic cocaine administration, these data suggest that chronic cocaine-induced up-regulation of delta-FosB may cause tolerance to the cognitive effects of subsequent acute cocaine administration.

It is unlikely that cocaine would be used therapeutically to treat impulsive choice; however, a preliminary experiment from our laboratory suggests that chronic cocaine administration similarly blocks the effects of D-amphetamine on impulsive choice (Simon et al., 2007b). Rats were given i.p. injections of cocaine (20 mg/kg, i.p. × 14 days) or saline vehicle, and three weeks later began testing on an intertemporal choice task, using the fixed delays procedure. Several months after cessation of drug treatment, the effects of acute administration of amphetamine or saline vehicle on task performance were examined. Consistent with some previous findings (Wade et al., 2000; van Gaalen et al., 2006), acute D-amphetamine (1.0 mg/kg) decreased impulsive choice in cocaine-naïve saline control rats; however, the same dose had no effect on impulsive choice in the rats with a history of cocaine administration (Figure 1), suggesting that, as in the Winstanley et al. (2007) study, chronic cocaine attenuated the effects of subsequent acute stimulant administration on impulsive choice (see also Dalley et al., 2004). Interestingly, a similar phenomenon was observed following acute administration of the non-specific dopamine receptor antagonist flupenthixol (which increased impulsive choice in the cocaine-naïve but not chronic cocaine-exposed rats), suggesting that chronic cocaine administration results in a general reduction in sensitivity to dopaminergic manipulations of impulsive choice (Figure 1). It remains to be determined whether drugs targeting other neurotransmitter systems (e.g. - atomoxetine targeting the norepinephrine transporter: Robinson et al., 2007) are effective in cocaine-exposed subjects, but the results described above argue against therapeutic targeting of the dopaminergic system for elevated impulsive choice in (at least cocaine) addiction.

Figure 1
Effects of acute administration of D-amphetamine sulfate or flupenthixol on performance in a fixed delays intertemporal choice task, in rats chronically exposed to cocaine or saline vehicle. Amphetamine (1 mg/kg) decreased impulsive choice in cocaine-naïve ...

Summary and Future Directions

The data reviewed above demonstrate that chronic administration of several stimulant drugs of abuse can increase impulsive choice, as assessed in several varieties of intertemporal choice tasks in rats. These effects appear to be most pronounced and long-lasting following chronic cocaine administration, consistent with observations in human cocaine users (Coffey et al., 2003; Kirby and Petry, 2004; Bornovalova et al., 2005; Heil et al., 2006; Roesch et al., 2007; Simon et al., 2007a). However, the effects on impulsive choice of chronic administration of other stimulants have been considerably less well investigated, and it therefore remains to be determined whether apparent differences between the effects of cocaine and other stimulants are due simply to the doses and regimens tested or to true differences in the drugs’ actions. This issue highlights several important shortcomings of the current literature on the effects of chronic drug administration on impulsive choice. First, there has been little or no systematic comparison of the effects of different chronic drug doses and administration regimens on impulsive choice; therefore, even for cocaine, it is not clear to what factor(s) differences in the effects of chronic administration between studies should be attributed. Second, to our knowledge, there are no animal studies investigating the effects of chronic administration of non-stimulant drugs of abuse on impulsive choice; such research would appear to be important in light of the steep delay discounting observed in, for example, alcoholics and heroin users (Vuchinich and Simpson, 1998; Kirby et al., 1999). Third, there has been little consideration of the role of withdrawal duration on chronic drug effects on impulsive choice. A comparison of the chronic cocaine administration studies described above suggests that increased impulsive choice may be more evident at longer withdrawal times, in agreement with a recent study in humans suggesting that current cocaine use may mask cognitive deficits that become evident at longer withdrawal times (Ardila et al., 1991; Berry et al., 1993; Woicik et al., 2008; Volkow et al., 2009). However, the absence of systematic comparisons of different withdrawal times in animal studies renders this idea speculative at present.

In addition to these shortcomings of the existing literature described above, there are several other research areas surrounding chronic drug-induced alterations in impulsive choice that deserve future scrutiny. One concerns the relationship between pre-existing levels of impulsive choice and chronic drug effects on impulsive choice. Studies in human subjects have suggested that a pre-existing tendency to discount delayed rewards more steeply may predispose individuals toward drug use (Bechara, 2005), and studies in rats have shown that greater pre-existing levels of impulsive choice are associated with subsequent faster acquisition of cocaine self-administration and escalation of intake, one of the hallmarks of addictive behavior (Perry and Carroll, 2008). However, the additive effects on impulsive choice of such pre-existing differences combined with chronic drug administration are currently unknown (for example, does chronic cocaine administration selectively increase impulsive choice in some vulnerable subpopulations but not others?). Moreover, it is important to keep in mind that such correlational studies do not reveal whether high levels of impulsive choice actually cause an increase in drug use (e.g. – via preference for immediate drug rewards over the delayed rewards of abstinence), or whether high levels of impulsive choice are merely correlated, but bear no causal relationship with, predisposition for drug use. Another area of future research concerns the relationship between drug-induced alterations in impulsive choice and other aspects of decision-making. Little or nothing is known about how chronic drug administration influences the effects of reward probability, effort, or punishment on decision-making (Mitchell, 1999, 2004; Floresco and Whelan, 2009; Simon et al., 2009b; Mendez et al., in preparation).

Finally, despite an enormous body of literature on neurobiological changes induced by chronic administration of drugs of abuse (particularly in brain regions implicated in regulating impulsive choice), it remains difficult to link these changes directly with drug-induced alterations in impulsive choice, as the majority of these neurobiological changes have been examined with less intense drug administration regimens and/or at shorter withdrawal times than those shown to affect impulsive choice. Clearly, more research is needed in this and all of the areas described above, in order to understand the relationships between chronic drug use and elevated impulsive choice. Such research could have considerable potential for our understanding of the causes and treatment of drug addiction, as well as other conditions such as gambling disorders and ADHD in which altered impulsive choice plays a role (Reynolds, 2006; Winstanley et al., 2006a).

Acknowledgements

We thank the Drug Supply Program at NIDA for kindly providing cocaine HCl, and Tara Busch, Andrew Richardson, and Kristy Walls for technical assistance. Supported by DA018764 (BS), MH65728 (IAM), and DA023331 (NWS) from the National Institutes of Health.

Supported by DA018764 (BS), MH65728 (IAM), and DA023331 (NWS) from the NIH

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • Adriani W, Boyer F, Gioiosa L, Macrì S, Dreyer JL, Laviola G. Increased impulsive behavior and risk proneness following lentivirus-mediated dopamine transporter over-expression in rats’ nucleus accumbens. Neurosci. 2009;159:47–58. [PubMed]
  • Ainslie G. Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol Bull. 1975;82:463–496. [PubMed]
  • Ardila A, Rosselli M, Strumwasser S. Neuropsychological deficits in chronic cocaine abusers. Int J Neurosci. 1991;57:73–79. [PubMed]
  • Barbelivien A, Billy E, Lazarus C, Kelche C, Majchrzak M. Rats with different profiles of impulsive choice behavior exhibit differences in responses to caffeine and d-amphetamine and in medial prefrontal cortex 5-HT utilization. Behav Brain Res. 2008;187:273–283. [PubMed]
  • Bechara A. Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nat Neurosci. 2005;8:1458–1463. [PubMed]
  • Bechara A, Tranel D, Damasio H. Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain. 2000;123:2189–2202. [PubMed]
  • Bechara A, Dolan S, Denburg N, Hindes A, Anderson SW, Nathan PE. Decision-making deficits, linked to dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia. 2001;39:376–389. [PubMed]
  • Belcher AM, O'Dell SJ, Marshall JF. Impaired object recognition memory following methamphetamine, but not p-chloroamphetamine- or d-amphetamine-induced neurotoxicity. Neuropsychopharmacol. 2005;30:2026–2034. [PubMed]
  • Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ. High impulsivity predicts the switch to compulsive cocaine-taking. Science. 2008;320:1352–1355. [PMC free article] [PubMed]
  • Berridge KC. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacol (Berl) 2007;191:391–331. [PubMed]
  • Berry J, van Gorp WG, Herzberg DS, Hinkin C, Boone K, Steinman L, et al. Neuropsychological deficits in abstinent cocaine abusers: preliminary findings after two weeks of abstinence. Drug Alcohol Depend. 1993;32:231–237. [PubMed]
  • Bezzina G, Cheung T, Asgari K, Hampson C, Body S, Bradshaw C, et al. Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis. Psychopharmacol (Berl) 2007;195:71–84. [PMC free article] [PubMed]
  • Bezzina G, Boon FSd, Hampson CL, Cheung THC, Body S, Bradshaw CM, et al. Effect of quinolinic acid-induced lesions of the subthalamic nucleus on performance on a progressive-ratio schedule of reinforcement: A quantitative analysis. Behav Brain Res. 2008;195:223–230. [PMC free article] [PubMed]
  • Bickel WK, Marsch L. Toward a behavioral economic understanding of drug dependence: delay discounting processes. Addiction. 2001;96:73–86. [PubMed]
  • Bickel WK, Odum AL, Madden GJ. Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers. Psychopharmacol (Berl) 1999;146:447–454. [PubMed]
  • Bickel WK, Miller ML, Yi R, Kowal BP, Lindquist DM, Pitcock JA. Behavioral and neuroeconomics of drug addiction: Competing neural systems and temporal discounting processes. Drug Alcohol Depend. 2007;90:S85–S91. [PMC free article] [PubMed]
  • Bolla KI, Eldreth DA, London ED, Kiehl KA, Mouratidis M, Contoreggi C, et al. Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decision-making task. Neuroimage. 2003;19:1085–1094. [PMC free article] [PubMed]
  • Bornovalova MA, Daughters SB, Hernandez GD, Richards JB, Lejuez CW. Differences in impulsivity and risk-taking propensity between primary users of crack cocaine and primary users of heroin in a residential substance-use program. Exp Clin Psychopharmacol. 2005;13:311–318. [PubMed]
  • Briand LA, Robinson TE, Maren S. Enhancement of auditory fear conditioning after housing in a complex environment is attenuated by prior treatment with amphetamine. Learn Mem. 2005;12:553–556. [PubMed]
  • Briand LA, Flagel SB, Garcia-Fuster MJ, Watson SJ, Akil H, Sarter M, et al. Persistent alterations in cognitive function and prefrontal dopamine D2 receptors following extended, but not limited, access to self-administered cocaine. Neuropsychopharmacol. 2008;33:2969–2980. [PMC free article] [PubMed]
  • Burke KA, Franz TM, Gugsa N, Schoenbaum G. Prior cocaine exposure disrupts extinction of fear conditioning. Learn Mem. 2006;13:416–421. [PMC free article] [PubMed]
  • Calu DJ, Stalnaker TA, Franz TM, Singh T, Shaham Y, Schoenbaum G. Withdrawal from cocaine self-administration produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats. Learn Mem. 2007;14:325–328. [PMC free article] [PubMed]
  • Cardinal RN. Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw. 2006;19:1277–1301. [PubMed]
  • Cardinal R, Howes N. Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 2005;6:37. [PMC free article] [PubMed]
  • Cardinal RN, Robbins TW, Everitt BJ. The effects of d-amphetamine, chlordiazepoxide, α-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacol (Berl) 2000;152:362–375. [PubMed]
  • Cardinal RN, Pennicot DR, Sugathapala CL, Robbins TW, Everitt BJ. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science. 2001;292:2499–2501. [PubMed]
  • Clark L, Cools R, Robbins TW. The neuropsychology of ventral prefrontal cortex: decision-making and reversal learning. Brain Cogn. 2004;55:41–53. [PubMed]
  • Coffey SF, Gudleski GD, Saladin ME, Brady KT. Impulsivity and rapid discounting of delayed rewards in cocaine-dependent individuals. Exp Clin Psychopharmacol. 2003;11:18–25. [PubMed]
  • Dallery J, Locey ML. Effects of acute and chronic nicotine on impulsive choice in rats. Behav Pharmacol. 2005;16:15–23. [PubMed]
  • Dalley JW, Theobald DEH, Berry D, Milstein JA, Laane K, Everitt BJ, et al. Cognitive sequelae of intravenous amphetamine self-administration in rats: Evidence for selective effects on attentional performance. Neuropsychopharmacol. 2004;30:525–537. [PubMed]
  • Dalley JW, Mar AC, Economidou D, Robbins TW. Neurobehavioral mechanisms of impulsivity: Fronto-striatal systems and functional neurochemistry. Pharmacol Biochem Behav. 2008;90:250–260. [PubMed]
  • Dalley JW, Fryer TD, Brichard L, Robinson ESJ, Theobald DEH, Laane K, et al. Nucleus Accumbens D2/3 Receptors Predict Trait Impulsivity and Cocaine Reinforcement. Science. 2007;315:1267–1270. [PMC free article] [PubMed]
  • de Wit H. Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addict Biol. 2009;14:22–31. [PubMed]
  • de Wit H, Richards J. Dual determinants of drug use in humans: reward and impulsivity. Nebraska Symp Motivation. 2003;50:19–55. [PubMed]
  • de Wit H, Enggasser JL, Richards JB. Acute administration of d-amphetamine decreases impulsivity in healthy volunteers. Neuropsychopharmacol. 2002;27:813–825. [PubMed]
  • Diergaarde L, Pattij T, Poortvliet I, Hogenboom F, de Vries W, Schoffelmeer ANM, et al. Impulsive choice and impulsive action predict vulnerability to distinct stages of nicotine seeking in rats. Biol Psychiatry. 2008;63:301–308. [PubMed]
  • Diller JW, Saunders BT, Anderson KG. Effects of acute and repeated administration of caffeine on temporal discounting in rats. Pharmacol Biochem Behav. 2008a;89:546–555. [PubMed]
  • Dom G, Sabbe B, Hulstijn W, van den Brink W. Substance use disorders and the orbitofrontal cortex. Br J Psychiatry. 2005;187:209–220. [PubMed]
  • Ersche K, Roiser J, Robbins T, Sahakian B. Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacol (Berl) 2008;197:421–431. [PubMed]
  • Evenden JL. Varieties of impulsivity. Psychopharmacol (Berl) 1999;146:348–361. [PubMed]
  • Evenden JL, Ryan CN. The pharmacology of impulsive behavior in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacol (Berl) 1996;128:161–170. [PubMed]
  • Floresco S, Whelan J. Perturbations in different forms of cost/benefit decision making induced by repeated amphetamine exposure. Psychopharmacol (Berl) 2009 In Press. [PubMed]
  • Floresco SB, St. Onge JR, Ghods-Sharifi S, Winstanley CA. Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision-making. Cogn Affect Behav Neurosci. 2008;8:375–389. [PubMed]
  • Garavan H, Stout JC. Neurocognitive insights into substance abuse. Trends Cogn Sci. 2005;9:195–201. [PubMed]
  • George O, Mandyam CD, Wee S, Koob GF. Extended access to cocaine self-administration produces long-lasting prefrontal cortex-dependent working memory impairments. Neuropsychopharmacol. 2008;33:2474–2482. [PMC free article] [PubMed]
  • Green L, Myerson J, Ostaszewski P. Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes. J Exp Psychol Learn Mem Cogn. 1999;25:418–427. [PubMed]
  • Grimm JW, Hope BT, Wise RA, Shaham Y. Neuroadaptation: Incubation of cocaine craving after withdrawal. Nature. 2001;412:141–142. [PMC free article] [PubMed]
  • Hariri AR, Brown SM, Williamson DE, Flory JD, de Wit H, Manuck SB. Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. J Neurosci. 2006;26:13213–13217. [PubMed]
  • Harmer CJ, Phillips GD. Enhanced appetitive conditioning following repeated pretreatment with d-amphetamine. Behav Pharmacol. 1998;9:299–308. [PubMed]
  • Heil SH, Johnston MW, Higgins ST, Bickel WK. Delay discounting in currently using and currently abstinent cocaine-dependent outpatients and non-drug-using matched controls. Addict Behav. 2006;31:1290–1294. [PubMed]
  • Hinson JM, Jameson TL, Whitney P. Impulsive decision making and working memory. J Exp Psychol Learn Mem Cog. 2003;29:298–306. [PubMed]
  • Ho MY, Mobini S, Chiang TJ, Bradshaw CM, Szabadi E. Theory and method in the quantitative analysis of ”impulsive choice” behaviour: implications for psychopharmacology. Psychopharmacol (Berl) 1999;146:362–372. [PubMed]
  • Hoffman W, Moore M, Templin R, McFarland B, Hitzemann R, Mitchell S. Neuropsychological function and delay discounting in methamphetamine-dependent individuals. Psychopharmacol (Berl) 2006;188:162–170. [PubMed]
  • Hoffman WF, Schwartz DL, Huckans MS, McFarland BH, Meiri G, Stevens AA, et al. Cortical activation during delay discounting in abstinent methamphetamine dependent individuals. Psychopharmacol (Berl) 2008;201:183–193. [PMC free article] [PubMed]
  • Jentsch JD, Taylor JR. Impulsivity resulting from frontostriatal dysfunction in drug abuse: implications for the control of behavior by reward-related stimuli. Psychopharmacol (Berl) 1999;146:373–390. [PubMed]
  • Kalivas PW, Volkow ND. The neural basis of addiction: a pathology off motivation and choice. Am J Psychiatry. 2005;162:1403–1413. [PubMed]
  • Kalivas PW, LaLumiere R, Knackstedt L, Shen H. Glutamate transmission in addiction. Neuropharmacol. 2009;56:169–173. [PMC free article] [PubMed]
  • Kheramin S, Body S, Mobini S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, et al. Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis. Psychopharmacol (Berl) 2002;165:9–17. [PubMed]
  • Kheramin S, Body S, Ho MY, Velázquez-Martinez DN, Bradshaw CM, Szabadi E, et al. Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis. Behav Processes. 2003;64:239–250. [PubMed]
  • Kirby KN, Petry NM. Heroin and cocaine abusers of higher discount rates for delayed rewards than alcoholics or non-drug-using controls. Addiction. 2004;99:461–471. [PubMed]
  • Kirby KN, Petry NM, Bickel WK. Heroin addicts have higher discounting rates for delayed rewards than non-drug-using controls. J Exp Psychol: General. 1999;128:78–87. [PubMed]
  • Logue AW. Research on self-control: an integrating framework. Behav Brain Sci. 1988;11:665–709.
  • Logue AW, Tobin H, Chelonis JJ, Wang RY, Geary N, Schacter S. Cocaine decreases self-control in rats: a preliminary study. Psychopharmacol (Berl) 1992;109:245–247. [PubMed]
  • Lu L, Grimm JW, Hope BT, Shaham Y. Incubation of cocaine craving after withdrawal: a review of preclinical data. Neuropharmacol. 2004;47:214–226. [PubMed]
  • Mar AC, Walker A, Pelloux Y, Theobald DEH, Eagle DM, Robbins TW. Effects of medial versus lateral orbitofrontal cortex lesions on temporal discounting in the rat. Soc Neurosci Abs. 2008 876.14/SS11.
  • Mazur JE. An adjusting procedure for studying delayed reinforcement. In: Commons ML, Mazur JE, Nevin JA, Rachlin H, editors. The effect of delay and of intervening events on reinforcement value. Erlbaum; Hillsdale, NJ: 1987. pp. 55–73.
  • McClure S, Laibson DI, Loewenstein G, Cohen JD. Separate neural systems value immediate and delayed monetary rewards. Science. 2004;15:503–507. [PubMed]
  • McFarland K, Lapish CC, Kalivas PW. Prefrontal glutamate release into the core of the nucleus accumbens mediates cocaine-induced reinstatement of drug-seeking behavior. J Neurosci. 2003;23:3531–3537. [PubMed]
  • Mendez IA, Montgomery KS, LaSarge CL, Simon NW, Bizon JL, Setlow B. Long-term effects of prior cocaine exposure on Morris water maze performance. Neurobiol Learn Mem. 2008;89:185–191. [PMC free article] [PubMed]
  • Mendez IA, Williams MT, Bhavsar A, Lu A, Bizon JL, Setlow B. Long-lasting sensitization of reward-directed behavior by amphetamine. Behav Brain Res. 2009;201:74–79. [PMC free article] [PubMed]
  • Mendez IA, Simon NW, Hart N, Mitchell MR, Nation JR, Wellman PJ, Setlow B. Self-administered cocaine causes lasting increases in impulsive choice in a delay discounting task. in preparation. [PMC free article] [PubMed]
  • Mitchell SH. Measures of impulsivity in cigarette smokers and non-smokers. Psychopharmacol (Berl) 1999;146:455–464. [PubMed]
  • Mitchell SH. Effects of short-term nicotine deprivation on decision-making: delay, uncertainty, and effort discounting. Nicotine and Tob Res. 2004;6:819–828. [PubMed]
  • Mobini S, Body S, Ho M-Y, Bradshaw CM, Szabadi E, Deakin JFW, et al. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacol (Berl) 2002;160:290–298. [PubMed]
  • Monterosso J, Ehrman R, Napier K, O'Brien CP, Childress AR. Three-decision-making tasks in cocaine-dependent patients; do they measure the same construct? Addiction. 2001;96:1825–1837. [PubMed]
  • Monterosso JR, Ainslie G, Xu J, Cordova X, Domier CP, London ED. Frontoparietal cortical activity of methamphetamine-dependent and comparison subjects performing a delay discounting task. Hum Brain Map. 2007;28:383–393. [PubMed]
  • Nestler EJ. Molecular mechanisms of drug addiction. Neuropharmacol. 2004;47:24–32. [PubMed]
  • Nordquist RE, Voorn P, de Mooij-van Malsen JG, Joosten RN, Pennartz CM, Vanderschuren LJ. Augmented reinforcer value and accelerated habit formation after repeated amphetamine treatment. Eur Neuropsychopharmacol. 2007;17:532–540. [PubMed]
  • O'Brien CP. Anticraving medications for relapse prevention: A possible new class of psychoactive medications. Am J Psychiatry. 2005;162:1423–1431. [PubMed]
  • Olausson P, Jentsch JD, Taylor JR. Repeated nicotine exposure enhances reward-related learning in the rat. Neuropsychopharmacol. 2003;28:1264–1271. [PubMed]
  • Olmstead MC. Animal models of drug addiction: where do we go from here? Q J Exp Psychol. 2006;59:625–653. [PubMed]
  • Olmstead M, Hellemans K, Paine T. Alcohol-induced impulsivity in rats: an effect of cue salience? Psychopharmacol (Berl) 2006;184:221–228. [PubMed]
  • Paine TA, Dringenberg HC, Olmstead MC. Effects of chronic cocaine on impulsivity: relation to cortical serotonin mechanisms. Behav Brain Res. 2003;147:135–147. [PubMed]
  • Pattij T, Vanderschuren LJMJ. The neuropharmacology of impulsive behaviour. Trends Pharmacol Sci. 2008;29:192–199. [PubMed]
  • Paulson PE, Robinson TE. Amphetamine-induced time-dependent sensitization of dopamine neurotransmission in the dorsal and ventral striatum: a microdialysis study in behaving rats. Synapse. 1995;19:56–65. [PMC free article] [PubMed]
  • Perry J, Carroll M. The role of impulsive behavior in drug abuse. Psychopharmacol (Berl) 2008;200:1–26. [PubMed]
  • Perry JL, Larson EB, German JP, Madden GJ, Carroll ME. Impulsivity (delay discounting) as a predictor of acquisition of IV cocaine self-administration in female rats. Psychopharmacol (Berl) 2005;154:193–201. [PubMed]
  • Perry JL, Nelson SE, Carroll ME. Impulsive choice as a predictor of acquisition of IV cocaine self-administration and reinstatement of cocaine-seeking behavior in male and female rats. Exp Clin Psychopharmacol. 2008;16:165–177. [PubMed]
  • Pitts RC, McKinney AP. Effects of methylphenidate and morphine on delay-discount functions obtained within sessions. J Exp Anal Behav. 2005;83:297–314. [PMC free article] [PubMed]
  • Pothuizen HJ, Jongen-Relo AL, Feldon J, Yee BK. Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats. Eur J Neurosci. 2005;22:2605–2616. [PubMed]
  • Rachlin H, Green L. Commitment, choice, and self-control. J Exp Anal Behav. 1972;17:15–22. [PMC free article] [PubMed]
  • Rachlin H, Raineri A, Cross D. Subjective probability and delay. J Exp Anal Behav. 1991;55:233–244. [PMC free article] [PubMed]
  • Reynolds B. A review of delay-discounting research with human: relations to drug and gambling. Behav Pharmacol. 2006;17:651–667. [PubMed]
  • Richards JB, Sabol KE, de Wit H. Effects of methamphetamine on the adjusting amount procedure, a model of impulsive behavior in rats. Psychopharmacol (Berl) 1999;146:432–439. [PubMed]
  • Robinson ESJ, Eagle DM, Mar AC, Bari A, Banerjee G, Jiang X, et al. Similar effects of the selective noradrenaline reuptake inhibitor atomoxetine on three distinct forms of impulsivity in the rat. Neuropsychopharmacol. 2007;33:1028–1037. [PubMed]
  • Robinson TE, Berridge KC. Incentive-sensitization and addiction. Addiction. 2001;96:103–114. [PubMed]
  • Robinson TE, Kolb B. Structural plasticity associated with exposure to drugs of abuse. Neuropharmacol. 2004;47:33–46. [PubMed]
  • Roesch MR, Taylor AR, Schoenbaum G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron. 2006;51:509–520. [PMC free article] [PubMed]
  • Roesch MR, Takahashi Y, Gugsa N, Bissonette GB, Schoenbaum G. Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude. J Neurosci. 2007;27:245–250. [PMC free article] [PubMed]
  • Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MFS. Separate neural pathways process different decision costs. Nat Neurosci. 2006;9:1161–1168. [PubMed]
  • Schoenbaum G, Setlow B. Integrating orbitofrontal cortex into prefrontal theory: common processing themes across species and subdivisions. Learn Mem. 2001;8:134–147. [PubMed]
  • Schoenbaum G, Setlow B. Cocaine makes actions insensitive to outcomes but not extinction: implications for altered orbitofrontal-amygdalar function. Cereb Cortex. 2005;15:1162–1169. [PubMed]
  • Schoenbaum G, Nugent SL, Saddoris MP, Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport. 2002;13:885–890. [PubMed]
  • Schoenbaum G, Ramus SJ, Shaham Y, Saddoris MP, Setlow B. Cocaine-experienced rats exhibit learning deficits in a task sensitive to orbitofrontal cortex lesions. Eur J Neurosci. 2004;19:1997–2002. [PubMed]
  • Shamosh NA, DeYoung CG, Green AE, L RD, R JM, A CAR, et al. Individual differences in delay discounting: Relation to intelligence, working memory, and anterior prefrontal cortex. Psychol Sci. 2008;19:904–911. [PubMed]
  • Simon NW, Mendez IA, Setlow B. Cocaine exposure causes long term increases in impulsive choice. Behav Neurosci. 2007a;121:543–549. [PMC free article] [PubMed]
  • Simon NW, Busch TN, Richardson AJ, Walls KJ, Setlow B. Cocaine exposure and impulsive choice: A parametric analysis. Soc Neurosci Abs. 2007b
  • Simon NW, Mendez IA, Setlow B. Effects of prior amphetamine exposure on approach strategy in appetitive Pavlovian conditioning in rats. Psychopharmacol (Berl) 2009a;202:699–709. [PubMed]
  • Simon NW, Gilbert RJ, Mayse JD, Bizon JL, Setlow B. Balancing risk and reward: A rat model of risky decision making. Neuropsychopharmacol. 2009b In Press. [PMC free article] [PubMed]
  • Stalnaker TA, Roesch MR, Franz TM, Burke KA, Schoenbaum G. Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision making. Eur J Neurosci. 2006;24:2643–2653. [PMC free article] [PubMed]
  • Stanis JJ, Marquez Avila H, White M, Gulley JM. Dissociation between long-lasting behavioral sensitization to amphetamine and impulsive choice in rats performing a delay-discounting task. Psychopharmacol (Berl) 2008a;199:539–548. [PMC free article] [PubMed]
  • Stanis JJ, Burns RM, Sherrill LK, Gulley JM. Disparate cocaine-induced locomotion as a predictor of choice behavior in rats trained in a delay-discounting task. Drug Alcohol Depend. 2008b;98:54–62. [PMC free article] [PubMed]
  • Takahashi Y, Roesch MR, Stalnaker TA, Schoenbaum G. Cocaine exposure shifts the balance of associative encoding from ventral to dorsolateral striatum. Front Integr Neurosci. 2007;1:1–20. [PMC free article] [PubMed]
  • Tarter RE, Kirisci L, Mezzich A, Cornelius JR, Pajer K, Vanyukov M, et al. Neurobehavioral disinhibition in childhood predicts early age at onset of substance use disorder. Am J Psychiatry. 2003;160:1078–1085. [PubMed]
  • Tindell AJ, Berridge KC, Zhang J, Pecina S, Aldridge JW. Ventral pallidal neurons code incentive motivation: amplification by mesolimbic sensitization and amphetamine. Eur J Neurosci. 2005;22:2617–2634. [PubMed]
  • van Gaalen MM, van Koten R, Schoffelmeer AN, Vanderschuren LJ. Critical involvement of dopaminergic neurotransmission in impulsive decision making. Biol Psychiatry. 2006;60:66–73. [PubMed]
  • Verdejo-Garcia A, Bechara A, Recknor EC, Perez-Garcia M. Executive dysfunction in substance dependent individuals during drug use and abstinence: an examination of the behavioral, cognitive, and emotional correlates of addiction. J Int Neuropsychol Soc. 2006;12:405–415. [PubMed]
  • Vocci FJ. Cognitive remediation in the treatment of stimulant abuse disorders: a research agenda. Exp Clin Psychopharmacol. 2008;16:484–497. [PubMed]
  • Volkow ND, Fowler JS, Wang G-J, Hitzemann R, Logan J, Schlyer DJ, et al. Decreased dopamine D2 receptor availability is associated with reduced frontal metabolism in cocaine abusers. Synapse. 1993;14:169–177. [PubMed]
  • Volkow ND, Fowler JS, Wang GJ, Baler R, Telang F. Imaging dopamine's role in drug abuse and addiction. Neuropharmacol. 2009;56:3–8. [PMC free article] [PubMed]
  • Vuchinich RE, Simpson CA. Hyperbolic temporal discounting in social drinkers and problem drinkers. Exp Clin Psychopharmacol. 1998;6:292–305. [PubMed]
  • Wade TR, De Wit H, Richards JB. Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacol (Berl) 2000;150:90–101. [PubMed]
  • Weber BJ, Huettel SA. The neural substrates of probabilistic and intertemporal decision making. Brain Res. 2008;1234:104–115. [PMC free article] [PubMed]
  • Winstanley CA, Theobald DEH, Cardinal RN, Robbins TW. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J Neurosci. 2004;24:4718–4722. [PubMed]
  • Winstanley CA, Eagle DM, Robbins TW. Behavioral models of impulsivity in relation to ADHD: Translation between clinical and preclinical studies. Clin Psychol Review. 2006a;26:379–395. [PMC free article] [PubMed]
  • Winstanley CA, Theobald DEH, Dalley JW, Cardinal RN, Robbins TW. Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice. Cereb Cortex. 2006b;16:106–114. [PubMed]
  • Winstanley CA, LaPlant Q, Theobald DE, Green TA, Bachtell RK, Perrotti LI, et al. DeltaFosB induction in orbitofrontal cortex mediates tolerance to cocaine-induced cognitive dysfunction. J Neurosci. 2007;27:10497–10507. [PubMed]
  • Woicik PA, Moeller SJ, Alia-Klein N, Maloney T, Lukasik TM, Yeliosof O, et al. The neuropsychology of cocaine addiction: Recent cocaine use masks impairment. Neuropsychopharmacol. 2008;34:1112–1122. [PMC free article] [PubMed]
  • Woolverton W, Anderson K. Effects of delay to reinforcement on the choice between cocaine and food in rhesus monkeys. Psychopharmacol (Berl) 2006;186:99–106. [PubMed]