Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Behav Processes. Author manuscript; available in PMC 2017 May 1.
Published in final edited form as:
PMCID: PMC4826824

Impulsive Choice and Pre-Exposure to Delays: III. Four-Month Test-Retest Outcomes in Male Wistar Rats


Delay discounting describes the tendency for organisms to devalue outcomes because they are delayed. Robust, positive correlations exist between excessive delay discounting and many maladaptive behaviors (e.g., substance abuse, obesity). Several studies have demonstrated that delay discounting can be reduced and this may hold promise for improving treatment outcomes. One method of reducing delay discounting provides rats with extended training with delayed reinforcement (i.e., delay-exposure training) and this significantly reduces impulsive choices, relative to rats trained with an equal number of immediate-reinforcement sessions (i.e., immediate-exposure training). To evaluate the stability of this effect, 12 weanling male Wistar rats were randomly assigned to receive either delay-exposure or immediate-exposure training for 120 sessions. Impulsive choice was assessed using an increasing-delay procedure immediately following training and 120 days after completion of the initial assessment. Delay-exposed rats discounted delayed food rewards significantly less than immediate-exposed rats in the initial assessment and the reassessment conducted 120 days later. These results are encouraging as they suggest that the effects of delay-exposure training are robust to the passage of time and intervening experience.

Keywords: Delay Discounting, Impulsivity, Impulsive Choice, Delay-exposure Training, Rats

1.1 Introduction

Impulsivity is a multifaceted construct that describes many forms of maladaptive behaviors (for review, see Evenden, 1999). One such form of impulsivity—impulsive choice—involves preference for smaller, sooner rewards (SSR) over larger, later rewards (LLR). Delay discounting describes the subjective devaluation of the LLR, and this process is thought to underlie impulsive choice (for reviews, see Madden and Johnson, 2010; Stein and Madden, 2013; Odum, 2011).

Strong, positive correlations have been observed between excessively discounting delayed rewards and many problematic behaviors such as substance abuse (e.g., Heil et al., 2006; Madden et al., 1997; Vuchinich & Simpson, 1998; for meta-analysis, see MacKillop et al., 2011), poor health behaviors (e.g., Bradford, 2010; Daugherty and Brase, 2010), and pathological gambling (e.g., Albein-Urios et al., 2012; Alessi and Petry, 2003; Petry, 2001). Because excessive delay discounting is a common process shared among many problematic behaviors, some consider it a trans-disease process (Bickel et al., 2012; Bickel and Mueller, 2009). If excessive delay discounting underlies poor decision-making, exploring techniques for reducing impulsive choice may yield therapeutic benefits for a wide range of behavioral maladies (Bickel et al., 2015; Gray and MacKillop, 2015; Koffarnus et al., 2013).

Impulsive choice can be experimentally reduced in humans (for review, see Gray and MacKillop, 2015; Koffarnus et al., 2013) and nonhumans (Mazur and Logue, 1978; Stein et al., 2013, 2015). In humans, reductions in impulsive choice have been observed using a number of strategies such as working-memory training (Bickel et al., 2011), contingency management of substance use (e.g., Landes et al., 2012; Yi et al., 2008), and episodic future thinking (e.g., Peters and Büchel, 2010; Lin and Epstein, 2014). In nonhumans, a training regimen involving early and extended exposure to delayed reinforcement resulted in significant decreases in impulsive choice (Stein et al., 2013, 2015). In the latter studies, one group of weanling Long Evans rats learned to press a lever for food delayed by 17.5 s; the rats subsequently completed 120 sessions of this Delay-Exposure (DE) training. A second group of rats completed the same sessions but food was delivered immediately after the lever press (Immediate-Exposure group; IE). At the post-training impulsive-choice assessment, DE rats made significantly fewer impulsive choices than IE rats (common language [CL] effect size1 = .80 and .82, respectively in Stein et al., 2013, 2015). These differences remained significant at retests conducted approximately 66 and 48 days, respectively, after rats were given the opportunity to consume oral alcohol.

One goal of the current study was to systematically replicate the methods of Stein et al. (2013, 2015) to evaluate the duration of the DE effect at a longer follow-up interval. As such, the follow-up interval was extended to 120 days and rats did not consume alcohol during the test-retest interval. A second goal of the current study to examine whether the DE effect generalizes to a different strain of rats. To that end, Wistar rats were used instead of Long Evans rats. Wistar rats are commonly used in studies assessing delay discounting as a predictor for cocaine self-administration (e.g., Anker et al., 2009; Perry et al., 2005; Perry et al., 2008; Regier, Claxton, Zlebnik, & Caroll, 2014; Broos et al., 2012). Evaluating the DE effect in Wistars was conducted as a precursor to a larger study of the effect of this training on cocaine self-administration.

2.1 Method

2.1.1 Subjects

Subjects were 12 naïve, male Wistar rats (Harlan, Indianapolis, IN) approximately 21 days old at intake. Rats were block-randomized to either the DE or IE groups (n = 6 per group). Rats were individually housed in an animal colony operating on a 12 hr light:dark cycle (light onset at 7:00 am). After 5 days of free access to food, rats were gradually food restricted to 85% of their dealer-supplied growth curve free-feeding weights. They were maintained at this weight for all behavioral assessments but were otherwise given free access to food during the test-retest interval. Throughout the experiment, rats had free access to water in their home cage. Experimental sessions were conducted daily at the same time. Supplemental food was provided 2 hrs post-session.

2.1.2 Apparatus

Six identical operant chambers (Med Associates, St. Albans, VT), housed within ventilated sound-attenuating cubicles and equipped with white-noise speakers, were used. Two low-profile retractable levers were positioned on the front wall of each chamber (10.5 cm above the grid floor). A pellet dispenser delivered 45-mg food pellets (Bio-Serv, Frenchtown, NJ) to a receptacle that was centered between the two front-wall levers. An identical lever was centered on the opposing chamber wall (10.5 cm above the grid floor). Above each lever was a 28-V cue light.

2.1.3 Procedures

Figure 1 depicts the order of experimental conditions and approximate age of the rats.

Figure 1
Order of experimental conditions and approximate age of the rats. Delay-Exposure and Immediate-Exposure Training

An autoshaping procedure was used to establish responding on the rear-wall lever (for a detailed description, see Stein et al., 2013). Autoshaping continued until rats pressed the rear-wall lever to earn ≥ 90% of the reinforcers for 2 consecutive sessions. Next, rats received 120 sessions of DE or IE training. For both groups, the presentation of the rear-wall lever and the cue light marked the beginning of each trial. For DE rats, one lever press retracted the lever and initiated a 17.5-s delay during which the cue light remained illuminated. Following the delay, the cue light was extinguished and two food pellets were delivered. For IE rats, one lever press immediately extinguished the cue light and two food pellets were delivered. For the remainder of the trial, no experimental stimuli were presented; trials began every 60 s. Failure to respond within 20 s of lever insertion was scored as an omission and omitted trials were repeated. Sessions ended after 80 completed trials or after 2 hrs, whichever came first. Impulsive-Choice Task

Immediately following DE or IE training, an increasing-delay procedure (Evenden and Ryan, 1996) was used to assess impulsive choice. Sessions were divided into three trial-blocks, each separated by a 7-min blackout. Each trial-block consisted of 6 forced- and 14 free-choice trials. In forced-choice trials, the rear-wall lever and cue light were presented at trial onset. One response retracted the lever, extinguished the light, and either the left or right front-wall lever and cue light were presented. One response to the front-wall lever retracted the lever, extinguished the light, and the reinforcer associated with that lever was delivered. A post-food blackout ensured that new trials began every 90 s. Free-choice trials were identical to forced-choice trials with the exception that both levers (and cue lights) were presented following a rear-wall lever press, and both levers were retracted following a choice. Failure to respond to any lever within 30 s was scored as an omission and forced-choice trials were repeated.

Amount-discrimination sessions were conducted initially; all free-choice trials in these sessions were between one and three food pellets sans delay. Following 2 consecutive days with ≥ 90% choice of the larger reward, the delay to the larger reward was increased across trial-blocks in the following order: 0, 15, and 30 s. If the LLR was selected, the cue light above the lever remained lit until the delay elapsed and three pellets were delivered. Following 7 sessions in the impulsive-choice procedure, a single amount-discrimination session was conducted to evaluate if choice was sensitive to reward amount and delay, or instead showed a habitual pattern of avoiding the larger reward after the first trial block. Subsequent to that probe session, testing continued for at least 6 more sessions and until the following stability criteria were met: 1) ≥ 80% choice of the larger reward in the 0-s delay block for 5 consecutive sessions, 2) percent LLR choice in each of the final 5 sessions did not deviate by more than 20% from the 5-day mean, and 3) no monotonic increasing or decreasing trend was observed over the last 3 sessions. Testing was terminated upon meeting this stability criterion.

Impulsive choice was reassessed 120 days after the completion of the initial assessment. During the test-retest interval, all rats completed approximately 22 sessions in an operant task not relevant to the current experiment (all rats completed the same task, with no significant differences in reinforcer rate across groups). Subsequently, rats were returned to their free-feeding weights and remained in their home cages until the impulsive-choice reassessment. Data Analysis

Separate, independent-samples t-tests were used to examine between-group differences in the following measures from training: 1) number of days to acquire rear-wall lever pressing and 2) trials completed, omissions, and response latencies during DE/IE training (averaged over the final 10 sessions).

Percent LLR choice was averaged across the final 5 days of the assessments. To quantify impulsive choice, area under the curve (AUC; see Myerson et al. 2011) was calculated from the average percent LLR choice for each rat; higher AUC values indicate greater preference for the LLR. Pearson’s r correlation coefficients were used to examine the relationship between post-training AUC scores and those obtained 120-days later at the retest. Separate, mixed-model ANOVAs were used to examine the main effects of the within-subject factor (Time), the between-subject factor (Group), and the interaction for the following measures: 1) AUC values, 2) number of days to meet the amount-discrimination criteria and the impulsive-choice stability criteria, 3) percent LLR choice during the amount-discrimination probe sessions, and 4) omissions and response latencies (averaged over the final 5 sessions). Post-hoc comparisons of significant findings were made by conducting separate, independent-samples t-tests. Bonferroni’s correction was applied to the post-hoc comparisons resulting in a criterion alpha value of .025. All other tests were deemed statistically significant at p < .05.

3.1 Results

The behavior of one DE rat was excluded from analysis because an intractable side bias was evident in the impulsive-choice reassessment; this exclusion did not affect the significance of the tests prior to the follow-up. No between-group difference was observed in the number of days to establish rear-wall lever pressing, p = .58 (see Table S1). During the final 10 sessions of DE/IE training, there were no significant between-group differences in the number of trials completed, omissions, or response latencies, p’s > .22 (see Table S1).

In the impulsive-choice assessments, there were no main effects of Time or Group and no Time × Group interaction in the number of days to meet the amount-discrimination criteria, p’s > .18, or in the number of sessions required to meet the stability criteria, p’s > .30 (see Table S2). Figure 1 depicts mean percent LLR choice (± SEM) in the initial assessment (left panel) and the reassessment (right panel) of impulsive choice. The inset bar graphs show the mean (± SEM) and individual-subject AUC values. From test to retest, there was no main effect of Time and no Time × Group interaction, p = .57 and p = .34, respectively. However, a significant main effect of Group was observed, F(1, 9) = 27.28, p < .001. Post-hoc comparisons revealed significant between-group differences in AUC in the initial assessment, t(9) = 7.49, p < .0001; CL = .99, and the reassessment, t(9) = 3.30, p < .01; CL = .92. In addition, there was a strong, positive correlation between the initial and reassessment of AUC scores in the DE group, r = .91, p < .05, but not in the IE group, r = .40, p = .44. No significant main effects of Time or Group and no Time × Group interaction were observed in percent LLR choice for the amount-discrimination probe sessions, p’s > .34 (see Table S2). Finally, there were no main effects of Time or Group and no Time × Group interaction in the number of omissions, p’s > .29, or the latency to respond on forced- and free-choice SSR or LLR trials, p’s > .08 (see Table S2).

4.1 Discussion

The present research examined the longer-term effects of DE training in male Wistar rats. As in prior research conducted with Long Evans rats (Stein et al., 2013, 2015), Wistar rats randomly assigned to the DE group made significantly fewer impulsive choices than those assigned to the IE group. The Stein et al. (2013, 2015) findings were extended to show that DE-training effects may be observed 120 days after the initial impulsive-choice assessment (the longest prior test-retest interval was 66 days). It is noteworthy that 120 days of the Wistar rats’ life corresponds to approximately 11 human years when one considers the relative lifespans of these species (Hubrecht and Kirkwood, 2010); thus, the effects of DE training last a significant portion of the rats’ lifespan.

In the Stein et al. (2013, 2015) studies, rats were given the opportunity to consume alcohol during the test-retest interval. Rats in the present study consumed no alcohol; however, both groups completed an operant task for a portion of the test-retest interval. Because reductions in impulsive choice were observed at follow-up in spite of this intervening task, the present study, when combined with those of Stein et al., suggests that the effect of DE training on impulsive choice is robust to a variety of intervening events. Future research might further explore how robust the DE-training effect is by examining if it generalizes to other impulsive-choice assessments (e.g., adjusting-delay task; Mazur, 1987) or to impulsive-choice tasks arranged in different chambers or with different rewards. If DE training proves robust in these generalization tests, studies should be undertaken to adapt the training for use with, for example, pre-school children. Embedding DE training into a game played in pre-school classrooms might reduce impulsive choice in the game, in the classroom, and perhaps beyond.

Finally, it is noteworthy that the effect of DE training was more pronounced in this study than in prior reports (Stein et al., 2013, 2015). The reason for the larger effect size can only be speculated upon. Because the procedures are unchanged from past studies, the rat-strain difference (Wistars vs. Long Evans) may be responsible, a possibility to be evaluated in future studies.

4.2.1 Conclusion

The present study provides encouraging results for researchers interested in producing large, long-lasting reductions in impulsive choice that are robust to intervening experiences. Such an effect may provide a useful baseline against which future studies are conducted (e.g., effects of reducing impulsive choice on subsequent acquisition of cocaine self-administration; Perry et al., 2005).

Figure 2
Mean percent LLR choice in the initial assessment (left panel) and reassessment (right panel) of impulsive choice. Solid and dashed data paths represent the DE and IE group, respectively. The inset graphs represent mean and individual-subject AUC values ...


  • Extended training with delayed reinforcement decreases impulsive choice in rats.
  • This study replicates the delay-exposure effect with a second strain of rats.
  • Reduced levels of impulsive choice remained 120 days after the initial assessment.

Supplementary Material


This research was supported financially by a grant from the National Institutes of Health: 1R01DA029605, awarded to the last author (G. J. Madden). Neither author has any real or potential conflict(s) of interest, including financial, personal, or other relationships with organizations or pharmaceutical/biomedical companies that may inappropriately influence the research and interpretation of the findings. Both authors would like to thank Jay E. Hinnenkamp and Jillian M. Rung for their thoughtful discussions regarding the present study and for their assistance in conducting the experiment. In addition, we would like to thank Thomas Argyle, Cassandra Gardner, Jacob Goddard, Stephanie Lenzini, Courtney Nielsen, Jodi Siri, Spencer Steadman, Michael Williams, and Benjamin Yang for their help in conducting the experiment.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1CL effect size was calculated, as it is robust to normality violations (see McGraw and Wong, 1992); as applied to these data, CL effect size is the likelihood that a randomly sampled DE rat will make fewer impulsive choices than a randomly sampled IE rat (Lakens, 2013).


  • Albein-Urios N, Martinez-Gonzalez JM, Lozano O, Clark L, Verdejo-Garcia A. Comparison of impulsivity and working memory in cocaine addiction and pathological gambling: Implications for cocaine-induced neurotoxicity. Drug Alcohol Depend. 2012;126:1–6. [PubMed]
  • Alessi SM, Petry NM. Pathological gambling severity is associated with impulsivity in a delay discounting procedure. Behav Processes. 2003;64:345–354. [PubMed]
  • Bickel WK, Jarmolowicz DP, Mueller ET, Koffarnus MN, Gatchalian KM. Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. Pharmacol Ther. 2012;134:287–97. [PMC free article] [PubMed]
  • Bickel WK, Mueller ET. Toward the Study of Trans-Disease Processes: A Novel Approach With Special Reference to the Study of Co-morbidity. J Dual Diagn. 2009;5:131–138. [PMC free article] [PubMed]
  • Bickel WK, Quisenberry AJ, Moody L, Wilson AG. Therapeutic Opportunities for Self-Control Repair in Addiction and Related Disorders: Change and the Limits of Change in Trans-Disease Processes. Clin Psychol Sci. 2015;3:140–153. [PMC free article] [PubMed]
  • Bickel WK, Yi R, Landes RD, Hill PF, Baxter C. Remember the future: working memory training decreases delay discounting among stimulant addicts. Biol Psychiatry. 2011;69:260–5. [PMC free article] [PubMed]
  • Bradford WD. The association between individual time preferences and health maintenance habits. Med Decis Making. 2010;30:99–112. [PubMed]
  • Daugherty JR, Brase GL. Taking time to be healthy: Predicting health behaviors with delay discounting and time perspective. Personality and Individual Differences. 2010;48:202–207.
  • Evenden JL. Varieties of impulsivity. Psychopharmacology (Berl) 1999;146:348–61. [PubMed]
  • Evenden JL, Ryan CN. The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl) 1996;128:161–70. [PubMed]
  • Gray JC, MacKillop J. Impulsive delayed reward discounting as a genetically-influenced target for drug abuse prevention: a critical evaluation. Front Psychol. 2015;6:1104. [PMC free article] [PubMed]
  • Hubrecht RC, Kirkwood J. The UFAW handbook on the care and management of laboratory and other research animals. Wiley-Blackwell; Oxford: 2010.
  • Koffarnus MN, Jarmolowicz DP, Mueller ET, Bickel WK. Changing delay discounting in the light of the competing neurobehavioral decision systems theory: a review. J Exp Anal Behav. 2013;99:32–57. [PMC free article] [PubMed]
  • Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013;4 [PMC free article] [PubMed]
  • Landes RD, Christensen DR, Bickel WK. Delay discounting decreases in those completing treatment for opioid dependence. Exp Clin Psychopharmacol. 2012;20:302–9. [PMC free article] [PubMed]
  • Lin H, Epstein LH. Living in the moment: effects of time perspective and emotional valence of episodic thinking on delay discounting. Behav Neurosci. 2014;128:12–9. [PMC free article] [PubMed]
  • MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, Munafo MR. Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology (Berl) 2011;216:305–21. [PMC free article] [PubMed]
  • Madden GJ, Johnson PS. A Delay-discounting Primer. In: Madden GJ, Bickel WK, editors. Impulsivity: The behavioral and neurological science of discounting. American Psychological Association; Washington, DC, US: 2010.
  • Madden GJ, Petry NM, Badger GJ, Bickel WK. Impulsive and self-control choices in opioid-dependent patients and non-drug-using control participants: Drug and monetary rewards. Experimental and Clinical Psychopharmacology. 1997;5:256–262. [PubMed]
  • Mazur JE. An adjusting procedure for studying delayed reinforcement. In: Commons ML, Mazur JE, Nevin JA, Rachlin H, editors. Quantitative analysis of behavior. Erlbaum; Hillsdale, NJ: 1987.
  • Mazur JE, Logue AW. Choice in a “self-control” paradigm: effects of a fading procedure. J Exp Anal Behav. 1978;30:11–7. [PMC free article] [PubMed]
  • McGraw KO, Wong SP. A common language effect size statistic. Psycho Bull. 1992;111:361–365.
  • Odum AL. Delay discounting: I’m a k, you’re a k. J Exp Anal Behav. 2011;96:427–39. [PMC free article] [PubMed]
  • Perry JL, Larson EB, German JP, Madden GJ, Carroll ME. Impulsivity (delay discounting) as a predictor of acquisition of IV cocaine self-administration in female rats. Psychopharmacology (Berl) 2005;178:193–201. [PubMed]
  • Peters J, Büchel C. Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neuron. 2010;66:138–48. [PubMed]
  • Petry NM. Pathological gamblers, with and without substance use disorders, discount delayed rewards at high rates. J Abnorm Psychol. 2001;110:482–7. [PubMed]
  • Stein JS, Johnson PS, Renda CR, Smits RR, Liston KJ, Shahan TA, Madden GJ. Early and prolonged exposure to reward delay: effects on impulsive choice and alcohol self-administration in male rats. Exp Clin Psychopharmacol. 2013;21:172–80. [PMC free article] [PubMed]
  • Stein JS, Madden GJ. Delay discounting and drug abuse: empirical, conceptual, and methodological considerations. In: MacKillop J, de Wit H, editors. The Wiley-Blackwell handbook of addiction psychopharmacology. Wiley-Blackwell; Oxford, UK: 2013.
  • Stein JS, Renda CR, Hinnenkamp JE, Madden GJ. Impulsive Choice, Alcohol Consumption, and Pre-Exposure to Delayed Rewards: II. Potential Mechanisms. Journal of the Experimental Analysis of Behavior 2015 [PMC free article] [PubMed]
  • Yi R, Johnson MW, Giordano LA, Landes RD, Badger GJ, Bickel WK. The Effects of Reduced Cigarette Smoking on Discounting Future Rewards: An Initial Evaluation. Psychol Rec. 2008;58:163–174. [PMC free article] [PubMed]