PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Neuropsychopharmacology. Author manuscript; available in PMC Mar 1, 2010.
Published in final edited form as:
PMCID: PMC2726909
NIHMSID: NIHMS111020
Balancing Risk and Reward: A Rat Model of Risky Decision-Making
Nicholas W. Simon, MS,1 Ryan J. Gilbert, BS,1 Jeffrey D. Mayse, BS,1 Jennifer L. Bizon, PhD,1,2 and Barry Setlow, PhD1,2
1Department of Psychology, Texas A&M University, College Station, TX 77843-4235
2Faculty of Neuroscience, Texas A&M University, College Station, TX 77843-4235
Address Correspondence to: Dr. Barry Setlow, Behavioral and Cellular Neuroscience Program, Department of Psychology, Texas A&M University, College Station, TX, 77843-4235, office: 979-845-2507, fax: 979-845-4727, Email: bsetlow/at/tamu.edu
We developed a behavioral task in rats to assess the influence of risk of punishment on decision-making. Male Long-Evans rats were given choices between pressing a lever to obtain a small, “safe” food reward and a large food reward associated with risk of punishment (footshock). Each test session consisted of 5 blocks of 10 choice trials, with punishment risk increasing with each consecutive block (0, 25, 50, 75, 100%). Preference for the large, “risky” reward declined with both increased probability and increased magnitude of punishment, and reward choice was not affected by the level of satiation or the order of risk presentation. Performance in this risky decision-making task was correlated with the degree to which the rats discounted the value of probabilistic rewards, but not delayed rewards. Finally, the acute effects of different doses of amphetamine and cocaine on risky decision-making were assessed. Systemic amphetamine administration caused a dose-dependent decrease in choice of the large risky reward (i.e. – it made rats more risk-averse). Cocaine did not cause a shift in reward choice, but instead impaired rats’ sensitivity to changes in punishment risk. These results should prove useful for investigating neuropsychiatric disorders in which risk taking is a prominent feature, such as attention deficit/hyperactivity disorder and addiction.
Keywords: risk, decision-making, choice, reward, punishment, amphetamine, cocaine
Few decisions with which individuals are faced on a daily basis are entirely without risk of adverse consequences. Even everyday decisions such as “what to do when faced with a yellow traffic signal while driving” (accelerate or slow down) involve consideration of both the risks and rewards associated with each option. A better understanding of this type of decision-making under conditions in which highly rewarding choices are accompanied by risks of adverse consequences (punishment) may have considerable implications for understanding psychopathological conditions characterized by alterations in risk-based decision-making, such as attention deficit/hyperactivity disorder, major depressive disorder, schizophrenia, and addiction (Bechara et al., 2001; Drechsler et al., 2008; Ernst et al., 2003; Heerey et al., 2008; Taylor Tavares et al., 2007). Such risky decision-making is commonly studied in human subjects (Lejuez et al., 2002; Leland and Paulus, 2005); however, there are few animal models that systematically assess the degree to which risk of punishment (rather than reward omission) influences reward-based choice (see Negus, 2005).
There were three main goals of the experiments described below: the first was to establish the performance parameters of a discrete-trials risky decision-making choice task, in which rats chose between a small, “safe” reward and a large reward associated with a risk of punishment that varied within each test session. The second goal was to determine how choice performance in the risky decision-making task was related to performance in other decision-making tasks that assess the degree to which choices are influenced by reward probability and delays to reward delivery (and which are also commonly altered in the same neuropsychiatric conditions in which altered risky decision-making is observed). The third goal was to determine how risky decision-making is affected by acute administration of psychostimulant drugs that have been shown previously to affect delay- and probability-based decision-making in both human and animal subjects (Cardinal et al., 2000; De Wit et al., 2002; DeVito et al., 2008; Evenden and Ryan, 1996; St Onge and Floresco, 2009; Stanis et al., 2008; Winstanley et al., 2007).
Subjects
Male Long-Evans rats (n=28, Charles River Laboratories, Raleigh, NC weighing 275–300 g upon arrival) were individually housed and kept on a 12 hour light/dark cycle (lights on at 0800) with free access to food and water except as noted. During testing, rats were maintained at 85% of their free-feeding weight, with allowances for growth. All procedures were conducted in accordance with the Texas A&M University Laboratory Animal Care and Use Committee and NIH guidelines.
Apparatus
Testing took place in standard behavioral test chambers (Coulbourn Instruments, Whitehall, PA) housed within sound attenuating cubicles. Each chamber was equipped with a recessed food pellet delivery trough fitted with a photobeam to detect head entries and a 1.12 W lamp to illuminate the food trough. The trough, into which 45 mg grain-based food pellets (PJAI, Test Diet: Richmond, IN) were delivered, was located 2 cm above the floor in the center of the front wall. Two retractable levers were located to the left and right of the food delivery trough, 11 cm above the floor. A 1.12-W house light was mounted on the rear wall of the sound-attenuating cubicle. The floor of the test chamber was composed of steel rods connected to a shock generator (Coulbourn) that delivered scrambled footshocks. Locomotor activity was assessed throughout each session with an overhead infrared activity monitor. Test chambers were interfaced with a computer running Graphic State software (Coulbourn), which controlled task event delivery and data collection.
Behavioral Procedures
Shaping
Shaping procedures were identical to those used previously (Simon et al., 2007b). Following magazine training, rats were trained to press a single lever (either left or right, counterbalanced across groups; the other was retracted during this phase of training) to receive a single food pellet. After reaching a criterion of 50 lever presses in 30 minutes, rats were shaped to press the opposite lever under the same criterion. This was followed by further shaping sessions in which both levers were retracted and rats were shaped to nose-poke into the food trough during simultaneous illumination of the trough and house lights. When a nose-poke occurred, a single lever was extended (left or right), and a lever press resulted in immediate delivery of a single food pellet. Immediately following the lever press, the house and trough lights were extinguished and the lever was retracted. Rats were trained to a criterion of at least 30 presses of each lever in 60 minutes. This shaping procedure was conducted only once at the start of all behavioral testing.
Risky Decision-Making Task
Test sessions were 60 minutes long and consisted of five blocks of 18 trials each. Each 40 s trial began with a 10 s illumination of the food trough and house lights. A nose poke into the food trough during this time extinguished the food trough light and triggered extension of either a single lever (forced choice trials) or of both levers simultaneously (choice trials). If the rats failed to nosepoke within the 10 s time window, the lights were extinguished and the trial was scored as an omission.
A press on one lever (either left or right, balanced across animals) resulted in one food pellet (the small, safe reward) delivered immediately following the lever press. A press on the other lever resulted in immediate delivery of three food pellets (the large reward). However, selection of this lever was also accompanied immediately by a possible 1 s footshock contingent on a preset probability specific to each trial block. The large food pellet reward was delivered following every choice of the large reward lever, regardless of whether or not the footshock occurred. The intensity of the footshock varied by experiment (see below). With the exception of Experiment 1C, the probability of footshock accompanying the large reward was set at 0% during the first 18-trial block. In subsequent 18-trial blocks, the probability of footshock increased to 25, 50, 75, and 100%. Each 18-trial block began with 8 forced choice trials used to establish the punishment contingencies (4 for each lever), followed by 10 choice trials (Cardinal and Howes, 2005; Simon et al., 2007b; St Onge and Floresco, 2009). Once either lever was pressed, both levers were immediately retracted. Food delivery was accompanied by re-illumination of both the food trough and house lights, which were extinguished upon entry to the food trough to collect the food or after 10 s, whichever occurred sooner.
Delay Discounting Task
A detailed description of this procedure is provided in Simon et al. (2007b). Each 100 minute session consisted of five blocks of 12 trials each. Each 100 s trial began with illumination of the food trough and house lights. A nose-poke into the food trough during this time extinguished the food trough light and triggered extension of either a single lever (forced choice trials) or of both levers simultaneously (choice trials). Trials on which rats failed to nose-poke during this window were scored as omissions.
Each block consisted of 2 forced choice trials followed by 10 choice trials. A press on one lever (either left or right, counterbalanced across subjects) resulted in one food pellet delivered immediately. A press on the other lever resulted in four food pellets delivered after a variable delay. Once either lever was pressed, both levers were retracted for the remainder of the trial. The delay duration increased with each block of trials (0s, 10s, 20s, 40s, 60s) (Cardinal et al., 2000; Evenden and Ryan, 1996; Simon et al., 2007b; Winstanley et al., 2003).
Probability Discounting Task
The parameters of this task were identical to the risky decision-making task, with the only difference following selection of the large reward lever. During the first block of trials, the large reward was delivered with 100% probability. During each of the four subsequent blocks, the probability of large reward delivery was systematically decreased (75, 50, 25, 0%). The large reward was accompanied by neither punishment nor a delay period.
Experiment 1: Establishing the risky decision-making task
Experiment 1A: Effects of Shock Intensity
To determine the optimal shock intensity for subsequent experiments, rats were divided into 3 groups and tested on the risky decision-making task for 20 sessions at shock intensities of 0.35 mA (n = 12), 0.4 mA (n=10), or 0.45 mA (n=6). Percent choice of the large reward lever was averaged across the final five days of testing, during which performance was stable.
Experiment 1B: Effects of Food Satiation Level
To assess the effects of alterations in food motivation on performance, rats trained at the 0.35 mA shock intensity were tested after being given access to freely-available food in their home cage for either one or 24 hours immediately prior to testing. This testing occurred over four days for each satiation level, with rats food restricted as normal (85% of free-feeding weight) on days 1 and 3, and with access to food on days 2 and 4. For each satiation level, the two satiation days and the two non-satiation days were averaged together for analysis.
Experiment 1C: Reversal of Punishment Probabilities
To determine whether task performance was specific to ascending risks of punishment, rats trained with a shock intensity of 0.35 mA were tested for 10 sessions in a modified version of the task in which the order of risk presentations accompanying the large reward was reversed (100% in the first block of trials, followed by 75, 50, 25, and 0%). All other aspects of the task remained constant. Choice behavior was averaged across the final five sessions.
Experiment 2: Comparing Decision-Making across Tasks
Following testing in the risky decision-making task, rats tested with a shock intensity of 0.35 mA were tested in the delay discounting task for 20 sessions, followed by testing in the probability discounting task for 15 sessions. In each task, performance was assessed by averaging across the final five sessions (during which performance was stable).
Experiment 3: Acute Drug Treatments
The acute effects of d-amphetamine sulfate and cocaine hydrochloride on risky decision-making were examined in rats tested at a shock intensity of 0.35 mA. Rats were tested following i.p. injections (1 ml/kg) of one of three doses of d-amphetamine sulfate (Sigma, St Louis, MO; 0.33, 1.0, 1.5 mg/kg) or 0.9% saline vehicle. Injections were administered prior to testing over a period of 6 days using the following schedule: saline, amphetamine dose 1, saline, amphetamine dose 2, saline, amphetamine dose 3. The order of drug doses was counterbalanced across subjects. This 6-day experimental procedure was later repeated (after stable performance was obtained) using cocaine hydrochloride (Drug Supply Program, NIDA; 5, 10, 15 mg/kg in 0.9% saline vehicle). All drug treatments were administered in the vivarium; between transportation to the test chambers and the initial five minutes of forced-choice trials, 10 minutes elapsed between drug administration and collection of choice preference data in the first block of trials.
Timeline of experiments
Rats in the .35 mA shock intensity group from Experiment 1 were the subjects in Experiment 2 and Experiment 3. The progression of tasks and treatments was as follows: Risky decision-making (RDM) / amphetamine treatment / baseline RDM / delay discounting / probability discounting / baseline RDM / cocaine treatment / baseline RDM / RDM satiation tests / baseline RDM / RDM reversed probabilities.
Data Analysis
Raw data files were exported from Graphic State software and compiled using a custom macro written for Microsoft Excel (Dr. Jonathan Lifshitz, University of Kentucky). Statistical analyses were conducted in SPSS 16.0. Stable behavior was defined by the absence of either a main effect of session or an interaction between session and trial block in a repeated measures ANOVA over a 5 session period (Cardinal et al., 2000; Simon et al., 2008b; Winstanley et al., 2006b). The effects of behavioral or pharmacological manipulations in all tasks were assessed using two way ANOVAs, with trial block (i.e. – level of risk) as a repeated measures variable. Performance between tasks was compared using bivariate Pearson’s correlations. In all cases, p values less than .05 were considered significant.
Experiment 1: Establishing the risky decision-making task
Experiment 1A: Effects of Shock Intensity
Performance was stable for rats in all three shock intensity groups in sessions 16–20. A two-way repeated measures ANOVA (shock intensity [0.35, 0.40, 0.45 mA] × risk [0, 25, 50, 75, 100%]) revealed main effects of both shock intensity (F(2,24) = 12.31, p <.001) and risk (F(4,96) = 14.75, p<.001), with choice of the large reward decreasing with both shock intensity and risk in a combined analysis of all three groups (Figure 1). There was no interaction between shock intensity groups (F(8,96) = 1.75, n.s.); however, LSD post hoc analyses revealed significant differences in reward choice between the three groups (ps <.01). Additional planned one-way repeated measures ANOVAs showed main effects of punishment risk in rats trained with the 0.35 mA shock (F(4,44) = 7.20, p <.001) and the 0.4 mA shock (F(4,32) = 12.18 p <.001), indicating that these groups discounted the large reward as a function of risk (i.e. – they were sensitive to risk of punishment). There was no effect of punishment risk for the 0.45 mA shock intensity (F(4,20), = 1.00, n.s.), with this group demonstrating almost complete preference for the small reward across all blocks (Figure 1).
Figure 1
Figure 1
Effects of shock intensity on reward choice in the risky decision-making task. Groups demonstrated differences in reward preference based on shock intensity, with the higher intensity groups preferring the small, safe reward.
Additionally, there was no main effect of shock intensity on the number of choice trials completed during testing (mean % completed trials, 0.35 mA, 97.00; 0.40 mA, 96.62, 0.45 mA, 99.93; F(2,24) = 1.52, n.s.), although there was a main effect of shock intensity on trials completed during forced choice trials with the large lever (with no option for the small, safe reward) when the punishment risk was 100% (mean % completed trials: 0.35 mA: 75.00, 0.40 mA: 53.89, 0.45 mA: 5.83; F(2,24) = 14.54, p <.001). LSD post hoc analyses conducted on these latter data revealed no difference in completed trials between the 0.35 and 0.40 mA groups (n.s.) but strong group differences between the 0.35 and 0.40 groups and the 0.45 mA group (ps < .001), indicating that rats would often forego selection of the large reward altogether at the high shock intensity when there was no safe reward option.
There was considerable variance in reward preference within both the 0.35 and 0.4 mA shock groups, such that some rats demonstrated a strong preference for the large, risky reward whereas others preferred the small, safe reward (Figure 2). In order to analyze the source of this variance, linear regressions were used to assess the ability of initial reward preference (during the first, 0% risk block) to predict reward choice averaged across the other four blocks. Reward choice during the 0% block was correlated with choice during the next four blocks for both the .35 (r = .86, p < .001) and .40 mA (r = .82, p < .01) shock intensity groups. For the 0.35 mA shock group, there were no correlations between reward preference (risk-taking) and baseline body weight (r = .33, n.s.) or shock reactivity as assessed by locomotion during the 1 second shock delivery period (r = −.15, n.s.). For the 0.4 mA shock group, there was a strong correlation between baseline body weight and reward preference (r = .82, p <.05) but not between reward preference and shock reactivity (r = −.13, n.s.) or body weight and shock reactivity (r = −.08, n.s.).
Figure 2
Figure 2
Individual variability in risky decision-making. There was a wide distribution of reward preference in rats in both the 0.35 and 0.4 mA shock intensity groups. Each curve represents data from a single subject.
Based on the data from Experiment 1A, the 0.35 mA shock intensity was used for all subsequent experiments.
Experiment 1B: Effects of Food Satiation Level
For the 1 hour satiation condition (Figure 3A), a two-way, repeated measures ANOVA showed a main effect of punishment risk (F(4,40)= 7.72, p <.001) but no main effect of satiation (F(1,40) = .68, n.s.), indicating that 1 hour of free feeding prior to testing did not influence choice behavior. There was also no effect of 1 hour satiation on the number of trials omitted during testing (F(1,10) = 1.61, n.s.).
Figure 3
Figure 3
Effects of satiation on reward choice in the risky decision-making task. Neither one nor 24 hours of free-feeding prior to testing affected reward choice.
The effects of 24 hours of free-feeding on choice performance were similar, in that there was a main effect of punishment risk (F(4,36) = 5.82, p <.01) but no main effect of satiation level (F(1,9) = 3.19, n.s.), indicating that even substantial satiation caused no change in reward choice (Figure 3b). However, rats completed significantly fewer trials after 24 hours of free-feeding than under food-restricted conditions (mean percentage of completed trials; satiated, 67.80, vs. restricted, 79.50; F(1,9) = 9.48, p <.05).
Experiment 1C: Reversal of Punishment Probabilities
After the order of presentation of punishment risks was reversed, there was still a main effect of risk (F(4,36) = 7.59, p <.001), with rats showing less preference for the large reward under conditions of greater risk. These data were consistent with data from Experiment 1A, as a within subjects comparison of performance under ascending and descending orders of risk presentation revealed no main effects or interactions involving the order of risk presentation (Fs < 1.45, n.s.) (Figure 4).
Figure 4
Figure 4
Effects of reversal of punishment risks. After the order of risk presentations was reversed, rats continued to demonstrate discounting of the large risky reward in a manner similar to that under ascending risk presentations. The ascending risk data are (more ...)
Experiment 2: Comparing Decision-Making Across Tasks
Risky Decision-Making Task
Rats tested at 0.35 mA in Experiment 1 were re-tested in the risky decision-making task (using an ascending order of punishment risk) until behavior was stable. There was a main effect of risk (F(4,44) = 4.72, p <.01), indicating that rats discounted the value of the large reward as a function of risk of punishment (Figure 5a). Moreover, rats’ performance in this session was highly correlated with their previous performance in Experiment 1, indicating that risky decision-making is stable across time (r = .92, p <.001).
Figure 5
Figure 5
Comparison of the risky decision-making task with other decision-making tasks. a. Performance on the risky decision-making task with 0.35 mA shock intensity. b. Performance on the delay discounting task. c. Performance on the probability discounting task. (more ...)
Delay Discounting Task
Behavior was stable across sessions 16–20. There was a main effect of delay during these sessions (F(4,44) = 32.36, p <.001), indicating that rats discounted the large reward as a function of delay duration (Figure 5b).
Probability Discounting Task
Behavior was stable across sessions 11–15. There was a main effect of reward probability during these sessions (F(4,44) = 37.61, p <.001), indicating that rats discounted the large reward as a function of the probability of its delivery (Figure 5c).
Relationships between Tasks
A Pearson’s correlation test revealed that choice of the large, risky reward in the risky decision-making task was not correlated with choice of the large, delayed reward in the delay discounting task (r = .23, n.s.). There was also no correlation between reward choice in the delay discounting and probability discounting tasks (r = .23, n.s.). However, there was a significant correlation in reward choice between the risky decision-making and probability discounting tasks (Figure 5, insets; r = .59, p <.05).
Experiment 3: Acute Drug Treatments
Amphetamine
There was no difference in performance across the three days of saline injections (F(2,22) = 2.53, n.s.), so the mean of these days was used in the analysis. A two-way repeated measures ANOVA (risk × drug dose) revealed a main effect of punishment risk (F(4,44) = 13.52, p <.001), indicating that, across doses, rats decreased their choice of the large reward with increasing risk (Figure 6a). Most importantly, there was a main effect of drug dose (F(3,33) = 3.87, p <.05) such that rats became more risk-averse with increasing doses of amphetamine (although the risk × drug interaction did not quite reach significance [F(12,132) = 1.80, p =.055]). Individual pair-wise comparisons between saline and amphetamine conditions showed that the 1.5 mg/kg dose caused a significant decrease in preference for the large reward (p < .05). In addition to its effects on reward choice, amphetamine also increased the number of omitted trials (F(3,33) = 3.92, p < .05), with omissions increasing as a function of dose (% completed choice trials: saline=98.66, .33 mg/kg=94.00, 1.0 mg/kg=94.34, 1.5 mg/kg=84.00). However, the effects of amphetamine on omissions appeared to be separate from its effects on reward choice, as there were no correlations between these two variables (rs <.35, ns). There was also no difference in shock reactivity (locomotion during the 1 s shock presentations) across drug doses (F(3,21) = .12, n.s.), indicating that the observed behavioral changes were likely not a result of amphetamine-induced alterations in shock sensitivity.
Figure 6
Figure 6
Effects of pharmacological treatments on risky decision-making. a. Rats were tested under the influence of systemic 0.33, 1.0, and 1.5 mg/kg doses of amphetamine. Amphetamine decreased preference for the large risky reward in a dose-dependent fashion, (more ...)
Cocaine
There were no differences between the three days of saline injections F(2,22) = 2.94, n.s.), so these days were averaged for the analysis. A two-way repeated measures ANOVA revealed a main effect of punishment risk when all doses were combined into a single analysis (F(4,44) = 4.00, p < .05) (Figure 6b). There was no main effect of drug treatment on reward choice (F(3,33) = .39, n.s.); however there was an interaction between drug dose and punishment risk (F(12,132) = 2.26, p < .05). The nature of this interaction was further investigated by performing one-way repeated measures ANOVAs assessing the effects of punishment risk on each drug dose individually. Simple main effects of punishment risk were evident under both saline and the 10 mg/kg dose of cocaine, such that the rats discounted the large reward as a function of risk (ps <.001). However, there was no main effect of risk when rats were given either the 5 mg/kg (F(4,44) = 1.79, n.s.) or 15 mg/kg (F(4,44) = .50, n.s.) doses of cocaine, indicating that risk of punishment failed to influence reward choice at these doses. Because reward choice during the first block (0% footshock accompanying large reward) appeared to differ between groups, a repeated measures ANOVA was performed between doses for the first block only. There was no main effect of dose (F(3,33) = 2.34, n.s.), and paired t-tests between saline and each dose individually revealed no differences in reward preference (n.s.), although the difference between saline and 5 mg/kg conditions approached significance (p = .07). There was no main effect of drug dose on the number of trials omitted (F(3,33) = 1.77, n.s.), and no main effect of drug dose on shock reactivity (F(3,9) = .32, n.s.). There were also no correlations between reward choice and shock reactivity for any doses (rs < .61, n.s.).
We developed a task that assessed the degree to which risk of punishment (footshock) influenced reward choice. This risky decision-making task differs from previous conflict paradigms such as the Geller-Seifter conflict and the thirsty-rat conflict tasks (File et al., 2003) in that 1.) rats are given a choice between the potentially punished response and a second, safe alternative response, and 2.) the risky response was only accompanied by punishment according to a specific probability that shifted within each session. We found that rats reliably shifted preference from a large, risky reward to a small, safe reward as the risk of punishment accompanying the large reward increased. This preference was mediated by the magnitude of the punishment, as preference for the small safe reward increased with greater shock intensity. Importantly, rats were able to recognize changes in punishment risk within sessions, as reward preference shifted as risk was altered. During initial testing, the risky reward began with 0% probability of shock and systematically ascended to 100% probability. It may be argued that the performance curves observed were a result of a lack of motivation due to satiation or frustration as trial blocks progressed. However, reversing the order of risk presentations did not alter performance (i.e., rats continued to show increased preference for the small safe reward with greater risks of punishment) and thus it is unlikely that the order of risk presentations was the cause of the observed pattern of reward preference.
Performance in the risky decision-making task showed large between-subjects variability (at least at lower shock intensities), but was stable across multiple test sessions over the course of several months and was not related to differences in body weight or shock reactivity in rats trained at the 0.35 mA shock intensity. In rats trained at the 0.4 mA shock intensity only, there was a correlation between body weight and risk-taking behavior such that heavier rats preferred the large, risky reward. This may have been due to the differences in body weight altering the experience of the footshock (e.g., the shock was less aversive to heavier rats), although there was no correlation between weight and shock reactivity in the 0.4 mA group. It is also possible that rats with higher baseline weights were simply more highly motivated to obtain food rather than less influenced by the shock; however, this alternative explanation also seems unlikely, as rats showed no shift in reward preference after a 24-hour satiation period (which caused an increase in body weight). This latter finding was somewhat surprising, as satiation can reduce food’s motivational value, resulting in reduced choice of the devalued food (Johnson et al., 2009). While we did observe an increase in overall omitted trials after satiation (indicating the effectiveness of this procedure), satiation did not significantly affect preference for the large, risky reward. This finding could indicate that choice behavior in this task (but not overall responding) is only minimally controlled by reinforcer value. Alternatively, it is possible that the long duration of testing experienced by the rats by that point in the experiment resulted in choice behavior being mediated by “stimulus-response” - type mechanisms (and thus less controlled by reinforcer value (Balleine and Dickinson, 1998)).
Further analysis of the individual variability demonstrated that risky decision-making performance was related to each subject’s selection of the large reward during the initial block, even though the reward was accompanied by a 0% probability of footshock during this block. Despite this observation, the patterns of decision-making observed here likely were not solely a function of response perseveration from baseline levels of responding. Rats were able to adjust their baseline responding throughout the 20 days of training (such that the relationship between baseline responding and overall responding shifted throughout training) (Simon et al., unpublished observations). Additionally, a separate experiment showed that rats were able to adjust responding when the shock intensity was increased (Simon et al., 2008a). Although the fact that some rats failed to choose the large reward even under 0% risk conditions is somewhat surprising, it can likely be accounted for by “carryover” effects from the previous day’s training (i.e. – because in the final block of trials, they were always shocked when choosing the large reward, this experience likely biased their choices on the following day). In support of this possibility, choice of the large reward in the 0% risk block varied directly with shock intensity (Figure 1), even though no shocks were received in this block.
The individual differences in risky decision-making observed in this task may mimic the diversity in propensity for risk-taking observed in human subjects (DeVito et al., 2008; Gianotti et al., 2009; Lejuez et al., 2003; Reyna and Farley, 2006; Sobanski et al., 2008; Taylor Tavares et al., 2007; Weber et al., 2004) but, importantly, the use of an animal model allows a degree of experimental control that is not possible in human studies. Thus, any behavioral differences observed are more likely due to intrinsic rather than experiential factors. This variability should prove useful in future studies for identifying behavioral and neurobiological correlates of risky decision-making.
Risky decision-making behavior was compared to behavior in two other reward-related decision-making tasks: delay discounting (commonly used to measure impulsive choice (Ainslie, 1975; Evenden and Ryan, 1996; Simon et al., 2007b; Winstanley et al., 2006a)) and probability discounting (characterized as an assessment of risky behavior (Cardinal and Howes, 2005; St Onge and Floresco, 2009)). Correlational evidence suggests that rats with a preference for the large, risky reward in the risky decision-making task also demonstrate preference for the large reward in the probability discounting task (preference for the large, probabilistic reward over the small, certain reward). These data suggest that either the assessment of probabilities (of punishment and reward omission, respectively) or the integration of probabilistic information with reward value may be mediated by similar neurobiological mechanisms. Conversely, rats with a greater propensity for risky choice did not consistently demonstrate greater impulsive choice in the delay discounting task (preference for the small, immediate reward over the large, delayed reward). Although this runs counter to some theoretical and experimental data suggesting similarities between the influence of delay and probability on reward value (see Hayden and Platt, 2007; Yi et al., 2006), other findings suggest that integration of delays with reward value requires a different set of neural substrates than probability assessment (Cardinal, 2006; Floresco et al., 2008; Kobayashi and Schultz, 2008; Mobini et al., 2000; Schultz et al., 2008). It could be argued that the correlation between performance in the risky decision-making and probability discounting tasks was a result of response perseveration, as both tasks used the same levers to produce the small and large rewards. However, rats were tested in the delay discounting task after the risky decision-making task, and demonstrated a sizable shift in behavior. Rats then shifted their behavior again when tested in the probability discounting task. Were perseveration the only explanation for the similarity between the risky decision-making and probability discounting tasks, the performance curves would be expected to follow similar trends for all three tasks.
Systemic amphetamine administration produced a dose-dependent increase in risk aversion, shifting rats’ preference toward the “safe” reward. It is possible that this shift in behavior, which led to an overall reduction in food consumption, was a result of amphetamine-induced suppression of food intake (Wellman et al., 2008). However, neither 1- nor 24-hour periods of free feeding prior to testing had an effect on reward choice, although 24-hour free-feeding did increase the number of trials omitted (an effect that was also observed under amphetamine). Additionally, when acute amphetamine was tested in the delay discounting task, reward preference was shifted in the opposite direction, toward greater choice of the large, delayed reward, resulting in greater food consumption (Simon et al., 2007a). Thus, it seems unlikely that amphetamine altered reward choice simply by altering hunger levels or food motivation. Another possibility is that the increased preference for the small, safe reward induced by amphetamine was a result of hypersensitivity to footshock. This explanation seems unlikely for two reasons: first, amphetamine has been characterized as an analgesic agent (Connor et al., 2000; Drago et al., 1984). If pain sensitivity were indeed the critical mediator of reward selection in this task, rats given amphetamine would be expected to find the shock less aversive and shift their preference toward the large, risky reward as a result of a higher pain threshold. Second, amphetamine did not alter locomotion during the footshock, which can be used as a behavioral marker for pain/shock sensitivity (Chhatwal et al., 2004).
Interestingly, results similar to those found here with amphetamine have been obtained in human subjects with various psychopathological disorders. Children with ADHD and patients with fronto-temporal dementia show reduced risky choices in the Cambridge gambling task when treated with methylphenidate, a monoamine reuptake inhibitor with effects similar to amphetamine (DeVito et al., 2008; Rahman et al., 2005). As methylphenidate is thought primarily to affect decision-making through actions on prefrontal cortex (Berridge et al., 2006), a structure that has been implicated in risky decision-making (Bechara et al., 2000; Clark et al., 2008; St. Onge and Floresco, 2008), it is possible that alterations in prefrontal cortex activity are responsible for the changes in risk-taking behavior observed after administration of amphetamine (in rats) or methylphenidate (in humans).
The amphetamine-induced decrease in risk-taking behavior observed in this study contrasts with the increased risk-taking behavior in rats tested in a probability discounting task observed by St. Onge & Floresco (2009). Although both tasks involve assessment of probabilities (indeed, we observed that reward choice was correlated between these two tasks), it is possible that amphetamine affects these types of decision-making in different ways. The risky decision-making task utilized in the current study used probabilities of punishment rather than reward omission as the discounting factor associated with the large reward. The difference in amphetamine’s effects may be a result of dopaminergic mediation of aversive states induced by expectation of footshock. The same mesolimbic dopaminergic structures implicated in reward (such as nucleus accumbens and ventral tegmental area) also appear to be involved with emotional reactions to aversive stimuli (Carlezon and Thomas, 2009; Liu et al., 2008; Setlow et al., 2003). Thus, it is possible that amphetamine-induced enhancements in dopamine transmission increase the ability of aversive stimuli to control behavior (rather than solely enhancing the influence of rewarding stimuli), which could explain the amphetamine-induced shift in reward choice away from the large, risky reward. This explanation is consistent with previous findings showing that acute amphetamine administration at doses similar to those used here increased the degree to which rats avoided making a response that produced an aversive conditioned stimulus previously associated with footshock (i.e. – amphetamine increased control over responding by the aversive conditioned stimulus (Killcross et al., 1997)).
Somewhat surprisingly, cocaine administration did not affect risky decision-making in the same manner as amphetamine. Subjects given cocaine at relatively high doses, although not high enough to confound performance with excessive stereotypy (Wellman et al., 2002), no longer demonstrated a shift in reward choice with increasing risk of punishment. This may be a result of a cocaine-induced enhancement in response perseveration (i.e., an inability to shift choice from the large reward to the smaller reward across the course of the session). Indeed, enhancements in perseverative behavior have been observed in human cocaine but not amphetamine abusers (Ersche et al., 2008). Interestingly, the dissociation between the effects of amphetamine and cocaine may be due in part to cocaine’s relatively higher affinity for the serotonin (5-HT) transporter (White and Kalivas, 1998). It has been suggested that 5-HT signaling may be critically involved in prediction of punishment (Daw et al., 2002). As acute depletion of the 5-HT precursor tryptophan enhances predictions of punishment in human subjects (Cools et al., 2008), it is possible that enhancements in 5-HT neurotransmission by cocaine might impair such predictions, resulting in apparent insensitivity to risk of punishment.
Another possibility is that the previous exposure to amphetamine influenced the subjects’ response to subsequent acute cocaine administration. A previous regimen of chronic cocaine administration can produce tolerance to cocaine’s acute effects on decision-making (Winstanley et al., 2007), although chronic amphetamine fails to influence the acute effects of amphetamine in a similar manner (Stanis et al., 2008). While this possibility cannot be entirely ruled out, it seems unlikely for behavioral tolerance to manifest itself given the short regimen of amphetamine administered to the subjects (three injections of ≤ 1.5 mg/kg across six days), as tolerance to the effects of psychostimulants on cognition has only been demonstrated with considerably higher doses and much longer regimens (Dalley et al., 2005; Simon et al., 2007a; Winstanley et al., 2007).
An interesting aspect of performance during this experiment is the discrepancy in reward choice between cocaine- and saline-exposed trials during the first block (0% risk). While there were no statistically significant differences between treatments, the lowest dose of cocaine caused a near-significant reduction in selection of the large reward during this block. This maladaptive shift in decision-making could be a result of an impaired ability to discriminate between the response levers, perhaps due to the anxiogenic properties of acute cocaine (Goeders, 1997).
Elevated risk-taking is characteristic of many psychopathological disorders, and can lead to persisting financial, social, and medical problems. A better understanding of the behavioral and neural substrates underlying risky decision-making will allow more efficacious treatment of patients affected adversely by excessive risk-taking. The risky decision-making task described here offers a novel method of assessing the role of punishment risk in decision-making. Given the large between-subjects variability and high test-retest reliability, this task may have great utility as a model of human risk-taking behavior, and for further investigation of its neurobiological substrates.
Acknowledgements
The authors thank Bradley Crain for technical assistance and Drs. Jim Grau, Brian Doss, and Aaron Taylor for statistical consulting and helpful comments on the manuscript. Supported by the Department of Psychology at Texas A&M University and NIH DA018764 (BS), NIH AG029421 (JLB) and NIH DA023331 (NWS).
Footnotes
Disclosure/Conflicts of Interest
The authors declare that this work was funded in part by NIH grants DA018764 (BS), AG029421 (JLB) and DA023331 (NWS). The authors further declare that, except for income received from their primary employer, no financial support or compensation has been received from any individual or corporate entity over the past three years for research or professional service and there are no personal financial holdings that could be perceived as constituting a potential conflict of interest.
  • Ainslie G. Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol Bull. 1975;82:463–496. [PubMed]
  • Balleine BW, Dickinson A. Goal-directed instrumental action: continency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. [PubMed]
  • Bechara A, Dolan S, Denburg N, Hindes A, Anderson SW, Nathan PE. Decision-making deficits, linked to dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia. 2001;39:376–389. [PubMed]
  • Bechara A, Tranel D, Damasio H. Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain. 2000;123:2189–2202. [PubMed]
  • Berridge CW, et al. Methylphenidate preferentially increases catecholamine neurotransmission within the prefrontal cortex at low doses that enhance cognitive function. Biol Psychiatry. 2006;60:1111–1120. [PubMed]
  • Cardinal R, Howes N. Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 2005;6:37. [PMC free article] [PubMed]
  • Cardinal RN. Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw. 2006;19:1277–1301. [PubMed]
  • Cardinal RN, Robbins TW, Everitt BJ. The effects of d-amphetamine, chlordiazepoxide, α-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology (Berl) 2000;152:362–375. [PubMed]
  • Carlezon WA, Thomas MJ. Biological substrates of reward and aversion: A nucleus accumbens activity hypothesis. Neuropharmacology. 2009;56:122–132. [PMC free article] [PubMed]
  • Chhatwal JP, Davis M, Maguschak KA, Ressler KJ. Enhancing Cannabinoid Neurotransmission Augments the Extinction of Conditioned Fear. Neuropsychopharmacology. 2004;30:516–524. [PubMed]
  • Clark L, Bechara A, Damasio H, Aitken MRF, Sahakian BJ, Robbins TW. Differential effects of insular and ventromedial prefrontal cortex lesions on risky decision-making. Brain. 2008;131:1311–1322. [PubMed]
  • Connor J, Makonnen E, Rostom A. Comparison of analgesic effects of khat (Catha edulis Forsk) extract, d-amphetamine, and ibuprofen in mice. J Pharm Pharmacol. 2000;52:107–110. [PubMed]
  • Cools R, Robinson OJ, Sahakian B. Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology. 2008;33:2291–2299. [PubMed]
  • Dalley JW, Laane K, Pena Y, Theobald DE, Everitt BJ, Robbins TW. Attentional and motivational deficits in rats withdrawn from intravenous self-administration of cocaine or heroin. Psychopharmacology (Berl) 2005;182:579–587. [PubMed]
  • Daw ND, Kakade S, Dayan P. Opponent interactions between serotonin and dopamine. Neural Netw. 2002;15:603–616. [PubMed]
  • De Wit H, Enggasser JL, Richards JB. Acute administration of d-amphetamine decreases impulsivity in healthy volunteers. Neuropsychopharmacology. 2002;27:813–825. [PubMed]
  • DeVito EE, et al. The effects of methylphenidate on decision-making in attention-deficit/hyperactivity disorder. Biol Psychiatry. 2008;64:636–639. [PMC free article] [PubMed]
  • Drago F, Caccamo G, Continella G, Scapagnini U. Amphetamine-induced analgesia does not involve brain opioids. Eur J Pharmacol. 1984;101:267–269. [PubMed]
  • Drechsler R, Rizzo P, Steinhausen HC. Decision-making on an explicit risk-taking task in preadolescents with attention-deficit/hyperactivity disorder. J Neural Transm. 2008;115:201–209. [PubMed]
  • Ernst M, et al. Neural substrates of decision making in adults with attention deficit hyperactivity disorder. Am J Psychiatry. 2003;160:1061–1070. [PubMed]
  • Ersche K, Roiser J, Robbins T, Sahakian B. Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology (Berl) 2008;197:421–431. [PubMed]
  • Evenden JL, Ryan CN. The pharmacology of impulsive behavior in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology (Berl) 1996;128:161–170. [PubMed]
  • File SE, Lippa AS, Beer B, Lippa MT. Animal Tests of Anxiety. Current Protocols in Neuroscience. 2003;8.3
  • Floresco SB, St Onge JR, Ghods-Sharifi S, Winstanley CA. Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision-making. Cogn Affect Behav Neurosci. 2008;8:375–389. [PubMed]
  • Gianotti LRR, et al. Tonic activity level in the right prefrontal cortex predicts individuals' risk-taking. Psychol Sci. 2009;20:33–38. [PubMed]
  • Goeders NE. A neuroendocrine role in cocaine reinforcement. Psychoneuroendocrinology. 1997;22:237–259. [PubMed]
  • Hayden BY, Platt ML. Temporal discounting predicts risk sensitivity in Rhesus macaques. Curr Biol. 2007;17:49–53. [PMC free article] [PubMed]
  • Heerey EA, Bell-Warren KR, Gold JM. Decision-making impairments in the context of intact reward sensitivity in schizophrenia. Biol Psychiatry. 2008;64:62–69. [PMC free article] [PubMed]
  • Johnson AW, Gallagher M, Holland PC. The basolateral amygdala is critical to the expression of Pavlovian and instrumental outcome-specific reinforcer devaluation effects. J. Neurosci. 2009;29:696–704. [PMC free article] [PubMed]
  • Killcross AS, Everitt BJ, Robbins TW. Symmetrical effects of amphetamine and alpha-flupenthixol on conditioned punishment and conditioned reinforcement: contrasts with midazolam. Psychopharmacology (Berl) 1997;129:141–152. [PubMed]
  • Kobayashi S, Schultz W. Influence of reward delays on responses of dopamine neurons. J. Neurosci. 2008;28:7837–7846. [PubMed]
  • Lejuez CW, Aklin WM, Zvolensky MJ, Pedulla CM. Evaluation of the Balloon Analogue Risk Task (BART) as a predictor of adolescent real-world risk-taking behaviours. J Adolesc. 2003;26:475–479. [PubMed]
  • Lejuez CW, et al. Evaluation of risk taking: the Balloon Analogue Risk Task (BART) J Exp Psychol Appl. 2002;8:75–84. [PubMed]
  • Leland DS, Paulus MP. Increased risk-taking decision-making but not altered response to punishment in stimulant-using young adults. Drug Alcohol Depend. 2005;78:83–90. [PubMed]
  • Liu Z-H, Shin R, Ikemoto S. Dual role of medial A10 dopamine neurons in affective encoding. Neuropsychopharmacology. 2008;33:3010–3020. [PMC free article] [PubMed]
  • Mobini S, Chiang TJ, Ho MY, Bradshaw CM, Szabadi E. Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl) 2000;152:390–397. [PubMed]
  • Negus S. Effects of punishment on choice between cocaine and food in rhesus monkeys. Psychopharmacology (Berl) 2005;181:244–252. [PubMed]
  • Rahman S, et al. Methylphenidate (‘Ritalin') can ameliorate abnormal risk-taking behavior in the frontal variant of frontotemporal dementia. Neuropsychopharmacology. 2005;31:651–658. [PMC free article] [PubMed]
  • Reyna V, Farley F. Risk and rationality in adolescent decision-making: Implications for theory, practice, and public policy. Psychological Science in the Public Interest. 2006;7:1–44.
  • Schultz W, et al. Review. Explicit neural signals reflecting reward uncertainty. Philos Trans R Soc Lond B Biol Sci. 2008;363:3801–3811. [PMC free article] [PubMed]
  • Setlow B, Schoenbaum G, Gallagher M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron. 2003;38:625–636. [PubMed]
  • Simon NW, Busch TN, Richardson AJ, Walls KJ, Setlow B. Cocaine exposure and impulsive choice: A parametric analysis. Society for Neuroscience Abstracts. 2007a 933.4/WW24.
  • Simon NW, Gilbert RJ, Mayse JD, Setlow B. A rat model of risky decision-making: The role of dopamine in balancing risk and reward. Society for Neuroscience Abstracts. 2008a 875.1/RR63.
  • Simon NW, et al. Good things come to those who wait: Attenuated discounting of delayed rewards in aged Fischer 344 rats. Neurobiol Aging. 2008b print copy in press. [PMC free article] [PubMed]
  • Simon NW, Mendez IA, Setlow B. Cocaine exposure causes long term increases in impulsive choice. Behav Neurosci. 2007b;121:543–549. [PMC free article] [PubMed]
  • Sobanski E, et al. Driving-related risks and impact of methylphenidate treatment on driving in adults with attention-deficit/hyperactivity disorder (ADHD) J Neural Transm. 2008;115:347–356. [PubMed]
  • St Onge JR, Floresco SB. Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology. 2009;34:681–697. [PubMed]
  • St Onge JR, Floresco SB. Dissociable roles for different subregions of the rat prefrontal cortex in risk-based decision making. Society for Neuroscience Abstracts. 2008 88.15/SS11.
  • Stanis J, Marquez Avila H, White M, Gulley J. Dissociation between long-lasting behavioral sensitization to amphetamine and impulsive choice in rats performing a delay-discounting task. Psychopharmacology (Berl) 2008;199:539–548. [PMC free article] [PubMed]
  • Taylor Tavares JV, Clark L, Cannon DM, Erickson K, Drevets WC, Sahakian BJ. Distinct profiles of neurocognitive function in unmedicated unipolar depression and bipolar II depression. Biol Psychiatry. 2007;62:917–924. [PubMed]
  • Weber EU, Shafir S, Blais AR. Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation. Psychol Rev. 2004;111:430–445. [PubMed]
  • Wellman P, Ho D, Cepeda-Benito A, Bellinger L, Nation J. Cocaine-induced hypophagia and hyperlocomotion in rats are attenuated by prazosin. Eur J Pharmacol. 2002;455:117–126. [PubMed]
  • Wellman PJ, Davis KW, Clifford PS, Rothman RB, Blough BE. Changes in feeding and locomotion induced by amphetamine analogs in rats. Drug Alcohol Depend. 2008 print copy in press. [PMC free article] [PubMed]
  • White FJ, Kalivas PW. Neuroadaptations involved in amphetamine and cocaine addiction. Drug Alcohol Depend. 1998;51:141–153. [PubMed]
  • Winstanley CA, Dalley JW, Theobald DE, Robbins MJ. Global 5-HT depletion attenuates the ability of amphetamine to decrease impulse choice on a delay-discounting task in rats. Psychopharmacology (Berl) 2003;170:320–331. [PubMed]
  • Winstanley CA, Eagle DM, Robbins TW. Behavioral models of impulsivity in relation to ADHD: Translation between clinical and preclinical studies. Clin Psychol Rev. 2006a;26:379–395. [PMC free article] [PubMed]
  • Winstanley CA, et al. DeltaFosB induction in orbitofrontal cortex mediates tolerance to cocaine-induced cognitive dysfunction. J Neurosci. 2007;27:10497–10507. [PubMed]
  • Winstanley CA, Theobald DEH, Dalley JW, Cardinal RN, Robbins TW. Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice. Cereb Cortex. 2006b;16:106–114. [PubMed]
  • Yi R, de la Piedad X, Bickel WK. The combined effects of delay and probability in discounting. Behav Processes. 2006;73:149–155. [PubMed]