PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Behav Processes. Author manuscript; available in PMC Dec 17, 2012.
Published in final edited form as:
PMCID: PMC3523357
NIHMSID: NIHMS424871
Deciding when to “cash in” when outcomes are continuously improving: An escalating interest task
Michael E. Young,* Tara L. Webb, and Eric A. Jacobs
Southern Illinois University at Carbondale, United States
*Corresponding author at: 281 LSII, Mailcode 6502, Department of Psychology, Southern Illinois University, Carbondale, IL 62901, 618/453-3567, United States. meyoung/at/siu.edu (M.E. Young)
A first-person shooter video game was adapted for the study of choice between smaller sooner and larger later outcomes. Participants chose when to fire a weapon that increased in damage potential over a 10 s interval, an escalating interest situation. Across two experiments, participants demonstrated sensitivity to the nature of the mathematical function that defined the relationship between waiting and damage potential. In Experiment 1, people tended to wait longer when doing so allowed them to eliminate targets more quickly. In Experiment 2, people tended to wait longer to increase the probability of a constant magnitude outcome than to increase the magnitude of a 100% certain outcome that was matched for the same expected value (i.e., probability times magnitude). The two experiments demonstrated sensitivity to the way in which an outcome improves when the outcome is continuously available. The results also demonstrate that this new video game task is useful for generating sensitivity to delay to reinforcement over time scales that are typically used in nonhuman animal studies.
Keywords: Impulsivity, Delay discounting, Video game, Probability, Magnitude
In the present work, we introduce a novel method for assessing delay and probability discounting in people. We sought to produce a new task that would allow us to study contingency-shaped performances in humans by embedding the experimental contingencies within the context of a first-person-shooter video game in which the outcome was the amount of damage produced by a player’s weapon. A participant could fire sooner and do less damage or wait for their weapon to increase in its damage potential. We hoped the consequences arranged in this virtual world (i.e., destroying a target) would impact behavior in a way that establishes sensitivity to reinforcement delay or probability over timescales that are more similar to those typically used in the study of nonhuman animal behavior. By embedding contingencies within the context of a video game, we allowed participants to continuously interact with an environment that produced consequences of immediate utility. A gaming platform may therefore be useful for addressing questions regarding behavioral sensitivity to proximal versus distal or temporally diffuse consequences.
Delay discounting is the process by which the effect of a consequence on behavior decreases as a function of the time between the occurrence of the behavior and the delivery of the consequence. Delay discounting is a basic behavioral process that is thought to underlie many problems of intertemporal choice. When a person acts impulsively and his or her behavior is governed by immediate consequences at the expense of long term outcomes, it is assumed that the efficacy of the future consequences is diminished as a function of delay to the point that they do not affect current behavior. The study of delay discounting has grown over the past decade because research has shown that severe delay discounting is often comorbid with impulse control disorders, such as smoking, pathological gambling, alcohol use, and drug abuse (e.g., Alessi and Petry, 2003; Bickel et al., 1999; Petry, 2001). That is, individuals with impulse control disorders unsurprisingly often show less sensitivity to delayed consequences than matched control participants.
A common procedure for assessing delay discounting in people is the hypothetical money choice task (HMCT) introduced by Rachlin et al. (1991). The task involves a series of choices involving hypothetical outcomes like, “Would you prefer $780 now or $1000 in a week?” The amount of the immediately available money is adjusted systematically across choice trials to identify a point of subjective equivalence (i.e., an indifference point) between the smaller immediate amount and the larger later amount. The delay to the larger later amount is manipulated across conditions to establish an indifference curve that displays how the subjective value of money is discounted as a function of delay.
The validity of the HMCT is strengthened by the fact that the indifference curves generated with the procedure are well described by a hyperbolic model that was introduced by Mazur (1987) and has been used to accurately describe delay discounting in a variety of nonhuman subjects. Despite the success of these methods, limitations remain regarding the generality of the data. The participants are typically responding to a single presentation of descriptions of the contingencies and, unlike the procedures used with nonhuman subjects, they do not have the opportunity to learn from their choices (but see Lagorio and Madden, 2005 for an exception). Thus, the human performances may be a function of instructional control and verbal mediation and may not be representative of steady-state response patterns generated by repeated exposure to the described contingencies.
Given the prevalence of maladaptive behavior (i.e., drug abuse, risky sex, poor financial decisions) related to delay discounting, it is important to reliably generate impulsive behavior in the laboratory. In nonhuman animals, impulsive choice is examined using operant methods that function on very different time scales than human discounting studies and consistently use immediately consumable reinforcers rather than hypothetical rewards or conditioned reinforcers (i.e., money or points). Developing experience based procedures for assessing delay discounting in people may provide a valuable tool for understanding impulsive behavior.
Early attempts to develop experiential procedures for studying choice and self-control in people used procedures that were intended to be functional analogues of operant methods used to study choice and self control in nonhuman subjects. Extending traditional operant methods to the study of human choice and self-control, however, initially met with limited success (see Logue, 1988; Mazur, 1998, for reviews). In studies of delay discounting with nonhuman animals, subjects make repeated choices between a smaller sooner reinforcer (SSR) and a larger later reinforcer (LLR). These methods have produced orderly data and have led to quantitative models of delay discounting (Green and Myerson, 2004; Mazur, 1987, 1988; Rachlin, 2006; Rachlin et al., 1991). Adult participants do not reliably discount delayed consequences (e.g., Logue et al., 1986). That is, unlike nonhuman animals, the adult participants responding for points exchangeable for money typically prefer the LLR at all delays examined (but cf. Lane et al., 2003a; Reynolds and Schiffbauer, 2005). Thus, adult human behavior in these contexts is often consistent with maximization of overall reinforcement density (i.e., amount x rate). In many operant approaches to the study of impulsive behavior, control by reinforcement immediacy is dissociated from control by overall reinforcement density by either including a delay following the reinforcer delivery to equate the durations of the terminal links (e.g., Rachlin and Green, 1972) or by arranging the onset of a choice trial according to a fixed cycle time (e.g., Mazur, 1988). Under these conditions, people responding for points exchangeable for money typically maximize reinforcement density (e.g., Flora and Pavlik, 1992). A few recent studies, however, have established selection of the SSR in adult humans responding for money by eliminating the postreinforcer delay (Lane et al., 2003a; Reynolds and Schiffbauer, 2005) or by also delivering the LLR probabilistically (Reynolds and Schiffbauer, 2005). Although these procedural innovations are promising, they introduce factors that may make it difficult to determine if the results are driven by sensitivity to reinforcement immediacy or by some other factors (e.g., overall reinforcement density, more rapid escape from the demands of the experiment, probability discounting, etc.).
One possible reason for the between-species disparity in traditional operant studies of choice and self-control may be differences in the nature of the consequences used in the different studies. In most studies with humans, behavior is maintained by points exchangeable for money, whereas in studies with nonhuman animals behavior is maintained by consumable consequences (e.g., food). Because all points earned are typically exchanged at the same time (i.e., at the end of the session or at the end of the entire experiment), there is no immediate utility to earning points sooner rather than later within the session (cf. Hyten et al., 1994). Some operant studies to show evidence of impulsive choice in adults have used consumable reinforcers in the form of escape from loud noise (Navarick, 1982; Solnick et al., 1980), juice (Jimura et al., 2009; McClure et al., 2007), or access to a video game (Millar and Navarick, 1984). The present research is an extension of this approach.
We designed a task in which an outcome would gradually increase in value as time passed, a paradigm that we will call an “escalating interest” task. Thus, a participant must decide when to cash-in, a decision analogous to that faced by investors for whom an asset is continuously increasing in value. A decision to cash in at a delay of 5 s indicates that the outcome available at that delay was of higher subjective value than all of the smaller sooner outcomes and all of the larger later ones. The choice that our participants faced is similar to that used in studies of deferred gratification in which the reward is always present during the waiting period (e.g., Mischel and Ebbesen, 1970; Reynolds and Schiffbauer, 2005), but in our task the reward available increased continuously during the delay. The only direct analogues we could uncover for our escalating interest task were (a) the single key impulsivity paradigm (SKIP, Swann et al., 2002) in which each mouse-click earned 1 cent for every 2 s since the previous response, or (b) an increasing food task in which nonhuman primates were given an increasing amount of food as the delay-to-consumption increased (Anderson et al., 2010; Beran and Evans, 2006, 2009; Pele et al., 2010). For example, Anderson et al. (2010) increased the amount of food available to a monkey over a delay, either by adding additional equally sized food items at regular intervals (thus magnitude was a linear function of delay) or by adding increasingly larger food items (thus magnitude was a power function of delay). Only the latter method induced waiting in their monkeys.
In order to study these types of choices, we designed a video game in which participants were told to simply destroy all of their targets (stationary monsters, “orcs”, distributed throughout a game environment involving hills, a lake, and buildings) in each of four levels of the game (for a clip, see http://bcs.siuc.edu/facultypages/young/Research/Supplemental.html). The four levels of the game were identical with only the behavior of the player’s weapon continuing to change; once a player completed all of the decisions in the game environment, the environment was reset (i.e., all orcs were resurrected) thus creating the next level of the game. This method was used to ensure that the participant would learn the spatial layout of the targets during the first level thus making for more efficient navigation in subsequent levels.
The key variable was the way in which the player’s weapon recharged; the weapon always obtained its full charge (maximum damage) 10 s after its previous shot. The player could choose to fire more quickly with less damage but with the advantage of being able to fire again soon or to fire more slowly in order to do more damage with each shot. In order to create situations in which impulsive behavior was detrimental to performance (i.e., firing early decreases overall rate of damage), we systematically changed the mathematical function dictating the recharge of the weapon. For some conditions (manipulated within-subject), a weapon recharged more slowly early in the 10 s interval, thus encouraging waiting, and for other conditions the weapon recharged more quickly early in the 10 s interval, thus encouraging firing sooner. Because multiple shots were required to destroy an orc, there were many decisions regarding how long to wait between shots thus allowing for ample opportunity for the participant to learn the properties of their weapon before the mathematical relationship changed.
The function producing the recharge behavior was the superellipsoid:
equation M1
(1)
Eq. (1) dictates the percentage of maximal damage done by firing the weapon t seconds after the previous shot. Examples of the application of this function are shown in Fig. 1. For powers greater than 1, it was beneficial to fire earlier. For powers less than 1, it was always beneficial to wait (i.e., impulsive behavior was detrimental to performance). Thus, under these conditions (power < 1), the procedure may have utility for assessing individual differences in sensitivity to delayed consequences. That is, a relatively “impulsive” participant would be more likely to “cash in” early before the weapon is fully charged (t < 10 s), leading to more immediate reinforcement in the short run but less efficient performance overall.
Fig. 1
Fig. 1
The four superellipsoid functions generated by Eq. (1) using a power value of 1.25 (top curve), 1.00, 0.75, and 0.50 (bottom curve).
If we assume that the participants’ goal is to destroy the targets as quickly as possible, then there is an optimal wait time for each level of power. Waiting has the benefit of increasing the weapon’s damage but the cost of reducing the firing rate. For weapons with a power value of 1.0, the linear increase in damage for waiting is directly offset by the decrease in firing rate and thus a participant will maximize the rate of target destruction for any wait time of 10 s or less (cf. the single key impulsivity paradigm or SKIP, Dougherty et al., 2005). In contrast, for weapons with a power less than 1.0, the optimal participant should always wait the full 10 s and no longer. The degree of sub-optimality is a function of both the deviation from this optimal value and the weapon’s power. For example, at a power of 0.5, firing every 5 s produces a rate of destruction only 17% of optimal whereas at a power of 0.75 the same rate of firing produces a rate of destruction that is 60% of optimal. If one were to wait for 9 s, a power of 0.5 produces a rate of destruction 52% of optimal whereas at a power of 0.75 these longer wait times produce a rate of destruction that is 85% of optimal.
Finally, for weapons with a power greater than 1.0, the participant should shoot as rapidly as possible (in our game, players can shoot as rapidly as once every 0.25 s). Again, degree of suboptimality for deviating from this optimum is a function of both the deviation from the optimal rate and the weapon’s power. For example, at a power of 1.25, firing every 5 s produces a rate of destruction 52% of optimal whereas at a power of 1.50, the same rate of firing produces a rate of destruction that is 33% of optimal. The penalty for waiting longer than 10 s is identical across all power values.
In Experiment 1, we sought to establish the basic issue of control by consequences as a function of power (Eq. (1)) in the context of the video game. Will conditions in which impulsive choice is beneficial to fast task completion (powers > 1.0) elicit faster firing whereas conditions in which impulsive choice is detrimental to quick completion (powers < 1.0) elicit slower firing? Once sensitivity to our key independent variable, power, was established, we were positioned to examine the effect of other environmental variables on both the sensitivity to power and the overall firing rate.
Experiment 2 was designed to assess behavior under two conditions. In the magnitude condition, the amount of damage increased as specified in Eq. (1) and thus replicates Experiment 1. In the probability condition, the probability of doing maximal damage increased as specified in Eq. (1). In terms of expected value, these two conditions were matched – if waiting 5 s produced 4 points of damage (of 10) for a given power value in the magnitude condition, then waiting 5 s produced a 40% chance of 10 points of damage (0.4 × 10 points = EV of 4 points) for the same power value in the probability condition. Thus, if participants are only sensitive to the expected value of the outcome, then the two conditions should give rise to the same behavioral tendencies toward impulsive choice as a function of the weapon’s power value. If, however, guaranteed small amounts are valued more highly than uncertain large amounts (Kahneman and Tversky, 1979; Shafir et al., 2008), then our players will cash-in more quickly in the magnitude condition than in the probability condition (because the uncertainty will evoke a tendency to wait in order to achieve certainty because of its greater perceived value).
1.1. Predicting game performance
A natural question that arises concerns the degree to which other measures of impulsivity would correlate with choice in our video game. Prior studies have shown moderate, weak, and no correlations among various measures of impulsivity (e.g., Lane et al., 2003b; Reynolds et al., 2006; Swann et al., 2002; Wingrove and Bond, 1997), so we did not anticipate strong correlations. Furthermore, we expected the task contingencies to be relatively strong thus dominating any small differences among our participants’ behavioral tendencies (either from prior history or biological differences). In Experiment 1, participants completed a delay discounting task assessing preference for immediate and delayed hypothetical monetary amounts to determine if discounting rate in the hypothetical choice task predicts choice in the video game (either sensitivity to consequences or the overall tendency to fire sooner rather than later). Participants also completed the Fagerström Test for Nicotine Dependence (Heatherton et al., 1991). Given that earlier studies have shown that smokers tend to have higher discount rates as assessed using traditional discounting procedures (Bickel et al., 1999; Johnson et al., 2007; Mitchell, 1999), smokers might produce shorter interresponse times in the video game. Our natural sampling of smokers in a college population produced small samples, however, and thus these results will be given little attention. In Experiment 2, participants completed the Barratt Impulsiveness Scale (or BIS) as well as a behavioral inhibition task in which a pre-potent response had to be withheld in the presence of a stop signal that occurred at random. The BIS is the most prevalent measure of impulsivity in the clinical literature and includes a set of questions (e.g., “I act on impulse” and “I am future oriented”) that are intended to assess self-reported behavioral tendencies that transcend any given decision or situation (Barratt, 1959; Patton et al., 1995). Given that a behavioral task that has some similarity to our game choices, the SKIP, has shown no or weak correlation with a choice-based delay discounting task, the BIS, and a behavioral inhibition task (e.g., Dougherty et al., 2009), we did not anticipate any significant correlations between these measures and choice in our video game. Regardless, we chose to revisit the issue in these experiments.
2.1. Method
2.1.1. Participants
A total of 40 introductory psychology students at Southern Illinois University at Carbondale received course credit for their voluntary participation. There were 26 women and 14 men, and 8 participants who reported having smoked at least 100 cigarettes in their lives (1 of which was not presently smoking). Most of our smokers were relatively light smokers; six reported smoking less than 10 cigarettes a day, and one 11–20 per day.
2.1.2. Procedure
The Torque Game Engine (obtained from www.garagegames.com) was adopted as the platform for game development. The game world included four levels each containing seven separate regions with each region populated by two visually identical orcs. The landscape and orcs were identical across game levels. For simplicity, the orcs were stationary and oriented toward a target region (e.g., a building) that the player was directed to protect. Every 4 s (on average), each orc fired its weapon. In each 2-orc region, the firing of one of the orcs produced contingent explosions in the target region; this design was used to make the player’s goal more realistic. The player’s task was to destroy all of the orcs within each game level.
The power value of the weapon changed each time the participant destroyed two orcs. We used a random sampling design in which the power value was randomly chosen from the 0.50 to 1.25 range (uniformly). The change in a weapon’s power was accompanied with a three-tone sequence (three 250 ms pure tones each separated by 250 ms of silence for a total duration of 1250 ms) with a pitch that was correlated with the new power level. Pitch varied from 440 Hz for a power of 0.50 to 740 Hz for a power of 1.25. The instructions included the following information: “A tone will also sound each time your weapon changes. The pitch of the tone indicates the nature of your weapon’s recharge.” Because there were 14 orcs in each level and the power value changed when 2 orcs were destroyed, each player experienced seven power values in each game level. The principal advantage to this design is that it allows the generalization of the results to the entire range of the functional relationship, thus having the same advantage as randomly sampling participants (Brunswik, 1955).
The program tracked the location of the participant’s avatar (i.e., the player’s location in the video game world). Approximately once every second, the avatar’s position (in x, y, z coordinate space) was recorded along with a time stamp. Although traversal time in the first level was more erratic because of unfamiliarity with the game environment (more stops and starts), in subsequent levels participants moved efficiently from one location to the next (10–20 s of travel time).
Students completed demographic sheets that asked their sex, age, cigarette habits (using the Fagerström Test of Nictonine Dependence, Heatherton et al., 1991), and previous video game experience (see Young and Nguyen, 2009, for details). Given the small sample of smokers, we used a liberal criterion and classified someone as a smoker if the provided answer was “yes” to the question: “Have you smoked more than 100 cigarettes in your life?” There was insufficient power to perform a finer analysis of nicotine use and its relationship to the variables under study.
Each student also performed a delay discounting task programmed in PsyScope (Cohen et al., 1993) using Johnson and Bickel’s (2002) titration algorithm. We used five delays (1 week, 1 month, 3 months, 1 year, and 5 years) and $1000 as the delayed amount. The convergence criterion for the indifference point was met when the difference between the outer upper limit and the outer lower limit was less than $10 (see Johnson and Bickel, 2002, for details on their algorithm). Half of the students completed the delay discounting task before the video game task and half completed the game first.
In our analyses, we dropped interresponse times (IRTs, equivalent to waiting time between shots) that exceeded 20 s in duration with the assumption that these were contaminated by inattention or excessive travel time. This resulted in a loss of 1.7% of the data with no more than 5.9% lost from any single participant. To allow for a logistic analysis of participant behavior in which IRTs were distributed between the range 0 s and 10 s, inclusively, we also made the simplifying assumption that responses greater than 10 s were equivalent to 10 s waiting times (this affected 11.8% of the responses; the median delay for these longer IRTs was 10.5 s).
2.2. Results
All of the video game players completed all four levels and the titration task in the allotted time of 1 h. Six of our participants were dropped from the analysis of the titration discounting data: five due to a program error that resulted in overly rapid convergence under unusual behavioral patterns (choosing the larger later amount on every trial for the shortest delay) and one due to a failure to complete the task in a timely fashion (this participant required 650 choices before meeting criterion for the first delay; the median number of choices to meet criterion for the shortest delay for the other participants was 30). Because game performance did not stabilize until the second level, the first level was dropped for subsequent analyses.
Most participants produced a bimodal IRT distribution either firing as quickly as possible or waiting around 10 s. Thus, in our subsequent analyses, we rescaled the IRTs to the 0–1 interval and used a logit link function in a generalized linear mixed effects analysis, thus assuming a logistic relationship between our predictors and IRT. Fig. 2 shows the results of the best fitting logistic regression lines when the game preceded or followed the delay discounting task (see later analysis). Participants changed their firing rate in response to changes in the power value of the weapon in the predicted direction. The sensitivity to the independent variable, power, was greater when the game preceded the delay discounting task.
Fig. 2
Fig. 2
Best fitting logistic curves in Experiment 1 when the game was played first (solid line) or after (dashed line) the delay discounting task. The standard error on the intercepts (i.e., the level of the curves) is 0.2 s.
The data were analyzed with a linear mixed effects analysis using R’s (available at www.r-project.org) lmer function and centering the power and experience variables (Gelman and Hill, 2006; Jonsson et al., 2000; Pinheiro and Bates, 2004). The mixed effects analysis allowed intercepts, level slope effects, and power slope effects to vary across participants. To incorporate individual differences, we first fit a model containing the within-subject variables (power and level) and task order; the power slope effect (i.e., IRTs varied as a function of power) and task order were retained as predictors. We then fit a series of models involving various combinations of self-reported amount of experience, smoking status, and sex as additional fixed effects in the mixed effects analysis; this stepwise approach was used because of high correlations among many of these variables thus creating multicollinearity. We did not use participants’ self-reports of types of video games played in this analysis because the additional degrees of freedom produced an overly complex mixed effects model – only amount of experience was considered. The model containing self-reported amount of video game experience and sex plus their interaction provided the best fit as judged by the Akaike Information Criterion or AIC (Akaike, 1974).
The estimated sensitivity to power was −1.77 (SE = 0.10) and −0.81 (SE = 0.14) when the game was played first or second, respectively; both slopes were significantly different from 0.0, t(37) = −17.07 and −10.46, ps < 0.01. The main effect of task order was not significant, t(37) = −28, p > 0.05, the Task Order × Power interaction was significant, t(37) = 6.61, equation M2, p < 0.05, the main effect of experience was significant, t(37) = −14.85, equation M3, p < 0.05, the main effect of sex was significant, t(37) = −3.31, equation M4, p < 0.05, and the Sex × Experience interaction was significant, t(37) = 9.58, equation M5, p < 0.05 (Note: We have provided a pseudo-R2 or equation M6 as a measure of effect size that approximates the increase in R2 when the predictor is included in a mixed effects model, Raudenbush and Bryk, 2002). Fig. 3 shows the model’s fits for individual participants’ power-curves as a function of self-reported video game experience (here separated into trichotomized groups for display purposes) and reveals the nature of the relationship.
Fig. 3
Fig. 3
Best fitting logistic curves for each participant in Experiment 1 by video game experience. Men’s curves are on the top of the figure (labeled “m”). The curves on the left are for those with the lowest of amount of self-reported (more ...)
To examine any possible relationship between the types of video games played and overall performance by a participant, we used the random effect estimates of the slope and intercepts derived from the mixed effects model as dependent variables in a series of simple t-tests for each of the games-played self-reports. We also examined an additional dependent variable, the discounting rate, k, derived by fitting a hyperbolic discounting curve to the data from the delay discounting choice task using nonlinear mixed effects modeling (for a discussion of the advantages of this analytical approach, see Lindstrom and Bates, 1990):
equation M7
(2)
Typical discounting curves were generated by most participants (average performance is shown in Fig. 4) and the best fitting fixed effect estimate of k was 0.0011 with 1st and 3rd quartiles of 0.0006 and 0.0015. The random effect (i.e., subject) estimates of k were approximately normally distributed and thus lacked the positive skew often observed in other studies using alternative two-stage statistical methods. Only one of the tests was significant using uncorrected t-tests: participants who self-reported playing fighting games had higher discounting rates than those who did not, t(32) = 2.37, equation M8, p < 0.05.
Fig. 4
Fig. 4
Average participant performance on the delay discounting titration task used in Experiment 1. Error bars are ±1 SE.
In a final analysis, although higher discounting rates were associated with shorter IRTs (using the best fitting individual participant intercept from the linear mixed effects model), this effect did not reach statistical significance, r(32) = −0.23, p = 0.19. Discounting rates were unrelated to sensitivity to the power manipulation (using the best fitting individual participant slope for power from the linear mixed effects model), r(32) = −0.05, p = 0.79.
2.3. Discussion
Participants showed good sensitivity to environmental changes in which short IRTs either facilitated (power > 1.0) or retarded (power < 1.0) faster completion of the game task. This result demonstrates that the game’s demands were sufficient to modulate behavior as a function of the power parameter of Eq. (1). People showed less sensitivity, however, if the game was played after the discounting task had been completed suggesting that we may have been observing some degree of fatigue or boredom as the experiment progressed, or the prior exposure to the discounting task may have had a carryover effect on game performance.
As expected, delay discounting did not correlate highly with impulsive behavior in the game (cf. Dougherty et al., 2009; Lane et al., 2003a,b; Reynolds et al., 2008). For example, Lane et al. (2003a) reported non-significant correlations between the AUC and k values derived from a standard hypothetical discounting task like ours and a trial-by-trial consequences task involving small monetary outcomes (less than 15 cents) and short experienced delays (less than 90 s); their correlations were −0.01 and −0.03, respectively. Nicotine dependence and self-reported types of video games played also did not substantially predict the likelihood of impulsive choice in the game (given the small number of participants with a history of smoking, no conclusions can be reached regarding this relation). In contrast, as women’s self-reported experience with games increased, their IRTs tended to decrease thus suggesting a relationship between heavy game play and impulsive choice. Given that this is the first such finding in our laboratory and the unstable nature of individual differences variables, we wanted to see the result replicated to prevent over-interpreting this outcome. Additionally, one should not conclude that more experience playing video games causes more impulsive behavior because these results could suggest that people, especially women, who tend toward impulsive choice in these types of environments are drawn to play more video games. There could, of course, be a third-factor that gives rise to both impulsive game choices and the attractiveness of video games and thus the correlation is spurious.
Given that participants showed sensitivity to our manipulation of the consequences of response rate via the power parameter, we next sought to identify the conditions that might alter either the sensitivity to these consequences (i.e., the slope of the model) or the mean IRT (i.e., the intercept of the model). In description-based studies of the trade-off between a smaller certain amount and a larger uncertain amount, people tend to prefer a smaller certain amount (e.g., $3.00) over a somewhat larger but less certain amount (e.g., $4.00 with 80% likelihood), even when the expected value of the outcomes favors the larger uncertain amount (Kahneman and Tversky, 1979). When tested in an experience-based decision making paradigm in which outcomes are actually experienced, people still showed a preference for certainty as long as the outcomes were not too easy to discriminate (Shafir et al., 2008). Thus, we were interested in whether people would be more likely to wait longer in order to obtain a greater certainty of a larger outcome than they would for a certain outcome with the same expected value. Or, in more concrete terms, would someone who fires halfway into the recharge interval in order to do a certain 8 points of damage (magnitude condition) be more reluctant to fire halfway into the interval when the weapon would have produced an 80% chance of 10 points of damage (probability condition), despite it producing the same expected value of 8 points (0.80 × 10)?
In this experiment we used two new measures of impulsivity commonly used in the literature, a behavioral inhibition task and the BIS (e.g., Acheson, Reynolds et al., 2006; Dougherty et al., 2009; Swann et al., 2002), and dropped the FTND and the delay discounting task used in Experiment 1. We were interested in whether other measures of impulsivity might predict how long participants would wait between shots and/or their sensitivity to the power parameter.
3.1. Method
3.1.1. Participants
A total of 60 introductory psychology students at Southern Illinois University at Carbondale received course credit for their voluntary participation. There were 33 women, 21 men, and 6 who did not specify their sex.
3.1.2. Procedure
The procedure for the magnitude condition was identical to that used in Experiment 1, except that the power value ranged from 0.5 to 1.5. In the probability condition, probability increased over the 10 s period in a manner directly analogous to that for the magnitude condition. For example, when power was 1.0 a weapon that was half-charged had a 50% chance of doing maximal damage (probability condition) rather than a 100% chance of half damage (magnitude condition). Thus, the probability and magnitude conditions were matched in the expected value of the outcome for any particular combination of power and delay.
Students completed demographic sheets that asked their sex, age, and previous video game experience. Each student also completed the BIS questionnaire and a behavioral inhibition (go/no-go) task. The go/no-go task was programmed using PsyScope (Cohen et al., 1993). The task consisted of 150 trials in which the word “PRESS” was presented in either green or red. The stimulus was presented at the center of the screen on a black background for 250-ms followed by a 500-ms inter-trial interval (ITI). We instructed participants to press the space bar as quickly as possible after seeing the word “PRESS” in green but not to respond if the word was in red; both accuracy and speed were emphasized. The program only recorded the first response occurring during the stimulus presentation or the immediately following ITI; if no response occurred during this interval, the trial was categorized as a no-response trial. We programmed the presentation of the green (go) and red (no-go) stimuli to appear on 75% and 25% of the trials, respectively. The likelihood of a red or green stimulus being sampled was a constant 0.75/0.25 (i.e., stimulus type was sampled with replacement) to ensure that there were no trial-to-trial dependencies that would enable a participant to anticipate the next trial type beyond what is possible from their respective base rates. Thus, the experienced proportion of green to red trials varied between participants but was always near the programmed 3:1 ratio. The program recorded the occurrence or non-occurrence of a response and the reaction time for responses. Half of the students completed the behavioral inhibition task and the BIS before the video game task and half completed the game first.
3.2. Results
All of the participants completed the BIS and the behavioral inhibition task in the allotted time of one hour. Not all participants completed the video game task in a timely manner; if playing the game first, their play was terminated early to ensure adequate time to complete the other tasks (5 participants were terminated after partially completing the fourth level but all were retained in our analyses).
The same analytical approach of Experiment 1 was used for the present experiment. We again dropped the first level in our analyses. Fig. 5 shows the best fitting power-curve regression lines across the two conditions and Fig. 6 shows the best fitting power-curves for individual participants across the two conditions. Participants changed their firing rate in response to changes in the power value of the weapon in the predicted direction and waited considerably longer in the probability condition than in the magnitude condition.
Fig. 5
Fig. 5
Best fitting logistic curves in Experiment 2 for participants in the probability condition (dashed line) and magnitude condition (solid line). The standard error on the intercepts (i.e., the level of the curves) is 0.2 s.
Fig. 6
Fig. 6
Best fitting logistic curves for each participant in Experiment 2 in the magnitude condition (left) and probability condition (right).
A series of model fits resulted in our retaining none of the individual differences variables of sex or video game experience, the variable of task order, nor the within-subject variable of game level; a model containing only the main effects of power and condition produced the best fit as assessed by the AIC. The estimated sensitivity to power was −1.43 (SE = 0.05, t(58) = −27.40, equation M9, p < 0.01) and the main effect of condition, t(58) = 19.46, equation M10, p < 0.01, reflected much longer IRTs for the probability condition than for the magnitude condition (see Figs. 5 and and66).
3.2.1. Behavioral inhibition task
In the analysis of the behavioral inhibition task, we aggregated the proportion of hits versus misses (pressing vs. not pressing the bar when PRESS was displayed in green) and false alarms versus correct rejections (pressing vs. not pressing when PRESS was displayed in red) for each subject. The hit rate (hits divided by hits + misses) and false alarm rate (false alarms divided by false alarms + correct rejections) were used to compute sensitivity (d′) and bias (c) in accordance with signal detection theory (Macmillan and Creelman, 2004). Sensitivity assesses a participant’s ability to discriminate the conditions under which they should or should not press the bar, and response bias assesses the overall tendency to press the bar. For example, a participant who could not detect red versus green or who had a general inattention to the task would have a low d′ value. A participant who generated a lot of false alarms due to a bias to respond would have a low c value. Both d′ and c were normally distributed in our sample (Ms = 3.35 and 0.55 and SDs = 0.77 and 0.31 for d′ and c, respectively). When included as predictors, neither d′ nor c improved the model fit, and neither was correlated with our other demographic variables (video games played, gaming experience, sex, or BIS score).
Response times on the behavioral inhibition task were also not related to our other behavioral measures. We did observe rational relationships between RTs and d′ (slower responders produced higher d′ values, t(53) = 2.21, equation M11, p < 0.05) and RTs and c (slower responders were less likely to respond, t(53) = 4.49, equation M12, p < 0.01). Furthermore, RTs for false alarms were shorter (M = 268 ms) than those for correct responses (M = 324 ms). Collectively, the data indicate that participants’ responding was systematic during the behavioral inhibition task and, therefore, that the task demonstrated good control over the participants’ behavior.
3.2.2. Barratt impulsiveness scale
To analyze BIS scores, we scored the BIS questionnaire to create an overall BIS score (Patton et al., 1995); scores ranged from 37 to 88 with a mean of 59.5 and SD of 10.3 and were approximately normally distributed (for those unfamiliar with the BIS, the lowest possible score is 30 indicating self-reporting a high degree of self-control and the highest possible score is a 120 indicating self-reporting a high degree of impulsivity). BIS was not correlated with our other demographic variables (video games played, gaming experience, or sex). However, adding BIS as a main effect to the model of video game behavior did improve the model, t(58) = −3.81, equation M13, p < 0.01, by accounting for some of the individual differences in the intercept (overall tendency to fire quickly or slowly).
To interpret the nature of the BIS effect, the predicted average IRT for a participant with a 25th percentile BIS (score = 53) in our study is 7.0 s whereas that for a participant with a 75th percentile BIS (score = 67) is 5.7 s; thus, the direction of the effect is consistent with participants who self-report greater self-control (low BIS scores) waiting longer to fire their weapon. Despite the relatively large difference of 1.3 s, this result must be considered in light of the considerable variability in this relationship; the simple correlation between the total BIS score and participant intercepts (i.e., overall waiting time regardless of the power parameter) was r(57) = −0.36, p < 0.01. Given that a prior published study revealed stronger correlations between the motor subscale of the BIS and waiting time in the SKIP (r = 0.44 for motor vs. 0.32 for total, Swann et al., 2002), we performed a similar analysis and found a correlation between the smaller 7-item motor impulsiveness scale and mean game IRTs, r(57) =−37, p < 0.01. When both the total BIS and the motor impulsiveness subscale were analyzed using a stepwise regression with the motor subscale entering the analysis first, adding the total BIS did not significantly improve the model’s fit (adjusted R2 = 0.096 vs. 0.107 and AIC = 14.29 vs. 14.49, for the motor-only vs. motor-and-total BIS models, respectively).
3.2.3. Types of video games played
To examine any possible relationship between the types of video games played and overall performance by a participant, we used the random effect estimates of the slope and intercepts derived from the mixed effects model as dependent variables in a series of simple t-tests for each of the games-played self-reports. Only two of the tests were significant using uncorrected t-tests: participants who self-reported playing musical and rhythm games (e.g., Guitar Hero, Dance Dance Revolution) had higher IRT intercepts, waiting 1.2 s longer, than those participants who did not, t(57) = 3.01, equation M14, p < 0.01, and participants who self-reported playing first-person shooter games had less sensitivity to the power manipulation, t(55) = 2.16, equation M15, p < 0.05.
3.3. Discussion
Experiment 2 replicated our general finding of participants showing sensitivity to the benefits versus costs of waiting as revealed by strong effects of the power manipulation on waiting time. The relationship between sex and game experience that we reported in Experiment 1 did not replicate. Furthermore, the small order effect observed in Experiment 1 was not observed in Experiment 2 when the game was accompanied by a behavioral inhibition task rather than a delay discounting task which suggests that Experiment 1’s order effect might have been specific to the use of a discounting procedure or due to Type I error.
The key result of Experiment 2 was that participants waited considerably longer when waiting produced a higher probability of the maximal outcome than when waiting produced a higher magnitude with the same expected value as the probabilistic outcome. Fig. 5 suggests that participants with the maximal incentive to fire sooner in the probability condition (power near 1.5) tended to wait the same amount of time as participants with the maximal incentive to fire later in the magnitude condition (powers near 0.5). Fig. 6 further reveals that the vast majority of participants in the probability condition waited longer than nearly all of the participants in the magnitude condition. This result replicates the aversive nature of probabilistic outcomes demonstrated in prior experiments of the certainty effect (Kahneman and Tversky, 1979) and extends prior work by documenting a tendency to wait longer in order to increase certainty.
Finally, waiting in the video game did not correlate with either the outcome sensitivity or response bias observed in a simple behavioral inhibition task, but waiting was moderately correlated with a trait measure of impulsivity, BIS score, in a rational direction (with higher impulsiveness associated with less waiting). This weak correlation is consistent with other observations of weak correlations between the BIS and various behavioral measures of impulsivity, including a task functionally similar to our video game task (SKIP) (Dougherty et al., 2008; Swann et al., 2002).
In an escalating interest situation, video game players showed an ability to withhold responding when doing so produced larger benefits (due to efficient destruction of targets), although participants differed in their average wait time and responsiveness to changes in the power value of Eq. (1) (see Figs. 3 and and6).6). When magnitude increased with delay under conditions of certainty, participants tended to cash-in earlier than when probability increased with delay under conditions that produced the same expected value as those extant in the certain-outcomes condition (Figs. 5 and and66).
Unlike the more common delay discounting tasks, our experiments used an escalating interest paradigm that is analogous to those used in deferred gratification tasks in which the outcome is continuously available during the delay (e.g., Mischel and Ebbesen, 1970; Reynolds and Schiffbauer, 2005) but in which the outcomes in the game continuously improved as the delay progressed. This continuous improvement could make it increasingly more difficult to continue to defer but also could make the advantages of waiting continuously salient (cf. Anderson et al., 2010; Beran and Evans, 2006). Although Rachlin (2000) made a strong argument that behavior in delay discounting and deferred gratification tasks is rooted in the same process, it is still unclear how best to extend the mathematical models that have shown so much success in modeling discounting behavior to continuously unfolding events. Identifying the best general models will require systematic study of the relationship among tasks in which an early choice commits one to an outcome delay (delay discounting tasks), in which the choice for the smaller sooner outcome can be made at any point during the delay (deferred gratification tasks), and in which choices during the delay result in increasingly larger outcomes as a function of delay (an escalating interest task).
4.1. Game choices as they relate to other measures of impulsivity
Although a measure of trait impulsivity (BIS) was moderately related to video game behavior (Experiment 2), discounting parameters (k) derived from the hypothetical choice task did not predict behavior on the video game task (Experiment 1) nor did measures derived from a behavioral inhibition task (Experiment 2). This predictive failure may indicate: (a) discounting on the hypothetical choice task may not translate to discounting performance on the time scales present in our video game, or (b) behavioral inhibition and delay discounting tasks measure different types of impulsivity than that leveraged in our game preparation. We next consider these alternatives in further detail.
First, it is possible that the behavioral processes that dictate choice between smaller sooner and larger later rewards vary between longer- and shorter-term consequences. The choice between the immediate benefits of smoking a cigarette or eating a piece of pie versus the long-term benefits of cleaner lungs or a healthier weight may be fundamentally different than the choice between making a quick decision to fire a weapon or waiting a few seconds in order to obtain a better shot, or between choosing an adequate word for your sentence and taking the time to consult a thesaurus in order to choose the perfect word (cf. Lane et al., 2003a). Furthermore, perhaps short-term decisions are much more heavily determined by the current environmental conditions than by the history of the organism (e.g., Reynolds et al., 2006).
Although there is a clear preference for a parsimonious model that assumes that the same behavioral process produces sensitivity to delay for both near-term and long-term delayed consequences, some scientists have provided evidence that two or more discounting functions may be necessary to model discounting behavior at different time scales (McClure et al., 2007). Evidence from imaging studies supports the dissociation of processes dictating immediate decisions from those involving intertemporal choice (McClure et al., 2007; Redish and Kurth-Nelson, 2009). The observed moderate correlation between BIS and game choices, however, suggests that the BIS provides some assessment of people’s awareness of their short-term behavioral tendencies (specifically in the motor component of the BIS).
Second, impulsivity has been hypothesized to be a multifaceted construct in which its components may be only weakly correlated (Acheson et al., 2006; Dougherty et al., 2008; Lane et al., 2003b; Winstanley, 2009). With continued study, we may be able to determine whether the types of decisions faced by participants in our game are more closely related to other behavioral tendencies that were not assessed by the other tasks that we examined. Impulsivity has been operationally defined in a variety of ways, and many of these definitions appear unrelated or weakly related to one another when behaviorally assessed. Although identifying relations among these behaviors can improve the understanding of choice, ideally this understanding will lead to effective manipulations that can change people’s behavior.
4.2. Environmental factors that may influence the emergence of impulsive behavior
Participants playing the video game were more willing to wait to increase the probability of a constant-magnitude outcome than to increase the magnitude of a certain outcome. This behavior reveals a general preference for certainty over uncertainty and is thus reminiscent of the certainty effect observed in human studies of choice (Kahneman and Tversky, 1979). When people are confronted with the choice between winning a certain $3.00 and an 80% chance of winning $4.00 (the latter with an expected value of 0.80 × $4.00 = $3.20), they tend to choose the smaller certain amount. In contrast, when given the choice between a 25% chance of winning $3.00 (EV = $0.75) and a 20% chance of winning $4.00 (EV = $0.80), they prefer the choice with the higher expected value. We hypothesized that this preference for certainty would also be reflected by a greater willingness to wait in order to gain greater certainty, a prediction that was confirmed in Experiment 2. This outcome suggests that interventions targeted at increasing self-control might be more effective if waiting was rewarded with a higher probability of a much larger outcome rather than with a certain somewhat larger outcome.
We believe that a greater understanding of the environmental factors that cause even self-controlled individuals to behave impulsively will have clear implications for behavioral modification. Although identifying behavioral tendencies (so-called trait variables) helps the diagnosis of those with impulsiveness problems, treatment necessitates an understanding of the environmental variables that influence decisions in order to improve long-term outcomes. If our escalating interest video game preparation proves to be an effective platform for studying a broad array of these variables and their effects on impulsive choice, we will be better positioned to manipulate environments to produce the behaviors desired in a way that benefits individuals and society.
Acknowledgments
This research was partially supported by a Southern Illinois University undergraduate REACH grant to Tara Webb and by NIDA 1R15-DA026290. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
  • Acheson A, Reynolds B, Richards JB, de Wit H. Diazepam impairs behavioral inhibition but not delay discounting or risk taking in healthy adults. Experimental and Clinical Psychopharmacology. 2006;14:190–198. [PubMed]
  • Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control. 1974;19:716–723.
  • Alessi SM, Petry NM. Pathological gambling severity is associated with impulsivity in a delay discounting procedure. Behavioural Processes. 2003;64 (3):345–354. [PubMed]
  • Anderson JR, Kuroshima H, Fujita K. Delay of gratification in capuchin monkeys (Cebus apella) and squirrel monkeys (Saimiri sciureus) Journal of Comparative Psychology. 2010;124:205–210. [PubMed]
  • Barratt ES. Anxiety and impulsiveness related to psychomotor efficiency. Perceptual and Motor Skills. 1959;9:191–198.
  • Beran MJ, Evans TA. Maintenance of delay of gratification by four chimpanzees (Pan troglodytes): the effects of delayed reward visibility, experimenter presence, and extended delay intervals. Behavioural Processes. 2006;73:315–324. [PubMed]
  • Beran MJ, Evans TA. Delay of gratification by chimpanzees (Pan troglodytes) in working and waiting situations. Behavioural Processes. 2009;80:177–181. [PMC free article] [PubMed]
  • Bickel WK, Odum AL, Madden GJ. Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers. Psychopharmacology. 1999;146 (4):447–454. [PubMed]
  • Brunswik E. Representative design and probabilistic theory in a functional psychology. Psychological Review. 1955;62:193–217. [PubMed]
  • Cohen JD, MacWhinney B, Flatt M, Provost J. PsyScope: a new graphic interactive environment for designing psychology experiments. Behavioral Research Methods, Instruments, and Computers. 1993;25:257–271.
  • Dougherty DM, Marsh-Richard DM, Hatzis ES, Nouvion SO, Mathias CW. A test of alcohol dose effects on multiple behavioral measures of impulsivity. Drug and Alcohol Dependence. 2008;96:111–120. [PMC free article] [PubMed]
  • Dougherty DM, Mathias CW, Marsh DM, Jagar AA. Laboratory behavioral measures of impulsivity. Behavior Research Methods. 2005;37:82–90. [PubMed]
  • Dougherty DM, Mathias CW, Marsh-Richard DM, Furr RM, Nouvion SO, Dawes MA. Distinctions in behavioral impulsivity: implications for substance abuse research. Additive Disorders and Their Treatment. 2009;8:61–73. [PMC free article] [PubMed]
  • Flora SR, Pavlik WB. Human self-control and the density of reinforcement. Journal of the Experimental Analysis of Behavior. 1992;57:201–208. [PMC free article] [PubMed]
  • Gelman A, Hill J. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press; New York: 2006.
  • Green L, Myerson J. A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin. 2004;130 (5):769–792. [PMC free article] [PubMed]
  • Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström test for nicotine dependence: a revision of the Fagerström tolerance questionnaire. British Journal of Addiction. 1991;86:1119–1127. [PubMed]
  • Hyten C, Madden GJ, Field DP. Exchange delays and impulsive choice in adult humans. Journal of the Experimental Analysis of Behavior. 1994;62 (2):225–233. [PMC free article] [PubMed]
  • Jimura K, Myerson J, Hilgard J, Braver TS, Green L. Are people really more patient than other animals? Evidence from human discounting of real liquid rewards. Psychonomic Bulletin & Review. 2009;16:1071–1075. [PubMed]
  • Johnson MW, Bickel WK. Within-subject comparison of real and hypothetical money rewards in delay discounting. Journal of the Experimental Analysis of Behavior. 2002;77:129–146. [PMC free article] [PubMed]
  • Johnson MW, Bickel WK, Baker F. Moderate drug use and delay discounting: a comparison of heavy, light, and never smokers. Experimental and Clinical Psychopharmacology. 2007;15:187–194. [PubMed]
  • Jonsson EN, Karlsson MO, Wade JR. Nonlinearity detection: advantages of nonlinear mixed-effects modeling. AAPS PharmSci. 2000;2(3):article 32. [PMC free article] [PubMed]
  • Kahneman D, Tversky A. Prospect theory: an analysis of decisions under risk. Econometrica. 1979;47:313–327.
  • Lagorio CH, Madden GJ. Delay discounting of real and hypothetical rewards III: steady-state assessments, forced-choice trials, and all real rewards. Behavioural Processes. 2005;69:173–187. [PubMed]
  • Lane SD, Cherek DR, Pietras CJ, Tcheremissine OV. Measurement of delay discounting using trial-by-trial consequences. Behavioural Processes. 2003a;64:287–303. [PubMed]
  • Lane SD, Cherek DR, Rhoades HM, Pietras CJ, Tcheremissine OV. Relationships among laboratory and psychometric measures of impulsivity: implications in substance abuse and dependence. Addictive Disorders and Their Treatment. 2003b;2:33–40.
  • Lindstrom MJ, Bates DM. Nonlinear mixed effects models for repeated measures data. Biometrics. 1990;46:673–687. [PubMed]
  • Logue AW. Research on self-control: an integrating framework. Behavioral & Brain Sciences. 1988;11:665–678.
  • Logue AW, Pena-Correal TE, Rodriguez ML, Kabela E. Self-control in adult humans: variation in positive reinforcer amount and delay. Journal of the Experimental Analysis of Behavior. 1986;46:159–173. [PMC free article] [PubMed]
  • Macmillan NA, Creelman CD. Detection Theory: A User’s Guide. Lawrence Erlbaum; Mahwah, NJ: 2004.
  • Mazur JE. An adjusting delay procedure for studying delayed reinforcement. In: Commons ML, Mazur JE, Nevin JA, Rachlin HC, editors. The Effect of Delay and Intervening Events on Reinforcement Value. Vol. 5. Erlbaum; Hillsdale, NJ: 1987. pp. 55–73.
  • Mazur JE. Estimation of indifference points with an adjusting-delay procedure. Journal of the Experimental Analysis of Behavior. 1988;49 (1):37–47. [PMC free article] [PubMed]
  • Mazur JE. Choice and self-control. In: Lattal KA, Perone M, editors. Handbook of Research Methods in Human Operant Behavior. Plenum Press; New York: 1998. pp. 131–161.
  • McClure SM, Ericson KM, Laibson DI, Loewenstein G, Cohen JD. Time discounting for primary rewards. Journal of Neuroscience. 2007;27:5796–5804. [PubMed]
  • Millar A, Navarick DJ. Self-control and choice in humans: effects of video game playing as a positive reinforcer. Learning and Motivation. 1984;15:203–218.
  • Mischel W, Ebbesen EB. Attention in delay of gratification. Journal of Personality and Social Psychology. 1970;(16):329–337. [PubMed]
  • Mitchell SH. Measures of impulsivity in cigarette smokers and non-smokers. Psychopharmacology. 1999;146 (4):455–464. [PubMed]
  • Navarick DJ. Negative reinforcement and choice in humans. Learning & Motivation. 1982;13:361–377.
  • Patton JH, Stanford MS, Barratt ES. Factor structure of the Barratt impulsiveness scale. Journal of Clinical Psychology. 1995;51:768–774. [PubMed]
  • Pele M, Dufour V, Micheletta J, Thierry B. Long-tailed macaques display unexpected waiting abilities in exchange tasks. Animal Cognition. 2010;13:263–271. [PubMed]
  • Petry NM. Delay discounting of money and alcohol in actively using alcoholics, currently abstinent alcoholics, and controls. Psychopharmacology. 2001;154:243–250. [PubMed]
  • Pinheiro JC, Bates DM. Mixed-Effects Models in S and S-PLUS. Springer; New York: 2004.
  • Rachlin HC. The Science of Self-Control. Harvard University Press; Cambridge, MA: 2000.
  • Rachlin HC. Notes on discounting. Journal of the Experimental Analysis of Behavior. 2006;85:425–535. [PMC free article] [PubMed]
  • Rachlin HC, Green L. Commitment, choice and self-control. Journal of the Experimental Analysis of Behavior. 1972;17:15–22. [PMC free article] [PubMed]
  • Rachlin HC, Raineri A, Cross D. Subjective probability and delay. Journal of the Experimental Analysis of Behavior. 1991;55:233–244. [PMC free article] [PubMed]
  • Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2. Sage Publishing; Newbury Park, CA: 2002.
  • Redish AD, Kurth-Nelson Z. Neural models of delay discounting. In: Madden GJ, Bickel WK, editors. Impulsivity: The Behavioral and Neurological Science of Discounting. American Psychological Association; Washington, DC: 2009.
  • Reynolds B, Penfold RB, Patak M. Dimensions of impulsive behavior in adolescents: laboratory behavioral assessments. Experimental and Clinical Psychopharmacology. 2008;16:124–131. [PubMed]
  • Reynolds B, Richards JB, de Wit H. Acute-alcohol effects on the Experiential Discounting Task (EDT) and a question-based measure of delay discounting. Pharmacology, Biochemistry and Behavior. 2006;83:194–202. [PubMed]
  • Reynolds B, Schiffbauer R. Delay of gratification and delay discounting: a unifying feedback model of delay-related impulsive behavior. Psychological Record. 2005;55:439–460.
  • Shafir S, Reich T, Tsur E, Erev I, Lotem A. Perceptual accuracy and conflicting effects of certainty on risk-taking behaviour. Nature. 2008;453:917–920. [PubMed]
  • Solnick JV, Kannenberg CH, Eckerman DA, Waller MB. An experimental analysis of impulsivity and impulse control in humans. Learning & Motivation. 1980;11:61–77.
  • Swann AC, Bjork JM, Moeller FG, Dougherty DM. Two models of impulsivity: relationship to personality traits and psychopathology. Biological Psychiatry. 2002;51:988–994. [PubMed]
  • Wingrove J, Bond AJ. Impulsivity: a state as well as trait variable does mood awareness explain low correlations between trait and behavioural measures of impulsivity? Personality and Individual Differences. 1997;22:333–339.
  • Winstanley CA. The neural and neurochemical basis of delay discounting. In: Madden GJ, Bickel WK, editors. Impulsivity: The Behavioral and Neurological Science of Discounting. American Psychological Association; Washington, DC: 2009.
  • Young ME, Nguyen N. The problem of delayed causation in a video game: constant, varied, and filled delays. Learning and Motivation. 2009;40:298–312.