Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
JAMA Psychiatry. Author manuscript; available in PMC 2013 December 11.
Published in final edited form as:
PMCID: PMC3859132

Reward Signals, Attempted Suicide, and Impulsivity in Late-Life Depression



Suicide can be viewed as an escape from unendurable punishment at the cost of any future rewards. Could faulty estimation of these outcomes predispose to suicidal behavior? In behavioral studies, many of those who have attempted suicide misestimate expected rewards on gambling and probabilistic learning tasks.


To describe the neural circuit abnormalities that underlie disadvantageous choices in people at risk for suicide and to relate these abnormalities to impulsivity, which is one of the components of vulnerability to suicide.


Case-control functional magnetic resonance imaging study of reward learning using a reinforcement learning model.


University hospital and outpatient clinic.


Fifty-three participants 60 years or older, including 15 depressed patients who had attempted suicide, 18 depressed patients who had never attempted suicide (depressed control subjects), and 20 psychiatrically healthy controls.


Components of the cortical blood oxygenation level–dependent response tracking expected and unpredicted rewards.


Depressed elderly participants displayed 2 distinct disruptions of control over reward-guided behavior. First, impulsivity and a history of suicide attempts (particularly poorly planned ones) were associated with a weakened expected reward signal in the paralimbic cortex, which in turn predicted the behavioral insensitivity to contingency change. Second, depression was associated with disrupted corticostriatothalamic encoding of unpredicted rewards, which in turn predicted the behavioral oversensitivity to punishment. These results were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular illness, and incipient dementia.


Altered paralimbic reward signals and impulsivity and/or carelessness may facilitate unplanned suicidal acts. This pattern, also seen in gambling and cocaine use, may reflect a primary deficit in the paralimbic cortex or in its mesolimbic input. The overreactivity to punishment in depression may be caused in part by a disruption of appetitive learning in the corticostriatothalamic circuits.

What causes a person to commit suicide is one of the central questions in psychiatry. We know that stressors precipitate suicidal behavior, particularly in the pathological states of depression, psychosis, intoxication, and pain. However, because only a minority of people facing these circumstances attempt suicide, the individual diathesis must play an important role. This individual diathesis is still not well understood. However, impulsive-aggressive traits,16 cognitive deficits,713 and persisting hopelessness1416 have emerged as important dimensions. Suicidal behavior is heterogeneous, and even less is known about the many pathways that lead to it. Suicidal acts follow a decision, and we argue that a propensity to make bad decisions may be part of one of the pathways to suicidal behavior. This view is supported by at least 3 converging lines of evidence. The first line is the association of suicidal behavior with problem gambling17 and drug addiction,18 behaviors defined by disadvantageous choice. In the second line, several although not all studies find that significant subgroups of individuals with a history of suicide attempts lose on gambling tasks, fail to learn from the experience of rewards and punishments,11,1922 and make shortsighted choices.23 In the third line, suicidal behavior has been linked to disruptions of signaling pathways in the basal ganglia,2430 disruptions that severely and somewhat selectively impair decision-making processes.3135 Akin to the analysis of suicide as an escape from self by Baumeister36 and the entrapment theory of Williams et al,37 one can view suicidal behavior as instrumental, aimed at achieving the subjectively preferred outcome in a desperate situation. Our interpretation of the evidence reviewed above is that at least some people who engage in suicidal behavior tend to misestimate future outcomes and see suicide as unrealistically attractive relative to other options. This perspective is supported by our behavioral findings of disrupted instrumental learning in older depressed people who attempt suicide (suicide attempters).38,39 To extend this evidence, we examined whether older depressed suicide attempters demonstrate disruptions in brain signals involved in reward prediction. We also considered whether impulsivity, a key dimension of the suicidal diathesis, exerts effects on these signals.

We focused on older adults because the rate of suicide is high in the elderly, and we believe that the age-dependent decline in decision competence40,41 puts this group at special risk. We view learning from rewards and punishments in a changing, ambiguous environment as a model of behavioral adaptation to change and uncertainty that accompanies a suicidal crisis. Consistent with this idea, a previous study38 observed a complete breakdown of reward learning in older depressed suicide attempters, with some suicide attempters overreacting to punishments and others displaying insensitivity to a contingency change. Older depressed suicide attempters also failed to grasp the changing contingency in an environment with no misleading feedback, the Wisconsin Card Sorting Task.39 This failure to grasp a changing contingency resembles the behavior of animals and humans with lesions of the ventromedial and ventrolateral prefrontal cortex (vmPFC and vlPFC).4244 To test whether the activity of these areas was disrupted during reward learning in older depressed suicide attempters, we conducted a functional magnetic resonance imaging (fMRI) study with participants facing uncertain and changing rewards and punishments on a probabilistic reversal learning task. To obtain more accurate and interpretable predictions of neural activity, we used a cognitive modeling approach, reinforcement learning, to estimate 2 key reward signals postulated by formal learning theory. The first signal is expected reward, and the second is the discrepancy between the experienced outcome and the prior expectation, or the prediction error.

We were further interested in how these neural signals are affected by impulsivity, a tendency strongly linked to suicidal behavior. We hypothesized that a history of suicide attempts and impulsivity would be associated with weaker expected reward signals in the vmPFC on the basis of studies mapping value representations to the vmPFC4550 and positron emission tomography findings of vmPFC alterations in suicide attempters51,52 and individuals with maladaptive impulsive behaviors.53

Overreactivity to negative feedback is typical of depression5458 and was very prominent among acutely depressed older suicide attempters in our behavioral study.38 On the probabilistic reversal learning task, depressed patients tend to switch response after a misleading punishment,58 as if forgetting the preceding history of rewards. According to learning theory,59,60 such responses have to result from a disproportionately big effect of negative prediction errors on choice. This effect could be caused by incorrect encoding of negative or preceding positive prediction errors or improper maintenance of expected reward representations. This reasoning led us to focus on the neural representation of prediction errors as a correlate of behavioral overreactivity to punishments in depression. A prior study61 has found altered error-related activity in the vlPFC of depressed individuals who overreacted to negative feedback during probabilistic reversal learning, although the study did not estimate prediction error signals. Studies using other reward-learning paradigms have found blunted prediction errors in the cingulate gyrus, striatum, and midbrain of depressed patients.62,63 However, these abnormalities were not linked to any particular behavioral tendency.



From March 28, 2008, through November 15, 2011, we recruited 53 participants 60 years or older, including 31 with major depression (15 suicide attempters and 18 with no lifetime history of suicidal behavior or ideation; we hereinafter refer to these 31 participants as depressed) and 20 psychiatrically healthy control subjects. Major depression was diagnosed by the Structured Clinical Interview for DSM-IV Axis I Disorders64,65 (SCID). Unlike participants in the previous behavioral study,38 all of whom had an acute depressive episode, participants in this study had different levels of depressive symptoms, from severe to partial remission (Table). To exclude individuals with clinical dementia and to ensure that participants could engage in the task, all were required to have a score of at least 24 on the Mini-Mental State Examination.66 Our reported sample excludes elderly individuals with sensory disorders that precluded cognitive testing, limited English, mental retardation, delirium, neurologic disorders, bipolar disorder, schizophrenia, schizoaffective disorder, exposure to electroconvulsive therapy in the previous 6 months, and circumstances precluding an fMRI assessment (eg, claustrophobia, metal implants [n = 7]). Participant flow is described in more detail in the material available on the authors’ website (

Table 1
Table Demographic, Clinical, and Cognitive Characteristics

All participants provided written informed consent. The University of Pittsburgh institutional review board approved the study.

Suicide attempters had engaged in a self-injurious act with an intent to die; 9 of 15 made their first suicide attempt after 50 years of age and 6 of them after 60 years of age. Fourteen of the 15 were recruited as inpatients, all of whom were admitted out of concern about suicide risk. These participants displayed a high level of suicidal intent (mean [SD] Suicide Intent Scale score, 18.1 [4.6]) during their attempts and severe suicidal ideation during the index episode (Scale for Suicide Ideation score, 23.9 [6.9]); 7 of 15 made repeated attempts. A history of suicide attempts was verified by a study psychiatrist (A.Y.D. or K.S.) based on the interview, medical records, and information from the treatment team and from family or friends. We excluded participants with significant discrepancies between these sources. None of the suicide attempters had experienced head injuries directly related to the attempt; however, we assessed potential anoxic-ischemic or toxic brain injury based on the Beck Lethality Scale,67 medical records, and the clinical interview. A study psychiatrist (A.Y.D.) identified any attempts with a score greater than 4 on the Beck Lethality Scale and any history of systemic hypotension of longer than 5 minutes, asphyxia, or neurotoxic ingestion in 3 of 15 participants. For 1 additional participant, we could not rule out brain injury during past attempts. Thus, we excluded these 4 participants in sensitivity analyses.

Nonsuicidal depressed elderly were included in the study to detect an association between decision making and suicidal behavior beyond the cognitive effects of depression. These participants had no current or lifetime history of suicide attempts or suicidal ideation as established by clinical interview, review of medical records, SCID, and the Scale for Suicidal Ideation (lifetime). One of the 18 nondepressed participants was recruited as an inpatient. Participants were excluded from this group if they had a current passive death wish or a history of indirect self-destructive behaviors.

Nondepressed controls were included as the benchmark group. They had no lifetime history of any psychiatric disorder as determined by SCID.

Clinical and cognitive assessments used to characterize the study groups and the 3-T MRI scanning parameters are described in detail on the authors’ website.

Probabilistic Reversal Learning

On each of the 300 trials of this instrumental learning task, participants chose between 2 pictures (eFigure 1 in the Supplement). One picture had a higher probability of reward when chosen (varied, .80 ≤ p ≤ .87), and the other, its additive inverse (.13 ≤ [1 − p] ≤ .20). In other words, even while choosing the right picture, participants received occasional misleading (probabilistic) negative feedback. Every 25 trials, the contingency (the good picture) changed without warning (reversal). On this task, one needs to trade off staying with the previously reinforced stimulus despite occasional misleading (probabilistic) feedback and switching when a true reversal occurs. The tendency to stay too long after reversal while ignoring negative feedback leads to perseverative errors. Conversely, the tendency to switch after a single misleading punishment results in probabilistic switch errors, previously linked to depression.58 We used the number of trials with misleading feedback during which the participant resisted this switch as a measure of resilience to it. Spontaneous or win/switch errors occurred when the subject switched away from a stimulus after being rewarded for choosing it. Because spontaneous errors are infrequent, their count is strongly and positively skewed with many values of 0. We also recorded response times, using their variation as an index of whether subjects were consistently on task or distracted at times. To ensure that even the most impaired participants are engaged in the task during scanning, we performed prior behavioral training with 4 reversals, and participants were no longer naive to the change in contingency. Thus, we were not expecting a complete breakdown of contingency learning in suicide attempters, unlike the earlier behavioral study with an unexpected reversal.38

Reinforcement Learning Model

To estimate abstract reward signals from participants’ reinforcement history and behavior, we adapted the Rescorla-Wagner reinforcement learning model, as in the earlier behavioral study.38 On this task, expected reward or expected value tracks the certainty of reward for the best available action. Here, the model’s estimate of expected value is high when the choice of one stimulus is almost certain to be rewarded. Meanwhile, a low expected value means ambiguity about which stimulus is likely to be rewarded. Prediction error is the discrepancy between the prior expectation of reward and the reward actually received on this trial. Prediction errors drive the updates of expected value after feedback. A reward for choosing a stimulus with a low prior expected value produces a positive prediction error and increases expected value. The omission of reward for a stimulus with a previously high expected value produces a negative prediction error and decreases expected value.

Measures of Impulsivity

We were interested in how neural indices of reward learning are affected by impulsivity for 2 reasons. First, strong evidence implicates impulsivity in the suicidal diathesis.5 Second, impulsivity involves a failure of higher-order control, which could plausibly undermine reward learning in an uncertain, changing environment. Assessments of impulsivity included the Impulsive/Careless Style subscale of the Social Problem-Solving Inventory,68,69 the Nonplanning and Attention/Cognitive subscales of the Barratt Impulsiveness Scale,70 and bets against the odds on the Cambridge Gamble Task (CGT),21 all found to be related to suicidal behavior.

Bets against the odds on the CGT, observed in older suicide attempters, reflect a neglect of information about outcome probability,21 possibly owing to a gambler’s fallacy or belief in a lucky streak. On each trial of the CGT, the participant is presented with an array of 10 boxes at the top of the screen, with each colored red or blue. The ratio of red to blue boxes varies from 1:9 to 9:1, in a pseudorandom order. The participant is instructed that the computer has hidden a token in 1 of the boxes and is told to register a guess as to whether the token is hidden under a red or a blue box by selecting 1 of the 2 corresponding colored panels. Given that the proportions of majority vs minority bets tend to follow a nonlinear, beta distribution, we dichotomized all participants into those who always bet on the majority color (good choices) vs those who sometimes bet on the minority color (bad choices). Of the 33 depressed participants included in the analysis, 2 were missing Social Problem-Solving Inventory data, 6 were missing Barratt Impulsiveness Scale data, and 9 were missing CGT data.

Statistical Analysis

Blood Oxygenation Level–Dependent Signal

Given the age-related variance in the hemodynamic response function (HRF), an empirical HRF was estimated in the control group using the (unbiased) switch/stay contrast in the vlPFC,71 as detailed on the authors’ website and in the Supplement (eFigure 2). This empirical HRF was then convolved with the expected value, positive prediction error, and negative prediction error signals from the reinforcement learning model for each subject. Voxelwise blood oxygenation level-dependent (BOLD) signal was regressed on these estimates in single-variable single-subjects analyses using an AFNI program (3dDeconvolve).72 Group differences were estimated by regressing the beta weights against 2 factors—depression and history of suicide attempts—across participants using another AFNI program (3dRegAna). To understand the variation in the HRF across regions, we also obtained estimates of the impulse response function for expected value signals using the 3dDeconvolve program with no assumed HRF and extracted the time series of resulting beta weights for each of the nondepressed controls.

To control type I error, we thresholded voxelwise tests at P < .005 and cluster thresholded them using Monte Carlo simulations based on the spatial autocorrelation of derived maps using the AFNI programs 3dFWHMx and 3dAlphaSim72,73 (67 voxels yielding P < .05, corrected). To estimate effects within the anatomical vmPFC mask from the Talairach atlas, 8 voxels at P < .005 controlled type I error at P < .05 with small-volume correction.

Correlations With Behavior and Self-Report: Extracting Network Activity Indices

To test the relationship between neural responses to expected value and prediction error on one hand and behavioral, clinical, and self-reported measures of impulsivity on the other, we conducted 2 independent analyses described in detail on the authors’ website. In the first, we mapped the networks responsive to expected value and prediction error in healthy controls (n = 20). In the second, we estimated the responses of these networks in the independent group of depressed participants with and without suicide attempts (n = 33) and correlated them with the measures of interest. In these correlational analyses, we chose to examine summary activations across networks responsive to expected value and prediction error instead of activations in single regions of interest to reduce dimensionality and control type I error. This choice was dictated by our initial analyses showing that responses to expected value and prediction error were distributed across large networks and were strongly intercorrelated across regions within each network (Tables 1 and 2 on the authors’ website). Although some regions were less active in suicide attempters and in depressed participants in general regardless of the history of suicide attempts as discussed in the Results section, we found no regions that were more active in the 2 depressed groups than in the controls. Thus, by applying the masks from the controls to the depressed participants, we did not leave out any ostensibly relevant regions.

In an exploratory analysis (described on the authors’ website along with the sensitivity analyses), we examined whether paralimbic structures encoded expected reward when the choices were being made vs when the reward/punishment was received after the choice. Given the low temporal resolution of fMRI, we tested this by comparing the timing of paralimbic response with that of the ostensibly choice-related response in the cortical task networks sensitive to ambiguity.


Suicide Attempts, Impulsivity, and Value Signals

In within-group analyses, controls demonstrated a positively modulated BOLD signal in the vmPFC (Brodmann areas [BAs] 10, 32, and 24) and other paralimbic structures, including the midcingulate cortex (BA 24), precuneus/posterior cingulate (BAs 7 and 31), and posterior insula for high expected reward (Figure 1A [in blue] and Table 1 on the authors’ website). In the whole-brain group comparison, as hypothesized, the pregenual cingulate subregion of the vmPFC (BA 32, 24, and 25 on the posterior periphery of the reward-modulated vmPFC region of interest) tracked the reinforcement history less reliably in suicide attempters than in the comparison groups, controlling for depression group status (P < .05, corrected; Figure 1B).

Figure 1
Modulation by Expected Reward

Correlations With Task Performance

Depressed participants whose paralimbic structures were less responsive to expected reward were more likely to ignore negative feedback after reversal (r33 = 0.47 [P = .006]; after excluding 1 outlier, r32 = 0.64 [P < .001]). However, their learning was otherwise relatively preserved, with no statistically reliable relationship between weak paralimbic modulation and inability to track the contingency, probabilistic, or spontaneous switches (r33 ≤ 0.24 [P ≥ .28]) (Figure 2A–C).

Figure 2
Impulsivity and Paralimbic Expected Reward Signals in Depressed Participants

Correlations With Impulsivity Measures

Confirming our hypothesis in depressed participants, weak paralimbic response to high expected reward was related to nonplanning impulsivity (r27 = −0.43 [P = .03]; after excluding 1 outlier, r26 = −0.50 [P = .01]) and bad choices on the CGT (ρ27 = 0.50 [P = .01]). The correlation with impulsive/careless problem-solving style was not significant (r31 = −0.20 [P = .29]) but became more apparent after excluding one outlier (r30 = −0.43 [P = .02]). We found no relationship with delay discounting (r23 = −0.10 [P = .64]). Among suicide attempters, weak paralimbic modulation was related to poor attempt planning (ρ15 = −0.58 [P = .03]) and not significantly to low attempt lethality (ρ15 = −0.48 [P = .07]) (Figure 2D–F and Supplement [eFigure 3]).

Timing of Reward Signals in the Paralimbic vs Other Cortical Networks

Did paralimbic structures represent the expected reward signal when the actual choice was being made? Our analysis of BOLD waveforms seems to suggest otherwise. Although the opercularinsular, dorsomedial PFC (dmPFC), and lateral-parietal responses to ambiguity peaked at a time point consistent with choice (Figure 3, in red), responses to high expected reward in the vmPFC and precuneus peaked earlier (Figure 3, in blue; repeated-measures analysis of variance, F99,228 = 9.3 [P < .001]). This peak fell approximately between feedback and the next choice. Thus, paralimbic structures appeared to broadly integrate reinforcement history rather than compare the value of presented options or select between actions.

Figure 3
Time Course of Reward Signals in Healthy Controls

Depression, Overreactivity to Punishments, and Prediction Error Signals

Unexpected rewards (positive prediction error) modulated the same networks as ambiguity, plus the midcingulate cortex, striatum, and thalamus (Figure 4A and Table 2 on the authors’ website). The behavioral overreactivity of depressed individuals to negative feedback was related to blunted representation of positive prediction errors in the cingulo-opercular and frontoparietal networks (r33 = −0.40 [P = .02]). As a group, depressed participants displayed weaker modulation by positive prediction error in the right thalamus, bilateral superior temporal gyrus (BA 22 and 39), bilateral operculoinsular cortex (BA 13, 45, and 46), bilateral postcentral gyrus (BA 40), and the bilateral supplementary motor area (P < .05, corrected; Figure 4B), and these effects were not related to a history of suicide attempts.

Figure 4
Modulation by Positive Prediction Error

Sensitivity Analyses

Group differences in expected reward and prediction error representations were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular pathological features, incipient dementia, sex, and model fits (see material on the authors’ website). Only depressive severity was a significant covariate, explaining additional variance in expected reward signal in the pericallosal cingulate cortex.


Our study of reward learning in late-life depression uncovered 2 neural patterns associated with disrupted control over reward-guided behavior. The first pattern, manifesting in a blunted modulation of paralimbic structures by expected reward, is associated with impulsivity and attempted suicide. The second pattern, associated with depression, is expressed in a weak thalamocorticostriatal response to unpredicted rewards and in maladaptive overreactivity to punishments. We discuss them in that order.

Reward Value Representations in the Paralimbic Structures, Impulsivity, and Attempted Suicide

In a dynamic environment, paralimbic structures guide approach behavior by tracking the value of available options.45,46 This function is sometimes related to the hedonic aspect of pursuing rewards (liking).45,74,75 In the impulsive depressed elderly in our study and in depressed suicide attempters, the paralimbic structures failed to track the reinforcement history accurately. This failure was seen mostly in patients who had made unplanned suicide attempts, again linking this anomaly to impulsive behavior. Participants displaying such a failure had the tendency to perseverate in approach behavior even when it no longer paid off, suggesting that they did not update the value of the option that was no longer rewarded. This interpretation is consistent with disrupted tracking of expected value by the paralimbic structures, which was associated with betting against the odds on a gambling task, a behavior previously observed in older suicide attempters.21 In both cases, impulsive individuals appear to be ignoring key information—odds (CGT) or reinforcement history (reversal learning)—and experiencing losses as a result. Our findings of the relationship among paralimbic structures’ expected reward signals, trait impulsivity, and bad choices on the CGT converge with those of Cox and colleagues,76 who found that vmPFC-insula resting-state functional connectivity was related to disadvantageous risk seeking. Our observations bear a curious resemblance to the behavioral and neural patterns seen in long-term cocaine users77,78 and in impulsive pathological gamblers79 on the reversal learning task. A similar pattern of perseveration has been produced by long-term cocaine administration in monkeys80 and by its withdrawal in rats.81 Whether these populations experience a primary disruption in the paralimbic cortex itself or in its mesolimbic input remains unclear.82,83 We found little support for the primary cortical deficit account from the literature concerning lesions in monkeys. Although lesions of the medial occipital-frontal cortex induce perseverative responding on a pavlovian reversal learning task,84 their effects on instrumental behavior are more complex and not limited to perseveration.85 Meanwhile, the fact that such perseveration can be induced and ameliorated86 by monoaminergic manipulation points toward a source in the ascending reward pathways.

Depression, Positive Prediction Errors, and Overreactivity to Punishment

Depressed individuals overreact to negative feedback, making error after error.5458 Abnormal response to errors in prefrontal areas, such as the vlPFC and the dmPFC, has been suggested to underlie this tendency.61 Extending that finding, we show that this behavior is also associated with blunted response of the cingulo-opercular and frontoparietal networks to positive prediction errors. Notably, this blunting was seen on a different set of trials: those with surprising positive feedback rather than misleading negative feedback. This effect may be best explained in terms of competition between 2 controllers of behavior. A coherent neurobiological account of such competition is offered by the rodent model of controllable vs uncontrollable stress used by Amat et al.87 In the controllable condition, once the animal encodes the action/reinforcement contingency, resulting cortical safety signals inhibit the primitive brainstem and limbic stress response. Conversely, weaker encoding of unpredicted rewards in depressed patients may fail to suppress a primitive response to noncontingent punishment, resembling the state of learned helplessness. Depressed individuals may thus experience punishments as uncontrollable, shifting toward a primitive, subcortical lose-switch policy instead of strategic choice based on reinforcement history. A plausible computational account of depressive overreactivity to punishments is that a primitive, automatic pavlovian controller responding to an aversive stimulus may override the more sophisticated controllers of instrumental learning. Pavlovian influences on instrumental behavior have been invoked to explain a range of disadvantageous approach-avoidance responses88 and appear to depend on limbic structures, such as the basolateral amygdala89 and the ventral striatum.90


Besides the cross-sectional, case-control design and a relatively small sample, several other design weaknesses limit our findings. Imaging assessments are too demanding for our sickest patients. Therefore, being unable to include these patients, we probably lack the opportunity to observe all the neural correlates of suicidal behavior. Because some of our patients were in partial remission, our findings may not apply to the most acutely depressed individuals. With respect to external validity, one can be more confident in the mechanistic findings linking impulsivity and overreactivity to punishment to specific neural aberrations than in group differences as such. Furthermore, expected reward is confounded with (lower) ambiguity and risk in reversal learning, limiting the interpretation of our findings. In addition, because depression status was confounded with antidepressant exposure, we cannot conclusively separate the two, although sensitivity analyses suggest that our results are robust. Also, our correlational study cannot disentangle the exact structure of relationships among the altered paralimbic reward signals, behaviors on reversal learning and gambling tasks, and self-reported impulsivity. Finally, the delta-rule model of associative learning explains only the most basic aspects of human brain activity during reversal learning (additional discussion can be found on the authors’ website). More sophisticated models49,91,92 might provide additional insight into anomalous reward learning in impulsive depressed elderly.

In summary, we found that in older adults, unplanned suicide attempts and impulsivity were associated with disrupted paralimbic tracking of expected reward. Depression was associated with disrupted corticostriatothalamic encoding of unexpected rewards.

Supplementary Material



Funding/Support: The study was supported by grants K23MH086620, K23MH070471, and R01MH085651 from the National Institute of Mental Health and by the American Foundation for Suicide Prevention.

Role of the Sponsors: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.


Previous Presentation: Some of the results reported herein were previously presented at the 50th Annual Meeting of the American College of Neuropsychopharmacology; December 7, 2011; Waikoloa, Hawaii.

Additional Contributions: Michael Hallquist, PhD, and Henry Chase, PhD, provided critical comments on the manuscript. Mandy Collier, BS, contributed to data collection and processing. Swathi Gujral, BS, Natalie Truty, BS, and Cori Shollenberger, BS, assisted with recruitment and assessments. Jan Kalkus, BS, provided help with graphics. Howard Aizenstein, MD, PhD, contributed to the structural imaging protocol.

Author Contributions: Dr Dombrovski had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Dombrovski, Clark, Reynolds, Siegle.

Acquisition of data: Dombrovski, Szanto, Reynolds, Siegle.

Analysis and interpretation of data: Dombrovski, Clark, Siegle.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: Dombrovski, Clark, Siegle.

Statistical analysis: Dombrovski, Siegle.

Obtained funding: Dombrovski, Clark.

Administrative, technical, and material support: Dombrovski, Reynolds.

Study supervision: Dombrovski, Szanto, Reynolds, Siegle.


1. Keilp JG, Gorlyn M, Oquendo MA, et al. Aggressiveness, not impulsiveness or hostility, distinguishes suicide attempters with major depression. Psychol Med. 2006;36(12):1779–1788. [PubMed]
2. McGirr A, Renaud J, Bureau A, Seguin M, Lesage A, Turecki G. Impulsive-aggressive behaviours and completed suicide across the life cycle: a predisposition for younger age of suicide. Psychol Med. 2008;38(3):407–417. [PubMed]
3. New AS, Gelernter J, Goodman M, et al. Suicide, impulsive aggression, and HTR1B genotype. Biol Psychiatry. 2001;50(1):62–65. [PubMed]
4. Swann AC, Dougherty DM, Pazzaglia PJ, Pham M, Steinberg JL, Moeller FG. Increased impulsivity associated with severity of suicide attempt history in patients with bipolar disorder. Am J Psychiatry. 2005;162(9):1680–1687. [PubMed]
5. Turecki G. Dissecting the suicide phenotype: the role of impulsive-aggressive behaviours. J Psychiatry Neurosci. 2005;30(6):398–408. [PMC free article] [PubMed]
6. Placidi GP, Oquendo MA, Malone KM, Huang YY, Ellis SP, Mann JJ. Aggressivity, suicide attempts, and depression: relationship to cerebrospinal fluid monoamine metabolite levels. Biol Psychiatry. 2001;50(10):783–791. [PubMed]
7. Gunnell D, Magnusson PK, Rasmussen F. Low intelligence test scores in 18 year old men and risk of suicide: cohort study. BMJ. 2005;330(7484):167. doi: 10.1136/bmj.38310.473565.8F. [PMC free article] [PubMed] [Cross Ref]
8. Keilp JG, Gorlyn M, Oquendo MA, Burke AK, Mann JJ. Attention deficit in depressed suicide attempters. Psychiatry Res. 2008;159(1–2):7–17. [PMC free article] [PubMed]
9. Keilp JG, Sackeim HA, Brodsky BS, Oquendo MA, Malone KM, Mann JJ. Neuropsychological dysfunction in depressed suicide attempters. Am J Psychiatry. 2001;158(5):735–741. [PubMed]
10. Bartfai A, Winborg IM, Nordström P, Asberg M. Suicidal behavior and cognitive flexibility: design and verbal fluency after attempted suicide. Suicide Life Threat Behav. 1990;20(3):254–266. [PubMed]
11. Jollant F, Bellivier F, Leboyer M, et al. Impaired decision making in suicide attempters. Am J Psychiatry. 2005;162(2):304–310. [PubMed]
12. Westheide J, Quednow BB, Kuhn KU, et al. Executive performance of depressed suicide attempters: the role of suicidal ideation. Eur Arch Psychiatry Clin Neurosci. 2008;258(7):414–421. [PubMed]
13. Cha CB, Najmi S, Park JM, Finn CT, Nock MK. Attentional bias toward suicide-related stimuli predicts suicidal behavior. J Abnorm Psychol. 2010;119(3):616–622. [PMC free article] [PubMed]
14. Beck AT, Kovacs M, Weissman A. Hopelessness and suicidal behavior: an overview. JAMA. 1975;234(11):1146–1149. [PubMed]
15. Young MA, Fogg LF, Scheftner W, Fawcett J, Akiskal H, Maser J. Stable trait components of hopelessness: baseline and sensitivity to depression. J Abnorm Psychol. 1996;105(2):155–165. [PubMed]
16. Szanto K, Reynolds CF, III, Conwell Y, Begley AE, Houck P. High levels of hopelessness persist in geriatric patients with remitted depression and a history of attempted suicide. J Am Geriatr Soc. 1998;46(11):1401–1406. [PubMed]
17. Wong PW, Cheung DY, Conner KR, Conwell Y, Yip PS. Gambling and completed suicide in Hong Kong: a review of coroner court files. Prim Care Companion J Clin Psychiatry. 2010;12(6) doi: 10.4088/PCC.09m00932blu. PCC.09m00932. [PubMed] [Cross Ref]
18. Vijayakumar L, Kumar MS, Vijayakumar V. Substance use and suicide. Curr Opin Psychiatry. 2011;24(3):197–202. [PubMed]
19. Jollant F, Lawrence NS, Olie E, et al. Decreased activation of lateral orbitofrontal cortex during risky choices under uncertainty is associated with disadvantageous decision-making and suicidal behavior. Neuroimage. 2010;51(3):1275–1281. [PubMed]
20. Malloy-Diniz LF, Neves FS, Abrantes SS, Fuentes D, Corrêa H. Suicide behavior and neuropsychological assessment of type I bipolar patients. J Affect Disord. 2009;112(1–3):231–236. [PubMed]
21. Clark L, Dombrovski AY, Siegle GJ, et al. Impairment in risk-sensitive decision-making in older suicide attempters with depression. Psychol Aging. 2011;26(2):321–330. [PMC free article] [PubMed]
22. Martino DJ, Strejilevich SA, Torralva T, Manes F. Decision making in euthymic bipolar I and bipolar II disorders. Psychol Med. 2011;41(6):1319–1327. [PubMed]
23. Dombrovski AY, Szanto K, Siegle GJ, et al. Lethal forethought: delayed reward discounting differentiates high- and low-lethality suicide attempts in old age. Biol Psychiatry. 2011;70(2):138–144. [PMC free article] [PubMed]
24. Ahearn EP, Jamison KR, Steffens DC, et al. MRI correlates of suicide attempt history in unipolar depression. Biol Psychiatry. 2001;50(4):266–270. [PubMed]
25. Vang FJ, Ryding E, Träskman-Bendz L, van Westen D, Lindström MB. Size of basal ganglia in suicide attempters, and its association with temperament and serotonin transporter density. Psychiatry Res. 2010;183(2):177–179. [PubMed]
26. Di Maio L, Squitieri F, Napolitano G, Campanella G, Trofatter JA, Conneally PM. Suicide risk in Huntington’s disease. J Med Genet. 1993;30(4):293–295. [PMC free article] [PubMed]
27. Foncke EM, Schuurman PR, Speelman JD. Suicide after deep brain stimulation of the internal globus pallidus for dystonia. Neurology. 2006;66(1):142–143. [PubMed]
28. Voon V, Krack P, Lang AE, et al. A multicentre study on suicide outcomes following subthalamic stimulation for Parkinson’s disease. Brain. 2008;131(pt 10):2720–2728. [PMC free article] [PubMed]
29. Dombrovski AY, Siegle GJ, Szanto K, Clark L, Reynolds CF, Aizenstein H. The temptation of suicide: striatal gray matter, discounting of delayed rewards, and suicide attempts in late-life depression [published online October 17, 2011] Psychol Med. 2012;42(6):1203–1215. doi: 10.1017/S0033291711002133. [PMC free article] [PubMed] [Cross Ref]
30. Soulas T, Gurruchaga JM, Palfi S, Cesaro P, Nguyen JP, Fenelon G. Attempted and completed suicides after subthalamic nucleus stimulation for Parkinson’s disease. J Neurol Neurosurg Psychiatry. 2008;79(8):952–954. [PubMed]
31. Housden CR, O’Sullivan SS, Joyce EM, Lees AJ, Roiser JP. Intact reward learning but elevated delay discounting in Parkinson’s disease patients with impulsive-compulsive spectrum behaviors. Neuropsychopharmacology. 2010;35(11):2155–2164. [PMC free article] [PubMed]
32. Voon V, Reynolds B, Brezing C, et al. Impulsive choice and response in dopamine agonist-related impulse control behaviors. Psychopharmacology (Berl) 2010;207(4):645–659. [PMC free article] [PubMed]
33. Weintraub D, Siderowf AD, Potenza MN, et al. Association of dopamine agonist use with impulse control disorders in Parkinson disease. Arch Neurol. 2006;63(7):969–973. [PMC free article] [PubMed]
34. Hälbig TD, Tse W, Frisina PG, et al. Subthalamic deep brain stimulation and impulse control in Parkinson’s disease. Eur J Neurol. 2009;16(4):493–497. [PubMed]
35. Frank MJ, Samanta J, Moustafa AA, Sherman SJ. Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science. 2007;318(5854):1309–1312. [PubMed]
36. Baumeister RF. Suicide as escape from self. Psychol Rev. 1990;97(1):90–113. [PubMed]
37. Williams JMG, Crane C, Barnhofer T, Duggan D. Psychology and suicidal behaviour: elaborating the entrapment model. In: Hawton K, editor. Prevention and Treatment of Suicidal Behaviour: From Science to Practice. New York, NY: Oxford University Press; 2005. pp. 71–89.
38. Dombrovski AY, Clark L, Siegle GJ, et al. Reward/punishment reversal learning in older suicide attempters. Am J Psychiatry. 2010;167(6):699–707. [PMC free article] [PubMed]
39. McGirr A, Dombrovski AY, Butters MA, Clark L, Szanto K. Deterministic learning and attempted suicide among older depressed individuals: cognitive assessment using the Wisconsin Card Sorting Task. J Psychiatr Res. 2012;46(2):226–232. [PMC free article] [PubMed]
40. Samanez-Larkin GR, Kuhnen CM, Yoo DJ, Knutson B. Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J Neurosci. 2010;30(4):1426–1434. [PMC free article] [PubMed]
41. Denburg NL, Cole CA, Hernandez M, et al. The orbitofrontal cortex, real-world decision making, and normal aging. Ann N Y Acad Sci. 2007;1121:480–498. [PMC free article] [PubMed]
42. Fellows LK, Farah MJ. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain. 2003;126(pt 8):1830–1837. [PubMed]
43. Tsuchida A, Doll BB, Fellows LK. Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J Neurosci. 2010;30(50):16868–16875. [PubMed]
44. Hornak J, O’Doherty J, Bramham J, et al. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J Cogn Neurosci. 2004;16(3):463–478. [PubMed]
45. Grabenhorst F, Rolls ET. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci. 2011;15(2):56–67. [PubMed]
46. Rushworth MF, Noonan MP, Boorman ED, Walton ME, Behrens TE. Frontal cortex and reward-guided learning and decision-making. Neuron. 2011;70(6):1054–1069. [PubMed]
47. Gläscher J, Hampton AN, O’Doherty JP. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2009;19(2):483–495. [PMC free article] [PubMed]
48. Plassmann H, O’Doherty J, Rangel A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci. 2007;27(37):9984–9988. [PubMed]
49. Hampton AN, Bossaerts P, O’Doherty JP. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci. 2006;26(32):8360–8367. [PubMed]
50. Jocham G, Neumann J, Klein TA, Danielmeier C, Ullsperger M. Adaptive coding of action values in the human rostral cingulate zone. J Neurosci. 2009;29(23):7489–7496. [PMC free article] [PubMed]
51. Leyton M, Paquette V, Gravel P, et al. Alpha-[11C]methyl-L-tryptophan trapping in the orbital and ventral medial prefrontal cortex of suicide attempters. Eur Neuropsychopharmacol. 2006;16(3):220–223. [PubMed]
52. Oquendo MA, Placidi GP, Malone KM, et al. Positron emission tomography of regional brain metabolic responses to a serotonergic challenge and lethality of suicide attempts in major depression. Arch Gen Psychiatry. 2003;60(1):14–22. [PubMed]
53. Fineberg NA, Potenza MN, Chamberlain SR, et al. Probing compulsive and impulsive behaviors, from animal models to endophenotypes: a narrative review. Neuropsychopharmacology. 2010;35(3):591–604. [PMC free article] [PubMed]
54. Nelson RE, Craighead WE. Selective recall of positive and negative feedback, self-control behaviors, and depression. J Abnorm Psychol. 1977;86(4):379–388. [PubMed]
55. Steffens DC, Wagner HR, Levy RM, Horn KA, Krishnan KR. Performance feedback deficit in geriatric depression. Biol Psychiatry. 2001;50(5):358–363. [PubMed]
56. Elliott R, Sahakian BJ, Herrod JJ, Robbins TW, Paykel ES. Abnormal response to negative feedback in unipolar depression: evidence for a diagnosis specific impairment. J Neurol Neurosurg Psychiatry. 1997;63(1):74–82. [PMC free article] [PubMed]
57. Elliott R, Sahakian BJ, McKay AP, Herrod JJ, Robbins TW, Paykel ES. Neuropsychological impairments in unipolar depression: the influence of perceived failure on subsequent performance. Psychol Med. 1996;26(5):975–989. [PubMed]
58. Murphy FC, Michael A, Robbins TW, Sahakian BJ. Neuropsychological impairment in patients with major depressive disorder: the effects of feedback on task performance. Psychol Med. 2003;33(3):455–467. [PubMed]
59. Bush RR, Mosteller F. Stochastic Models for Learning. Oxford, England: John Wiley & Sons Inc; 1955.
60. Rescorla RA, Wagner AR. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical Conditioning II. New York, NY: Appleton-Century-Crofts; 1972. pp. 64–99.
61. Taylor Tavares JV, Clark L, Furey ML, Williams GB, Sahakian BJ, Drevets WC. Neural basis of abnormal response to negative feedback in unmedicated mood disorders. Neuroimage. 2008;42(3):1118–1126. [PMC free article] [PubMed]
62. Kumar P, Waiter G, Ahearn T, Milders M, Reid I, Steele JD. Abnormal temporal difference reward-learning signals in major depression. Brain. 2008;131(pt 8):2084–2093. [PubMed]
63. Gradin VB, Kumar P, Waiter G, et al. Expected value and prediction error abnormalities in depression and schizophrenia. Brain. 2011;134(pt 6):1751–1764. [PubMed]
64. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4. Washington, DC: American Psychiatric Association; 1994.
65. First M, Sr, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV Axis I Disorders–Patient Edition (SCID-I/P), Version 2. New York: Biometrics Research Dept, New York State Psychiatric Institute; 1995.
66. Folstein MF, Folstein SE, McHugh PR. “Mini-Mental State”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–198. [PubMed]
67. Beck AT, Beck R, Kovacs M. Classification of suicidal behaviors, I: quantifying intent and medical lethality. Am J Psychiatry. 1975;132(3):285–287. [PubMed]
68. Gibbs LM, Dombrovski AY, Morse J, Siegle GJ, Houck PR, Szanto K. When the solution is part of the problem: problem solving in elderly suicide attempters. Int J Geriatr Psychiatry. 2009;24(12):1396–1404. [PMC free article] [PubMed]
69. D’Zurilla TJ, Nezu AM. Development and preliminary evaluation of the Social Problem-Solving Inventory: psychological assessment. J Consult Clin Psychol. 1990;2(2):156–163.
70. Patton JH, Stanford MS, Barratt ES. Factor structure of the Barratt Impulsiveness Scale. J Clin Psychol. 1995;51(6):768–774. [PubMed]
71. Cools R, Clark L, Owen AM, Robbins TW. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 2002;22(11):4563–4567. [PubMed]
72. Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res. 1996;29(3):162–173. [PubMed]
73. Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med. 1995;33(5):636–647. [PubMed]
74. Kringelbach ML. The human orbitofrontal cortex: linking reward to hedonic experience. Nat Rev Neurosci. 2005;6(9):691–702. [PubMed]
75. Peters J, Büchel C. Neural representations of subjective reward value. Behav Brain Res. 2010;213(2):135–141. [PubMed]
76. Cox CL, Gotimer K, Roy AK, Castellanos FX, Milham MP, Kelly C. Your resting brain CAREs about your risky behavior. PLoS One. 2010;5(8):e12296. doi: 10.1371/journal.pone.0012296. [PMC free article] [PubMed] [Cross Ref]
77. Fernández-Serrano MJ, Perales JC, Moreno-López L, Pérez-García M, Verdejo-García A. Neuropsychological profiling of impulsivity and compulsivity in cocaine dependent individuals. Psychopharmacology (Berl) 2012;219(2):673–683. [PubMed]
78. Ersche KD, Roiser JP, Robbins TW, Sahakian BJ. Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology (Berl) 2008;197(3):421–431. [PMC free article] [PubMed]
79. de Ruiter MB, Veltman DJ, Goudriaan AE, Oosterlaan J, Sjoerds Z, van den Brink W. Response perseveration and ventral prefrontal sensitivity to reward and punishment in male problem gamblers and smokers. Neuropsychopharmacology. 2009;34(4):1027–1038. [PubMed]
80. Jentsch JD, Olausson P, De La Garza R, II, Taylor JR. Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology. 2002;26(2):183–190. [PubMed]
81. Calu DJ, Stalnaker TA, Franz TM, Singh T, Shaham Y, Schoenbaum G. Withdrawal from cocaine self-administration produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats. Learn Mem. 2007;14(5):325–328. [PMC free article] [PubMed]
82. Chase HW, Clark L. Gambling severity predicts midbrain response to near-miss outcomes. J Neurosci. 2010;30(18):6180–6187. [PMC free article] [PubMed]
83. Stalnaker TA, Roesch MR, Calu DJ, Burke KA, Singh T, Schoenbaum G. Neural correlates of inflexible behavior in the orbitofrontal-amygdalar circuit after cocaine exposure. Ann N Y Acad Sci December. 2007;1121:598–609. [PMC free article] [PubMed]
84. Chudasama Y, Robbins TW. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci. 2003;23(25):8771–8780. [PubMed]
85. Noonan MP, Walton ME, Behrens TE, Sallet J, Buckley MJ, Rushworth MF. Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc Natl Acad Sci U S A. 2010;107(47):20547–20552. [PubMed]
86. Ersche KD, Roiser JP, Abbott S, et al. Response perseveration in stimulant dependence is associated with striatal dysfunction and can be ameliorated by a D(2/3) receptor agonist. Biol Psychiatry. 2011;70(8):754–762. [PubMed]
87. Amat J, Baratta MV, Paul E, Bland ST, Watkins LR, Maier SF. Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat Neurosci. 2005;8(3):365–371. [PubMed]
88. De Martino B. The effect of context on choice and value. In: Sharot T, Dolan RJ, editors. Neuroscience of Preference and Choice. London, England: Academic Press; 2012. pp. 97–106.
89. Corbit LH, Balleine BW. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J Neurosci. 2005;25(4):962–970. [PubMed]
90. Talmi D, Seymour B, Dayan P, Dolan RJ. Human pavlovian-instrumental transfer. J Neurosci. 2008;28(2):360–368. [PMC free article] [PubMed]
91. Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS. Learning the value of information in an uncertain world. Nat Neurosci. 2007;10(9):1214–1221. [PubMed]
92. O’Reilly RC, Frank MJ, Hazy TE, Watz B. PVLV: the primary value and learned value Pavlovian learning algorithm. Behav Neurosci. 2007;121(1):31–49. [PubMed]