The neurobehavioral underpinnings of pathological gambling are not well understood. Insight might be gained by understanding pharmacological effects on the reward system in patients with Parkinson’s disease (PD). Treatment with dopamine agonists (DAs) has been associated with pathological gambling in PD patients. However, how DAs are involved in the development of this form of addiction is unknown. We tested the hypothesis that tonic stimulation of dopamine receptors specifically desensitizes the dopaminergic reward system by preventing decreases in dopaminergic transmission that occurs with negative feedback. Using functional magnetic resonance imaging, we studied PD patients during three sessions of a probabilistic reward task in random order: off medication, after levodopa (LD) treatment, and after an equivalent dose of DA (pramipexole). For each trial, a reward prediction error value was computed using outcome, stake, and probability. Pramipexole specifically changed activity of the orbitofrontal cortex (OFC) in two ways that were both associated with increased risk taking in an out-of-magnet task. Outcome-induced activations were generally higher with pramipexole compared with LD or off medication. In addition, only pramipexole greatly diminished trial-by-trial correlation with reward prediction error values. Further analysis yielded that this resulted mainly from impaired deactivation in trials with negative errors in reward prediction. We propose that DAs prevent pauses in dopamine transmission and thereby impair the negative reinforcing effect of losing. Our findings raise the question of whether pathological gambling may in part stem from an impaired capacity of the OFC to guide behavior when facing negative consequences.
PMID: 19741594 CAMSID: cams1534
fMRI; impulse control disorder; dopamine agonist; reward; addiction; reinforcement
The primary motor cortex is important for motor learning and response selection, functions that require information on the expected and actual outcomes of behavior. Therefore, it should receive signals related to reward and pathways from reward centers to motor cortex exist in primates. Previously, we showed that gamma aminobutyric acid-A(GABAA)-mediated inhibition in motor cortex, measured by paired transcranial magnetic stimulation (TMS), changes with expectation and uncertainty of money rewards generated by a slot machine simulation.
We examined the role of dopamine in this phenomenon by testing 13 mildly affected Parkinson disease patients, off and on dopaminergic medications, and 13 healthy, age-matched controls.
Consistent with a dopaminergic mechanism, reward expectation or predictability modulated the response to paired TMS in controls, but not in unmedicated patients. A single dose of pramipexole restored this effect of reward, mainly by increasing the paired TMS response amplitude during low expectation. Levodopa produced no such effect. Both pramipexole and levodopa increased risk-taking behavior on the Iowa Gambling Task. However, pramipexole increased risk-taking behavior more in patients showing lower paired TMS response amplitude during low expectation.
These results provide evidence that modulation of motor cortex inhibition by reward is mediated by dopamine signaling and that physiological states in the motor cortex are associated with levels of risk-taking behavior in patients on pramipexole. The cortical response to reward expectation may represent an endophenotype for risk-taking behavior in patients on agonist treatment.
Transcranial magnetic stimulation (TMS); dopamine; gambling; motor cortex
Pramipexole and other direct dopamine agonist medications have been implicated in the development of impulsive behavior such as pathological gambling among those taking the drug to control symptoms of Parkinson’s disease or restless leg syndrome. Few laboratory studies examining pramipexole’s effects on gambling-like behavior have been conducted.
The present study used a rodent model approximating some aspects of human gambling to examine within-subject effects of acute pramipexole (0.03, 0.1, 0.18, & 0.3 mg/kg) on rat’s choices to earn food reinforcement by completing variable-ratio (i.e., gambling-like) or fixed-ratio response requirements.
In a condition in which the variable-ratio alternative was rarely selected, all but the lowest dose of pramipexole significantly increased choice of the variable-ratio alternative (an average of 15% above saline).. The same doses did not affect choice significantly in a control condition designed to evaluate the involvement of nonspecific drug effects. Pramipexole increased latencies to initiate trials (+ 9.12 s) and to begin response runs on forced-choice trials (variable-ratio: + 0.21 s; fixed-ratio: + 0.88 s), but did not affect measures of response perseveration (conditional probabilities of “staying”).
The findings are consistent with clinical reports linking pramipexole to the expression of increased gambling in humans. Results are discussed in the context of neurobehavioral evidence suggesting that dopamine agonists increase sensitivity to reward delay and disrupt appropriate feedback from negative outcomes.
pramipexole; dopamine agonist; gambling; impulsive behavior; Parkinson’s disease; rat
The dopaminergic system, particularly D2-like dopamine receptors, has been strongly implicated in reward processing. Animal studies have emphasized the role of phasic dopamine (DA) signaling in reward-related learning, but these processes remain largely unexplored in humans.
To evaluate the effect of a single, low dose of a D2/D3 agonist—pramipexole—on reinforcement learning in healthy adults. Based on prior evidence indicating that low doses of DA agonists decrease phasic DA release through autoreceptor stimulation, we hypothesized that 0.5 mg of pramipexole would impair reward learning due to presynaptic mechanisms.
Using a double-blind design, a single 0.5 mg dose of pramipexole or placebo was administered to 32 healthy volunteers, who performed a probabilistic reward task involving a differential reinforcement schedule as well as various control tasks.
As hypothesized, response bias toward the more frequently rewarded stimulus was impaired in the pramipexole group, even after adjusting for transient adverse effects. In addition, the pramipexole group showed reaction time and motor speed slowing and increased negative affect; however, when adverse physical side effects were considered, group differences in motor speed and negative affect disappeared.
These findings show that a single low dose of pramipexole impaired the acquisition of reward-related behavior in healthy participants, and they are consistent with prior evidence suggesting that phasic DA signaling is required to reinforce actions leading to reward. The potential implications of the present findings to psychiatric conditions, including depression and impulse control disorders related to addiction, are discussed.
Dopamine; D2 agonists; Reward Processing; Depression; Mesolimbic System; Addiction
This pharmacological fMRI study shows that during reward-based sensory decision-making, dopamine is crucially involved in reward-related modulation of human primary sensory cortex.
Reward-related dopaminergic influences on learning and overt behaviour are well established, but any influence on sensory decision-making is largely unknown. We used functional magnetic resonance imaging (fMRI) while participants judged electric somatosensory stimuli on one hand or other, before being rewarded for correct performance at trial end via a visual signal, at one of four anticipated financial levels. Prior to the procedure, participants received either placebo (saline), a dopamine agonist (levodopa), or an antagonist (haloperidol). Principal findings: higher anticipated reward improved tactile decisions. Visually signalled reward reactivated primary somatosensory cortex for the judged hand, more strongly for higher reward. After receiving a higher reward on one trial, somatosensory activations and decisions were enhanced on the next trial. These behavioural and neural effects were all enhanced by levodopa and attenuated by haloperidol, indicating dopaminergic dependency. Dopaminergic reward-related influences extend even to early somatosensory cortex and sensory decision-making.
The rewards one receives during decision-making has a profound impact on learning. Much recent interest has focused on the role of the neurotransmitter dopamine in the basal ganglia for influencing learning and behaviour. Here, we ask whether reward can influence low-level sensory processing, for instance in primary sensory cortex, and how dopamine mediates this process. We show in humans that dopamine level, as manipulated with a dopamine agonist and antagonist in a double-blind placebo-controlled design, is involved in reward modulation of primary somatosensory cortex. Higher anticipated reward improved tactile decisions, and receipt of visual reward signals reactivated primary somatosensory cortex for the judged hand as measured using functional neuroimaging. After receiving a higher reward on one trial, somatosensory activations and decisions were enhanced on the next trial, suggesting that reward outcome provides a form of teaching signal that may be fed back to task-relevant sensory cortex. All these behavioural and neural effects of reward on somatosensory decision-making were strongly modulated by the availability of dopamine as the mediating neurotransmitter. These findings raise the tantalising new possibility that reward manipulations in conjunction with dopaminergic drugs might be used to enhance pathologically deficient or lapsed sensory processes, analogous to how rewards can be used to shape or correct behaviour.
We recently reported that the D2/D3 agonist pramipexole may have pro-cognitive effects in euthymic patients with bipolar disorder (BPD); however, the emergence of impulse-control disorders has been documented in Parkinson's disease (PD) after pramipexole treatment. Performance on reward-based tasks is altered in healthy subjects after a single dose of pramipexole, but its potential to induce abnormalities in BPD patients is unknown. We assessed reward-dependent decision making in euthymic BPD patients pre- and post 8 weeks of treatment with pramipexole or placebo by using the Iowa Gambling Task (IGT). The IGT requires subjects to choose among four card decks (two risky and two conservative) and is designed to promote learning to make advantageous (conservative) choices over time. Thirty-four BPD patients completed both assessments (18 placebo and 16 pramipexole). Baseline performance did not differ by treatment group (F=0.63; p=0.64); however, at week 8, BPD patients on pramipexole demonstrated a significantly greater tendency to make increasingly high-risk, high-reward choices across the five blocks, whereas the placebo group's pattern was similar to that reported in healthy individuals (treatment × time × block interaction, p<0.05). Analyses of choice strategy using the expectancy valence model revealed that after 8 weeks on pramipexole, BPD patients attended more readily to feedback related to gains than to losses, which could explain the impaired learning. There were no significant changes in mood symptoms over the 8 weeks, and no increased propensity toward manic-like behaviors were reported. Our results suggest that the enhancement of dopaminergic activity influences risk-associated decision-making performance in euthymic BPD. The clinical implications remain unknown.
Behavioral Science; bipolar disorder; decision-making; Dopamine; gambling; Mood/Anxiety/Stress Disorders; Neuropharmacology; pramipexole; bipolar disorder; dopamine; pramipexole; decision-making
OBJECTIVE: To determine the frequency of new-onset compulsive gambling or hypersexuality among regional patients with Parkinson disease (PD), ascertaining the relationship of these behaviors to PD drug use.
PATIENTS AND METHODS: We retrospectively reviewed the medical records of patients from 7 rural southeastern Minnesota counties who had at least 1 neurology appointment for PD between July 1, 2004, and June 30, 2006. The main outcome measure was compulsive gambling or hypersexuality developing after parkinsonism onset, including the temporal relationship to PD drug use.
RESULTS: Of 267 patients with PD who met the study inclusion criteria, new-onset gambling or hypersexuality was documented in 7 (2.6%). All were among the 66 patients (10.6%) taking a dopamine agonist. Moreover, all 7 (18.4%) were among 38 patients taking therapeutic doses (defined as ≥2 mg of pramipexole or 6 mg of ropinirole daily). Behaviors were clearly pathologic and disabling in 5: 7.6% of all patients taking an agonist and 13.2% of those taking therapeutic doses. Of the 5 patients, 2 had extensive treatment for what was considered a primary psychiatric problem before the agonist connection was recognized.
CONCLUSION: Among the study patients with PD, new-onset compulsive gambling or hypersexuality was documented in 7 (18.4%) of 38 patients taking therapeutic doses of dopamine agonists but was not found among untreated patients, those taking subtherapeutic agonist doses, or those taking carbidopa/levodopa alone. Behaviors abated with discontinuation of agonist therapy or dose reduction. Because this is a retrospective study, cases may have been missed, and hence this study may reflect an underestimation of the true frequency. Physicians who care for patients taking these drugs should recognize the drug's potential to induce pathologic syndromes that sometimes masquerade as primary psychiatric disease.
In patients with Parkinson disease, new-onset compulsive gambling or hypersexuality was documented in 7 of 38 patients taking therapeutic doses of dopamine agonists but was not found among untreated patients, those taking subtherapeutic agonist doses, or those taking carbidopa/levodopa alone.
An influential model suggests that dopamine signals the difference between predicted and experienced reward. In this way, dopamine can act as a learning signal that can shape behaviors to maximize rewards and avoid punishments. Dopamine is also thought to invigorate reward seeking behavior. Loss of dopamine signaling is the major abnormality in Parkinson’s disease. Dopamine agonists have been implicated in the occurrence of impulse control disorders in Parkinson’s disease patients, the most common being pathological gambling, compulsive sexual behavior, and compulsive buying. Recently, a number of functional imaging studies investigating impulse control disorders in Parkinson’s disease have been published. Here we review this literature, and attempt to place it within a decision-making framework in which potential gains and losses are evaluated to arrive at optimum choices. We also provide a hypothetical but still incomplete model on the effect of dopamine agonist treatment on these value and risk assessments. Two of the main brain structures thought to be involved in computing aspects of reward and loss are the ventral striatum (VStr) and the insula, both dopamine projection sites. Both structures are consistently implicated in functional brain imaging studies of pathological gambling in Parkinson’s disease.
impulse control disorders; impulsivity; reward; loss aversion; insula; ventral striatum
Gambling is a common recreational activity that becomes dysfunctional in a subset of individuals, with DSM ‘pathological gambling’ regarded as the most severe form. During gambling, players experience a range of cognitive distortions that promote an over-estimation of the chances of winning. Near-miss outcomes are thought to fuel these distortions. We observed previously that near-misses recruited overlapping circuitry to monetary wins in a study in healthy volunteers (Clark et al. 2009). The present study sought to extend these observations in regular gamblers and relate brain responses to an index of gambling severity. Twenty regular gamblers, who varied in their involvement from recreational players to probable pathological gamblers, were scanned whilst performing a simplified slot-machine task that delivered occasional monetary wins, as well as near-miss and full-miss non-win outcomes. In the overall group, near-miss outcomes were associated with a significant response in the ventral striatum, which was also recruited by monetary wins. Gambling severity, measured with the South Oaks Gambling Screen, predicted a greater response in the dopaminergic midbrain to near-miss outcomes. This effect survived controlling for clinical co-morbidities that were present in the regular gamblers. Gambling severity did not predict win-related responses in the midbrain or elsewhere. These results demonstrate that near-miss events during gambling recruit reward-related brain circuitry in regular players. An association with gambling severity in the midbrain suggests that near-miss outcomes may enhance dopamine transmission in disordered gambling, which extends neurobiological similarities between pathological gambling and drug addiction.
Gambling; Cognitive; Addiction; Dopamine; Striatum; Midbrain
Dysregulation of mesolimbic dopamine transmission is implicated in a number of psychiatric illnesses characterised by disruption of reward processing and goal-directed behaviour, including schizophrenia, drug addiction and impulse control disorders associated with chronic use of dopamine agonists. Amphetamine sensitization (AS) has been proposed to model the development of this aberrant dopamine signalling and the subsequent dysregulation of incentive motivational processes. However, in humans the effects of AS on the dopamine-sensitive neural circuitry associated with reward processing remains unclear. Here we describe the effects of acute amphetamine administration, following a sensitising dosage regime, on blood oxygen level dependent (BOLD) signal in dopaminoceptive brain regions during a rewarded gambling task performed by healthy volunteers. Using a randomised, double-blind, parallel-groups design, we found clear evidence for sensitization to the subjective effects of the drug, while rewarded reaction times were unchanged. Repeated amphetamine exposure was associated with reduced dorsal striatal BOLD signal during decision making, but enhanced ventromedial caudate activity during reward anticipation. The amygdala BOLD response to reward outcomes was blunted following repeated amphetamine exposure. Positive correlations between subjective sensitization and changes in anticipation- and outcome-related BOLD signal were seen for the caudate nucleus and amygdala, respectively. These data show for the first time in humans that AS changes the functional impact of acute stimulant exposure on the processing of reward-related information within dopaminoceptive regions. Our findings accord with pathophysiological models which implicate aberrant dopaminergic modulation of striatal and amygdala activity in psychosis and drug-related compulsive disorders.
The nucleus accumbens (NAc) serves as an integral node within cortico-limbic circuitry that regulates various forms of cost–benefit decision making. The dopamine (DA) system has also been implicated in enabling organisms to overcome a variety of costs to obtain more valuable rewards. However, it remains unclear how DA activity within the NAc may regulate decision making involving reward uncertainty. This study investigated the contribution of different DA receptor subtypes in the NAc to risk-based decision making, assessed with a probabilistic discounting task. In well-trained rats, D1 receptor blockade with SCH 23 390 decreased preference for larger, uncertain rewards, which was associated with enhanced negative-feedback sensitivity (ie, an increased tendency to select a smaller/certain option after an unrewarded risky choice). Treatment with a D1 agonist (SKF 81 297) optimized decision making, increasing choice of the risky option when reward probability was high, and decreasing preference under low probability conditions. In stark contrast, neither blockade of NAc D2 receptors with eticlopride, nor stimulation of these receptors with quinpirole or bromocriptine influenced risky choice. In comparison, infusion of the D3-preferring agonist PD 128 907 decreased reward sensitivity and risky choice. Collectively, these results show that mesoaccumbens DA refines risk–reward decision biases via dissociable mechanisms recruiting D1 and D3, but not D2 receptors. D1 receptor activity mitigates the effect of reward omissions on subsequent choices to promote selection of reward options that may have greater long-term utility, whereas excessive D3 receptor activity blunts the impact that larger/uncertain rewards have in promoting riskier choices.
Animal models; Cognition; D1 receptor; D3 receptor; Dopamine; probabilistic discounting; Psychopharmacology; rat; ventral striatum; probabilistic discounting; D1 receptors; D3 receptors; rat; ventral striatum; neuroeconomics
In models of dopaminergic neuronal loss, the dopamine agonist pramipexole has exhibited neuroprotective properties. The Pramipexole On Underlying Disease (PROUD) study was designed to identify whether early versus delayed pramipexole initiation has clinical and neuroimaging benefits in patients with Parkinson's disease (PD).
Between May 24, 2006, and April 22, 2009, at 98 centres, we recruited patients with PD diagnosed within 2 years and aged 30–79 years. We randomly assigned eligible patients (ratio 1:1), by a centralised, computerised randomisation schedule, to receive double-blind either placebo or pramipexole (1·5 mg a day) and followed them up for 15 months. At 9 months, or as early as 6 months if considered necessary, placebo recipients were assigned to pramipexole. In a neuroimaging substudy, striatal dopamine-transporter binding was assessed by SPECT. All patients, investigators, and independent raters were masked to study treatment. The primary endpoint was the 15-month change from baseline in total score on the unified Parkinson's disease rating scale (UPDRS). This trial is registered with ClinicalTrials.gov, number NCT00321854.
Of 535 patients, 261 were randomly assigned to receive pramipexole and 274 to receive placebo. At 15 months (n=411), adjusted mean change in UPDRS total score showed no significant difference between early and delayed pramipexole (−0·4 points, 95% CI −2·2 to 1·4, p=0·65). 62 patients in the early pramipexole group and 61 patients in the delayed pramipexole group were included in the neuroimaging substudy, for which the adjusted mean 15-month change in striatal 123I-FP-CIT binding was −15·1% (SE 2·1) for early and −14·6% (2·0) for delayed pramipexole (difference −0·5 percentage points, 95% CI −5·4 to 4·4, p=0·84). Overall, 180 (81%) of patients given early pramipexole and 179 (84%) patients given delayed pramipexole reported adverse events (most frequently nausea), and 22 (10%) patients in the early pramipexole group and 17 (8%) in the delayed pramipexole group had serious events, two of which (hallucinations and orthostatic hypotension) were deemed related to study drug.
By clinical and neuroimaging measures, pramipexole showed little evidence differentiating 15-month usage from usage delayed for 6–9 months. The results do not support the hypothesis that pramipexole has disease-modifying effects.
Boehringer Ingelheim GmbH.
Parkinson's disease is characterized by the degeneration of dopaminergic pathways projecting to the striatum. These pathways are implicated in reward prediction. In this study, we investigated reward and punishment processing in young, never-medicated Parkinson's disease patients, recently medicated patients receiving the dopamine receptor agonists pramipexole and ropinirole and healthy controls. The never-medicated patients were also re-evaluated after 12 weeks of treatment with dopamine agonists. Reward and punishment processing was assessed by a feedback-based probabilistic classification task. Personality characteristics were measured by the temperament and character inventory. Results revealed that never-medicated patients with Parkinson's disease showed selective deficits on reward processing and novelty seeking, which were remediated by dopamine agonists. These medications disrupted punishment processing. In addition, dopamine agonists increased the correlation between reward processing and novelty seeking, whereas these drugs decreased the correlation between punishment processing and harm avoidance. Our finding that dopamine agonist administration in young patients with Parkinson's disease resulted in increased novelty seeking, enhanced reward processing, and decreased punishment processing may shed light on the cognitive and personality bases of the impulse control disorders, which arise as side-effects of dopamine agonist therapy in some Parkinson's disease patients.
Parkinson's disease; reward; novelty seeking; dopamine; pramipexole; ropinirole
Impulse control disorders such as pathological gambling (PG) are a serious and common adverse effect of dopamine (DA) replacement medication in Parkinson’s disease (PD). Patients with PG have increased impulsivity and abnormalities in striatal DA, in common with behavioural and substance addictions in the non-PD population. To date, no studies have investigated the role of extrastriatal dopaminergic abnormalities in PD patients with PG. We used the PET radiotracer, [11C] FLB-457, with high-affinity for extrastriatal DA D2/3 receptors. 14 PD patients on DA agonists were imaged while they performed a gambling task involving real monetary reward and a control task. Trait impulsivity was measured with the Barratt Impulsivity Scale (BIS). Seven of the patients had a history of PG that developed subsequent to DA agonist medication. Change in [11C] FLB-457 binding potential (BP) during gambling was reduced in PD with PG patients in the midbrain, where D2/D3 receptors are dominated by autoreceptors. The degree of change in [11C] FLB-457 binding in this region correlated with impulsivity. In the cortex, [11C] FLB-457 BP was significantly greater in the anterior cingulate cortex (ACC) in PD patients with PG during the control task, and binding in this region was also correlated with impulsivity. Our findings provide the first evidence that PD patients with PG have dysfunctional activation of DA autoreceptors in the midbrain and low DA tone in the ACC. Thus, altered striatal and cortical DA homeostasis may incur vulnerability for the development of PG in PD, linked with the impulsive personality trait.
PMID: 22766031 CAMSID: cams2373
Parkinson’s disease; Dopamine agonists; Pathological gambling; Impulsivity
Clinical reports, primarily with Parkinson’s patients, note an association between the prescribed use of pramipexole (and other direct-acting dopamine agonist medications) and impulse control disorders, particularly pathological gambling. Two experiments examined the effects of acute pramipexole on rats’ impulsive choices where impulsivity was defined as selecting a smaller-sooner over a larger-later food reward. In Experiment 1, pramipexole (0.1 to 0.3 mg/kg) significantly increased impulsive choices in a condition in which few impulsive choices were made during a stable baseline. In a control condition, in which impulsive choices predominated during baseline, pramipexole did not significantly change the same rats’ choices. Experiment 2 explored a wider range of doses (0.01 to 0.3 mg/kg) using a choice procedure in which delays to the larger-later reinforcer delivery increased across trial blocks within each session. At the doses used in Experiment 1, pramipexole shifted choice toward indifference regardless of the operative delay. At lower doses of pramipexole (0.01 & 0.03 mg/kg), a trend toward more impulsive choice was observed at the 0.03 mg/kg dose. The difference in outcomes across experiments may be due to the more complex discriminations required in Experiment 2; i.e., multiple discriminations between changing delays within each session.
Pramipexole; D2/D3 agonist; Impulsivity; Choice; Gambling
The dopamine agonists ropinirole and pramipexole exhibit highly specific affinity for the cerebral dopamine D3 receptor. Use of these medications in Parkinson's disease has been complicated by the emergence of pathologic behavioral patterns such as hypersexuality, pathologic gambling, excessive hobbying, and other circumscribed obsessive-compulsive disorders of impulse control in people having no history of such disorders. These behavioral changes typically remit following discontinuation of the medication, further demonstrating a causal relationship. Expression of the D3 receptor is particularly rich within the limbic system, where it plays an important role in modulating the physiologic and emotional experience of novelty, reward, and risk assessment. Converging neuroanatomical, physiological, and behavioral science data suggest the high D3 affinity of these medications as the basis for these behavioral changes. These observations suggest the D3 receptor as a therapeutic target for obsessive-compulsive disorder and substance abuse, and improved understanding of D3 receptor function may aid drug design of future atypical antipsychotics.
Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge.
Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate.
MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate.
Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.
Anhedonia; Major depressive disorder; Depression; Reinforcement learning; Reward learning; Prediction error; Computational; Meta-analysis; Reward sensitivity; Learning rate
To optimize behavior organisms evaluate the risks and benefits of available choices. The mesolimbic dopamine (DA) system encodes information about response costs and reward delays that bias choices. However, it remains unclear whether subjective value associated with risk-taking behavior is encoded by DA release.
Here, rats (n = 11) were trained on a risk-based decision making task in which visual cues predicted the opportunity to respond for smaller certain (safer) or larger uncertain (riskier) rewards. Following training, DA release within the NAc was monitored on a rapid time scale using fast-scan cyclic voltammetry during the risk-based decision making task.
Individual differences in risk-taking behavior were observed as animals displayed a preference for either safe or risky rewards. When only one response option was available, reward predictive cues evoked increases in DA concentration in the NAc core that scaled with each animal’s preferred reward contingency. However, when both options were presented simultaneously, cue-evoked DA release signaled the animals preferred reward contingency, regardless of the future choice. Further, DA signaling in the NAc core also tracked unexpected presentations or omissions of rewards following prediction error theory.
These results suggest that the dopaminergic projections to the NAc core encode the subjective value of future rewards that may function to influence future decisions to take risks.
risk-taking; dopamine; nucleus accumbens; decision making; reward; value
Pramipexole (PPX) is a dopamine agonist medication that has been implicated in the development of pathological gambling and other impulse control disorders. Johnson, Madden, Brewer, Pinkston, and Fowler (2011) reported that PPX increased male rats’ preference for gambling-like rewards (those arranged according to a variable-ratio schedule) over predictable rewards (those obtained from a fixed-ratio schedule). The present experiment explored the possibility that Johnson et al. underestimated the effects of PPX on gambling-like choices by constraining their rats’ daily income. In the present experiment conducted in a closed economy, PPX produced a dose-related increase in choice of the gambling-like alternative. In a control condition, PPX did not disrupt choice, suggesting the increased preference for gambling-like rewards was not due to nonspecific drug effects. Our findings are qualitatively consistent with those of Johnson et al., although the dose-related effect and larger effect size in the current study suggest that the effect of PPX on gambling-like choices is more pronounced when income was not constrained. This finding is consistent with clinical reports suggesting PPX is related to the development of problem gambling in humans.
pramipexole; dopamine agonist; gambling; Parkinson’s disease; rat
In patients with Parkinson's disease, aberrant or excessive dopaminergic stimulation is commonly indicated as the trigger factor in unmasking impulse control disorders (ICDs) such as pathological gambling. We had the opportunity to follow a patient who experienced Parkinson's disease 7 years ago when he was using pramipexole and again, recently, when he was treated with levodopa (L-dopa) and low frequency stimulation of the nucleus of the pedunculopontine tegmentus (PPTg) but no dopamine agonists. The same patient had shown, when studied with fluorodeoxyglucose-positron emission tomography in the condition PPTg-ON, a peculiar increased activity in the left ventral striatum. This case report confirms that, in a predisposed personality, ICD may arise from the perturbation of endogenous pathways, which connect the brainstem to the basal ganglia.
Pathological gambling is an impulse control disorder reported in association with dopamine agonists used to treat Parkinson’s disease. Although impulse control disorders are conceptualized as lying within the spectrum of addictions, little neurobiological evidence exists to support this belief. Functional imaging studies have consistently demonstrated abnormalities of dopaminergic function in patients with drug addictions, but to date no study has specifically evaluated dopaminergic function in Parkinson’s disease patients with impulse control disorders. We describe results of a [11C] raclopride positron emission tomography (PET) study comparing dopaminergic function during gambling in Parkinson’s disease patients, with and without pathological gambling, following dopamine agonists. Patients with pathological gambling demonstrated greater decreases in binding potential in the ventral striatum during gambling (13.9%) than control patients (8.1%), likely reflecting greater dopaminergic release. Ventral striatal bindings at baseline during control task were also lower in patients with pathological gambling. Although prior imaging studies suggest that abnormality in dopaminergic binding and dopamine release may be markers of vulnerability to addiction, this study presents the first evidence of these phenomena in pathological gambling. The emergence of pathological gambling in a number of Parkinson’s disease patients may provide a model into the pathophysiology of this disorder.
PMID: 19346328 CAMSID: cams2369
Parkinson’s disease; dopamine; impulse control disorders; pathological gambling; PET; functional imaging
Trait sensation-seeking, defined as a need for varied, complex, and intense sensations, represents a relatively underexplored hedonic drive in human behavioral neuroscience research. It is related to increased risk for a range of behaviors including substance use, gambling, and risky sexual practice. Individual differences in self-reported sensation-seeking have been linked to brain dopamine function, particularly at D2-like receptors, but so far no causal evidence exists for a role of dopamine in sensation-seeking behavior in humans. Here, we investigated the effects of the selective D2/D3 agonist cabergoline on performance of a probabilistic risky choice task in healthy humans using a sensitive within-subject, placebo-controlled design. Cabergoline significantly influenced the way participants combined different explicit signals regarding probability and loss when choosing between response options associated with uncertain outcomes. Importantly, these effects were strongly dependent on baseline sensation-seeking score. Overall, cabergoline increased sensitivity of choice to information about probability of winning; while decreasing discrimination according to magnitude of potential losses associated with different options. The largest effects of the drug were observed in participants with lower sensation-seeking scores. These findings provide evidence that risk-taking behavior in humans can be directly manipulated by a dopaminergic drug, but that the effectiveness of such a manipulation depends on baseline differences in sensation-seeking trait. This emphasizes the importance of considering individual differences when investigating manipulation of risky decision-making, and may have relevance for the development of pharmacotherapies for disorders involving excessive risk-taking in humans, such as pathological gambling.
Continued gambling to recover losses—‘loss chasing'—is a prominent feature of social and pathological gambling. However, little is known about the neuromodulators that influence this behavior. In three separate experiments, we investigated the role of serotonin activity, D2/D3 receptor activity, and beta-adrenoceptor activity on the loss chasing of age and IQ-matched healthy adults randomized to treatment or an appropriate control/placebo. In Experiment 1, participants consumed amino-acid drinks that did or did not contain the serotonin precursor, tryptophan. In Experiment 2, participants received a single 176 μg dose of the D2/D3 receptor agonist, pramipexole, or placebo. In Experiment 3, participants received a single 80 mg dose of the beta-adrenoceptor blocker, propranolol, or placebo. Following treatment, participants completed a computerized loss-chasing game. Mood and heart rate were measured at baseline and following treatment. Tryptophan depletion significantly reduced the number of decisions made to chase losses, and the number of consecutive decisions to chase, in the absence of marked changes in mood. By contrast, pramipexole significantly increased the value of losses chased and diminished the value of losses surrendered. Propranolol markedly reduced heart rate, but produced no significant changes in loss-chasing behavior. Loss chasing can be thought of as an aversively motivated escape behavior controlled, in part, by the marginal value of continued gambling relative to the value of already accumulated losses. Serotonin and dopamine appear to play dissociable roles in the tendency of individuals to gamble to recover, or to seek to ‘escape' from, previous losses. Serotonergic activity seems to promote the availability of loss chasing as a behavioral option, whereas D2/D3 receptor activity produces complex changes in the value of losses judged worth chasing. Sympathetic arousal, at least as mediated by beta-adrenoceptors, does not play a major role in laboratory-based loss-chasing choices.
serotonin; dopamine; loss chasing; gambling; persistence; value; addiction and substance abuse; behavioral science; dopamine; serotonin; gambling; loss chasing; persistence; value
Impulse control disorders are common in Parkinson's; disease, occurring in 13.6% of patients. Using a pharmacological manipulation and a novel risk taking task while performing functional magnetic resonance imaging, we investigated the relationship between dopamine agonists and risk taking in patients with Parkinson's; disease with and without impulse control disorders. During functional magnetic resonance imaging, subjects chose between two choices of equal expected value: a ‘Sure’ choice and a ‘Gamble’ choice of moderate risk. To commence each trial, in the ‘Gain’ condition, individuals started at $0 and in the ‘Loss’ condition individuals started at −$50 below the ‘Sure’ amount. The difference between the maximum and minimum outcomes from each gamble (i.e. range) was used as an index of risk (‘Gamble Risk’). Sixteen healthy volunteers were behaviourally tested. Fourteen impulse control disorder (problem gambling or compulsive shopping) and 14 matched Parkinson's; disease controls were tested ON and OFF dopamine agonists. Patients with impulse control disorder made more risky choices in the ‘Gain’ relative to the ‘Loss’ condition along with decreased orbitofrontal cortex and anterior cingulate activity, with the opposite observed in Parkinson's; disease controls. In patients with impulse control disorder, dopamine agonists were associated with enhanced sensitivity to risk along with decreased ventral striatal activity again with the opposite in Parkinson's; disease controls. Patients with impulse control disorder appear to have a bias towards risky choices independent of the effect of loss aversion. Dopamine agonists enhance sensitivity to risk in patients with impulse control disorder possibly by impairing risk evaluation in the striatum. Our results provide a potential explanation of why dopamine agonists may lead to an unconscious bias towards risk in susceptible individuals.
Parkinson's; disease; dopamine; gambling; decision making; risk
An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.
What are the physiological changes that take place in the brain when we solve a problem or learn a new skill? It is commonly assumed that behavior adaptations are realized on the microscopic level by changes in synaptic efficacies. However, this is hard to verify experimentally due to the difficulties of identifying the relevant synapses and monitoring them over long periods during a behavioral task. To address this question computationally, we develop a spiking neuronal network model of actor-critic temporal-difference learning, a variant of reinforcement learning for which neural correlates have already been partially established. The network learns a complex task by means of an internally generated reward signal constrained by recent findings on the dopaminergic system. Our model combines top-down and bottom-up modelling approaches to bridge the gap between synaptic plasticity and system-level learning. It paves the way for further investigations of the dopaminergic system in reward learning in the healthy brain and in pathological conditions such as Parkinson's disease, and can be used as a module in functional models based on brain-scale circuitry.