PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (645178)

Clipboard (0)
None

Related Articles

1.  Reward processing abnormalities in Parkinson disease 
Background
The primary motor cortex is important for motor learning and response selection, functions that require information on the expected and actual outcomes of behavior. Therefore, it should receive signals related to reward and pathways from reward centers to motor cortex exist in primates. Previously, we showed that gamma aminobutyric acid-A(GABAA)-mediated inhibition in motor cortex, measured by paired transcranial magnetic stimulation (TMS), changes with expectation and uncertainty of money rewards generated by a slot machine simulation.
Methods
We examined the role of dopamine in this phenomenon by testing 13 mildly affected Parkinson disease patients, off and on dopaminergic medications, and 13 healthy, age-matched controls.
Results
Consistent with a dopaminergic mechanism, reward expectation or predictability modulated the response to paired TMS in controls, but not in unmedicated patients. A single dose of pramipexole restored this effect of reward, mainly by increasing the paired TMS response amplitude during low expectation. Levodopa produced no such effect. Both pramipexole and levodopa increased risk-taking behavior on the Iowa Gambling Task. However, pramipexole increased risk-taking behavior more in patients showing lower paired TMS response amplitude during low expectation.
Conclusions
These results provide evidence that modulation of motor cortex inhibition by reward is mediated by dopamine signaling and that physiological states in the motor cortex are associated with levels of risk-taking behavior in patients on pramipexole. The cortical response to reward expectation may represent an endophenotype for risk-taking behavior in patients on agonist treatment.
doi:10.1002/mds.23701
PMCID: PMC3150632  PMID: 21538525
Transcranial magnetic stimulation (TMS); dopamine; gambling; motor cortex
2.  Effects of Acute Pramipexole on Preference for Gambling-like Schedules of Reinforcement in Rats 
Psychopharmacology  2010;213(1):11-18.
Rationale
Pramipexole and other direct dopamine agonist medications have been implicated in the development of impulsive behavior such as pathological gambling among those taking the drug to control symptoms of Parkinson’s disease or restless leg syndrome. Few laboratory studies examining pramipexole’s effects on gambling-like behavior have been conducted.
Objectives
The present study used a rodent model approximating some aspects of human gambling to examine within-subject effects of acute pramipexole (0.03, 0.1, 0.18, & 0.3 mg/kg) on rat’s choices to earn food reinforcement by completing variable-ratio (i.e., gambling-like) or fixed-ratio response requirements.
Results
In a condition in which the variable-ratio alternative was rarely selected, all but the lowest dose of pramipexole significantly increased choice of the variable-ratio alternative (an average of 15% above saline).. The same doses did not affect choice significantly in a control condition designed to evaluate the involvement of nonspecific drug effects. Pramipexole increased latencies to initiate trials (+ 9.12 s) and to begin response runs on forced-choice trials (variable-ratio: + 0.21 s; fixed-ratio: + 0.88 s), but did not affect measures of response perseveration (conditional probabilities of “staying”).
Conclusions
The findings are consistent with clinical reports linking pramipexole to the expression of increased gambling in humans. Results are discussed in the context of neurobehavioral evidence suggesting that dopamine agonists increase sensitivity to reward delay and disrupt appropriate feedback from negative outcomes.
doi:10.1007/s00213-010-2006-5
PMCID: PMC3747984  PMID: 20814781
pramipexole; dopamine agonist; gambling; impulsive behavior; Parkinson’s disease; rat
3.  Dopaminergic Influences on Emotional Decision Making in Euthymic Bipolar Patients 
Neuropsychopharmacology  2013;39(2):274-282.
We recently reported that the D2/D3 agonist pramipexole may have pro-cognitive effects in euthymic patients with bipolar disorder (BPD); however, the emergence of impulse-control disorders has been documented in Parkinson's disease (PD) after pramipexole treatment. Performance on reward-based tasks is altered in healthy subjects after a single dose of pramipexole, but its potential to induce abnormalities in BPD patients is unknown. We assessed reward-dependent decision making in euthymic BPD patients pre- and post 8 weeks of treatment with pramipexole or placebo by using the Iowa Gambling Task (IGT). The IGT requires subjects to choose among four card decks (two risky and two conservative) and is designed to promote learning to make advantageous (conservative) choices over time. Thirty-four BPD patients completed both assessments (18 placebo and 16 pramipexole). Baseline performance did not differ by treatment group (F=0.63; p=0.64); however, at week 8, BPD patients on pramipexole demonstrated a significantly greater tendency to make increasingly high-risk, high-reward choices across the five blocks, whereas the placebo group's pattern was similar to that reported in healthy individuals (treatment × time × block interaction, p<0.05). Analyses of choice strategy using the expectancy valence model revealed that after 8 weeks on pramipexole, BPD patients attended more readily to feedback related to gains than to losses, which could explain the impaired learning. There were no significant changes in mood symptoms over the 8 weeks, and no increased propensity toward manic-like behaviors were reported. Our results suggest that the enhancement of dopaminergic activity influences risk-associated decision-making performance in euthymic BPD. The clinical implications remain unknown.
doi:10.1038/npp.2013.177
PMCID: PMC3870768  PMID: 23884342
Behavioral Science; bipolar disorder; decision-making; Dopamine; gambling; Mood/Anxiety/Stress Disorders; Neuropharmacology; pramipexole; bipolar disorder; dopamine; pramipexole; decision-making
4.  Dopamine agonists and risk: impulse control disorders in Parkinson's; disease 
Brain  2011;134(5):1438-1446.
Impulse control disorders are common in Parkinson's; disease, occurring in 13.6% of patients. Using a pharmacological manipulation and a novel risk taking task while performing functional magnetic resonance imaging, we investigated the relationship between dopamine agonists and risk taking in patients with Parkinson's; disease with and without impulse control disorders. During functional magnetic resonance imaging, subjects chose between two choices of equal expected value: a ‘Sure’ choice and a ‘Gamble’ choice of moderate risk. To commence each trial, in the ‘Gain’ condition, individuals started at $0 and in the ‘Loss’ condition individuals started at −$50 below the ‘Sure’ amount. The difference between the maximum and minimum outcomes from each gamble (i.e. range) was used as an index of risk (‘Gamble Risk’). Sixteen healthy volunteers were behaviourally tested. Fourteen impulse control disorder (problem gambling or compulsive shopping) and 14 matched Parkinson's; disease controls were tested ON and OFF dopamine agonists. Patients with impulse control disorder made more risky choices in the ‘Gain’ relative to the ‘Loss’ condition along with decreased orbitofrontal cortex and anterior cingulate activity, with the opposite observed in Parkinson's; disease controls. In patients with impulse control disorder, dopamine agonists were associated with enhanced sensitivity to risk along with decreased ventral striatal activity again with the opposite in Parkinson's; disease controls. Patients with impulse control disorder appear to have a bias towards risky choices independent of the effect of loss aversion. Dopamine agonists enhance sensitivity to risk in patients with impulse control disorder possibly by impairing risk evaluation in the striatum. Our results provide a potential explanation of why dopamine agonists may lead to an unconscious bias towards risk in susceptible individuals.
doi:10.1093/brain/awr080
PMCID: PMC3097893  PMID: 21596771
Parkinson's; disease; dopamine; gambling; decision making; risk
5.  Single Dose of a Dopamine Agonist Impairs Reinforcement Learning in Humans: Behavioral Evidence from a Laboratory-based Measure of Reward Responsiveness 
Psychopharmacology  2007;196(2):221-232.
Rationale
The dopaminergic system, particularly D2-like dopamine receptors, has been strongly implicated in reward processing. Animal studies have emphasized the role of phasic dopamine (DA) signaling in reward-related learning, but these processes remain largely unexplored in humans.
Objectives
To evaluate the effect of a single, low dose of a D2/D3 agonist—pramipexole—on reinforcement learning in healthy adults. Based on prior evidence indicating that low doses of DA agonists decrease phasic DA release through autoreceptor stimulation, we hypothesized that 0.5 mg of pramipexole would impair reward learning due to presynaptic mechanisms.
Methods
Using a double-blind design, a single 0.5 mg dose of pramipexole or placebo was administered to 32 healthy volunteers, who performed a probabilistic reward task involving a differential reinforcement schedule as well as various control tasks.
Results
As hypothesized, response bias toward the more frequently rewarded stimulus was impaired in the pramipexole group, even after adjusting for transient adverse effects. In addition, the pramipexole group showed reaction time and motor speed slowing and increased negative affect; however, when adverse physical side effects were considered, group differences in motor speed and negative affect disappeared.
Conclusions
These findings show that a single low dose of pramipexole impaired the acquisition of reward-related behavior in healthy participants, and they are consistent with prior evidence suggesting that phasic DA signaling is required to reinforce actions leading to reward. The potential implications of the present findings to psychiatric conditions, including depression and impulse control disorders related to addiction, are discussed.
doi:10.1007/s00213-007-0957-y
PMCID: PMC2268635  PMID: 17909750
Dopamine; D2 agonists; Reward Processing; Depression; Mesolimbic System; Addiction
6.  Influence of Dopaminergically Mediated Reward on Somatosensory Decision-Making 
PLoS Biology  2009;7(7):e1000164.
This pharmacological fMRI study shows that during reward-based sensory decision-making, dopamine is crucially involved in reward-related modulation of human primary sensory cortex.
Reward-related dopaminergic influences on learning and overt behaviour are well established, but any influence on sensory decision-making is largely unknown. We used functional magnetic resonance imaging (fMRI) while participants judged electric somatosensory stimuli on one hand or other, before being rewarded for correct performance at trial end via a visual signal, at one of four anticipated financial levels. Prior to the procedure, participants received either placebo (saline), a dopamine agonist (levodopa), or an antagonist (haloperidol). Principal findings: higher anticipated reward improved tactile decisions. Visually signalled reward reactivated primary somatosensory cortex for the judged hand, more strongly for higher reward. After receiving a higher reward on one trial, somatosensory activations and decisions were enhanced on the next trial. These behavioural and neural effects were all enhanced by levodopa and attenuated by haloperidol, indicating dopaminergic dependency. Dopaminergic reward-related influences extend even to early somatosensory cortex and sensory decision-making.
Author Summary
The rewards one receives during decision-making has a profound impact on learning. Much recent interest has focused on the role of the neurotransmitter dopamine in the basal ganglia for influencing learning and behaviour. Here, we ask whether reward can influence low-level sensory processing, for instance in primary sensory cortex, and how dopamine mediates this process. We show in humans that dopamine level, as manipulated with a dopamine agonist and antagonist in a double-blind placebo-controlled design, is involved in reward modulation of primary somatosensory cortex. Higher anticipated reward improved tactile decisions, and receipt of visual reward signals reactivated primary somatosensory cortex for the judged hand as measured using functional neuroimaging. After receiving a higher reward on one trial, somatosensory activations and decisions were enhanced on the next trial, suggesting that reward outcome provides a form of teaching signal that may be fed back to task-relevant sensory cortex. All these behavioural and neural effects of reward on somatosensory decision-making were strongly modulated by the availability of dopamine as the mediating neurotransmitter. These findings raise the tantalising new possibility that reward manipulations in conjunction with dopaminergic drugs might be used to enhance pathologically deficient or lapsed sensory processes, analogous to how rewards can be used to shape or correct behaviour.
doi:10.1371/journal.pbio.1000164
PMCID: PMC2709435  PMID: 19636360
7.  Effects of Pramipexole on Impulsive Choice in Male Wistar Rats 
Clinical reports, primarily with Parkinson’s patients, note an association between the prescribed use of pramipexole (and other direct-acting dopamine agonist medications) and impulse control disorders, particularly pathological gambling. Two experiments examined the effects of acute pramipexole on rats’ impulsive choices where impulsivity was defined as selecting a smaller-sooner over a larger-later food reward. In Experiment 1, pramipexole (0.1 to 0.3 mg/kg) significantly increased impulsive choices in a condition in which few impulsive choices were made during a stable baseline. In a control condition, in which impulsive choices predominated during baseline, pramipexole did not significantly change the same rats’ choices. Experiment 2 explored a wider range of doses (0.01 to 0.3 mg/kg) using a choice procedure in which delays to the larger-later reinforcer delivery increased across trial blocks within each session. At the doses used in Experiment 1, pramipexole shifted choice toward indifference regardless of the operative delay. At lower doses of pramipexole (0.01 & 0.03 mg/kg), a trend toward more impulsive choice was observed at the 0.03 mg/kg dose. The difference in outcomes across experiments may be due to the more complex discriminations required in Experiment 2; i.e., multiple discriminations between changing delays within each session.
doi:10.1037/a0019244
PMCID: PMC3021944  PMID: 20545391
Pramipexole; D2/D3 agonist; Impulsivity; Choice; Gambling
8.  Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis 
Background
Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge.
Methods
Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate.
Results
MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate.
Conclusion
Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.
doi:10.1186/2045-5380-3-12
PMCID: PMC3701611  PMID: 23782813
Anhedonia; Major depressive disorder; Depression; Reinforcement learning; Reward learning; Prediction error; Computational; Meta-analysis; Reward sensitivity; Learning rate
9.  Pramipexole in patients with early Parkinson's disease (PROUD): a randomised delayed-start trial 
Lancet Neurology  2013;12(8):747-755.
Summary
Background
In models of dopaminergic neuronal loss, the dopamine agonist pramipexole has exhibited neuroprotective properties. The Pramipexole On Underlying Disease (PROUD) study was designed to identify whether early versus delayed pramipexole initiation has clinical and neuroimaging benefits in patients with Parkinson's disease (PD).
Methods
Between May 24, 2006, and April 22, 2009, at 98 centres, we recruited patients with PD diagnosed within 2 years and aged 30–79 years. We randomly assigned eligible patients (ratio 1:1), by a centralised, computerised randomisation schedule, to receive double-blind either placebo or pramipexole (1·5 mg a day) and followed them up for 15 months. At 9 months, or as early as 6 months if considered necessary, placebo recipients were assigned to pramipexole. In a neuroimaging substudy, striatal dopamine-transporter binding was assessed by SPECT. All patients, investigators, and independent raters were masked to study treatment. The primary endpoint was the 15-month change from baseline in total score on the unified Parkinson's disease rating scale (UPDRS). This trial is registered with ClinicalTrials.gov, number NCT00321854.
Findings
Of 535 patients, 261 were randomly assigned to receive pramipexole and 274 to receive placebo. At 15 months (n=411), adjusted mean change in UPDRS total score showed no significant difference between early and delayed pramipexole (−0·4 points, 95% CI −2·2 to 1·4, p=0·65). 62 patients in the early pramipexole group and 61 patients in the delayed pramipexole group were included in the neuroimaging substudy, for which the adjusted mean 15-month change in striatal 123I-FP-CIT binding was −15·1% (SE 2·1) for early and −14·6% (2·0) for delayed pramipexole (difference −0·5 percentage points, 95% CI −5·4 to 4·4, p=0·84). Overall, 180 (81%) of patients given early pramipexole and 179 (84%) patients given delayed pramipexole reported adverse events (most frequently nausea), and 22 (10%) patients in the early pramipexole group and 17 (8%) in the delayed pramipexole group had serious events, two of which (hallucinations and orthostatic hypotension) were deemed related to study drug.
Interpretation
By clinical and neuroimaging measures, pramipexole showed little evidence differentiating 15-month usage from usage delayed for 6–9 months. The results do not support the hypothesis that pramipexole has disease-modifying effects.
Funding
Boehringer Ingelheim GmbH.
doi:10.1016/S1474-4422(13)70117-0
PMCID: PMC3714436  PMID: 23726851
10.  The Iowa Gambling Task and the three fallacies of dopamine in gambling disorder 
Gambling disorder sufferers prefer immediately larger rewards despite long term losses on the Iowa Gambling Task (IGT), and these impairments are associated with dopamine dysfunctions. Dopamine is a neurotransmitter linked with temporal and structural dysfunctions in substance use disorder, which has supported the idea of impaired decision-making and dopamine dysfunctions in gambling disorder. However, evidence from substance use disorders cannot be directly transferred to gambling disorder. This article focuses on three hypotheses of dopamine dysfunctions in gambling disorder, which appear to be “fallacies,” i.e., have not been supported in a series of positron emission tomography (PET) studies. The first “fallacy” suggests that gambling disorder sufferers have lower dopamine receptor availability, as seen in substance use disorders. However, no evidence supported this hypothesis. The second “fallacy” suggests that maladaptive decision-making in gambling disorder is associated with higher dopamine release during gambling. No evidence supported the hypothesis, and the literature on substance use disorders offers limited support for this hypothesis. The third “fallacy” suggests that maladaptive decision-making in gambling disorder is associated with higher dopamine release during winning. The evidence did not support this hypothesis either. Instead, dopaminergic coding of reward prediction and uncertainty might better account for dopamine dysfunctions in gambling disorder. Studies of reward prediction and reward uncertainty show a sustained dopamine response toward stimuli with maximum uncertainty, which may explain the continued dopamine release and gambling despite losses in gambling disorder. The findings from the studies presented here are consistent with the notion of dopaminergic dysfunctions of reward prediction and reward uncertainty signals in gambling disorder.
doi:10.3389/fpsyg.2013.00709
PMCID: PMC3792697  PMID: 24115941
gambling disorder; Iowa Gambling Task (IGT); dopamine; addiction; positron-emission tomography
11.  Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients* 
Brain  2009;132(9):2385-2395.
Parkinson's disease is characterized by the degeneration of dopaminergic pathways projecting to the striatum. These pathways are implicated in reward prediction. In this study, we investigated reward and punishment processing in young, never-medicated Parkinson's disease patients, recently medicated patients receiving the dopamine receptor agonists pramipexole and ropinirole and healthy controls. The never-medicated patients were also re-evaluated after 12 weeks of treatment with dopamine agonists. Reward and punishment processing was assessed by a feedback-based probabilistic classification task. Personality characteristics were measured by the temperament and character inventory. Results revealed that never-medicated patients with Parkinson's disease showed selective deficits on reward processing and novelty seeking, which were remediated by dopamine agonists. These medications disrupted punishment processing. In addition, dopamine agonists increased the correlation between reward processing and novelty seeking, whereas these drugs decreased the correlation between punishment processing and harm avoidance. Our finding that dopamine agonist administration in young patients with Parkinson's disease resulted in increased novelty seeking, enhanced reward processing, and decreased punishment processing may shed light on the cognitive and personality bases of the impulse control disorders, which arise as side-effects of dopamine agonist therapy in some Parkinson's disease patients.
doi:10.1093/brain/awp094
PMCID: PMC2766178  PMID: 19416950
Parkinson's disease; reward; novelty seeking; dopamine; pramipexole; ropinirole
12.  From uncertainty to reward: BOLD characteristics differentiate signaling pathways 
BMC Neuroscience  2009;10:154.
Background
Reward value and uncertainty are represented by dopamine neurons in monkeys by distinct phasic and tonic firing rates. Knowledge about the underlying differential dopaminergic pathways is crucial for a better understanding of dopamine-related processes. Using functional magnetic resonance blood-oxygen level dependent (BOLD) imaging we analyzed brain activation in 15 healthy, male subjects performing a gambling task, upon expectation of potential monetary rewards at different reward values and levels of uncertainty.
Results
Consistent with previous studies, ventral striatal activation was related to both reward magnitudes and values. Activation in medial and lateral orbitofrontal brain areas was best predicted by reward uncertainty. Moreover, late BOLD responses relative to trial onset were due to expectation of different reward values and likely to represent phasic dopaminergic signaling. Early BOLD responses were due to different levels of reward uncertainty and likely to represent tonic dopaminergic signals.
Conclusions
We conclude that differential dopaminergic signaling as revealed in animal studies is not only represented locally by involvement of distinct brain regions but also by distinct BOLD signal characteristics.
doi:10.1186/1471-2202-10-154
PMCID: PMC2806405  PMID: 20028546
13.  The Effect of Dopamine Agonists on Adaptive and Aberrant Salience in Parkinson's Disease 
Neuropsychopharmacology  2011;37(4):950-958.
Clinical evidence suggests that after initiation of dopaminergic medications some patients with Parkinson's disease (PD) develop psychotic symptoms, such as hallucinations and delusions. Here, we tested the hypothesis that the neurocognitive basis of this phenomenon can be defined as the formation of arbitrary and illusory associations between conditioned stimuli and reward signals, called aberrant salience. Young, never-medicated PD patients and matched controls were assessed on a speeded reaction time task in which the probe stimulus was preceded by conditioned stimuli that could signal monetary reward by color or shape. The patients and controls were re-evaluated after 12 weeks during which the patients received a dopamine agonist (pramipexole or ropinirole). Results indicated that dopamine agonists increased both adaptive and aberrant salience in PD patients, that is, formation of real and illusory associations between conditioned stimuli and reward, respectively. This effect was present when associations were assessed by means of faster responding after conditioned stimuli signaling reward (implicit salience) and overt rating of stimulus–reward links (explicit salience). However, unusual feelings and experiences, which are subclinical manifestations of psychotic-like symptoms, were specifically related to irrelevant and illusory stimulus–reward associations (aberrant salience) in PD patients receiving dopamine agonists. The learning of relevant and real stimulus–reward associations (adaptive salience) was not related to unusual experiences. These results suggest that dopamine agonists may increase psychotic-like experiences in young patients with PD, possibly by facilitating dopaminergic transmission in the ventral striatum, which results in aberrant associations between conditioned stimuli and reward.
doi:10.1038/npp.2011.278
PMCID: PMC3280658  PMID: 22089321
Parkinson's disease; dopamine agonists; psychosis; reward; salience; cognition; neuropharmacology; dopamine; cognition; learning and memory; Parkinson's disease; reward; dopamine agonists
14.  An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning 
PLoS Computational Biology  2011;7(5):e1001133.
An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.
Author Summary
What are the physiological changes that take place in the brain when we solve a problem or learn a new skill? It is commonly assumed that behavior adaptations are realized on the microscopic level by changes in synaptic efficacies. However, this is hard to verify experimentally due to the difficulties of identifying the relevant synapses and monitoring them over long periods during a behavioral task. To address this question computationally, we develop a spiking neuronal network model of actor-critic temporal-difference learning, a variant of reinforcement learning for which neural correlates have already been partially established. The network learns a complex task by means of an internally generated reward signal constrained by recent findings on the dopaminergic system. Our model combines top-down and bottom-up modelling approaches to bridge the gap between synaptic plasticity and system-level learning. It paves the way for further investigations of the dopaminergic system in reward learning in the healthy brain and in pathological conditions such as Parkinson's disease, and can be used as a module in functional models based on brain-scale circuitry.
doi:10.1371/journal.pcbi.1001133
PMCID: PMC3093351  PMID: 21589888
15.  Impulse control disorders in Parkinson’s disease: recent advances 
Current opinion in neurology  2011;24(4):324-330.
Purpose of review
To review the recent advances in the epidemiology and pathophysiology of impulse control disorders (ICD) in Parkinson’s disease (PD).
Recent findings
Large cross-sectional and case-control multicentre studies show that ICDs in PD are common with a frequency of 13.6%. These behaviours are associated with impaired functioning and with depressive, anxiety and obsessive symptoms, novelty seeking and impulsivity. Behavioural subtypes demonstrate differences in novelty seeking and impulsivity suggesting pathophysiological differences. Observational and neurophysiological studies point towards a potential mechanistic overlap between the behavioural (ICDs) and motor (dyskinesias) dopaminergic sequelae. Converging data suggest dopamine agonists in ICDs appear to enhance learning from rewarding outcomes and impulsive choice. ICD patients also have enhanced risk preference and impaired working memory. Neuroimaging data points towards enhanced bottom-up ventral striatal dopamine release to incentive cues, gambling tasks and reward prediction, and possibly inhibition of top-down orbitofrontal influences. Dopamine agonist-related ventral striatal hypoactivity to risk is consistent with impaired risk evaluation.
Summary
Recent large scale studies and converging findings are beginning to provide an understanding of mechanisms underlying ICDs in PD which can guide prevention of these behaviours and optimize therapeutic approaches.
doi:10.1097/WCO.0b013e3283489687
PMCID: PMC3154756  PMID: 21725242
Impulse control disorders; Parkinson’s disease; dopamine agonists; pathological gambling; impulsivity
16.  Frequency of New-Onset Pathologic Compulsive Gambling or Hypersexuality After Drug Treatment of Idiopathic Parkinson Disease 
Mayo Clinic Proceedings  2009;84(4):310-316.
OBJECTIVE: To determine the frequency of new-onset compulsive gambling or hypersexuality among regional patients with Parkinson disease (PD), ascertaining the relationship of these behaviors to PD drug use.
PATIENTS AND METHODS: We retrospectively reviewed the medical records of patients from 7 rural southeastern Minnesota counties who had at least 1 neurology appointment for PD between July 1, 2004, and June 30, 2006. The main outcome measure was compulsive gambling or hypersexuality developing after parkinsonism onset, including the temporal relationship to PD drug use.
RESULTS: Of 267 patients with PD who met the study inclusion criteria, new-onset gambling or hypersexuality was documented in 7 (2.6%). All were among the 66 patients (10.6%) taking a dopamine agonist. Moreover, all 7 (18.4%) were among 38 patients taking therapeutic doses (defined as ≥2 mg of pramipexole or 6 mg of ropinirole daily). Behaviors were clearly pathologic and disabling in 5: 7.6% of all patients taking an agonist and 13.2% of those taking therapeutic doses. Of the 5 patients, 2 had extensive treatment for what was considered a primary psychiatric problem before the agonist connection was recognized.
CONCLUSION: Among the study patients with PD, new-onset compulsive gambling or hypersexuality was documented in 7 (18.4%) of 38 patients taking therapeutic doses of dopamine agonists but was not found among untreated patients, those taking subtherapeutic agonist doses, or those taking carbidopa/levodopa alone. Behaviors abated with discontinuation of agonist therapy or dose reduction. Because this is a retrospective study, cases may have been missed, and hence this study may reflect an underestimation of the true frequency. Physicians who care for patients taking these drugs should recognize the drug's potential to induce pathologic syndromes that sometimes masquerade as primary psychiatric disease.
In patients with Parkinson disease, new-onset compulsive gambling or hypersexuality was documented in 7 of 38 patients taking therapeutic doses of dopamine agonists but was not found among untreated patients, those taking subtherapeutic agonist doses, or those taking carbidopa/levodopa alone.
PMCID: PMC2665974  PMID: 19339647
17.  Effects of Acute Pramipexole on Male Rats’ Preference for Gambling-like Rewards II 
Pramipexole (PPX) is a dopamine agonist medication that has been implicated in the development of pathological gambling and other impulse control disorders. Johnson, Madden, Brewer, Pinkston, and Fowler (2011) reported that PPX increased male rats’ preference for gambling-like rewards (those arranged according to a variable-ratio schedule) over predictable rewards (those obtained from a fixed-ratio schedule). The present experiment explored the possibility that Johnson et al. underestimated the effects of PPX on gambling-like choices by constraining their rats’ daily income. In the present experiment conducted in a closed economy, PPX produced a dose-related increase in choice of the gambling-like alternative. In a control condition, PPX did not disrupt choice, suggesting the increased preference for gambling-like rewards was not due to nonspecific drug effects. Our findings are qualitatively consistent with those of Johnson et al., although the dose-related effect and larger effect size in the current study suggest that the effect of PPX on gambling-like choices is more pronounced when income was not constrained. This finding is consistent with clinical reports suggesting PPX is related to the development of problem gambling in humans.
doi:10.1037/a0027117
PMCID: PMC3482126  PMID: 22288460
pramipexole; dopamine agonist; gambling; Parkinson’s disease; rat
18.  The role of dopamine in risk taking: a specific look at Parkinson’s disease and gambling 
An influential model suggests that dopamine signals the difference between predicted and experienced reward. In this way, dopamine can act as a learning signal that can shape behaviors to maximize rewards and avoid punishments. Dopamine is also thought to invigorate reward seeking behavior. Loss of dopamine signaling is the major abnormality in Parkinson’s disease. Dopamine agonists have been implicated in the occurrence of impulse control disorders in Parkinson’s disease patients, the most common being pathological gambling, compulsive sexual behavior, and compulsive buying. Recently, a number of functional imaging studies investigating impulse control disorders in Parkinson’s disease have been published. Here we review this literature, and attempt to place it within a decision-making framework in which potential gains and losses are evaluated to arrive at optimum choices. We also provide a hypothetical but still incomplete model on the effect of dopamine agonist treatment on these value and risk assessments. Two of the main brain structures thought to be involved in computing aspects of reward and loss are the ventral striatum (VStr) and the insula, both dopamine projection sites. Both structures are consistently implicated in functional brain imaging studies of pathological gambling in Parkinson’s disease.
doi:10.3389/fnbeh.2014.00196
PMCID: PMC4038955  PMID: 24910600
impulse control disorders; impulsivity; reward; loss aversion; insula; ventral striatum
19.  Dopamine Agonist Increases Risk Taking but Blunts Reward-Related Brain Activity 
PLoS ONE  2008;3(6):e2479.
The use of D2/D3 dopaminergic agonists in Parkinson's disease (PD) may lead to pathological gambling. In a placebo-controlled double-blind study in healthy volunteers, we observed riskier choices in a lottery task after administration of the D3 receptor-preferring agonist pramipexole thus mimicking risk-taking behavior in PD. Moreover, we demonstrate decreased activation in the rostral basal ganglia and midbrain, key structures of the reward system, following unexpected high gains and therefore propose that pathological gambling in PD results from the need to seek higher rewards to overcome the blunted response in this system.
doi:10.1371/journal.pone.0002479
PMCID: PMC2423613  PMID: 18575579
20.  Adolescent Changes in Dopamine D1 Receptor Expression in Orbitofrontal Cortex and Piriform Cortex Accompany an Associative Learning Deficit 
PLoS ONE  2013;8(2):e56191.
The orbitofrontal cortex (OFC) and piriform cortex are involved in encoding the predictive value of olfactory stimuli in rats, and neural responses to olfactory stimuli in these areas change as associations are learned. This experience-dependent plasticity mirrors task-related changes previously observed in mesocortical dopamine neurons, which have been implicated in learning the predictive value of cues. Although forms of associative learning can be found at all ages, cortical dopamine projections do not mature until after postnatal day 35 in the rat. We hypothesized that these changes in dopamine circuitry during the juvenile and adolescent periods would result in age-dependent differences in learning the predictive value of environmental cues. Using an odor-guided associative learning task, we found that adolescent rats learn the association between an odor and a palatable reward significantly more slowly than either juvenile or adult rats. Further, adolescent rats displayed greater distractibility during the task than either juvenile or adult rats. Using real-time quantitative PCR and immunohistochemical methods, we observed that the behavioral deficit in adolescence coincides with a significant increase in D1 dopamine receptor expression compared to juvenile rats in both the OFC and piriform cortex. Further, we found that both the slower learning and increased distractibility exhibited in adolescence could be alleviated by experience with the association task as a juvenile, or by an acute administration of a low dose of either the dopamine D1 receptor agonist SKF-38393 or the D2 receptor antagonist eticlopride. These results suggest that dopaminergic modulation of cortical function may be important for learning the predictive value of environmental stimuli, and that developmental changes in cortical dopaminergic circuitry may underlie age-related differences in associative learning.
doi:10.1371/journal.pone.0056191
PMCID: PMC3578843  PMID: 23437091
21.  Gambling severity predicts midbrain response to near-miss outcomes 
Gambling is a common recreational activity that becomes dysfunctional in a subset of individuals, with DSM ‘pathological gambling’ regarded as the most severe form. During gambling, players experience a range of cognitive distortions that promote an over-estimation of the chances of winning. Near-miss outcomes are thought to fuel these distortions. We observed previously that near-misses recruited overlapping circuitry to monetary wins in a study in healthy volunteers (Clark et al. 2009). The present study sought to extend these observations in regular gamblers and relate brain responses to an index of gambling severity. Twenty regular gamblers, who varied in their involvement from recreational players to probable pathological gamblers, were scanned whilst performing a simplified slot-machine task that delivered occasional monetary wins, as well as near-miss and full-miss non-win outcomes. In the overall group, near-miss outcomes were associated with a significant response in the ventral striatum, which was also recruited by monetary wins. Gambling severity, measured with the South Oaks Gambling Screen, predicted a greater response in the dopaminergic midbrain to near-miss outcomes. This effect survived controlling for clinical co-morbidities that were present in the regular gamblers. Gambling severity did not predict win-related responses in the midbrain or elsewhere. These results demonstrate that near-miss events during gambling recruit reward-related brain circuitry in regular players. An association with gambling severity in the midbrain suggests that near-miss outcomes may enhance dopamine transmission in disordered gambling, which extends neurobiological similarities between pathological gambling and drug addiction.
doi:10.1523/JNEUROSCI.5758-09.2010
PMCID: PMC2929454  PMID: 20445043
Gambling; Cognitive; Addiction; Dopamine; Striatum; Midbrain
22.  Increased striatal dopamine release in Parkinsonian patients with pathological gambling: a [11C] raclopride PET study 
Brain : a journal of neurology  2009;132(Pt 5):1376-1385.
Pathological gambling is an impulse control disorder reported in association with dopamine agonists used to treat Parkinson’s disease. Although impulse control disorders are conceptualized as lying within the spectrum of addictions, little neurobiological evidence exists to support this belief. Functional imaging studies have consistently demonstrated abnormalities of dopaminergic function in patients with drug addictions, but to date no study has specifically evaluated dopaminergic function in Parkinson’s disease patients with impulse control disorders. We describe results of a [11C] raclopride positron emission tomography (PET) study comparing dopaminergic function during gambling in Parkinson’s disease patients, with and without pathological gambling, following dopamine agonists. Patients with pathological gambling demonstrated greater decreases in binding potential in the ventral striatum during gambling (13.9%) than control patients (8.1%), likely reflecting greater dopaminergic release. Ventral striatal bindings at baseline during control task were also lower in patients with pathological gambling. Although prior imaging studies suggest that abnormality in dopaminergic binding and dopamine release may be markers of vulnerability to addiction, this study presents the first evidence of these phenomena in pathological gambling. The emergence of pathological gambling in a number of Parkinson’s disease patients may provide a model into the pathophysiology of this disorder.
doi:10.1093/brain/awp054
PMCID: PMC3479148  PMID: 19346328 CAMSID: cams2369
Parkinson’s disease; dopamine; impulse control disorders; pathological gambling; PET; functional imaging
23.  Effects of a Dopamine Agonist on the Pharmacodynamics of Levodopa in Parkinson Disease 
Archives of Neurology  2010;67(1):27-32.
Background
Treatment of Parkinson disease commonly includes levodopa and dopamine agonists; however, the interaction of these 2 drugs is poorly understood.
Objective
To examine the effects of a dopamine agonist on the motor response to levodopa.
Design
Double-blind, randomized, placebo-controlled, crossover clinical trial.
Setting
Ambulatory academic referral center.
Patients
Thirteen patients with idiopathic Parkinson disease taking levodopa and experiencing motor fluctuations and dyskinesia.
Interventions
Eligible individuals were randomly assigned to receive pramipexole dihydrochloride or placebo for 4 weeks followed by a 2-hour intravenous levodopa infusion on consecutive days at 2 rates and with blinded assessments. They were then crossed over to the alternate oral therapy for 4 weeks followed by levodopa infusion and reassessment.
Main Outcome Measures
Change in finger-tapping speed, measured using the area under the curve (AUC) for finger taps per minute across time; peak finger-tapping speed; duration of response; time to “ON” (defined as a 10% increase in finger-tapping speed above baseline); walking speed; and dyskinesia AUC.
Results
Pramipexole with levodopa infusion increased finger-tapping speed beyond the change in baseline by a mean (SE) of 170 (47.2) per minute×minutes (P=.006) and more than doubled the AUC for finger-tapping speed. Pramipexole increased peak finger-tapping speed by a mean (SE) of 18 (8.5) taps per minute (P=.02) and improved mean (SE) walking speed (15.9 [0.70] vs 18.9 [0.70] seconds, P=.004). Pramipexole prolonged duration of response after levodopa infusion and shortened time to ON. Pramipexole increased mean (SE) baseline dyskinesia scores (26.0 [5.85] vs 12.1 [5.85] points, P = .05) and peak dyskinesia scores with levodopa infusion.
Conclusions
Pramipexole augmented the motor response to levodopa beyond a simple additive effect and increased the severity of levodopa-induced dyskinesia. When considering a combination of these therapies, an appropriate balance should be maintained regarding gain of motor function vs worsening of dyskinesia.
Trial Registration
clinicaltrials.gov Identifier: NCT00666653
doi:10.1001/archneurol.2009.287
PMCID: PMC3390306  PMID: 20065126
24.  Single Dose of a Dopamine Agonist Impairs Reinforcement Learning in Humans: Evidence from Event-related Potentials and Computational Modeling of Striatal-Cortical Function 
Human brain mapping  2009;30(7):1963-1976.
Animal findings have highlighted the modulatory role of phasic dopamine (DA) signaling in incentive learning, particularly in the acquisition of reward-related behavior. In humans, these processes remain largely unknown. In a recent study we demonstrated that a single low dose of a D2/D3 agonist (pramipexole) – assumed to activate DA autoreceptors and thus reduce phasic DA bursts – impaired reward learning in healthy subjects performing a probabilistic reward task. The purpose of the present study was to extend these behavioral findings using event-related potentials and computational modeling. Compared to the placebo group, participants receiving pramipexole showed increased feedback-related negativity to probabilistic rewards and decreased activation in dorsal anterior cingulate regions previously implicated in integrating reinforcement history over time. Additionally, findings of blunted reward learning in participants receiving pramipexole were simulated by reduced presynaptic DA signaling in response to reward in a neural network model of striatal-cortical function. These preliminary findings offer important insights on the role of phasic DA signals on reinforcement learning in humans, and provide initial evidence regarding the spatio-temporal dynamics of brain mechanisms underlying these processes.
doi:10.1002/hbm.20642
PMCID: PMC3034238  PMID: 18726908
25.  Drug-induced deactivation of inhibitory networks predicts pathological gambling in PD (e–Pub ahead of print)  
Neurology  2010;75(19):1711-1716.
Objective:
Some patients with Parkinson disease (PD) develop pathological gambling when treated with dopamine agonists (DAs). However, little is known about DA-induced changes in neuronal networks that may underpin this drug-induced change in behavior in vulnerable individuals. In this case-control study, we aimed to investigate DA-induced changes in brain activity that may differentiate patients with PD with DA-induced pathological gambling (gamblers) from patients with PD without such a history (controls).
Methods:
Following overnight withdrawal of antiparkinsonian medication, patients were studied with H2 15O PET before and after administration of DA (3 mg apomorphine) to measure changes in regional cerebral blood flow as an index of regional brain activity during a card selection game with probabilistic feedback.
Results:
We observed that the direction of DA-related activity change in brain areas that are implicated in impulse control and response inhibition (lateral orbitofrontal cortex, rostral cingulate zone, amygdala, external pallidum) distinguished gamblers from controls. DA significantly increased activity in these areas in controls, while gamblers showed a significant DA-induced reduction of activity.
Conclusions:
We propose that in vulnerable patients with PD, DAs produce an abnormal neuronal pattern that resembles those found in nonparkinsonian pathological gambling and drug addiction. DA-induced disruption of inhibitory key functions—outcome monitoring (rostral cingulate zone), acquisition and retention of negative action-outcome associations (amygdala and lateral orbitofrontal cortex)—together with restricted access of those areas to executive control (external pallidum)—may well explain loss of impulse control and response inhibition in vulnerable patients with PD, thereby fostering the development of pathological gambling.
GLOSSARY
= analysis of variance;
= dopamine agonist;
= Gambling Symptom Assessment Scale;
= external pallidum;
= Montréal Neurological Institute;
= orbitofrontal cortex;
= Parkinson disease;
= regional cerebral blood flow;
= rostral cingulated zone;
= Unified Parkinson's Disease Rating Scale.
doi:10.1212/WNL.0b013e3181fc27fa
PMCID: PMC3033606  PMID: 20926784

Results 1-25 (645178)