PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of brainLink to Publisher's site
 
Brain. Sep 2009; 132(9): 2385–2395.
Published online May 4, 2009. doi:  10.1093/brain/awp094
PMCID: PMC2766178
Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients*
Nikoletta Bódi,1 Szabolcs Kéri,corresponding author1 Helga Nagy,2 Ahmed Moustafa,3 Catherine E. Myers,4 Nathaniel Daw,5 György Dibó,6 Annamária Takáts,2 Dániel Bereczki,2 and Mark A. Gluck3
1 Department of Psychiatry and Psychotherapy, Semmelweis University, Budapest, Hungary
2 Department of Neurology, Semmelweis University, Budapest, Hungary
3 Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ, USA
4 Department of Psychology, Rutgers University, Newark, NJ, USA
5 Department of Psychology, Center for Neural Science, New York University, New York, NY, USA
6 Albert Szent-Györgyi Clinical Center, Neurology Outpatient Unit, University of Szeged, Szeged, Hungary
corresponding authorCorresponding author.
*This article is dedicated to the memory of Marvin Blume, who passed away in 2008, following a long battle with Parkinson's disease.
Correspondence to: Dr Szabolcs Kéri, Department of Psychiatry and Psychotherapy, Semmelweis University, Balassa u. 6., Budapest H1083, Hungary E-mail: keriszabolcs/at/psych.sote.hu
Received December 15, 2008; Revised February 25, 2009; Accepted March 9, 2009.
Parkinson's disease is characterized by the degeneration of dopaminergic pathways projecting to the striatum. These pathways are implicated in reward prediction. In this study, we investigated reward and punishment processing in young, never-medicated Parkinson's disease patients, recently medicated patients receiving the dopamine receptor agonists pramipexole and ropinirole and healthy controls. The never-medicated patients were also re-evaluated after 12 weeks of treatment with dopamine agonists. Reward and punishment processing was assessed by a feedback-based probabilistic classification task. Personality characteristics were measured by the temperament and character inventory. Results revealed that never-medicated patients with Parkinson's disease showed selective deficits on reward processing and novelty seeking, which were remediated by dopamine agonists. These medications disrupted punishment processing. In addition, dopamine agonists increased the correlation between reward processing and novelty seeking, whereas these drugs decreased the correlation between punishment processing and harm avoidance. Our finding that dopamine agonist administration in young patients with Parkinson's disease resulted in increased novelty seeking, enhanced reward processing, and decreased punishment processing may shed light on the cognitive and personality bases of the impulse control disorders, which arise as side-effects of dopamine agonist therapy in some Parkinson's disease patients.
Keywords: Parkinson's disease, reward, novelty seeking, dopamine, pramipexole, ropinirole
Converging evidence from animal neurophysiology, neurochemistry and cognitive neuroscience in humans suggests that the basal ganglia and its dopaminergic projections from the midbrain are important for learning to predict rewarding outcomes (Schultz, 2006; Balleine et al., 2007). This reward prediction signal is critical in tasks during which responses to salient stimuli are modified by feedback. Given the well-known loss of dopaminergic signals in the basal ganglia in Parkinson's disease, it is not surprising that these patients show impairments on several tasks requiring feedback-based learning (Packard and Knowlton, 2002; Shohamy et al., 2008). However, this type of learning markedly depends on medication status and task demands. Specifically, l-DOPA medication, which enhances global dopamine levels, impairs certain types of feedback-based learning presumably because of the ‘overdosing’ of dopamine in brain areas less affected in Parkinson's disease (Cools et al., 2001, 2003; Shohamy et al., 2006). Kish et al. (1988) postulated that in early Parkinson's disease dopaminergic loss is more pronounced in the dorsal than in the ventral striatum. Therefore, l-DOPA doses that restore dopamine levels in the dorsal striatum may lead to dopamine ‘overdose’ in the less affected ventral striatum. This imbalanced effect may improve motor functions and cognitive flexibility, but, at the same time, may induce impulsivity and decreased performance on some tasks involving feedback-based learning (Cools et al., 2001, 2003; Shohamy et al., 2006).
The controversial effects of dopaminergic medications on cognition was further investigated by Frank et al. (2004, 2007) who demonstrated that Parkinson's disease patients off l-DOPA medication are better at learning from punishment (negative outcome) than from reward (positive outcome) during stimulus–response procedural learning tasks, and that medication reverses this pattern, enhancing learning from punishment at the expense of reward. Such a pattern had been suggested by computational models based on recordings from dopaminergic neurons (Schultz et al., 1997; Frank et al., 2004; Frank, 2005), which hypothesize that phasic excitation and inhibition of these neurons drives learning from better or worse than expected outcomes, respectively. Based on these models and on pharmacological studies in healthy participants (Frank and O’Reilly, 2006), it is possible that by enhancing dopaminergic action, l-DOPA facilitates learning from positive feedback (reward), but impairs learning from negative feedback (punishment). Cools et al. (2006) further confirmed this hypothesis using a feedback-based reversal learning task. In the study of Cools et al. (2006), a medication-induced deficit in reversal learning signalled by unexpected punishment was particularly pronounced in patients who received the dopamine D3 receptor agonist pramipexole.
Beyond laboratory tasks of cognition, dopamine and sensitivity to reward play a crucial role in more complex behavioural phenomena, such as human personality. The most widely investigated dopamine-related personality trait is novelty seeking, including exploratory excitability, impulsiveness, extravagance and disorderliness, as measured by the temperament and character inventory (TCI) (Cloninger, 1994). Anecdotal reports indicate rigid, punctual and introverted personality in Parkinson's disease, but results from controlled studies failed to provide equivocal evidence (Menza, 2000; Jacobs et al., 2001; Tomer and Aharon-Peretz, 2004). In accordance with decreased dopaminergic transmission, some reports suggest decreased novelty seeking in Parkinson's disease (Menza, 2000), but others also demonstrated increased scores in Parkinson's disease on another TCI dimension, harm avoidance (anticipatory worry, fear of uncertainty, shyness and fatigability), which may be related to depression (Jacobs et al., 2001).
An important problem in the literature of both laboratory cognition and personality measurement is that the vast majority of studies include chronic elderly patients with Parkinson's disease receiving multiple medications who display an advanced stage of the illness, severely affected cognition and mood disorders. Longitudinal follow-up studies are virtually missing from the literature. Finally, the relationship between feedback-based cognitive tasks and complex personality traits is not known. The general aim of this study was to clarify these issues. We were particularly interested in young patients with Parkinson's disease. Young-onset Parkinson's disease is associated with slower progression of motor symptoms, longer disease course with spared cognitive function, but an earlier appearance of motor fluctuations, dyskinesias and psychiatric symptoms (Schrag and Schott, 2006). The pathology is more circumscribed than in late-onset Parkinson's disease, but in some cases cell loss in the substantia nigra can be more pronounced (Gibb and Lees, 1988).
First, we recruited young, never-medicated patients with Parkinson's disease and a matched sample of Parkinson's disease patients who were on dopamine agonist therapy (cross-sectional part of the study) and did not receive any other drugs (e.g. l-DOPA, antidepressants). Second, we followed-up the never-medicated sample after the initiation of dopamine agonist therapies pramipexole or ropinirole (longitudinal, within-subject part of the study). We used a feedback-based probabilistic classification learning task developed by Myers, Daw and colleagues at Rutgers University, Newark (Bolikal et al., 2007) that enabled us to investigate stimulus-response learning guided by positive and negative feedback (winning and losing virtual money) (Fig. 1, Table 1). Results from this feedback-based task were compared with personality traits as measured by the TCI. Our key hypotheses were as follows: (i) Never-medicated patients with Parkinson's disease should show decreased novelty seeking, decreased sensitivity to reward and enhanced punishment learning on the feedback-based task; (ii) Parkinson's disease patients receiving dopamine agonist therapy should show increased sensitivity to reward and increased novelty seeking in both cross sectional and longitudinal designs; and (iii) novelty seeking should correlate with sensitivity to positive feedback and harm avoidance should correlate with negative feedback.
Figure 1
Figure 1
The feedback-based probabilistic classification task. (A) On each trial, the participant saw one of four stimuli and was asked whether this stimulus belonged to category A or B. (B) For some stimuli, correct responses were rewarded with visual feedback (more ...)
Table 1
Table 1
Category and feedback structure of the probabilistic classification task
Participants
Participants were patients with idiopathic Parkinson's disease who had never received dopaminergic medications or who had recently begun medication with dopamine receptor agonists. These patients were compared with healthy volunteers without a history of neurological or psychiatric disorders. The clinical and demographic data are shown in Table 2. The mean dose of pramipexole (n = 12) was 4.5 mg/day (range 2.5–6.0 mg/day), the mean dose of ropinirole (n = 10) was 5.5 mg/day (range 2.0–7.0 mg/day). After baseline testing, never-medicated Parkinson's disease patients started dopamine agonist therapy and were followed-up for 12 weeks (pramipexole n = 14, mean dose at follow-up: 4.0 mg/day, range 2.0–6.0 mg/day; ropinirole n = 12, mean dose at follow-up: 5.5 mg/day, range 2.0–7.5 mg/day). After this period, participants were re-evaluated.
Table 2
Table 2
Clinical and demographic characteristics of the participants
The symptoms of Parkinson's disease were evaluated by the Hoehn–Yahr Scale (Hoehn and Yahr, 1967) and the Unified Parkinson's Disease Rating Scale (UPDRS) (Lang and Fahn, 1989). The Hamilton Depression Rating Scale (HAM-D) and the Hamilton Anxiety Rating Scale (HAM-A) were used to evaluate mood and anxiety symptoms, respectively (Mountjoy and Roth, 1982). The socio-economic status was evaluated by the Hollingshead Four-Factor Index (Cirino et al., 2002). General intellectual abilities were determined using the revised version of the Wechsler Adult Intelligence Scale (WAIS-R) (Wechsler, 1981). All scales were administered by trained experts who were blind to personality measures, test performances and medication status. All participants gave written informed consent and the study was approved by the institutional ethics board.
Feedback-based probabilistic classification task
All participants were administered a computer-based probabilistic classification task (Bolikal et al., 2007). On each trial, participants viewed one of four images (Fig. 1), and were asked to guess whether it belonged to Category A or B. For each participant, the four images were randomly assigned to be stimuli S1, S2, S3 and S4. A second set of similar images (S5–S8) were used for repeated testing (test–retest reliability based on the repeated testing of controls: r = 0.76). On any given trial, stimuli S1 and S3 belonged to Category A with 80% probability and to Category B with 20% probability, while stimuli S2 and S4 belonged to Category B with 80% probability and to Category A with 20% probability (Table 1). Stimuli S1 and S2 were used in the reward-learning task. Two stimuli per valence were employed in order to balance category outcome frequencies, so that one stimulus in each task would be associated with each outcome. Thus, if the participant correctly guessed category membership on a trial with either of these stimuli, a reward of +25 points was received; if the participant guessed incorrectly, no feedback appeared. Stimuli S3 and S4 were used in the punishment-learning task. Thus, if the participant guessed incorrectly on a trial with either of these stimuli, a punishment of –25 was received; correct guesses received no feedback.
The experiment was conducted on a Macintosh i-book, programmed in the SuperCard language. The participant was seated in a quiet testing room at a comfortable viewing distance from the screen. The keyboard was masked except for two keys, labelled ‘A’ and ‘B’ which the participant could use to enter responses. At the start of the experiment, the participant read the following instructions: ‘In this experiment, you will be shown pictures, and you will guess whether those pictures belong to Category A or Category B. A picture does not always belong to the same category each time you see it. If you guess correctly, you may win points. If you guess wrong, you may lose points. You will see a running total of your points as you play. We will start you off with a few points now. Press the mouse button to begin practice’.
The practice phase then walked the participant through an example of a correct and an incorrect response to a sample trial in the punishment-learning task and an example of a correct and incorrect response to a sample trial in the reward-learning task. These examples used images other than those assigned to S1–S4. The participant saw a practice image, with a prompt to choose Category A or B, and a running tally of points at the lower right corner of the screen. The tally is initialized to 500 points at the start of practice. The participant was first instructed to press the ‘A’ key, which resulted in a punishment of –25 and updated point tally and then the ‘B’ key, which resulted in no feedback. The participant then saw a second practice figure and was instructed first to press the ‘B’ key which resulted in a reward of +25 and updated point tally and then the ‘A’ key, which resulted in no feedback.
After these two practice trials, a summary of instructions appeared: ‘So … for some pictures, if you guess CORRECTLY, you WIN points (but, if you guess incorrectly, you win nothing). For other pictures, if you guess INCORRECTLY, you LOSE points (but, if you guess correctly, you lose nothing). Your job is to win all the points you can – and lose as few as you can. Remember that the same picture does not always belong to the same category. Press the mouse button to begin the experiment’. From here, the experiment began. In each trial, the participant saw one of the four stimuli (S1, S2, S3 and S4) and was prompted to guess whether it was an ‘A’ or a ‘B’. On trials in the reward-learning task (with stimuli S1 or S2), correct answers were rewarded with positive feedback and a gain of 25 points; incorrect answers received no feedback. On trials in the punishment-learning task (with Stimuli S3 or S4), incorrect answers were punished with negative feedback and a loss of 25 points; correct answers received no feedback. The task contained 160 trials. Within a block, trial order was randomized. Trials were separated by a 2 s interval, during which time the screen was blank. Within each block, each stimulus appeared 10 times, 8 times with the more common outcome (e.g. category ‘A’ for S1 and S3 and ‘B’ for S2 and S4) and 2 times with the less common outcome. Thus, training on the reward-learning task (S1 and S2) and punishment-learning task (S3 and S4) were intermixed. The no-feedback outcome, when it arrived, was ambiguous, as it could signal lack of reward (if received during a trial with S1 or S2) or lack of punishment (if received during a trial with S3 or S4). At the end of the 160 trials, if the participant's running tally of points was <525 (i.e. no more than the points awarded at the start of the experiment), additional trials were added on which the participant's response was always taken as correct, until the tally was at least 525. This was done in an attempt to minimize frustration in participants by ensuring that all participants terminated the experiment with more points than they had started with. Data from any such additional trials were not analysed. On each trial, the computer recorded whether the participant made the optimal response (i.e. Category A for S1 and S3 and Category B for S2 and S4) regardless of actual outcome.
Personality measures
Following the probabilistic classification task, all participants were administered the Hungarian version of the TCI questionnaire, which has a good test–retest reliability (Rózsa et al. 2005). The TCI is suitable for the assessment of temperament and character traits. In this study, we focused on the temperament traits of novelty seeking (exploratory excitability, impulsiveness, extravagance and disorderliness), harm avoidance (anticipatory worry, fear of uncertainty, shyness and fatigability), reward dependence (sentimentality, openness to warm communication, attachment and dependence) and persistence (eagerness of effort, work–hardened, ambitious and perfectionist) (Cloninger, 1994). Thus, in addition to the main focus on novelty seeking and harm avoidance, data were also collected on reward dependence and persistence in order to test the specificity of possible alterations in personality traits.
Data analysis
The normality of data distribution was checked using Kolmogorov–Smirnov tests. All data were normally distributed (P > 0.1). Analyses of variance (ANOVAs) using the general linear model panel of the STATISTICA 7.0 software (StatSoft, Inc., Tulsa) were used to compare controls, never-medicated and recently medicated Parkinson's disease patients, and to compare the performance of patients at baseline (no medication) and at follow-up (dopamine agonists). ANOVAs were followed by planned F-tests and Tukey Honestly Significant Difference (HSD) tests. Two-tailed t-tests were used for the analysis of demographic data and personality measures. Pearson's product–moment correlation coefficients were calculated between test performance and personality measures. The Williams test was used to compare the correlation coefficients. The level of significance was set at α < 0.05.
Differences between never-medicated and recently medicated Parkinson's disease patients in sensitivity to positive and negative feedback
The results from the feedback-based task are shown in Fig. 2. The ANOVA, in which group (controls, never-medicated and recently medicated Parkinson's disease patients) was the between-subject factor and feedback-type (positive and negative) and trial blocks were the within-subject factors, revealed significant main effects of group [F(2,65) = 10.76, P < 0.001] and trial blocks [F(3,195) = 91.40, P < 0.001]. The two-way interaction between group and feedback-type was significant [F(2,65) = 210.11, P < 0.001], as was the three-way interaction among group, feedback-type and trial blocks [F(6,195) = 22.84, P < 0.001]. The main effect of feedback-type [F(1,65) = 0.45, P = 0.5], the interaction between feedback-type and trial blocks [F(3,195) = 2.06, P = 0.1], and the interaction between group by trial blocks [F(6,195) = 1.82, P = 0.1] did not reach the level of significance.
Figure 2
Figure 2
Results from the feedback-based probabilistic classification task. The never-medicated Parkinson's patients (nm PD) outperformed the recently medicated patients (m PD) in the punishment condition, whereas the recently medicated patients outperformed the (more ...)
This three-way interaction was further investigated by F-tests for linear trend. First, the controls were compared with the never-medicated Parkinson's disease patients. This analysis revealed a significant interaction among group, feedback-type and trial blocks [F(1,65) = 10.53, P < 0.001]. Second, the controls were compared with the recently medicated Parkinson's disease patients. This analysis also revealed a three-way interaction [F(1,65) = 34.14, P < 0.001]. Finally, a similar interaction was observed when the never-medicated and the recently medicated patients were compared [F(1,65) = 91.45, P < 0.001].
Tukey HSD tests conducted on the group-by-feedback-type interactions revealed that the never-medicated Parkinson's disease patients displayed significantly impaired performance on reward learning as compared with the controls (P < 0.001), whereas the opposite effect was found for punishment learning: the patients out-performed the controls (P < 0.01) (Fig. 2). When the recently medicated Parkinson's disease patients were compared with the controls, there was no significant difference for reward learning (P = 0.19), but the patients displayed significantly impaired performance on punishment learning (P < 0.001) (Fig. 2). Finally, when the recently medicated and the never-medicated Parkinson's disease patients were compared, we found that the recently medicated group out-performed the never-medicated group in the reward condition (P < 0.001), whereas the opposite was found in the punishment condition (P < 0.001) (Fig. 2).
These differences were not due to confounding variables such as age, education, IQ, or socio-economic status, because the patient groups and controls were similar in these measures. In addition, the above-described analyses remained the same when the time since diagnosis and UPDRS scores were included as covariants (time since diagnosis was significantly different between never-medicated and recently medicated patients, whereas UPDRS scores were not) (Table 2).
Personality measures in never-medicated and recently medicated Parkinson's disease patients
Data from the TCI are shown in Table 2. One-way ANOVAs indicated a significant main effect of group only in the case of novelty seeking [F(2,65) = 13.72, P < 0.0001]. The never-medicated Parkinson's disease patients exhibited significantly lower novelty-seeking scores compared with controls [t(44) = 3.34, P < 0.005] and with recently medicated patients [t(46) = –4.66, P < 0.0001]. In addition, the recently medicated patients exhibited significantly higher novelty-seeking scores compared with the controls [t(40) = –2.34, P < 0.05].
Correlation between performance on the feedback-based task and personality measures
In the healthy control group, there was a significant positive relationship between the percent of optimal choices on the feedback-based task for positive feedback (reward) and novelty-seeking scores (r = 0.49, P < 0.05). A similar tendency was observed in never-medicated Parkinson's disease patients, but this did not reach the level of statistical significance (r = 0.31, P > 0.1). Finally, we observed the strongest positive correlation in recently medicated Parkinson's disease patients (r = 0.75, P < 0.001) (Fig. 3). The correlation coefficients from the never-medicated and the recently medicated group showed a significant difference (Williams test, P < 0.05).
Figure 3
Figure 3
Correlations between novelty seeking and reward learning in controls (black), never-medicated Parkinson's patients (blue) and recently medicated patients (red).
In the healthy control group, we also observed a significant positive correlation between the percent of optimal choices on the feedback-based task for negative feedback (punishment) and harm avoidance scores (r = 0.67, P < 0.01), which also was present in never-medicated Parkinson's disease patients (r = 0.40, P < 0.05) but not in recently medicated patients (r = 0.11, P > 0.1) (Fig. 4). The correlation coefficients from the controls and the recently medicated group showed a significant difference (Williams test, P < 0.05).
Figure 4
Figure 4
Correlations between harm avoidance and punishment learning in controls (black), never-medicated Parkinson's patients (blue) and recently medicated patients (red).
When the correlation analysis was corrected for multiple comparisons (Bonferroni, alpha adjusted to 0.002), only the correlation between novelty seeking and reward learning in the recently medicated Parkinson's disease group, and the correlation between harm avoidance and punishment learning in the controls reached the level of significance.
Longitudinal result from the feedback-based task: retesting the never-medicated Parkinson's disease patients after the initialization of dopamine agonist therapy
At the follow-up phase, the mean UPDRS score was 26.4 (SD = 6.4), which was significantly lower than the score at baseline testing [mean 30.8, SD = 6.4, t(48) = 2.43, P < 0.05]. The HAM-D (mean at follow-up 4.1, SD = 2.1) and HAM-A (mean at follow-up 3.2, SD = 1.8) scores did not change relative to the baseline.
The longitudinal results from the feedback-based task are shown in Fig. 5. We tested how medication affected feedback-based task performance in patients with Parkinson's disease using an ANOVA in which group (controls versus Parkinson's disease) was the between-subject factor and testing time (baseline versus follow-up), feedback-type (positive versus negative) and trial blocks were the within-subject factors. This analysis revealed significant main effects of group [F(1,44) = 13.47, P < 0.01] and trial blocks [F(3,132) = 58.08, P < 0.001]. The main effects of testing time [F(1,44) = 2.04, P = 0.2] and feedback-type [F(1,44) = 0.98, P = 0.3] were not significant. There were significant two-way interactions between group and testing time [F(1,44) = 15.42, P < 0.001], group and feedback-type [F(1,44) =4.46, P < 0.05], group and trial blocks [F(3,132) = 2.75, P = 0.05] and testing time and feedback-type [F(1,44) = 149.18, P < 0.001]. The two-way interactions between testing time and trial blocks [F(3,132) = 0.41, P = 0.7] and feedback and trial blocks [F(3,132) =1.52, P = 0.2] were not significant. The three-way interactions among group, testing time and feedback-type [F(1,44) =148.62, P < 0.001] and group, testing type and trial blocks [F(3,132) = 3.77, P < 0.05] were significant, whereas the interaction among group, testing time and trial blocks [F(3,132) = 1.24, P = 0.3] was not significant. Finally, the four-way interaction among group, testing time, feedback-type and trial blocks was significant [F(3,132) = 14.21, P < 0.001]. The four-way interaction was examined by an F-test for linear trend, which confirmed the interaction [F(1,44) = 29.08, P < 0.001].
Figure 5
Figure 5
Results from the feedback-based probabilistic classification task at baseline and at follow-up when Parkinson's patients (PD) received pramipexole and ropinirole. In reward learning, performance in the unmedicated baseline condition (base) was signifcantly (more ...)
These complex results were further analysed using Tukey HSD tests, which were conducted on the critical group by testing time by feedback-type interaction. As an important control condition, these tests indicated that the performance of the controls was similar at baseline and follow-up for both reward and punishment (P > 0.5). Critically, in patients with Parkinson's disease, there were significant differences between the baseline and follow-up results: dopaminergic medications robustly improved reward-learning (P < 0.001) and disrupted punishment-learning (P < 0.001) (Fig. 5). At the follow-up assessment, the Parkinson's disease patients did not differ from controls on reward learning (P = 0.12), whereas they performed less effectively than controls on punishment-learning (P < 0.001).
Longitudinal data from personality measures
Dopaminergic medications significantly increased novelty seeking [mean at follow-up 20.3, SD = 6.2, t(48) = –2.26, P < 0.05], whereas harm avoidance (mean at follow-up 14.9, SD = 3.2) and reward dependence (mean at follow-up 16.0, SD = 4.2) did not change significantly (P > 0.1). This was not accompanied by clinical changes in mood and anxiety, because HAM-D and HAM-A scores were similar at the baseline and at the follow-up assessment.
Effect of different dopamine agonists, illness duration and symptoms
Data from the feedback-based task and TCI did not differ between patients receiving pramipexole and ropinirole (F < 2, P > 0.1). There were no significant correlations between the primary measures (performance on the feedback-based task and TCI scores), illness duration and UPDRS/HAM-A/HAM-D scores (all P > 0.1).
The effect of dopamine agonists on reward and punishment processing
The data from the present study are consistent with the findings originally reported by Frank et al. (2004) and replicated using different tasks (Cools et al., 2006; Frank et al., 2007; Moustafa et al., 2008). These results are extended and confirmed in young patients both cross-sectionally and longitudinally as a function of dopamine D2/3 receptor agonist medication only. We demonstrated that young, never-medicated patients with Parkinson's disease exhibit markedly reduced novelty seeking and reward processing, which are related to each other. Dopamine agonist pramipexole and ropinirole increased both novelty seeking and reward processing, as indicated by the data from the cross-sectional comparison of medicated and non-medicated patients and by the follow-up assessment of patients who received dopaminergic medications for the very first time in their life. In addition, dopamine agonists increased the correlation between reward processing and novelty seeking, whereas these drugs decreased the correlation between punishment processing and harm avoidance. Additionally, although punishment learning and harm avoidance were coupled in controls, dopamine agonists disrupted punishment learning without similarly affecting harm avoidance. This suggests that punishment learning and personality measures of harm avoidance are not as closely related as reward learning and novelty seeking in relation to dopaminergic changes. A possible explanation is that serotonin may also play an important role in these functions (Daw et al., 2002; Cools et al., 2008). Such a serotonergic mechanism could be modulated only indirectly by dopamine agonists, such as via opponency between the neuromodulators (Daw et al., 2002). Accordingly, the differential effect of medication on punishment learning and harm avoidance may reflect a more complex mechanism of dopaminergic action on either of these measures.
In summary, dopaminergic medications improved reward processing in the feedback-based task, whereas punishment learning was less efficient in medicated than in non-medicated states, which is consistent with the assumptions of Frank (2005). However, Frank et al. (2004) found cognitive enhancement but not impairment in Parkinson's disease patients receiving l-DOPA medications in comparison with controls. The current data show definitive impairment but not enhancement on the learning task relative to controls. This discrepancy may be explained by the different characteristics of participants (elderly versus young), stage of disease, medications (patients on and off l-DOPA versus dopamine agonists; chronically medicated versus never-medicated and recently medicated) and task differences. It is also worthy to note that our data are highly similar to the predictions of the neural network model of Frank et al. (2004), which indicated impaired reward learning in simulated Parkinson's disease and impaired punishment learning but less dominantly improved reward learning in simulated dopamine medications relative to an ‘intact’ condition.
In contrast to the task of Frank et al. (2004, 2007) and Cools et al. (2006), the feedback-based probabilistic classification test of this study is simple and allows a more direct assessment of reward and punishment processing; that is, we were able to depict a learning curve displaying performance changes across trials. The learning curves demonstrated a linear increment of performance in controls, which was less pronounced in Parkinson's disease (for reward learning in the unmedicated and for punishment learning in the medicated condition). In the task of Frank et al. (2004, 2007), reward/punishment learning was inferred from a transfer phase, in which further learning could occur. In addition, there could be context effects in both the acquisition and transfer phases of the Frank et al. (2004, 2007) task, because participants always learned to choose a stimulus in the context of another stimulus (i.e. they choose one of two stimuli). In other words, it is possible that, for example, when participants select one stimulus and receive positive feedback, they concurrently learn that the other stimulus they have not chosen is not rewarding. The same logic applies to learning from negative feedback: participants might learn that the other stimulus they have not chosen is rewarding. This might confound the claim that subjects learn to select or avoid stimuli based solely on positive or negative feedback. This, however, is not a concern in our task because subjects are presented with one stimulus on each trial.
Striatal dopamine and reward processing
Dopaminergic transmission in the striatum and its cortical projections play an important role in personality traits such as novelty seeking (Wittmann et al., 2008; Cohen et al., 2009), reward processing (Montague et al., 2006; Schultz, 2006; Balleine et al., 2007) and incentive motivation (Schmidt et al., 2008), which are affected in Parkinson's disease. The relationship between reward and novelty is not surprising, given that even early data from animal studies indicated that midbrain dopaminergic neurons respond to both reward and novel cues (Ljungberg et al., 1992). Functional imaging studies demonstrated that patients with Parkinson's disease activate a compensatory cortical network during reward processing, which is significantly different from that of controls (Künig et al., 2000; Goerendt et al., 2004; Keitz et al., 2008; Rowe et al., 2008). Leyton et al. (2002) demonstrated that increased extracellular dopamine in the ventral striatum is related to interest in obtaining rewards and novelty seeking. Novelty seeking is also associated with increased vulnerability to sensitization to psychostimulants eliciting dopamine release in the ventral striatum (Boileau et al., 2006). During instrumental learning, functional magnetic resonance imaging correlates of reward prediction error in the striatum are modulated by the administration of drugs enhancing (l-DOPA) or reducing (haloperidol) dopaminergic function (Pessiglione et al., 2006). Participants receiving l-DOPA are likely to choose the most rewarding action, as compared with people treated with the dopamine antagonist haloperidol (Pessiglione et al., 2006). Others suggested that dopaminergic mechanisms outside the basal ganglia may be implicated in novelty and reward, for instance, in the insula, which participates in emotional processing (Suhara et al., 2001), and the striatum may also be implicated in aversive learning and negative prediction errors (Seymour et al., 2004; for review, see Delgado et al., 2008). Nevertheless, predominant stimulation of the dopamine D3 receptors, which can be found in a high density in the ventral striatum (Sokoloff et al., 1990), may increase reward processing. Both pramipexole and ropinirole exhibit a high affinity to D3 receptors (Gerlach et al., 2003), which may explain why these medications increased reward learning in the feedback-based task.
Personality and Parkinson's disease
Personality measures in Parkinson's disease provided mixed results (Ishihara and Brayne, 2006). As reviewed by Menza (2000), several decades of research indicated that Parkinson's disease is associated with industriousness, punctuality, inflexibility, cautiousness and lack of novelty seeking, which might be seen in the premorbid phase and which persists after the onset of the motor illness. The most plausible explanation may be that decreased dopaminergic transmission results in low novelty seeking in Parkinson's disease. However, in a positron emission tomography study, Kaasinen et al. (2001) found that novelty seeking was not associated with (18)F-dopa uptake in any of the brain regions studied in patients with Parkinson's disease. In contrast, harm avoidance, associated with anxiety and depression, was not only increased in Parkinson's disease patients, but showed a paradoxical positive correlation with the (18)F-dopa uptake in the right caudate nucleus. Tomer and Aharon-Peretz (2004) suggested that patients with greater dopamine loss in the left hemisphere showed reduced novelty seeking, whereas patients with reduced dopamine in the right hemisphere reported higher harm avoidance compared with healthy controls. Kaasinen et al. (2004) found that decreased novelty seeking in Parkinson's disease patients may be related to altered dopaminergic transmission in the insula.
Our data from never-medicated, young, non-depressed patients clearly show that decreased novelty seeking and reward processing are early signs of Parkinson's disease. Our sample was not large enough to test differences between patients with right- and left-sided motor symptoms, but many of our patients (n = 20) displayed right-sided symptoms (left-hemisphere dopamine deficiency), which is consistent with the results of Tomer and Aharon-Peretz (2004).
Limitations and further directions
The most important limitation of the study is the small sample size, especially with regards to the personality measures and correlation analysis. However, young and never-medicated patients with Parkinson's disease are relatively rare and difficult to recruit for longitudinal studies. Although this study was not a randomized controlled trial to compare different types of dopamine agonists, patients receiving pramipexole and ropinirole were highly similar regarding demographical, clinical, personality and test characteristics, probably due to the homogeneous sample selected for the study.
We speculate that the marked and selective deficit of novelty seeking and reward processing, and its robust response to dopaminergic medications (increased novelty seeking/reward processing and decreased punishment processing), may be a characteristic feature of some early-onset Parkinson's disease patients and may represent a susceptibility factor to dopamine dysregulation syndrome and impulse control disorders, including compulsive dopaminergic drug use, pathological gambling, binge eating, hyperlibidinous behaviour, compulsive shopping and punding (Ferrara and Stacy, 2008; Wolters et al., 2008). Pathological gambling, for example, can be conceptualized as a form of excessive ‘exploitation’ of available rewards (Daw et al., 2006); such intensified focus on reward may also be coupled to an increased drive to seek and explore novel or salient stimuli, which dopaminergic and striatal mechanisms appear to treat similarly to rewards (Wittmann et al. 2008). Evans et al. (2005) demonstrated that age at onset and novelty-seeking personality traits are the two strongest predictors to dopamine dysregulation syndrome. Ondo and Lai (2008) arrived at a similar conclusion. It has been suggested that the D3 agonist pramipexole may have a special potency to elicit pathological gambling (Dodd et al., 2005). Patients with Parkinson's disease are characterized by risky choices in experimental games, paradoxically when cognitive functions are better in the early stage of the disease (Perretta et al., 2005; Pagonabarraga et al., 2007). In our sample, no patients presented clinically impulsive behaviour, which, however, does not exclude the possibility that increased novelty seeking and reward processing may be a progenitor or a latent marker for impulse controls symptoms. To further investigate this hypothesis, a more extended follow-up of larger samples is warranted in order to examine the relationship among task performance, personality measures and impulse control disorders.
Funding
Dekker Foundation; Bachman-Strauss Dystonia; Parkinson Foundation; NIH-NINDS (#R01-NS047[per 10 thousand]434-02 to M.A.G.).
  • Balleine BW, Delgado MR, Hikosaka O. The role of the dorsal striatum in reward and decision-making. J Neurosci. 2007;27:8161–5. [PubMed]
  • Boileau I, Dagher A, Leyton M, Gunn RN, Baker GB, Diksic M, et al. Modeling sensitization to stimulants in humans: an [11C]raclopride/positron emission tomography study in healthy men. Arch Gen Psychiatry. 2006;63:1386–95. [PubMed]
  • Bolikal P, Myers CE, Patel R, Ropp L, Daw N, Gluck MA. Cognitive Neuroscience Society Annual Meeting Program. 2007. Punishment-based learning correlates with a putative index of serotonin in healthy young adults. F148 [abstract]
  • Cirino PT, Chin CE, Sevcik RA, Wolf M, Lovett M, Morris RD. Measuring socioeconomic status: reliability and preliminary validity for different approaches. Assessment. 2002;9:145–55. [PubMed]
  • Cloninger RC. The temperament and character inventory (TCI): a guide to its development and use. St. Louis, MO: Center for Psychobiology of Personality, Washington University; 1994.
  • Cohen MX, Schoene-Bake JC, Elger CE, Weber B. Connectivity-based segregation of the human striatum predicts personality characteristics. Nat Neurosci. 2009;12:32–4. [PubMed]
  • Cools R, Altamirano L, D’Esposito M. Reversal learning in Parkinson's disease depends on medication status and outcome valence. Neuropsychologia. 2006;44:1663–73. [PubMed]
  • Cools R, Barker RA, Sahakian BJ, Robbins TW. Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex. 2001;11:1136–43. [PubMed]
  • Cools R, Barker RA, Sahakian BJ, Robbins TW. L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson's disease. Neuropsychologia. 2003;41:1431–41. [PubMed]
  • Cools R, Roberts AC, Robbins TW. Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn Sci. 2008;12:31–40. [PubMed]
  • Daw ND, Kakade S, Dayan P. Opponent interactions between serotonin and dopamine. Neural Netw. 2002;15:603–16. [PubMed]
  • Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ. Cortical substrates for exploratory decisions in humans. Nature. 2006;441:876–9. [PMC free article] [PubMed]
  • Delgado MR, Li J, Schiller D, Phelps EA. The role of the striatum in aversive learning and aversive prediction errors. Philos Trans R Soc Lond B Biol Sci. 2008;363:3787–800. [PMC free article] [PubMed]
  • Dodd ML, Klos KJ, Bower JH, Geda YE, Josephs KA, Ahlskog JE. Pathological gambling caused by drugs used to treat Parkinson's disease. Arch Neurol. 2005;62:1377–81. [PubMed]
  • Evans AH, Lawrence AD, Potts J, Appel S, Lees AJ. Factors influencing susceptibility to compulsive dopaminergic drug use in Parkinson disease. Neurology. 2005;65:1570–4. [PubMed]
  • Ferrara JM, Stacy M. Impulse-control disorders in Parkinson's disease. CNS Spectr. 2008;13:690–8. [PubMed]
  • Frank MJ. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci. 2005;17:51–72. [PubMed]
  • Frank MJ, O’Reilly RC. A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci. 2006;120:497–517. [PubMed]
  • Frank MJ, Samanta J, Moustafa AA, Sherman SJ. Hold your horses: impulsivity, deep brain stimulation, and medication in Parkinsonism. Science. 2007;318:1309–12. [PubMed]
  • Frank MJ, Seeberger LC, O’Reilly RC. By carrot or by stick: cognitive reinforcement learning in Parkinsonism. Science. 2004;306:1940–3. [PubMed]
  • Gerlach M, Double K, Arzberger T, Leblhuber F, Tatschner T, Riederer P. Dopamine receptor agonists in current clinical use: comparative dopamine receptor binding profiles defined in the human striatum. J Neural Transm. 2003;110:1119–27. [PubMed]
  • Gibb WR, Lees AJ. A comparison of clinical and pathological features of young- and old-onset Parkinson's disease. Neurology. 1988;38:1402–6. [PubMed]
  • Goerendt IK, Lawrence AD, Brooks DJ. Reward processing in health and Parkinson's disease: neural organization and reorganization. Cereb Cortex. 2004;14:73–80. [PubMed]
  • Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology. 1967;17:427–42. [PubMed]
  • Ishihara L, Brayne C. What is the evidence for a premorbid parkinsonian personality: a systematic review. Mov Disord. 2006;21:1066–72. [PubMed]
  • Jacobs H, Heberlein I, Vieregge A, Vieregge P. Personality traits in young patients with Parkinson's disease. Acta Neurol Scand. 2001;103:82–7. [PubMed]
  • Kaasinen V, Aalto S, Någren K, Rinne JO. Insular dopamine D2 receptors and novelty seeking personality in Parkinson's disease. Mov Disord. 2004;19:1348–51. [PubMed]
  • Kaasinen V, Nurmi E, Bergman J, Eskola O, Solin O, Sonninen P, et al. Personality traits and brain dopaminergic function in Parkinson's disease. Proc Natl Acad Sci USA. 2001;98:13272–7. [PubMed]
  • Keitz M, Koerts J, Kortekaas R, Renken R, de Jong BM, Leenders KL. Prefrontal cortex and striatal activation by feedback in Parkinson's disease. Brain Res. 2008;1236:225–33. [PubMed]
  • Kish SJ, Shannak K, Hornykiewicz O. Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease. Pathophysiologic and clinical implications. N Engl J Med. 1988;318:876–80. [PubMed]
  • Künig G, Leenders KL, Martin-Sölch C, Missimer J, Magyar S, Schultz W. Reduced reward processing in the brains of Parkinsonian patients. Neuroreport. 2000;11:3681–7. [PubMed]
  • Lang AE, Fahn S. Assessment of Parkinson's disease. In: Munsat TL, editor. Quantification of neurologic deficit. Boston, MA: Butterworth-Heinemann; 1989. pp. 285–309.
  • Leyton M, Boileau I, Benkelfat C, Diksic M, Baker G, Dagher A. Amphetamine-induced increases in extracellular dopamine, drug wanting, and novelty seeking: a PET/[11C]raclopride study in healthy men. Neuropsychopharmacology. 2002;27:1027–35. [PubMed]
  • Ljungberg T, Apicella P, Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol. 1992;67:145–63. [PubMed]
  • Menza M. The personality associated with Parkinson's disease. Curr Psychiatry Rep. 2000;2:421–6. [PubMed]
  • Montague PR, King-Casas B, Cohen JD. Imaging valuation models in human choice. Annu Rev Neurosci. 2006;29:417–48. [PubMed]
  • Mountjoy CQ, Roth M. Studies in the relationship between depressive disorders and anxiety states. Part 1. Rating scales. J Affect Disord. 1982;4:127–47. [PubMed]
  • Moustafa AA, Cohen MX, Sherman SJ, Frank MJ. A role for dopamine in temporal decision making and reward maximization in parkinsonism. J Neurosci. 2008;28:12294–304. [PMC free article] [PubMed]
  • Ondo WG, Lai D. Predictors of impulsivity and reward seeking behavior with dopamine agonists. Parkinsonism Relat Disord. 2008;14:28–32. [PubMed]
  • Packard MG, Knowlton BJ. Learning and memory functions of the basal ganglia. Annu Rev Neurosci. 2002;25:563–93. [PubMed]
  • Pagonabarraga J, García-Sánchez C, Llebaria G, Pascual-Sedano B, Gironell A, Kulisevsky J. Controlled study of decision-making and cognitive impairment in Parkinson's disease. Mov Disord. 2007;22:1430–5. [PubMed]
  • Perretta JG, Pari G, Beninger RJ. Effects of Parkinson disease on two putative non-declarative learning tasks: probabilistic classification and gambling. Cogn Behav Neurol. 2005;18:185–92. [PubMed]
  • Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature. 2006;442:1042–5. [PMC free article] [PubMed]
  • Rowe JB, Hughes L, Ghosh BC, Eckstein D, Williams-Gray CH, Fallon S, et al. Parkinson's disease and dopaminergic therapy–differential effects on movement, reward and cognition. Brain. 2008;131:2094–105. [PubMed]
  • Rózsa S, Kállai J, Osváth A, Bánki MC. Temperamentum és Karakter: Cloninger pszichobiológiai modellje. A Cloninger-féle Temperamentum és Karakter Kérdőív felhasználói kézikönyve. Medicina: Budapest. 2005
  • Schmidt L, d’Arc BF, Lafargue G, Galanaud D, Czernecki V, Grabli D, et al. Disconnecting force from money: effects of basal ganglia damage on incentive motivation. Brain. 2008;131:1303–10. [PubMed]
  • Schrag A, Schott JM. Epidemiological, clinical, and genetic characteristics of early-onset Parkinsonism. Lancet Neurol. 2006;5:355–63. [PubMed]
  • Schultz W. Behavioural theories and the neurophysiology of reward. Annu Rev Psychol. 2006;57:87–115. [PubMed]
  • Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–9. [PubMed]
  • Seymour B, O’Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, et al. Temporal difference models describe higher-order learning in humans. Nature. 2004;429:664–7. [PubMed]
  • Shohamy D, Myers CE, Geghman KD, Sage J, Gluck MA. L-dopa impairs learning, but spares generalization, in Parkinson's disease. Neuropsychologia. 2006;44:774–84. [PMC free article] [PubMed]
  • Shohamy D, Myers CE, Kalanithi J, Gluck MA. Basal ganglia and dopamine contributions to probabilistic category learning. Neurosci Biobehav Rev. 2008;32:219–36. [PMC free article] [PubMed]
  • Sokoloff P, Giros B, Martres MP, Bouthenet ML, Schwartz JC. Molecular cloning and characterization of a novel dopamine eceptor (D3) as a target for neuroleptics. Nature. 1990;347:146–51. [PubMed]
  • Suhara T, Yasuno F, Sudo Y, Yamamoto M, Inoue M, Okubo Y, et al. Dopamine D2 receptors in the insular cortex and the personality trait of novelty seeking. Neuroimage. 2001;13:891–5. [PubMed]
  • Tomer R, Aharon-Peretz J. Novelty seeking and harm avoidance in Parkinson's disease: effects of asymmetric dopamine deficiency. J Neurol Neurosurg Psychiatry. 2004;75:972–5. [PMC free article] [PubMed]
  • Wechsler D. Wechsler Adult Intelligence Scale—Revised Manual. New York, NY: Psychological Corporation; 1981.
  • Wittmann BC, Daw ND, Seymour B, Dolan RJ. Striatal activity underlies novelty-based choice in humans. Neuron. 2008;58:967–73. [PMC free article] [PubMed]
  • Wolters ECh, van der Werf YD, van den Heuvel OA. Parkinson's disease-related disorders in the impulsive-compulsive spectrum. J Neurol. 2008;255(Suppl 5):48–56. [PubMed]
Articles from Brain are provided here courtesy of
Oxford University Press