PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Hum Brain Mapp. Author manuscript; available in PMC 2013 December 1.
Published in final edited form as:
PMCID: PMC3731774
NIHMSID: NIHMS483305

Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence

Abstract

Fluid intelligence represents the capacity for flexible problem solving and rapid behavioral adaptation. Rewards drive flexible behavioral adaptation, in part via a teaching signal expressed as reward prediction errors in the ventral striatum, which has been associated with phasic dopamine release in animal studies. We examined a sample of 28 healthy male adults using multimodal imaging and biological parametric mapping with 1) functional magnetic resonance imaging during a reversal learning task and 2) in a subsample of 17 subjects also with positron emission tomography using 6-[18F]fluoro-L-DOPA to assess dopamine synthesis capacity. Fluid intelligence was measured using a battery of nine standard neuropsychological tests. Ventral striatal BOLD correlates of reward prediction errors were positively correlated with fluid intelligence and, in the right ventral striatum, also inversely correlated with dopamine synthesis capacity (FDOPA Kinapp). When exploring aspects of fluid intelligence, we observed that prediction error signaling correlates with complex attention and reasoning. These findings indicate that individual differences in the capacity for flexible problem solving may be driven by ventral striatal activation during reward-related learning, which in turn proved to be inversely associated with ventral striatal dopamine synthesis capacity.

Keywords: prediction error, dopamine synthesis, fluid intelligence, ventral striatum, fMRI, PET

Introduction

The fluid intelligence quotient (fluid IQ) [Horn and Cattell, 1966;Sternberg, 2000] represents the capacity of an individual for interpreting novel stimuli and flexible behavioral adaptation, whereas crystallized IQ reflects learning over the lifespan and has been associated with a neurobiological signature expressed in cortical structure [Choi et al., 2008]. Fluid IQ is a general factor comprising attributes such as attention, cognitive speed, working memory, reasoning, and episodic memory, which have been linked to activation in brain areas such as the dorsolateral prefrontal cortex (dlPFC) [Fuster, 2000;Goldman-Rakic et al., 2000] and striatum [Cools et al., 2008;Landau et al., 2009]. Fluid IQ declines over the human lifespan [Salthouse, 1992], and it has been suggested that individual differences in cognitive functions such as cognitive speed and working memory performance are associated with alterations in dopaminergic neurotransmission in the striatum [Cools et al., 2008;Landau et al., 2009;Salthouse, 1992].

In animal studies of reward-related learning, the firing properties of dopaminergic midbrain neurons correlate with trial-by-trial changes in errors of reward prediction (PEs), which reflect the difference between the expected reward and the reinforcement that was actually received [Schultz et al., 1997]. This finding was paralleled in human neuroimaging studies, which showed that functional activity in ventral striatum (VS), a target area of dopaminergic midbrain neurons, is correlated with PEs [O'Doherty, 2004;Pessiglione et al., 2006]. PE signals can be used to update the reward values of stimuli or actions in striatum [e.g., Frank and Claus, 2006] and hence play a key role in trial-and-error learning. As such, they are a key force driving flexible adaptation [Frank and Claus, 2006], and are intimately related to broad aspects of decision making in disease [Murray et al., 2008;Park et al., 2010] and across the life span [Lindenberger and Baltes, 1997].

Despite these findings, a relationship between individual differences in fluid IQ, reward-related learning and dopamine neurotransmission has not yet been formally demonstrated. In this study, our main aim was to relate blood oxygen level-dependent (BOLD) VS PE signals derived from a model-based analysis of learning performance during a reversal learning task [Park et al., 2010] with 1) individual differences in a composite measure of fluid IQ [Lindenberger and Baltes, 1997] and 2) VS dopamine synthesis capacity as measured with positron emission tomography (PET) with 6-[18F]fluoro-L-DOPA [Kienast et al., 2008]. We used biological parametric mapping to test for an association between functional magnetic resonance imaging (fMRI) and PET data in a voxel-wise manner while controlling for grey matter volume differences.

Materials and Methods

Subjects and screening instruments

A previously unpublished sample of 28 right-handed healthy men with a mean age of 36.9 years (SD = 12.4, range 22–61) underwent fMRI and neuropsychological testing. A subgroup of 17 participants (39.4 years SD = 12.1, range 19–61 years) was also investigated with FDOPA PET. Subjects with axis I and II psychiatric disorders according to DSM IV were excluded through the Structured Clinical Interview [First et al., 2001;First et al., 1997] and drug abuse was further excluded with urine tests. The study was approved by the local Ethics Committee of the Charité – Universitätsmedizin Berlin according to the Declaration of Helsinki, and written informed consent was obtained from all participants.

Neuropsychological assessment and intelligence measures

A neuropsychological battery was given during an initial session within two months of PET and fMRI measurements. Components of fluid and crystallized IQ were measured with an adaptation of the standard battery used in the Berlin Aging Study [Lindenberger and Baltes, 1997]. Fluid IQ was measured with a battery of nine tests comprising cognitive speed (i.e, the Digit Symbol Substitution test [Wechsler, 1955]; Reitan Trailmaking test, part A [REITAN, 1955]), attention and executive function (Reitan Trailmaking test, part B [REITAN, 1955]; Stroop color-word interference test [Stroop and ., 1935]), working memory (forward and backward digit span tests [Wechsler, 1955], episodic memory (Rivermead Behavioral Memory Test [Beckers et al., 1992]; auditory verbal learning test [Helmstaedter et al., 2001]), and reasoning (figural analogies [Horn, 1983]: for details, see Supplementary Methods S1).

Fluid IQ scores were derived from a factorial analysis of the raw scores of each of these tests using SPSS 11.0 for Macintosh (SPSS, Inc.). Specifically, we used a Varimax rotation with an Eigenvalue cutoff set to 1.0; the final (single factor) solution accounted for 74.3% of the variance in the nine tests provided). The final composite measure of fluid IQ hence reflects z-scores of aggregates of the nine tests provided derived from factorial analysis for the measured sample.

Furthermore, in order to allow for correlations within subdomains of fluid IQ we averaged z-scores of each test for each domain (i.e., for cognitive speed, DSST and Trailmaking A; for complex attention, Trailmaking B and Stroop; for working memory, forward and backward digit span; for episodic memory, RBMT and AVLT, for reasoning, LPS-3 scores were used). Correlations between mean standardized scores in each domain (cognitive speed, attention, working memory, episodic memory, and reasoning) and the composite fluid IQ score ranged from .54 to .97 (all p-values <.001).

Crystallized IQ was estimated using a verbal knowledge test [Schmidt and Metzler, 1992], during which subjects are required to identify each one meaningful word from a total of 42 lists of five words (of which four are nonsense words), which are ordered in increasing difficulty as reflected by the frequency of word use, and raw scores were used as the variable of interest.

Reversal learning task

During fMRI acquisition, subjects performed a reversal learning task [Cools et al., 2002;Kahnt et al., 2009;Park et al., 2010] known to evoke a BOLD PE signal in the striatum [O'Doherty et al., 2004;O'Doherty, 2004]. In each of 200 trials (100 per session), subjects first saw two abstract targets on the screen and were asked to choose one of them as quickly as possible by pressing the left or right button with the left or right thumb on a response box (maximum response time: 2s). A blue box surrounding their chosen target and feedback (either a green smiley face for positive feedback or a red frowning face for negative feedback) were simultaneously shown for 1s. The trials were separated with a jittered interval of 1–6.5s.

Participants went through a random sequence of three types of blocks. In block type 1, a reward was delivered for choosing the right stimulus if less than 80% of the recent right choices had been rewarded, and a punishment delivered otherwise. Conversely, a punishment was delivered for choosing the left stimulus if less than 80% of the recent left choices had been punished, and a reward delivered otherwise. In block type 2 the contingencies were simply reversed for left and right. In block type 3, the probabilities were 50/50 instead of 80/20. Switches between blocks happened always after 16 trials, or any time after 10 trials if subjects reached 70% correct choices.

Computational modeling of reinforcement learning

The trial-by-trial sequence of choices for each subject was fit by a simple Rescorla-Wagner (RW) model [Sutton and Barto, 1998]. This model assumes that the likelihood of a subject choosing action a on trial t is proportional to a value Qt(a) and given by the softmax

equation M1

The value Qt(a) in turn is the expected value of that action, i.e. the expected reinforcement conditional on taking the action. It is updated iteratively

equation M2

where ε is the learning rate. The variable Rt represents the effective reinforcement sensitivity as expressed by the effect of the reinforcement on the subject’s choice behavior. This variable took on value Rt = βrew if a reward was obtained and -βpun if a punishment was obtained. To fit the models, parameters are transformed to lie on the real line vector of parameters. Letting θ=[ε’, βpun’, βrew’] denote the vector of transformed parameters, we report the maximum a posteriori estimates of these parameters using a Gaussian prior with mean and variance parameters µ and ∑:

equation M3

where Ai represents all the actions by subject i and where the dependence of each individual action probability on the parameters θ determining the Q value was emphasized. Importantly, we set the prior parameters empirically using Expectation Maximization to find the maximum likelihood estimates of µ and ∑ given all the data by all the subjects. Separate parameters were fitted to each subject. An empirical Bayesian approach was used to constrain the individual parameters and fit the prior distribution to the data directly.

Based on the individually fitted parameters θi for each of the subjects, a temporal sequence of PEs was computed for each subject i as

equation M4

FMRI protocol

fMRI acquisition

Functional imaging was conducted using a 3.0 Tesla GE Signa scanner with an 8 channel phase array head coil to acquire gradient echo T2*-weighted echo-planar images as previously described [Kahnt et al., 2009;Park et al., 2010]. For each of the two sessions, 310 EPI volumes (~12 min) containing 29 slices (4mm thick) were acquired (repetition time (TR) = 2300 ms, echo time (TE) = 27 ms, matrix size 128×128 and a field of view (FOV) 256 × 256 mm2, thus yielding an in-plane voxel resolution of 2.7 mm2, flip angle α=90 degree). A 3D anatomical image of the entire brain was obtained by using a T1-weighted 3D spoiled-gradient echo pulse sequence with (TR = 7.8 ms, TE=3.2 ms, matrix size 256 × 256, FOV 256 × 256 mm2, 1 mm slice thickness, flip angle α =20 degree, voxel size 1 mm × 1 mm × 1mm).

fMRI data preprocessing

Functional imaging data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). ArtRepair was used to apply to remove noise spikes and bad slices within one scan and bad slices were repair by interpolation (“Noise Filtering”, http://cibsr.stanford.edu/tools/ArtRepair/ArtRepair.htm). After that the following preprocessing steps were performed: acquisition time and motion correction, coregistration of the mean EPI to the anatomical T1 image, spatial normalization and segmentation into tissue classes of the T1 image using the unified segmentation approach as implemented in SPM8 [Ashburner and Friston, 2005], application of the normalization parameters to all functional images, and spatial smoothing with an isotropic Gaussian kernel of 8mm full width at half maximum (FWHM) kernel.

Statistical analysis

The images were analyzed in an event related manner using the general linear model approach (GLM) as implemented in SPM8, Neuronal activity was modeled for win and loss trials separately by stick functions at the onsets of the feedback. We used a parametric design [Buchel et al., 1998;O'Doherty et al., 2007], in which the trial-by-trial PE values from the Rescorla-Wagner (RW) model modulated the amplitude of the trial related stick. Regressors of interest for the BOLD-responses corresponding to the trial-wise PEs were generated by convolving the modulated stimulus functions with the canonical hemodynamic response function (HRF), provided by SPM8. To account for signal fluctuations associated to the movement by susceptibility interaction, the six movement parameters from the realignment preprocessing step were included in the model as additional regressors. The individual contrast images for the contrast of the PE modulated feedback (combining win and loss feedback) were then taken to a random effects group-level analysis using a one sample t-test. To test for associations with measures of IQ, these measures were entered as covariates into additional random effects analyses. To control for age and individual fit of the Rescorla-Wagner model, these variables were added as an additional covariate in the SPM analyses. Correlations were plotted using the mean PE-related signal in VS VOIs (described below) and fluid IQ.

Correction for multiple comparisons

Small volume correction was used within the VS volume of interest (VOI). The VS VOI was constructed based on coordinates of previous findings using an in house tool provided by one of the authors (TW) to create an fMRI-literature based probabilistic VOI for the VS. To this end, we selected 16 recent papers containing data from 325 subjects [Bray and O'Doherty, 2007;Cohen, 2007;Cohen and Ranganath, 2005;D'Ardenne et al., 2008;Gershman et al., 2009;Kahnt et al., 2009;Krugel et al., 2009;Murray et al., 2008;O'Doherty et al., 2004;O'Doherty et al., 2003;Palminteri et al., 2009;Pessiglione et al., 2006;Rodriguez et al., 2006;Schonberg et al., 2010;Tobler et al., 2006;Valentin and O'Doherty, 2009]. From each study, the coordinates of PE related activation for right and the left VS were extracted (see Supplementary Methods S2).

PET protocol

PET acquisition

We used PET with FDOPA [Heinz et al., 2005;Kienast et al., 2008;Kumakura et al., 2007;Meyer-Lindenberg et al., 2002] to define the magnitude of the net blood brain clearance of FDOPA, designated as Kinapp, which has units of cerebral blood flow (ml g−1 min−1). Subjects reclined on the scanning bed and their head positioned within the aperture of the PET/CT (Siemens Biograph 16) scanner in 3-D mode. After a low dose CT-scan, a dynamic „list-mode“ emission recording lasting 124 minutes started immediately after intravenous bolus administration of 200 MBq FDOPA. After CT-based tissue attenuation correction and scatter correction, listmode data were iteratively reconstructed (OSEM, 16 iterations with 6 subsets) and framed (30 frames: 3 × 20 s, 3 × 1 min, 3 × 2 min, 3 × 3 min, 15 × 5 min, 3 × 10 min). Arterial blood samples were collected during the emission recording (in the first 6 min continuous measuring using a blood sampler, then manually at intervals), and the total radioactivity concentration in plasma samples was measured using a well-counter cross-calibrated to the PET. The fractions of untransformed FDOPA and the main metabolite O-methyl-[18F]-fluoro-L-DOPA (OMFD) were measured in plasma extracts from blood collected at 5, 15, 30, 45, and 60 post injection by reversed phase HPLC, and the continuous arterial input functions were calculated by bi-exponential fitting of the measured fractions [Gillings et al., 2001]. Initially we also planned to calculate Vd similar to [Kienast et al., 2008]. Due to sever motion artifacts in 9 out of 17 participants during the second scanning hour Kinapp was used for the first scanning hour (frame 1 to 20).

PET data preprocessing

PET data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). The emission recording frames and the individual T1 image were coregistered to frame 12. The individual anatomical T1 image was spatially normalized using the unified segmentation approach of SPM8 [Ashburner and Friston, 2005], and the computed normalization parameters were applied to all frames.

Voxel-wise quantification of net blood brain clearance of FDOPA

The net blood-brain clearance of FDOPA from plasma to brain (Kinapp, ml g-1 min-1) was calculated voxel-wise by Gjedde-Patlak Linear Graphic analysis [Patlak and Blasberg, 1985] after subtracting the radioactivity measured in the cerebellum (using the mean activity within a standard cerebellum mask as defined in the WFU Pick Atlas), and using the frames of the first 60min of the emission recording for the linear analysis [Cumming and Gjedde, 1998;Kumakura and Cumming, 2009]. The Kinapp images were spatially smoothed with a Gaussian kernel of 8 mm full width at half maximum (FWHM). This net blood-brain clearance is a macroparameter defined as K1*k3/(k2 + k3), where K1 and k2 describe the partitioning across the blood brain barrier, and k3 is the relative activity of DOPA decarboxylase with respect to exogenous FDOPA. As such FDOPA Kinapp describes the capacity to synthesize the dopamine from exogenous dopamine, as distinct from the dopamine synthesis rate, which depends on the unknown brain activity of tyrosine hydroxylase.

Biological Parametric Mapping (BPM)

To test for association of local Kinapp and local PE related BOLD response Biological Parametric Mapping [BPM; Casanova et al., 2007] was used. Because both values appear to depend on local neuronal structures as reflected in grey matter density [Goense and Logothetis, 2008;Woodward et al., 2009], data should be corrected for this unspecific – usually age related – proportion of variance. Group models containing locally specific variable values can take advantage of this multimodal information and can best preserve the physiological meaningful local relationships. However, the algorithms currently implemented in SPM are not able to fit such locally different models to the data. To overcome this limitation, we used the BPM-Toolbox [Casanova et al., 2007]. The conceptual difference between this approach and a conventional SPM style group analysis is in the use of other images as covariates. To evaluate the impact of dopamine synthesis capacity on the fMRI PE signal a BPM ANCOVA design was used with fMRI as primary modality, FDOPA Kinapp maps and grey matter density from the structural MRI as imaging covariate and age as non-imaging covariate. Modulated gray matter density images of each participant were calculate from the structural high-resolution T1-weigthed MRI images using the unified segmentation approach [Ashburner and Friston, 2005] as implemented in SPM8 and smoothed with a 8mm full width at half maximum Gaussian kernel.

Results

Participants made on average 72.8% +/− 6.3% correct responses and reached criterion on 5.5 +/− 1.3 conditions with a learning speed of 6.8 +/− 1.0 trials. In the fMRI group of 28 healthy controls, we observed a significant PE signal in VS (right VS: t=3.113; x=20, y=6, z=−5; pFWE-corrected for VS VOI =0.019; left VS: t=3.230 x=−11, y=8, z=−3; pFWE-corrected for VS VOI =0.019) (Figure 1).

Figure 1
Prediction error signal in the bilateral ventral striatum

Fluid IQ was significantly and positively correlated with the BOLD PE signal in bilateral VS (right VS: t=3.617; x=20, y=3, z=−8; pFWE-corrected for VS VOI=0.007; and left VS: t=2.994; x=−11, y=8, z=−3; pFWE-corrected for VS VOI=0.031), while crystalline IQ displayed no association, even when applying a much lower statistical threshold (pFWE-corrected for VS VOI>0.2). There was no correlation between fluid IQ and correct responses on the reversal learning task (r=−.015, p>.9).

Fluid IQ declined with age (r=−.647; p<0.001). Upon controlling for age by introducing age as an additional covariate into the SPM analysis, VS PE signal remained associated with fluid IQ (right VS: t=3.959; x=20, y=6, z=−8; pFWE-corrected for VS VOI=0.004; left VS: t=3.970; x=−11, y=11, z=−3; pFWE-corrected for VS VOI=0.004), suggesting that this association was not simply explained by an age-related decline in fluid IQ (Figure 2). The positive correlation between VS PE signal and fluid IQ also remained significant when individual model fit (predictive probability) was introduced as an additional covariate (right VS: t=3.47; x=17, y=6, z=−8; pFWE-corrected for VS VOI=0.010; left VS: t=3.32; x=−11, y=11, z=−3; pFWE-corrected for VS VOI=0.001).

Figure 2
Positive Correlation between prediction error signal and fluid intelligence

Regarding exploratory analyses of associations between components of fluid IQ (attention, reasoning, working memory, episodic memory, and cognitive speed) while controlling for age, we observed a significant positive correlation of VS PE-related signals with attention (left VS: t=3.117; x=−11, y=11, z=−3; pFWE-corrected for VS VOI =0.026) and reasoning performance (right VS: t=2.735; x=15, y=8, z=−5; pFWE-corrected for VS VOI =0.043; left VS: t=3.024; x=−16, y=11, z=−3; pFWE-corrected for VS VOI =0.030).

A subgroup of the participants (n=17; 39.4 years SD = 12.1, range 19–61 years) also underwent PET with 6-[18F]fluoro-L-DOPA (FDOPA) to directly measure individual differences in dopamine metabolism. Dopamine synthesis capacity (Kinapp; ml g−1 min−1), which represents the net influx of FDOPA to the brain relative to the metabolite-corrected plasma input, was computed on a voxel-wise basis [Kienast et al., 2008].

The association between local BOLD response elicited by PE and local dopamine synthesis capacity Kinapp was tested on a voxel-by-voxel basis to best take advantage of the multimodal imaging information. The Biological Parametric Mapping Toolbox (BPM) was applied [Casanova et al., 2007], which in contrast to standard SPM analyses allows the use of other parametric images such as Kinapp as covariates. This analysis revealed an inverse correlation between local FDOPA Kinapp and BOLD PE signal in right VS (t=4.40; x=15, y=13, z=−8; pFWE-corrected for VS VOI=0.011), but not on the left side (pFWE-corrected for VS VOI.>0.8; uncorrected p=0.020).

Discussion

We learn from making mistakes and need to adapt our predictions in the face of changing circumstances; iterative learning via PEs plays a major role in such learning processes [O'Doherty et al., 2004;Park et al., 2010;Pessiglione et al., 2006]. Our data show for the first time that individual neuronal signatures of PEs were directly related to individual differences in fluid IQ, even when correcting for the decline of fluid IQ with age: The age-independent association between fluid IQ and the functional correlate of a VS PE signal suggests that the VS contributed to individual differences in cognitive flexibility.

This interpretation was given credence by two further analyses. First, more detailed analyses of the sub-components of fluid IQ, which revealed that VS PE signaling was associated particularly with capacities such as complex attention and reasoning performance that are intimately related to error detection (20). Second, the correlation between fluid IQ and the VS BOLD correlate of PEs remained significant when we controlled for how well the PEs accounted for behavior (i.e. when controlling for the model fit). This latter result is critical. If the VS BOLD correlates strongly with PEs in one subject but not another one, then that might just be because one subject used PEs to guide their behavior, while the other subject did not. Had the correlation with fluid IQ not survived correction for model fit, then the conclusion would have had to be that subjects with higher IQ were more likely to learn using PEs than subjects with lower IQ, and thus that the VS correlate was a reflection of this strategic difference. Furthermore, no direct relationship was observed between fluid IQ and performance on the reversal learning task. Our data do not support that conclusion.

An additional consideration of importance is that several types of learning can co-exist, and be expressed at different times. In rats, lesions of different parts of the prefrontal cortex can reveal ‘latent’ habitually learned responses when goal-directed decisions are normally expressed [Killcross and Coutureau, 2003]. In this paper, the use of PEs is explicitly motivated by their neurobiological face value in terms of DA phasic responses [Pessiglione et al., 2006;Schultz et al., 1997]. However, such iterative PE learning forfeits much of the intricate structure of reversal learning tasks (for instance Hidden Markov Models; [for instance Hidden Markov Models; Hampton et al., 2006]. As such, the BOLD correlates of PE learning we observe is best taken as an index of PE learning that is ongoing even though it may not be fully expressed (and thus not be correlated with behavioral fit). Thus, subjects with higher fluid IQ show a higher subcortical, ongoing, RW learning even when this is not in charge. This suggests an interpretation whereby high fluid IQ subjects show a more varied, multifaceted approach to learning: rather than only exploiting one aspect of the reinforcement feedback, they exploit multiple interpretations of reinforcements, possibly allowing for more flexible future deployment of a larger variety of behavioral strategies. Our data point to VS PE signals as one key ingredient of such fluid flexibility.

The PE signal closely matches the temporal properties of a phasic dopamine response [Schultz et al., 1997] and appears to be modulated by dopamine agonists [Pessiglione et al., 2006]. While acute striatal dopamine release correlated with functional activation of the substantia nigra/ventral tegmental area during reward anticipation [Schott et al., 2008], it has to date not been explored whether individual differences in dopaminergic neurotransmission as assessed in vivo with PET are directly correlated with the PE related BOLD response [O'Doherty, 2004]. Like functional activation during reward anticipation, the PE signal may well be triggered by phasic alterations in firing rates of dopaminergic neurons and their respective effect on striatal neurotransmission [Pessiglione et al., 2006], while the magnitude of dopamine synthesis capacity Kinapp likely reflects the local parenchymal brain capacity to form [18F]fluorodopamine from plasma FDOPA, and to retain that product within vesicles, mainly located in dopaminergic nerve terminals. FDOPA Kinapp is thus widely accepted as a surrogate marker for the activity of DOPA-decarboxylase and is interpreted as an index of dopamine synthesis capacity [Kumakura and Cumming, 2009]. Studies in Parkinson’ disease patients with a loss of nigrostriatal nerve terminals found reduced FDOPA Kinapp [Kumakura and Cumming, 2009] and therefore this macroparameter may reflect the density of dopaminergic innervations [Pate et al., 1993]. Therefore, one may have expected to find a positive correlation between VS PE signaling and dopamine synthesis capacity.

Indeed, dopamine depletion via blockade of synthesis reduces electrically evoked dopamine release [Venton et al., 2006]; however, such a rather profound stimulation of dopamine release rapidly depletes presynaptic dopamine storage pools and may impair continuous tonic as well as stimulus-driven phasic [Goto et al., 2007] dopamine release. In our study, we observed a negative correlation in right VS between dopamine synthesis capacity and the functional PE signal. One explanation for this comes from animal and human studies with DA agonists and antagonists, which suggest that dopamine synthesis capacity is regulated by presynaptic autoreceptors [Cumming et al., 1997;Vernaleken et al., 2006]. Hassoun [Hassoun et al., 2005] examined the Kin from 60 to 90 minutes after injection and compared the baseline measurement with a Kin calculated during sensory stimulation and found that increased dopamine release induced by the sensory stimulation was associated with a decreased striatal synthesis capacity. Using a different approach compared to our study, which only assessed baseline dopamine synthesis capacity, the observation of Hassoun et al. thus suggests inverse correlations between dopamine synthesis capacity and release. Interestingly, a recent study using PET with [11C]raclopride and L-[β-11C]DOPA found a negative correlation between the binding potential of dopamine D2 receptors and endogenous dopamine synthesis rate in the striatum [Ito et al., 2011]. In line with our findings this negative correlation points towards a compensative relation between presynaptic and postsynaptic dopaminergic functions may mediated by D2 autoreceptors, which may indicate an inverse relationship between synthesis capacity and dopamine release, hypothetically mediated by D2 autoreceptors.

Therefore, our finding of an inverse relationship between synthesis capacity and functional PE signal may indicate a homeostatic regulation of dopamine synthesis, under the control of autoreceptors. Indeed, dopaminergic autoreceptor availability in human mesencephalon was inversely correlated with striatal dopamine release evoked by amphetamine [Buckholtz et al., 2010]. Animal research has also suggested that a rather continuous, tonic mode of dopamine release is negatively correlated with a phasic dopamine signal [Goto et al., 2007;Grace, 2000]. The FDOPA Kinapp is a steady-state parameter measured during an hour, presumably reflecting the long-lasting rather than phasic aspects of dopaminergic neurotransmission, although the exact mechanism of synthesis capacity and dopamine release remains to be elucidated. Therefore, one possible interpretation of the observed inverse correlation between the VS PE signal and local dopamine synthesis capacity is that these two measures reflect a phasic versus tonic component of dopamine-related neurotransmission.

Higher dopamine synthesis capacity in some subjects does not necessarily reflect hereditary differences in DA neurotransmission; instead, elevated DA levels have, for instance, been found in socially stressed primates [Morgan et al., 2002;Nader et al., 2006]. In subjects with rather high levels of DA synthesis capacity and putatively higher stress levels (such as patients with psychosis [such as patients with psychosis van Os et al., 2010], both impaired cognitive performance and a reduced VS PE signal has been reported [Heinz and Schlagenhauf, 2010;Murray et al., 2008]. Further studies are needed to assess acute and chronic stress effects on DA neurotransmission and their respective effects on cognitive performance.

Limitations of our study include the cross-sectional nature of our data, the limitation to male subjects, and the relatively limited sample size of n=17 for the multimodal imaging group, which nevertheless yielded significant correlations between VS PE signaling and dopamine synthesis capacity in VS. We only observed a significant correlation between FDOPA Kinapp and the functional PE signal in the right but not left VS, and further studies may be required to clarify whether there are indeed lateralization differences in the mediation of PE signaling by striatal dopamine.

Altogether, our data reveal a potential mechanism directly linking individual differences in functional brain activation associated with reward PEs to flexible cognitive performance. Indeed, phasic alterations in striatal PE signaling appear to preferentially facilitate complex reasoning and staying on task [Aalto et al., 2005;Goto and Grace, 2005]. It has been suggested that cognitive performance can be targeted pharmacologically by dopaminergic agents such as modafinil [Grady et al., 2010]; however, the inverse correlation observed in this study between right VS dopamine synthesis and PE signaling in VS suggests that pharmacological intervention seeking to optimize cognitive abilities may face unexpected side-effects due to the apparently complex nature of striatal dopaminergic signaling.

Figure 3
Negative correlation between dopamine synthesis capacity as assessed in vivo with FDOPA PET and the BOLD prediction error signal in the ventral striatum

Supplementary Material

supplemental material

Acknowledgements

We thank R. Lorenz, M. Keitel, A. Goldmann, and B. Neumann for assistance during data acquisition; N. Fonyuy and E. Jaeschke for organization and assistance during FDOPA PET; R. Michel and A. Gerhardt for radiochemical analysis; T. Mell and T. Dembler for assistance in neuropsychological testing. This study was supported by the German Science Foundation (DFG HE2597/4-3&7-3 and DFG Exc 257), by the German Ministry for Education and Research (BMBF 01QG87164 & 01GS08159). RJD and QH’s contribution is supported by the Max Planck Society for the Advancement of Science award to RJD.

Reference List

  • Aalto S, Bruck A, Laine M, Nagren K, Rinne JO. Frontal and temporal dopamine release during working memory and attention tasks in healthy humans: a positron emission tomography study using the high-affinity dopamine D-2 receptor ligand [C-11]FLB 457. Journal of Neuroscience. 2005;25:2471–2477. [PubMed]
  • Ashburner J, Friston KJ. Unified segmentation. Neuroimage. 2005;26:839–851. [PubMed]
  • Beckers K, Behrends U, Canavan A. Deutsche Version des Rivermead Behavioral Memory Test. Bury St. Edmunds: Thames Valley Test Company; 1992.
  • Bray S, O'Doherty J. Neural coding of reward-prediction error signals during classical conditioning with attractive faces. Journal of Neurophysiology. 2007;97:3036–3045. [PubMed]
  • Buchel C, Holmes AP, Rees G, Friston KJ. Characterizing stimulus-response functions using nonlinear regressors in parametric fMRI experiments. Neuroimage. 1998;8:140–148. [PubMed]
  • Buckholtz JW, Treadway MT, Cowan RL, Woodward ND, Li R, Ansari MS, Baldwin RM, Schwartzman AN, Shelby ES, Smith CE, Kessler RM, Zald DH. Dopaminergic network differences in human impulsivity. Science. 2010;329:532. [PMC free article] [PubMed]
  • Casanova R, Srikanth R, Baer A, Laurienti PJ, Burdette JH, Hayasaka S, Flowers L, Wood F, Maldjian JA. Biological parametric mapping: A statistical toolbox for multimodality brain image analysis. Neuroimage. 2007;34:137–143. [PMC free article] [PubMed]
  • Choi YY, Shamosh NA, Cho SH, DeYoung CG, Lee MJ, Lee JM, Kim SI, Cho ZH, Kim K, Gray JR, Lee KH. Multiple Bases of Human Intelligence Revealed by Cortical Thickness and Neural Activation. Journal of Neuroscience. 2008;28:10323–10329. [PubMed]
  • Cohen MX. Individual differences and the neural representations of reward expectation and reward prediction error. Social Cognitive and Affective Neuroscience. 2007;2:20–30. [PMC free article] [PubMed]
  • Cohen MX, Ranganath C. Behavioral and neural predictors of upcoming decisions. Cognitive Affective & Behavioral Neuroscience. 2005;5:117–126. [PubMed]
  • Cools R, Clark L, Owen AM, Robbins TW. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J.Neurosci. 2002;22:4563–4567. [PubMed]
  • Cools R, Gibbs SE, Miyakawa A, Jagust W, D'Esposito M. Working memory capacity predicts dopamine synthesis capacity in the human striatum. J Neurosci. 2008;28:1208–1212. [PubMed]
  • Cumming P, Ase A, Laliberte C, Kuwabara H, Gjedde A. In vivo regulation of DOPA decarboxylase by dopamine receptors in rat brain. J Cereb.Blood Flow Metab. 1997;17:1254–1260. [PubMed]
  • Cumming P, Gjedde A. Compartmental analysis of dopa decarboxylation in living brain from dynamic positron emission tomograms. Synapse. 1998;29:37–61. [PubMed]
  • D'Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 2008;319:1264–1267. [PubMed]
  • First MB, Spitzer RL, Gibbon M, Williams J. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition With Psychotic Screen (SCID-I/P W/ PSY SCREEN) New York: New York State Psychiatric Institute; 2001.
  • First M, Spitzer R, Gibbon M, Williams J. Structured Clinical Interview for DSM-IV Personality Disorders, (SCID-II) Washington, DC: American Psychiatric Press, Inc.; 1997.
  • Frank MJ, Claus ED. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol.Rev. 2006;113:300–326. [PubMed]
  • Fuster JM. Executive frontal functions. Exp Brain Res. 2000;133:66–70. [PubMed]
  • Gershman SJ, Pesaran B, Daw ND. Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values. Journal of Neuroscience. 2009;29:13524–13531. [PMC free article] [PubMed]
  • Gillings NM, Bender D, Falborg L, Marthi K, Munk OL, Cumming P. Kinetics of the metabolism of four PET radioligands in living minipigs. Nucl Med Biol. 2001;28:97–104. [PubMed]
  • Goense JB, Logothetis NK. Neurophysiology of the BOLD fMRI signal in awake monkeys. Curr.Biol. 2008;18:631–640. [PubMed]
  • Goldman-Rakic PS, Muly EC, III, Williams GV. D(1) receptors in prefrontal cells and circuits. Brain Res.Brain Res.Rev. 2000;31:295–301. [PubMed]
  • Goto Y, Grace AA. Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat.Neurosci. 2005;8:805–812. [PubMed]
  • Goto Y, Otani S, Grace AA. The Yin and Yang of dopamine release: a new perspective. Neuropharmacology. 2007;53:583–587. [PMC free article] [PubMed]
  • Grace AA. The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. Addiction. 2000;95(Suppl 2):S119–S128. [PubMed]
  • Grady S, Aeschbach D, Wright KP, Jr, Czeisler CA. Effect of modafinil on impairments in neurobehavioral performance and learning associated with extended wakefulness and circadian misalignment. Neuropsychopharmacology. 2010;35:1910–1920. [PMC free article] [PubMed]
  • Hampton AN, Bossaerts P, O'Doherty JP. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci. 2006;26:8360–8367. [PubMed]
  • Hassoun W, Thobois S, Ginovart N, Garcia-Larrea L, Cavorsin ML, Guillouet S, Bonnefoi F, Costes N, Lavenne F, Martin JP, Broussolle E, Leviel V. Striatal dopamine during sensorial stimulations: a [18F]FDOPA PET study in human and cats. Neurosci.Lett. 2005;383:63–67. [PubMed]
  • Heinz A, Schlagenhauf F. Dopaminergic dysfunction in schizophrenia: salience attribution revisited. Schizophr.Bull. 2010;36:472–485. [PMC free article] [PubMed]
  • Heinz A, Siessmeier T, Wrase J, Buchholz HG, Grunder G, Kumakura Y, Cumming P, Schreckenberger M, Smolka MN, Rosch F, Mann K, Bartenstein P. Correlation of alcohol craving with striatal dopamine synthesis capacity and D2/3 receptor availability: a combined [18F]DOPA and [18F]DMFP PET study in detoxified alcoholic patients. Am.J Psychiatry. 2005;162:1515–1520. [PubMed]
  • Helmstaedter C, Lendt M, Lux S. Auflage. Göttingen: Hogrefe Verlag; 2001. VLMT-Verbaler Lern- und Merkfähigkeitstest 1.
  • Horn JL, Cattell RB. Age differences in primary mental ability factors. J.Gerontol. 1966;21:210–220. [PubMed]
  • Horn W. Auflage. Göttingen: Hogrefe; 1983. L-P-S Leistungsprüfsystem. 2.
  • Ito H, Kodaka F, Takahashi H, Takano H, Arakawa R, Shimada H, Suhara T. Relation between Presynaptic and Postsynaptic Dopaminergic Functions Measured by Positron Emission Tomography: Implication of Dopaminergic Tone. The Journal of Neuroscience. 2011;31(21):7886–7890. Ref Type: Journal (Full) [PubMed]
  • Kahnt T, Park SQ, Cohen MX, Beck A, Heinz A, Wrase J. Dorsal striatal-midbrain connectivity in humans predicts how reinforcements are used to guide decisions. J.Cogn Neurosci. 2009;21:1332–1345. [PubMed]
  • Kienast T, Hariri AR, Schlagenhauf F, Wrase J, Sterzer P, Buchholz HG, Smolka MN, Grunder G, Cumming P, Kumakura Y, Bartenstein P, Dolan RJ, Heinz A. Dopamine in amygdala gates limbic processing of aversive stimuli in humans. Nat.Neurosci. 2008;11:1381–1382. [PubMed]
  • Killcross S, Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb.Cortex. 2003;13:400–408. [PubMed]
  • Krugel LK, Biele G, Mohr PNC, Li SC, Heekeren HR. Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proceedings of the National Academy of Sciences of the United States of America. 2009;106:17951–17956. [PubMed]
  • Kumakura Y, Cumming P. PET Studies of Cerebral Levodopa Metabolism: A Review of Clinical Findings and Modeling Approaches. Neuroscientist. 2009;15:635–650. [PubMed]
  • Kumakura Y, Cumming P, Vernaleken I, Buchholz HG, Siessmeier T, Heinz A, Kienast T, Bartenstein P, Grunder G. Elevated [18F]fluorodopamine turnover in brain of patients with schizophrenia: an [18F]fluorodopa/positron emission tomography study. J Neurosci. 2007;27:8080–8087. [PubMed]
  • Landau SM, Lal R, O'Neil JP, Baker S, Jagust WJ. Striatal dopamine and working memory. Cereb.Cortex. 2009;19:445–454. [PMC free article] [PubMed]
  • Lindenberger U, Baltes PB. Intellectual functioning in old and very old age: cross-sectional results from the Berlin Aging Study. Psychol.Aging. 1997;12:410–432. [PubMed]
  • Meyer-Lindenberg A, Miletich RS, Kohn PD, Esposito G, Carson RE, Quarantelli M, Weinberger DR, Berman KF. Reduced prefrontal activity predicts exaggerated striatal dopaminergic function in schizophrenia. Nat.Neurosci. 2002;5:267–271. [PubMed]
  • Morgan D, Grant KA, Gage HD, Mach RH, Kaplan JR, Prioleau O, Nader SH, Buchheimer N, Ehrenkaufer RL, Nader MA. Social dominance in monkeys: dopamine D2 receptors and cocaine self-administration. Nat.Neurosci. 2002;5:169–174. [PubMed]
  • Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, Jones PB, Bullmore ET, Robbins TW, Fletcher PC. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol.Psychiatry. 2008;13:239, 267–239, 276. [PMC free article] [PubMed]
  • Nader MA, Morgan D, Gage HD, Nader SH, Calhoun TL, Buchheimer N, Ehrenkaufer R, Mach RH. PET imaging of dopamine D2 receptors during chronic cocaine self-administration in monkeys. Nat.Neurosci. 2006;9:1050–1056. [PubMed]
  • O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–454. [PubMed]
  • O'Doherty JP. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr.Opin.Neurobiol. 2004;14:769–776. [PubMed]
  • O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ. Temporal difference models and reward-related learning in the human brain. Neuron. 2003;38:329–337. [PubMed]
  • O'Doherty JP, Hampton A, Kim H. Model-based fMRI and its application to reward learning and decision making. Ann.N.Y.Acad.Sci. 2007;1104:35–53. [PubMed]
  • Palminteri S, Boraud T, Lafargue G, Dubois B, Pessiglione M. Brain Hemispheres Selectively Track the Expected Value of Contralateral Options. Journal of Neuroscience. 2009;29:13465–13472. [PubMed]
  • Park SQ, Kahnt T, Beck A, Cohen MX, Dolan RJ, Wrase J, Heinz A. Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. J Neurosci. 2010;30:7749–7753. [PMC free article] [PubMed]
  • Pate BD, Kawamata T, Yamada T, McGeer EG, Hewitt KA, Snow BJ, Ruth TJ, Calne DB. Correlation of striatal fluorodopa uptake in the MPTP monkey with dopaminergic indices. Ann.Neurol. 1993;34:331–338. [PubMed]
  • Patlak CS, Blasberg RG. Graphical evaluation of blood-to-brain transfer constants from multiple-time uptake data. Generalizations. J Cereb.Blood Flow Metab. 1985;5:584–590. [PubMed]
  • Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature. 2006;442:1042–1045. [PMC free article] [PubMed]
  • REITAN RM. The relation of the trail making test to organic brain damage. J.Consult Psychol. 1955;19:393–394. [PubMed]
  • Rodriguez PF, Aron AR, Poldrack RA. Ventral-striatal/nucleus-accumbens sensitivity to prediction errors during classification learning. Human Brain Mapping. 2006;27:306–313. [PubMed]
  • Salthouse TA. Cognition and Context. Science. 1992;257:982–983. [PubMed]
  • Schmidt K, Metzler P. Wortschatztest (WST) Weinheim: Beltz test; 1992.
  • Schonberg T, O'Doherty JP, Joel D, Inzelberg R, Segev Y, Daw ND. Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study. Neuroimage. 2010;49:772–781. [PubMed]
  • Schott BH, Minuzzi L, Krebs RM, Elmenhorst D, Lang M, Winz OH, Seidenbecher CI, Coenen HH, Heinze HJ, Zilles K, Duzel E, Bauer A. Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release. J Neurosci. 2008;28:14311–14319. [PubMed]
  • Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. [PubMed]
  • Sternberg RJ. Cognition. The holey grail of general intelligence. Science. 2000;289:399–401. [PubMed]
  • Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol. 1935;18:643–662.
  • Sutton RS, Barto AG. Reinforcement learning: an introduction. Cambridge, MA: MIT Press; 1998.
  • Tobler PN, O'Doherty JP, Dolan RJ, Schultz W. Human neural learning depends on reward prediction errors in the blocking paradigm. Journal of Neurophysiology. 2006;95:301–310. [PMC free article] [PubMed]
  • Valentin VV, O'Doherty JP. Overlapping Prediction Errors in Dorsal Striatum During Instrumental Learning With Juice and Money Reward in the Human Brain. Journal of Neurophysiology. 2009;102:3384–3391. [PubMed]
  • van Os J, Kenis G, Rutten BP. The environment and schizophrenia. Nature. 2010;468:203–212. [PubMed]
  • Venton BJ, Seipel AT, Phillips PE, Wetsel WC, Gitler D, Greengard P, Augustine GJ, Wightman RM. Cocaine increases dopamine release by mobilization of a synapsin-dependent reserve pool. J Neurosci. 2006;26:3206–3209. [PubMed]
  • Vernaleken I, Kumakura Y, Cumming P, Buchholz HG, Siessmeier T, Stoeter P, Muller MJ, Bartenstein P, Grunder G. Modulation of [18F]fluorodopa (FDOPA) kinetics in the brain of healthy volunteers after acute haloperidol challenge. Neuroimage. 2006;30:1332–1339. [PubMed]
  • Wechsler D. Wechsler Adult Intelligence Scale Manual. New York: Psychological Coropration; 1955.
  • Woodward ND, Zald DH, Ding Z, Riccardi P, Ansari MS, Baldwin RM, Cowan RL, Li R, Kessler RM. Cerebral morphology and dopamine D2/D3 receptor distribution in humans: a combined [18F]fallypride and voxel-based morphometry study. Neuroimage. 2009;46:31–38. [PMC free article] [PubMed]