PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Neuropsychopharmacology. Author manuscript; available in PMC 2013 August 15.
Published in final edited form as:
PMCID: PMC3744058
NIHMSID: NIHMS76748

Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers

Abstract

One prevalent theory of learning states that dopamine neurons signal mismatches between expected and actual outcomes, called temporal difference errors (TDEs). Evidence indicates that dopamine system dysfunction plays a role in negative symptoms of schizophrenia (SZ), including avolition and anhedonia. As such, we predicted that brain responses to TDEs in dopamine midbrain nuclei and target areas would be abnormal in SZ. Eighteen clinically-stable patients with chronic schizophrenia and 18 controls participated in an fMRI study, which used a passive conditioning task. In the task, the delivery of a small amount of juice followed a light stimulus by exactly 6 seconds on approximately 75% of 78 total trials, and was further delayed by 4–7 s on the remaining trials. The delayed juice delivery was designed to elicit the two types of TDE signals, associated with the recognition that a reward was omitted at the expected time, and delivered at an unexpected time. Main effects of TDE valence and group differences in the positive – negative TDE contrast (unexpected juice deliveries – juice omissions) were assessed through whole-brain and regions-of-interest (ROI) analyses. Main effects of TDE valence were observed for the entire sample in the midbrain, left putamen, left cerebellum, and primary gustatory cortex, bilaterally. Whole-brain analyses revealed group differences in the positive – negative TDE contrast in the right putamen and left precentral gyrus, while ROI analyses revealed additional group differences in the midbrain, insula and parietal operculum, on the right, the putamen and cerebellum, on the left, and the frontal operculum, bilaterally. Further, these group differences were generally driven by attenuated responses in patients to positive TDEs (unexpected juice deliveries), whereas responses to negative TDEs (unexpected juice omissions) were largely intact. Patients also showed reductions in responses to juice deliveries on standard trials, and more blunted reinforcer responses in the left putamen corresponded to higher ratings of avolition. These results provide evidence that SZ patients show abnormal brain responses associated with the processing of a primary reinforcer, which may be a source of motivational deficits.

Keywords: schizophrenia, dopamine, reinforcement, basal ganglia, temporal difference error

Introduction

In addition to positive, or psychotic symptoms, schizophrenia (SZ) is characterized by negative symptoms, reflecting deficits in areas such as motivation, emotional expression, and speech production. These negative symptoms have been shown to relate closely to functional impairments exhibited by patients (Green et al 2000; Tamminga et al 1998), particularly with regards to engagement in goal-directed behavior. The fact that patients also show deficits on experimental tasks of reinforcement learning suggests that dysfunctional reinforcement processing mechanisms may contribute to negative symptoms. This idea is further supported by evidence that brain dopamine (DA) systems, which are known to play a pivotal role in modulating reinforcement learning (Montague et al 2004), are disrupted in schizophrenia (Abi-Dargham et al 2000; Breier et al 1997; Laruelle et al 1996; Okubo et al 1997). As such, the purpose of our study was to investigate whether disrupted reward processing mechanisms contribute to deficits in motivated behavior in schizophrenia by examining the role of DA in reward processing.

One prevalent theory of DA system function is that DA cells signal, through phasic modulations of their firing rates, the registration of mismatches between reinforcer expectations and outcomes (Schultz et al 1997), known as “reward prediction errors,” or “temporal difference errors” (TDEs; if they pertain to the timing of reinforcement). Neuroimaging studies with human subjects have observed activation changes in both (presumably) dopaminergic midbrain nuclei (Aron et al 2004) and dopamine target structures (Berns et al 2001; McClure et al 2003; Seymour et al 2004) in association with the perception of prediction errors. McClure and colleagues (2003) found that an area of the left putamen showed significantly decreased activation, relative to baseline, when a juice reinforcer was omitted at the expected time (corresponding to a negative TDE). This area also showed increased activation when the reinforcer was unexpectedly delivered 10 seconds after the light cue (corresponding to a positive TDE).

The results of several studies (Jensen et al 2008; Murray et al 2007), in fact, suggest that SZ patients have specific impairment in the perception and representation of prediction errors. Murray et al. (2007) found differential activations between psychotic patients and controls in the dopaminergic midbrain in response to unexpected monetary outcomes in an operant learning paradigm. In learning to choose between stimuli predicting monetary outcomes, patients showed attenuated neural responses to outcomes that were better or worse than expected, whereas their responses to neutral outcomes were somewhat elevated, relative to those of controls. Corlett et al. (2007) found that prediction-error evoked fMRI activity in prefrontal cortex (PFC) was strongly predictive of clinical ratings of delusional thought content, as measured by the Brief Psychiatric Rating Scale (BPRS; Overall and Gorman 1962).

Study objectives and predictions

Multiple studies (Corlett et al 2007; Murray et al 2007) have identified abnormalities in prediction signaling in the context of higher-level cognitive tasks using a symbolic reinforcer (e.g., money). Determining whether or not there are alterations in responses prediction errors regarding a primary reinforcer (e.g., a food reinforcer) can make an important contribution to the interpretation of the above findings, by helping to establish whether these findings generalize to more elementary forms of learning. In order to address this issue, we implemented a version of the Pavlovian classical conditioning paradigm employed by McClure and colleagues (2003), which used a juice reinforcer, in conjunction with fMRI. This paradigm enabled us to separately investigate neural responses to positive and negative prediction errors in SZ patients. Furthermore, our study was specifically designed to examine the signaling of errors in predictions about the timing of a reinforcer (TDEs) in schizophrenia. Finally, because we hypothesized that faulty TDE processing is at the root of deficits in reward-driven learning and behavior, our goal was to investigate relationships between reinforcer responses and ratings of negative symptoms.

Based on evidence of both disrupted DA function in SZ (Abi-Dargham et al 2000; Breier et al 1997; Laruelle et al 1996; Okubo et al 1997) and deficits in reinforcement learning driven by both positive and negative feedback (Prentice et al in press; Waltz et al 2007; Waltz and Gold 2007), we hypothesized that patients would show abnormal brain correlates of both positive and negative TDEs. Based on our formulation that negative symptoms in SZ reflect a reduced ability of rewards and punishments to modulate learning and motivate behavior in SZ patients, we further hypothesized that patients would show systematic relationships between ratings of negative symptoms, such as avolition, and brain responses to the presence and absence of rewards.

Participants and Methods

Recruiting and Screening of Participants

Eighteen patients and 18 demographically-matched healthy controls participated in the study (Table 1). All participants provided written informed consent to a protocol approved by the institutional review boards at the National Institute on Drug Abuse and the University of Maryland School of Medicine. All participants were right-handed, as determined by the Edinburgh Handedness Inventory (Oldfield 1971).

Table 1
Subject Descriptive Information.

All patients were on stable antipsychotic medication regimens (no changes for four weeks), almost all with second-generation antipsychotic drugs (Supplementary Table 1). The diagnosis of schizophrenia or schizoaffective disorder in patients was confirmed using the SCID-I (First et al 1997), as was the absence of Axis I diagnoses in control participants. Control participants diagnosed with Axis II personality disorders (based on screening with the SIDP-R; Pfohl et al 1989) were also excluded. Participants in both experimental groups underwent medical screening, involving a medical history and physical exam, and subjects with evidence of any neurological or medical condition that might confound data interpretation were excluded (such as significant head injury, stroke or severe vascular problems, chronic untreated diabetes, etc.). Further exclusionary criteria included: pregnancy, current illegal drug use (both verified by urine screens), and admission of past substance dependence.

Procedures

Events prior to MRI scanning

Participants were instructed to abstain from alcohol for 24 hours prior to each visit, which was verified by a breathalyzer test prior to each session. Subjects were fluid-deprived for 3 hours prior to the actual MRI session to increase the value of the juice, which served as a primary reinforcer (see below).

Experimental task (TDE paradigm)

Participants underwent a classical conditioning task (Berns et al 2001; McClure et al 2003; see Figure 1). Prior to actual MRI scanning, participants were trained in a mock scanner to associate the receipt of a juice reward with the presentation of a light stimulus (a large yellow disc on a black background, which was displayed on a screen behind the subject’s head and viewed through a mirror attached to the head coil). During training, participants completed three runs of 26 ‘standard’ trials each, during which the light-juice interval was always 6 seconds. Following training, subjects performed the same task in the MRI scanner. During scanning, however, a number of ‘catch’ events were introduced. On catch trials the juice was not delivered at the ‘expected’ time point, but rather 4–7 seconds after the expected delivery. Based on McClure et al. (2003), we predicted that this type of catch event would induce a negative prediction error at the time when juice was not delivered as expected, and that a subsequent positive prediction error would be incurred by the ‘unexpected’ delivery of juice. The number of paired training events outnumbered the catch events by a ratio of approximately 3:1 in scanning sessions. The time between individual trials ranged from 4–12 seconds.

Figure 1
Classical Conditioning Task. Timing of events on normal and “catch” trials. On catch events, delivery of the juice reward occurs 10–13 second after the light cue, instead of the usual 6 seconds. During the training session (outside ...

Prior to training, participants chose one type of commercially available juice from three options: apple, grape, or fruit punch. Juice was delivered through small-bore IV tubing connected to syringes set into an MR-compatible syringe pump (Harvard Apparatus; Boston, Massachusetts). The end of the tubing was placed in the participant’s mouth, attached to a commercial sports mouth-guard to stabilize the tube. A 600-μl squirt of juice was delivered on each trial, at a rate of 1 ml/second (duration=0.6 s). Between runs of the TDE task, subjects rated the pleasantness of the juice stimuli, by moving a cursor on a visual-analog scale using a wheel manipulandum, with a rating of 8 representing extreme liking, and rating of 0 representing extreme aversion.

Interval Timing Test

The capacity of an individual to learn the association between the light cue and the juice reward is reliant upon their ability to accurately estimate the time at which they should (based on previous learning) expect receipt of the reward. To assess time estimation abilities, participants completed a short test of timing function, based on the work of Rao and colleagues (Harrington et al 2004; Hinton et al 2004), outside of the MRI scanner and following all other experimental phases. On each trial, subjects were presented with three temporal intervals, defined by four beeps, and asked to judge whether the third interval was shorter than, longer than, or the same as the two preceding intervals. The first two intervals were 6 s in all cases, while the third ranged from 4.5 to 7.5 s, in increments of 0.5 s. Thus, the timing paradigm assessed the ability to estimate a similar time interval as was used in the TDE/classical conditioning paradigm, albeit demanding much greater precision.

Other Psychological Assessments

Cognitive function was assessed in all participants using three standard measures: the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler 1999), the Wechsler Test of Adult Reading (WTAR; Wechsler 2001), and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph et al 1998; Wilk et al 2005). In order to quantify the ability of study participants to experience pleasure both physically and in social contexts, we had all subjects complete the Scales for Physical and Social Anhedonia (Chapman et al 1976). Standard symptom ratings were obtained for all patients using the BPRS (Overall and Gorman 1962), the Scale for the Assessment of Negative Symptoms (SANS; Andreasen 1984), and the Calgary Depression Scale (CDS; Addington et al 1992).

Acquisition and Analysis of fMRI Data

MRI scanning

A 3-T Siemens Allegra scanner (Erlangen, Germany) acquired whole-brain functional EPI images for measurement of T2*-weighted BOLD effects (4-mm oblique axial slices, 30° axial to coronal; 64 × 64 matrix; FOV = 22 × 22 cm; TR = 2 s; TE = 27 ms; FA = 80°). In each scanning session, a whole-brain oblique axial T1-weighted structural image (MPRAGE) was acquired for anatomical reference (1-mm3 isotropic voxels; TR = 2.5 s; TE = 4.38 ms; FA = 80°).

Whole-brain analyses of MRI data

All preprocessing and first-level analyses of MRI data were performed using the AFNI software package (Analysis of Functional NeuroImages; Cox 1996). Preprocessing steps included volume-registration for motion correction, slice-timing correction, and temporal normalization. In order to generate statistical parametric maps for individual subjects, we used an approach similar to McClure et al. (2003) and D’Ardenne et al. (2008). We used three types of events as regressors in regression analyses: juice deliveries on standard trials, and positive and negative TDEs (juice deliveries and omissions on catch trials, respectively). Regressors were modeled by an idealized hemodynamic response function and its temporal derivative, time-locked to the event onsets. Additional regressors of no-interest included the six motion-correction curves.

For the purposes of this study, we examined responses only after learning because it provides the strongest method for studying TDE responses in isolation, while avoiding dependencies on exact learning-model parameters. That is, when learning is complete, positive and negative prediction errors evoked by changing the time of reward can be fully modeled without dependence on how effective TDEs were in driving learning.

Beta weights from the first-level analysis were spatially normalized to Talairach space and smoothed with a 4.2 mm FWHM Gaussian kernel prior to second-level (i.e. group) analyses. The main second-level analyses were two separate repeated-measures ANOVAs, performed to determine brain areas showing significant main effects of group, event type, and group x event type interactions: one with factors of group (patients vs. control) and TDE valence (positive vs. negative), and one with factors of group and trial-type (catch vs. standard, to compare responses to ‘unexpected’ and ‘expected’ deliveries of the reinforcer). Correction for multiple comparisons was accomplished using a Monte Carlo simulation, which determined a minimum cluster size required for a given voxel-wise threshold. This simulation determined that, for our chosen voxel-wise threshold (p<0.002), a minimum cluster size of 11 voxels (424 μl) was required to achieve an overall significance level of p < 0.05.

Analyses of group differences in regions of interest (ROI)

In order to compare the performance of the two groups in regions involved in the processing of the reinforcer, as well as prediction errors, we selected all significant clusters showing significant main effects of TDE valence and all clusters showing significant group X TDE valence interactions. This criterion yielded 11 ROIs, described below. Based on the results of McClure et al. (2003), which used the same paradigm to examine prediction-error-related brain activity in healthy volunteers, we selected two additional regions of interest a priori, in the left and right putamen by drawing spheres of 5 mm radius around their Talairach coordinates (±18, 2, 8). Thus, we performed further analyses on 13 total ROIs. Reported p-values based on this group analysis for a priori regions of interest were corrected for the number of comparisons made within each region.

Correlation analyses

We used Spearman correlation analyses to assess relationships among timing task performance, clinical ratings of avolition (to assess reductions in goal-directed behavior in schizophrenia), and BOLD responses to juice deliveries in the 11 ROIs identified functionally, and the 2 ROIs identified a priori. Clinical ratings of avolition were determined by summing the four items from the avolition subscale of the 25-item SANS.

Results

Brain regions distinguishing positive from negative TDEs

A two-way ANOVA with factors of group and TDE valence was performed to determine the brain regions distinguishing positive from negative TDEs in the entire sample (N=36; see Table 2a). This ANOVA produced five brain regions showing main effects of TDE valence, all of which resulted from activations to positive TDEs and deactivations to negative TDEs. Regions showing main effects of TDE valence included left putamen and the right midbrain, consistent with previous findings (D’Ardenne et al 2008; McClure et al 2003; Murray et al 2007; see Figure 2a–c). Large areas also emerged, centered on the left and right frontal and parietal operculum, extending into insular cortex. These areas correspond to primary gustatory cortex, as identified in previous studies (Lee et al 1998; Small et al 1999). For the purpose of ROI analysis, we divided each of the large areas into three components, based on their overlap with precentral gyrus, postcentral gyrus, and insula, as identified by the Talairach daemon (Lancaster et al 2000). Together with an additional cluster emerging in the declive of the cerebellum, this yielded 9 ROIs for further analysis. Note that a main effect of group, without respect to TDE valence, is difficult to interpret, because we expected (and observed) activations for positive TDEs, and deactivations for negative TDEs; thus, we do not report this effect.

Figure 2
Regions emerging from whole-brain analyses. a–c. Regions showing differential responses to positive and negative TDEs within the entire sample (N=36). Midbrain contrasts are shown in panels a and b, while contrasts in putamen and insula are depicted ...
Table 2
Brain regions showing significant main effects of TDE valence (positive vs. negative) or significant group × TDE valence interactions.

Two brain regions, the right putamen and in the left frontal operculum, showed significant group × TDE valence interactions (Table 2b, Figure 2d–f). In these two areas, controls showed BOLD activations to positive TDEs and deactivations to negative TDEs, whereas patients showed the opposite (aberrant) pattern (see Figure 3b–c, ROIs 10 and 11).

Figure 3
a. Contrast in MRI activity evoked by positive and negative TDEs in each group, in each ROI. b. Analysis of MRI activity evoked by positive TDEs, relative to baseline. c. Analysis of MRI activity evoked by negative TDEs, relative to baseline. d. Analysis ...

ROI analyses of group differences in the positive – negative TDE contrast

With 11 regions emerging from whole-brain analyses, and 2 regions chosen a priori, we performed additional analyses on 13 total ROIs. In 9 of the 13 ROIs investigated, the two subject groups differed in the magnitude of the [positive – negative] TDE contrast (the difference trended toward significance in ROI 13, the right putamen; Figure 3a). This was true despite the fact that in nine of these regions (ROIs 1–9), the magnitude of the [positive – negative] TDE contrast was significantly different from zero within the patient group (all t’s > 2.75; see Figure 3a).

Further analyses revealed that group differences in the [positive – negative] TDE contrast were generally driven by attenuated responses to positive TDEs in the patient group (all t’s > 3.10; Figure 3b). Whereas controls showed significant activations to positive TDEs, relative to baseline, in all 13 ROIs, patients showed significant activations to positive TDEs only in the pre- and postcentral gyrus, bilaterally (ROIs 5–8). In response to negative TDEs, however, patients and controls did not differ significantly in their responses to negative TDEs in any of the ROIs (Figure 3c), with patients showing robust deactivations, relative to baseline, in nine regions (ROIs 1–9), including the midbrain and left putamen.

Together, these analyses indicate that group differences in BOLD signal modulations, due to the valence of prediction errors, generally result from attenuated responses in patients to unexpected administrations of the reinforcer. In order to determine whether patients’ responses to positive TDEs were selectively disrupted, or if patients showed more general dysfunction in neural processing of the reinforcer, we performed analyses comparing subjects’ responses to unexpected (catch) versus expected (standard) juice deliveries.

MRI responses to non-TDE events

Juice deliveries on standard vs. catch trials

A two-way ANOVA of group X trial-type was performed to compare unexpected deliveries of reward (positive TDEs) with expected reward delivery (null TDEs). This analysis revealed significant main effects of group in numerous areas, regardless of whether they occurred on standard or catch trials (Table 3). Controls showed more positive responses to juice delivery than patients, regardless of trial type, in all of these areas, which overlapped with many of the ROIs identified above, including midbrain, left and right putamen, left and right frontal operculum, and left insular cortex.

Table 3
Clusters showing main effects of group for juice delivery, regardless of trial type

Also in our previously identified ROIs, participants within each group showed similar responses to juice deliveries on standard and catch trials, with controls showing significant activations to both in all regions (relative to baseline; compare Figures 3b and 3d). In three of the 13 ROIs, in fact, BOLD responses to expected juice deliveries and unexpected juice deliveries correlated significantly in the patient group (see Supplementary Table 2).

No region showed a significant main effect of trial-type, and no brain region showed a significant group X trial-type interaction. Thus, we did not find evidence of an enhanced neural response to (presumably unexpected) juice deliveries on catch trials, relative to juice deliveries on standard trials.

MRI responses to the light cue

It should be noted that patients did not show attenuated activations, or deactivations to all event types; brain responses to the reward-predicting cue (the conditioned stimulus) were largely intact in SZ patients (see Supplementary Table 3; Supplementary Figure 1).

Behavioral and Correlation Analyses

Timing Task Performance

Patients and controls differed significantly in their performance on the interval timing task done outside the scanner [t(31)=3.38; p=0.002]. Whereas patients correctly estimated only 45.8% (SD=12.1%) of test intervals, controls correctly estimated 61.6% (SD=14.7%) of test intervals.

When we computed Spearman correlation coefficients for relationships between scores on the interval timing task and MRI responses to standard juice deliveries, we observed a single significant correlation: timing task performance in patients related systematically to responses to standard juice deliveries in the left frontal operculum/precentral gyrus (Table 4).

Table 4
Spearman Correlation between Standard Juice Responses and Behavioral Measures in Patients

Anhedonia and Avolition

Exploratory analyses of relationships between BOLD data and rating of clinical symptoms revealed that avolition subscores from the SANS correlated negatively with MRI responses to standard juice deliveries in the left frontal and parietal operculum (pre- and postcentral gyrus; Table 4). Furthermore significant negative correlations were observed between avolition ratings and MRI responses to standard juice deliveries in the left and right a priori putamen ROIs, and six additional ROIs showed correlations of medium effect size (d > 0.3; Cohen 1992). In all of these ROIs, patients with the highest ratings of avolition showed the least positive BOLD responses to juice deliveries. Supplementary Figure 2 illustrates this pattern in the left frontal operculum and the left putamen ROIs.

By contrast, avolition ratings bore no relationship to self-reports of reinforcer enjoyment (Spearman’s rho = −0.023), which did not differ between patients and controls [mean rating of 5.8 (1.4) for patients vs. mean of 5.3 (1.1) for controls; t(34)=1.25; p>0.10]. Avolition ratings also bore no relationship to self-reports of physical (ρ = −0.287) and social anhedonia (ρ = −0.037) from the Chapman scales. Patients and controls also did not differ significantly on these measures [t(34)<1.6; p>0.10 in both cases; see Table 1].

Discussion

In this functional MRI study of prediction error-related activity, our data led us to draw the following conclusions: 1. Multiple components of reward circuits showed responses that distinguished between positive and negative TDEs in the entire sample of participants; 2. Patients with schizophrenia showed reduced contrasts in brain activity evoked by positive and negative TDEs in multiple brain areas, driven largely by attenuated responses to positive TDEs; 3. Patients showed attenuated responses in numerous brain regions to delivery of the juice reinforcer on standard trials, as well; and 4. Clinical ratings of avolition correlated significantly with brain activity evoked by standard juice delivery in the primary gustatory cortex and putamen. Each of these findings will be addressed in turn.

Contrasts between positive and negative TDEs

When combined across groups, neural activity distinguished between positive and negative temporal difference errors in multiple brain regions. Consistent with the results of McClure and colleagues (D’Ardenne et al 2008; McClure et al 2003), these areas included the midbrain and the left putamen. Both ventral and dorsal areas of the neostriatum have been identified as components of circuits for reward-based learning (Haber et al 2006; Voorn et al 2004), and a role for these areas in outcome processing, in particular, is supported by the results of several previous neuroimaging studies (Delgado et al 2005; Delgado et al 2000; McClure et al 2003). Our results provide further evidence that projections from the midbrain to the basal ganglia comprise a critical component of circuits for processing outcomes, and mismatches between expectations and outcomes.

Additionally, significant contrasts between unexpected juice delivery and omissions were observed, bilaterally, in the insula and the frontal and parietal operculum, consistent with the results of previous studies that have linked these areas to gustation (Lee et al 1998), general interoception (Craig 2003; Critchley et al 2004), and the processing of outcomes (Paulus et al 2005; Seymour et al 2004).

Group differences in responses to TDEs

Whole-brain analyses revealed that patients showed a reduced BOLD contrast between positive and negative TDEs in right putamen and the left frontal operculum. Additional ROI analyses revealed group differences in the [positive – negative] TDE contrast in the right midbrain, the left putamen, right primary gustatory cortex, and left cerebellum.

Evidence of abnormal prediction-error responses in schizophrenia suggests a possible factor underlying reinforcement learning deficits commonly observed in patients (Kemali et al 1987; Keri 2008; Schwartz et al 2003). Our finding of attenuated prediction-error-related activity in schizophrenia is consistent with the results of a recent neuroimaging study (Murray et al 2007), which found evidence of abnormal prediction error responses in the midbrain, among other brain regions, in SZ patients performing an operant learning paradigm. Several other studies have identified attenuated responses in the striatum in SZ patients (Kumari et al 2002; Reiss et al 2006), associated with the feedback-driven learning of procedures.

We observed group differences between patients and controls mainly for positive prediction errors (unexpected juice delivery), rather than negative prediction errors (unexpected juice omissions). This finding suggests that sensitivity to outcomes that are worse than expected (negative TDEs) might be relatively preserved in schizophrenia. The relatively intact response to negative TDEs in patients in the present study was somewhat unexpected, in light of the behavioral evidence pointing to impaired online error-correction behavior in schizophrenia (Prentice et al in press; Waltz and Gold 2007).

It is possible, however, that the type of the learning involved, specifically in terms of whether subjects need to explicitly represent feedback and use it to make rapid adjustments in behavior, may influence the extent to which brain signals associated with negative prediction errors are abnormal in schizophrenia. Despite relatively clear evidence of deficits in the ability to use feedback to make rapid adjustments in behavior, other behavioral results indicate that punishment-driven learning may be unimpaired in SZ patients if the learning task is of a probabilistic or procedural nature (Waltz et al 2007). The fact that the passive conditioning task in the current study has been shown to depend largely on striatal regions, implicated in procedural learning, might explain our finding of largely intact neural responses to negative TDEs in patients. It is possible, furthermore, that the negative TDE error signal is intact in schizophrenia, but that the ability of target brain areas to use error information to modulate response selection and learning is disrupted.

Responses to reinforcer deliveries on standard trials

Contrary to our expectations, patients also showed attenuated responses in numerous brain regions to delivery of the juice reinforcer on standard trials. Furthermore, we observed that neuronal activity evoked by juice delivery in the left primary gustatory cortex and bilateral putamen correlated significantly with clinical ratings of avolition. These findings, together with our observation that BOLD responses to expected and unexpected juice deliveries correlated significantly in three ROIs in the patient group, suggest that abnormal processing of the juice reinforcer was a contributing factor to the attenuated positive prediction error signals in SZ patients, and that reductions in goal-directed behavior frequently observed in schizophrenia patients may be influenced by abnormal processing of rewards.

Findings regarding the experience of rewards in schizophrenia are mixed. Evidence from behavioral studies (Cohen and Minor 2008; Gard et al 2007; Germans and Kring 2000) supports, in large part, the idea that patients with schizophrenia have intact self-reported experience of rewards. Indeed, patients in the present study reported finding the juice just as pleasant as controls did. These findings, however, appear to be contradicted by the results of multiple neuroimaging studies (Crespo-Facorro et al 2001; Paradiso et al 2003; Plailly et al 2006), which have found evidence of abnormal neural responses to pleasant stimuli in schizophrenia (especially primary reinforcers, such as pleasant odors). Furthermore, multiple studies have found evidence of reduced gray matter volume in components of reward processing circuitry, such as insula and ventral prefrontal cortex (Crespo-Facorro et al 2000; Davatzikos et al 2005).

One possible explanation for this apparent dichotomy is that, even if patients report normal hedonic experiences, the physiology underlying those hedonic experiences might still be abnormal. Several previous studies have found differential brain responses between SZ patients and controls, despite similar behavioral performance on learning tasks (Murray et al 2007; Zedkova et al 2006), as well as similar reports of emotional experience (Takahashi et al 2004). A disconnect between the reported experience of rewards, and the associated physiology, may partly explain the reduced ability of rewards and punishments to motivate behavior in schizophrenia. In short, patients who do not report a reduced experienced of pleasure might function like those who do, if the physiology underlying the experience of pleasure is abnormal. This view is supported by our finding that ratings of avolition in patients correlate most strongly with neural responses to the juice reinforcer, whereas avolition ratings bore no relationship to self-reports of reinforcer enjoyment.

It is possible that the strong response to juice delivery on standard trials, in components of reward circuits in controls, stemmed from the frequency of “catch” trials in our paradigm (roughly ¼ of total trials). A much lower proportion of nonstandard events has been used in some paradigms where infrequency was used to enhance the salience of an event (see, e.g., Zink et al 2006; Zink et al 2004). In our study, expectations may have changed across the course of the session, causing MRI responses to the reinforcer on standard and catch trials to become more similar.

Our observation of a significant correlation between timing task performance in patients, and neural responses to standard juice delivery in the left frontal operculum, suggests that it is also possible that group differences in interval timing abilities contributed to group differences in brain responses to TDEs. However, the fact that SZ patients showed largely intact responses to omissions of the reinforcer (negative TDEs) contradicts this, suggesting that patients had developed a relatively normal expectation as to the timing of standard juice deliveries. Thus, poor interval estimation is likely to make only a minor contribution to attenuated physiological responses to prediction errors in SZ patients.

Might the altered neurophysiological response to juice rewards reflect the impact of antipsychotic medications that attenuate dopamine signaling rather than an effect that can be considered to be a consequence of the illness? Previous studies have not produced a clear picture of what the impact of antipsychotic medications on reward processing might be. At least two studies using pharmacological challenge in normal volunteers have found that antipsychotic drugs modulate feedback-related brain activity (Abler et al 2007; Zirnheld et al 2004). On the other hand, results from studies of reward processing in patients with schizophrenia are mixed. Juckel and colleagues (2006a; 2006b) found that unmedicated SZ patients and patients on first-generation (typical) antipsychotics showed attenuated activity in ventral striatum, relative to controls. However, treatment with olanzapine appears to produce a relative normalization of reward anticipation BOLD response in the ventral striatum (relative to activity observed in patients treated with typical antipsychotics; Juckel et al 2006a; Schlagenhauf et al 2008). Thus, at least in the case of a symbolic reinforcer, some antipsychotic medications may actually enhance brain activity related to reward anticipation in SZ patients.

The design of the present study does not allow us to address this question in a straightforward fashion. In the absence of random assignment to drug type and dose, any post-hoc analysis of medication effects is confounded. That is, it is not possible to evaluate drug dose and type independently of the illness features and treatment history that led to the choice of that drug and dose. This may be a particular concern in the patient group studied here, where 1/3 of individuals were receiving clozapine – evidence of treatment resistance. Thus, in principle, our findings of abnormal reward processing physiology may be limited to treated patients. In order to answer the critical question of whether these abnormalities are intrinsic to the illness would require a very ambitious study design where patients are evaluated off medication and then again on medication, with the need to randomly assign drug type and dose in order to comprehensively address this issue.

The fact that our findings may only generalize to treated patients does not diminish the clinical importance of the results, because the great majority of patients with schizophrenia take antipsychotic medications. Such patients demonstrate altered reward processing, and the correlations between brain reward signal modulation and negative symptoms suggest that this physiological response is related to core features of the illness – features that are highly treatment resistant and are highly predictive of functional disability.

Supplementary Material

Acknowledgments

This research was made possible by Grant # 1 K12 RR023250-01, # P30 MH068580-01, # 1 R24 MH72647-01A1 from the National Institutes of Mental Health, and the Intramural Research Program of the National Institute on Drug Abuse. Sharon August and Kimberly Warren assisted with screening and neuropsychological assessments.

Footnotes

Disclosure

These data were presented, in part, at the 13th annual meeting of the Organization for Human Brain Mapping, in Chicago, Illinois, and at the 37th annual meeting of the Society for Neuroscience, in San Diego, California.

Dr. Gold receives royalties from sales of the Brief Assessment of Cognition in Schizophrenia (BACS) battery and has served as a consultant or an advisory board member for Pfizer, AstraZeneca, and GlaxoSmithKline. All other authors declare that, except for income received from their primary employers, no financial support or compensation has been received from any individual or corporate entity over the past three years for research or professional service, and there are no personal financial holdings that could be perceived as constituting a potential conflict of interest.

References

  • Abi-Dargham A, Rodenhiser J, Printz D, Zea-Ponce Y, Gil R, Kegeles LS, et al. Increased baseline occupancy of D2 receptors by dopamine in schizophrenia. Proc Natl Acad Sci U S A. 2000;97:8104–8109. [PubMed]
  • Abler B, Erk S, Walter H. Human reward system activation is modulated by a single dose of olanzapine in healthy subjects in an event-related, double-blind, placebo-controlled fMRI study. Psychopharmacology (Berl) 2007;191:823–833. [PubMed]
  • Addington D, Addington J, Maticka-Tyndale E, Joyce J. Reliability and validity of a depression rating scale for schizophrenics. Schizophr Res. 1992;6:201–208. [PubMed]
  • Andreasen NC. The Scale for the Assessment of Negative Symptoms (SANS) Iowa City, IA: University of Iowa; 1984.
  • Aron AR, Shohamy D, Clark J, Myers C, Gluck MA, Poldrack RA. Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol. 2004;92:1144–1152. [PubMed]
  • Berns GS, McClure SM, Pagnoni G, Montague PR. Predictability modulates human brain response to reward. J Neurosci. 2001;21:2793–2798. [PubMed]
  • Breier A, Su TP, Saunders R, Carson RE, Kolachana BS, de Bartolomeis A, et al. Schizophrenia is associated with elevated amphetamine-induced synaptic dopamine concentrations: evidence from a novel positron emission tomography method. Proc Natl Acad Sci U S A. 1997;94:2569–2574. [PubMed]
  • Chapman LJ, Chapman JP, Raulin ML. Scales for physical and social anhedonia. J Abnorm Psychol. 1976;85:374–382. [PubMed]
  • Cohen AS, Minor KS. Emotional Experience in Patients With Schizophrenia Revisited: Meta-analysis of Laboratory Studies. Schizophr Bull 2008 [PMC free article] [PubMed]
  • Cohen J. A power primer. Psychological Bulletin. 1992;112:155–159. [PubMed]
  • Corlett PR, Murray GK, Honey GD, Aitken MR, Shanks DR, Robbins TW, et al. Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain. 2007;130:2387–2400. [PMC free article] [PubMed]
  • Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res. 1996;29:162–173. [PubMed]
  • Craig AD. Interoception: the sense of the physiological condition of the body. Curr Opin Neurobiol. 2003;13:500–505. [PubMed]
  • Crespo-Facorro B, Kim J, Andreasen NC, O’Leary DS, Magnotta V. Regional frontal abnormalities in schizophrenia: a quantitative gray matter volume and cortical surface size study. Biol Psychiatry. 2000;48:110–119. [PubMed]
  • Crespo-Facorro B, Paradiso S, Andreasen NC, O’Leary DS, Watkins GL, Ponto LL, Hichwa RD. Neural mechanisms of anhedonia in schizophrenia: a PET study of response to unpleasant and pleasant odors. Jama. 2001;286:427–435. [PubMed]
  • Critchley HD, Wiens S, Rotshtein P, Ohman A, Dolan RJ. Neural systems supporting interoceptive awareness. Nat Neurosci. 2004;7:189–195. [PubMed]
  • D’Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 2008;319:1264–1267. [PubMed]
  • Davatzikos C, Shen D, Gur RC, Wu X, Liu D, Fan Y, et al. Whole-brain morphometric study of schizophrenia revealing a spatially complex set of focal abnormalities. Arch Gen Psychiatry. 2005;62:1218–1227. [PubMed]
  • Delgado MR, Miller MM, Inati S, Phelps EA. An fMRI study of reward-related probability learning. Neuroimage. 2005;24:862–873. [PubMed]
  • Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol. 2000;84:3072–3077. [PubMed]
  • First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV- Axis I Disorders (SCID-I) Washington, DC: American Psychiatric Press; 1997.
  • Gard DE, Kring AM, Gard MG, Horan WP, Green MF. Anhedonia in schizophrenia: distinctions between anticipatory and consummatory pleasure. Schizophr Res. 2007;93:253–260. [PMC free article] [PubMed]
  • Germans MK, Kring AM. Hedonic deficit in anhedonia: Support for the role of approach motivation. Personality and Individual Differences. 2000;28:659–672.
  • Green MF, Kern RS, Braff DL, Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the “right stuff”? Schizophr Bull. 2000;26:119–136. [PubMed]
  • Haber SN, Kim KS, Mailly P, Calzavara R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci. 2006;26:8368–8376. [PubMed]
  • Harrington DL, Boyd LA, Mayer AR, Sheltraw DM, Lee RR, Huang M, Rao SM. Neural representation of interval encoding and decision making. Brain Res Cogn Brain Res. 2004;21:193–205. [PubMed]
  • Hinton SC, Harrington DL, Binder JR, Durgerian S, Rao SM. Neural systems supporting timing and chronometric counting: an FMRI study. Brain Res Cogn Brain Res. 2004;21:183–192. [PubMed]
  • Jensen J, Willeit M, Zipursky RB, Savina I, Smith AJ, Menon M, et al. The formation of abnormal associations in schizophrenia: neural and behavioral evidence. Neuropsychopharmacology. 2008;33:473–479. [PubMed]
  • Juckel G, Schlagenhauf F, Koslowski M, Filonov D, Wustenberg T, Villringer A, et al. Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics. Psychopharmacology (Berl) 2006a;187:222–228. [PubMed]
  • Juckel G, Schlagenhauf F, Koslowski M, Wustenberg T, Villringer A, Knutson B, et al. Dysfunction of ventral striatal reward prediction in schizophrenia. Neuroimage. 2006b;29:409–416. [PubMed]
  • Kemali D, Maj M, Galderisi S, Monteleone P, Mucci A. Conditional associative learning in drug-free schizophrenic patients. Neuropsychobiology. 1987;17:30–34. [PubMed]
  • Keri S. Interactive memory systems and category learning in schizophrenia. Neurosci Biobehav Rev. 2008;32:206–218. [PubMed]
  • Kumari V, Gray JA, Honey GD, Soni W, Bullmore ET, Williams SC, et al. Procedural learning in schizophrenia: a functional magnetic resonance imaging investigation. Schizophr Res. 2002;57:97–107. [PubMed]
  • Lancaster JL, Woldorff MG, Parsons LM, Liotti M, Freitas CS, Rainey L, et al. Automated Talairach atlas labels for functional brain mapping. Hum Brain Mapp. 2000;10:120–131. [PubMed]
  • Laruelle M, Abi-Dargham A, van Dyck CH, Gil R, D’Souza CD, Erdos J, et al. Single photon emission computerized tomography imaging of amphetamine-induced dopamine release in drug-free schizophrenic subjects. Proc Natl Acad Sci U S A. 1996;93:9235–9240. [PubMed]
  • Lee BC, Hwang SH, Rison R, Chang GY. Central pathway of taste: clinical and MRI study. Eur Neurol. 1998;39:200–203. [PubMed]
  • McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron. 2003;38:339–346. [PubMed]
  • Montague PR, Hyman SE, Cohen JD. Computational roles for dopamine in behavioural control. Nature. 2004;431:760–767. [PubMed]
  • Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, et al. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiatry. 2007;13:267–276. [PMC free article] [PubMed]
  • Okubo Y, Suhara T, Suzuki K, Kobayashi K, Inoue O, Terasaki O, et al. Decreased prefrontal dopamine D1 receptors in schizophrenia revealed by PET. Nature. 1997;385:634–636. [PubMed]
  • Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9:97–113. [PubMed]
  • Overall JE, Gorman DR. The Brief Psychiatric Rating Scale. Psychological Reports. 1962;10:799–812.
  • Paradiso S, Andreasen NC, Crespo-Facorro B, O’Leary DS, Watkins GL, Boles Ponto LL, Hichwa RD. Emotions in unmedicated patients with schizophrenia during evaluation with positron emission tomography. Am J Psychiatry. 2003;160:1775–1783. [PubMed]
  • Paulus MP, Feinstein JS, Leland D, Simmons AN. Superior temporal gyrus and insula provide response and outcome-dependent information during assessment and action selection in a decision-making situation. Neuroimage. 2005;25:607–615. [PubMed]
  • Pfohl B, Blum N, Zimmerman M, Stangl D. Structured Interview for DSM-III-R Personality Disorders (SIDP-R) Iowa City, IA: University of Iowa, Department of Psychiatry; 1989.
  • Plailly J, d’Amato T, Saoud M, Royet JP. Left temporo-limbic and orbital dysfunction in schizophrenia during odor familiarity and hedonicity judgments. Neuroimage. 2006;29:302–313. [PubMed]
  • Prentice KJ, Gold JM, Buchanan RW. The Wisconsin Card Sorting impairment in schizophrenia is evident in the first four trials. Schizophr Res (in press) [PMC free article] [PubMed]
  • Randolph C, Tierney MC, Mohr E, Chase TN. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. J Clin Exp Neuropsychol. 1998;20:310–319. [PubMed]
  • Reiss JP, Campbell DW, Leslie WD, Paulus MP, Ryner LN, Polimeni JO, et al. Deficit in schizophrenia to recruit the striatum in implicit learning: a functional magnetic resonance imaging investigation. Schizophr Res. 2006;87:127–137. [PubMed]
  • Schlagenhauf F, Juckel G, Koslowski M, Kahnt T, Knutson B, Dembler T, et al. Reward system activation in schizophrenic patients switched from typical neuroleptics to olanzapine. Psychopharmacology (Berl) 2008;196:673–684. [PubMed]
  • Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. [PubMed]
  • Schwartz BL, Howard DV, Howard JH, Jr, Hovaguimian A, Deutsch SI. Implicit learning of visuospatial sequences in schizophrenia. Neuropsychology. 2003;17:517–533. [PubMed]
  • Seymour B, O’Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, et al. Temporal difference models describe higher-order learning in humans. Nature. 2004;429:664–667. [PubMed]
  • Small DM, Zald DH, Jones-Gotman M, Zatorre RJ, Pardo JV, Frey S, Petrides M. Human cortical gustatory areas: a review of functional neuroimaging data. Neuroreport. 1999;10:7–14. [PubMed]
  • Takahashi H, Koeda M, Oda K, Matsuda T, Matsushima E, Matsuura M, et al. An fMRI study of differential neural response to affective pictures in schizophrenia. Neuroimage. 2004;22:1247–1254. [PubMed]
  • Tamminga CA, Buchanan RW, Gold JM. The role of negative symptoms and cognitive dysfunction in schizophrenia outcome. Int Clin Psychopharmacol. 1998;13(Suppl 3):S21–26. [PubMed]
  • Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 2004;27:468–474. [PubMed]
  • Waltz JA, Frank MJ, Robinson BM, Gold JM. Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biol Psychiatry. 2007;62:756–764. [PMC free article] [PubMed]
  • Waltz JA, Gold JM. Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophr Res. 2007;93:296–303. [PMC free article] [PubMed]
  • Wechsler D. Wechsler Abbreviated Scale of Intelligence (WASI) San Antonio TX: The psychological corporation; 1999.
  • Wechsler D. Wechsler Test of Adult Reading (WTAR) San Antonio, TX: The Psychological Corporation; 2001.
  • Wilk CM, Gold JM, McMahon RP, Humber K, Iannone VN, Buchanan RW. No, it is not possible to be schizophrenic yet neuropsychologically normal. Neuropsychology. 2005;19:778–786. [PubMed]
  • Zedkova L, Woodward ND, Harding I, Tibbo PG, Purdon SE. Procedural learning in schizophrenia investigated with functional magnetic resonance imaging. Schizophr Res. 2006;88:198–207. [PubMed]
  • Zink CF, Pagnoni G, Chappelow J, Martin-Skurski M, Berns GS. Human striatal activation reflects degree of stimulus saliency. Neuroimage. 2006;29:977–983. [PMC free article] [PubMed]
  • Zink CF, Pagnoni G, Martin-Skurski ME, Chappelow JC, Berns GS. Human striatal responses to monetary reward depend on saliency. Neuron. 2004;42:509–517. [PubMed]
  • Zirnheld PJ, Carroll CA, Kieffaber PD, O’Donnell BF, Shekhar A, Hetrick WP. Haloperidol impairs learning and error-related negativity in humans. J Cogn Neurosci. 2004;16:1098–1112. [PubMed]