PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Neurosci. Author manuscript; available in PMC Dec 9, 2010.
Published in final edited form as:
PMCID: PMC2896864
NIHMSID: NIHMS212072
Updating beliefs for a decision: Neural correlates of uncertainty and underconfidence
Emily R. Stern,1 Richard Gonzalez,2 Robert C. Welsh,1,3 and Stephan F. Taylor1
1Department of Psychiatry
2Department of Psychology
3Department of Radiology
Corresponding Author: Emily R. Stern, Ph.D., Department of Psychiatry, University of Michigan, 4250 Plymouth Road, 2506 Rachel Upjohn Building, Ann Arbor, MI 48109, emistern/at/med.umich.edu, Phone: (734) 936-1323, Fax: (834) 936-7868
Some decisions are made after obtaining several pieces of information, while others are reached quickly. Such differences may depend on the quality of information acquired, as well as individual variability in how cautiously evidence is evaluated. The current study examined neural activity while subjects accumulated sequential pieces of evidence and then made a decision. Uncertainty was updated with each piece of evidence, with individual ratings of subjective uncertainty characterizing underconfidence when observing evidence. Increased uncertainty during evidence accumulation was associated with activity in dorsal anterior cingulate cortex, whereas greater uncertainty when executing a decision uniquely elicited lateral frontal and parietal activity. Greater underconfidence when observing evidence correlated with activity in ventromedial prefrontal cortex. These results suggest that neural mechanisms of uncertainty depend on the stage of decision-making (belief updating versus decision), and that greater subjective uncertainty when evaluating evidence is associated with activity in ventromedial brain regions, even in the absence of overt risk.
Keywords: Decision, fMRI, Prefrontal cortex, ventromedial, cingulate, Emotion, Cognition
People often make decisions after evaluating several pieces of information, such as when an investor tracks performance measures prior to purchasing a stock. The normative process of belief updating through evidence accumulation can be described using Bayesian calculations, where the posterior likelihood of an event is computed by aggregating current evidence with prior information. Although Bayesian formalism describes the uncertainty that should be experienced by a rational decision-maker when choosing between options, it does not always reflect the subjective experience of uncertainty, which varies between individuals with access to the same information (Weber and Milliman, 1997). For example, some investors may be able to choose a stock after observing a few weeks of performance, while others may be uncertain even after following the stock for several months or years. This latter type of cautious decision-making has been observed in the laboratory, where healthy subjects acquire more information than is needed to make a decision (Edwards and Slovic, 1965) and are underconfident when estimating event likelihoods (Edwards, 1968). Little is known about the neural mechanisms underlying individual differences in underconfidence, despite clear relevance for understanding psychiatric disorders characterized by pathological uncertainty, such as obsessive-compulsive disorder (Fear and Healy, 1997).
Previous neuroimaging studies of uncertainty have mostly focused on activity when making a decision or anticipating an outcome, with parametric increases in uncertainty correlated with greater activity in dorsal medial frontal/anterior cingulate cortex (MFC/ACC) (Volz et al., 2003; Grinband et al., 2006; Krain et al., 2006) and paralimbic regions including lateral orbitofrontal cortex (OFC) and anterior insula (Critchley et al., 2001; Tobler et al., 2007; Preuschoff et al., 2008). In one study examining uncertainty during evidence accumulation, Huettel and colleagues (2005) found lateral fronto-parietal and insular activity when subjects observed stimuli associated with greater uncertainty for an upcoming decision. However, it remains unclear how the brain updates uncertainty in response to each sequential piece of evidence, and how such updating may be different from uncertainty experienced when actually executing a decision, when consequences for the outcome could be more salient.
In the current event-related fMRI study, participants rated subjective uncertainty while brain activity was measured separately on each of four sequential pieces of evidence, followed by a binary decision. The posterior probability computed from Bayes rule using the binomial likelihood and known 50:50 prior (“objective uncertainty”, in contrast to ratings of subjective uncertainty) was dynamically updated as information was acquired. To the extent that individual differences in subjective uncertainty (i.e., the self-reported posterior) are due to variability in risk appraisal and negative affect/arousal when observing evidence, underconfidence should be associated with ventral prefrontal and insular activity (Bechara et al., 2000; Paulus and Stein 2006). We further hypothesized that objective uncertainty during evidence accumulation would be tracked by regions involved in monitoring behaviorally relevant information, such as dorsal MFC/ACC (Carter et al., 1998), whereas executing a decision under uncertainty would uniquely engage decision or outcome risk areas, such as OFC and anterior insula (Rolls, 2004; Preuschoff et al., 2008).
Subjects
Twenty subjects free of psychiatric or neurological disorders were recruited to participate in the experiment. All subjects were right-handed, had normal or corrected-to-normal vision, and were not color-blind. Three subjects were excluded because they failed to understand the task adequately (e.g., they did not learn that all draws were from the same deck), leaving 17 subjects for analysis (mean age: 22.4, range: 19–32 years, 12 females). All subjects provided written informed consent as approved by the institutional review board of the University of Michigan Medical School.
Task
Instructions and practice trials for the task were presented prior to the fMRI session. While in the fMRI scanner, subjects received 72 sequences in which subjective uncertainty was rated while observing four pieces of evidence used to make a decision. On 54 of these sequences, two decks each containing red and blue cards were displayed at the top of the screen. Deck A (left side of screen) contained 80% red cards and 20% blue cards, while Deck B (right side of screen) contained 20% red cards and 80% blue cards. Subjects were informed of the percentages of red and blue cards contained in each deck, which were graphically represented by two rectangles containing a representative proportion of space colored in red and blue (Figure 1). At the start of each sequence, one of the decks was selected by the experimenter, and subjects needed to identify which deck had been selected by viewing four sequential “draws” of colored cards from the chosen deck (“evidence accumulation”). On each draw, subjects rated how certain they were that the deck providing the draws was located on the left (Deck A) or right side (Deck B) of the screen using a 9-point scale with “uncertain” at the center and “certain” at either end. Each subject was informed that the choices located in-between “uncertain” and “certain” on the scale should be interpreted as evenly-spaced intermediate categories between uncertainty and certainty. After viewing the four draws, subjects were given the option to choose which deck was supplying the cards or to decline to make a choice (“execution of decision”). Following this, feedback was presented and a new sequence was started. Unlike during evidence accumulation, decision execution was associated with incentive consequences so that a correct choice resulted in a gain of 50 points, an incorrect choice resulted in a loss of 50 points, and declining to choose resulted in no change in points. Importantly, the amount of points obtained (based on accuracy of decision) determined a monetary bonus received at the end of the experiment. This aspect of the paradigm was aimed at increasing the motivational salience of uncertainty when executing the decision, where subjects could potentially win or lose money, relative to uncertainty when observing draws during evidence accumulation, where ratings would have no monetary consequences. This feature was designed to mimic real-world rewards and punishments, which normally are based on decisions stemming from the evaluation of an entire set of information that has been accumulated rather than on the appraisal of individual pieces of evidence. All responses were made using MRI-compatible button gloves attached to both hands.
Figure 1
Figure 1
Sequences of trials in the evidence accumulation task
On each of these sequences, the likelihood of Deck A or Deck B being the deck supplying the observed draws changed dynamically as evidence accumulated, with each draw updating probabilities throughout a sequence. Subjects were informed that both decks had equal likelihood of being selected to supply the draws (prior of 50:50), with posteriors after each draw computed for both decks using Bayes’ theorem:
equation M1
  • Where P (X) = probability of one of the two decks supplying the cards
  • P (X’) = probability that the other deck is supplying the cards
  • P (Y) = probability of observing a particular sequence of colored cards
“Objective uncertainty” was defined as the calculated posterior probability for the most likely deck (based upon the observed evidence) after each card was drawn (throughout this paper, we report probabilities as percentages). The closer the calculated posterior probability was to 100%, the less the objective uncertainty. This objective uncertainty measure can be conceptualized as the absolute difference between the calculated posterior and 50% (a smaller absolute difference represents larger objective uncertainty), and thus is related to other measures of uncertainty that measure deviation from 50%, such as entropy. Importantly, the measure of objective uncertainty contrasts with a different measure we used, subjective uncertainty, which involved subjects’ actual ratings (described below) rather than calculated posteriors. We have taken an “experience sampling” approach (Hertwig et al., 2004) by computing the posterior probability based on information directly available to the participant (the sequence of draws they actually observed) rather than computing the posterior probability of the true deck (i.e. the one actually supplying the cards). For instance, even in cases where the true deck was Deck A (containing 80% blue cards and 20% red cards), a red card on the first draw would be calculated with a posterior probability of 80% (for Deck B) based on the evidence available to the subject at the time. This way, the uncertainty measure did not require that subjects have knowledge of the chosen deck and the calculated posterior could never fall below 50% for the more likely deck. By manipulating the order of presentation of red and blue cards in a sequence, the posterior probabilities that either deck was supplying the cards was varied on a draw-by-draw basis, reflecting the accumulation of evidence favoring one deck over the other, or not, depending upon the sequence of draws.
The relative number of red versus blue cards presented in a sequence dictated whether the calculated posteriors (objective uncertainty) increased or decreased between the first and last draw of the sequence, with the posteriors on the last draw also determining the uncertainty associated with the decision. There were three types of sequences (18 presentations each type): 1) “Type 2:2” sequences presented 2 cards of each color (final posterior of 50% for each deck; associated with high objective uncertainty), 2) “Type 3:1” sequences presented 3 cards of one color and 1 card of the other color (final posterior of 94.12% for the deck containing a majority of the dominant color observed; associated with moderate to low objective uncertainty), and 3) “Type 4:0” sequences presented 4 cards all of the same color (final posterior of 99.61 % for the deck containing a majority of the dominant color observed; associated with very low objective uncertainty). In addition to sequences where each deck contained both red and blue cards, 18 control sequences where one deck contained only blue cards and the other contained only red cards were included to account for non-specific effects occurring over the sequence (such as fatigue or habituation) that were unrelated to uncertainty. Subjects were informed that the observed evidence would fairly represent what would occur by chance based on the probability distributions of the decks. Furthermore, subjects were told that the number of cards contained in each deck was “infinite” and, hence, that counting of cards would not aid performance.
Deck A supplied the cards on half of sequences across the entire experiment and within sequence types, while Deck B supplied the cards on the other half. For sequences where both red and blue cards were drawn (Types 2:2 and 3:1), the order of cards was varied, with each order occurring in equal numbers when possible. Further, the majority color among the four draws in a sequence corresponded to the majority color of the deck selected to supply the draws (i.e., 3 red cards and 1 blue card were drawn from a deck that had 80% red and 20% blue cards). The only instance where this did not occur was on one Type 3:1 sequence, where 3 red cards and 1 blue card were drawn from a deck that contained 80% blue cards and 20% red cards. Consistent with our aim of having events in the task correspond to what would actually occur by chance, this sequence was used because the Bayesian probability of such an event is 5.88%, which is roughly equivalent to 1 out of our total of 18 (5.56 %) Type 3:1 sequences.
Each sequence began by showing the two decks for 500 ms, followed by the four draws (2000 ms each) and then the decision (2000 ms). All stimuli remained on-screen until the 2000 ms had elapsed regardless of when individual responses (subjective uncertainty ratings and decision) were made. If responses were not made within this time frame, a message appeared on-screen asking subjects to speed up on future trials. After each draw and decision, an inter-stimulus-interval (ISI) blank screen was presented for 500–4500 ms in order to isolate the BOLD signal for each event. Feedback was jittered between 2000–4000 ms after the decision, followed immediately by the start of the next sequence. These timings were determined using a design optimization program written in Matlab by RCW, which minimized mulitcolinearity between events for contrasts of interest.
Functional MRI Acquisition and Preprocessing
MRI scanning occurred on a GE 3T Signa scanner (LX [8.3] release). A T1-weighted image was acquired in the same prescription as the functional images to facilitate co-registration. Functional images were acquired with a T2*-weighted, reverse spiral acquisition sequence (GRE, TR = 2000, TE = 30, FA = 90, FOV = 20, 40 sl, 3.0/0, matrix dia. 71 - equivalent to 64 × 64) sensitive to signal in ventral medial frontal regions (Yang et al., 2002). Subjects underwent 6 runs, each consisting of 171 volumes plus 4 initial, discarded volumes to allow for thermal equilibration of scanner signal. After acquisition of functional volumes, a high resolution T1 SPGR scan was obtained for anatomic normalization.
Images were presented by a BrainLogics (PST, Inc., Pittsburgh, PA) digital MR projector, which provides high resolution video (1024 × 768) by back projection. Preprocessing was performed using the Statistical Parametric Mapping (SPM) 2 package (Wellcome Institute of Cognitive Neurology, London), with the exception of realignment, which used MCFLIRT (Jenkinson et al., 2002) and slice-time correction, which was interpolated with an 8-point sinc kernel multiplied by a Hanning window. Re-alignment parameters were inspected as a proxy for subject movement, in order to ensure that movement did not exceed either 3 mm translation or 1 degree rotation within each run. Parameters for anatomic normalization to the MNI152 brain, an average of 152 T1 images from the Montreal Neurological Institute, were derived from the high resolution SPGR T1 image and applied to the time series of co-registered, functional volumes, which were re-sliced and smoothed with a 5 mm isotropic Gaussian smoothing kernel.
Data Analysis
Behavioral analysis examined mean subjective uncertainty ratings in a 2 (draw 1, draw 4) × 3 (sequence types 2:2, 3:1, 4:0) repeated measures ANOVA and mean reaction times (RT) in a 3 (draw 1, draw 4, decision) × 3 (sequence types 2:2, 3:1, 4:0) ANOVA. In order to examine whether RT to make the subjective uncertainty rating depended on the level of objective or subjective uncertainty experienced by subjects, separate univariate ANOVAs examined mean RT for calculated posteriors (50, 80, 94.12, 98.46, 99.61 %) and self-reported posteriors (50, 67.5, 75, 87.5, 100 %). Finally, frequency of choosing the wrong deck (i.e., errors) and declining to choose a deck were also examined.
All analyses of fMRI BOLD signal were event-related. Regressors specifying the onset of each draw separately, the decision, and feedback were convolved with the canonical hemodynamic response function (HRF) using the general linear model as implemented in SPM2. Omission trials where subjects did not make a response within the deadline were infrequent (8.29 +/− 11.6 trials) and were included in the model to reduce error variance but were not analyzed further.
Four different models were used to analyze the data. To examine regions tracking objective uncertainty during evidence accumulation, one regressor specified for all draws, irrespective of sequence type and time (i.e., draw number 1, 2, 3, or 4), was parametrically modulated by the posterior probability for the most likely deck (50, 80, 94.12, 98.46, 99.61 %) (model 1). First-level contrasts of this modulation identified regions where brain activity became greater as the posterior for the most likely deck decreased (i.e., objective uncertainty increased); group-level analysis used one-sample t-tests. Regions activated in group-level analyses were further examined to determine the pattern of activity over time (draw 1 vs. draw 4) based on sequence type. In order to do this, a second model (model 2) specified separate regressors for each draw (1, 2, 3, and 4) for each sequence type, with no modulation. First-level contrasts were used to obtain parameter estimates for each model 2 regressor against baseline in those areas activated at the group level in model 1. These parameter estimates were averaged across all voxels in the activated region and separately submitted to 2 (draw 1, draw 4) × 3 (sequence types 2:2, 3:1, 4:0) repeated-measures ANOVAs. Values were extracted only from draws 1 and 4 because probabilities for these draws did not differ based on the ordering of the sequence, and there were not enough trials contained in the different combinations of orderings to reliably examine the pattern of activity for draws 2 and 3 based on sequence type.
To directly compare uncertainty-related activations when accumulating evidence vs. executing a decision that contained incentive consequences (reward or punishment), a third model (model 3) specified separate regressors for the fourth (last) draw and when executing the decision, irrespective of sequence type, each parametrically modulated by posterior probability for the most likely deck (similar to model 1, with the exception that only the fourth draw and the decision were parametrically modulated in model 3). Separate first-level contrasts of the modulations identified areas showing greater activation as the posterior for the most likely deck decreased on draw 4 alone and when executing the decision, as well as a direct comparison identifying regions correlated with posteriors when executing the decision but not on the fourth draw. As no further evidence was provided following draw 4 – and hence, Bayesian probabilities were the same for draw 4 and the decision – differences identified in this comparison can be interpreted as due to an interaction between objective uncertainty and the psychological process involved (i.e., observing evidence on draw 4 vs. making a decision).
To facilitate comparisons between regions tracking calculated objective uncertainty vs. subjective uncertainty, subjective uncertainty ratings for each draw were converted into self-reported posteriors through a linear transformation. A rating of “uncertain” was converted to a 50% probability that either deck was supplying the draws while a rating of “certain” for either deck was converted to a 100% probability, with intervals of 12.5% spanning intermediate ratings (62.5, 75, and 87.5 %). Self-reported posteriors were then used in a fourth model (model 4) to parametrically modulate a regressor for all draws (just as in model 1, irrespective of sequence type), with first-level contrasts of the modulation identifying regions where brain activity became greater as self-reported posteriors decreased (i.e., subjective uncertainty increased). Models 1 and 4 were run separately to avoid the regressor multicolinearity that would be present in a unified model due to the high correlation between calculated and self-reported posteriors (r = 0.68).
The design also permitted the analysis of underconfidence during evidence accumulation by comparing calculated posteriors with self-reported posteriors, and correlating these differences with brain activation while observing evidence. A difference score was obtained for each subject by subtracting self-reported posteriors from calculated posteriors for the most likely deck for each of the four draws, with larger scores representing greater underconfidence/subjective uncertainty. This type of underconfidence occurs when the point estimate of one’s prediction is lower than the actual value, and differs from a related bias of overly narrow confidence intervals around a prediction, i.e., second-order overconfidence (Lichtenstein et al., 1982). First-level contrasts obtained from model 2 (separate regressors for each draw, without parametric modulations), compared regions of activation for each draw regressor, averaged across sequence type, against baseline. Simple regression at the group level identified between-subjects correlations of neural activation with mean deviation score for each draw (irrespective of sequence type) to identify brain regions showing increased activity for subjects exhibiting greater underconfidence during evidence accumulation.
For the models described above, all events in the task were specified as regressors (draws, decisions, feedback) even if not the focus of the analysis. Furthermore, events in control sequences were also included in all models, but these events were not tested in the parametric analysis, i.e. they were not given Bayesian probabilities, and were not included in the contrasts. However, for comparison purposes, activity in those regions identified during non-control sequences is also presented for control sequences in the results and figures. Except where noted, all results are significant at p < .05, corrected for multiple comparisons across the whole brain at the cluster level (voxel-wise threshold of z >2.58).
Behavioral
Mean subjective uncertainty ratings and RT for these ratings are shown in Figure 2. Analysis of subjective uncertainty ratings based on draw number and sequence type revealed main effects and an interaction between factors (interaction: F (2,32) = 200.8, p < .001). Certainty increased between draw 1 and draw 4 on sequences where all four cards were the same color (i.e., Type 4:0) or where three of the cards were the same color and one card was a different color (i.e., Type 3:1), but decreased when two cards of each color were presented (i.e., Type 2:2). Although updating of subjective uncertainty paralleled Bayesian probabilities, some subjects were consistently more underconfident than others (i.e., they rated subjective uncertainty higher), especially on the first draw (see scatterplots in Figure 5).
Figure 2
Figure 2
Subjective uncertainty ratings and reaction times
Figure 5
Figure 5
Greater VMPFC activity associated with underconfidence during evidence accumulation
Analyses of RT also identified main effects and an interaction between factors interaction: (F (4, 64) = 18.5, p < .001, see Figure 2). On Type 4:0 sequences, RTs were faster when rating uncertainty on draw 4 as compared to draw 1, but were similar for draw 4 and when making a decision. However, on both Type 3:1 and Type 2:2 sequences, RTs were similar when rating uncertainty for draws 1 and 4 and then decreased when making a decision.
Analysis of RT based on calculated posteriors indicated that subjects were significantly slower to make subjective uncertainty ratings when the Bayesian probability for the most likely deck was lower (i.e. objective uncertainty was higher) (F (4, 64) = 21.3, p < .001; 50 %: 941.7, 80 %: 975.8, 94.12 %: 857.2, 98.46 %: 769.6, 99.16 %: 665.1 ms). Similarly, there was a main effect of self-reported posterior probability on RT (F (4, 106) = 9.1, p < .001; 50 %: 977.9, 62.5 %: 1016.6. 75 %: 918.0, 87.5 %: 826.5, 100 %: 752.7 ms), indicating that ratings were slower when subjects reported more subjective uncertainty.
Subjects declined to choose a deck on 87.4 % of Type 2:2 sequences, compared with 1 % and 0 % decline choices on Type 3:1 and Type 4:0 sequences, respectively. When subjects did choose a deck on Type 4:0 and 3:1 sequences, accuracy was very high, with only one subject making one error on a Type 3:1 sequence and no errors for any subjects on Type 4:0 sequences.
Functional neuroimaging
Objective uncertainty when accumulating evidence
Results from model 1, where all draws were modulated by their Bayesian probability irrespective of draw number and sequence type, identified regions in dorsal anterior cingulate cortex (dACC), precuneus, and left postcentral gyrus that showed increasing activity as posterior probability decreased (i.e., objective uncertainty increased) (Table 1, Figure 3). A post-hoc repeated-measures ANOVA on dACC activity as evidence accumulated between the first and last draws (model 2) revealed an interaction between draw number and sequence type (F (2,32) = 7.8, p = .002) such that activity increased between the first and last draw for Type 2:2 and 3:1 sequences, but decreased for Type 4:0 sequences (Figure 3). Post-hoc t-tests (two-tailed) indicated that dACC activity did not differ between any of the three sequence types on draw 1 (p > .3 for all comparisons), but that Type 2:2 activity was greater than Type 3:1 (t (16) = −2.7, p = .017), and Type 3:1 activity was greater than Type 4:0 (t (16) = −2.8, p = .012), on draw 4. Due to the differential temporal pattern of activity in dACC based on sequence type, we hypothesized that this region was involved in detecting when information was inconsistent with expectations that had been established through prior observation of draws in the sequence. To directly test this hypothesis, we performed a secondary analysis in which draws 2, 3, and 4 were sorted based on whether they were consistent or inconsistent with prior draws in the sequence (see Supplemental Figure 1). Consistency or inconsistency was defined as a draw that respectively increased or decreased the posterior probability. Results from this analysis supported our hypothesis, revealing robust activation in the same region of dACC for the comparison of inconsistent > consistent draws (see Supplemental Figure 2).
Table 1
Table 1
Activity associated with increasing objective uncertainty when observing evidence and when executing a decision at p < .05 corrected for multiple comparisons across the whole brain. BA = Brodmann’s areas; k = number of voxels; coordinates (more ...)
Figure 3
Figure 3
Neural activity related to increasing objective uncertainty during evidence accumulation
Similar to dACC, an ANOVA on precuneus activation indicated no difference on draw 1 for any of the sequence types (p > .2 for all comparisons), and an increase in precuneus activity between draws 1 and 4 for Type 2:2 and 3:1, but not Type 4:0, sequences (interaction: F (2,32) = 9.0, p = .001). However, unlike dACC activity, the amount of increase between draws 1 and 4 was the same for Type 2:2 and 3:1 sequences, as indicated by a lack of difference between these sequence types on draw 4 (p = . 47). As can be seen from Figure 3, precuneus activity on draw 4 was greater for Type 2:2 and 3:1 sequences as compared to Type 4:0 sequences (t (16) = −3.3, p = .004 and t (16) = −3.5, p = .003, respectively).
Finally, given that reaction times were slower when calculated and self-reported posteriors were smaller (i.e., higher objective and subjective uncertainty), we sought to determine whether these relationships between neural activity and uncertainty were due to differences in RT. A supplemental model specifying the same regressors as model 1 and including RT as a covariate on each draw identified relationships between posterior probability and BOLD signal in the same regions as in model 1 (dACC: x = 6, y = 27, z = 30, z-score = 3.43, k = 43; precuneus: x = −33, y = −87, z = 30, z-score = 3.97, k = 315; postcentral gyrus: x = −57, y = −18, z = 51, z-score = 5.0, k = 79) (see Supplemental Figure 3). Thus, although response times for uncertainty ratings were different for varying levels of calculated and self-reported posterior probabilities, the brain activations correlated with uncertainty during evidence accumulation cannot be explained by this difference in RT.
Objective uncertainty when executing a decision
To compare uncertainty-related activations when observing evidence with those at the time of the decision, the fourth draw and decision were separately modulated by Bayesian probability (model 3). For the fourth draw only, uncertainty-related increases in activity were found in the same regions as for the entire sequence of draws (Table 1), although medial frontal activations spanned a larger region of cortex, encompassing anterior regions of MFC in addition to dACC. Increased activity in dorsal MFC/ACC was also related to greater uncertainty when executing the decision; in addition, decision-related uncertainty elicited activity in a distributed network of lateral frontal, parietal, and temporal cortex (Table 1 and Figure 4).
Figure 4
Figure 4
Neural activity related to increasing objective uncertainty on the last draw of evidence accumulation and during execution of the decision
Although subjects selected the “decline to choose” option on the majority of Type 2:2 sequences (87 % of trials on average), it is possible that anticipation of receiving error feedback on trials where subjects did not choose this option (i.e. when they guessed which deck was providing the draws), rather than greater uncertainty, could be influencing these findings. Thus, we conducted a secondary analysis that excluded all trials in which subjects guessed which deck was providing the draws on Type 2:2 sequences. Results were largely unchanged, identifying similar areas of activation excepting the right middle temporal region (see Supplemental Table 1), confirming that increasing activation in this widespread fronto-parietal network when executing a decision under uncertainty was not due to greater expectation of receiving error feedback in response to guesses on Type 2:2 sequences.
A direct comparison between regions modulated by posterior probability during evidence accumulation and execution of the decision revealed several areas that increased in activity as objective uncertainty became greater during the decision but not when observing the last piece of evidence (even though probabilities were identical), including left hemisphere regions of OFC/inferior frontal cortex (IFC), superior and middle frontal gyri, and putamen, as well as bilateral dorsomedial frontal and inferior parietal cortices (Figure 4). Two regions in right pre/postcentral gryus and posterior MFC also emerged from the contrast; however, these activations were due to uncertainty-related decreases on draw 4 rather than uncertainty-related increases when executing a decision.
Subjective uncertainty and underconfidence when accumulating evidence
Neural regions that correlated with self-reported posterior probability for all draws during evidence accumulation (model 4) were similar to those correlated with calculated posteriors for all draws in model 1 (precuneus, left postcentral gyrus), albeit with reduced significance, indicated by lower z scores (Table 2). With correction across the whole brain, dACC activity was not related to subjective uncertainty; at an uncorrected threshold, a very small cluster of activation in dACC (−6, 21, 36, k = 5) was identified. This overlap in uncertainty-related activations, with smaller z scores in regions tracking self-reported as compared to calculated posteriors, is consistent with our behavioral results indicating that subjective ratings were appropriately sensitive to changes in Bayesian probability overall, but with notable deviations (i.e., underconfidence) exhibited by some subjects potentially impacting neural activation at the group level.
Table 2
Table 2
Activity associated with increasing subjective uncertainty and underconfidence when observing evidence at p < .05 corrected for multiple comparisons across the whole brain. BA = Brodmann’s areas; k = number of voxels; coordinates are in (more ...)
In order to examine brain regions involved in underconfidence, mean deviation scores (calculated minus self-reported posterior probability, with larger values corresponding to greater underconfidence) for each of the four draws were separately correlated with their respective parameter estimates at each voxel in the brain, irrespective of sequence type (model 2). During the first draw, subjects with greater activity in ventromedial prefrontal cortex (VMPFC) exhibited greater underconfidence when making subjective ratings (Table 2 and Figure 5). Overlapping regions of VMPFC were also positively correlated with underconfidence on draw 2 (−9, 15,−21, k = 6), draw 3 (−12, 18,−24, k = 7), and draw 4 (−3, 9,−21, k = 13), although these activations were not significant when corrected for multiple comparisons. In addition, increased activity in right precentral gyrus and bilateral postcentral gyrus/paracentral lobule was associated with greater underconfidence on draw 4.
The current study examined how the brain evaluates information when updating uncertainty about a future decision. Although dACC activity was correlated with objective uncertainty when observing evidence, the pattern of activity between the first and last draw suggests that this region was not tracking probabilities in a strict sense. Activity in dACC became greater between draw 1 and 4 as calculated posterior probabilities decreased (i.e., objective uncertainty increased) in Type 2:2 sequences, yet it also increased (albeit to a lesser degree) when posterior probabilities increased (i.e., objective uncertainty decreased) in Type 3:1 sequences. Indeed, even at an uncorrected threshold, we did not find any regions that behaved as if they were tracking objective probabilities (i.e., decreasing activity between draw 1 and draw 4 in Type 4:0 and 3:1 sequences in parallel with increasing posteriors, and increasing activity in Type 2:2 sequences in parallel with decreasing posteriors).
These findings may be interpreted in light of major theories of dACC functioning that have focused on evaluative and monitoring processes. Error detection theory suggests that dACC monitors mismatches between actual and intended responses (Scheffers and Coles, 2000), while conflict monitoring theory states that dACC detects conflict between co-activated responses (Botvinick et al., 2001). By contrast, Brown and Braver (2005) proposed that dACC does not respond to errors or conflicts per se, but instead is activated in situations where error likelihood is greatest, and, as such, has a role in risk prediction. Finally, reinforcement learning (RL) theory suggests that dACC is involved in learning after obtaining an outcome that is worse than expected, i.e. a reward prediction error (Holroyd and Coles, 2002), with later expansion of this theory to include outcomes that are merely different than expected (Oliveira et al., 2007), i.e., prediction errors not restricted to conditions of reward omission.
The pattern of results in dACC cannot be explained by existing error detection, conflict monitoring, or error likelihood theories. There were no overt errors during evidence accumulation, and both the response conflict associated with choosing a deck and the likelihood of making a future error in response to the decision would have been reduced in Type 3:1 sequences as uncertainty decreased, yet our data show that the dACC signal increased between the first and last draw. Our findings are instead compatible with a role for dACC in responding to unexpected or unpredicted outcomes (Oliveira et al., 2007). During evidence accumulation, dACC may have increased between draws 1 and 4 on Type 3:1 and 2:2 sequences because at least one draw was observed that was inconsistent with expectations established through prior observations of draws in a sequence (see supplemental Figures 1 and 2), with the largest increase occurring in situations when expectations were repeatedly violated (i.e., Type 2:2 sequences). By contrast, dACC activity did not increase as evidence accumulated on Type 4:0 sequences because all draws consistently pointed to the same deck.
Frontal regions including OFC/IFC, superior and middle frontal gyri, and dorsomedial frontal cortex, as well as bilateral inferior parietal cortex, were uniquely associated with greater uncertainty when executing a decision at the end of the sequence as compared to observing the last draw, even though objective uncertainty was the same. Although making ratings and executing the decision both involve choices regarding the identity of the deck supplying the cards, only the decision made at the end of the sequence is associated with incentive consequences. For this reason, unlike other studies that have identified OFC in uncertainty processing (Critchley et al., 2001; Tobler et al., 2007), the current paradigm was able to disentangle activations related to the cognitive experience of uncertainty from those associated with motivational processes such as reward risk that often vary along with uncertainty. The lack of significant modulation of OFC by uncertainty when subjects accumulated evidence in the absence of risk suggests that this region does not detect cognitive uncertainty per se, but instead responds to uncertainty only when there are incentive consequences. This is consistent with the well-documented role of OFC in associating stimuli with the value of outcomes (Rushworth et al., 2007; Wallis, 2007), with lateral regions preferentially involved in processing punishment and disappointment (O’Doherty et al., 2001; Ursu and Carter, 2005). Indeed, uncertain sequences in our task should elicit a greater expectation of not receiving a reward, which may lead to disappointment when executing the decision. Alternatively, OFC/IFC activity may reflect a “safety signal” associated with opting-out of making a decision, given prior work identifying a relationship between safe choices among risk-averse individuals and activity in a region of IFC located slightly superior to our own (Christopoulos et al., 2009).
Although we did find activity in OFC/IFC, we did not replicate previous findings linking decision uncertainty to nearby anterior insula (Grinband et al., 2006; Huettel, 2006, Preuschoff et al., 2008; Volz et al., 2003), which may be due to the fact that greater decision uncertainty in prior studies often also signals an increased likelihood for error. Errors robustly activate anterior insula, perhaps due to the aversive emotional response they elicit (Simmons et al., 2004), even in the absence of incentives or feedback (see Taylor et al., 2007). Thus, the lack of insula activation in the present study may be due to our design, which dissociated increased uncertainty from error likelihood by allowing subjects to avoid an error by declining to make a decision on uncertain sequences.
Previous studies, including the analyses described above, have examined brain regions responding to uncertainty as defined by experimental stimuli. However, uncertainty is not only determined by external information, but is also modulated by internal cognitive or emotional states, which vary between individuals. The VMPFC activity that correlated with underconfidence during evidence accumulation is not likely to be due to greater difficulty among underconfident subjects, as considerable evidence suggests that VMPFC activation decreases with cognitive effort (Simpson et al., 2001). Instead, we suggest that increased VMPFC activity may be related to greater arousal in autonomic and emotional systems when evaluating evidence. VMPFC has been conceptualized as part of a network important for integrating emotional-somatic states with factual information in order to guide decision-making (Damasio et al., 1996), due to strong interconnectivity with limbic/autonomic regions including the amygdala, nucleus accumbens, hypothalamus, and midbrain (Kringelbach and Rolls, 2004; Price, 2007). VMPFC activity is associated with greater arousal as measured by skin conductance (Critchley et al., 2000), and is part of the “default mode” network that deactivates during directed attention (Simpson et al., 2001). Damage to this region leads to an increase in risk-taking behavior (Clark et al., 2008), possibly because patients do not generate appropriate emotional-somatic anticipatory reactions prior to making a risky choice (Bechara et al., 2000).
Our results indicate that VMPFC activity is associated with relatively conservative ratings of certainty, even in the absence of overt risk when subjects are merely evaluating evidence. In theory, increased subjective uncertainty could be driven by a cognitive mechanism, such as an incorrect judgment of likelihood, or an emotional/motivational process involving the experience of increased arousal or appraisal of risk when evaluating evidence. While correlational data cannot completely rule out the influence of aberrant cognition, our findings in VMPFC suggest that somatic/emotional mechanisms play an important role in determining subjective uncertainty, at least among healthy subjects. Of interest, individual variability in underconfidence was greater, and the relationship between VMPFC and underconfidence was stronger, on the first draw as compared to later draws. This suggests that the neural signatures of conservatism have their greatest influence when observing limited information, and that increasing evidence allows underconfident subjects to recover from initial uncertainty.
Overall, our data provide insight into how the brain updates beliefs during evidence accumulation, and how emotional responses may interact with cognitive functioning to process uncertainty. Updating beliefs in a roughly Bayesian manner involves the activation of dorsal ACC regions to evaluate whether evidence is consistent or inconsistent with expectations, while the simultaneous activation of emotional-related processes by VMPFC may lead to a phenomenological experience of uncertainty that is greater (i.e., more underconfident) than what would be expected from the monitoring of dACC output alone.
While results from the current study serve to elucidate the mechanisms of uncertainty during decision-making, limitations and questions remain for future research. Although subjective ratings after each piece of information served a critical role in our design and analysis, it is possible that the mere process of rating uncertainty (in addition to experiencing uncertainty) lead to artificial reductions in emotionality (Taylor et al., 2003), which may have minimized the involvement of limbic/paralimbic brain regions in our task. Further, methodological constraints required that we use a categorical scale for uncertainty ratings, yet a continuous visual-analog scale would have been preferable for detecting subtle differences between subjects. As such, our results may underestimate the full scope of brain regions involved in underconfidence. Despite these limitations, the current results represent an important step toward understanding how uncertainty is processed during different phases of decision-making, and point to the need to investigate evidence accumulation in disorders characterized by pathological uncertainty.
Supplementary Material
Supp1
Acknowledgments
Support for this research has been provided by National Institute of Mental Health National Research Service Award F32 MH082573 and the University of Michigan fMRI Center pilot funding awarded to ERS and National Institute of Mental Health R01 MH071821 awarded to SFT.
  • Bechara A, Damasio H, Damasio AR. Emotion, decision making and the orbitofrontal cortex. Cereb Cortex. 2000;10:295–307. [PubMed]
  • Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. Conflict monitoring and cognitive control. Psychol Rev. 2001;108:624–652. [PubMed]
  • Brown JW, Braver TS. Learned predictions of error likelihood in the anterior cingulate cortex. Science. 2005;307:1118–1121. [PubMed]
  • Carter CS, Braver TS, Barch DM, Botvinick MM, Noll D, Cohen JD. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science. 1998;280:747–749. [PubMed]
  • Christopoulos GI, Tobler PN, Bossaerts P, Dolan RJ, Schultz W. Neural correlates of value, risk, and risk aversion contributing to decision making under risk. J Neurosci. 2009;29:12574–12583. [PMC free article] [PubMed]
  • Clark L, Bechara A, Damasio H, Aitken MRF, Sahakian BJ, Robbins TW. Differential effects of insular and ventromedial prefrontal cortex lesions on risky decision-making. Brain. 2008;131:1311–1322. [PubMed]
  • Critchley HD, Mathias CJ, Dolan RJ. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron. 2001;29:537–545. [PubMed]
  • Critchley HD, Elliott R, Mathias CJ, Dolan RJ. Neural activity relating to generation and representation of galvanic skin conductance responses: a functional magnetic resonance imaging study. J Neurosci. 2000;20:3033–3040. [PubMed]
  • Damasio AR. The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos Trans R Soc Lond B Biol Sci. 1996;351:1413–1420. [PubMed]
  • Edwards W. Conservatism in human information processing. In: Kleinmuntz B, editor. Formal Representation of Human Judgment. New York: John Wiley and Sons; 1968. pp. 17–52.
  • Edwards W, Slovic P. Seeking information to reduce the risk of decisions. Am J Psychol. 1965;78:188–197. [PubMed]
  • Fear CF, Healy D. Probabilistic reasoning in obsessive-compulsive and delusional disorders. Psychol Med. 1997;27:199–208. [PubMed]
  • Grinband J, Hirsch J, Ferrera VP. A neural representation of categorization uncertainty in the human brain. Neuron. 2006;49:757–763. [PubMed]
  • Hertwig R, Barron G, Weber EU, Erev I. Decisions from experience and the effect of rare events on risky choice. Psychol Sci. 2004;15:534–539. [PubMed]
  • Holroyd CB, Coles MG. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev. 2002;109:679–709. [PubMed]
  • Huettel SA. Behavioral, but not reward, risk modulates activation of prefrontal, parietal, and insular cortices. Cogn Affect Behav Neurosci. 2006;6:141–151. [PubMed]
  • Huettel SA, Song AW, McCarthy G. Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J Neurosci. 2005;25:3304–3311. [PubMed]
  • Jenkinson M, Bannister P, Brady M, Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage. 2002;17:825–841. [PubMed]
  • Krain AL, Hefton S, Pine DS, Ernst M, Castellanos FX, Klein RG, Milham MP. An fMRI examination of developmental differences in the neural correlates of uncertainty and decision-making. J Child Psychol Psychiatry. 2006;47:1023–1030. [PubMed]
  • Kringelbach ML, Rolls ET. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog Neurobiol. 2004;72:341–372. [PubMed]
  • Lichtenstein S, Fischoff B, Phillips LD. Calibration of probabilities: The state of the art to 1980. In: Kahneman D, Slovic P, Tversky A, editors. Judgment under uncertainty: Heuristics and biases. Cambridge: University Press; 1982. pp. 306–334.
  • O'Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci. 2001;4:95–102. [PubMed]
  • Oliveira FT, McDonald JJ, Goodman D. Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J Cogn Neurosci. 2007;19:1994–2004. [PubMed]
  • Paulus MP, Stein MB. An insular view of anxiety. Biol Psychiatry. 2006;60:383–387. [PubMed]
  • Preuschoff K, Quartz SR, Bossaerts P. Human insula activation reflects risk prediction errors as well as risk. J Neurosci. 2008;28:2745–2752. [PubMed]
  • Price JL. Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions. Ann N Y Acad Sci. 2007;1121:54–71. [PubMed]
  • Rolls ET. The functions of the orbitofrontal cortex. Brain Cogn. 2004;55:11–29. [PubMed]
  • Rushworth MFS, Behrens TEJ, Rudebeck PH, Walton ME. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn Sci. 2007;11:168–176. [PubMed]
  • Scheffers MK, Coles MG. Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. J Exp Psychol Hum Percept Perform. 2000;26:141–151. [PubMed]
  • Simmons A, Matthews SC, Stein MB, Paulus MP. Anticipation of emotionally aversive visual stimuli activates right insula. Neuroreport. 2004;15:2261–2265. [PubMed]
  • Simpson JR, Snyder AZ, Gusnard DA, Raichle ME. Emotion-induced changes in human medial prefrontal cortex: I. During cognitive task performance. Proc Natl Acad Sci U S A. 2001;98:683–687. [PubMed]
  • Taylor SF, Phan KL, Decker LR, Liberzon I. Subjective rating of emotionally salient stimuli modulates neural activity. Neuroimage. 2003;18:650–659. [PubMed]
  • Taylor SF, Stern ER, Gehring WJ. Neural systems for error monitoring: Recent findings and theoretical perspectives. Neuroscientist. 2007;13:160–172. [PubMed]
  • Tobler PN, O'Doherty JP, Dolan RJ, Schultz W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J Neurophysiol. 2007;97:1621–1632. [PMC free article] [PubMed]
  • Ursu S, Carter CS. Outcome representations, counterfactual comparisons and the human orbitofrontal cortex: Implications for neuroirnaging studies of decision-making. Cognitive Brain Research. 2005;23:51–60. [PubMed]
  • Volz KG, Schubotz RI, von Cramon DY. Predicting events of varying probability: uncertainty investigated by fMRI. Neuroimage. 2003;19:271–280. [PubMed]
  • Wallis JD. Orbitofrontal cortex and its contribution to decision-making. Annu Rev Neurosci. 2007;30:31–56. [PubMed]
  • Weber EU, Milliman RA. Perceived risk attitudes: relating risk perception to risky choice. Manage Sci. 1997;43:123–144.
  • Yang Y, Gu H, Zhan W, Xu S, Silbersweig DA, Stern E. Simultaneous perfusion and BOLD imaging using reverse spiral scanning at 3T: characterization of functional contrast and susceptibility artifacts. Magn Reson Med. 2002;48:278–289. [PubMed]