Acetylcholine (ACh) is a neuromodulatory transmitter implicated in perception and learning under uncertainty. This study combined computational simulations and pharmaco-electroencephalography in humans, to test a formulation of perceptual inference based upon the free energy principle. This formulation suggests that acetylcholine enhances the precision of bottom-up synaptic transmission in cortical hierarchies by optimising the gain of supragranular pyramidal cells. Simulations of a mismatch negativity paradigm predicted a rapid trial-by-trial suppression of evoked sensory prediction error (PE) responses that is attenuated by cholinergic neuromodulation. We confirmed this prediction empirically with a placebo-controlled study of cholinesterase inhibition. Furthermore – using dynamic causal modelling – we found that drug-induced differences in PE responses could be explained by gain modulation in supragranular pyramidal cells in primary sensory cortex. This suggests that acetylcholine adaptively enhances sensory precision by boosting bottom-up signalling when stimuli are predictable, enabling the brain to respond optimally under different levels of environmental uncertainty.
Free Energy Principle; Predictive Coding; Neuromodulation; Acetylcholine; Galantamine; Oddball Response; Precision; Dynamic Causal Modelling
The aging brain shows a progressive loss of neuropil, which is accompanied by subtle changes in neuronal plasticity, sensory learning and memory. Neurophysiologically, aging attenuates evoked responses—including the mismatch negativity (MMN). This is accompanied by a shift in cortical responsivity from sensory (posterior) regions to executive (anterior) regions, which has been interpreted as a compensatory response for cognitive decline. Theoretical neurobiology offers a simpler explanation for all of these effects—from a Bayesian perspective, as the brain is progressively optimized to model its world, its complexity will decrease. A corollary of this complexity reduction is an attenuation of Bayesian updating or sensory learning. Here we confirmed this hypothesis using magnetoencephalographic recordings of the mismatch negativity elicited in a large cohort of human subjects, in their third to ninth decade. Employing dynamic causal modeling to assay the synaptic mechanisms underlying these non-invasive recordings, we found a selective age-related attenuation of synaptic connectivity changes that underpin rapid sensory learning. In contrast, baseline synaptic connectivity strengths were consistently strong over the decades. Our findings suggest that the lifetime accrual of sensory experience optimizes functional brain architectures to enable efficient and generalizable predictions of the world.
While studies of aging are widely framed in terms of their demarcation of degenerative processes, the brain provides a unique opportunity to uncover the adaptive effects of getting older. Though intuitively reasonable, that life-experience and wisdom should reside somewhere in human cortex, these features have eluded neuroscientific explanation. The present study utilizes a “Bayesian Brain” framework to motivate an analysis of cortical circuit processing. From a Bayesian perspective, the brain represents a model of its environment and offers predictions about the world, while responding, through changing synaptic strengths to novel interactions and experiences. We hypothesized that these predictive and updating processes are modified as we age, representing an optimization of neuronal architecture. Using novel sensory stimuli we demonstrate that synaptic connections of older brains resist trial by trial learning to provide a robust model of their sensory environment. These older brains are capable of processing a wider range of sensory inputs – representing experienced generalists. We thus explain how, contrary to a singularly degenerative point-of-view, aging neurobiological effects may be understood, in sanguine terms, as adaptive and useful.
Asymmetry in distributions of potential outcomes (i.e. skewness), and whether those potential outcomes reflect gains or losses (i.e. their valence), both exert a powerful influence on value-based choice. How valence affects the impact of skewness on choice is unknown. Here by orthogonally manipulating the skewness and valence of economic stimuli we show that both have an influence on choice. We show that the influence of skewness on choice is independent of valence, both across and within subjects. fMRI data revealed skew-related activity in bilateral anterior insula and dorsomedial prefrontal cortex, which shows no interaction with valence. Further, the expression of skew-related activity depends on an individual’s preference for skewness, and this was again independent of valence-related preference. Our findings highlight the importance of skewness in choice and show that its influence, both behaviourally and neurally, is distinct from an influence of valence.
Neural encoding of value-based stimuli is suggested to involve representations of summary statistics, including risk and expected value (EV). A more complex, but ecologically more common, context is when multiple risky options are evaluated together. However, it is unknown whether encoding related to option evaluation in these situations involves similar principles. Here we employed fMRI during a task that parametrically manipulated EV and risk in two simultaneously presented lotteries, both of which contained either gains or losses. We found representations of EV in medial prefrontal cortex and anterior insula, an encoding that was dependent on which option was chosen (i.e. chosen and unchosen EV) and whether the choice was over gains or losses. Parietal activity reflected whether the riskier or surer option was selected, whilst activity in a network of regions that also included parietal cortex reflected both combined risk and difference in risk for the two options. Our findings provide support for the idea that summary statistics underpin a representation of value-based stimuli, and further that these summary statistics undergo distinct forms of encoding.
•We examine choice between multiple risky options.•fMRI revealed encoding of value-based stimuli as expected value (EV) and risk.•Risk and EV underwent distinct types of encoding in distinct brain regions.•Neural and RT data were also consistent with model-free influences on choice.
Risk; Loss; fMRI; Approach–avoidance
In humans, dopamine is implicated in reward and risk-based decision-making. However, the specific effects of dopamine augmentation on risk evaluation are unclear. Here we sought to measure the effect of 100 mg oral levodopa, which enhances synaptic release of dopamine, on choice behaviour in healthy humans. We use a paradigm without feedback or learning, which solely isolates effects on risk evaluation. We present two studies (n = 20; n = 20) employing a randomised, placebo-controlled, within-subjects design. We manipulated different dimensions of risk in a controlled economic paradigm. We test effects on risk-reward tradeoffs, assaying both aversion to variance (the spread of possible outcomes) and preference for relative losses and gains (asymmetry of outcomes - skewness), dissociating this from potential non-specific effects on choice randomness using behavioural modelling. There were no systematic effects of levodopa on risk attitudes, either for variance or skewness. However, there was a drift towards more risk-averse behaviour over time, indicating that this paradigm was sensitive to detect changes in risk-preferences. These findings suggest that levodopa administration does not change the evaluation of risk. One possible reason is that dopaminergic influences on decision making may be due to changing the response to reward feedback.
Value-based choices are influenced both by risk in potential outcomes and by whether outcomes reflect potential gains or losses. These variables are held to be related in a specific fashion, manifest in risk aversion for gains and risk seeking for losses. Instead, we hypothesized that there are independent impacts of risk and loss on choice such that, depending on context, subjects can show either risk aversion for gains and risk seeking for losses or the exact opposite. We demonstrate this independence in a gambling task, by selectively reversing a loss-induced effect (causing more gambling for gains than losses and the reverse) while leaving risk aversion unaffected. Consistent with these dissociable behavioral impacts of risk and loss, fMRI data revealed dissociable neural correlates of these variables, with parietal cortex tracking risk and orbitofrontal cortex and striatum tracking loss. Based on our neural data, we hypothesized that risk and loss influence action selection through approach–avoidance mechanisms, a hypothesis supported in an experiment in which we show valence and risk-dependent reaction time effects in line with this putative mechanism. We suggest that in the choice process risk and loss can independently engage approach–avoidance mechanisms. This can provide a novel explanation for how risk influences action selection and explains both classically described choice behavior as well as behavioral patterns not predicted by existing theory.
The neuroscience of human decision-making has focused on localizing brain activity correlating with decision variables and choice, most commonly using functional MRI (fMRI). Poor temporal resolution means these studies are agnostic in relation to how decisions unfold in time. Consequently, here we address the temporal evolution of neural activity related to encoding of risk using magnetoencephalography (MEG), and show modulations of electromagnetic power in posterior parietal and dorsomedial prefrontal cortex (DMPFC) which scale with both variance and skewness in a lottery, detectable within 500 ms following stimulus presentation. Electromagnetic responses in somatosensory cortex following this risk encoding predict subsequent choices. Furthermore, within anterior insula we observed early and late effects of subject-specific risk preferences, suggestive of a role in both risk assessment and risk anticipation during choice. The observation that cortical activity tracks specific and independent components of risk from early time-points in a decision-making task supports the hypothesis that specialized brain circuitry underpins risk perception.
decision-making; magnetoencephalography (MEG); risk; neuroeconomics; cortex
Bradykinesia is a cardinal feature of Parkinson’s disease (PD). Despite its disabling impact, the precise cause of this symptom remains elusive. Recent thinking suggests that bradykinesia may be more than simply a manifestation of motor slowness, and may in part reflect a specific deficit in the operation of motivational vigour in the striatum. In this paper we test the hypothesis that movement time in PD can be modulated by the specific nature of the motivational salience of possible action-outcomes.
We developed a novel movement time paradigm involving winnable rewards and avoidable painful electrical stimuli. The faster the subjects performed an action the more likely they were to win money (in appetitive blocks) or to avoid a painful shock (in aversive blocks). We compared PD patients when OFF dopaminergic medication with controls. Our key finding is that PD patients OFF dopaminergic medication move faster to avoid aversive outcomes (painful electric shocks) than to reap rewarding outcomes (winning money) and, unlike controls, do not speed up in the current trial having failed to win money in the previous one. We also demonstrate that sensitivity to distracting stimuli is valence specific.
We suggest this pattern of results can be explained in terms of low dopamine levels in the Parkinsonian state leading to an insensitivity to appetitive outcomes, and thus an inability to modulate movement speed in the face of rewards. By comparison, sensitivity to aversive stimuli is relatively spared. Our findings point to a rarely described property of bradykinesia in PD, namely its selective regulation by everyday outcomes.
Humans bargaining over money tend to reject unfair offers, whilst chimpanzees bargaining over primary rewards of food do not show this same motivation to reject. Whether such reciprocal fairness represents a predominantly human motivation has generated considerable recent interest. We induced either moderate or severe thirst in humans using intravenous saline, and examined responses to unfairness in an Ultimatum Game with water. We ask if humans also reject unfair offers for primary rewards. Despite the induction of even severe thirst, our subjects rejected unfair offers. Further, our data provide tentative evidence that this fairness motivation was traded-off against the value of the primary reward to the individual, a trade-off determined by the subjective value of water rather than by an objective physiological metric of value. Our data demonstrate humans care about fairness during bargaining with primary rewards, but that subjective self-interest may limit this fairness motivation.
Impulse control disorders are common in Parkinson's; disease, occurring in 13.6% of patients. Using a pharmacological manipulation and a novel risk taking task while performing functional magnetic resonance imaging, we investigated the relationship between dopamine agonists and risk taking in patients with Parkinson's; disease with and without impulse control disorders. During functional magnetic resonance imaging, subjects chose between two choices of equal expected value: a ‘Sure’ choice and a ‘Gamble’ choice of moderate risk. To commence each trial, in the ‘Gain’ condition, individuals started at $0 and in the ‘Loss’ condition individuals started at −$50 below the ‘Sure’ amount. The difference between the maximum and minimum outcomes from each gamble (i.e. range) was used as an index of risk (‘Gamble Risk’). Sixteen healthy volunteers were behaviourally tested. Fourteen impulse control disorder (problem gambling or compulsive shopping) and 14 matched Parkinson's; disease controls were tested ON and OFF dopamine agonists. Patients with impulse control disorder made more risky choices in the ‘Gain’ relative to the ‘Loss’ condition along with decreased orbitofrontal cortex and anterior cingulate activity, with the opposite observed in Parkinson's; disease controls. In patients with impulse control disorder, dopamine agonists were associated with enhanced sensitivity to risk along with decreased ventral striatal activity again with the opposite in Parkinson's; disease controls. Patients with impulse control disorder appear to have a bias towards risky choices independent of the effect of loss aversion. Dopamine agonists enhance sensitivity to risk in patients with impulse control disorder possibly by impairing risk evaluation in the striatum. Our results provide a potential explanation of why dopamine agonists may lead to an unconscious bias towards risk in susceptible individuals.
Parkinson's; disease; dopamine; gambling; decision making; risk
Risky choice entails a need to appraise all possible outcomes and integrate this information with individual risk preference. Risk is frequently quantified solely by statistical variance of outcomes, but here we provide evidence that individuals’ choice behaviour is sensitive to both dispersion (variance) and asymmetry (skewness) of outcomes. Using a novel behavioural paradigm in humans, we independently manipulated these ‘summary statistics’ while scanning subjects with fMRI. We show that a behavioural sensitivity to variance and skewness is mirrored in neuroanatomically dissociable representations of these quantities, with parietal cortex showing sensitivity to the former and prefrontal cortex and ventral striatum to the latter. Furthermore, integration of these objective risk metrics with subjective risk preference is expressed in a subject-specific coupling between neural activity and choice behaviour in anterior insula. Our findings show that risk is neither monolithic from a behavioural nor neural perspective and its decomposition is evident both in distinct behavioural preferences and in segregated underlying brain representations.
► Behavioural influence of variance and skewness (asymmetry) of risky decisions. ► Posterior parietal cortex is sensitive to variance. ► Distributed skewness representation in striatum, prefrontal and insular cortex. ► Integration of decision statistics and individual risk preferences in anterior insula.
Decision making; fMRI; Risk; Variance; Skewness
Perception of fairness can influence outcomes in human exchange. However, an inherent subjectivity in attribution renders it difficult to manipulate fairness experimentally. Here using a modified Ultimatum Game, within a varying social context, we induced a bias in human subjects’ acceptance of objectively identical offers. To explain this fairness-related behaviour we use a computational model to specify metrics for the objective and contextual aspects of fairness, testing for correlations between these model parameters and brain activity determined using fMRI. We show that objective social inequality, as defined by our model, is tracked in posterior insula cortex. Crucially, this inequality is integrated with social context in posterior and mid-insula, consistent with construction of a fairness motivation that flexibly adapted to the social environment. We suggest the dual importance of objective and contextual aspects to fairness we highlight might explain seemingly inconsistent societal phenomena, including public attitudes to income disparities.
Game theory; fMRI; fairness; Ultimatum Game; insula; prefrontal
Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling.
► Humans learn interdependence of multiple environmental outcomes ► FMRI activity in insula pertains to trial-by-trial estimate of outcome correlation ► Correlation estimate updated by prediction error-based learning mechanism ► Subjects are able to use correlation information to make risk optimal choices
The contribution of dopamine to working memory has been studied extensively [1–3]. Here, we exploited its well characterized effects [1–3] to validate a novel human in vivo assay of ongoing synaptic [4, 5] processing. We obtained magnetoencephalographic (MEG) measurements from subjects performing a working memory (WM) task during a within-subject, placebo-controlled, pharmacological (dopaminergic) challenge. By applying dynamic causal modeling (DCM), a Bayesian technique for neuronal system identification , to MEG signals from prefrontal cortex, we demonstrate that it is possible to infer synaptic signaling by specific ion channels in behaving humans. Dopamine-induced enhancement of WM performance was accompanied by significant changes in MEG signal power, and a DCM assay disclosed related changes in synaptic signaling. By estimating the contribution of ionotropic receptors (AMPA, NMDA, and GABAA) to the observed spectral response, we demonstrate changes in their function commensurate with the synaptic effects of dopamine. The validity of our model is reinforced by a striking quantitative effect on NMDA and AMPA receptor signaling that predicted behavioral improvement over subjects. Our results provide a proof-of-principle demonstration of a novel framework for inferring, noninvasively, neuromodulatory influences on ion channel signaling via specific ionotropic receptors, providing a window on the hidden synaptic events mediating discrete psychological processes in humans.
► We present a DCM capable of assaying neurotransmitter function during human cognition ► We demonstrate this using dopaminergic modulation of working memory and MEG ► We find changes in ionotropic receptors commensurate with dopaminergic enhancement ► We uncover quantitative effects that can predict individual behavioral improvements
Making the best choice when faced with a chain of decisions requires a person to judge both anticipated outcomes and future actions. Although economic decision-making models account for both risk and reward in single choice contexts there is a dearth of similar knowledge about sequential choice. Classical utility-based models assume that decision-makers select and follow an optimal pre-determined strategy, irrespective of the particular order in which options are presented. An alternative model involves continuously re-evaluating decision utilities, without prescribing a specific future set of choices. Here, using behavioral and functional magnetic resonance imaging (fMRI) data, we studied human subjects in a sequential choice task and use these data to compare alternative decision models of valuation and strategy selection. We provide evidence that subjects adopt a model of re-evaluating decision utilities, where available strategies are continuously updated and combined in assessing action values. We validate this model by using simultaneously-acquired fMRI data to show that sequential choice evokes a pattern of neural response consistent with a tracking of anticipated distribution of future reward, as expected in such a model. Thus, brain activity evoked at each decision point reflects the expected mean, variance and skewness of possible payoffs, consistent with the idea that sequential choice evokes a prospective evaluation of both available strategies and possible outcomes.
Decision-making; strategy; fMRI; risk; dynamic programming; skewness
Animals' attitudes to risk are profoundly influenced by metabolic state (hunger and baseline energy stores). Specifically, animals often express a preference for risky (more variable) food sources when below a metabolic reference point (hungry), and safe (less variable) food sources when sated. Circulating hormones report the status of energy reserves and acute nutrient intake to widespread targets in the central nervous system that regulate feeding behaviour, including brain regions strongly implicated in risk and reward based decision-making in humans. Despite this, physiological influences per se have not been considered previously to influence economic decisions in humans. We hypothesised that baseline metabolic reserves and alterations in metabolic state would systematically modulate decision-making and financial risk-taking in humans.
We used a controlled feeding manipulation and assayed decision-making preferences across different metabolic states following a meal. To elicit risk-preference, we presented a sequence of 200 paired lotteries, subjects' task being to select their preferred option from each pair. We also measured prandial suppression of circulating acyl-ghrelin (a centrally-acting orexigenic hormone signalling acute nutrient intake), and circulating leptin levels (providing an assay of energy reserves). We show both immediate and delayed effects on risky decision-making following a meal, and that these changes correlate with an individual's baseline leptin and changes in acyl-ghrelin levels respectively.
We show that human risk preferences are exquisitely sensitive to current metabolic state, in a direction consistent with ecological models of feeding behaviour but not predicted by normative economic theory. These substantive effects of state changes on economic decisions perhaps reflect shared evolutionarily conserved neurobiological mechanisms. We suggest that this sensitivity in human risk-preference to current metabolic state has significant implications for both real-world economic transactions and for aberrant decision-making in eating disorders and obesity.