Behavioral changes driven by reinforcement and punishment are referred to as simple or model-free reinforcement learning. Animals can also change their behaviors by observing events that are neither appetitive nor aversive, when these events provide new information about payoffs available from alternative actions. This is an example of model-based reinforcement learning, and can be accomplished by incorporating hypothetical reward signals into the value functions for specific actions. Recent neuroimaging and single-neuron recording studies showed that the prefrontal cortex and the striatum are involved not only in reinforcement and punishment, but also in model-based reinforcement learning. We found evidence for both types of learning, and hence hybrid learning, in monkeys during simulated competitive games. In addition, in both the dorsolateral prefrontal cortex and orbitofrontal cortex, individual neurons heterogeneously encoded signals related to actual and hypothetical outcomes from specific actions, suggesting that both areas might contribute to hybrid learning.
belief learning; decision making; game theory; reinforcement learning; reward
The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.
A study of single neurons in the ventral striatum reveals signatures of value comparison and selection during a risky choice task, suggesting that the cortex and its striatal targets make overlapping contributions to the choice process. Read the accompanying Primer.
The neural calculations underlying reward-based choice are closely associated with a network of brain areas including the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC). Most theories ascribe distinct roles to these two structures during choice, but these differences have yet to be confirmed at the level of single neurons. We compared responses of VS neurons to those of vmPFC neurons recorded in rhesus macaques choosing between potential gambles for water rewards. We found widespread similarities in the way that VS and vmPFC neurons fire during the choice process. Neurons in both areas encoded the value of the offered gamble, the difference in value between offered gambles, and the gamble outcome. Additionally, both areas showed stronger coding for the chosen gamble than for the unchosen one and predicted choice even when we controlled for offer value. Interestingly, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, similarities between vmPFC and VS suggest that cortex and its striatal targets make overlapping contributions to reward-based choice.
Game theory analyses optimal strategies for multiple decision makers interacting in a social group. However, the behaviours of individual humans and animals often deviate systematically from the optimal strategies described by game theory. The behaviours of rhesus monkeys (Macaca mulatta) in simple zero-sum games showed similar patterns, but their departures from the optimal strategies were well accounted for by a simple reinforcement-learning algorithm. During a computer-simulated zero-sum game, neurons in the dorsolateral prefrontal cortex often encoded the previous choices of the animal and its opponent as well as the animal's reward history. By contrast, the neurons in the anterior cingulate cortex predominantly encoded the animal's reward history. Using simple competitive games, therefore, we have demonstrated functional specialization between different areas of the primate frontal cortex involved in outcome monitoring and action selection. Temporally extended signals related to the animal's previous choices might facilitate the association between choices and their delayed outcomes, whereas information about the choices of the opponent might be used to estimate the reward expected from a particular action. Finally, signals related to the reward history might be used to monitor the overall success of the animal's current decision-making strategy.
prefrontal cortex; decision making; reward
The value of an object acquired by a particular action often determines the motivation to produce that action. Previous studies found neural signals related to the values of different objects or goods in the orbitofrontal cortex, while the values of outcomes expected from different actions are broadly represented in multiple brain areas implicated in movement planning. However, how the brain combines the values associated with various objects and the information about their locations is not known. In this study, we tested whether the neurons in the dorsolateral prefrontal cortex (DLPFC) and striatum in rhesus monkeys might contribute to translating the value signals between multiple frames of reference. Monkeys were trained to perform an oculomotor intertemporal choice in which the color of a saccade target and the number of its surrounding dots signaled the magnitude of reward and its delay, respectively. In both DLPFC and striatum, temporally discounted values (DVs) associated with specific target colors and locations were encoded by partially overlapping populations of neurons. In the DLPFC, the information about reward delays and DVs of rewards available from specific target locations emerged earlier than the corresponding signals for target colors. Similar results were reproduced by a simple network model built to compute DVs of rewards in different locations. Therefore, DLPFC might play an important role in estimating the values of different actions by combining the previously learned values of objects and their present locations.
intertemporal choice; prefrontal cortex; reward; temporal discounting; utility
The primate prefrontal cortex contributes to stimulus-guided behavior, but the functional specializations among its areas remain uncertain. To better understand such specializations, we contrasted neuronal activity in the dorsolateral prefrontal cortex (PFdl) and the orbital prefrontal cortex (PFo). The task required rhesus monkeys to use a visual cue to choose a saccade target. Some cues instructed the monkeys to repeat their most recent response; others instructed them to change it. Responses were followed by feedback: fluid reward if correct, visual feedback if incorrect. Previous studies, using different tasks, have reported that PFo neurons did not encode responses. We found PFo did encode responses in this task, but only near feedback time, after the response had been completed. PFdl differed from PFo in several respects. As reported previously, some PFdl neurons encoded responses from the previous trial and others encoded planned responses. PFo neurons did not have these properties. After feedback, PFdl encoded rewarded responses better than unrewarded ones and thus combined response and outcome information. PFo, in contrast, encoded the responses chosen, rewarded or not. These findings suggest that PFdl and PFo contribute differently to response knowledge, with PFo using an outcome-independent signal to monitor current responses at feedback time.
Decision; feedback; monitoring; evaluation; frontal lobe; prefrontal cortex
Human behaviors can be more powerfully influenced by conditioned reinforcers, such as money, than by primary reinforcers. Moreover, people often change their behaviors to avoid monetary losses. However, the effect of removing conditioned reinforcers on choices has not been explored in animals, and the neural mechanisms mediating the behavioral effects of gains and losses are not well understood. To investigate the behavioral and neural effects of gaining and losing a conditioned reinforcer, we trained rhesus monkeys for a matching pennies task in which the positive and negative values of its payoff matrix were realized by the delivery and removal of a conditioned reinforcer. Consistent with the findings previously obtained with non-negative payoffs and primary rewards, the animal’s choice behavior during this task was nearly optimal. Nevertheless, the gain and loss of a conditioned reinforcer significantly increased and decreased, respectively, the tendency for the animal to choose the same target in subsequent trials. We also found that the neurons in the dorsomedial frontal cortex, dorsal anterior cingulate cortex, and dorsolateral prefrontal cortex often changed their activity according to whether the animal earned or lost a conditioned reinforcer in the current or previous trial. Moreover, many neurons in the dorsomedial frontal cortex also signaled the gain or loss occurring as a result of choosing a particular action as well as changes in the animal’s behaviors resulting from such gains or losses. Thus, primate medial frontal cortex might mediate the behavioral effects of conditioned reinforcers and their losses.
cingulate cortex; decision making; prefrontal cortex; reinforcement learning; reward; punishment; neuroeconomics
Previous work indicates that economic decisions can be made independently of the visuo-motor contingencies of the choice task (space of goods). However, the neuronal mechanisms through which the choice outcome (the chosen good) is transformed into a suitable action plan remain poorly understood. Here we show that neurons in lateral prefrontal cortex reflect the early stages of this good-to-action transformation. Monkeys chose between different juices. The experimental design dissociated in space and time the presentation of the offers and the saccade targets associated with them. We recorded from the orbital, ventrolateral and dorsolateral prefrontal cortices (OFC, LPFCv and LPFCd, respectively). Prior to target presentation, neurons in both LPFCv and LPFCd encoded the choice outcome in goods space. After target presentation, they gradually came to encode the location of the targets and the upcoming action plan. Consistent with the anatomical connectivity, all spatial and action-related signals emerged in LPFCv before LPFCd.
Humans and animals often must choose between rewards that differ in their qualities, magnitudes, immediacy, and likelihood, and must estimate these multiple reward parameters from their experience. However, the neural basis for such complex decision making is not well understood. To understand the role of the primate prefrontal cortex in determining the subjective value of delayed or uncertain reward, we examined the activity of individual prefrontal neurons during an inter-temporal choice task and a computer-simulated competitive game. Consistent with the findings from previous studies in humans and other animals, the monkey’s behaviors during inter-temporal choice were well accounted for by a hyperbolic discount function. In addition, the activity of many neurons in the lateral prefrontal cortex reflected the signals related to the magnitude and delay of the reward expected from a particular action, and often encoded the difference in temporally discounted values that predicted the animal’s choice. During a computerized matching pennies game, the animals approximated the optimal strategy, known as Nash equilibrium, using a reinforcement learning algorithm. We also found that many neurons in the lateral prefrontal cortex conveyed the signals related to the animal’s previous choices and their outcomes, suggesting that this cortical area might play an important role in forming associations between actions and their outcomes. These results show that the primate lateral prefrontal cortex plays a central role in estimating the values of alternative actions based on multiple sources of information.
game theory; inter-temporal choice; reinforcement learning; utility theory; temporal discounting
Our choices often require appropriate actions in order to obtain a preferred outcome, but the neural underpinnings that link decision making and action selection remain largely undetermined. Recent theories propose that action selection occurs simultaneously, i.e. parallel in time, with the decision process. Specifically, it is thought that action selection in motor regions originates from a competitive process which is gradually biased by evidence signals originating in other regions, such as those specialized in value computations. Biases reflecting the evaluation of choice options should thus emerge in the motor system before the decision process is complete. Using transcranial magnetic stimulation, we sought direct physiological evidence for this prediction by measuring changes in cortico-spinal excitability in human motor cortex during value-based decisions. We found that excitability for chosen versus unchosen actions distinguishes the forthcoming choice before completion of the decision process. Both excitability and reaction times varied as a function of the subjective value-difference between chosen and unchosen actions, consistent with this effect being value-driven. This relationship was not observed in the absence of a decision. Our data provide novel evidence in humans that internally generated value-based decisions influence the competition between action representations in motor cortex before the decision process is complete. This is incompatible with models of serial processing of stimulus, decision, and action.
Functional impairment of the orbital and medial prefrontal cortex underlies deficits in executive control that characterize addictive disorders, including alcohol addiction. Previous studies indicate that alcohol alters glutamate neurotransmission and one substrate of these effects may be through the reconfiguration of the subunits constituting ionotropic glutamate receptor (iGluR) complexes. Glutamatergic transmission is integral to cortico-cortical and cortico-subcortical communication, and alcohol-induced changes in the abundance of the receptor subunits and/or their splice variants may result in critical functional impairments of prefrontal cortex in the alcohol-addicted state.
Methods and results
The effects of chronic ethanol self-administration on glutamate receptor ionotropic NMDA (GRIN), as well as GRIN1 splice variant mRNA expression was studied in the orbitofrontal cortex (OFC; Area 13), dorsolateral prefrontal cortex (DLPFC; Area 46) and anterior cingulate cortex (ACC; Area 24) of male cynomolgus monkeys. Chronic ethanol self-administration resulted in significant changes in the expression of NMDA subunit mRNA expression in the DLPFC and OFC, but not the ACC. In DLPFC, the overall expression of NMDA subunits was significantly decreased in ethanol treated monkeys. Slight but significant changes were observed for synaptic associated protein 102 kD (SAP102) and neuronal nitric oxide synthase (nNOS) mRNAs. In OFC, the NMDAR1 variant GRIN1-1 was reduced while GRIN1-2 was increased. Furthermore, no significant changes in GFAP protein levels were observed in either the DLPFC or OFC.
Results from these studies provide the first demonstration of post-transcriptional regulation of iGluR subunits in the primate brain following long-term ethanol self-administration. Furthermore, changes in these transcripts do not appear to reflect changes in glial activation or loss. Further studies examining the expression and cellular localization of subunit proteins and receptor pharmacology would shed more light on the findings reported here.
Ethanol; Glutamate; messenger RNA; Prefrontal Cortex; qPCR; Primate
We assessed the involvement of the orbital prefrontal cortex (PFo), the prelimbic region of the medial prefrontal cortex (PL), and the amygdala in goal-directed behavior. Rhesus monkeys were trained on a task in which two different instrumental responses were linked to two different outcomes. One response, called ‘Tap’, required the monkeys to repeatedly touch a colored square on a video monitor, to produce one kind of food reward. The other response, called ‘Hold’, required persistent contact of an identical stimulus, and it produced a different kind of food reward. Following training, we assessed the effects of satiety-specific reinforcer devaluation as a way to probe each monkey’s use of goal-directed behavior. In this procedure, monkeys were allowed to consume one of the two foods to satiety, and were then tested for Tap/Hold preference under extinction. Unoperated control monkeys showed a reduction in the response associated with obtaining the devalued food, called the devaluation effect, a hallmark of goal-directed behavior. Monkeys with bilateral lesions of PFo or the amygdala exhibited significantly reduced devaluation effects. Results from monkeys with PL lesions were equivocal. We conclude that both PFo and the amygdala play a significant role in goal-directed behavior in monkeys. Notably, the findings for PFo challenge the idea that orbital and medial prefrontal regions are exclusively dedicated to object- and action-based processes, respectively.
Studies in monkeys show clear anatomical and functional distinctions among networks connecting with subregions within the prefrontal cortex. Three such networks are centered on lateral orbitofrontal cortex, medial frontal and cingulate cortex, and lateral prefrontal cortex and all have been identified with distinct cognitive roles. Although these areas differ in a number of their cortical connections, some of the first anatomical evidence for these networks came from tracer studies demonstrating their distinct patterns of connectivity with the mediodorsal (MD) nucleus of the thalamus. Here, we present evidence for a similar topography of MD thalamus prefrontal connections, using non-invasive imaging and diffusion tractography (DWI–DT) in human and macaque. DWI–DT suggested that there was a high probability of interconnection between medial MD and lateral orbitofrontal cortex, between caudodorsal MD and medial frontal/cingulate cortex, and between lateral MD and lateral prefrontal cortex, in both species. Within the lateral prefrontal cortex a dorsolateral region (the principal sulcus in the macaque and middle frontal gyrus in the human) was found to have a high probability of interconnection with the MD region between the regions with a high probability of interconnection with other parts of the lateral prefrontal cortex and with the lateral orbitofrontal cortex. In addition to suggesting that the thalamic connectivity in the macaque is a good guide to human prefrontal cortex, and therefore that there are likely to be similarities in the cognitive roles played by the prefrontal areas in both species, the present results are also the first to provide insight into the topography of projections of an individual thalamic nucleus in the human brain.
Anatomy; DTI; Human; Macaque; Thalamus
Studies investigating response reversal consistently implicate regions of medial and lateral prefrontal cortex when reinforcement contingencies change. However, it is unclear from these studies how these regions give rise to the individual components of response reversal, such as reinforcement value encoding, response inhibition, and response change. Here we report a novel instrumental learning task designed to determine whether regions implicated in processing reversal errors are uniquely involved in this process, or whether they play a more general role in representing response competition, reinforcement value, or punishment value in the absence of demands for response change. In line with previous findings, reversal errors activated orbitofrontal cortex, dorsomedial prefrontal cortex, ventrolateral prefrontal cortex, caudate, and dorsolateral prefrontal cortex. These regions also showed increased activity to errors in the absence of contingency changes. In addition, ventrolateral PFC, caudate, and dorsolateral PFC each exhibited increased activity following correct reversals. Activity in these regions was not significantly modulated by changes in reinforcement value that were not sufficient to make an alternative response advantageous. These data do not support punishment-processing or prepotent reponse inhibition accounts of ventrolateral prefrontal cortex function. Instead, they support recent conceptualizations of ventrolateral prefrontal cortex function that implicate this region in resolving response competition by manipulating the representation of either motor response options, or object features. These data also suggest that dorsolateral prefrontal cortex plays a role in reversal learning, probably through top down attentional control of object or reinforcement features when task demands increase.
Response reversal; affective shift; response competition; ventrolateral prefrontal cortex; decision-making
Motivational interventions to improve health behaviors based on conventional cognitive and behavioral theories have been extensively studied; however, advances in neuroimaging technology make it possible to assess the neurophysiological basis of health behaviors, such as physical activity. The goals of this approach are to support new interventions to achieve optimal outcomes.
This study used functional magnetic resonance imaging (fMRI) to assess differences in brain responses in healthy weight to obese midlife women during a goal-directed decision task.
Thirty nondiabetic, midlife (age 47–55 years) women with body mass index (BMI) ranging from 18.5 to 40 kg/m2 were recruited. A descriptive, correlational design was used to assess the relationship between brain activations and weight status. Participants underwent a goal-directed behavior task in the fMRI scanner consisting of a learning and implementation phase. The task was designed to assess both goal-directed and habitual behaviors. One participant was omitted from the analysis because of excessive motion (>4 mm), and six were omitted because of fewer than 50% correct responses on the exit survey. Four participants developed claustrophobia in the scanner and were disqualified from further participation. The remaining 19 participants were included in the final analysis.
Brain responses while participants learned goal-directed behavior showed a positive correlation with BMI in the dorsomedial prefrontal cortex (dmPFC) and a negative correlation with BMI in the insula. During the implementation of goal-directed behavior, brain responses in the dorsolateral prefrontal cortex (dlPFC) negatively correlated with BMI.
These results indicate that overweight women activate regions associated with cognitive control to a greater degree than healthy weight women during goal-directed learning. The brain regions activated (dmPFC, dlPFC, insula) are associated with cognitive control and self-regulation. On the other hand, healthy weight women activate regions associated with emotion processing, planning, and self-regulation (lateral orbitofrontal cortex, anterior insula) to a greater degree than overweight women during goal-directed learning and implementation of goal-directed behavior. Overweight women activate cognitive control regions while learning associations between actions and outcomes; however, this is not the case during the implementation phase—which may make it more difficult to transform goals into action (e.g., maintain physical activity over time). Overall, these results indicate that overweight midlife women respond differently during learning and implementation of actions that lead to positive outcomes during a general test of goal-directed behavior. Future study is needed to assess the transfer of goal-directed and habitual behavior to specific aspects of energy balance to improve health outcomes.
fMRI; health behavior; neuroimaging; neurophysiology; obesity; women’s health
Mentalizing is defined as the inference of mental states of fellow humans, and is a particularly important skill for social interactions. Here we assessed whether activity in brain areas involved in mentalizing is specific to the processing of mental states or can be generalized to the inference of non-mental states by comparing brain responses during the interaction with an intentional and an artificial agent. Participants were scanned using fMRI during interactive rock-paper-scissors games while believing their opponent was a fellow human (Intentional agent, Int), a humanoid robot endowed with an artificial intelligence (Artificial agent, Art), or a computer playing randomly (Random agent, Rnd). Participants' subjective reports indicated that they adopted different stances against the three agents. The contrast of brain activity during interaction with the artificial and the random agents didn't yield any cluster at the threshold used, suggesting the absence of a reproducible stance when interacting with an artificial intelligence. We probed response to the artificial agent in regions of interest corresponding to clusters found in the contrast between the intentional and the random agents. In the precuneus involved in working memory, the posterior intraparietal suclus, in the control of attention and the dorsolateral prefrontal cortex, in executive functions, brain activity for Art was larger than for Rnd but lower than for Int, supporting the intrinsically engaging nature of social interactions. A similar pattern in the left premotor cortex and anterior intraparietal sulcus involved in motor resonance suggested that participants simulated human, and to a lesser extend humanoid robot actions, when playing the game. Finally, mentalizing regions, the medial prefrontal cortex and right temporoparietal junction, responded to the human only, supporting the specificity of mentalizing areas for interactions with intentional agents.
social cognition; neuroscience; artificial intelligence; fMRI
Recent directions in the treatment of ADHD have involved both a broadening of pharmacological perspectives to include nor-adrenergic as well as dopaminergic agents. A review of animal and human studies of pharmacological and therapeutic directions in ADHD suggests that the D1 receptor is a specific site for dopaminergic regulation of the PFC, but optimal levels of dopamine (DA) are required for beneficial effects on working memory. Animal and human studies indicate that the alpha-2A receptor is also important for prefrontal regulation, leaving open the question of the relative importance of these receptor sites. The therapeutic effects of ADHD medications in the prefrontal cortex have focused attention on the development of working memory capacity in ADHD.
The actions of dopaminergic vs noradrenergic agents, currently available for the treatment of ADHD have overlapping, but different actions in the prefrontal cortex (PFC) and subcortical centers. While stimulants act on D1 receptors in the dorsolateral prefrontal cortex, they also have effects on D2 receptors in the corpus striatum and may also have serotonergic effects at orbitofrontal areas. At therapeutic levels, dopamine (DA) stimulation (through DAT transporter inhibition) decreases noise level acting on subcortical D2 receptors, while NE stimulation (through alpha-2A agonists) increases signal by acting preferentially in the PFC possibly on DAD1 receptors. On the other hand, alpha-2A noradrenergic transmission is more limited to the prefrontal cortex (PFC), and thus less likely to have motor or stereotypic side effects, while alpha-2B and alpha-2C agonists may have wider cortical effects. The data suggest a possible hierarchy of specificity in the current medications used in the treatment of ADHD, with guanfacine likely to be most specific for the treatment of prefrontal attentional and working memory deficits. Stimulants may have broader effects on both vigilance and motor impulsivity, depending on dose levels, while atomoxetine may have effects on attention, anxiety, social affect, and sedation via noradrenergic transmission.
Tests of the hypothesis
At a theoretical level, the advent of possible specific alpha-2A noradrenergic therapies has posed the question of the role of working memory in ADHD. Head to head comparisons of stimulant and noradrenergic alpha-2A, alpha-2B and alpha-2C agonists, utilizing vigilance and affective measures should help to clarify pharmacological and therapeutic differences.
Chronic cocaine administration regulates the expression of several proteins related to dopaminergic signaling and synaptic function in the mesocorticolimbic pathway, including the prefrontal cortex. Functional abnormalities in the prefrontal cortex are hypothesized to be due in part to the expression of proteins involved in dopamine signaling and plasticity. Adult male rhesus monkeys self-administered cocaine (i.v.) under limited (n = 4) and extended access conditions (n = 6). The abundance of surrogate markers of dopamine signaling and plasticity in the dorsolateral prefrontal cortex (DLPFC), orbitofrontal cortex (OFC), and anterior cingulate cortex (ACC) were examined: glycosylated and non-glycosylated forms of the dopamine transporter (efficiency of dopamine transport), tyrosine hydroxylase (TH; marker of dopamine synthesis) and phosphorylated TH at Serine 30 and 40 (markers of enzyme activity), extracellular signal-regulated kinase 1 and 2 (ERK1 and ERK 2), and phosphorylated ERK1 and ERK2 (phosphorylates TH Serine 31; markers of synaptic plasticity), and markers of synaptic integrity, spinophilin and post-synaptic density protein 95 (roles in dopamine signaling and response to cocaine). Extended cocaine access increased non-glycosylated and glycosylated DAT in DLPFC and OFC. While no differences in TH expression were observed between groups for any of the regions, extended access induced significant elevations in pTHSer31 in all regions. In addition, a slight but significant reduction in phosphorylated pTHSer40 was found in the DLPFC. Phosphorylated ERK2 was increased in all regions; however, pERK1 was decreased in ACC and OFC but increased in DLPFC. PSD-95 was increased in the OFC but not in DLPFC or ACC. Furthermore, extended cocaine self-administration elicited significant increases in spinophilin protein expression in all regions. Results from the study provide insight into the biochemical alterations occurring in primate prefrontal cortex.
cocaine; dopamine; orbitofrontal cortex; anterior cingulate cortex; dorsolateral prefrontal cortex; rhesus monkey
Functional impairment of the orbital and medial prefrontal cortex underlies deficits in executive control that characterize addictive disorders, including alcohol addiction. Previous studies indicate that alcohol alters glutamate neurotransmission and one substrate of these effects may be through the reconfiguration of the subunits constituting ionotropic glutamate receptor (iGluR) complexes. Glutamatergic transmission is integral to cortico-cortical and cortico-subcortical communication and alcohol-induced changes in the abundance of the receptor subunits and/or their splice variants may result in critical functional impairments of prefrontal cortex in alcohol dependence. To this end, the effects of chronic ethanol self-administration on glutamate receptor ionotropic AMPA (GRIA) subunit variant and kainate (GRIK) subunit mRNA expression were studied in the orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DLPFC), and anterior cingulate cortex (ACC) of male cynomolgus monkeys. In DLPFC, total AMPA splice variant expression and total kainate receptor subunit expression were significantly decreased in alcohol drinking monkeys. Expression levels of GRIA3 flip and flop and GRIA4 flop mRNAs in this region were positively correlated with daily ethanol intake and blood ethanol concentrations (BEC) averaged over the 6 months prior to necropsy. In OFC, AMPA subunit splice variant expression was reduced in the alcohol treated group. GRIA2 flop mRNA levels in this region were positively correlated with daily ethanol intake and BEC averaged over the 6 months prior to necropsy. Results from these studies provide further evidence of transcriptional regulation of iGluR subunits in the primate brain following chronic alcohol self-administration. Additional studies examining the cellular localization of such effects in the framework of primate prefrontal cortical circuitry are warranted.
ethanol; AMPA; kainate; messenger RNA; prefrontal cortex; qPCR; primate
Social subordination in female macaques represents a well-described model of chronic psychosocial stress. Additionally, a length polymorphism (5HTTLPR) in the regulatory region of the serotonin (5HT) transporter (5HTT) gene (SLC6A4) is present in rhesus macaques, which has been linked to adverse outcomes similar to what has been described in humans with an analogous 5HTTLPR polymorphism. The present study determined the effects of social status and the 5HTTLPR genotype on 5HT1A receptor binding potential (5HT1A BPND) in brain regions implicated in emotional regulation and stress reactivity in ovariectomised female monkeys, and then assessed how these effects were altered by 17β-oestradiol (E2) treatment. Areas analyzed included the prefrontal cortex [anterior cingulate (ACC); medial prefrontal cortex (mPFC); dorsolateral prefrontal cortex; orbitofrontal prefrontal cortex], amygdala, hippocampus, hypothalamus and raphe nucleui. Positron emission tomography (PET) using p-[18F]MPPF was performed to determine the levels of 5HT1A BPND under a non-E2 and a 3-wk E2 treatment condition. The short variant (s-variant) 5HTTLPR genotype produced a significant reduction in 5HT1A BPND in the mPFC regardless of social status, and subordinate s-variant females showed a reduction in 5HT1A BPND within the ACC. Both these effects of 5HTTLPR were unaffected by E2. Additionally, E2 reduced 5HT1A BPND in the dorsal raphe of all females irrespective of psychosocial stress or 5HTTLPR genotype. Hippocampal 5HT1A BPND was attenuated in subordinate females regardless of 5HTTLPR genotype during the non-E2 condition, an effect that was normalised with E2. Similarly, 5HT1A BPND in the hypothalamus was significantly lower in subordinate females regardless of 5HTTLPR genotype, an effect reversed with E2. Together, the data indicate that the effect of E2 on modulation of central 5HT1A BPND may only occur in brain regions that show no 5HTTLPR genotype-linked control of 5HT1A binding.
oestradiol; social subordination; psychosocial stress; 5HT1A receptor; 5HTTLPR; monkeys
Prefrontal cortex mediates cognitive control by means of circuitry organized along dorso-ventral and rostro-caudal axes. Along the dorso-ventral axis, ventrolateral PFC controls semantic information, whereas dorsolateral PFC encodes task rules. Along the rostro-caudal axis, anterior prefrontal cortex encodes complex rules and relationships between stimuli, whereas posterior prefrontal cortex encodes simple relationships between stimuli and behavior. Evidence of these gradients of prefrontal cortex organization has been well documented in fMRI studies, but their functional correlates have not been examined with regard to integrity of underlying white matter tracts. We hypothesized that (a) the integrity of specific white matter tracts is related to cognitive functioning in a manner consistent with the dorso-ventral and rostro-caudal organization of the prefrontal cortex, and (b) this would be particularly evident in healthy older adults. We assessed three cognitive processes that recruit the prefrontal cortex and can distinguish white matter tracts along the dorso-ventral and rostro-caudal dimensions –episodic memory, working memory, and reasoning. Correlations between cognition and fractional anisotropy as well as fiber tractography revealed: (a) Episodic memory was related to ventral prefrontal cortex-thalamo-hippocampal fiber integrity; (b) Working memory was related to integrity of corpus callosum body fibers subserving dorsolateral prefrontal cortex; and (c) Reasoning was related to integrity of corpus callosum body fibers subserving rostral and caudal dorsolateral prefrontal cortex. These findings confirm the ventrolateral prefrontal cortex's role in semantic control and the dorsolateral prefrontal cortex's role in rule-based processing, in accordance with the dorso-ventral prefrontal cortex gradient. Reasoning-related rostral and caudal superior frontal white matter may facilitate different levels of task rule complexity. This study is the first to demonstrate dorso-ventral and rostro-caudal prefrontal cortex processing gradients in white matter integrity.
It is generally assumed that choice between different actions reflects the difference between their action values yet little direct evidence confirming this assumption has been reported. Here we assess whether the brain calculates the absolute difference between action values or their relative advantage, that is, the probability that one action is better than the other alternatives. We use a two-armed bandit task during functional magnetic resonance imaging and modelled responses to determine both the size of the difference between action values (D) and the probability that one action value is better (P). The results show haemodynamic signals corresponding to P in right dorsolateral prefrontal cortex (dlPFC) together with evidence that these signals modulate motor cortex activity in an action-specific manner. We find no significant activity related to D. These findings demonstrate that a distinct neuronal population mediates action-value comparisons, and reveals how these comparisons are implemented to mediate value-based decision-making.
In humans, choice between actions depends on the ability to compare action–outcome values. Here, the authors show that action–outcome values are compared on the basis of the relative advantage of a particular action over alternative actions, which takes place in the right dorsolateral prefrontal cortex of the brain.
Monkeys adjust their behavior in response to outcomes that they have observed but not directly experienced, and single neurons within the anterior cingulate cortex respond to these fictive rewards they same way they respond to experienced rewards.
The neural mechanisms supporting the ability to recognize and respond to fictive outcomes, outcomes of actions that one has not taken, remain obscure. We hypothesized that neurons in anterior cingulate cortex (ACC), which monitors the consequences of actions and mediates subsequent changes in behavior, would respond to fictive reward information. We recorded responses of single neurons during performance of a choice task that provided information about the reward values of unchosen options. We found that ACC neurons signal fictive reward information, and use a coding scheme similar to that used to signal experienced outcomes. Thus, individual ACC neurons process both experienced and fictive rewards.
We compared neuronal activity in the dorsolateral (PFdl), orbital (PFo) and polar (PFp) prefrontal cortex as monkeys performed three tasks. In two tasks, a cue instructed one of two strategies: stay with the previous response or shift to the alternative. Visual stimuli served as cues in one of these tasks; in the other, fluid rewards did so. In the third task, visuospatial cues instructed each response. A delay period followed each cue. As reported previously, PFdl encoded strategies (stay or shift) and responses (left or right) during the cue and delay periods, while PFo encoded strategies and PFp encoded neither strategies nor responses; during the feedback period, all three areas encoded responses, not strategies. Four novel findings emerged from the present analysis. (1) The strategy encoded by PFdl and PFo cells during the cue and delay periods was modality specific. (2) The response encoded by PFdl cells was task- and modality specific during the cue period, but during the delay and feedback periods it became task- and modality general. (3) Although some PFdl and PFo cells responded to or anticipated rewards, we could rule out reward effects for most strategy-and response-related activity. (4) Immediately before feedback, only PFp signaled responses that were correct according to the cued strategy; after feedback, only PFo signaled the response that had been made, whether correct or incorrect. These signals support a role in generating responses by PFdl, assigning outcomes to choices by PFo, and assigning outcomes to cognitive processes by PFp.
frontal pole; orbitofrontal cortex; periprincipal cortex; frontal lobe; rules; decisions; choices
Action is often executed according to information provided by a visual signal. As this type of behavior integrates two distinct neural representations, perception and action, it has been thought that identification of the neural mechanisms underlying this process will yield deeper insights into the principles underpinning goal-directed behavior. Based on a framework derived from conditional visuomotor association, prior studies have identified neural mechanisms in the dorsal premotor cortex (PMd), dorsolateral prefrontal cortex (dlPFC), ventrolateral prefrontal cortex (vlPFC), and basal ganglia (BG). However, applications resting solely on this conceptualization encounter problems related to generalization and flexibility, essential processes in executive function, because the association mode involves a direct one-to-one mapping of each visual signal onto a particular action. To overcome this problem, we extend this conceptualization and postulate a more general framework, conditional visuo-goal association. According to this new framework, the visual signal identifies an abstract behavioral goal, and an action is subsequently selected and executed to meet this goal. Neuronal activity recorded from the four key areas of the brains of monkeys performing a task involving conditional visuo-goal association revealed three major mechanisms underlying this process. First, visual-object signals are represented primarily in the vlPFC and BG. Second, all four areas are involved in initially determining the goals based on the visual signals, with the PMd and dlPFC playing major roles in maintaining the salience of the goals. Third, the cortical areas play major roles in specifying action, whereas the role of the BG in this process is restrictive. These new lines of evidence reveal that the four areas involved in conditional visuomotor association contribute to goal-directed behavior mediated by conditional visuo-goal association in an area-dependent manner.
sensorimotor integration; visuomotor integration; goal; action; globus pallidus; executive function
Activity of the neurons in the lateral intra-parietal cortex or LIP displays a mixture of sensory, motor, and memory signals. Moreover, they often encode signals reflecting the accumulation of sensory evidence that certain eye movements might lead to a desirable outcome. However, when the environment changes dynamically, animals are also required to combine the information about its previously chosen actions and their outcomes appropriately to update continually the desirabilities of alternative actions. Here, we investigated whether LIP neurons encoded signals necessary to update an animal’s decision making strategies adaptively during a computer-simulated matching pennies game. Using a reinforcement learning algorithm, we estimated the value functions that best predicted the animal’s choices on a trial-by-trial basis. We found that immediately before the animal revealed its choice, approximately 18% of LIP neurons changed their activity according to the difference in the value functions for the two targets. In addition, a somewhat higher fraction of LIP neurons displayed signals related to the sum of the value function, which might correspond to the state value function or an average rate of reward used as a reference point. Similar to the neurons in the prefrontal cortex, many LIP neurons also encoded the signals related to the animal’s previous choices. Thus, the posterior parietal cortex might be a part of the network that provides the substrate for forming appropriate associations between actions and outcomes.
decision; parietal; reward; feedback; control; cognition