Behavioral changes driven by reinforcement and punishment are referred to as simple or model-free reinforcement learning. Animals can also change their behaviors by observing events that are neither appetitive nor aversive, when these events provide new information about payoffs available from alternative actions. This is an example of model-based reinforcement learning, and can be accomplished by incorporating hypothetical reward signals into the value functions for specific actions. Recent neuroimaging and single-neuron recording studies showed that the prefrontal cortex and the striatum are involved not only in reinforcement and punishment, but also in model-based reinforcement learning. We found evidence for both types of learning, and hence hybrid learning, in monkeys during simulated competitive games. In addition, in both the dorsolateral prefrontal cortex and orbitofrontal cortex, individual neurons heterogeneously encoded signals related to actual and hypothetical outcomes from specific actions, suggesting that both areas might contribute to hybrid learning.
belief learning; decision making; game theory; reinforcement learning; reward
Game theory analyses optimal strategies for multiple decision makers interacting in a social group. However, the behaviours of individual humans and animals often deviate systematically from the optimal strategies described by game theory. The behaviours of rhesus monkeys (Macaca mulatta) in simple zero-sum games showed similar patterns, but their departures from the optimal strategies were well accounted for by a simple reinforcement-learning algorithm. During a computer-simulated zero-sum game, neurons in the dorsolateral prefrontal cortex often encoded the previous choices of the animal and its opponent as well as the animal's reward history. By contrast, the neurons in the anterior cingulate cortex predominantly encoded the animal's reward history. Using simple competitive games, therefore, we have demonstrated functional specialization between different areas of the primate frontal cortex involved in outcome monitoring and action selection. Temporally extended signals related to the animal's previous choices might facilitate the association between choices and their delayed outcomes, whereas information about the choices of the opponent might be used to estimate the reward expected from a particular action. Finally, signals related to the reward history might be used to monitor the overall success of the animal's current decision-making strategy.
prefrontal cortex; decision making; reward
The value of an object acquired by a particular action often determines the motivation to produce that action. Previous studies found neural signals related to the values of different objects or goods in the orbitofrontal cortex, while the values of outcomes expected from different actions are broadly represented in multiple brain areas implicated in movement planning. However, how the brain combines the values associated with various objects and the information about their locations is not known. In this study, we tested whether the neurons in the dorsolateral prefrontal cortex (DLPFC) and striatum in rhesus monkeys might contribute to translating the value signals between multiple frames of reference. Monkeys were trained to perform an oculomotor intertemporal choice in which the color of a saccade target and the number of its surrounding dots signaled the magnitude of reward and its delay, respectively. In both DLPFC and striatum, temporally discounted values (DVs) associated with specific target colors and locations were encoded by partially overlapping populations of neurons. In the DLPFC, the information about reward delays and DVs of rewards available from specific target locations emerged earlier than the corresponding signals for target colors. Similar results were reproduced by a simple network model built to compute DVs of rewards in different locations. Therefore, DLPFC might play an important role in estimating the values of different actions by combining the previously learned values of objects and their present locations.
intertemporal choice; prefrontal cortex; reward; temporal discounting; utility
Recent directions in the treatment of ADHD have involved both a broadening of pharmacological perspectives to include nor-adrenergic as well as dopaminergic agents. A review of animal and human studies of pharmacological and therapeutic directions in ADHD suggests that the D1 receptor is a specific site for dopaminergic regulation of the PFC, but optimal levels of dopamine (DA) are required for beneficial effects on working memory. Animal and human studies indicate that the alpha-2A receptor is also important for prefrontal regulation, leaving open the question of the relative importance of these receptor sites. The therapeutic effects of ADHD medications in the prefrontal cortex have focused attention on the development of working memory capacity in ADHD.
The actions of dopaminergic vs noradrenergic agents, currently available for the treatment of ADHD have overlapping, but different actions in the prefrontal cortex (PFC) and subcortical centers. While stimulants act on D1 receptors in the dorsolateral prefrontal cortex, they also have effects on D2 receptors in the corpus striatum and may also have serotonergic effects at orbitofrontal areas. At therapeutic levels, dopamine (DA) stimulation (through DAT transporter inhibition) decreases noise level acting on subcortical D2 receptors, while NE stimulation (through alpha-2A agonists) increases signal by acting preferentially in the PFC possibly on DAD1 receptors. On the other hand, alpha-2A noradrenergic transmission is more limited to the prefrontal cortex (PFC), and thus less likely to have motor or stereotypic side effects, while alpha-2B and alpha-2C agonists may have wider cortical effects. The data suggest a possible hierarchy of specificity in the current medications used in the treatment of ADHD, with guanfacine likely to be most specific for the treatment of prefrontal attentional and working memory deficits. Stimulants may have broader effects on both vigilance and motor impulsivity, depending on dose levels, while atomoxetine may have effects on attention, anxiety, social affect, and sedation via noradrenergic transmission.
Tests of the hypothesis
At a theoretical level, the advent of possible specific alpha-2A noradrenergic therapies has posed the question of the role of working memory in ADHD. Head to head comparisons of stimulant and noradrenergic alpha-2A, alpha-2B and alpha-2C agonists, utilizing vigilance and affective measures should help to clarify pharmacological and therapeutic differences.
Mentalizing is defined as the inference of mental states of fellow humans, and is a particularly important skill for social interactions. Here we assessed whether activity in brain areas involved in mentalizing is specific to the processing of mental states or can be generalized to the inference of non-mental states by comparing brain responses during the interaction with an intentional and an artificial agent. Participants were scanned using fMRI during interactive rock-paper-scissors games while believing their opponent was a fellow human (Intentional agent, Int), a humanoid robot endowed with an artificial intelligence (Artificial agent, Art), or a computer playing randomly (Random agent, Rnd). Participants' subjective reports indicated that they adopted different stances against the three agents. The contrast of brain activity during interaction with the artificial and the random agents didn't yield any cluster at the threshold used, suggesting the absence of a reproducible stance when interacting with an artificial intelligence. We probed response to the artificial agent in regions of interest corresponding to clusters found in the contrast between the intentional and the random agents. In the precuneus involved in working memory, the posterior intraparietal suclus, in the control of attention and the dorsolateral prefrontal cortex, in executive functions, brain activity for Art was larger than for Rnd but lower than for Int, supporting the intrinsically engaging nature of social interactions. A similar pattern in the left premotor cortex and anterior intraparietal sulcus involved in motor resonance suggested that participants simulated human, and to a lesser extend humanoid robot actions, when playing the game. Finally, mentalizing regions, the medial prefrontal cortex and right temporoparietal junction, responded to the human only, supporting the specificity of mentalizing areas for interactions with intentional agents.
social cognition; neuroscience; artificial intelligence; fMRI
Prefrontal cortex mediates cognitive control by means of circuitry organized along dorso-ventral and rostro-caudal axes. Along the dorso-ventral axis, ventrolateral PFC controls semantic information, whereas dorsolateral PFC encodes task rules. Along the rostro-caudal axis, anterior prefrontal cortex encodes complex rules and relationships between stimuli, whereas posterior prefrontal cortex encodes simple relationships between stimuli and behavior. Evidence of these gradients of prefrontal cortex organization has been well documented in fMRI studies, but their functional correlates have not been examined with regard to integrity of underlying white matter tracts. We hypothesized that (a) the integrity of specific white matter tracts is related to cognitive functioning in a manner consistent with the dorso-ventral and rostro-caudal organization of the prefrontal cortex, and (b) this would be particularly evident in healthy older adults. We assessed three cognitive processes that recruit the prefrontal cortex and can distinguish white matter tracts along the dorso-ventral and rostro-caudal dimensions –episodic memory, working memory, and reasoning. Correlations between cognition and fractional anisotropy as well as fiber tractography revealed: (a) Episodic memory was related to ventral prefrontal cortex-thalamo-hippocampal fiber integrity; (b) Working memory was related to integrity of corpus callosum body fibers subserving dorsolateral prefrontal cortex; and (c) Reasoning was related to integrity of corpus callosum body fibers subserving rostral and caudal dorsolateral prefrontal cortex. These findings confirm the ventrolateral prefrontal cortex's role in semantic control and the dorsolateral prefrontal cortex's role in rule-based processing, in accordance with the dorso-ventral prefrontal cortex gradient. Reasoning-related rostral and caudal superior frontal white matter may facilitate different levels of task rule complexity. This study is the first to demonstrate dorso-ventral and rostro-caudal prefrontal cortex processing gradients in white matter integrity.
The primate prefrontal cortex contributes to stimulus-guided behavior, but the functional specializations among its areas remain uncertain. To better understand such specializations, we contrasted neuronal activity in the dorsolateral prefrontal cortex (PFdl) and the orbital prefrontal cortex (PFo). The task required rhesus monkeys to use a visual cue to choose a saccade target. Some cues instructed the monkeys to repeat their most recent response; others instructed them to change it. Responses were followed by feedback: fluid reward if correct, visual feedback if incorrect. Previous studies, using different tasks, have reported that PFo neurons did not encode responses. We found PFo did encode responses in this task, but only near feedback time, after the response had been completed. PFdl differed from PFo in several respects. As reported previously, some PFdl neurons encoded responses from the previous trial and others encoded planned responses. PFo neurons did not have these properties. After feedback, PFdl encoded rewarded responses better than unrewarded ones and thus combined response and outcome information. PFo, in contrast, encoded the responses chosen, rewarded or not. These findings suggest that PFdl and PFo contribute differently to response knowledge, with PFo using an outcome-independent signal to monitor current responses at feedback time.
Decision; feedback; monitoring; evaluation; frontal lobe; prefrontal cortex
Human behaviors can be more powerfully influenced by conditioned reinforcers, such as money, than by primary reinforcers. Moreover, people often change their behaviors to avoid monetary losses. However, the effect of removing conditioned reinforcers on choices has not been explored in animals, and the neural mechanisms mediating the behavioral effects of gains and losses are not well understood. To investigate the behavioral and neural effects of gaining and losing a conditioned reinforcer, we trained rhesus monkeys for a matching pennies task in which the positive and negative values of its payoff matrix were realized by the delivery and removal of a conditioned reinforcer. Consistent with the findings previously obtained with non-negative payoffs and primary rewards, the animal’s choice behavior during this task was nearly optimal. Nevertheless, the gain and loss of a conditioned reinforcer significantly increased and decreased, respectively, the tendency for the animal to choose the same target in subsequent trials. We also found that the neurons in the dorsomedial frontal cortex, dorsal anterior cingulate cortex, and dorsolateral prefrontal cortex often changed their activity according to whether the animal earned or lost a conditioned reinforcer in the current or previous trial. Moreover, many neurons in the dorsomedial frontal cortex also signaled the gain or loss occurring as a result of choosing a particular action as well as changes in the animal’s behaviors resulting from such gains or losses. Thus, primate medial frontal cortex might mediate the behavioral effects of conditioned reinforcers and their losses.
cingulate cortex; decision making; prefrontal cortex; reinforcement learning; reward; punishment; neuroeconomics
Activity of the neurons in the lateral intra-parietal cortex or LIP displays a mixture of sensory, motor, and memory signals. Moreover, they often encode signals reflecting the accumulation of sensory evidence that certain eye movements might lead to a desirable outcome. However, when the environment changes dynamically, animals are also required to combine the information about its previously chosen actions and their outcomes appropriately to update continually the desirabilities of alternative actions. Here, we investigated whether LIP neurons encoded signals necessary to update an animal’s decision making strategies adaptively during a computer-simulated matching pennies game. Using a reinforcement learning algorithm, we estimated the value functions that best predicted the animal’s choices on a trial-by-trial basis. We found that immediately before the animal revealed its choice, approximately 18% of LIP neurons changed their activity according to the difference in the value functions for the two targets. In addition, a somewhat higher fraction of LIP neurons displayed signals related to the sum of the value function, which might correspond to the state value function or an average rate of reward used as a reference point. Similar to the neurons in the prefrontal cortex, many LIP neurons also encoded the signals related to the animal’s previous choices. Thus, the posterior parietal cortex might be a part of the network that provides the substrate for forming appropriate associations between actions and outcomes.
decision; parietal; reward; feedback; control; cognition
Humans and animals often must choose between rewards that differ in their qualities, magnitudes, immediacy, and likelihood, and must estimate these multiple reward parameters from their experience. However, the neural basis for such complex decision making is not well understood. To understand the role of the primate prefrontal cortex in determining the subjective value of delayed or uncertain reward, we examined the activity of individual prefrontal neurons during an inter-temporal choice task and a computer-simulated competitive game. Consistent with the findings from previous studies in humans and other animals, the monkey’s behaviors during inter-temporal choice were well accounted for by a hyperbolic discount function. In addition, the activity of many neurons in the lateral prefrontal cortex reflected the signals related to the magnitude and delay of the reward expected from a particular action, and often encoded the difference in temporally discounted values that predicted the animal’s choice. During a computerized matching pennies game, the animals approximated the optimal strategy, known as Nash equilibrium, using a reinforcement learning algorithm. We also found that many neurons in the lateral prefrontal cortex conveyed the signals related to the animal’s previous choices and their outcomes, suggesting that this cortical area might play an important role in forming associations between actions and their outcomes. These results show that the primate lateral prefrontal cortex plays a central role in estimating the values of alternative actions based on multiple sources of information.
game theory; inter-temporal choice; reinforcement learning; utility theory; temporal discounting
The dorsolateral prefrontal and posterior parietal cortex play critical roles in mediating attention, working memory, and executive function. Despite proposed dynamic modulation of connectivity strength within each area according to task demands, scant empirical data exist about the time course of the strength of effective connectivity, particularly in tasks requiring information to be sustained in working memory. We investigated this question by performing time-resolved cross-correlation analysis for pairs of neurons recorded simultaneously at distances of 0.2–1.5 mm apart of each other while monkeys were engaged in working memory tasks. The strength of effective connectivity determined in this manner was higher throughout the trial in the posterior parietal cortex than the dorsolateral prefrontal cortex. Significantly higher levels of parietal effective connectivity were observed specifically during the delay period of the task. These differences could not be accounted for by differences in firing rate, or electrode distance in the samples recorded in the posterior parietal and prefrontal cortex. Differences were present when we restricted our analysis to only neurons with significant delay period activity and overlapping receptive fields. Our results indicate that dynamic changes in connectivity strength are present but area-specific intrinsic organization is the predominant factor that determines the strength of connections between neurons in each of the two areas.
Our choices often require appropriate actions in order to obtain a preferred outcome, but the neural underpinnings that link decision making and action selection remain largely undetermined. Recent theories propose that action selection occurs simultaneously, i.e. parallel in time, with the decision process. Specifically, it is thought that action selection in motor regions originates from a competitive process which is gradually biased by evidence signals originating in other regions, such as those specialized in value computations. Biases reflecting the evaluation of choice options should thus emerge in the motor system before the decision process is complete. Using transcranial magnetic stimulation, we sought direct physiological evidence for this prediction by measuring changes in cortico-spinal excitability in human motor cortex during value-based decisions. We found that excitability for chosen versus unchosen actions distinguishes the forthcoming choice before completion of the decision process. Both excitability and reaction times varied as a function of the subjective value-difference between chosen and unchosen actions, consistent with this effect being value-driven. This relationship was not observed in the absence of a decision. Our data provide novel evidence in humans that internally generated value-based decisions influence the competition between action representations in motor cortex before the decision process is complete. This is incompatible with models of serial processing of stimulus, decision, and action.
We assessed the involvement of the orbital prefrontal cortex (PFo), the prelimbic region of the medial prefrontal cortex (PL), and the amygdala in goal-directed behavior. Rhesus monkeys were trained on a task in which two different instrumental responses were linked to two different outcomes. One response, called ‘Tap’, required the monkeys to repeatedly touch a colored square on a video monitor, to produce one kind of food reward. The other response, called ‘Hold’, required persistent contact of an identical stimulus, and it produced a different kind of food reward. Following training, we assessed the effects of satiety-specific reinforcer devaluation as a way to probe each monkey’s use of goal-directed behavior. In this procedure, monkeys were allowed to consume one of the two foods to satiety, and were then tested for Tap/Hold preference under extinction. Unoperated control monkeys showed a reduction in the response associated with obtaining the devalued food, called the devaluation effect, a hallmark of goal-directed behavior. Monkeys with bilateral lesions of PFo or the amygdala exhibited significantly reduced devaluation effects. Results from monkeys with PL lesions were equivocal. We conclude that both PFo and the amygdala play a significant role in goal-directed behavior in monkeys. Notably, the findings for PFo challenge the idea that orbital and medial prefrontal regions are exclusively dedicated to object- and action-based processes, respectively.
It is generally assumed that choice between different actions reflects the difference between their action values yet little direct evidence confirming this assumption has been reported. Here we assess whether the brain calculates the absolute difference between action values or their relative advantage, that is, the probability that one action is better than the other alternatives. We use a two-armed bandit task during functional magnetic resonance imaging and modelled responses to determine both the size of the difference between action values (D) and the probability that one action value is better (P). The results show haemodynamic signals corresponding to P in right dorsolateral prefrontal cortex (dlPFC) together with evidence that these signals modulate motor cortex activity in an action-specific manner. We find no significant activity related to D. These findings demonstrate that a distinct neuronal population mediates action-value comparisons, and reveals how these comparisons are implemented to mediate value-based decision-making.
In humans, choice between actions depends on the ability to compare action–outcome values. Here, the authors show that action–outcome values are compared on the basis of the relative advantage of a particular action over alternative actions, which takes place in the right dorsolateral prefrontal cortex of the brain.
Functional impairment of the orbital and medial prefrontal cortex underlies deficits in executive control that characterize addictive disorders, including alcohol addiction. Previous studies indicate that alcohol alters glutamate neurotransmission and one substrate of these effects may be through the reconfiguration of the subunits constituting ionotropic glutamate receptor (iGluR) complexes. Glutamatergic transmission is integral to cortico-cortical and cortico-subcortical communication, and alcohol-induced changes in the abundance of the receptor subunits and/or their splice variants may result in critical functional impairments of prefrontal cortex in the alcohol-addicted state.
Methods and results
The effects of chronic ethanol self-administration on glutamate receptor ionotropic NMDA (GRIN), as well as GRIN1 splice variant mRNA expression was studied in the orbitofrontal cortex (OFC; Area 13), dorsolateral prefrontal cortex (DLPFC; Area 46) and anterior cingulate cortex (ACC; Area 24) of male cynomolgus monkeys. Chronic ethanol self-administration resulted in significant changes in the expression of NMDA subunit mRNA expression in the DLPFC and OFC, but not the ACC. In DLPFC, the overall expression of NMDA subunits was significantly decreased in ethanol treated monkeys. Slight but significant changes were observed for synaptic associated protein 102 kD (SAP102) and neuronal nitric oxide synthase (nNOS) mRNAs. In OFC, the NMDAR1 variant GRIN1-1 was reduced while GRIN1-2 was increased. Furthermore, no significant changes in GFAP protein levels were observed in either the DLPFC or OFC.
Results from these studies provide the first demonstration of post-transcriptional regulation of iGluR subunits in the primate brain following long-term ethanol self-administration. Furthermore, changes in these transcripts do not appear to reflect changes in glial activation or loss. Further studies examining the expression and cellular localization of subunit proteins and receptor pharmacology would shed more light on the findings reported here.
Ethanol; Glutamate; messenger RNA; Prefrontal Cortex; qPCR; Primate
Monkeys adjust their behavior in response to outcomes that they have observed but not directly experienced, and single neurons within the anterior cingulate cortex respond to these fictive rewards they same way they respond to experienced rewards.
The neural mechanisms supporting the ability to recognize and respond to fictive outcomes, outcomes of actions that one has not taken, remain obscure. We hypothesized that neurons in anterior cingulate cortex (ACC), which monitors the consequences of actions and mediates subsequent changes in behavior, would respond to fictive reward information. We recorded responses of single neurons during performance of a choice task that provided information about the reward values of unchosen options. We found that ACC neurons signal fictive reward information, and use a coding scheme similar to that used to signal experienced outcomes. Thus, individual ACC neurons process both experienced and fictive rewards.
Humans with drug addiction exhibit compulsive drug-seeking associated with impairment of prefrontal cortex cognitive function. Whether prefrontal cortex dysfunction is a consequence of chronic drug exposure, or mediates the transition from drug use to drug dependence, is unknown. The current study investigates whether a history of escalated vs controlled cocaine intake is associated with specific working memory impairments, and long-lasting alterations of the dorsomedial prefrontal cortex and orbitofrontal cortex in rats. Working memory was assessed in rats with a history of extended (6 h per session) or limited (1 h per session) access to cocaine (0.5 mg/kg per injection), 3–17 days after the last self-administration session, using a delayed nonmatching-to-sample task. The density of neurons, oligodendrocytes, and astrocytes was quantified in the dorsomedial prefrontal cortex and orbitofrontal prefrontal cortex 2 months after the last self-administration session. Working memory impairments were observed after a history of chronic and escalated cocaine intake, but not after repeated limited access to cocaine. Moreover, working memory impairments were correlated with a decreased density of neurons and oligodendrocytes but not astrocytes in the dorsomedial prefrontal cortex, and with a decreased density of oligodendrocytes in the orbitofrontal cortex. Considering the role of the prefrontal cortex in goal-directed behavior, the prefrontal cortex dysfunctions observed here may exacerbate the loss of control associated with increased drug use and facilitate the progression to drug addiction.
psychostimulant; prefrontal cortex; self-administration; working memory; compulsivity; dependence
Reinforcement of behavioral responses involves a complex cerebral circuit engaging specific neuronal networks that are modulated by cortical oversight systems affiliated with emotion, memory, judgment, and decision making (collectively referred to in this study as the “extended reward and oversight system” or ”reward-network”). We examined whether reward-network brain volumes are reduced in alcoholics, and how volumes of subcomponents within this system are correlated with memory and drinking-history.
Morphometric analysis was performed on magnetic resonance brain scans in 21 abstinent long-term chronic alcoholic men and 21 healthy control men, group-matched on age, verbal IQ, and education. We derived volumes of total brain and volumes of cortical and subcortical reward-related structures including the dorsolateral-prefrontal, orbitofrontal, and cingulate cortices, and the insula, as well as the amygdala, hippocampus, nucleus accumbens septi (NAc), and ventral diencephalon.
Morphometric analyses of reward-related regions revealed decreased total reward-network volume in alcoholic subjects. Volume reduction was most pronounced in right dorsolateral-prefrontal cortex, right anterior insula, and right NAc, as well as left amygdala. In alcoholics, NAc and anterior insula volumes increased with length-of-abstinence, and total reward-network and amygdala volumes correlated positively with memory scores.
The observation of decreased reward-network volume suggests that alcoholism is associated with alterations in this neural reward system. These structural reward system deficits, and their correlation with memory scores, elucidate underlying structural-functional relationships between alcoholism and emotional and cognitive processes.
Alcoholism; MRI; reward system; amygdala; nucleus accumbens; dorsolateral-prefrontal cortex
Vagus nerve stimulation (VNS) can improve depression. Cognitive models of depression highlight an over-representation of negative thoughts and memories, with depressed individuals showing memory facilitation for negative material. We hypothesized that the antidepressant action of VNS may emerge through corrective influences on ‘negativity bias’ in memory. We therefore examined the impact of VNS on emotional memory and its underlying brain activity.
We tested a single patient undergoing VNS for treatment-resistant depression (TRD). Stimulation was set at a 30/66 second on/off cycle during three encoding blocks when the patient viewed randomly presented positive, negative and neutral words. Following each block, VNS was switched off and the patient identified previously seen words from distractors in a subsequent recognition memory task. The patient was scanned using functional magnetic resonance imaging (fMRI) during the first encoding block.
There was robust recall of negative material viewed during ‘off cycles’ of VNS but subsequent memory of negative words was attenuated during active VNS (on cycles). These effects were not apparent for neutral and positive words. In neuroimaging, direct modulatory effects of VNS were observed in dorsomedial, dorsolateral and orbital regions of prefrontal cortex. Moreover, during encoding of negative words, compared to positive and neutral words, VNS also modulated activity within orbitofrontal, ventromedial and polar prefrontal cortices, mid cingulate cortex and brainstem.
Our observations show that VNS can interfere with memory for negative information, an effect that may contribute to its antidepressant role. Neuroimaging implicates regions including ventral and medial prefrontal cortex as an underlying neural substrate.
Motivational interventions to improve health behaviors based on conventional cognitive and behavioral theories have been extensively studied; however, advances in neuroimaging technology make it possible to assess the neurophysiological basis of health behaviors, such as physical activity. The goals of this approach are to support new interventions to achieve optimal outcomes.
This study used functional magnetic resonance imaging (fMRI) to assess differences in brain responses in healthy weight to obese midlife women during a goal-directed decision task.
Thirty nondiabetic, midlife (age 47–55 years) women with body mass index (BMI) ranging from 18.5 to 40 kg/m2 were recruited. A descriptive, correlational design was used to assess the relationship between brain activations and weight status. Participants underwent a goal-directed behavior task in the fMRI scanner consisting of a learning and implementation phase. The task was designed to assess both goal-directed and habitual behaviors. One participant was omitted from the analysis because of excessive motion (>4 mm), and six were omitted because of fewer than 50% correct responses on the exit survey. Four participants developed claustrophobia in the scanner and were disqualified from further participation. The remaining 19 participants were included in the final analysis.
Brain responses while participants learned goal-directed behavior showed a positive correlation with BMI in the dorsomedial prefrontal cortex (dmPFC) and a negative correlation with BMI in the insula. During the implementation of goal-directed behavior, brain responses in the dorsolateral prefrontal cortex (dlPFC) negatively correlated with BMI.
These results indicate that overweight women activate regions associated with cognitive control to a greater degree than healthy weight women during goal-directed learning. The brain regions activated (dmPFC, dlPFC, insula) are associated with cognitive control and self-regulation. On the other hand, healthy weight women activate regions associated with emotion processing, planning, and self-regulation (lateral orbitofrontal cortex, anterior insula) to a greater degree than overweight women during goal-directed learning and implementation of goal-directed behavior. Overweight women activate cognitive control regions while learning associations between actions and outcomes; however, this is not the case during the implementation phase—which may make it more difficult to transform goals into action (e.g., maintain physical activity over time). Overall, these results indicate that overweight midlife women respond differently during learning and implementation of actions that lead to positive outcomes during a general test of goal-directed behavior. Future study is needed to assess the transfer of goal-directed and habitual behavior to specific aspects of energy balance to improve health outcomes.
fMRI; health behavior; neuroimaging; neurophysiology; obesity; women’s health
Studies in monkeys show clear anatomical and functional distinctions among networks connecting with subregions within the prefrontal cortex. Three such networks are centered on lateral orbitofrontal cortex, medial frontal and cingulate cortex, and lateral prefrontal cortex and all have been identified with distinct cognitive roles. Although these areas differ in a number of their cortical connections, some of the first anatomical evidence for these networks came from tracer studies demonstrating their distinct patterns of connectivity with the mediodorsal (MD) nucleus of the thalamus. Here, we present evidence for a similar topography of MD thalamus prefrontal connections, using non-invasive imaging and diffusion tractography (DWI–DT) in human and macaque. DWI–DT suggested that there was a high probability of interconnection between medial MD and lateral orbitofrontal cortex, between caudodorsal MD and medial frontal/cingulate cortex, and between lateral MD and lateral prefrontal cortex, in both species. Within the lateral prefrontal cortex a dorsolateral region (the principal sulcus in the macaque and middle frontal gyrus in the human) was found to have a high probability of interconnection with the MD region between the regions with a high probability of interconnection with other parts of the lateral prefrontal cortex and with the lateral orbitofrontal cortex. In addition to suggesting that the thalamic connectivity in the macaque is a good guide to human prefrontal cortex, and therefore that there are likely to be similarities in the cognitive roles played by the prefrontal areas in both species, the present results are also the first to provide insight into the topography of projections of an individual thalamic nucleus in the human brain.
Anatomy; DTI; Human; Macaque; Thalamus
Tactile object discrimination is an essential human skill that relies on functional connectivity between the neural substrates of motor, somatosensory and supramodal areas. From a theoretical point of view, such distributed networks elude categorical analysis because subtraction methods are univariate. Thus, the aim of this study was to identify the neural networks involved in somatosensory object discrimination using a voxel-based principal component analysis (PCA) of event-related functional magnetic resonance images.
Seven healthy, right-handed subjects aged between 22 and 44 years were required to discriminate with their dominant hand the length differences between otherwise identical parallelepipeds in a two-alternative forced-choice paradigm. Of the 34 principal components retained for analysis according to the ‘bootstrapped’ Kaiser-Guttman criterion, t-tests applied to the subject-condition expression coefficients showed significant mean differences between the object presentation and inter-stimulus phases in PC 1, 3, 26 and 32. Specifically, PC 1 reflected object exploration or manipulation, PC 3 somatosensory and short-term memory processes. PC 26 evinced the perception that certain parallelepipeds could not be distinguished, while PC 32 emerged in those choices when they could be. Among the cerebral regions evident in the PCs are the left posterior parietal lobe and premotor cortex in PC 1, the left superior parietal lobule (SPL) and the right cuneus in PC 3, the medial frontal and orbitofrontal cortex bilaterally in PC 26, and the right intraparietal sulcus, anterior SPL and dorsolateral prefrontal cortex in PC 32.
The analysis provides evidence for the concerted action of large-scale cortico-subcortical networks mediating tactile object discrimination. Parallel to activity in nodes processing object-related impulses we found activity in key cerebral regions responsible for subjective assessment and validation.
Studies investigating response reversal consistently implicate regions of medial and lateral prefrontal cortex when reinforcement contingencies change. However, it is unclear from these studies how these regions give rise to the individual components of response reversal, such as reinforcement value encoding, response inhibition, and response change. Here we report a novel instrumental learning task designed to determine whether regions implicated in processing reversal errors are uniquely involved in this process, or whether they play a more general role in representing response competition, reinforcement value, or punishment value in the absence of demands for response change. In line with previous findings, reversal errors activated orbitofrontal cortex, dorsomedial prefrontal cortex, ventrolateral prefrontal cortex, caudate, and dorsolateral prefrontal cortex. These regions also showed increased activity to errors in the absence of contingency changes. In addition, ventrolateral PFC, caudate, and dorsolateral PFC each exhibited increased activity following correct reversals. Activity in these regions was not significantly modulated by changes in reinforcement value that were not sufficient to make an alternative response advantageous. These data do not support punishment-processing or prepotent reponse inhibition accounts of ventrolateral prefrontal cortex function. Instead, they support recent conceptualizations of ventrolateral prefrontal cortex function that implicate this region in resolving response competition by manipulating the representation of either motor response options, or object features. These data also suggest that dorsolateral prefrontal cortex plays a role in reversal learning, probably through top down attentional control of object or reinforcement features when task demands increase.
Response reversal; affective shift; response competition; ventrolateral prefrontal cortex; decision-making
Research in animal learning and behavioral neuroscience has distinguished between two forms of action control: a habit-based form, which relies on stored actio n values, and a goal-dir ected form, which forecasts and compares action outcomes based on a model of the environment. While habit-based control has been the subject of extensive computational research, the computational principles underlying goal-directed control in animals have so far received less attention. In the present paper, we advance a computational framework for goal-directed control in animals and humans. We take three empirically motivated points as founding premises: (1) Neurons in dorsolateral prefrontal cortex represent action policies, (2) Neurons in orbitofrontal cortex represent rewards, and (3) Neural computation, across domains, can be appropriately understood as performing structured probabilistic inference. On a purely computational level, the resulting account relates closely to previous work using Bayesian inference to solve Markov decision problems, but extends this work by introducing a new algorithm, which provably converges on optimal plans. On a cognitive and neuroscientific level, the theory provides a unifying framework for several different forms of goal-directed action selection, placing emphasis on a novel form, within which orbitofrontal reward representations directly drive policy selection.
Cannabis use is associated with both impaired cognitive functions, including working memory, and an increased risk of schizophrenia. Schizophrenia is characterized by impairments in working memory that are associated with reduced γ-aminobutyric acid (GABA) neurotransmission in the dorsolateral prefrontal cortex. The cannabinoid 1 receptor (CB1R) is highly expressed in the dorsolateral prefrontal cortex, is contained in the axon terminals of a subpopulation of perisomatic-targeting GABA neurons, and, when activated, suppresses the release of GABA.
To determine the potential relationship between CB1R signaling and altered GABA neurotransmission in schizophrenia by evaluating CB1R messenger RNA (mRNA) and protein expression in the dorsolateral pre-frontal cortex.
In situ hybridization and immunocytochemistry techniques were used to examine the cortical levels of CB1R mRNA and protein, respectively.
Brain specimens were obtained from autopsies conducted at the Allegheny County Medical Examiner’s Office, Pittsburgh, Pennsylvania.
Postmortem brain specimens from 23 pairs of subjects with schizophrenia and age-, sex-, and postmortem interval–matched comparison subjects, as well as brain specimens from 18 macaque monkeys with long-term exposure to haloperidol, olanzapine, or placebo.
Main Outcome Measures
Optical density measures of CB1R mRNA expression and protein levels and correlations with previously reported glutamic acid decarboxylase 67 and cholecystokinin mRNA measures.
Levels of CB1R mRNA were significantly lower by 14.8% in the subjects with schizophrenia. Similarly, CB1R protein levels, assessed by radioimmunocytochemistry and standard immunocytochemistry, were significantly decreased by 11.6% and 13.9%, respectively. Group differences in CB1R mRNA levels were significantly correlated with those in glutamic acid decarboxylase 67 and cholecystokinin mRNA levels. Expression of CB1R mRNA was not changed in antipsychotic-exposed monkeys, and neither CB1R mRNA levels nor protein levels were affected by potential confounding factors in the subjects with schizophrenia.
This combination of findings suggests the testable hypothesis that reduced CB1R mRNA and protein levels in schizophrenia represent a compensatory mechanism to increase GABA transmission from perisomatic-targeting cholecystokinin interneurons with impaired GABA synthesis.