PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1695366)

Clipboard (0)
None

Related Articles

1.  Prefrontal cortex and hybrid learning during iterative competitive games 
Behavioral changes driven by reinforcement and punishment are referred to as simple or model-free reinforcement learning. Animals can also change their behaviors by observing events that are neither appetitive nor aversive, when these events provide new information about payoffs available from alternative actions. This is an example of model-based reinforcement learning, and can be accomplished by incorporating hypothetical reward signals into the value functions for specific actions. Recent neuroimaging and single-neuron recording studies showed that the prefrontal cortex and the striatum are involved not only in reinforcement and punishment, but also in model-based reinforcement learning. We found evidence for both types of learning, and hence hybrid learning, in monkeys during simulated competitive games. In addition, in both the dorsolateral prefrontal cortex and orbitofrontal cortex, individual neurons heterogeneously encoded signals related to actual and hypothetical outcomes from specific actions, suggesting that both areas might contribute to hybrid learning.
doi:10.1111/j.1749-6632.2011.06223.x
PMCID: PMC3302724  PMID: 22145879
belief learning; decision making; game theory; reinforcement learning; reward
2.  Capturing the temporal evolution of choice across prefrontal cortex 
eLife  null;4:e11945.
Activity in prefrontal cortex (PFC) has been richly described using economic models of choice. Yet such descriptions fail to capture the dynamics of decision formation. Describing dynamic neural processes has proven challenging due to the problem of indexing the internal state of PFC and its trial-by-trial variation. Using primate neurophysiology and human magnetoencephalography, we here recover a single-trial index of PFC internal states from multiple simultaneously recorded PFC subregions. This index can explain the origins of neural representations of economic variables in PFC. It describes the relationship between neural dynamics and behaviour in both human and monkey PFC, directly bridging between human neuroimaging data and underlying neuronal activity. Moreover, it reveals a functionally dissociable interaction between orbitofrontal cortex, anterior cingulate cortex and dorsolateral PFC in guiding cost-benefit decisions. We cast our observations in terms of a recurrent neural network model of choice, providing formal links to mechanistic dynamical accounts of decision-making.
DOI: http://dx.doi.org/10.7554/eLife.11945.001
eLife digest
In 1848, a railroad worker named Phineas Gage suffered an accident that was to secure him a place in neuroscience lore. While constructing a new railway line, a mistimed explosion propelled an iron bar into the base of his skull, where it passed behind his left eye before exiting through the top of his head. Gage survived the accident, but those who knew him reported significant changes in his personality and behaviour.
Gage’s ability to make decisions was particularly impaired by his injury. Decision-making involves weighing up the costs and benefits associated with alternative courses of action. It entails looking into the future to decide whether an anticipated reward will justify the effort or expense necessary to obtain it. This process is dependent on a region of the brain called the prefrontal cortex, the area that sustained the most damage in Phineas Gage.
While many studies have shown correlations between activity in particular parts of prefrontal cortex and the outcome of decisions, little is known about how this activity evolves over time as a decision is made. To explore this process, Hunt et al. trained macaque monkeys to choose between pairs of images that were associated with specific rewards (quantities of fruit juice) and costs (either amounts of work or fixed delays).
Electrode recordings revealed changes in prefrontal activity that varied over time as the monkeys deliberated over each pair of images, choosing for example between a large reward after a long delay versus a smaller reward immediately. This activity was consistent with a mathematical model of decision-making, which also explains data from brain imaging experiments in humans. This provides an important link between human data and electrode recordings in animals.
However, some of the patterns of activity observed in both macaques and humans appeared to reflect the speed at which decisions were made, rather than the outcome of the decisions themselves. By extracting information about decision speed on each decision from each region, it was shown that communication between regions of prefrontal cortex changes when choices are between two different amounts of work, as opposed to two different delays. Further experiments are needed to explore this phenomenon and to determine how other brain regions interact with the prefrontal cortex to support the decision-making process.
DOI: http://dx.doi.org/10.7554/eLife.11945.002
doi:10.7554/eLife.11945
PMCID: PMC4718814  PMID: 26653139
electrophysiology; Decision-making; magnetoencephalography; Reaction time; Neural dynamics; Prefrontal cortex; Human; Macaque monkey
3.  Signatures of Value Comparison in Ventral Striatum Neurons 
PLoS Biology  2015;13(6):e1002173.
The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.
A study of single neurons in the ventral striatum reveals signatures of value comparison and selection during a risky choice task, suggesting that the cortex and its striatal targets make overlapping contributions to the choice process. Read the accompanying Primer.
Author Summary
The neural calculations underlying reward-based choice are closely associated with a network of brain areas including the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC). Most theories ascribe distinct roles to these two structures during choice, but these differences have yet to be confirmed at the level of single neurons. We compared responses of VS neurons to those of vmPFC neurons recorded in rhesus macaques choosing between potential gambles for water rewards. We found widespread similarities in the way that VS and vmPFC neurons fire during the choice process. Neurons in both areas encoded the value of the offered gamble, the difference in value between offered gambles, and the gamble outcome. Additionally, both areas showed stronger coding for the chosen gamble than for the unchosen one and predicted choice even when we controlled for offer value. Interestingly, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, similarities between vmPFC and VS suggest that cortex and its striatal targets make overlapping contributions to reward-based choice.
doi:10.1371/journal.pbio.1002173
PMCID: PMC4472856  PMID: 26086735
4.  Prefrontal and Striatal Activity Related to Values of Objects and Locations 
The value of an object acquired by a particular action often determines the motivation to produce that action. Previous studies found neural signals related to the values of different objects or goods in the orbitofrontal cortex, while the values of outcomes expected from different actions are broadly represented in multiple brain areas implicated in movement planning. However, how the brain combines the values associated with various objects and the information about their locations is not known. In this study, we tested whether the neurons in the dorsolateral prefrontal cortex (DLPFC) and striatum in rhesus monkeys might contribute to translating the value signals between multiple frames of reference. Monkeys were trained to perform an oculomotor intertemporal choice in which the color of a saccade target and the number of its surrounding dots signaled the magnitude of reward and its delay, respectively. In both DLPFC and striatum, temporally discounted values (DVs) associated with specific target colors and locations were encoded by partially overlapping populations of neurons. In the DLPFC, the information about reward delays and DVs of rewards available from specific target locations emerged earlier than the corresponding signals for target colors. Similar results were reproduced by a simple network model built to compute DVs of rewards in different locations. Therefore, DLPFC might play an important role in estimating the values of different actions by combining the previously learned values of objects and their present locations.
doi:10.3389/fnins.2012.00108
PMCID: PMC3398315  PMID: 22822390
intertemporal choice; prefrontal cortex; reward; temporal discounting; utility
5.  Cortical mechanisms for reinforcement learning in competitive games 
Game theory analyses optimal strategies for multiple decision makers interacting in a social group. However, the behaviours of individual humans and animals often deviate systematically from the optimal strategies described by game theory. The behaviours of rhesus monkeys (Macaca mulatta) in simple zero-sum games showed similar patterns, but their departures from the optimal strategies were well accounted for by a simple reinforcement-learning algorithm. During a computer-simulated zero-sum game, neurons in the dorsolateral prefrontal cortex often encoded the previous choices of the animal and its opponent as well as the animal's reward history. By contrast, the neurons in the anterior cingulate cortex predominantly encoded the animal's reward history. Using simple competitive games, therefore, we have demonstrated functional specialization between different areas of the primate frontal cortex involved in outcome monitoring and action selection. Temporally extended signals related to the animal's previous choices might facilitate the association between choices and their delayed outcomes, whereas information about the choices of the opponent might be used to estimate the reward expected from a particular action. Finally, signals related to the reward history might be used to monitor the overall success of the animal's current decision-making strategy.
doi:10.1098/rstb.2008.0158
PMCID: PMC2607365  PMID: 18829430
prefrontal cortex; decision making; reward
6.  Specialized areas for value updating and goal selection in the primate orbitofrontal cortex 
eLife  null;4:e11695.
The macaque orbitofrontal cortex (OFC) is essential for selecting goals based on current, updated values of expected reward outcomes. As monkeys consume a given type of reward to satiety, its value diminishes, and OFC damage impairs the ability to shift goal choices away from devalued outcomes. To examine the contributions of OFC’s components to goal selection, we reversibly inactivated either its anterior (area 11) or posterior (area 13) parts. We found that neurons in area 13 must be active during the selective satiation procedure to enable the updating of outcome valuations. After this updating has occurred, however, area 13 is not needed to select goals based on this knowledge. In contrast, neurons in area 11 do not need to be active during the value-updating process. Instead, inactivation of this area during choices causes an impairment. These findings demonstrate selective and complementary specializations within the OFC.
DOI: http://dx.doi.org/10.7554/eLife.11695.001
eLife digest
Everyone knows that somehow, somewhere, the brain translates knowledge into action. In some people, however, knowledge and action become disconnected. These people behave in a way that either ignores or contradicts the knowledge that they have. They know what to do and can explain it to others, but – when the time comes to act – they do something else, something wrong.
Murray et al. have now investigated how a brain region called the orbitofrontal cortex helps to link knowledge and action in macaque monkeys, which, unlike rodents, have all of the main brain areas that make up the orbitofrontal cortex of humans. The monkeys learned to associate images with different types of food, and then performed a task where they chose between two images in order to get the food they wanted. On some days, one of the foods was less ‘valuable’ because the monkeys had already eaten a lot of it. In these circumstances, monkeys chose fewer of the images associated with that food.
By temporarily inactivating either the front or back region of the monkey’s orbitofrontal cortex at different times, Murray et al. showed that these regions make different contributions to decision making. Inactivating the back region of the orbitofrontal cortex disrupted the ability of monkeys to update their knowledge about the value of a particular foodstuff. However, inactivating the front part of the orbitofrontal cortex disrupted their ability to use this knowledge to select the images that led to the most valuable food. This contradicts the widely held belief that the orbitofrontal cortex acts as a single entity to update values and translate this knowledge into action.
Future work will need to investigate how, having translated knowledge into a chosen action, the orbitofrontal cortex stimulates the motor areas of the brain to generate the movements needed to perform that action.
DOI: http://dx.doi.org/10.7554/eLife.11695.002
doi:10.7554/eLife.11695
PMCID: PMC4739757  PMID: 26673891
macaque monkey; decision making; reinforcer devaluation; goal neglect; Other
7.  Behavioral and Neural Changes Following Gains and Losses of Conditioned Reinforcers 
Human behaviors can be more powerfully influenced by conditioned reinforcers, such as money, than by primary reinforcers. Moreover, people often change their behaviors to avoid monetary losses. However, the effect of removing conditioned reinforcers on choices has not been explored in animals, and the neural mechanisms mediating the behavioral effects of gains and losses are not well understood. To investigate the behavioral and neural effects of gaining and losing a conditioned reinforcer, we trained rhesus monkeys for a matching pennies task in which the positive and negative values of its payoff matrix were realized by the delivery and removal of a conditioned reinforcer. Consistent with the findings previously obtained with non-negative payoffs and primary rewards, the animal’s choice behavior during this task was nearly optimal. Nevertheless, the gain and loss of a conditioned reinforcer significantly increased and decreased, respectively, the tendency for the animal to choose the same target in subsequent trials. We also found that the neurons in the dorsomedial frontal cortex, dorsal anterior cingulate cortex, and dorsolateral prefrontal cortex often changed their activity according to whether the animal earned or lost a conditioned reinforcer in the current or previous trial. Moreover, many neurons in the dorsomedial frontal cortex also signaled the gain or loss occurring as a result of choosing a particular action as well as changes in the animal’s behaviors resulting from such gains or losses. Thus, primate medial frontal cortex might mediate the behavioral effects of conditioned reinforcers and their losses.
doi:10.1523/JNEUROSCI.4726-08.2009
PMCID: PMC2750005  PMID: 19295166
cingulate cortex; decision making; prefrontal cortex; reinforcement learning; reward; punishment; neuroeconomics
8.  Monkey orbitofrontal cortex encodes response choices near feedback time 
The primate prefrontal cortex contributes to stimulus-guided behavior, but the functional specializations among its areas remain uncertain. To better understand such specializations, we contrasted neuronal activity in the dorsolateral prefrontal cortex (PFdl) and the orbital prefrontal cortex (PFo). The task required rhesus monkeys to use a visual cue to choose a saccade target. Some cues instructed the monkeys to repeat their most recent response; others instructed them to change it. Responses were followed by feedback: fluid reward if correct, visual feedback if incorrect. Previous studies, using different tasks, have reported that PFo neurons did not encode responses. We found PFo did encode responses in this task, but only near feedback time, after the response had been completed. PFdl differed from PFo in several respects. As reported previously, some PFdl neurons encoded responses from the previous trial and others encoded planned responses. PFo neurons did not have these properties. After feedback, PFdl encoded rewarded responses better than unrewarded ones and thus combined response and outcome information. PFo, in contrast, encoded the responses chosen, rewarded or not. These findings suggest that PFdl and PFo contribute differently to response knowledge, with PFo using an outcome-independent signal to monitor current responses at feedback time.
doi:10.1523/JNEUROSCI.5777-08.2009
PMCID: PMC2684962  PMID: 19244532
Decision; feedback; monitoring; evaluation; frontal lobe; prefrontal cortex
9.  Contributions of Orbitofrontal and Lateral Prefrontal Cortices to Economic Choice and the Good-to-action Transformation 
Neuron  2014;81(5):1140-1151.
SUMMARY
Previous work indicates that economic decisions can be made independently of the visuo-motor contingencies of the choice task (space of goods). However, the neuronal mechanisms through which the choice outcome (the chosen good) is transformed into a suitable action plan remain poorly understood. Here we show that neurons in lateral prefrontal cortex reflect the early stages of this good-to-action transformation. Monkeys chose between different juices. The experimental design dissociated in space and time the presentation of the offers and the saccade targets associated with them. We recorded from the orbital, ventrolateral and dorsolateral prefrontal cortices (OFC, LPFCv and LPFCd, respectively). Prior to target presentation, neurons in both LPFCv and LPFCd encoded the choice outcome in goods space. After target presentation, they gradually came to encode the location of the targets and the upcoming action plan. Consistent with the anatomical connectivity, all spatial and action-related signals emerged in LPFCv before LPFCd.
doi:10.1016/j.neuron.2014.01.008
PMCID: PMC3951647  PMID: 24529981
10.  How do we think machines think? An fMRI study of alleged competition with an artificial intelligence 
Mentalizing is defined as the inference of mental states of fellow humans, and is a particularly important skill for social interactions. Here we assessed whether activity in brain areas involved in mentalizing is specific to the processing of mental states or can be generalized to the inference of non-mental states by comparing brain responses during the interaction with an intentional and an artificial agent. Participants were scanned using fMRI during interactive rock-paper-scissors games while believing their opponent was a fellow human (Intentional agent, Int), a humanoid robot endowed with an artificial intelligence (Artificial agent, Art), or a computer playing randomly (Random agent, Rnd). Participants' subjective reports indicated that they adopted different stances against the three agents. The contrast of brain activity during interaction with the artificial and the random agents didn't yield any cluster at the threshold used, suggesting the absence of a reproducible stance when interacting with an artificial intelligence. We probed response to the artificial agent in regions of interest corresponding to clusters found in the contrast between the intentional and the random agents. In the precuneus involved in working memory, the posterior intraparietal suclus, in the control of attention and the dorsolateral prefrontal cortex, in executive functions, brain activity for Art was larger than for Rnd but lower than for Int, supporting the intrinsically engaging nature of social interactions. A similar pattern in the left premotor cortex and anterior intraparietal sulcus involved in motor resonance suggested that participants simulated human, and to a lesser extend humanoid robot actions, when playing the game. Finally, mentalizing regions, the medial prefrontal cortex and right temporoparietal junction, responded to the human only, supporting the specificity of mentalizing areas for interactions with intentional agents.
doi:10.3389/fnhum.2012.00103
PMCID: PMC3347624  PMID: 22586381
social cognition; neuroscience; artificial intelligence; fMRI
11.  Converging Structural and Functional Connectivity of Orbitofrontal, Dorsolateral Prefrontal, and Posterior Parietal Cortex in the Human Striatum 
The Journal of Neuroscience  2015;35(9):3865-3878.
Modification of spatial attention via reinforcement learning (Lee and Shomstein, 2013) requires the integration of reward, attention, and executive processes. Corticostriatal pathways are an ideal neural substrate for this integration because these projections exhibit a globally parallel (Alexander et al., 1986), but locally overlapping (Haber, 2003), topographical organization. Here we explore whether there are unique striatal regions that exhibit convergent anatomical connections from orbitofrontal cortex, dorsolateral prefrontal cortex, and posterior parietal cortex. Deterministic fiber tractography on diffusion spectrum imaging data from neurologically healthy adults (N = 60) was used to map frontostriatal and parietostriatal projections. In general, projections from cortex were organized according to both a medial–lateral and a rostral–caudal gradient along the striatal nuclei. Within rostral aspects of the striatum, we identified two bilateral convergence zones (one in the caudate nucleus and another in the putamen) that consisted of voxels with unique projections from orbitofrontal cortex, dorsolateral prefrontal cortex, and parietal regions. The distributed cortical connectivity of these striatal convergence zones was confirmed with follow-up functional connectivity analysis from resting state fMRI data, in which a high percentage of structurally connected voxels also showed significant functional connectivity. The specificity of this convergent architecture to these regions of the rostral striatum was validated against control analysis of connectivity within the motor putamen. These results delineate a neurologically plausible network of converging corticostriatal projections that may support the integration of reward, executive control, and spatial attention that occurs during spatial reinforcement learning.
doi:10.1523/JNEUROSCI.2636-14.2015
PMCID: PMC4461697  PMID: 25740516
diffusion imaging and fMRI; executive function; reinforcement learning; reward; spatial attention; striatum
12.  Valuation of uncertain and delayed rewards in primate prefrontal cortex 
Humans and animals often must choose between rewards that differ in their qualities, magnitudes, immediacy, and likelihood, and must estimate these multiple reward parameters from their experience. However, the neural basis for such complex decision making is not well understood. To understand the role of the primate prefrontal cortex in determining the subjective value of delayed or uncertain reward, we examined the activity of individual prefrontal neurons during an inter-temporal choice task and a computer-simulated competitive game. Consistent with the findings from previous studies in humans and other animals, the monkey’s behaviors during inter-temporal choice were well accounted for by a hyperbolic discount function. In addition, the activity of many neurons in the lateral prefrontal cortex reflected the signals related to the magnitude and delay of the reward expected from a particular action, and often encoded the difference in temporally discounted values that predicted the animal’s choice. During a computerized matching pennies game, the animals approximated the optimal strategy, known as Nash equilibrium, using a reinforcement learning algorithm. We also found that many neurons in the lateral prefrontal cortex conveyed the signals related to the animal’s previous choices and their outcomes, suggesting that this cortical area might play an important role in forming associations between actions and their outcomes. These results show that the primate lateral prefrontal cortex plays a central role in estimating the values of alternative actions based on multiple sources of information.
doi:10.1016/j.neunet.2009.03.010
PMCID: PMC2693219  PMID: 19375276
game theory; inter-temporal choice; reinforcement learning; utility theory; temporal discounting
13.  Differential Contributions of Dorso-Ventral and Rostro-Caudal Prefrontal White Matter Tracts to Cognitive Control in Healthy Older Adults 
PLoS ONE  2013;8(12):e81410.
Prefrontal cortex mediates cognitive control by means of circuitry organized along dorso-ventral and rostro-caudal axes. Along the dorso-ventral axis, ventrolateral PFC controls semantic information, whereas dorsolateral PFC encodes task rules. Along the rostro-caudal axis, anterior prefrontal cortex encodes complex rules and relationships between stimuli, whereas posterior prefrontal cortex encodes simple relationships between stimuli and behavior. Evidence of these gradients of prefrontal cortex organization has been well documented in fMRI studies, but their functional correlates have not been examined with regard to integrity of underlying white matter tracts. We hypothesized that (a) the integrity of specific white matter tracts is related to cognitive functioning in a manner consistent with the dorso-ventral and rostro-caudal organization of the prefrontal cortex, and (b) this would be particularly evident in healthy older adults. We assessed three cognitive processes that recruit the prefrontal cortex and can distinguish white matter tracts along the dorso-ventral and rostro-caudal dimensions –episodic memory, working memory, and reasoning. Correlations between cognition and fractional anisotropy as well as fiber tractography revealed: (a) Episodic memory was related to ventral prefrontal cortex-thalamo-hippocampal fiber integrity; (b) Working memory was related to integrity of corpus callosum body fibers subserving dorsolateral prefrontal cortex; and (c) Reasoning was related to integrity of corpus callosum body fibers subserving rostral and caudal dorsolateral prefrontal cortex. These findings confirm the ventrolateral prefrontal cortex's role in semantic control and the dorsolateral prefrontal cortex's role in rule-based processing, in accordance with the dorso-ventral prefrontal cortex gradient. Reasoning-related rostral and caudal superior frontal white matter may facilitate different levels of task rule complexity. This study is the first to demonstrate dorso-ventral and rostro-caudal prefrontal cortex processing gradients in white matter integrity.
doi:10.1371/journal.pone.0081410
PMCID: PMC3846728  PMID: 24312550
14.  Time-dependent changes in human cortico-spinal excitability reveal value-based competition for action during decision processing 
Our choices often require appropriate actions in order to obtain a preferred outcome, but the neural underpinnings that link decision making and action selection remain largely undetermined. Recent theories propose that action selection occurs simultaneously, i.e. parallel in time, with the decision process. Specifically, it is thought that action selection in motor regions originates from a competitive process which is gradually biased by evidence signals originating in other regions, such as those specialized in value computations. Biases reflecting the evaluation of choice options should thus emerge in the motor system before the decision process is complete. Using transcranial magnetic stimulation, we sought direct physiological evidence for this prediction by measuring changes in cortico-spinal excitability in human motor cortex during value-based decisions. We found that excitability for chosen versus unchosen actions distinguishes the forthcoming choice before completion of the decision process. Both excitability and reaction times varied as a function of the subjective value-difference between chosen and unchosen actions, consistent with this effect being value-driven. This relationship was not observed in the absence of a decision. Our data provide novel evidence in humans that internally generated value-based decisions influence the competition between action representations in motor cortex before the decision process is complete. This is incompatible with models of serial processing of stimulus, decision, and action.
doi:10.1523/JNEUROSCI.0270-12.2012
PMCID: PMC3399779  PMID: 22699917
15.  Ethanol self-administration modulation of NMDA receptor subunit and related synaptic protein mRNA expression in prefrontal cortical fields 
Brain Research  2010;1318:144-154.
Background
Functional impairment of the orbital and medial prefrontal cortex underlies deficits in executive control that characterize addictive disorders, including alcohol addiction. Previous studies indicate that alcohol alters glutamate neurotransmission and one substrate of these effects may be through the reconfiguration of the subunits constituting ionotropic glutamate receptor (iGluR) complexes. Glutamatergic transmission is integral to cortico-cortical and cortico-subcortical communication, and alcohol-induced changes in the abundance of the receptor subunits and/or their splice variants may result in critical functional impairments of prefrontal cortex in the alcohol-addicted state.
Methods and results
The effects of chronic ethanol self-administration on glutamate receptor ionotropic NMDA (GRIN), as well as GRIN1 splice variant mRNA expression was studied in the orbitofrontal cortex (OFC; Area 13), dorsolateral prefrontal cortex (DLPFC; Area 46) and anterior cingulate cortex (ACC; Area 24) of male cynomolgus monkeys. Chronic ethanol self-administration resulted in significant changes in the expression of NMDA subunit mRNA expression in the DLPFC and OFC, but not the ACC. In DLPFC, the overall expression of NMDA subunits was significantly decreased in ethanol treated monkeys. Slight but significant changes were observed for synaptic associated protein 102 kD (SAP102) and neuronal nitric oxide synthase (nNOS) mRNAs. In OFC, the NMDAR1 variant GRIN1-1 was reduced while GRIN1-2 was increased. Furthermore, no significant changes in GFAP protein levels were observed in either the DLPFC or OFC.
Conclusion
Results from these studies provide the first demonstration of post-transcriptional regulation of iGluR subunits in the primate brain following long-term ethanol self-administration. Furthermore, changes in these transcripts do not appear to reflect changes in glial activation or loss. Further studies examining the expression and cellular localization of subunit proteins and receptor pharmacology would shed more light on the findings reported here.
doi:10.1016/j.brainres.2009.12.050
PMCID: PMC3272763  PMID: 20043891
Ethanol; Glutamate; messenger RNA; Prefrontal Cortex; qPCR; Primate
16.  Differential effects of amygdala, orbital prefrontal cortex and prelimbic cortex lesions on goal-directed behavior in rhesus macaques 
We assessed the involvement of the orbital prefrontal cortex (PFo), the prelimbic region of the medial prefrontal cortex (PL), and the amygdala in goal-directed behavior. Rhesus monkeys were trained on a task in which two different instrumental responses were linked to two different outcomes. One response, called ‘Tap’, required the monkeys to repeatedly touch a colored square on a video monitor, to produce one kind of food reward. The other response, called ‘Hold’, required persistent contact of an identical stimulus, and it produced a different kind of food reward. Following training, we assessed the effects of satiety-specific reinforcer devaluation as a way to probe each monkey’s use of goal-directed behavior. In this procedure, monkeys were allowed to consume one of the two foods to satiety, and were then tested for Tap/Hold preference under extinction. Unoperated control monkeys showed a reduction in the response associated with obtaining the devalued food, called the devaluation effect, a hallmark of goal-directed behavior. Monkeys with bilateral lesions of PFo or the amygdala exhibited significantly reduced devaluation effects. Results from monkeys with PL lesions were equivocal. We conclude that both PFo and the amygdala play a significant role in goal-directed behavior in monkeys. Notably, the findings for PFo challenge the idea that orbital and medial prefrontal regions are exclusively dedicated to object- and action-based processes, respectively.
doi:10.1523/JNEUROSCI.4374-12.2013
PMCID: PMC3711145  PMID: 23426666
17.  Topography of connections between human prefrontal cortex and mediodorsal thalamus studied with diffusion tractography 
Neuroimage  2010;51(2):555-564.
Studies in monkeys show clear anatomical and functional distinctions among networks connecting with subregions within the prefrontal cortex. Three such networks are centered on lateral orbitofrontal cortex, medial frontal and cingulate cortex, and lateral prefrontal cortex and all have been identified with distinct cognitive roles. Although these areas differ in a number of their cortical connections, some of the first anatomical evidence for these networks came from tracer studies demonstrating their distinct patterns of connectivity with the mediodorsal (MD) nucleus of the thalamus. Here, we present evidence for a similar topography of MD thalamus prefrontal connections, using non-invasive imaging and diffusion tractography (DWI–DT) in human and macaque. DWI–DT suggested that there was a high probability of interconnection between medial MD and lateral orbitofrontal cortex, between caudodorsal MD and medial frontal/cingulate cortex, and between lateral MD and lateral prefrontal cortex, in both species. Within the lateral prefrontal cortex a dorsolateral region (the principal sulcus in the macaque and middle frontal gyrus in the human) was found to have a high probability of interconnection with the MD region between the regions with a high probability of interconnection with other parts of the lateral prefrontal cortex and with the lateral orbitofrontal cortex. In addition to suggesting that the thalamic connectivity in the macaque is a good guide to human prefrontal cortex, and therefore that there are likely to be similarities in the cognitive roles played by the prefrontal areas in both species, the present results are also the first to provide insight into the topography of projections of an individual thalamic nucleus in the human brain.
doi:10.1016/j.neuroimage.2010.02.062
PMCID: PMC2877805  PMID: 20206702
Anatomy; DTI; Human; Macaque; Thalamus
18.  The contribution of ventrolateral and dorsolateral prefrontal cortex to response reversal 
Behavioural brain research  2007;187(1):80-87.
Studies investigating response reversal consistently implicate regions of medial and lateral prefrontal cortex when reinforcement contingencies change. However, it is unclear from these studies how these regions give rise to the individual components of response reversal, such as reinforcement value encoding, response inhibition, and response change. Here we report a novel instrumental learning task designed to determine whether regions implicated in processing reversal errors are uniquely involved in this process, or whether they play a more general role in representing response competition, reinforcement value, or punishment value in the absence of demands for response change. In line with previous findings, reversal errors activated orbitofrontal cortex, dorsomedial prefrontal cortex, ventrolateral prefrontal cortex, caudate, and dorsolateral prefrontal cortex. These regions also showed increased activity to errors in the absence of contingency changes. In addition, ventrolateral PFC, caudate, and dorsolateral PFC each exhibited increased activity following correct reversals. Activity in these regions was not significantly modulated by changes in reinforcement value that were not sufficient to make an alternative response advantageous. These data do not support punishment-processing or prepotent reponse inhibition accounts of ventrolateral prefrontal cortex function. Instead, they support recent conceptualizations of ventrolateral prefrontal cortex function that implicate this region in resolving response competition by manipulating the representation of either motor response options, or object features. These data also suggest that dorsolateral prefrontal cortex plays a role in reversal learning, probably through top down attentional control of object or reinforcement features when task demands increase.
doi:10.1016/j.bbr.2007.08.034
PMCID: PMC2752857  PMID: 17950474
Response reversal; affective shift; response competition; ventrolateral prefrontal cortex; decision-making
19.  Influence of monkey dorsolateral prefrontal and posterior parietal activity on behavioral choice during attention tasks 
The European journal of neuroscience  2014;40(6):2910-2921.
The dorsolateral prefrontal and the posterior parietal cortex have both been implicated in the guidance of visual attention. Traditionally, posterior parietal cortex has been thought to guide visual bottom-up attention, whereas prefrontal cortex to bias attention through top-down information. More recent studies suggest a parallel time course of activation of the two areas in bottom-up attention tasks, suggesting a common involvement, though these results do not necessarily imply identical roles, either. To address the specific roles of the two areas, we examined the influence of neuronal activity recorded from the prefrontal and parietal cortex of monkeys as they performed attention tasks based on choice probability and correlation between reaction time and neuronal activity. The results revealed that posterior parietal but not dorsolateral prefrontal activity correlated with behavioral choice during the fixation period, prior to the appearance of the stimulus, resembling a bias factor. This preferential influence of posterior parietal activity on behavior was transient, so that dorsolateral prefrontal activity predicted choice after the appearance of the stimulus. Additionally, reaction time was better predicted by posterior parietal activity. These findings confirm an involvement of both dorsolateral prefrontal and posterior parietal cortex in the bottom-up guidance of visual attention but indicate different roles of the two areas in the guidance of attention and a dynamic time course of their effects, influencing behavior at different stages of the task.
doi:10.1111/ejn.12662
PMCID: PMC4172489  PMID: 24964224
monkey; neurophysiology; principal sulcus; intraparietal sulcus
20.  Neuroimaging of Goal-Directed Behavior in Midlife Women 
Nursing Research  2014;63(6):388-396.
Background
Motivational interventions to improve health behaviors based on conventional cognitive and behavioral theories have been extensively studied; however, advances in neuroimaging technology make it possible to assess the neurophysiological basis of health behaviors, such as physical activity. The goals of this approach are to support new interventions to achieve optimal outcomes.
Objectives
This study used functional magnetic resonance imaging (fMRI) to assess differences in brain responses in healthy weight to obese midlife women during a goal-directed decision task.
Methods
Thirty nondiabetic, midlife (age 47–55 years) women with body mass index (BMI) ranging from 18.5 to 40 kg/m2 were recruited. A descriptive, correlational design was used to assess the relationship between brain activations and weight status. Participants underwent a goal-directed behavior task in the fMRI scanner consisting of a learning and implementation phase. The task was designed to assess both goal-directed and habitual behaviors. One participant was omitted from the analysis because of excessive motion (>4 mm), and six were omitted because of fewer than 50% correct responses on the exit survey. Four participants developed claustrophobia in the scanner and were disqualified from further participation. The remaining 19 participants were included in the final analysis.
Results
Brain responses while participants learned goal-directed behavior showed a positive correlation with BMI in the dorsomedial prefrontal cortex (dmPFC) and a negative correlation with BMI in the insula. During the implementation of goal-directed behavior, brain responses in the dorsolateral prefrontal cortex (dlPFC) negatively correlated with BMI.
Discussion
These results indicate that overweight women activate regions associated with cognitive control to a greater degree than healthy weight women during goal-directed learning. The brain regions activated (dmPFC, dlPFC, insula) are associated with cognitive control and self-regulation. On the other hand, healthy weight women activate regions associated with emotion processing, planning, and self-regulation (lateral orbitofrontal cortex, anterior insula) to a greater degree than overweight women during goal-directed learning and implementation of goal-directed behavior. Overweight women activate cognitive control regions while learning associations between actions and outcomes; however, this is not the case during the implementation phase—which may make it more difficult to transform goals into action (e.g., maintain physical activity over time). Overall, these results indicate that overweight midlife women respond differently during learning and implementation of actions that lead to positive outcomes during a general test of goal-directed behavior. Future study is needed to assess the transfer of goal-directed and habitual behavior to specific aspects of energy balance to improve health outcomes.
doi:10.1097/NNR.0000000000000060
PMCID: PMC4213232  PMID: 25186027
fMRI; health behavior; neuroimaging; neurophysiology; obesity; women’s health
21.  Neuroimaging of Goal-Directed Behavior in Midlife Women 
Nursing research  2014;63(6):388-396.
Background
Motivational interventions to improve health behaviors based on conventional cognitive and behavioral theories have been extensively studied; however, advances in neuroimaging technology make it possible to assess the neurophysiological basis of health behaviors, such as physical activity. The goals of this approach are to support new interventions to achieve optimal outcomes.
Objectives
This study used functional magnetic resonance imaging (fMRI) to assess differences in brain responses in healthy weight to obese midlife women during a goal-directed decision task.
Methods
Thirty nondiabetic, midlife (age 47-55 years) women with body mass index (BMI) ranging from 18.5 to 40 kg/m2 were recruited. A descriptive, correlational design was used to assess the relationship between brain activations and weight status. Participants underwent a goal-directed behavior task in the fMRI scanner consisting of a learning and implementation phase. The task was designed to assess both goal-directed and habitual behaviors. One participant was omitted from the analysis due to excessive motion (> 4 mm), and six were omitted due to fewer than 50% correct responses on the exit survey. Four participants developed claustrophobia in the scanner and were disqualified from further participation. The remaining 19 participants were included in the final analysis.
Results
Brain responses while participants learned goal-directed behavior showed a positive correlation with BMI in the dorsal medial prefrontal cortex (dmPFC) and a negative correlation with BMI in the insula. During the implementation of goal-directed behavior, brain responses in the dorsolateral prefrontal cortex (dlPFC) negatively correlated with BMI.
Discussion
These results indicate that overweight women activate regions associated with cognitive control to a greater degree than healthy weight women during goal-directed learning. The brain regions activated (dmPFC, dlPFC, insula) are associated with cognitive control and self-regulation. On the other hand, healthy weight women activate regions associated with emotion processing, planning, and self-regulation (lateral orbitofrontal cortex, anterior insula) to a greater degree than overweight women during goal-directed learning and implementation of goal-directed behavior. Overweight women activate cognitive control regions while learning associations between actions and outcomes; however, this is not the case during the implementation phase—which may make it more difficult to transform goals into action (e.g., maintain PA over time). Overall, these results indicate that overweight midlife women respond differently during learning and implementation of actions that lead to positive outcomes during a general test of goal-directed behavior. Future study is needed to assess the transfer of goal-directed and habitual behavior to specific aspects of energy balance to improve health outcomes.
doi:10.1097/NNR.0000000000000060
PMCID: PMC4213232  PMID: 25186027
fMRI; health behavior; neuroimaging; neurophysiology; obesity; women's health
22.  Neuroscientific Model of Motivational Process 
Considering the neuroscientific findings on reward, learning, value, decision-making, and cognitive control, motivation can be parsed into three sub processes, a process of generating motivation, a process of maintaining motivation, and a process of regulating motivation. I propose a tentative neuroscientific model of motivational processes which consists of three distinct but continuous sub processes, namely reward-driven approach, value-based decision-making, and goal-directed control. Reward-driven approach is the process in which motivation is generated by reward anticipation and selective approach behaviors toward reward. This process recruits the ventral striatum (reward area) in which basic stimulus-action association is formed, and is classified as an automatic motivation to which relatively less attention is assigned. By contrast, value-based decision-making is the process of evaluating various outcomes of actions, learning through positive prediction error, and calculating the value continuously. The striatum and the orbitofrontal cortex (valuation area) play crucial roles in sustaining motivation. Lastly, the goal-directed control is the process of regulating motivation through cognitive control to achieve goals. This consciously controlled motivation is associated with higher-level cognitive functions such as planning, retaining the goal, monitoring the performance, and regulating action. The anterior cingulate cortex (attention area) and the dorsolateral prefrontal cortex (cognitive control area) are the main neural circuits related to regulation of motivation. These three sub processes interact with each other by sending reward prediction error signals through dopaminergic pathway from the striatum and to the prefrontal cortex. The neuroscientific model of motivational process suggests several educational implications with regard to the generation, maintenance, and regulation of motivation to learn in the learning environment.
doi:10.3389/fpsyg.2013.00098
PMCID: PMC3586760  PMID: 23459598
motivation; neuroeducation; educational neuroscience; reward; value; goal; decision-making; self-regulation
23.  Prefrontal Neurons Encode Actions and Outcomes in Conjunction with Spatial Location in Rats Performing a Dynamic Delayed Non-Match to Position Task 
PLoS ONE  2016;11(2):e0149019.
To respond adaptively to change organisms must utilize information about recent events and environmental context to select actions that are likely to produce favorable outcomes. We developed a dynamic delayed nonmatching to position task to study the influence of spatial context on event-related activity of medial prefrontal cortex neurons during reinforcement-guided decision-making. We found neurons with responses related to preparation, movement, lever press responses, reinforcement, and memory delays. Combined event-related and video tracking analyses revealed variability in spatial tuning of neurons with similar event-related activity. While all correlated neurons exhibited spatial tuning broadly consistent with relevant task events, for instance reinforcement-related activity concentrated in locations where reinforcement was delivered, some had elevated activity in more specific locations, for instance reinforcement-related activity in one of several locations where reinforcement was delivered. Timing analyses revealed a limited set of distinct response types with activity time-locked to critical behavioral events that represent the temporal organization of dDNMTP trials. Our results suggest that reinforcement-guided decision-making emerges from discrete populations of medial prefrontal neurons that encode information related to planned or ongoing movements and actions and anticipated or actual action-outcomes in conjunction with information about spatial context.
doi:10.1371/journal.pone.0149019
PMCID: PMC4743997  PMID: 26848579
24.  Increased Firing Irregularity as an Emergent Property of Neural-State Transition in Monkey Prefrontal Cortex 
PLoS ONE  2013;8(12):e80906.
Flexible behaviors are organized by complex neural networks in the prefrontal cortex. Recent studies have suggested that such networks exhibit multiple dynamical states, and can switch rapidly from one state to another. In many complex systems such as the brain, the early-warning signals that may predict whether a critical threshold for state transitions is approaching are extremely difficult to detect. We hypothesized that increases in firing irregularity are a crucial measure for predicting state transitions in the underlying neuronal circuits of the prefrontal cortex. We used both experimental and theoretical approaches to test this hypothesis. Experimentally, we analyzed activities of neurons in the prefrontal cortex while monkeys performed a maze task that required them to perform actions to reach a goal. We observed increased firing irregularity before the activity changed to encode goal-to-action information. Theoretically, we constructed theoretical generic neural networks and demonstrated that changes in neuronal gain on functional connectivity resulted in a loss of stability and an altered state of the networks, accompanied by increased firing irregularity. These results suggest that assessing the temporal pattern of neuronal fluctuations provides important clues regarding the state stability of the prefrontal network. We also introduce a novel scheme that the prefrontal cortex functions in a metastable state near the critical point of bifurcation. According to this scheme, firing irregularity in the prefrontal cortex indicates that the system is about to change its state and the flow of information in a flexible manner, which is essential for executive functions. This metastable and/or critical dynamical state of the prefrontal cortex may account for distractibility and loss of flexibility in the prefrontal cortex in major mental illnesses such as schizophrenia.
doi:10.1371/journal.pone.0080906
PMCID: PMC3857743  PMID: 24349020
25.  The Good, the Bad, and the Irrelevant: Neural Mechanisms of Learning Real and Hypothetical Rewards and Effort 
The Journal of Neuroscience  2015;35(32):11233-11251.
Natural environments are complex, and a single choice can lead to multiple outcomes. Agents should learn which outcomes are due to their choices and therefore relevant for future decisions and which are stochastic in ways common to all choices and therefore irrelevant for future decisions between options. We designed an experiment in which human participants learned the varying reward and effort magnitudes of two options and repeatedly chose between them. The reward associated with a choice was randomly real or hypothetical (i.e., participants only sometimes received the reward magnitude associated with the chosen option). The real/hypothetical nature of the reward on any one trial was, however, irrelevant for learning the longer-term values of the choices, and participants ought to have only focused on the informational content of the outcome and disregarded whether it was a real or hypothetical reward. However, we found that participants showed an irrational choice bias, preferring choices that had previously led, by chance, to a real reward in the last trial. Amygdala and ventromedial prefrontal activity was related to the way in which participants' choices were biased by real reward receipt. By contrast, activity in dorsal anterior cingulate cortex, frontal operculum/anterior insula, and especially lateral anterior prefrontal cortex was related to the degree to which participants resisted this bias and chose effectively in a manner guided by aspects of outcomes that had real and more sustained relationships with particular choices, suppressing irrelevant reward information for more optimal learning and decision making.
SIGNIFICANCE STATEMENT In complex natural environments, a single choice can lead to multiple outcomes. Human agents should only learn from outcomes that are due to their choices, not from outcomes without such a relationship. We designed an experiment to measure learning about reward and effort magnitudes in an environment in which other features of the outcome were random and had no relationship with choice. We found that, although people could learn about reward magnitudes, they nevertheless were irrationally biased toward repeating certain choices as a function of the presence or absence of random reward features. Activity in different brain regions in the prefrontal cortex either reflected the bias or reflected resistance to the bias.
doi:10.1523/JNEUROSCI.0396-15.2015
PMCID: PMC4532756  PMID: 26269633
effort; frontal pole; hypothetical; learning; reward; vmPFC

Results 1-25 (1695366)