In this article, the authors show that the neural representation for control of a neuroprosthetic device undergoes a process of consolidation, after which it is stable, readily recalled, and resistant to interference.
Cortical control of neuroprosthetic devices is known to require neuronal adaptations. It remains unclear whether a stable cortical representation for prosthetic function can be stored and recalled in a manner that mimics our natural recall of motor skills. Especially in light of the mixed evidence for a stationary neuron-behavior relationship in cortical motor areas, understanding this relationship during long-term neuroprosthetic control can elucidate principles of neural plasticity as well as improve prosthetic function. Here, we paired stable recordings from ensembles of primary motor cortex neurons in macaque monkeys with a constant decoder that transforms neural activity to prosthetic movements. Proficient control was closely linked to the emergence of a surprisingly stable pattern of ensemble activity, indicating that the motor cortex can consolidate a neural representation for prosthetic control in the presence of a constant decoder. The importance of such a cortical map was evident in that small perturbations to either the size of the neural ensemble or to the decoder could reversibly disrupt function. Moreover, once a cortical map became consolidated, a second map could be learned and stored. Thus, long-term use of a neuroprosthetic device is associated with the formation of a cortical map for prosthetic function that is stable across time, readily recalled, resistant to interference, and resembles a putative memory engram.
Brain–machine interfaces (BMIs) have the potential to revolutionize the care of neurologically impaired patients. Numerous studies have now shown the feasibility of direct “brain control” of a neuroprosthetic device, yet it remains unclear whether the neural representation for prosthetic control can become consolidated and remain stable over time. This question is especially intriguing given the evidence demonstrating that the neural representation for natural movements can be unstable: BMIs provide a window into the plasticity of cortical circuits in awake-behaving subjects. Here, we show that long-term neuroprosthetic control leads to the formation of a remarkably stable cortical map. Interestingly, this map has the putative attributes of a memory trace, namely, it is stable across time, readily recalled, and resistant to the storage of a second map. The demonstration of such a cortical map for prosthetic control indicates that neuroprosthetic devices could eventually be controlled through the effortless recall of motor memory in a manner that mimics natural skill acquisition and motor control.
Learning to perform a complex motor task requires the optimization of specific behavioral features to cope with task constraints. We show that when mice learn a novel motor paradigm they differentially refine specific behavioral features. Animals trained to perform progressively faster sequences of lever presses to obtain reinforcement reduced variability in sequence frequency, but increased variability in an orthogonal feature (sequence duration). Trial-to-trial variability of the activity of motor cortex and striatal projection neurons was higher early in training and subsequently decreased with learning, without changes in average firing rate. As training progressed, variability in corticostriatal activity became progressively more correlated with behavioral variability, but specifically with variability in frequency. Corticostriatal plasticity was required for the reduction in frequency variability, but not for variability in sequence duration. These data suggest that during motor learning corticostriatal dynamics encode the refinement of specific behavioral features that change the probability of obtaining outcomes.
Learning a new motor skill typically involves a degree of trial and error. Movements that achieve the desired outcome—from catching a ball to playing scales—are repeated and refined until they can be produced on demand. This process is made more difficult as the activity of individual neurons and muscle fibers can vary at random, and this reduces the ability to reproduce a given movement precisely and reliably.
It has been suggested that the motor system overcomes this problem by identifying those parts of a task that are essential for achieving the end goal, and then focusing resources on reducing the variability in the performance of those parts alone. Santos et al. now provide direct evidence in support of this proposal by recording the activity of neurons in motor regions of the mouse brain as the animals learn a lever pressing task.
By giving mice a food reward each time they pressed the lever four times in a row, Santos et al. trained the animals to press the lever in bouts. The experiment was then slightly modified, so that the mice had to perform the four lever presses more rapidly in order to earn their reward. Consistent with predictions, the average speed of lever pressing initially varied greatly, but this variability decreased as the animals learned the task. By contrast, the total duration of individual bouts of lever pressing—which depends largely on the number of times the mice press the lever—was just as variable after training as before.
A similar pattern emerged for the activity of individual motor neurons in the mouse brain. Whereas their activity initially varied greatly, this variability decreased over training. Moreover, it became increasingly linked to the variability in the speed of lever pressing, but not with the variability in the duration of individual bouts.
The work of Santos et al. has thus shown in real time how the motor system focuses its efforts on reducing variability in those specific parts of a task that are essential for achieving a goal. Without a process called corticostriatal plasticity, by which the motor system adapts, mice could not refine this variability.
striatum; cortex; action; skill; motor; plasticity; mouse
Mutations in the human FOXP2 gene cause impaired speech development and linguistic deficits, which have been best characterised in a large pedigree called the KE family. The encoded protein is highly conserved in many vertebrates and is expressed in homologous brain regions required for sensorimotor integration and motor-skill learning, in particular corticostriatal circuits. Independent studies in multiple species suggest that the striatum is a key site of FOXP2 action. Here, we used in vivo recordings in awake-behaving mice to investigate the effects of the KE-family mutation on the function of striatal circuits during motor-skill learning. We uncovered abnormally high ongoing striatal activity in mice carrying an identical mutation to that of the KE family. Furthermore, there were dramatic alterations in striatal plasticity during the acquisition of a motor skill, with most neurons in mutants showing negative modulation of firing rate, starkly contrasting with the predominantly positive modulation seen in control animals. We also observed striking changes in the temporal coordination of striatal firing during motor-skill learning in mutants. Our results indicate that FOXP2 is critical for the function of striatal circuits in vivo, which are important not only for speech but also for other striatal-dependent skills.
Foxp2; in vivo recording; KE family; motor-skill learning; speech and language; striatum
Dopamine (DA)-dependent corticostriatal plasticity is thought to underlie incremental procedural learning. A primary effector of striatal DA signaling is cAMP, yet its role in corticostriatal plasticity and striatum-dependent learning remains unclear. Here, we show that genetic deletion of a striatum-enriched isoform of adenylyl cyclase, AC5 (AC5KO), impairs two forms of striatum-dependent learning and corticostriatal synaptic plasticity. AC5KO mice were severely impaired in acquisition of a response strategy in the cross maze, a striatum dependent task requiring a correct body turn to find a goal arm. In addition, AC5KO mice were impaired in acquisition of a motor skill, as assessed by the accelerated rotarod. Slice electrophysiology revealed a deficit in corticostriatal LTD following high frequency stimulation of tissue from AC5KO mice. LTD was rescued by activation of either presynaptic cannabinoid type 1 (CB1) receptors, or postsynaptic metabotropic glutamate receptors (mGluRs), suggesting a postsynaptic role of AC5-cAMP, upstream of endocannabinoid release. In striatopallidal projecting medium spiny neurons (MSNs), DA D2 receptors are negatively coupled to cAMP production and activation of these receptors is required for endocannabinoid release and corticostriatal LTD. Recordings from striatopallidal neurons indicated that this is mediated by AC5, as co-activation of D2 and mGluR receptors could induce LTD in WT, but not in AC5KO neurons. To further examine the role of cAMP in corticostriatal plasticity, we elevated cAMP in striatal neurons of wild-type mice via the recording electrode. Under these conditions corticostriatal LTD was eliminated. Together, these data suggest an AC5-cAMP-endocannabinoid-CB1 signaling pathway in corticostriatal plasticity and striatum-dependent learning.
Adenylyl cyclase; striatum; motor learning; plasticity; dopamine; LTD
The striatum is the major input nucleus of basal ganglia, an ensemble of interconnected sub-cortical nuclei associated with fundamental processes of action-selection and procedural learning and memory. The striatum receives afferents from the cerebral cortex and the thalamus. In turn, it relays the integrated information towards the basal ganglia output nuclei through which it operates a selected activation of behavioral effectors. The striatal output neurons, the GABAergic medium-sized spiny neurons (MSNs), are in charge of the detection and integration of behaviorally relevant information. This property confers to the striatum the ability to extract relevant information from the background noise and select cognitive-motor sequences adapted to environmental stimuli. As long-term synaptic efficacy changes are believed to underlie learning and memory, the corticostriatal long-term plasticity provides a fundamental mechanism for the function of the basal ganglia in procedural learning. Here, we reviewed the different forms of spike-timing dependent plasticity (STDP) occurring at corticostriatal synapses. Most of the studies have focused on MSNs and their ability to develop long-term plasticity. Nevertheless, the striatal interneurons (the fast-spiking GABAergic, NO-synthase and cholinergic interneurons) also receive monosynaptic afferents from the cortex and tightly regulated corticostriatal information processing. Therefore, it is important to take into account the variety of striatal neurons to fully understand the ability of striatum to develop long-term plasticity. Corticostriatal STDP with various spike-timing dependence have been observed depending on the neuronal sub-populations and experimental conditions. This complexity highlights the extraordinary potentiality in term of plasticity of the corticostriatal pathway.
spike-timing dependent plasticity; corticostriatal; striatum; GABAergic interneurons; cholinergic interneurons; LTP; LTD; basal ganglia
Corticostriatal synapse plasticity of medium spiny neurons is regulated by glutamate input from the cortex and dopamine input from the substantia nigra. While cortical stimulation alone results in long-term depression (LTD), the combination with dopamine switches LTD to long-term potentiation (LTP), which is known as dopamine-dependent plasticity. LTP is also induced by cortical stimulation in magnesium-free solution, which leads to massive calcium influx through NMDA-type receptors and is regarded as calcium-dependent plasticity. Signaling cascades in the corticostriatal spines are currently under investigation. However, because of the existence of multiple excitatory and inhibitory pathways with loops, the mechanisms regulating the two types of plasticity remain poorly understood. A signaling pathway model of spines that express D1-type dopamine receptors was constructed to analyze the dynamic mechanisms of dopamine- and calcium-dependent plasticity. The model incorporated all major signaling molecules, including dopamine- and cyclic AMP-regulated phosphoprotein with a molecular weight of 32 kDa (DARPP32), as well as AMPA receptor trafficking in the post-synaptic membrane. Simulations with dopamine and calcium inputs reproduced dopamine- and calcium-dependent plasticity. Further in silico experiments revealed that the positive feedback loop consisted of protein kinase A (PKA), protein phosphatase 2A (PP2A), and the phosphorylation site at threonine 75 of DARPP-32 (Thr75) served as the major switch for inducing LTD and LTP. Calcium input modulated this loop through the PP2B (phosphatase 2B)-CK1 (casein kinase 1)-Cdk5 (cyclin-dependent kinase 5)-Thr75 pathway and PP2A, whereas calcium and dopamine input activated the loop via PKA activation by cyclic AMP (cAMP). The positive feedback loop displayed robust bi-stable responses following changes in the reaction parameters. Increased basal dopamine levels disrupted this dopamine-dependent plasticity. The present model elucidated the mechanisms involved in bidirectional regulation of corticostriatal synapses and will allow for further exploration into causes and therapies for dysfunctions such as drug addiction.
Recent brain imaging and neurophysiological studies suggest that the striatum, the start of the basal ganglia circuit, plays a major role in value-based decision making and behavioral disorders such as drug addiction. The plasticity of synaptic input from the cerebral cortex to output neurons of the striatum, which are medium spiny neurons, depends on interactions between glutamate input from the cortex and dopaminergic input from the midbrain. It also links sensory and cognitive states in the cortex with reward-oriented action outputs. The mechanisms involved in molecular cascades that transmit glutamate and dopamine inputs to changes in postsynaptic glutamate receptors are very complex and it is difficult to intuitively understand the mechanism. Therefore, a biochemical network model was constructed, and computer simulations were performed. The model reproduced dopamine-dependent and calcium-dependent forms of long-term depression (LTD) and potentiation (LTP) of corticostriatal synapses. Further in silico experiments revealed that a positive feedback loop formed by proteins, the protein specifically expressed in the striatum, served as the major switch for inducing LTD and LTP. This model could allow us to understand dynamic constraints in reward-dependent learning, as well as causes and therapies of dopamine-related disorders such as drug addiction.
Dopaminergic neurons in the ventral tegmental area, the major midbrain nucleus projecting to the motor cortex, play a key role in motor skill learning and motor cortex synaptic plasticity. Dopamine D1 and D2 receptor antagonists exert parallel effects in the motor system: they impair motor skill learning and reduce long-term potentiation. Traditionally, D1 and D2 receptor modulate adenylyl cyclase activity and cyclic adenosine monophosphate accumulation in opposite directions via different G-proteins and bidirectionally modulate protein kinase A (PKA), leading to distinct physiological and behavioral effects. Here we show that D1 and D2 receptor activity influences motor skill acquisition and long term synaptic potentiation via phospholipase C (PLC) activation in rat primary motor cortex. Learning a new forelimb reaching task is severely impaired in the presence of PLC, but not PKA-inhibitor. Similarly, long term potentiation in motor cortex, a mechanism involved in motor skill learning, is reduced when PLC is inhibited but remains unaffected by the PKA inhibitor. Skill learning deficits and reduced synaptic plasticity caused by dopamine antagonists are prevented by co-administration of a PLC agonist. These results provide evidence for a role of intracellular PLC signaling in motor skill learning and associated cortical synaptic plasticity, challenging the traditional view of bidirectional modulation of PKA by D1 and D2 receptors. These findings reveal a novel and important action of dopamine in motor cortex that might be a future target for selective therapeutic interventions to support learning and recovery of movement resulting from injury and disease.
Motor learning occurs through interactions between the cerebellar circuit and cellular plasticity at different sites. Previous work has established plasticity in brain slices and suggested plausible sites of behavioral learning. We now reveal what actually happens in the cerebellum during short-term learning. We monitor the expression of plasticity in the simple-spike firing of cerebellar Purkinje cells during trial-over-trial learning in smooth pursuit eye movements of monkeys. Our findings imply that: 1) a single complex-spike response driven by one instruction for learning causes short-term plasticity in a Purkinje cell’s mossy fiber/parallel-fiber input pathways; 2) complex-spike responses and simple-spike firing rate are correlated across the Purkinje cell population; and 3) simple-spike firing rate at the time of an instruction for learning modulates the probability of a complex-spike response, possibly through a disynaptic feedback pathway to the inferior olive. These mechanisms may participate in long-term motor learning.
Practice makes perfect in many areas of life, such as playing sport or even just drinking coffee from a cup without spilling any. Our brains can learn and improve these motor skills through trial, error and learning, with such “motor learning” depending on the cerebellum, a part of the brain that helps to coordinate all kinds of movements.
Motor learning is a product of the organization of the cerebellar circuit, which is well understood, and the “plasticity” in the synapses that determine how cerebellar neurons interact with each other. The cerebellum contains cells called Purkinje cells that receive distinctive inputs from two pathways: a pathway involving inputs from many parallel fibers, which convey signals related to sensory events or motor commands; and a pathway involving input from a single climbing-fiber, which conveys signals from a part of the brain called the inferior olive nucleus.
Research on slices of brain has revealed many sites and forms of cerebellar plasticity that could participate in motor learning. In one form of plasticity, the strength of the synapses between the parallel fibers and the Purkinje cell can be changed when a signal sent along the climbing fiber arrives the Purkinje cell.
Yang and Lisberger have now taken the next step by studying the cerebellum of a monkey as it performs a motor learning task. Remarkably these experiments show that the climbing fiber inputs cause plasticity of Purkinje cell activity, just as happens in the experiments on brain slices. Further, some learning in the cerebellum restricts further learning, so that the cerebellum puts boundaries on its own learning. Overall the results make clear how learning is a property of groups of neurons working together in a circuit, rather than simply of changes in the strength of specific synapses.
By shedding light on what happens in the cerebellum during short-term motor learning, the work of Yang and Lisberger will benefit efforts to understand how the cerebellum is involved in motor learning on all time scales.
non-human primate; smooth pursuit eye movements; climbing fiber; cerebellar learning; trial-over-trial learning; floccular complex; Other
The spinal cord participates in the execution of skilled movements by translating high-level cerebral motor representations into musculotopic commands. Yet, the extent to which motor skill acquisition relies on intrinsic spinal cord processes remains unknown. To date, attempts to address this question were limited by difficulties in separating spinal local effects from supraspinal influences through traditional electrophysiological and neuroimaging methods. Here, for the first time, we provide evidence for local learning-induced plasticity in intact human spinal cord through simultaneous functional magnetic resonance imaging of the brain and spinal cord during motor sequence learning. Specifically, we show learning-related modulation of activity in the C6–C8 spinal region, which is independent from that of related supraspinal sensorimotor structures. Moreover, a brain–spinal cord functional connectivity analysis demonstrates that the initial linear relationship between the spinal cord and sensorimotor cortex gradually fades away over the course of motor sequence learning, while the connectivity between spinal activity and cerebellum gains strength. These data suggest that the spinal cord not only constitutes an active functional component of the human motor learning network but also contributes distinctively from the brain to the learning process. The present findings open new avenues for rehabilitation of patients with spinal cord injuries, as they demonstrate that this part of the central nervous system is much more plastic than assumed before. Yet, the neurophysiological mechanisms underlying this intrinsic functional plasticity in the spinal cord warrant further investigations.
Simultaneous neuroimaging of brain and spinal cord reveals intrinsic plasticity in the spinal cord during motor sequence learning in humans, independent from that of related sensorimotor structures in the brain.
When we acquire a new motor skill—for example, learning how to play a musical instrument—new synaptic connections are induced in a distributed network of brain areas. There is ample evidence from human neuroimaging studies for this high plasticity of the brain, but what about the spinal cord, the main link between the brain and the peripheral nervous system? Literature on animal models has recently hinted that spinal cord neurons can learn during various conditioning paradigms. However, human learning models by tradition assume that the spinal cord acts as a passive relay of information from the cortex to the muscles. In this study, we simultaneously acquired functional images of both the brain and the cervical spinal cord through functional magnetic resonance imaging, and we provide evidence for local spinal cord plasticity during a well-studied motor learning task in humans. We also demonstrate a dynamic change in the interaction of the brain and spinal cord regions over the course of motor learning. The present findings have important clinical implications for rehabilitation of patients with spinal cord injuries, as they demonstrate that this part of the central nervous system is much more plastic than it was assumed before.
Motor dysfunction in Parkinson’s disease is believed to arise primarily from pathophysiology in the dorsal striatum and its related corticostriatal and thalamostriatal circuits during progressive dopamine denervation. One function of these circuits is to provide a filter that selectively facilitates or inhibits cortical activity to optimize cortical processing, making motor responses rapid and efficient. Corticostriatal synaptic plasticity mediates the learning that underlies this performance-optimizing filter. Under dopamine denervation, corticostriatal plasticity is altered, resulting in aberrant learning that induces inappropriate basal ganglia filtering that impedes rather than optimizes cortical processing. Human imaging suggests that increased cortical activity may compensate for striatal dysfunction in PD patients. In this Perspective article, we consider how aberrant learning at corticostriatal synapses may impair cortical processing and learning and undermine potential cortical compensatory mechanisms. Blocking or remediating aberrant corticostriatal plasticity may protect cortical function and support cortical compensatory mechanisms mitigating the functional decline associated with progressive dopamine denervation.
corticostriatal plasticity; striatopallidal pathway; dorsolateral striatum; cortical compensation; basal ganglia
Cues predictive of food availability are powerful modulators of appetite as well as food-seeking and ingestive behaviors. The neurobiological underpinnings of these conditioned responses are not well understood. Monitoring regional immediate early gene expression is a method used to assess alterations in neuronal metabolism resulting from upstream intracellular and extracellular signaling. Furthermore, assessing the expression of multiple immediate early genes offers a window onto the possible sequelae of exposure to food cues, since the function of each gene differs. We used immediate early gene and proenkephalin expression as a means of assessing food cue-elicited regional activation and alterations in functional connectivity within the forebrain.
Contextual cues associated with palatable food elicited conditioned motor activation and corticosterone release in rats. This motivational state was associated with increased transcription of the activity-regulated genes homer1a, arc, zif268, ngfi-b and c-fos in corticolimbic, thalamic and hypothalamic areas and of proenkephalin within striatal regions. Furthermore, the functional connectivity elicited by food cues, as assessed by an inter-regional multigene-expression correlation method, differed substantially from that elicited by neutral cues. Specifically, food cues increased cortical engagement of the striatum, and within the nucleus accumbens, shifted correlations away from the shell towards the core. Exposure to the food-associated context also induced correlated gene expression between corticostriatal networks and the basolateral amygdala, an area critical for learning and responding to the incentive value of sensory stimuli. This increased corticostriatal-amygdalar functional connectivity was absent in the control group exposed to innocuous cues.
The results implicate correlated activity between the cortex and the striatum, especially the nucleus accumbens core and the basolateral amygdala, in the generation of a conditioned motivated state that may promote excessive food intake. The upregulation of a number of genes in unique patterns within corticostriatal, thalamic, and hypothalamic networks suggests that food cues are capable of powerfully altering neuronal processing in areas mediating the integration of emotion, cognition, arousal, and the regulation of energy balance. As many of these genes play a role in plasticity, their upregulation within these circuits may also indicate the neuroanatomic and transcriptional correlates of extinction learning.
The design of efficient neuroprosthetic devices has become a major challenge for the long-term goal of restoring autonomy to motor-impaired patients. One approach for brain control of actuators consists in decoding the activity pattern obtained by simultaneously recording large neuronal ensembles in order to predict in real-time the subject's intention, and move the prosthesis accordingly. An alternative way is to assign the output of one or a few neurons by operant conditioning to control the prosthesis with rules defined by the experimenter, and rely on the functional adaptation of these neurons during learning to reach the desired behavioral outcome. Here, several motor cortex neurons were recorded simultaneously in head-fixed awake rats and were conditioned, one at a time, to modulate their firing rate up and down in order to control the speed and direction of a one-dimensional actuator carrying a water bottle. The goal was to maintain the bottle in front of the rat's mouth, allowing it to drink. After learning, all conditioned neurons modulated their firing rate, effectively controlling the bottle position so that the drinking time was increased relative to chance. The mean firing rate averaged over all bottle trajectories depended non-linearly on position, so that the mouth position operated as an attractor. Some modifications of mean firing rate were observed in the surrounding neurons, but to a lesser extent. Notably, the conditioned neuron reacted faster and led to a better control than surrounding neurons, as calculated by using the activity of those neurons to generate simulated bottle trajectories. Our study demonstrates the feasibility, even in the rodent, of using a motor cortex neuron to control a prosthesis in real-time bidirectionally. The learning process includes modifications of the activity of neighboring cortical neurons, while the conditioned neuron selectively leads the activity patterns associated with the prosthesis control.
brain-machine interface; neuroprosthesis; learning; neuronal plasticity
Compulsive behaviors in obsessive–compulsive disorder (OCD) may be related to deficits in reward processing mediated by corticostriatal circuitry, a brain network implicated in the pathophysiology of OCD. Performing compulsive actions can be perceived as a reward to OCD patients because it temporarily reduces the anxiety provoked by obsessions. Although most OCD literature provides evidence of altered regional activity in these corticostriatal circuits, very little is known about the connectivity between individual regions of the corticostriatal-limbic circuits, including the cognitive and affective neural circuitry associated with OCD. Thus, this study investigated the differences in functional connectivity (FC) patterns in this network during resting-state and incentive processing. Nineteen patients with OCD and 18 well-matched healthy controls were scanned during resting-state and a monetary incentive delay task (task state). FC was assessed using both voxel-wise and region-of-interest (ROI)-wise analyses. Voxel-wise FC analysis with the nucleus accumbens seed revealed that patients with OCD exhibited increased FC between the nucleus accumbens and the lateral orbitofrontal cortex during resting-state. Additionally, these patients showed decreased FC between the nucleus accumbens and limbic areas such as the amygdala during incentive processing. Exploratory ROI-wise FC analysis revealed that OCD patients demonstrated enhanced FC between the nucleus accumbens and the lateral orbitofrontal cortex and increased total connectivity of the lateral orbitofrontal cortex during resting-state. Additionally, patients showed alterations in FC between resting and task state. This study provides evidence that patients with OCD have altered FC in the corticostriatal-limbic network, particularly in striatal-amygdala and striatal-orbitofrontal circuitry, during incentive processing and resting-state. These findings also emphasize that functional connections in the network are modulated by affective/motivational states and further suggest that OCD patients may have abnormalities of such modulation in this network.
•Corticostriatal-limbic FC analysis of task-based and resting-state fMRI data in OCD•Dysfunctional connectivity in striatal–amygdala during reward task in OCD•Dysfunctional connectivity in striatal–orbitofrontal cortex at rest in OCD•FC levels in corticostriatal-limbic network are modulated by affective state.•OCD patients have deficits in this modulation of the corticostriatal-limbic network.
Corticostriatal circuitry; Functional connectivity; Obsessive–compulsive disorder; Reward; Resting-state
Abundant evidence points to a key role of dopamine in motor skill learning, although the underlying cellular and molecular mechanisms are still poorly understood. Here, we used a skilled-reaching paradigm to first examine changes in the expression of the plasticity-related gene Arc to map activity in cortico-striatal circuitry during different phases of motor skill learning in young animals. In the early phase, Arc mRNA was significantly induced in the medial prefrontal cortex (mPFC), cingulate cortex, primary motor cortex, and striatum. In the late phase, expression of Arc did not change in most regions, except in the mPFC and dorsal striatum. In the second series of experiments, we studied the learning-induced changes in the phosphorylation state of dopamine and cAMP-regulated phosphoprotein, 32k Da (DARPP-32). Western blot analysis of the phosphorylation state of DARPP-32 and its downstream target cAMP response element-binding protein (CREB) in the striatum revealed that the early, but not late, phase of motor skill learning was associated with increased levels of phospho-Thr34-DARPP-32 and phospho-Ser133-CREB. Finally, we used the DARPP-32 knock-in mice with a point mutation in the Thr34 regulatory site (i.e., protein kinase A site) to test the significance of this pathway in motor skill learning. In accordance with our hypothesis, inhibition of DARPP-32 activity at the Thr34 regulatory site strongly attenuated the motor learning rate and skilled reaching performance of mice. These findings suggest that the cAMP/PKA/DARPP-32 signaling pathway is critically involved in the acquisition of novel motor skills, and also demonstrate a dynamic shift in the contribution of cortico-striatal circuitry during different phases of motor skill learning.
Perceptual decisions are based on the activity of sensory cortical neurons, but how organisms learn to transform this activity into appropriate actions remains unknown. Projections from the auditory cortex to the auditory striatum carry information that drives decisions in an auditory frequency discrimination task1. To assess the role of these projections in learning, we developed a Channelrhodopsin-2-based assay to selectively probe for synaptic plasticity associated with corticostriatal neurons representing different frequencies. Here we report that learning this auditory discrimination preferentially potentiates corticostriatal synapses from neurons representing either high or low frequencies, depending on reward contingencies. We observed frequency-dependent corticostriatal potentiation in vivo over the course of training, and in vitro in striatal brain slices. Our findings suggest a model in which the corticostriatal synapses made by neurons tuned to different features of the sound are selectively potentiated to enable the learned transformation of sound into action.
The dorsal striatum is a large forebrain region involved in action initiation, timing, control, learning and memory. Learning and remembering skilled movement sequences requires the dorsal striatum, and striatal subregions participate in both goal-directed (action-outcome) and habitual (stimulus-response) learning. Modulation of synaptic transmission plays a large part in controlling input to as well as the output from striatal medium spiny projection neurons (MSNs). Synapses in this brain region are subject to short-term modulation, including allosteric alterations in ion channel function and prominent presynaptic inhibition. Two forms of long-term synaptic plasticity have also been observed in striatum, long-term potentiation (LTP) and long-term depression (LTD). LTP at glutamatergic synapses onto MSNs involves activation of NMDA-type glutamate receptors and D1 dopamine or A2A adenosine receptors. Expression of LTP appears to involve postsynaptic mechanisms. LTD at glutamatergic synapses involves retrograde endocannabinoid signaling stimulated by activation of metabotropic glutamate receptors (mGluRs) and D2 dopamine receptors. While postsynaptic mechanisms participate in LTD induction, maintained expression involves presynaptic mechanisms. A similar form of LTD has also been observed at GABAergic synapses onto MSNs. Studies have just begun to examine the roles of synaptic plasticity in striatal-based learning. Findings to date indicate that molecules implicated in induction of plasticity participate in these forms of learning. Neurotransmitter receptors involved in LTP induction are necessary for proper skill and goal-directed instrumental learning. Interestingly, receptors involved in LTP and LTD at glutamatergic synapses onto MSNs of the “indirect pathway” appear to have important roles in habit learning. More work is needed to reveal if and when synaptic plasticity occurs during learning and if so what molecules and cellular processes, both short- and long-term, contribute to this plasticity.
Long-term plasticity; Dopamine; Glutamate; Endocannabinoid; Instrumental learning; Skill Learning
Brain machine interfaces (BMIs) transform activity of neurons recorded in motor areas of the brain into movements of external actuators. Representation of movements by neuronal populations varies over time, during both voluntary limb movements and movements controlled through BMIs, due to motor learning, neuronal plasticity, and instability in recordings. To assure accurate BMI performance over long time spans, BMI decoders must adapt to these changes. We propose the Bayesian regression self-training method for updating the parameters of an unscented Kalman filter decoder. This novel paradigm uses the decoder’s output to periodically update the decoder’s neuronal tuning model in a Bayesian linear regression. We use two previously-known statistical formulations of Bayesian linear regression: (1) a joint formulation which allows fast and exact inference, and (2) a factorized formulation which allows the addition and temporary omission of neurons from updates, but requires approximate variational inference. To evaluate these methods, we performed off-line reconstructions and closed-loop experiments with Rhesus monkeys implanted cortically with micro-wire electrodes. Off-line reconstructions used data recorded in areas M1, S1, PMd, SMA, and PP of 3 monkeys while they controlled a cursor using a hand-held joystick. The Bayesian regression self-training updates significantly improved the accuracy of offline reconstructions compared to the same decoder without updates. We performed 11 sessions of real-time, closed-loop experiments with a monkey implanted in areas M1 and S1. These sessions spanned 29 days. The monkey controlled the cursor using the decoder with and without updates. The updates maintained control accuracy and did not require information about monkey hand movements, assumptions about desired movements, or knowledge of the intended movement goals as training signals. These results indicate that Bayesian regression self-training can maintain BMI control accuracy over long time periods, making clinical neuroprosthetics more viable.
brain machine interface; decoding; adaptive filtering
The frontal lobes may be organized hierarchically such that more rostral frontal regions modulate cognitive control operations in caudal regions. In our companion paper (Frank MJ, Badre D. 2011. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits I: computational analysis. 22:509–526), we provide novel neural circuit and algorithmic models of hierarchical cognitive control in cortico–striatal circuits. Here, we test key model predictions using functional magnetic resonance imaging (fMRI). Our neural circuit model proposes that contextual representations in rostral frontal cortex influence the striatal gating of contextual representations in caudal frontal cortex. Reinforcement learning operates at each level, such that the system adaptively learns to gate higher order contextual information into rostral regions. Our algorithmic Bayesian “mixture of experts” model captures the key computations of this neural model and provides trial-by-trial estimates of the learner’s latent hypothesis states. In the present paper, we used these quantitative estimates to reanalyze fMRI data from a hierarchical reinforcement learning task reported in Badre D, Kayser AS, D’Esposito M. 2010. Frontal cortex and the discovery of abstract action rules. Neuron. 66:315--326. Results validate key predictions of the models and provide evidence for an individual cortico–striatal circuit for reinforcement learning of hierarchical structure at a specific level of policy abstraction. These findings are initially consistent with the proposal that hierarchical control in frontal cortex may emerge from interactions among nested cortico–striatal circuits at different levels of abstraction.
basal ganglia; cognitive control; fMRI; prefrontal cortex, reinforcement learning
Action potentials are thought to be determinant for the induction of long-term synaptic plasticity, the cellular basis of learning and memory. However, neuronal activity does not lead systematically to an action potential but also, in many cases, to synaptic depolarizing subthreshold events. This is particularly exemplified in corticostriatal information processing. Indeed, the striatum integrates information from the whole cerebral cortex and, due to the membrane properties of striatal medium spiny neurons, cortical inputs do not systematically trigger an action potential but a wide range of subthreshold postsynaptic depolarizations. Accordingly, we have addressed the following question: does a brief subthreshold event act as a Hebbian signal and induce long-term synaptic efficacy changes?
Here, using perforated patch-clamp recordings on rat brain corticostriatal slices, we demonstrate, that brief (30 ms) subthreshold depolarizing events in quasi-coincidence with presynaptic activity can act as Hebbian signals and are sufficient to induce long-term synaptic plasticity at corticostriatal synapses. This “subthreshold-depolarization dependent plasticity” (SDDP) induces strong, significant and bidirectional long-term synaptic efficacy changes at a very high occurrence (81%) for time intervals between pre- and postsynaptic stimulations (Δt) of −110<Δt<+110 ms. Such subthreshold depolarizations are able to induce robust long-term depression (cannabinoid type-1 receptor-activation dependent) as well as long-term potentiation (NMDA receptor-activation dependent).
Our data show the existence of a robust, reliable and timing-dependent bidirectional long-term plasticity induced by brief subthreshold events paired with presynaptic activity. The existence of a subthreshold-depolarization dependent plasticity extends considerably, beyond the action potential, the neuron's capabilities to express long-term synaptic efficacy changes.
Changes in basal ganglia plasticity at the corticostriatal and thalamostriatal level are required for motor learning. Endocannabinoid-dependent long-term depression (eCB-LTD) is known to be a dominant form of synaptic plasticity expressed at these glutamatergic inputs; however, whether eCB-LTD can be induced at all inputs on all striatal neurons is still debatable. Using region-specific Cre mouse lines combined with optogenetic techniques, we directly investigated and distinguished between corticostriatal and thalamostriatal projections. We found that eCB-LTD was successfully induced at corticostriatal synapses, independent of postsynaptic striatal spiny projection neuron (SPN) subtype. Conversely, eCB-LTD was only nominally present at thalamostriatal synapses. This dichotomy was attributable to the minimal expression of cannabinoid type-1 (CB1) receptors on thalamostriatal terminals. Furthermore, co-activation of dopamine receptors on SPNs during LTD induction re-established SPN subtype-dependent eCB-LTD. Altogether, our findings lay the groundwork for understanding corticostriatal and thalamostriatal synaptic plasticity and for striatal eCB-LTD in motor learning.
Striatum; spiny projection neuron; glutamatergic synapse; corticostriatal synapse; thalamostriatal synapse; LTD; endocannabinoid; long-term plasticity
Dopamine is released in the striatum during development and impacts the activity of Protein Kinase A (PKA) in striatal spiny projection neurons (SPNs). We examined whether dopaminergic neuromodulation regulates activity-dependent glutamatergic synapse formation in the developing striatum. Systemic in vivo treatment with Gαs-coupled G-protein receptors (GPCRs) agonists enhanced excitatory synapses on direct pathway striatal spiny projection neurons (dSPNs), whereas rapid production of excitatory synapses on indirect pathway neurons (iSPNs) required the activation of Gαs GPCRs in SPNs of both pathways. Nevertheless, in vitro Gαs activation was sufficient to enhance spinogenesis induced by glutamate photolysis in both dSPNs and iSPNs, suggesting that iSPNs in intact neural circuits have additional requirements for rapid synaptic development. We evaluated the in vivo effects of enhanced glutamate release from corticostriatal axons and postsynaptic PKA and discovered a mechanism of developmental plasticity wherein rapid synaptogenesis is promoted by the coordinated actions of glutamate and postsynaptic Gαs-coupled receptors.
The brain is composed of intricate circuits of connected neurons that communicate via a combination of electrical and chemical signals. Some signals (referred to as excitatory signals) increase the probability that the neuron receiving the chemical message will produce an electrical impulse. On the other hand, inhibitory messages decrease the likelihood of this activity. Both of these kinds of signals are fast, and act over milliseconds. There is also a diverse set of slower signals, referred to as neuromodulation, which regulates the faster signals. A signaling chemical called dopamine is involved in neuromodulation and is essential for rewarding behavior and complex motor actions. The importance of dopamine is clear from the profound lack of movement seen in individuals with Parkinson’s disease, which is caused by the death of dopamine producing brain cells.
Many nerve endings from dopamine-releasing neurons connect to a part of the brain’s reward system called the striatum. The neurons in this region are organized into two pathways that have opposing impacts on behavior. Dopamine activates different kinds of receptors called “G protein-coupled dopamine receptors” on neurons from each pathway. This allows dopamine to alter the activity of a protein called Protein Kinase A (or PKA) and alter the signaling state of these neurons.
The impact of dopamine on neural circuits in adults has been extensively studied. However it was unknown whether dopamine might influence how neural circuits are wired during brain development. Because the nerve endings from dopamine-releasing neurons reach the striatum before most excitatory connections between the neurons are formed, dopamine stands to influence the development of connections in the striatum.
Kozorovitskiy et al. have now investigated the role of neuromodulation in brain development in young mice. This involved measuring the formation of excitatory connections or synapses and the electrical activity of different striatal neurons during the maturation of brain circuits that occurs after birth. This analysis revealed that turning on dopamine receptors that increase PKA activity rapidly enhances the number of excitatory synapses on the neurons that express this receptor.
Kozorovitskiy et al. then used a variety of approaches to investigate whether there is cooperation between G protein-coupled receptors, PKA activity and a signaling molecule called glutamate in striatal development. This revealed a more general mechanism by which the activation of G-protein-coupled receptors interacts with glutamate (the primary excitatory signal sent between neurons) in order to produce new synapses. These results reveal a previously unknown role for neuromodulation in “wiring up” the brain and open the possibility of new therapies to treat neurodevelopmental and neurodegenerative disorders.
basal ganglia; circuit development; corticostriatal; dopamine; pka; Mouse
Cerebellar climbing fiber activity encodes performance errors during many motor learning tasks, but the role of these error signals in learning has been controversial. We compared two motor learning paradigms that elicited equally robust putative error signals in the same climbing fibers: learned increases and decreases in the gain of the vestibulo-ocular reflex (VOR). During VOR-increase training, climbing fiber activity on one trial predicted changes in cerebellar output on the next trial, and optogenetic activation of climbing fibers to mimic their encoding of performance errors was sufficient to implant a motor memory. In contrast, during VOR-decrease training, there was no trial-by-trial correlation between climbing fiber activity and changes in cerebellar output, and climbing fiber activation did not induce VOR-decrease learning. Our data suggest that the ability of climbing fibers to induce plasticity can be dynamically gated in vivo, even under conditions where climbing fibers are robustly activated by performance errors.
The cerebellum (or ‘little brain’) is located underneath the cerebral hemispheres. Despite comprising around 10% of the brain’s volume, the cerebellum contains roughly half of the brain’s neurons. Many of the functions of the cerebellum are related to the control and fine-tuning of movement, and people whose cerebellum has been damaged have problems with balance and coordination, and with learning new motor skills.
One of the roles of the cerebellum is to control a reflex known as the vestibulo-ocular reflex, which enables us to keep our gaze fixed on an object as we turn our heads. The cerebellum relays information about head movements to the muscles that control the eyes, instructing the eyes to move in the opposite direction to the head. This keeps the image of the object we are looking at stable on the retina.
The vestibulo-ocular reflex is controlled by a circuit that includes Purkinje cells (which are the main output cells of the cerebellum) and climbing fibres (which originate in the brainstem). Any failure of the vestibulo-ocular reflex to fully compensate for head movements generates an error signal that activates the climbing fibres. These in turn modify the output of Purkinje cells, leading ultimately to adjustments in eye movements.
However, Kimpo et al. have now obtained evidence that Purkinje cells can modulate their response to the instructions they receive from climbing fibres. Monkeys sat in a rotating chair while a visual object they were trained to track with their eyes was moved to induce errors in the vestibulo-ocular reflex. When the object was moved so that a bigger reflexive eye movement was required to stabilize the image, the activation of the climbing fibres in response to the error led to a change in the response of the Purkinje cells, as expected. However, when a smaller reflexive eye movement was needed, the error-driven responses of the climbing fibres did not alter the responses of Purkinje cells. Similar results were obtained using pulses of light to artificially activate climbing fibres and thus simulate error signals.
The work of Kimpo et al. indicates that the cerebellum does not blindly follow the instructions it receives from the brainstem, but can instead modulate its responses to incoming information about performance errors. Further work is now required to identify factors that influence the responsiveness of the cerebellum: such information could ultimately be used to improve learning of motor skills and recovery from injury.
rhesus macaque; climbing fibers; cerebellum; motor learning; vestibulo-ocular reflex; supervised learning; mouse; other
The generation of purposive movement by mammals involves coordinated activity in the corticospinal and corticostriatal systems, which are involved in different aspects of motor control. In the motor cortex, corticospinal and corticostriatal neurons are closely intermingled, raising the question of whether and how information flows intracortically within and across these two channels. To explore this, we developed an optogenetic technique based on retrograde transfection of neurons with deletion-mutant rabies virus encoding channelrhodopsin-2 (RV-ChR2), and used this in conjunction with retrograde anatomical labeling to stimulate and record from identified projection neurons in mouse motor cortex. We also used paired recordings to measure unitary connections. Both corticospinal and callosally projecting corticostriatal neurons in layer 5B formed within-class (recurrent) connections, with higher connection probability among corticostriatal than among corticospinal neurons. In contrast, across-class connectivity was extraordinarily asymmetric, essentially unidirectional from corticostriatal to corticospinal. Corticostriatal neurons in layer 5A and corticocortical neurons (callosal projection neurons similar to corticostriatal neurons) similarly received a paucity of corticospinal input. Connections involving presynaptic corticostriatal neurons had greater synaptic depression, and those involving postsynaptic corticospinal neurons had faster decaying EPSPs. Consequently, the three connections displayed a diversity of dynamic properties reflecting the different combinations of pre- and postsynaptic projection neurons. Collectively, these findings delineate a four-way specialized excitatory microcircuit formed by corticospinal and corticostriatal neurons. The “rectifying” corticostriatal-to-corticospinal connectivity implies a hierarchical organization and functional compartmentalization of corticospinal activity via unidirectional signaling from higher-order (corticostriatal) to lower order (corticospinal) output neurons.
The membrane protein Nogo-A is known as an inhibitor of axonal outgrowth and regeneration in the CNS. However, its physiological functions in the normal adult CNS remain incompletely understood. Here, we investigated the role of Nogo-A in cortical synaptic plasticity and motor learning in the uninjured adult rodent motor cortex. Nogo-A and its receptor NgR1 are present at cortical synapses. Acute treatment of slices with function-blocking antibodies (Abs) against Nogo-A or against NgR1 increased long-term potentiation (LTP) induced by stimulation of layer 2/3 horizontal fibers. Furthermore, anti-Nogo-A Ab treatment increased LTP saturation levels, whereas long-term depression remained unchanged, thus leading to an enlarged synaptic modification range. In vivo, intrathecal application of Nogo-A-blocking Abs resulted in a higher dendritic spine density at cortical pyramidal neurons due to an increase in spine formation as revealed by in vivo two-photon microscopy. To investigate whether these changes in synaptic plasticity correlate with motor learning, we trained rats to learn a skilled forelimb-reaching task while receiving anti-Nogo-A Abs. Learning of this cortically controlled precision movement was improved upon anti-Nogo-A Ab treatment. Our results identify Nogo-A as an influential molecular modulator of synaptic plasticity and as a regulator for learning of skilled movements in the motor cortex.
two-photon; in vivo; LTP; motor learning; Nogo-A; synaptic plasticity
Learning a novel motor skill is associated with well characterized structural and functional plasticity in the rodent motor cortex. Furthermore, neuroimaging studies of visuomotor learning in humans have suggested that structural plasticity can occur in white matter (WM), but the biological basis for such changes is unclear. We assessed the influence of motor skill learning on WM structure within sensorimotor cortex using both diffusion MRI fractional anisotropy (FA) and quantitative immunohistochemistry. Seventy-two adult (male) rats were randomly assigned to one of three conditions (skilled reaching, unskilled reaching, and caged control). After 11 d of training, postmortem diffusion MRI revealed significantly higher FA in the skilled reaching group compared with the control groups, specifically in the WM subjacent to the sensorimotor cortex contralateral to the trained limb. In addition, within the skilled reaching group, FA across widespread regions of WM in the contralateral hemisphere correlated significantly with learning rate. Immunohistological analysis conducted on a subset of 24 animals (eight per group) revealed significantly increased myelin staining in the WM underlying motor cortex in the hemisphere contralateral (but not ipsilateral) to the trained limb for the skilled learning group versus the control groups. Within the trained hemisphere (but not the untrained hemisphere), myelin staining density correlated significantly with learning rate. Our results suggest that learning a novel motor skill induces structural change in task-relevant WM pathways and that these changes may in part reflect learning-related increases in myelination.