|Home | About | Journals | Submit | Contact Us | Français|
Neural prosthetic systems aim to help disabled patients suffering from a range of neurological injuries and disease by using neural activity from the brain to directly control assistive devices. This approach in effect bypasses the dysfunctional neural circuitry, such as an injured spinal cord. To do so, neural prostheses depend critically on a scientific understanding of the neural activity that drives them. We review here several recent studies aimed at understanding the neural processes in premotor cortex that precede arm movements and lead to the initiation of movement. These studies were motivated by hypotheses and predictions conceived of within a dynamical systems perspective. This perspective concentrates on describing the neural state using as few degrees of freedom as possible and on inferring the rules that govern the motion of that neural state. Although quite general, this perspective has led to a number of specific predictions that have been addressed experimentally. It is hoped that the resulting picture of the dynamical role of preparatory and movement-related neural activity will be particularly helpful to the development of neural prostheses, which can themselves be viewed as dynamical systems under the control of the larger dynamical system to which they are attached.
It is difficult to appreciate just how central movement is to everyday life until this ability is lost due to neurological injury or disease. Moving is how we interact and communicate with the world. We move our legs and feet to walk, we move our arms and hands to manipulate the objects that surround us, and we move our tongues and vocal cords to speak. Movement is not only central to these critical aspects of life, but also to self-image and psychological well-being. In fact, the fundamental reason that tetrapelgics wish most for the restored use of their arms is to regain some degree of independence (Anderson, 2004).
Fortunately, it appears that a confluence of knowledge and technology from the fields of (1) systems motor neuroscience, (2) neuroengineering, and (3) electrical engineering and computer science may soon provide a new class of electronic medical systems (termed neural prosthetic systems, brain machine interfaces, or brain computer interfaces) aimed at increasing the quality of life for severely disabled patients.
First, basic neuroscience research across the past several decades has elucidated many of the fundamental principles underlying movement generation and control. A substantial body of knowledge regarding the cortical control of arm movements, particularly in rhesus macaques, now exists (e.g., Evarts, 1964; Georgopoulos et al., 1982, 1986; Schwartz, 1994; Tanji and Evarts, 1976). This literature is reviewed elsewhere (e.g., Kalaska, 2009; Kalaska et al., 1997; Scott, 2004; Wise, 1985). As discussed below, this understanding has been sufficient to help guide the design of first generation prosthetic systems. Yet continued focus on underlying neural mechanisms (in both monkeys and humans), how neural populations behave across timescales, and how neural populations participate in the ongoing control of movement, is essential for creating second generation prostheses capable of higher performance and a greater range of capabilities (e.g., Cunningham et al., 2010; Green and Kalaska, 2010; Truccolo et al., 2008, 2010).
Second, basic neuroengineering research has provided proof-of-concept demonstrations of neural prosthetic systems which translate the electrical activity (action potentials and local field potentials, LFPs) from populations of intracortically recorded neurons into control signals for guiding computer cursors, prosthetic arms, or stimulating the paralyzed musculature. More specifically, a series of designs and demonstrations across the past decade have produced compelling laboratory evidence that intracortical neural signals from rodents (e.g., Chapin et al., 1999), monkeys (e.g., Carmena et al., 2003; Chase et al., 2009; Fetz, 1969; Fraser et al., 2009; Ganguly and Carmena, 2009; Gilja et al., 2010b, c; Heliot et al., 2009; Humphrey et al., 1970; Isaacs et al., 2000; Jackson et al., 2006; Jarosiewicz et al., 2008; Moritz et al., 2008; Mulliken et al., 2008; Musallam et al., 2004; Nuyujukian et al., 2010; Santhanam et al., 2006; Serruya et al., 2002; Shenoy et al., 2003; Taylor et al., 2002; Velliste et al., 2008; Wessberg et al., 2000; Wu et al., 2004), and humans (e.g., Hochberg et al., 2006; Kim et al., 2008) can control prosthetic devices that may provide meaningful quality of life improvement to paralyzed patients. This literature is reviewed elsewhere (e.g., Andersen et al., 2010; Millan and Carmena, 2010; Donoghue, 2008; Donoghue et al., 2007; Fetz, 2007; Hatsopoulos and Donoghue, 2009; Linderman et al., 2008; Nicolelis and Lebedev, 2009; Ryu and Shenoy, 2009; Scherberger, 2009; Schwartz, 2007).
Finally, the semiconductor electronics, optoelectronic telecommunications, micro-electromechanical systems (MEMS), and information technology revolutions over the past four decades have produced extraordinary and relevant technologies. These include low-power and high computational-density circuits and systems, lowpower wireless telemetric systems, advanced light sources and imaging modalities, and small sensor systems that are capable of running sophisticated signal processing algorithms. These technologies have progressed extremely quickly, as described by Moore’s Law, and have been leveraged and adapted to create new neurotechnologies for basic neuroscience and neuroengineering applications such as neural prosthetic systems. It is now possible to record from hundreds of neurons simultaneously with bio-MEMS electrode arrays (e.g., Chestek et al., 2009a, 2011; Jackson and Fetz, 2007; Mavoori et al., 2005; Santhanam et al., 2007), filter and “spike sort” all channels in real time (e.g., O’Driscoll et al., 2006; Santhanam et al., 2004, 2006), “decode” the intended arm movement with advanced algorithms (e.g., Achtman et al., 2007; Cunningham et al., 2008; Kemere et al., 2004, 2008; Santhanam et al., 2009; Wu et al., 2006; Yu et al., 2007, 2010), wirelessly telemeter the resulting prosthetic arm control signals with just a few tens of milliwatts of power (e.g., Chestek et al., 2009b; Gilja et al., 2010a; Harrison et al., 2007, 2009), and soon, this will likely all be possible in fully implantable systems (e.g., Borton et al., 2009; Harrison, 2008; Nurmikko et al., 2010).
While these laboratory proof-of-concept systems and initial FDA phase-I clinical trials are encouraging (e.g., Hochberg, 2008; Hochberg and Taylor, 2007), several barriers remain. If these barriers are unaddressed, they could substantially limit the prospect of intracortically based neural prosthetic systems having a broad and important clinical impact. We recently reviewed what we consider to be three of the most important neuroengineering, bioengineering, electrical engineering, and computer science challenges and opportunities for intracortically based neural prostheses (Gilja et al., 2011). We review here what we consider to be one of the most central and important basic systems-level motor neuroscience questions. The knowledge gained while investigating this question should directly advance our ability to design high-performance neural prostheses. The central question we have been asking is: what are the neural processes that precede movement and lead to the initiation of movement? Neural prostheses will benefit from a deeper and more comprehensive understanding of the neural activity upon which they are based (Green and Kalaska, 2010). This includes activity during both movement preparation and movement generation. We need to understand both because prostheses use both (e.g., Yu et al., 2010), and because the two are presumably causally linked and likely impossible to understand fully if studied in isolation (discussed further in the final section, and Fig. 12). Prostheses should thus benefit from having a firm scientific understanding of how preparatory activity relates to upcoming arm movements, and how this preparatory activity evolves on a millisecond timescale. These are the questions and topics discussed in this review.
Why should one prepare and then move, as opposed to starting the movement as soon as possible? In some cases, it is critical to move right away, such as when withdrawing a hand from a flame. Animals have evolved low-latency circuits to help in these cases and these circuits underlie a wide range of reflexive movements. However, animals have also evolved circuits to enable voluntary movements which are intentional and purposeful. Voluntary movements require the ability to change, refine, and suppress possible actions before they are actually executed. A simple example is how we swat a fly. One approach would be to see a fly and start moving right away. Unless the nervous system can execute perfectly, this is unlikely to be a good strategy, and if the initial movement is not successful, the fly is likely to depart before a correction can be made. It would thus be beneficial to take slightly more time to initiate the movement, assuming that, in doing so, greater accuracy can be gained. Presumably, we use this slight addition of time to create and refine movement plans until the moment is right and then we initiate the movement. It is this form of deliberate, goal-driven movement that we seek to better understand, both neurally and behaviorally, both out of scientific curiosity and because it could lead to superior prosthetic designs.
There is indeed evidence that voluntary movements are prepared before they are initiated (e.g., Day et al., 1989; Ghez et al., 1997; Keele, 1968; Kutas and Donchin, 1974; Riehle and Requin, 1993; Rosenbaum, 1980; Wise, 1985). An important line of evidence comes from “instructed-delay tasks” where a temporal delay separates an instruction stimulus from a subsequent “go” cue. Figure 1 illustrates the experimental arrangement and task timing, along with example hand position and electromyographic (EMG) measurements. This task is widely employed and is the behavioral task used in the recent studies reviewed here.
At the behavioral level, reaction times (RTs), defined as the time from the go cue until movement onset, are shorter after an instructed-delay period. Figure 2 illustrates how RT decreases and then plateaus as a function of delay period. This RT reduction with delay, largely occurring during the first 200 ms, suggests that some timeconsuming preparatory process is given a head start by the delay (e.g., Crammond and Kalaska, 2000; Riehle and Requin, 1989; Rosenbaum, 1980). It is straightforward to interpret the importance of this head start on preparation in the context of the fly swatting example offered above. There the goal was not to move instantaneously as soon as the fly landed or was seen. Instead, the goal was to move swiftly and accurately, at a particular speed and along a particular path that perhaps approaches from behind, and to be able to start that movement as quickly as possible when it is decided that the time is right. Thus, a good strategy is to prepare the desired movement as soon as possible, so that one is ready to move as soon as possible when called upon to do so.
The ability to prepare a movement ahead of time is presumably related to the preparatory activity that is widespread in cortex and subcortical structures. Neurons in a number of cortical areas including dorsal premotor cortex (PMd) and primary motor cortex (M1) show changes in activity during the delay period (e.g., Crammond and Kalaska, 2000; Godschalk et al., 1985; Kalaska et al., 1997; Kurata, 1989; Messier and Kalaska, 2000; Riehle and Requin, 1989; Snyder et al., 1997; Tanji and Evarts, 1976; Weinrich et al., 1984). Figure 3 shows four example PMd neurons. While it is typical for the average action potential emission (firing) rate during the delay period to change following target onset, the temporal structure is widely varying across cells: some increase their firing rate, some decrease, some arrive at an approximate plateau level, while others undulate.
This variety of neural responses stands in stark contrast to the simple monotonic decline in behavioral RT as shown in Fig. 2. The central question is, therefore, how does neural activity in the first 200–300 ms of the delay period relate to the decrease in RT? Asked in the context of the fly swatting example, what does this neural activity need to accomplish during the delay so that we are maximally poised to generate the planned movement and, after initiating the movement, successfully hit the fly?
We have been investigating this question using a dynamical systems perspective (e.g., Briggman et al., 2005; Churchland et al., 2007; Stopfer et al., 2003). What this means in essence is that we wish to understand (1) how the activity of a neural population evolves and achieves the needed preparatory state, (2) how this preparatory state impacts the subsequent arm movement, and (3) what the underlying dynamics (rules) of the neural circuitry are (Churchland et al., 2007; Yu et al., 2006, 2009). We start with as simple an assumption as possible: the arm movement made (M(t)) depends upon preparatory activity (P) at the time movement activity begins to be generated (t0). In other words, M(t) depends on P(t0).
It is important to note that there are likely sources of variability that impact M(t) but are not accounted for in P(t0), such as downstream variability in the state of the spinal cord or muscles. Thus, to be strictly true, a noise source should be included, or P(t0) would need to be the initial state of the entire animal. However, for the moment, we avoid this issue and simply concentrate on the hypothesis that the movement you make is in large part a function of the plan that was present just before movement began. Also, note that the above conception does not rule out a strong (or even dominant) role for feedback. Such feedback could be part of the causal mechanism by which the plan produces the movement.
The central implication of our assumption that M(t) depends on P(t0) is that motor preparation may be the act of optimizing preparatory activity (i.e., bringing P to the state needed at t0) so that the generated movement has the desired properties. In the case of monkeys performing reaching movements, the desired movement can be defined as a reach that is accurate enough to result in reward. Consider the space of all possible preparatory states (all possible Ps). For a given reach, there is presumably some small subregion of space containing those values of P that are adequate to produce a successful reach. Although the response of each neuron (i.e., tuning) may not be easily parameterized, there is nonetheless a smooth relationship between firing rate and movement. Therefore, the small subregion of space is conceived of as being contiguous.
Figure 4 illustrates this idea. We conceive of all possible preparatory states as forming a space, with the firing rate of each neuron contributing an axis. Each possible state—each vector of possible firing rates—is then a point in this space. For a given reach (e.g., rightwards), there will be some subset of states (gray region in Fig. 4, referred to as the optimal subspace) that will result in a successful reach that garners a reward. Under this optimal subspace hypothesis, the central goal of motor preparation is to bring the neural state within this subspace before the movement is triggered. This may occur in different ways on different trials (trial 1 and trial 2 in Fig. 4). This framework, though rather general, has provided us with a number of specific and testable predictions, which we review below.
Before doing so, it is worth considering that an almost-trivial prediction of the optimal subspace hypothesis is that different movements require different initial states. If preparatory activity has a strong role in determining movement, then making different movements will require different patterns of preparatory activity. The overall neural state, and thus the state of individual neurons, should therefore vary with different movements. This is of course consistent with the observation that preparatory activity is tuned for reach parameters such as direction and distance (e.g., Messier and Kalaska, 2000). In fact, under the optimal subspace hypothesis, neural activity should appear tuned for essentially every controllable aspect of the upcoming reach (a prediction we will return to shortly).
As a brief aside on the topic of tuning, we note that one could conceive of each axis in Fig. 4 as capturing not the activity of a single neuron, but rather the activity of a population of neurons that are all tuned for the same thing. Thus, the three axes might capture, respectively, the average activity of neurons tuned for direction, distance, and speed. If so, the preparatory state could be thought of as an explicit representation of direction, distance, and speed. However, it has been argued that few individual neurons appear tuned for reach parameters in the straightforward and invariant way that one might hope (e.g., Churchland et al., 2006b; Churchland and Shenoy, 2007b; Cisek, 2006; Fetz, 1992; Scott, 2004, 2008; Todorov, 2000). The optimal subspace hypothesis is largely agnostic to this debate. So long as there is a systematic relationship between preparatory activity and movement, the optimal subspace conception remains viable. Put another way, the space illustrated in Fig. 4 could have axes that capture well-defined parameters, but it need not, and there are reasons to suspect that it does not.
A related and critical point is that the space in which neural activity evolves is certainly larger than the three dimensions illustrated in Fig. 4. Movements vary fromone another in more than three different ways. Similarly, neural activity varies across movements in more than three different ways (Churchland and Shenoy, 2007b). Thus, care should be taken when gleaning intuition from illustrations such as that in Fig. 4, to keep in mind that what is illustrated is a projection of a larger and richer space (Churchland et al., 2007; Yu et al., 2009).
We now review a number of specific and testable predictions of the optimal subspace hypothesis.
Figure 5a illustrates in state space our first prediction under the optimal subspace hypothesis: preparatory activity should covary with other meaningful aspects of movement, including peak reach speed. Confirming this would be consistent with our assumption above, whereas failing to find this would be consistent with preparatory activity having a higher-level, perhaps more sensory role reflecting the target location but not the more detailed aspects of movement.
To test this prediction, we trained monkeys to reach to targets in a variant of the instructed-delay task. Reaches must be made somewhat faster ($1.5 m/s peak speed) when the target was red and somewhat slower ($1.0 m/s peak speed) when the target was green (Churchland et al., 2006b). All other movement metrics such as reach path remained similar. Delay-period activity was substantially different ahead of fast and slow arm movements to the same target location. Figure 5b shows the average response of an example neuron, ahead of reaches to a particular target, where the delay-period activity was greater ahead of fast reaches (red) than ahead of slow reaches (green). Figure 5c and d show two more example neurons where this difference in preparatory activity ahead of fast (red) and slow (green) reaches is emphasized by collapsing across all reach target locations. Some neurons had higher average firing rates ahead of fast movements (Fig. 5c), while other neurons had higher average rates ahead of slow movements (Fig. 5d).
In sum, prediction 1 as illustrated in Fig. 5a appears to be correct.
Figure 5a also illustrates in state space our second prediction under the optimal subspace hypothesis: preparatory activity should correlate, on a trialby- trial basis, with the peak reach speed. Our assumption that M(t) depends on P(t0) predicts that even a slightly different P(t0) value should lead to a different M(t). If P(t0) reflects the result of a difficult optimization, then variability in P(t0) is likely. Therefore, it should be possible to observe a trial-by-trial correlation between P(t0) and movement metrics M(t).
To test this prediction, we again employed the reach-speed variant of the instructed-delay task. We found trial-by-trial correlations between the firing rate of individual neurons before the go cue and peak reach speed (Churchland et al., 2006a). Consider the instructed-fast condition (red) in Fig. 5c. The horizontal spread of points (one point per trial) reflects the trial-to-trial variance in peak reach speed. The vertical spread of points largely reflects the trial-to-trial variance in estimated firing rate, which is the inevitable result of it being difficult to assess the firing rate of a single neuron on a single trial from a handful of stochastically occurring spikes. Nevertheless a statistically significant correlation was found for most neurons and for both instructed speeds.
Importantly, the state-space illustration (Fig. 5a) further predicts that within the instructed-fast condition, for example, a trial with a slightly slower peak reach speed should have a preparatory state slightly closer to those found in the instructed-slow condition. In other words, if we assume that movement parameters are mapped smoothly from firing rate, the slope of the within-condition correlation (black lines) should agree with the slope of the across-condition mean line fit (gray line) both when the instructed-fast condition had a higher average firing rate (Fig. 5c) and when it had a lower average firing rate (Fig. 5d). We found this to be the case in the majority of neurons (Churchland et al., 2006a).
In sum, prediction 2 as illustrated in Fig. 5a appears to be correct.
Figure 6a illustrates in state space our third prediction under the optimal subspace hypothesis: preparatory activity should become, through time, quite accurate and therefore quite similar across trials. Before the target appears, “baseline” neural activity can be somewhat different from trial to trial, leading to some amount of across-trial firing-rate variance (black circles in panel labeled “before target onset” in Fig. 6a, top). After target onset, and for the coming 200–300 ms, preparatory activity on each trial is nominally being optimized and brought to reside within the optimal subspace. The optimal subspace is presumably rather restricted by virtue of the behavioral task constraints and thus should have less across-trial firing-rate variance (black circles within the optimal subspace shaded gray, in the panel labeled “$200 ms after target onset’ in Fig. 6a, bottom).
To test this prediction, we analyzed data from an instructed-delay task using the Fano factor: the across-trial spike-count variance divided by the mean (Churchland et al., 2006c, 2007, 2010c). Normalization, and an additional set of controls, is necessary to ensure that the measured changes in variance are not simply due to the well-known scaling of spike-count variance with spike-count mean (as happens, e.g., for a Poisson process; Churchland et al., 2007, 2010c; Rickert et al., 2009). As shown in Figure 6b, we found that the Fano factor declines over the course of approximately 200 ms and then approximately plateaus (Churchland et al., 2006c). This is somewhat remarkable, as it so closely resembles the decline and plateau seen in the behavioral curves (RT versus delay, Fig. 2).
In sum, prediction 3 as illustrated in Fig. 6a appears to be correct. Moreover, it appears that the across-trial firing-rate variance (as measured by the Fano factor) parallels the reduction in RT: both drop over the course of approximately 200 ms and then hold at that level. This possibility is explored below, as predictions 4-I and 4-II.
As a brief aside, it could be the case that this reduction in across-trial firing-rate variance is principally a motor phenomenon and is specific to the preparation of arm movements. However, we found that this same general structure of a reduction in across-trial firing-rate variance following a stimulus onset is present across much if not all of cerebral cortex (Churchland et al., 2010c). Figure 7 shows a substantial reduction in Fano factor following stimulus onset in numerous areas, across all four cortical lobes, and in a variety of behaviors. This reduction seems to be a general property of the nervous system responding to an input, much as the mean (across-trial) firing-rate changing is a general property of cortical neurons. This reduction in across-trial firing-rate variance in each area may be correlated with the relevant functions performed therein (e.g., sensation, cognition, behavior), again just as the mean firing rate is well known to correlate with the function of each area.
Figure 6c illustrates the first part of our fourth prediction (prediction 4-I) under the optimal subspace hypothesis: the lower the across-trial firingrate variance at the time of the go cue, the lower too should be the RT. Having seen the similarity between how the Fano factor descends and holds as a function of delay duration (Fig. 6b), and how RT descends and holds as a function of RT (Fig. 2), it is natural to predict that there should exist a positive correlation between Fano factor and RT. For example, one expects that short delays should lead to high Fano factors and high RTs, while long delays should lead to low Fano factors and low RTs.
To test this prediction, we analyzed short delayduration trials from the instructed-delay task. Figure 2 shows representative RT data from monkey G when 30, 130, and 230 ms delay durations were used. Figure 6c shows the Fano factor at the three critical times: 30, 130, and 230 ms after target onset. Figure 6d shows RT data plotted against Fano factor data, from the same trials in Monkey G, and a clear relationship is seen. The lower the across-trial firing-rate variance at the time of the go cue (as measured by the Fano factor), the lower the RT (Churchland et al., 2006c).
Figure 8a illustrates in state space the second part of our fourth prediction (prediction 4-II) under the optimal subspace hypothesis: the lower the across-trial firing-rate variance at the time of the go cue, the lower too should be the RT, even in long delay-duration trials where sufficient time has elapsed for “complete” motor preparation to result. For long delay durations (e.g., >200–300 ms), the Fano factor has nominally plateaued, as has the RT, at a low level. But, as depicted in Fig. 8a, there could still remain some variability. On trials that “wander outside” the optimal subspace (red circles), some additional time (i.e., increased RT) should be required to complete preparatory optimization following a go cue. In contrast, on trials where the preparatory state is within the optimal subspace (green dots) and therefore motor preparation is complete and ready for execution, movement can begin with a minimum of latency following the go cue (i.e., low RT).
To test this prediction, we started with all trials with 200 ms or longer delay durations, across 7 days of experiments using a 96-channel electrode array. This helped assure sufficient data. Second, we sorted trials according to whether the RT was shorter than or longer than the median RT. Third, we calculated the across-trial firing-rate variance (Fano factor) for the half of trials with shorter than median RT, and the same for the half of trials with longer than median RT. We did so for times ranging from 200 ms before the go cue until 200 ms after the go cue in order to assess the robustness of the result. Figure 8b plots the Fano factor curve for shorter than median trials (green curve) and longer than median trials (red curve). These curves are statistically significantly different (not shown), and as predicted, the lower across-trial firing-rate variance trials (green curve) are associated with lower RTs (Churchland et al., 2006c).
In sum, prediction 4-II as illustrated in Fig. 8a appears to be correct. When combined with the experiments and results associated with prediction 4-I, it appears clear that there is a close relationship between the across-trial firing-rate variability at the time of the go cue and the resulting RT. Recently, similar results have been found in area V4 ahead of saccadic eye movements, suggesting that this relationship is not limited to the arm movement system alone (Steinmetz and Moore, 2010).
The inset of Fig. 9 illustrates in state space the fifth prediction under the optimal subspace hypothesis: perturbing the preparatory state out of the optimal subspace should result in an increased RT. But it should not reduce movement accuracy.
The first four predictions of the optimal subspace hypothesis were correlative, and their affirmation provides important evidence supporting the optimal subspace hypothesis. The inset of Fig. 9 illustrates a causal prediction, wherein a preparatory state within the optimal subspace is deliberately perturbed (curly line with displaced preparatory state, black circle) and it is predicted that the RT should increase. This follows from the reasoning that if the goal of motor preparation is to help make accurate movements, then the brain must somehow be able to monitor preparatory activity and determine when it is accurate enough to initiate movement. If preparatory activity were optimized and within the optimal subspace, but were then perturbed away from the optimal subspace, the brain should wait for the plan to reoptimize to the optimal subspace (i.e., recover) before initiating movement (red dashed arrow labeled “reoptimization”). Importantly, after taking time to reoptimize preparatory activity, the resulting movements should be as accurate as on nonstimulated trials.
To test this prediction, we delivered subthreshold electrical microstimulation to PMd on a subset of trials and did so at various times relative to the go cue (Churchland and Shenoy, 2007a). Figure 9 plots experimental results from all (30) stimulation sites in PMd in one monkey. RTs are increased when microstimulation is delivered around the time of the go cue. This is seen as a rightward shift in the red hand-speed curve (stimulation around time of go cue, as indicated by the red bar) relative to the black hand-speed curve (no stimulation). This is consistent with time having been consumed (increased RT) to reoptimize preparatory activity. Importantly, aside from delaying the onset of movement, all other movement metrics were extremely similar to the nonstimulated trials (consistent with prediction 5). Note that the red and green averaged curves in Fig. 9 have lower peak hand speed due to staggered RTs, but individual trials do achieve the same, higher peak hand speed (see Churchland and Shenoy, 2007a for details).
As it is critical to establish effect specificity when conducting causal perturbation experiments, we performed several additional control experiments (Churchland and Shenoy, 2007a). Four are briefly summarized here. First, we found that stimulating well before the go cue (Fig. 9, green bar) had little impact on the RT. This can be seen in Fig. 9 by noting that the green curve largely overlaps with the black curve. This result is consistent with there being sufficient time for reoptimization to occur before the go cue appears. This is an important temporal control and indicates that the effect of subthreshold microstimulation exerts its influence just when the preparatory state is most needed (consistent with prediction 5). Second, stimulating on zero delay-duration trials where there was presumably no optimized preparatory activity present to perturb did not alter RT. This is an important control as it confirms the necessity of there first existing a preparatory state near the optimal subspace (consistent with prediction 5). Third, stimulating in M1 where there is relatively less preparatory activity resulted in little RT increase. The importance of this control is twofold. (i) It confirms that perturbing motor preparation is easier in an area where preparatory activity is prevalent (PMd) than in an area where it is not (M1). (ii) It confirms area specificity. M1 is just a few millimeters from PMd, but the effect is dramatically reduced and thereby helps assure that subthreshold microstimulation is not just a generalized distraction. Both findings are consistent with prediction 5. Fourth and finally, the effect of microstimulation was specific to arm movements and produced little increase in saccadic eye movement RT. This is an important control for the possibility that microstimulation is altering attention, which should impact both effectors equivalently.
In sum, prediction 5 as illustrated in Fig. 9 is borne out. This contributes causal supportive evidence for the optimal subspace hypothesis, which complements predictions 1–4 as well as 6 and 7 (below).
The conceptual sketch in Fig. 4 illustrates in state space the sixth prediction under the optimal subspace hypothesis: it should be possible to construct single-trial state-space neural trajectories and use them to directly confirm that across-trial firing-rate variability decreases through time.
To test this prediction, we must begin by measuring many neurons simultaneously. This is essential as we seek an accurate estimate of the preparatory state on each individual trial and on a fine time scale. Both require data from many neurons, instead of the more traditional technique of trial averaging, in order to mathematically reduce the deleterious effects of spiking noise (Churchland et al., 2007; Yu et al., 2009). These measurements can be made with electrode arrays, which have been developed substantially as part of neural prosthesis research. The analyses can be performed using modern dimensionality reduction and visualization methods such as Gaussian Process Factor Analysis (GPFA; Yu et al., 2009). Dimensionality reduction is needed for two reasons. First, reducing the dimensionality of the data from its original ~100 D space (e.g., 100 neurons measured simultaneously constitutes a 100 D space) down to 10–15 D appears to be possible without significant loss of information and has the benefit of effectively denoising the data (Yu et al., 2009). This can be thought of as essentially performing a weighted average to combine the responses of neurons that share some important aspect of their response. Second, while reducing the dimensionality further (below 10–15 D) does result in a loss of information, it can be quite useful for visualization purposes. This is because the two or three dimensions used in drawings can be the two or three dimensions that capture the greatest variance in the data, and the resulting plots are still sufficient to spur on hypotheses and predictions as described above.
Figure 10a shows multiple single-trial neural trajectories in a 2D state space created with GPFA (Churchland et al., 2010c; Yu et al., 2009). This is the first time that true single-trial neural trajectories (gray lines in Fig. 10a), as opposed to the cartoon depictions in Fig. 4, are plotted in this review. It is reassuring to see in Fig. 10a that the scatter in across-trial preparatory states at each point in time (black dots) reduces as the trial progresses. As the trial progresses from before target onset (100 ms pretarget) to just after target onset when the preparatory state is evolving toward the optimal subspace (100 ms post-target), and on to when the neural state on each trial is presumably within the optimal subspace (200 ms post-target), the variance (scatter) of the preparatory state reduces. This is consistent with the results presented above, inferred with Fano factor analyses.
Figure 10b again shows multiple single-trial neural trajectories in a 2D state space created with GPFA but now goes on to show data until the time of movement onset (Churchland et al., 2010c). This reveals for the first time that neural trajectories (gray lines) follow a largely stereotyped path through state space, after the initial convergence following target onset. They start in the baseline pretarget state (blue circles), progress, and slow-down (if an extended delay period) in the optimal subspace until the time of the go cue (green), and then arch around and arrive at a small region where the arm movement is first detected (black). Highlighted in red is one outlier trial which had a substantially longer RT than typical. With single-trial visualization of even (entirely) internal neural processing, it is now possible to ask, for the first time, what the reason might be. On this trial, the preparatory process appears to have completed normally. The green circle is within the (presumed) optimal subspace and surrounded by other trials that had normal RTs. While we cannot conclude what the cause was from this data alone, we appear to be able to rule out incomplete motor preparation. Figure 10c plots data from a different data set. Again, one trial with a particularly long RT is highlighted in red. This trial’s neural trajectory undergoes an entire loop between the go cue and movement onset.
In sum, prediction 6 as illustrated in Fig. 4 is borne out. It is possible to construct single-trial neural trajectories and use these trajectories to directly see key features that can only be inferred less directly with single-neuron recordings. It also appears to now be possible to begin to investigate the reasons for outlier and other types of unique trials. Intriguingly, it should now also be possible to design experiments aimed at creating inherently single-trial phenomenon such as single- trial decision making, which could shed considerable insight on internal cognitive processing and neural dynamics (Kalmar et al., 2010; Rivera-Alvidrez et al., 2008). Further, as tasks become more complex (e.g., Churchland et al., 2008) and naturalistic (e.g., Chestek et al., 2009b; Gilja et al., 2010a; Jackson et al., 2006; Santhanam et al., 2007), both behavior and the preceding neural processes will likely become less stereotyped and may therefore often require single- trial analyses.
We have posited that the preparatory state has a large impact on the subsequent movement. We have also seen several predictions that stem from the optimal subspace hypothesis along with evidence supporting these predictions. It does seem to be the case that the preparatory state at the time of the go cue has a substantial influence on the subsequent movement. But why should this be?
One possibility is illustrated in Fig. 11a. It could be that the preparatory state at the time of the go cue (green circle) acts as the initial state of a subsequent dynamical system that serves to generate muscle activity and create movement (green and blue arrows; Churchland et al., 2010a). As such, it is important that the preparatory state be within the optimal subspace in order to help create the desired movement. Thus some region or regions of the brain appear to monitor and wait for this to be true before “pulling the trigger” to initiate movement. After the movement trigger has been pulled, if the preparatory state happens to be farther along the “loop” and moving in the standard direction around the loop (see arrows), then RT may be further lowered.
Figure 11b illustrates in state space the seventh prediction under the optimal subspace hypothesis: the farther the preparatory state is along the loop when the movement trigger is pulled, and the faster it is moving along the loop in the standard direction (not shown in Fig. 11b), the shorter the RT should be. Figure 11b depicts the “loop” structure seen in Fig. 10b, where most individual trials follow a stereotyped path in state space. The two single-trial neural trajectories in Fig. 11b exit the optimal subspace in a particular direction, as was the case for all single-trial neural trajectories (gray lines) in Fig. 10b and c. Prediction 7 states that a trial like trial 1 in Fig. 11b should have a shorter RT than trial 2, because (i) the preparatory state at the time of the go cue is within the optimal subspace for both trials (and thus the movement trigger is presumably pulled at the same time) and (ii) the preparatory state for trial 1 (green circle) is nearer the exitedge of the optimal subspace and is thus farther along the stereotyped path that it will need to take to generate movement. If the preparatory state also happened to be moving along the stereotyped path in the standard direction, as opposed to not moving or moving in the opposite direction, then the RT ought to be shorter still.
To test this hypothesis, we correlated, on a trial-by-trial basis, how far along the loop the preparatory state was (at the time of the go cue) with RT. As predicted, we found a statistically significant negative correlation, and primarily in just the exit-edge direction (Afshar et al., 2011). Also as predicted, we found a statistically significant correlation between the direction of movement of the preparatory state at the time of the go cue and RT: preparatory states that were moving in the direction of (subsequent) loop travel had lower RTs than comparably positioned preparatory states moving in the opposite direction (Afshar et al., 2011).
This appears to suggest that a trial with preparatory activity at the time of the go cue (green circle) that is (i) within the optimal subspace and (ii) farther along, and moving in, the standard “loop” direction is in some sense “doubly advantaged” because it is both (i) a well-optimized preparatory state (i.e., within the optimal subspace so the movement trigger can be pulled straight away) and (ii) fortuitously positioned and already headed along the path it will need to take to generate movement. In sum, prediction 7 as illustrated in Fig. 11b is borne out.
The above predictions were derived from a dynamical systems perspective, and to some degree their confirmation argues for that perspective. Yet the most central questions remain largely unaddressed. What is the nature of the relevant dynamics (e.g., Yu et al., 2006)? Do they relate to the dynamics of movement-generating circuits in simpler organisms (e.g., Grillner, 2006; Kristan and Calabrese, 1976)? How and why do dynamics change as a function of overall state (e.g., resting vs. planning vs. moving)? What is the nature of the circuitry, both local and feedback, that produces those dynamics? Answering such questions will likely depend on progress in three domains: (1) the ability to better perturb and probe dynamics, (2) the ability to resolve dynamical structure in neural data, and (3) the ability to relate the recorded “neural trajectories” to externally measurable parameters such as muscle activity and hand movement. We consider these in turn.
First, when reverse-engineering any system, the ability to perturb and observe is critical. As described above, we used intracortical electrical microstimulation to ask how a perturbation of neural activity influenced RT. Pharmacological manipulations are also possible and would offer cell-type specific manipulation of the system. Recent advances in optogenetic stimulation of neurons in rhesus monkeys may also provide important new insights due to the ability to excite and inhibit neurons in a cell-type specific manner (unlike electrical microstimulation), do so on a millisecond timescale (unlike pharmacological manipulations), and not interfere with simultaneous electrical recordings (unlike electrical microstimulation; Diester et al., 2011; Han et al., 2009). This could enable the direct visualization of neural trajectories throughout trials where the neural state is optically perturbed at various times. Recall the Fig. 9 inset, where the curly black line and dashed red line could only illustrate our speculation about how the neural trajectory might evolve during and directly following stimulation, because electrical microstimulation interferes with electrical array recordings. Also, while permanently altering the underlying neural circuitry transgenically is not currently possible in rhesus monkeys, it is possible to reversibly lesion brain regions, cell-types, and neuronal projections pharmacologically and optogenetically. Understanding how the subsequent alteration of neural trajectories relates to altered behavior would deepen our understanding of motor preparation and generation, and the role of specific cells and connectivity (Kaufman et al., 2009, 2010a; Lerchner et al., 2011).
Second, any real understanding of dynamics will hinge upon the ability to go beyond merely plotting state-space trajectories. One wishes to take those seemingly complex neural trajectories, which evolve inmany dimensions, and infermeaningful and parsimonious underlying dynamics (Yu et al., 2006). Indeed, if this cannot be done—if the proposed dynamics are not simpler than the data they seek to explain—then the dynamical systems perspective may have little to offer. Fortunately, it appears that simple dynamics may well be able to explain a considerable amount of the structure of the data (Churchland et al., 2010b, 2011; Cunningham et al., 2011; Macke et al., 2011; Petreska et al., 2011), but further progress in this realm will depend on the continued development of analysis methods that can capture how one neural state leads to the next.
Finally, while the observed state-space trajectories often appear rather abstract, they must exist for a concrete purpose: producing movement. That is, there must be some direct and causal relationship between the neural trajectory and some externally measurable quantity such as muscle activation or arm kinematics. Historically, the relationship between movement- period activity and the parameters of movement has been contentious (e.g., Kalaska, 2009). The dynamical systems perspective will not on its own resolve this debate, but there are a number of contributions it can make. The dimensionality- reduction techniques that produce the statespace trajectories force the experimenter to focus on those patterns that are most strongly present in the data (e.g., Rivera-Alvidrez et al., 2009, 2010a, b). Also, the relatively high dimensionality of the state space makes it clear that not all aspects of neural activity can or should be related directly to external factors: some dimensions may be important to the overall dynamics but may not exert any direct influence on the periphery (Kaufman et al., 2010b, 2011).
Progress in the above domains should also increase the breadth of questions addressable under the dynamical systems perspective. Already it has been possible to ask whether neural variability decreases during learning (Mandelblat- Cerf et al., 2009). More generally, we wish to know what the “state-space” correlate of motor learning might look like. For example, does the location of the optimal subspace change following learning? Or does learning change the dynamics that determine the trajectory away from that planned state? Of course, one suspects that both such mechanisms might be at play, perhaps depending on the type of learning (e.g., the former strategy might be more rapid but less flexible). The further development of critical tools, as described above, could open the door to many experiments of this type.
Neural prosthetic system design depends critically on a fundamental and comprehensive scientific understanding of how populations of neurons prepare and generate natural movements (Green and Kalaska, 2010). How neural populations evolve on a fast timescale is of particular interest, as neural prostheses must operate rapidly to ensure accurate and stable control. Figure 12 highlights perhaps the most fundamental problem that basic neuroscience (reviewed above) and neural prosthetic systems have in common: how to understand noisy electrical activity from a population of neurons on a millisecond timescale and on a single-trial basis. Figure 12 shows the now familiar instructed-delay reach task and, for the first time in this review, reasonably raw and unprocessed electrode-array neural data. The most striking feature is how noisy spiking data really are. Staring at this figure for a few moments reveals the subtle difference in response pattern between the preparatory period following target onset and the baseline period preceding it. More obvious is the difference between the movement period and preparatory period. While the eye is often poor at discerning signal from noise, particularly without the visual benefit of trial averaging, this is a useful exercise as it helps one appreciate the challenge before us as neuroscientists and neuroengineers. We seek to understand how these neural responses arise, how they support behavior, and how we can use these volitional signals as an information source for controlling neural prostheses. While several important design insights for prostheses have already resulted from a deeper scientific understanding, some of which are discussed briefly below, there is little doubt that the most important leaps forward in prostheses will result from future scientific discovery. This has historically been the case, with science deeply informing engineering.
In addition to how the basic scientific understanding of brain organization and movement control has already informed neural prosthetic system design, as described in the Introduction, there are two additional points to briefly note here. First, communication prostheses rely on preparatory activity, as can motor prostheses (e.g., Musallam et al., 2004; Santhanam et al., 2006; Shenoy et al., 2003; Yu et al., 2007, 2010). These systems can make use of new discoveries such as preparatory activity in PMd reflecting the speed of the upcoming movement. Second, at the heart of the dynamical systems perspective and the associated quest for single-trial neural trajectories is time. How long does it take for the neural trajectory to actually traverse from baseline to the optimal subspace where, once there, the neural activity can be fruitfully decoded and used to guide a prosthesis? This is precisely the “transit time” we needed to know as part of our recent prosthesis research, so that we could skip this transition period to avoid inadvertently decoding neural activity that is still in flux. This time (Tskip; Santhanam et al., 2006) is approximately 200 ms as measured with single-trial neural trajectories in scientific experiments (as described above) and agrees with measurements from neural prosthetic experiments. Similarly, for prosthetics designs, it is important to know how long neural activity should be integrated (Tint), so as to best estimate the parameters of interest, and this is related to how stable preparatory neural activity is while in and around the optimal subspace. Single- trial neural trajectories can, and have, revealed important features which will continue to inform the design of neural prosthetic systems.
The ability to move voluntarily is central to the human experience. By pursuing a deeper scientific understanding of the neural control of natural movement, it should be possible to advance the design of neural prostheses, with the goal of helping patients who have lost their ability to move. A potentially underappreciated part of controlling movement is preparing movement. Preparation is, after all, how each movement begins. Motor preparation can be studied in many different ways. We have elected to adopt a dynamical systems perspective in order to facilitate the construction of hypotheses, and set about testing their predictions. The optimal subspace hypothesis has led to seven tested predictions. It appears that this dynamical systems perspective, and closely associated state-space diagrams, is helping to generate an ongoing series of testable predictions. As a result of the recent studies reviewed above, numerous questions are now more apparent and remain to be addressed as described in the Future Directions. We believe that the dynamical systems perspective should continue to help generate new and testable ideas and lead to deeper insights for both basic and applied neuroscience.
This work was supported by Burroughs Wellcome Fund Career Awards in the Biomedical Sciences (K. V. S and M. M. C.), DARPA REPAIR N66001-10-C-2010 and NIH-NINDS CRCNS R01-NS-054283 (K. V. S. and M. S.), an NIH Director’s Pioneer Award 1DP1OD006409 (K. V. S.), a National Science Foundation graduate research fellowship (M. T. K.), and the Gatsby Charitable Foundation (M. S.).