Worldwide, over 130,000 people each year survive spinal cord injury (SCI) with significant paralysis 4
. Roughly half of these injuries occur above the 6th
cervical vertebra, thereby affecting all four limbs. Most of these patients indicate that regaining the ability to grasp objects, would provide them the greatest practical benefit 5
For this reason, considerable effort has been devoted to the development of functional electrical stimulation (FES) systems to restore voluntary grasp 1,2,6
. These systems rely on residual movement or muscle activity to control electrical activation of hand muscles. Because of the complexity of the necessary patterns of muscle activation, current FES systems produce only one or two grasps using preprogrammed stimulus trains customized for each user 7
. This is effective because many objects can be grasped adequately with only palmar or pinch grasp. However, normal hand use is much more complex than this. Furthermore, using the motion of one body part to control that of another inevitably increases the associated cognitive burden. If FES is to provide more nearly normal hand movement, a higher dimensional, more natural control signal will be necessary.
Fortunately, the rapid development of the brain machine interface (BMI) provides a promising new means by which more flexible and dexterous movements might be controlled. However, despite the initial demonstration by Evarts of the strong force-related discharge in the primary motor cortex (M1)8
, virtually all existing BMIs extract only kinematic information from the brain. This bias is the more ironic, as the first study to decode signals from simultaneously recorded neurons was that of Humphrey, who also found force to be more strongly represented than movement in M1 9
Only a small number of groups, including ours, have pursued the possibility of using kinetic (force-related) information as a real-time control signal for a BMI, through the prediction of grip force 10,11
, joint torque 12
, or muscle activity 10,13,14
. We showed previously that despite paralysis produced by peripheral nerve block, monkeys could accurately modulate the magnitude of isometric flexion and extension wrist torque using cortically-controlled FES 15,16
. Related results using a rather different approach, were subsequently reported by a group that operantly conditioned monkeys to modulate the activity of one or two individual neurons, whose discharge directly controlled stimulation of individual muscles 17
We performed the current experiments with two monkeys trained to pick up weighted rubber balls and to convey them to an opening at the top of a dispenser (). After training, each monkey was implanted with a multi-electrode recording array in the hand area of M1. In a separate surgical procedure, we implanted intra-muscular electrodes for recording and stimulation of hand and forearm muscles. illustrates the neural discharge recorded under normal conditions from a representative session. Most of these 104 neuronal signals were well modulated during at least some portion of the task. Offline, typically 50–75% of the neuronal signals could be discriminated as single neurons, based on the consistency of their waveform shape and inter-spike interval histogram distribution. However, under real-time conditions, only about one-third of the inputs were well-discriminated single units, the remainder being signals that included action potentials from more than one neuron. Panel B shows the discharge of these neurons averaged over 229 trials, aligned to the time of contact with the ball. The varied phasing of the different neurons is quite evident.
Figure 1 Brain-controlled Functional Electrical Stimulation (FES). The monkey’s forearm and digit flexor muscles were temporarily paralyzed by a peripheral nerve block. Recordings from the motor cortex were used to infer the monkey’s attempted (more ...)
Figure 2 Grasp-related raw data collected during normal conditions. A) Firing rates of 104 neuronal signals recorded during two grasps. B) Ensemble average of 229 trials aligned to time of ball contact. C) Actual and predicted EMG during the same period as (A), (more ...)
Simultaneously with the neural recordings, we recorded from flexor and extensor muscles of the hand and forearm (). One can appreciate in panel C, the variation both in magnitude and duration that occurred from trial to trial. The muscles have been ordered by the relative times of their onset, and panel D shows rather clearly, the difference in average timing and patterns of activation of the different muscles.
We were able to predict EMG activity with very good accuracy, typically from approximately 100 neural signals (; red traces), using Wiener cascade decoders. These consisted of multiple-input, linear impulse response functions between the neural inputs and each muscle, followed by a static nonlinearity. Each impulse response was composed of 10 lags, extending 500 ms in the past. At the beginning of each week, we collected 20 minutes of data under normal conditions which we used to compute a decoder used for the remainder of the week. Accuracy was represented by R2
, calculated using a multi-fold cross-validation procedure described in the online supplementary material
. Using these real-time predictions of muscle activity, we stimulated up to five electrodes in three different muscles (FCR, and medial and lateral sites in FDS and FDP). By this means, we have restored in two monkeys the ability to pick up and move objects despite complete paralysis of the flexor muscles in the forearm and hand. We began each FES experimental session by collecting data under normal conditions to establish baseline performance. Following these baseline recordings, we injected lidocaine through nerve cuffs implanted proximal to the elbow that blocked the median and ulnar nerves. After 15–20 minutes the nerve block was complete, as determined by the loss of flexor muscle EMG activity (see supporting online material, Figs. S1
) and the onset of profound motor deficits. We made periodic tests of nerve block effectiveness throughout each session (Fig S2
), and we used a standardized stimulus train to evaluate the level of fatigue induced by the stimulation (Fig. S2
A series of four trials is shown in , illustrating typical neural discharge, predicted EMG and stimulus commands, as well as markers of the monkey’s performance. Although the common digit flexors (FDS and FDP) are normally activated nearly synchronously, FDS activation tended be more sustained, while FDP was more phasic. The pulse widths of the stimulus trains were determined from the predictions using a mapping procedure described in the supplemental materials and Fig. S3
. The distribution of these pulse widths from zero to 200 μs suggests that the monkey was able to grade the strength of contraction continuously (Fig. S4
Figure 3 Grasp performance during four consecutive brain-controlled FES trials. A) Neural data B) Predicted EMG signals (red traces) transformed into stimulus commands (black traces). Vertical dashed lines: go tone (“Go”), time of initial ball (more ...)
During the FES trials, the monkey grasped and moved the ball reliably. The movements did not differ sufficiently from normal to be obvious to casual observation (see Videos #1 and #2 for representative examples from both monkeys). On occasional “catch” trials, we turned off the neuroprosthesis at the beginning of the trial, to test the ability of the monkey without FES. On the single example illustrated here (note flat stimulus trace in ), the monkey was unable to grasp the ball despite the considerable effort apparent in the neural discharge and predicted EMG.
After the onset of paralysis, each experimental session consisted of a series of 10 minute sets of trials like those in , in which the monkey attempted to complete the grasp task either with or without FES assistance. Two complete sessions for both monkeys are summarized in . The success rate in these sessions using the neuroprosthesis was approximately 80% and 90% for the two monkeys, respectively (heavy green lines). In stark contrast, the average catch trial success rate was 5% for Monkey T and 0% for monkey J (blue lines). The average number of trials per session varied fairly substantially across sessions, with a mean of 272±84 for Monkey T, and 208±112 for monkey J. Although we explored different types of balls, we did not systematically examine the effects of size, weight or texture on the monkey's performance. It is likely that the FES success rate would have been lower with significantly heavier or slipperier balls. We used balls that seemed to represent a “normal” level of difficulty (e.g., a human picking up an apple).
summarizes both monkeys’ overall success rate across all sessions, with the FES neuroprosthesis and during catch trials. Both monkeys achieved about 80% using the neuroprosthesis, a level that was highly significantly different (p ~ 0) from that of the catch trial condition. In addition to effecting the greatly improved success rate, the FES neuroprosthesis also significantly increased the speed at which the monkeys completed successful trials (not shown; p<0.0001 for both monkeys, two-tailed Mann Whitney test). In order to test force control more systematically we conducted a second set of experiments with Monkey J, who was trained to control the vertical displacement of a cursor that moved in proportion to palmar grip force. Using the neuroprosthesis, the monkey was able to squeeze a pneumatic tube, and to track as many as three different targets ranging from 15 to 80% of his normal maximum voluntary contraction (MVC) each target having a width of approximately 20% of MVC. To be successful, the monkey needed to maintain the target force for 0.5 seconds. shows a short sequence of data during this target tracking task. One of these four trials was a catch trial. The monkey was unable to generate any significant force during the catch trial despite two attempts that are evident in the predicted EMG signals.
Figure 4 FES used to produce controlled palmar grip force during the palmar grasp task. Rectangles indicate the level and time of appearance of force targets. This segment shows three target levels, with the two extremes non-overlapping. The white trace is the (more ...)
We quantified this performance by measuring the mean force and stimulation pulse width during the target-hold periods of the initial and final 10 minutes of the session. Despite considerable FES-induced fatigue, the monkey remained able to achieve the required force throughout the session by voluntarily increasing the mean stimulus pulse width (see Fig S5
). The increased pulse width reflects an increase in cortical activity and resultant EMG predictions. The monkey apparently overcame the fatigue much as it would have under normal conditions, increasing its effort in order to regulate force accurately.
The monkey’s ability to control both a well-regulated palmar grip as well as to execute the unconstrained, natural grasp is powerful evidence of the impact this FES neuroprosthesis could have in eventual clinical application. Our neuroprosthesis makes use of patterns of activity in M1 that reflect the patterns that occur naturally during grasp. By matching patterns of neuronal activity to those muscles with which they are normally most closely correlated, we hope to maintain the natural coupling between cortical activity and motor output.
It is important to note that this process in no way limits the ability of the brain to adapt further, to compensate for inaccuracies in the decoded signals. However, even with adaptation or conditioning, it is difficult to imagine how a small number of individually conditioned, randomly selected neurons could yield an adequate level of control without the type of preprogrammng that is necessary with existing FES systems. Indeed, there is no evidence that it is possible to learn to associate the simultaneous activity of two, three, or more neurons with independent patterns of muscle activity. Even if possible, the cognitive load associated with this effort would presumably be rather high, while the reliability of a neuroprosthesis relying on a small number of conditioned neurons would be quite low.
Our model of paralysis avoided many of the complications of actual spinal cord injury, including muscle denervation and spasticity 19,20
. Furthermore, it was limited to the forearm and digit flexors. Patients with a C5-C6 spinal cord injury retain voluntary control of proximal arm muscles while losing full control of the more distal limb. Many retain or regain some level of voluntary wrist extension 21
. As we did not paralyze the monkey’s extensor muscles in this experiment, it is important to recognize the good coordination between the remaining natural muscle control and that achieved through the neuroprosthesis. We did routinely obtain extensor EMG predictions that were actually somewhat more accurate than those of the flexors. In future experiments, we intend to expand our control to these muscles.
This technology may offer even greater advantages to patients with injuries at higher levels, who have greater need for replaced function, yet possess even fewer available sources of control22
. In addition to the distal limb muscles considered in this study, we showed previously the ability to predict the reach-related activity of proximal limb muscles, suggesting the possibility of extending this control to these muscles 13
. In addition to providing patients greater independence, FES is also established as an effective means for exercising the muscles of paralyzed patients, bringing a range of health benefits: stronger muscles and bones, improved metabolism, cardiorespiratory health and reduced propensity to pressure sores 23,24
. It may well be that drawing on a conscious process to restore natural movement will bring the additional benefit of improved psychological health 25