We have developed an intracortical NIS that enabled persons with tetraplegia to move a computer cursor to an arbitrary position on the screen, stop the cursor, click on the area of interest and move to another position, without any interruption, automated recentering of the cursor or other external intervention used in previous BCIs. One participant (S3) was able to smoothly control cursor velocity and switch between cursor movement and clicking using different imagined movements. The other participant (A1) was able to move the cursor and click on targets, but cursor velocity control was relatively poor in the single session appropriate for analysis. A key advance of this study is the demonstration of online decoding of both continuous and discrete motor signals from a small population of motor cortical neurons in the dominant arm-hand area of humans with tetraplegia. The decoding model combines two methods: Fisher discriminant analysis and the Kalman filter. The simplicity of these two methods makes real-time implementation practical and decoder training easy. We modified previous filter building methods [13
] for users with paralysis by incorporating a new discrete state training method into the paradigm. We found that some particular imagined motions (e.g., squeezing the hand for S3 or opening the hand for A1) modulated motor cortical activity in a way that could be readily discriminated from continuous cursor movements and, therefore, could be used to generate a click signal. With well-established measurements for the effectiveness of non-keyboard pointing devices [29
], we have quantified the point-and-click performance of two participants (including one over multiple recording sessions). This assessment approach provided more comprehensive and practical measures of cortical cursor control than conventional measures such as mean squared errors or correlation coefficients. We believe that these measures will be useful for future BCI research, for which there is no currently accepted set of standard performance metrics [32
We found motor cortical units that exhibited distinct tuning to both cursor direction and “click” movements. We cannot rule out the possibility, however, that these units were, in fact, multi-units of two distinct neurons, each being tuned to a different movement feature, because mixtures of neurons can occur with these fixed electrodes. Importantly, other work has found that hand and arm information is intermingled within the MI arm region, as sampled by this sensor [13
]. From the perspective of neural prosthetic development, however, the important observation is that such a classification is possible using only this small population of cells.
It would be of practical interest to know how many neurons are needed to achieve reasonable control of both pointing and clicking. One possible way is to address this point is performing the neuron dropping analysis [3
] in which we remove one or more neurons at a time from the decoder and evaluate the effect on accuracy during closed-loop control. As we observed diverse groups of neurons tuned to velocity, clicking or both, the neuron dropping analysis may be done for each tuning group independently. Of course, the better-tuned units are likely to contribute more to generating correct control signals. If there is a linear relationship between the tuning depth and the contribution to control, we might be able to predict the level of cursor control from the degree of the tuning of all recorded units even before executing a closed-loop experiment. This could be useful for determining what level of control may be possible and thus allow a technician to change properties of the interface to suit the conditions (e.g., making the targets larger or smaller, or changing the integration window for click decoding).
2-D cursor control performance with the intracortical NIS shown in this study differs from previous BCI studies [8
]. After our initial report on point-and-click cursor control in a human [34
], McFarland et al.
demonstrated 2-D cursor control with target selection using EEG signals in humans [9
]. Besides fundamental differences in cortical signals, an important difference is the way the cursor was controlled. First, we simultaneously decoded both continuous cursor kinematics and click signals from the same neural ensemble while McFarland et al.
operated cursor control in two separate modes—continuous cursor movement and discrete target selection—and the switch between modes was conducted based on non-neural signals (i.e., whether or not a target was reached). Second, in our study, the cursor was always present and had to be held on the target by the participant until the target was selected. In McFarland et al.
, the cursor movement was frozen once it came in contact with a known target location. Hence, the user could focus only on the target selection with no need to control the cursor movement. Third, we did not use an automatic recentering of the cursor in the workspace while most previous studies have relied on this. Fourth, McFarland et al.
updated the decoder parameters using information about the target after acquisition. However, in a natural cursor control task, the true target is unknown, making such updating infeasible in practice. Consequently, we trained the decoder parameters and then held them fixed during testing.
A1 participated in three point-and-click sessions through the entire pilot trial and did not achieve point-and-click control in two of them. In these two sessions, the neuronal ensemble did not modulate well with cursor direction or click state during training, so we ended the session without evaluation. In the single session reported here, A1 achieved some level of point-and-click control. A1’s click performance was only slightly worse than S3 (1 versus 0.74 of FCR), showing that A1 could volitionally generate clicks when intended. A1’s velocity control was quantitatively worse than S3’s, resulting in an overall 47.4% error rate; this error rate was due entirely to runs ending as a result of the timeout. This error rate was, however, roughly comparable to previous studies of 2-D velocity control in which A1 had a 31.8% error rate on average with a simpler four-target task [14
]. With a more complicated cursor control task (smaller targets, click selection and all targets displayed) as in this study, the roughly 17% increase in error rate is not surprising. However, it is an obvious problem that the neural cursor tended to move in a particular direction (i.e., lower-left corner of the screen). The cause of the movement bias to the lower-left portion of the screen is not known (see [14
] for a discussion of related issues). Current work is exploring the possible sources of performance variability. It is noteworthy, however, that control was achieved in participants with both ALS, a progressive neurodegenerative disease, and after long-standing stroke.
Although the cursor was controlled reasonably well by motor cortical activity, there is still significant room to improve control, which might be accomplished by improving the encoding and decoding models. For instance, the maximum speed of the neural cursor was markedly slower than cursor speed generated by able-bodied users. We hypothesize that this slower cursor movement might be in part caused by the linearity assumption in our decoding model. Future work should explore nonlinear models as well as acceleration decoding. To better discriminate movement and click states, we are currently considering more accurate classification methods such as a logistic regression, a Bayes classifier with reduced input dimensions, or a support vector machine with a linear kernel. A more sophisticated temporal model of changes in discrete state, such as a hidden Markov model, might improve the FCR. We also aim to determine the optimal classification threshold for our current classifier from training data using cross-validation.
In our study, a click state meant that the cursor came to a stop. However, it is possible to define the discrete state (γk) in different ways for different computer applications. For instance, if a given application requires dragging an item from one location to another on the screen, we should decode both the velocity and the click at the same time. In that case, we can simply change the outputs of the discrete state such that γ(1) keeps the cursor moving and γ(0) simply generates a click without stopping the cursor. As the imagined motions for the continuous and discrete states are made independently, we can adjust the discrete state outputs flexibly to the user applications.
The demonstration of reliable and accurate point-and-click cursor control provides further evidence that even the use of a single implanted microelectrode array might be useful for neural cursor control applications. The development of a natural, reliable, and fast point-and-click interface for people with tetraplegia would be extremely valuable. A reliable control signal could be used to operate most computer software as well as commercial assistive technology. This study, together with [14
], presents the first demonstration of an intracortical point-and-click cursor neural interface used by humans with tetraplegia. The results provide a proof of concept that a direct neural interface system can provide people with paralysis the ability to achieve continuous control and make goal selections using a small ensemble of MI neurons even when paralyzed for years.