|Home | About | Journals | Submit | Contact Us | Français|
Future generations of upper limb prosthesis will have dexterous hand with individual fingers and will be controlled directly by neural signals. Neurons from the primary motor (M1) cortex code for finger movements and provide the source for neural control of dexterous prosthesis. Each neuron’s activation can be quantified by the change in firing rate before and after finger movement, and the quantified value is then represented by the neural activity over each trial for the intended movement. Since this neural activity varies with the intended movement, we define the relative importance of each neuron independent of specific intended movements. The relative importance of each neuron is determined by the inter-movement variance of the neural activities for respective intended movements. Neurons are ranked by the relative importance and then a subpopulation of rank-ordered neurons is selected for the neural decoding. The use of the proposed neuron selection method in individual finger movements improved decoding accuracy by 21.5% in the case of decoding with only 5 neurons and by 9.2% in the case of decoding with only 10 neurons. With only 15 highly-ranked neurons, a decoding accuracy of 99.5% was achieved. The performance improvement is still maintained when combined movements of two fingers were included though the decoding accuracy fell to 95.7%. Since the proposed neuron selection method can achieve the targeting accuracy of decoding algorithms with less number of input neurons, it can be significant for developing brain-machine interfaces for direct neural control of hand prostheses.
A Brain-Machine Interface (BMI) is a methodology which enables a brain to communicate with an external device bypassing normal neuromuscular systems. Currently, a BMI has drawn much interest as an appropriate alternative for restoring both motor control [1–4] and sensory feedback to amputees so that they can again perceive heat, cold, pressure, and the position of a limb in space . BMI systems collect neural activities from various cortical areas, such as the primary motor, premotor and posterior parietal cortex, and interpret the encoded motor-intent into control commands or kinematic parameters [2–4]. Up to now, many relevant studies have been exploited such as a closed-loop control of a computer cursor and target tracking, reaching and grasping task of a hand [5–7].
Presently dexterous, multi fingered prosthetic limbs are under development. Neural control of dexterous hands will require signals from a population of neurons coding for the hand and finger movements. Hence, an important problem in neural prosthesis control is to select, and preferably rank in terms of their relative importance, neurons coding for individual finger movements. Solution to this problem requires a trade-offs between achieving high decoding accuracy and low computational complexity. Since not all recorded neurons contribute equally to the all movements, since some neurons are related weakly or not at all to the specific movements, the use of as many neurons as possible does not guarantee high decoding accuracy and may even degrade the performance of the decoding algorithm. In addition, the increase in the number of input neurons puts a computational burden on finding an optimal solution especially when the goal is to implement such decoding algorithms in an experimental hardware [8, 9]. Therefore, developing a metric for evaluating the contribution of neurons selected for BMI tasks is at the core of designing an efficient real-time BMI. Researchers have developed some techniques to evaluate the relative importance and select the best neurons coding for the information [9–12]. Sensitivity analysis and single neuron correlation analysis through a vector linear model were proposed to quantitatively rate the importance of neurons in neural to motor mapping [9, 10]. These analyses depend on the decoding model and thus they are not easy to interpret from neurophysiologic points of view . In another approach, a neuron’s individual removal error was defined and then used for representing its importance in the population vector neural decoding method . Recently, information theoretical analysis based on an instantaneous tuning model was applied to extract the important neuron subset for neural decoding on BMI . However, these quantification methods of neurons’ importance have been developed for predicting intended reach and cursor control and are not yet targeted towards achieving dexterous hand and finger control. Thus, a very active area of research currently is to develop neural control of dexterous hand prosthesis, i.e. provide realistic control strategies for actuation and control of individual and combined finger movements [13–23]. However, methods of neuron selection for complex finger movements, have not been well developed, and as such is the subject of this paper.
We present a new simple metric for quantifying the relative contribution of a neuron towards finger movements. Based on the change in firing rate before and after the starting moment of each instructed finger movement, we define a random variable for the change in firing rate. Then, finding the means of each random variable over six trials for each movement, we finally compute the variance of the means over whole movements for each neuron. A larger variance of neural activations over each finger movement means that the corresponding neuron is activated distinguishably for each finger movement, and it can contribute to accurate decoding performance of all finger movements. Thus, we use these variances as a new metric for ordering neurons. With the ordered neurons we performed maximum-likelihood (ML) neural decoding  and then compared the performance with that of randomly selected neurons. Our objective is to demonstrate an improvement in decoding accuracy.
The remainder of the paper is organized as follows: In Section II, neuron selection based on neural activity is developed and ML neural decoding with the selected neurons is introduced. Section III shows the performance improvements with the selected neurons by comparing ML decoding performances with and without selected neurons. We analyze the decoding performance when both individual and combined finger movements are included. Section IV presents the conclusion.
Three male rhesus (Macaca mulatta) monkeys—K, G, C—were trained to perform visually-cued movements of individual fingers and the wrist movements. In addition, the monkey K was trained to perform combined finger movements involving two digits in order to test the decoding accuracy of more dexterous and complex movements. The monkeys were prepared for single-unit recording by surgically implanting both a head-holding device and a rectangular Lucite recording chamber that permitted access to the area encompassing M1 contralateral to the trained hand . These recording were obtained using self-made, glass-coated, Pt-Ir, microelectrodes. Recording tips were etched to be parabolic in shape and approximately 10 μm wide 20 μm back from the tip . There were 12 distinct individual movements: flexion (f) and extension (e) of each of the fingers (1=thumb, …, 5=little), the wrist (w) of the right hand, and six combined two-finger movements: f12, f23, f45, e12, e23, e45. The monkeys placed their right hand in a pistol-grip manipulandum; this grip separated each finger into a different slot. The pistol grip manipulandum was also mounted on an axis allowing flexion and extension of the wrist. The monkeys were instructed to flex or extend a single digit until a microswitch was closed. The duration of each trial was approximately 2 sec, and for analysis all trials were aligned such that switch closure occurred at 1 sec . Throughout these investigations, the monkeys were cared for according to the “Guiding Principles for Research Involving Animals and Human Beings” accepted by the American Physiological Society . A detailed description of the methods used to train the monkey and the actual experimental protocols can be found in  and . Single-unit activities were recorded from 115 task-related neurons in the M1 cortex of the monkey. Independent trials of each type of movements were recorded six times.
To define the neural activity, we need a random variable representing firing rate of a neuron for each finger movement. Let rn(m) be a random variable of firing rate of a neuron n for a movement type of m. Specifically, rn(0m) denotes the baseline activity of the neuron n before the movement of m. Then, we can define the neural activity considering only movement by introducing the following random variable :
Since the random variable of xn(m) represents the change in firing rate before and after the starting moment of instructed finger movement, it can be used as a metric of neuron’s activation for respective movements. Considering the randomness of neural activity, we can determine the neuron’s sensitivity to a particular finger movement m by obtaining the ensemble average of xn(m), i.e., E[xn(m)]. The estimate of E[xn(m)] is usually computed by averaging the recorded neural activation, xn(m, k), for possible training sets, that is
where P is the number of independent training sets. As P increases, the reliability of the metric can be also improved. However, the increase of P means that more training data are needed and thus there is a trade-off between the data size and the metric reliability. Letting M be the total number of tested movement types, then the neuron n has M μn(m)s and each μn(m) represents the estimate of the neuron’s activity corresponding to the movement of m. As a result, the value of μn(m) can be considered a straightforward metric for the absolute degree of neural activation ascertaining how much the neuron n contributes to a particular movement of m independent of other movements. This metric, however, cannot be directly applied for selecting the input neurons because the goal of neural decoding is to find the unknown movement from recorded spike signals. For any metric to be available for ascertaining the importance of a neuron when selecting an appropriate input neuron set for neural decoding, it should reflect the relative difference of activations among all the tested movements, not the absolute magnitude of activation for a particular movement. To achieve this goal, we define a relative importance of a neuron n with the inter-movement variance of neural activities as the following equation,
where n is the mean of μn(m) for the tested movements given by
For a neuron to be distinguishably activated for each finger movement, the μn(m) of a neuron should be separable for each m so as to mitigate the probability of false detection in the neural decoder. That is even if the μn(m) of a neuron is large for several different movements, there could be a case where those movements are analogous to each other, then its contribution for neural decoding becomes relatively low as shown in Fig. 1(a). On the other hand, Fig. 1(b) shows the opposite case where a neuron is slightly activated for most of movements but its neural activations, μn(m)s, are separable from each other, then this neuron has high impact contribution for neural decoding. Fig. 2 briefly describes the process to calculate the relative importance of each neuron: we compute the estimates of the neural activities, μn(m) (m=1, …, M), for a neuron n over each trial data and then find the inter-movement variance of the neural activities. Specially, in the ML decoding method, the neural activities, μn(m), represent the estimates of the ensemble means of each likelihood function . After computing the relative importance of each neuron, Vn, we can rank neurons by Vn for neural decoding.
Neuron selection based on the proposed method of assigning relative importance of each neuron to a given movement is independent of the model used for neural decoding and thus may contribute to reduction of the required number of neurons for achieving the target performance. Several different methods have been presented for decoding finger movements from neural activity [12–20] and we can use any of these to show the effectiveness of our neuron selection method. Specifically, we will use the ML decoding method described in  which is based on Skellam and Gaussian distributions.
ML decoding estimates an unknown parameter, m, corresponding to the intended movement, so that the probability function Pr(x1, x2, …, xN|m) is maximized, i.e.
where N is the total number of neurons used for ML decoding. Without any preference of neurons, we may need a large number of neurons to obtain the desired decoding performance because N neurons should be randomly selected and thus they might include neurons irrelevant to the movement. If the limiting factor is the total number of neurons, N, it is best to choose neurons with high importance for the movement. We choose highly-ranked N neurons in the neurons ordered by Vn.
In order to use (5), we need to define the probability distribution function corresponding to the random variable xn(m). In , Skellam and Gaussian distributions were considered and the performance was good for both cases. Skellam and Gaussian distributions are given by
where Δt is the time interval for observation of neural spikes,
Ix(z) is the modified Bessel function of the first kind,
Assuming that xi and xj (j≠i) are independent , we get
for Skellam distribution and
for Gaussian distribution, respectively.
The proposed method for neuron selection was examined by comparing the performance of the ML decoding based on Skellam and Gaussian distribution, using randomly selected neurons and highly-ranked neurons by the proposed method. In order to test the decoding result of various finger movements, we mainly used the spike data recorded from monkey K trained for both individual and combined finger movements. In (1), the firing rate of rn(m) was obtained by averaging the number of spikes for 300ms after the movement of m and the baseline activity of rn(0m) was obtained by averaging the number of spikes for 800ms before the movement of m. As shown in Fig. 3, six independent trials were recorded for each movement, with five trials out of them used for training, i.e., P=5, and the remaining trial was used for testing. Thus, six different combinations can be used for training and testing in the form of a Jackknife test. In , it is shown that the converged performance of the ML decoding based on statistical estimation with five training sets (P=5). However, to make a reliable judgment about the validity of the proposed method, we performed the ML decoding of individual finger movements using the spike signals recorded from the other monkeys (G and C).
If N neurons are randomly selected among total 115 neurons, there are 115CN combinations of input sets. To raise the reliability of decoding performance, 400 input sets out of 115CN combinations were randomly selected. On the other hand, for the rank-ordered neurons, if N neurons are used for ML decoding, there is only one input neuron set, consisting of the first-ranked neuron to the N-th ranked one. In order to increase the possible neuron sets, we added next-ranked L neurons to the highly-ranked N neurons, resulting in (N+L) ranked neurons. The value of L was empirically set to L=10 in our testing. Then, a subset of N neurons was randomly selected from (N+L) neurons. So, the possible number of selections becomes N+LCN and thus the total number of tested movements for neural decoding was 12×6×N+LCN in the case of 12 finger movements and 6 trials.
We first computed the estimates of the neural activities μn(m), which represent the absolute contribution impact of each neuron for the 12 individual movements. This computation was performed to verify capability of the metric for determining neurons’ importance. Fig. 4 shows that each neuron has various values of μn(m) for some different types of movements. As previously mentioned, each neuron has 12 μn(m)’s and we can get 12 rank-ordered lists as in Table I. To clearly show the effectiveness of μn(m), we chose one ordered neuron set out of a list of 12 ranked neurons, which corresponds to one of 12 columns in Table I. Then, we used highly ranked neurons for ML neural decoding and compared the false detection rates over the desired movement, which we denoted as md. Since μn(m) is most highly related to the movement m, it reflects the lowest false detection rate (FDR) when m equals md. Fig. 5 shows the FDRs for each desired movement, using either 5 (top) or 10 (bottom) neurons ordered by μn(1), μn(3), μn(5), as well as the case of a random selection. In the first case, the numbers of 1, 3, and 5 denote the finger movement f1, f3 and f5, respectively. For both N = 5 and N = 10, the FDR was lowest when the desired movement was the same as the movement type whose neural activity was used for ordering neurons. The difference between the lowest FDR and those of the remaining given movement types decreased with 10 ordered neurons.
The ordered result by μn(m) only shows the activity level of each neuron involved in the particular movement m, so it does not assure the accurate neural decoding for the remaining 11 movements. Because the desired movement is not revealed in real environments, the neural activity is not appropriate as a metric for ascertaining the importance of neurons.
The neural activity for the specific movement gives the information mainly related to the specific movement and thus we need to obtain the relative importance by computing the variance of the neural activity, Vn. For the 12 individual movements, we computed Vn for each neuron and arranged them in the decreasing order. Fig. 6 shows Vn and n for the ordered neurons. There we can find that some neurons have larger values of n than others. Since n is the mean value of μn(m) for all movements, it represents the estimate of the neural activity of the neuron n. This means that some neurons with high neural activity may have low relative importance if they have the similar contributions for all movements. From the plot of Vn we can also find that a knee occurs near the 10th neuron, which makes us expect that neural decoding performance would saturate around 10 neurons.
To investigate neural dependency on movement classifications, we also calculated Vn values for combined two-finger movements and the total movements including both individual and combined movements. To clarify the ordering criterion, let us denote the three variances of the individual, the combined, and the total movements by Vn,I, Vn,C, and Vn,T, respectively. Figs. 7 and and88 show the values of Vn,C and Vn,T for the combined and the total movements, respectively. It is clear that these three types of movement classifications have different orderings but we need to know how the ordering is changed depending on the movement classifications. From Table II, it can be seen that some neurons have large standard deviations and thus are differently ranked according to movement classifications. For example, neuron ‘K20301’, which has the largest standard deviation, is ranked 6th, 101th, and 12th for the individual, the combined, and the total movements, respectively. From this analysis we can predict that neural decoding of combined movements with the ordered neurons based on the individual movement does not have comparable performance to that with the ordered neurons by the combined movements.
The effect of neuron selection is evaluated by the ML decoding performance with ordered neurons and with randomly selected neurons. Fig. 9 shows the improved decoding performance by incorporating the proposed neuron selection methods. The criterion for the neuron selection was the variance of the individual movement activity, Vn,I. As predicted in the previous section, the decoding accuracy with only 5 neurons approaches 86.8% while the use of randomly selected neurons yields an accuracy of 62.3% with the same number of neurons, and it shows a performance improvement of decoding accuracy by 24.5%. In case of using 10 neurons, the decoding accuracy is also improved by 9.2% compared to the case of randomly-selected neurons. Also, we know that the proposed neuron selection method is robust to decoding algorithms because the performance improvement using Gaussian distribution is similar to that of Skellam distribution. In addition, Fig. 9 shows the better decoding performance than the previous selection method using the tuning depth of a neuron, which was defined as the difference between the maximum and minimum values in the cellular tuning . In addition, we performed the ML decoding using the spike signals recorded from the other subjects (monkey G and C) to generalize the validity of the proposed method. In both cases, the decoding results using the selected neurons by the proposed method show better decoding accuracies compared to the case of using randomly selected neurons as shown in Fig. 10.
We also checked the possibility of using the estimate of the neural activity for neuron selection. As addressed in the previous section, each estimate depends on the corresponding movement type and thus we have to choose one of 12 estimates of μn(m) which are related to the 12 individual movements, respectively. Fig. 9(a) shows the ML decoding performance of individual movements using different neuron groups chosen by different criteria. As discussed in section III. A, the use of μn(m) does not give any remarkable merit for neural decoding because it is the partial information only corresponding to the specific movement.
Next, we tried to test the availability of the neuron selection method for decoding of the six combined two-finger movements. Fig. 11 shows the ML decoding performance of combined movements using the different neuron selection methods. For decoding combined movements, the neuron-selection based on the relative importance of the combined movement activity produced an outstanding improvement in decoding performance of 98.3% with 10 neurons. On the other hand, neuron-selection based on the relative importance of individual movements in combined movements does not perform as well, as it yielded an accuracy of 93.5% with 10 neurons. This means that, in order to achieve the improved performance, we need training data for obtaining the required parameters.
Finally we tested the neuron selection methods for total movements. Fig. 12 shows the ML decoding performance of total movements using ordered neurons or not. In this case, the decoding performance does not converge to 100% for any of the neuron selection methods, not even when the number of neurons was increased. Because the usual grasping or control of an object is synthesized by the combination of many individual finger movements, the accurate neural decoding of total finger movements is essential to implement more practical BMI applications and requires further analysis.
The average decoding accuracy approaches 99.5% with only 15 neurons when individual movements (Fig. 9(a)) and combined-finger movements (Fig. 11) are separately decoded. However when decoding total movements, the decoding accuracy saturates at 95.7% even with 40 neurons. This phenomenon insinuates that there may be some interrelations between the individual and combined finger movements which confuse the decoder when decoding total movements.
To investigate this problem, we analyzed the main cause of false detections. Table III summarizes the total movement decoding results for each of the individual and combined two-finger movements using 40 neurons. False detections happened mostly in decoding the combined-finger movement tasks of f23, f45, e45 and the individual movement tasks of f5, e2, e5. The detection-rate table shows why the two-finger flexion task of ring and little fingers has the poorest performance. The false detection falls mainly into the little finger flexion, f5, which shows that control independence of finger movements affects brain activity. When the subjects intend to flex only a little finger, the ring finger also flinches irrespective of our intention. Therefore, the flexion of the little finger might have similar neural activity to those of both ring and little fingers. Table III verifies our conjecture because the most false detections for f5 appeared at f45. In the same way, we can explain the poor decoding performance for the cases of the two-finger extension task of ring and little fingers, e45 and of the individual extension task of the little finger, e5. Control dependence also occurs when monkey intends to extend only the index finger corresponding to e2. In that case, the middle finger also flinches irrespective of monkey’s intention. This makes the decoder confused in finding the correct solution between e2 and e23. As a result, the poor decoding accuracy for the total finger movements is likely caused by confusion between some individual movements and combined two-finger movements.
We have presented a neuron selection method by defining the relative importance of each neuron contributing to motor movements. Neuron selection improved the neural decoding performance remarkably due to the use of highly-tuned neurons. Furthermore, neuron selection may decrease the required number of neurons for neural decoding and as a result the number of electrodes needed and the computational power to decode neural activity for achieving dexterous finger actuation in a neuroprosthetic system. By analyzing the improved decoding performance of single- and multi-finger movements, we have taken a step towards the strategy of neural control of dexterous multi-fingered hand.
This research was supported in part by the Korea Research Foundation Grant KRF-2009-013-D00090 (to H-NK) and in part by grants ECCS-0835554 from the National Science Foundation (to NVT), and 1R01EB010100-01 from the National Institutes of Health (to NVT).
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Hyoung-Nam Kim, Department of Electronics Engineering, Pusan National University, Busan 609-735, Korea. Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, MD 21205 USA.
Yong-Hee Kim, Department of Electronics Engineering, Pusan National University, Busan 609-735, Korea.
Hyun-Chool Shin, School of Electronics Engineering, Soongsil University, Seoul 156-734, Korea.
Vikram Aggarwal, Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, MD 21205 USA.
Marc H. Schieber, Department of Neurology, Neurobiology and Anatomy, Brain and Cognitive Sciences and Physical Medicine and Rehabilitation, University of Rochester Medical Center, Rochester, NY 14642 USA.
Nitish V. Thakor, Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, MD 21205 USA.