|Home | About | Journals | Submit | Contact Us | Français|
Previous research has shown that neuronal activity can be used to continuously decode the kinematics of gross movements involving arm and hand trajectory. However, decoding the kinematics of fine motor movements, such as the manipulation of individual fingers, has not been demonstrated. In this study, single unit activities were recorded from task-related neurons in M1 of two trained rhesus monkey as they performed individuated movements of the fingers and wrist. The primates’ hand was placed in a manipulandum, and strain gauges at the tips of each finger were used to track the digit’s position. Both linear and non-linear filters were designed to simultaneously predict kinematics of each digit and the wrist, and their performance compared using mean squared error and correlation coefficients. All models had high decoding accuracy, but the feedforward ANN (R=0.76–0.86, MSE=0.04–0.05) and Kalman filter (R=0.68–0.86, MSE=0.04–0.07) performed better than a simple linear regression filter (0.58–0.81, 0.05–0.07). These results suggest that individual finger and wrist kinematics can be decoded with high accuracy, and be used to control a multi-fingered prosthetic hand in real-time.
Previous work has shown that neuronal ensemble activity from various motor areas can be used to continuously predict the kinematics for gross movements of a single effector, such as during reach [1,2] or control of a computer cursor [3,4]. However, in order to achieve neural control of advanced upper-limb neuroprostheses, there is also a need to develop Brain-Machine Interfaces (BMI) for dexterous movements, such as the manipulation of individual fingers.
We have recently demonstrated neural decoding of discrete flexions and extensions of individual fingers and the wrist [5,6], but for truly dexterous control of a multi-fingered prosthetic hand it will be necessary to continuously decode the kinematics of multiple digits. Single-unit activities were recorded from a population of neurons in the primary motor cortex (M1) hand area of two male rhesus monkeys during individuated flexion and extension movements of each digit and the wrist. Simultaneous kinematics of each digit were obtained through strain gauges mounted on microswitches on the tip of each finger.
Popular methods for decoding activity from neuronal ensembles include population vectors (PVs) , linear filters, and artificial neural networks (ANNs). PVs have proven successful in decoding discrete movements in center-out tasks , and may not be well suited for continuous decoding. Linear regression filters and ANNs have been effective for real-time neural control of a 2D cursor  and prediction of hand trajectory , but both lack a clear probabilistic model and do not incorporate temporal information. More recently, Kalman filters have been used to decode hand position  and trajectory of a computer cursor  using a recursive, probabilistic approach.
In order to determine the best approach to model the data for this particular task, three different algorithms were used to decode the kinematics of each finger and the wrist from the same population of M1 neurons – a linear regression filter, a feedforward ANN, and a Kalman filter. The results of each model were then compared using the mean squared error and Pearson correlation coefficients.
This work demonstrates how neural activity can be used to simultaneously predict the kinematics of multiple end-effectors, and lays the foundation for dexterous manipulation of a multi-fingered hand neuroprosthesis.
Two male rhesus monkeys (monkey C, monkey K) were trained to individually flex or extend each digit and the wrist of the right hand by operating a pistol-grip manipulandum (see Fig. 1, inset). By flexing or extending each digit a few millimeters, the monkey closed microswitches at the tips of each finger as shown in Fig. 1. The position of each finger was obtained from switch-mounted strain gauges on either sides of the fingertip, and a potentiometer transduced wrist flexion and extension.
A row of LEDs above each switch were illuminated instructing the monkey to perform 12 distinct movements. Each instructed movement is abbreviated with the first letter of the movement type (f=flexion, e=extension), and number of the instructed digit (d1=thumb…d5=little finger, d6 or w=wrist; e.g. ‘e4’ indicates extension of ring finger). Fig. 1 shows the average analog traces from strain-gauges during instructed flexion movements for Monkey K. A detailed description of the behavioral task can be found in .
Well-isolated single units with task-related activity were recorded sequentially in the M1 hand area (anterior bank of the central sulcus), contralateral to the trained hand (monkey C, n=49; monkey K, n = 115). Up to 15 trials per movement type were recorded during daily 2-to-3-hour recording sessions. Simultaneously recorded activity of multiple single units was simulated by aligning the activity of each unit at the time of switch closure.
Three different models were used to decode the kinematics of each finger and the wrist from the same population of M1 neurons. All models were trained using the neuronal firing rate over a 100 ms window shifting every 20 ms. Mutually exclusive trials were used for training (100 sets), validation (50 sets), and testing (100 sets).
The first approach used a linear filter to represent each decoded parameter as a weighted sum of the firing rate ,
where yk is the kinematic parameter recorded from the strain gauge for digit k, xi is the firing rate of neuron i, wi is the weight for the neuron, N is the total number of neurons, and bk is a bias term.
Multiple linear filters were used to simultaneously extract the kinematics of each digit and the wrist. The system of equations was set up as follows:
where Y is the matrix of kinematic parameters recorded from the strain gauges, X is the matrix of neuronal firing rates, and W are the weights of the model. The bias term was calculated by appending a row of ones to matrix X. The optimal weights were calculated using the least squares solution,
The second approach was to use a multilayer, feedforward Artificial Neural Network (ANN), which have been widely used in non-linear regression, function approximation, and classification . As before, the relationship between finger position and neural activity can be modeled as,
but where g(x) is a non-linear transformation of the firing rate activity. The ANN was designed with a single hidden layer containing 25 neurons with a tan sigmoidal transfer function. Dimensionality of the input space was reduced by performing Principle Component Analysis (PCA), and retaining those components that cumulatively contributed to >95% of the total variance. The networks were trained offline in MATLAB 7.4 (Mathworks Inc.), with the optimal weights calculated using the scaled conjugate gradient algorithm and early validation stop to prevent overfitting.
As opposed to the feedforward model, a Kalman filter  models the relationship between neural activity and finger and wrist position by using a probabilistic approach that incorporates prior events.
In the Kalman framework described in detail in , the position of each end-effector is modeled as the system state, Y, and the firing rate is modeled as the observation, X.
The Kalman model makes two important assumptions:
At each time step, we first calculate an a priori estimate Y′ using Eq. 5 and calculate its error covariance matrix, P′,
We then calculate the Kalman gain, K, update the estimate with an a posteriori estimate using new measurement data and calculate the posterior error covariance matrix, P,
The Kalman gain produces a state estimate that minimizes the mean squared error .
The sample decoding results in Fig. 3 from monkey K, demonstrate that the predicted output (red) was highly correlated with the actual hand kinematics (blue) using all three decoding models. Each plot shows the simultaneous reconstructed kinematics for all five digits and the wrist during instructed flexion movements.
Two different performance metrics were used to evaluate the decoding accuracy: mean squared error (MSE) and Pearson correlation coefficients (R). Table I and Table II summarize the decoding results for monkey C (n = 49) and monkey K (n = 115) respectively. Fig. 4 compares the MSE and correlation coefficients for all three decoding approaches, and across each end effector. Fig. 4A and 4B show the results for monkey C, while Fig. 4D and 4E show the results for monkey K.
As can be seen, the feedforward ANN appears to have the best performance for both monkey C (R=0.76; MSE=0.05) and monkey K (R=0.86; MSE=0.04). Furthermore, for both monkeys the poorest performance appears with decoding of the wrist kinematics (two-way ANOVA, p<0.05). This is consistent across all three decoding models (monk C: linear, R=0.25; ANN, R=0.54, Kalman, R=0.38 – monk K: linear, R=0.66; ANN, R=0.77, Kalman, R=0.72) which suggest that this movement parameter is especially difficult to decode given the available neuron population.
Fig. 4C (monkey C) and Fig. 4F (monkey K) show the average correlation coefficient (R) as a function of randomly selected subpopulations of neurons for the linear regression (blue), ANN (red), and Kalman (green) filters. The results were averaged across five random subsets of a given number of neurons. Both the feedforward ANN and Kalman filter perform statistically significantly better than the linear regression filter (two-way ANOVA, p<0.05).
This work demonstrates that it is indeed possible to decipher the neural coding of individual finger and wrist kinematics, which paves the way for dexterous manipulation of a multi-fingered hand prosthetic hand. The results do indicate, however, a negative bias towards decoding of wrist movements for both monkeys. This is likely due to the fact that the recording location of electrodes may have been biased to the lateral M1 hand area, and thus did not include as many neurons coding for wrist movements . Furthermore, mechanical stops on the manipulandum may have prevented complete flexion and extension of the wrist (note saturation of wrist measurement during ‘fw’ in Fig. 2).
Furthermore, it appears that for this particular task a nonlinear decoding filter may be appropriate. Although the feedforward ANN and Kalman filter show comparable correlation coefficients, the ANN has smaller MSE values-particularly for fewer neurons (data not shown). Given the dexterity of the motor movements, and the fact that we are tracking multiple end-effectors simultaneously, it is not surprising that a nonlinear filter is better able to decode finger and wrist kinematics.
It is important to note, however, that even a simpler linear regression algorithm still provides respectable decoding accuracy (see Fig. 4C,F), and thus would be an acceptable low-cost alternative for embedding in the hardware controller of a prosthetic arm.
Future experiments aim to extend the techniques developed in this paper, in order to decode entire kinematics of the hand and arm as subjects perform less constrained movements. By demonstrating the decoding of individual finger primitives, these findings can be extended to more complex dexterous tasks involving multiple fingers and different grasp conformations of the hand.
This work was sponsored in part by the Johns Hopkins University Applied Physics L under the DARPA Revolutionizing Prosthetics program, contract N66001-06-C-8005, by the National Science and Engineering Research Council of Canada (NSERC), and by NIH R01/R37 NS27686.
Vikram Aggarwal, Department of Biomedical Engineering at The Johns Hopkins University, Baltimore, MD.
Francesco Tenore, Johns Hopkins University Applied Physics Laboratory (APL), Laurel, MD.
Soumyadipta Acharya, Department of Biomedical Engineering at The Johns Hopkins University, Baltimore, MD.
Marc H. Schieber, Department of Neurology, Neurobiology, and Anatomy, at the University of Rochester Medical Center, Rochester, NY.
Nitish V. Thakor, Department of Biomedical Engineering at The Johns Hopkins University, Baltimore, MD.