This study was conducted with five patients who had undergone TMR surgery 11–70 months prior to testing. For comparison, five non-amputee control participants were included in the study. This study was approved by the Northwestern University Institutional Review Board and conducted between January 2007 and January 2008 at the Rehabilitation Institute of Chicago. All participants gave informed consent and provided permissions for publication of videos and photographs for scientific and educational purposes.
Five of the six shoulder-disarticulation or transhumeral amputees who had undergone TMR surgery in collaboration with the Rehabilitation Institute of Chicago agreed to participate in this study. Three of these participants had shoulder-disarticulation amputations. Patient S1 was a 54-year-old man who underwent bilateral shoulder-disarticulation amputations in May 2001 following high-voltage electrical injuries to both arms. During TR surgery, which took place in February 2002, his residual musculocutaneous, median, radial, and ulnar nerves were transferred to the pectoralis major and pectoralis minor muscles ().11, 17, 18
Patient S2 was a 24-year-old woman with a left shoulder-disarticulation (very short residual humerus) amputation resulting from a motor-vehicle accident in May 2005. During TMR surgery, the musculocutaneous, median, radial and ulnar nerves were transferred to portions of the pectoralis major and serratus anterior muscles ().13
Patient S2 began fitting with her TMR prosthesis in February 2007. Patient S3 was a 37-year-old man who underwent right shoulder-disarticulation amputation in February 2005 following severe electrical burns. During TMR surgery, which took place in July 2006, the musculocutaneous, median, radial and ulnar nerves were transferred to the pectoralis major, pectoralis minor, and latissimus muscles (). Two patients with transhumeral amputations also participated in the study. A 50-year-old man with a right transhumeral amputation resulting from a motor-vehicle accident in April 2004 (T4) had the median nerve transferred to the medial biceps and the distal radial nerve transferred to the brachialis muscle during TMR surgery in January 2005.14
A 38-year-old woman with a left transhumeral amputation due to a motor-vehicle accident in April 2006 (T5) had the median nerve transferred to the medial biceps, and the distal radial nerve transferred to the lateral triceps during TMR surgery in October 2006. For comparison, five healthy non-amputees (three males and two females, aged 20 to 45 years) participated in the study. The control participants were chosen to have representation of both genders and an age range similar to the TMR patients.
Schematic of TMR surgery in a participants S1 (a), S2 (b) and S3 (c).
EMG Data Collection
For each TMR patient, 12 self-adhesive bipolar EMG electrodes were placed on the skin over the reinnervated muscles. Four electrodes were placed at sites chosen previously through clinical evaluation to control the amputees' prostheses.10–12
The eight additional sites were determined by an electrode-placement optimization algorithm16
which sought to maximize the classification accuracy for different movements. For control participants, 12 electrodes were used to record EMG signals from physiologically appropriate muscles in the arm and hand. One electrode was placed over the biceps muscle, and a second over the triceps muscle; six electrodes were placed around the proximal forearm; one electrode was placed on the dorsal side of the wrist; and three electrodes were placed on the hand (medial and lateral thenar eminence and hypothenar eminence). The EMG signals were amplified and band-pass filtered from 5–400 Hz. Data were sampled at 1 kHz by an analog-to-digital converter (Measurement Computing, USB-116FS) and processed in real time on a desktop computer using the software platform Matlab (The Mathworks, Natick, MA).
Classifier Training and Testing
The 11 motion classes were elbow flexion, elbow extension, wrist flexion, wrist extension, wrist pronation, wrist supination, hand opening, three types of hand grasps and a no-movement class. TMR patients were allowed to try five different hand-grasp patterns: three-jaw chuck, fine pinch, key grip, power grip and tool grip (). Each patient chose three of these grips based on relative ease and reliability of control. For control participants, the three grasps were three-jaw chuck, fine pinch and tool grip; the three most commonly used grasps chosen by the patients. The participants were instructed to follow a demonstration of each movement displayed in random order on the computer screen () and to perform the movement with a comfortable and consistent level of effort. The prompt was displayed with a countdown during the rest time between trials to give patients time to prepare. EMG data were collected in eight consecutive trials. In each trial, each motion was repeated twice and held for 4 s, producing 8 s of EMG recordings per motion. There was a 3 s time interval between motions in the four even-numbered trials. A variable rest time of 0–3 s was used in the four odd-numbered trials in an attempt to keep the participants engaged and enhance the classifier's robustness. EMG data from the eight trials were split into two groups: the four odd-numbered trials were combined and used to train the classifier; the four even-numbered trials were combined and used to test the classifier. The pattern-recognition algorithm used in this study was implemented as follows: EMG recordings were segmented into a series of 150 ms analysis windows with 50 ms of overlap, resulting in a new classification every 100 ms. Four time-domain features3, 15
were extracted from EMG signals in each analysis window. The combined features from the even-numbered trials were used to train a linear discriminant analysis (LDA) classifier.3, 15, 16, 19
This LDA classifier was then used to classify the combined features from the testing set. The classification accuracy for each movement was the percentage of total analysis windows for that class which were correctly classified. The overall classification accuracy was the average of these values for all (11) movements. The LDA classifier was then used in real-time to classify features extracted from real-time EMG signals, produce a new prediction of the motion class every 100 ms, and control a virtual reality arm or a physical prosthesis, as described below. Computational time for each analysis window was less than 3 ms.
(a) Five hand-grasp patterns used in this study. (b) Screen shot of the prompted movement and responding virtual arm.
Virtual Prosthesis Control
Experiments with a virtual prosthesis were performed immediately after classifier training. Participants were instructed to follow visual prompts for each movement, and a virtual arm that responded to the classifier output was displayed on the screen (). Once the participants correctly selected the desired movement, they were asked to maintain it until the virtual arm completed the movement. The time of movement onset was identified as the time of the last `no movement' classification (). Each of the 10 motions was randomly presented three times in a trial and the trials were repeated six times for a total of 180 movements: 72 hand-grasp motions and 108 elbow and wrist motions. These data were used to evaluate the speed and consistency of control using real-time pattern recognition.
Figure 4 Two performance metrics: motion-selection time (MSt) and motion-completion time (MCt). The target motion classes are shown by green dots and the decisions of the classifier are depicted as blue circles. Each target movement started from a state of rest. (more ...)
The performance metrics used to assess virtual prosthesis control were motion-selection time, motion-completion time, and motion-completion (or `success') rate. The motion-selection time was the time taken to correctly select a target motion and was defined as the time from movement onset to the first correct classification (). This quantity measures how quickly motor commands can be translated into correct motion predictions. The motion-completion time was defined as the time from movement onset to the tenth correct classification (which represented the full range of motion for any movement) (). The fastest possible speed to complete any motion was 1 s, corresponding to 10 consecutive correct classifications with new classifications occurring every 100 ms. If the correct class was not selected within a 5 s time limit, the movement was considered a failure. The motion-completion rate was the percentage of successfully completed motions out of the total attempted motions (72 attempted motions for the hand, and 108 attempted motions for the elbow and wrist) within the time limit. Because the motion-selection and motion-completion data for each participant was highly skewed, the median value for all six arm movements (elbow and wrist) and all four hand movements (hand open and three hand-grasps) were calculated for each participant, and these values were averaged across the five TMR patients and five control participants.
Preliminary research demonstrated that hand-grasp patterns were more difficult to perform than elbow and wrist movements. Therefore, the control scheme for hand grasps was modified. A hand grasp could only be selected when the hand was fully open. Once a grasp was selected, any hand-grasp pattern would close the hand in the initially selected grasp. However, if the initial hand-grasp pattern selected was incorrect, the patient would have to fully open the hand and try again.
Physical Prosthesis Control
Three of the TMR patients were able to test advanced upper arm prosthesis prototypes developed under the Defense Advanced Research Project Agency's Revolutionizing Prosthetics program. Video of this initial testing is presented as supplemental information.
Johns Hopkins University Applied Physics Lab (JHUAPL) and their collaborators developed a seven-degree-of-freedom prosthetic arm that was tested with patient S1 in January 2007. S1 controlled flexion and extension of the motorized shoulder by using residual shoulder motion to operate a mechanical rocker switch. A motorized humeral rotator was controlled with EMG signals from the residual deltoid and latissimus dorsi muscles. Powered elbow flexion/extension, wrist pronation/supination, wrist flexion/extension and a hand that allowed three-jaw chuck and lateral pinch grip were controlled with EMG signals from TMR muscles and the pattern recognition algorithm.
DEKA Integrated Solutions Corporation and collaborators developed a 10 degree-of-freedom prosthetic arm system that was tested with patients S1, S2 and T5 in May, June and July 2007, respectively. A shoulder controller operated with residual shoulder movement allowed shoulder-disarticulation patients to simultaneously operate shoulder flexion/extension and abduction/adduction. Humeral rotation was controlled with EMG signals from the latissimus dorsi and deltoid muscles. The powered elbow, wrist and hand were controlled with pattern recognition of EMG signals recorded over TMR muscles. For patient T5, the humeral rotator was controlled with a switch, while the elbow, wrist, and hand were controlled with pattern recognition of EMG signals recorded over TMR muscles. The DEKA hand had multiple motors and was able to form a variety of hand-grasp patterns including those shown in .
Surface electrodes were either self-adhesive or built into the patients' prosthetic sockets. The arm systems were trained at the beginning of each session with a short pattern-recognition protocol similar to the one described above. Training and testing with the prostheses occurred over a two-week period for each patient. Sessions generally lasted two to three hours with one session in the morning and one in the afternoon.