This study demonstrates that ECoG activity can support two-dimensional cursor control. In contrast to a recent study using EEG [2
], two-dimensional control was achieved here using different locations on the same hemisphere. This may prove beneficial for BCI applications for patients with unilateral hemiparesis. After brief one-dimensional training similar to what we and our collaborators have previously reported [18
], all five subjects achieved this two-dimensional control within minutes. The level of control over the two movement signals and the speed of the movement reported here (see ) were comparable to those that have previously been achieved after extended training using EEG in humans or in highly controlled experiments using intracortical microelectrodes in non-human primates (see Tables and in [2
]). The time course of control acquisition is much faster than the weeks or months reported in a recent EEG study [2
], and may be comparable to the rapid control acquisition suggested by anecdotal evidence for intracortical studies in monkeys. In summary, by showing that ECoG can support rapid acquisition of robust two-dimensional control without penetrating the cortex, this paper further demonstrates that ECoG is an excellent signal recording modality for BCI applications that may combine high performance with technical and clinical practicality.
While all subjects studied in this paper successfully achieved two-dimensional control, the present experimental approach has three important limitations. The first limitation is that, at present, the only subjects available for these ECoG studies are patients with epilepsy who are temporarily implanted with electrode grids prior to surgery. The second limitation relates to the current difficulty of identifying appropriate ECoG signal features. The third limitation stems from the arbitrary nature of the motor/imagery tasks used for BCI control. These three limitations are described in more detail below.
Patient volunteers who are transiently implanted with subdural electrodes for clinical evaluation offer a rare opportunity to conduct scientific studies. However, this opportunity is constrained by the clinical needs of the patients undergoing treatment. These constraints include variation in patients’ cognitive status, restricted time for experimentation, and, for our research purpose, typically suboptimal cortical coverage by the electrode array. The cognitive status of the patients is often somewhat impaired due to the medical condition that results in epilepsy [40
], seizures during the course of their ECoG monitoring, and concurrent administration of narcotic medication to control their pain. In addition, the subjects’ willingness to participate is often influenced by the highly variable nature of the clinical course of diagnosis and treatment, and consequently may wax and wane unpredictably. The time available for experimentation is curtailed by the limited duration of electrode placement: two of the five subjects (B, D) had their electrodes implanted for a total of 7 days each. Post-operative recovery (2-3 days) and other factors reduced the time available for experimentation to about 3-4 days. In addition, clinical testing (e.g., electrical cortical mapping) or seizures (and subsequent post-ictal periods) often further limit available time. Because the study protocols are approved for only relatively short experimental periods per day, there were limited opportunities for experimentation in these patients. Patients A, C and E had a prolonged monitoring course (five, two, and two weeks, respectively) due to clinical requirements, which made additional testing possible. Variable cognitive status and reduced time were the most limiting factors in conducting our multi-step experimental protocol (i.e., signal identification, sequential one-dimensional training, concurrent two-dimensional control). In addition to the patient and time limitations, configuration and placement of the grids are optimized for clinical and not for BCI purposes. The configuration (i.e., 1-cm inter-electrode distance) of the clinical grids is almost certainly significantly coarser than that previous studies have suggested to be optimal (i.e., the optimum spatial sampling resolution is probably 1.25 mm [41
]). In fact, we often observed task-related correlations limited to only one or a few recording sites. The placement of the electrodes is dictated by clinical requirements and thus, rarely covers all relevant motor areas and is highly variable between subject. These patient-related issues greatly increase inter- and intra-subject experimental variability and significantly impair the use of a consistent and systematic experimental procedure, and will thus ultimately limit the amount of information that can be extracted using this subject population. The experience with these issues in the present study and in related studies suggests several important areas for future investigation. These include controlled animal studies that determine the long-term effects of subdural/epidural implants, that assess the difference between subdural and epidural recordings with regard to signal-to-noise ratio, and that define the optimum spacing and placement of the electrodes. These efforts should be accompanied by the development and testing of wireless telemetry systems. We expect that appropriate implementation and integration of this optimized ECoG recording platform will result in a small and potentially epidural implant with wireless transmission, which will substantially reduce the clinical risks currently associated with a large and transcutaneous implant. We anticipate that this and future studies will provide ample evidence of the utility of the ECoG platform and will thus support FDA approval of long-term human ECoG-based BCI use for the purpose of communication and restoration of mobility and functional interactions.
The second major limitation is the identification of optimal ECoG signal features. As described in Section 2.3, our first step in utilizing ECoG for real-time BCI control was the identification of the features (i.e., signal amplitudes at particular locations and frequencies) that would be most effectively modulated by the subject using a particular task (i.e., a signal identification procedure). Because there is no strong a-priori basis for making this selection, and because ECoG contains more signal features that are responsive to more tasks than does EEG, this choice is also more difficult. Moreover, ancillary analyses supported the notion reported in [20
] that signal features typically change between the signal identification procedure and the real-time experiment. Traditional signal translation schemes (such as classification or regression) assume that the signal properties are the same during initial signal identification and during BCI feedback. Since this assumption is often not met, it tends to reduce performance. A good example is the generally large difference in performance between the optimized features determined offline and the set of features used online (). Full exploitation of the promise of ECoG signals will require better ways to initially identify good signal features, and subsequently track them as they evolve. One such possibility is to simply detect changes from a signal baseline (e.g., at one location per control dimension), rather than trying to identify and use specific features or sets of features. In this approach, a set of features recorded during rest is modeled, and deviations from this model (such as those associated with production of a task) are measured. These measurements can be used for device control. Initial testing [42
] demonstrates that this methodology can be used effectively in situations in which there is little a priori
knowledge about signal features (such as in the experiments discussed here).
The third limitation is related to the non-intuitive and arbitrary tasks utilized for BCI control. With an ideal brain-computer interface, a person with a motor disability would simply intend a particular movement. This intended action would be detected by the BCI system and translated into appropriate device action. In contrast, the subjects in typical human BCI experiments (such as those described in this paper) use arbitrary and non-intuitive imagery (such as imagined tongue or hand movements) to drive a computer cursor. While a recent EEG-based study [2
] and this paper demonstrate that subjects can make good use of such non-intuitive imagery‡
, it appears likely that multi-dimensional control could be more e ciently achieved and potentially further improved by using more intuitive tasks such as directed actual or imagined hand movements. It has been widely assumed that only microelectrodes implanted within cortex can provide the signal fidelity necessary for effective realization of this approach. However, evidence in recent and ongoing studies [45
] strongly suggests that this is not the case; that, in fact, ECoG supports decoding of multidimensional movement parameters with a fidelity that is comparable to that achieved previously only with microelectrodes implanted within cortex.
In summary, in the present paper, we describe two-dimensional BCI control using ECoG in humans after minimal training, and we outline several promising directions for further research. These results should help to move BCI research closer to practical realization of powerful and reliable BCI systems for long-term use by people with severe neuromuscular disorders.