The introduction of a third control dimension allowed for successful navigation of three-dimensional space that was fast, accurate, and continuous. Two of the experimental subjects (Subject 1 and 2) presented in this work had trained previously with the virtual helicopter using a modified reductionist control strategy that allowed for three-dimensional control by using only two control signals 
. It is important to note that the differences in subject training prior to participation in this study preclude an in depth examination of the absolute difficulty subjects faced when learning the task. However, observations of subject experience are qualitatively informative. The transition to three-dimensional control appears to have been straightforward for Subject 1, as evidenced by a consistently high level of performance over the course of the five consecutive experimental sessions. Subjects 2 and 3 adjusted to the new control paradigm more gradually; both gained proficiency over the course of the experimental sessions. With the presented work proving the efficacy of the proposed system in the 3D control task, a more rigorous treatment of learning in multidimensional virtual helicopter control may be an informative direction for future efforts.
Unlike Subject 1 and 2, Subject 3 had no previous experience using the modified reductionist control strategy to control the virtual helicopter. Rather, this subject completed only 1- and 2-D cursor task training prior to her involvement with 3D virtual helicopter task. Despite her relative inexperience, this subject demonstrated higher PVC/PTC and ARR scores than Subject 2, a more extensively trained subject. This lends credence to a less extensive training regimen for future subjects prior to introduction to the 3D virtual helicopter task. Moreover, Subject 3 also exhibited improved performance over the course of the 5 sessions, leading us to believe that further improvement in performance could be achieved with additional experience in the helicopter task.
Subjects were trained to accurately fly the helicopter through three-dimensional space. An important part of this training was the requirement that they pass through the ring without hitting its edges. The PVC and PTC accuracy assessments presented in take this additional requirement into account and are stricter measures of control accuracy than those reported in conventional BCI cursor tasks. By imposing these conditions, subjects must not only reach the target space, but also plan and execute an appropriate flight path that avoids the ring's edge. This action often necessitated the simultaneous orchestration of multiple control states, continuous adaptation to system feedback and modulation of the strength of imaginations. Subject 1 in particular described how, over the course of training, this process transitioned from the use of definitive imaginative tasks to the ability to shift awareness to the arms, legs, or tongue. Thus, a subject's motor imagery abilities may evolve from representational imaginations to more abstract and intuitive control over the course of training.
Subjects used the FB control signal to rapidly fly through rings after properly aligning the helicopter. This is seen in several of the supplementary videos and reflects the general strategy for ring acquisition. Between individual trials, subjects were able to request adjustment in the relative strength of each of the control components. Recurring subject selections resulted in an optimization of directional velocities, including a general attenuation of the backward control strength. However, the backward control remained a viable option for breaking or backing away to avoid obstacles. This is probably because the third person view presented to the subject was linked to the helicopter's motion and was oriented forward. Therefore, subjects could not see obstacles or rings that were behind them when moving backward. This strategy is not uncommon in the real world. Real helicopters and cars use slow backward motion for adjustment or obstacle avoidance even though more rapid backward motion is possible. When subjects wanted to go quickly in the opposite direction, they preferred to rotate the helicopter 180 degrees and then use the forward control. Yet, when asked to do so, all subjects were capable of flying through rings in reverse. By optimizing the control signal, the independent control component weighting, and the strategy employed, subjects were able to pursue rings quickly through 3D space. This speed of control is reflected in the ARAV values reported in .
Continuity of control was an important objective of this study. To be considered continuous, control must allow for the acquisition of greater than one target in an unbroken control path. Continuous control was achieved by presenting subjects with a series of randomly oriented targets throughout a 3D environment. reflects the degree of continuous control achieved by each subject. All subjects averaged more than 1 ring prior to a reset for the 5 consecutive experimental sessions (4.5, 1.5, and 2.0 for Subjects 1, 2, and 3 respectively).
The experimental protocol was designed to reward the development of control that was fast, accurate and continuous. By requiring that subjects fly from one ring to the next, subjects learned to modulate their control before, during and after ring acquisition. Significant time penalties were associated with resetting after collisions with objects. Thus, intentionally colliding with an object to be reset within the target domain when presented with a difficult ring was not an effective strategy. The requirement that the subject needed to pass through the rings without touching them added an additional level of difficulty to the task, and trained subjects to establish and modify the flight path as each situation required. The capacity for adjustment of the control plan during all stages of control is essential for real world applications. In these applications, goals will not be imposed by the system, but by the will of the user. Therefore, it is essential to allow the user to alter the flight path at any time to respond to a change in intent or as a reaction to an unexpected event.
The adapted multidimensional control system presented here is innovative in its incorporation of both force and displacement actuation to achieve fluid movement through the control, 3-dimensional space. The rotation of the helicopter was linearly tied to the control signal generated through learned modulation of ERD and ERS arising from movement of the right and left hands. Tying this control signal to the change in the helicopter's rotational angle at each simulation frame allowed subjects to use the most well established and easily learned control signal, generated from right versus left hand imagination, to rotate the helicopter with high resolution and set a course for the desired target. Displacements in the vertical and horizontal axes were controlled using force actuation. This is an appropriate choice since the summative nature of forces allows the subject to dynamically affect the direction of vertical and horizontal displacement, but once the desired movement is actuated, the forces applied will cause the helicopter to continue to drift in the desired direction. The subject may then focus attention on modulation of the high-resolution rotational control to ensure proper alignment in relation to the ring. This removes the requirement for a subject to be able to perform multiple motor imageries simultaneously, a task often complicated for the inexperienced user. At the same time, the arrangement preserves the potential for complex control through the use of simultaneous motor imaginations by the experienced user. The interplay between high and low-resolution movement, leveraged in the design of this system, is an essential component of the brain's ability to coordinate movement in three-dimensional space. When grasping a target with the hand, cognition is dedicated to small collections of motor units to coordinate the fine movement of the fingers. At the same time, larger groups of motor units are recruited to move the arm in the general desired direction. This intrinsic arrangement of the human nervous system is reflected in the design of this novel BCI.
Three-dimensional control that is fast, accurate and continuous is a prerequisite for many of the useful applications envisioned for BCI. Here we present a novel system that allows users to navigate to a series of randomly positioned targets in 3D space. The system enables them to fly quickly and accurately through a series of rings in an unbroken path, characteristic of continuous control. Furthermore, utilizing BCI2000, a well-established software platform, we were able to successfully expand the limits of motor imagery based BCI into three dimensions. No BCI applications to date have allowed for this continuous, three-dimensional control along an unbroken path to multiple targets through the use of non-invasive EEG. By placing emphasis on the interplay between the methodologies used to train the user and the functionality of this novel system, the possibilities for non-invasive BCIs for potential applications to neuroprosthetics, rehabilitative medicine or other fields will continue to expand.