PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
IEEE Trans Neural Syst Rehabil Eng. Author manuscript; available in PMC 2010 September 20.
Published in final edited form as:
PMCID: PMC2941890
NIHMSID: NIHMS221153

Development of Closed-Loop Neural Interface Technology in a Rat Model: Combining Motor Cortex Operant Conditioning With Visual Cortex Microstimulation

Timothy Charles Marzullo, Member, IEEE, Mark J. Lehmkuhle, Member, IEEE, Gregory J. Gage, Student Member, IEEE, and Daryl R. Kipke, Member, IEEE

Abstract

Closed-loop neural interface technology that combines neural ensemble decoding with simultaneous electrical microstimulation feedback is hypothesized to improve deep brain stimulation techniques, neuromotor prosthetic applications, and epilepsy treatment. Here we describe our iterative results in a rat model of a sensory and motor neurophysiological feedback control system. Three rats were chronically implanted with microelectrode arrays in both the motor and visual cortices. The rats were subsequently trained over a period of weeks to modulate their motor cortex ensemble unit activity upon delivery of intra-cortical microstimulation (ICMS) of the visual cortex in order to receive a food reward. Rats were given continuous feedback via visual cortex ICMS during the response periods that was representative of the motor cortex ensemble dynamics. Analysis revealed that the feedback provided the animals with indicators of the behavioral trials. At the hardware level, this preparation provides a tractable test model for improving the technology of closed-loop neural devices.

Index Terms: Closed-loop control, microstimulation, motor cortex, operant conditioning, rat, visual cortex

I. Introduction

Technological developments over the past decade have contributed to a dramatic growth in the capabilities of brain-controlled applications in rats [2], [3], monkeys [4]–[7], and humans [8], [9]. Such work has focused on exploiting/creating output signals via electrode interfaces to the neocortex of the brain, with either visual or auditory feedback given to the experimental subjects indicating the state of the respective neural ensembles. The scientific community speculates the next technological growth area will be in delivering feedback of neural output states directly back to sensory areas of the central nervous system [10], [11], as doing so could potentially improve neuroprosthetics performance [12]. Such closed-loop feedback systems in implanted neural devices are also speculated to improve deep brain stimulation applications [13] and epilepsy treatment [14].

Though pioneering operant conditioning studies in monkeys revealed subjects could successfully modulate their motor cortex neurons with minimal or no feedback (beyond food reward) [15], complex control systems coupled to the nervous system will require utilizing feedback control. Thus, concurrent with the development of neuroprosthetic output devices, some groups have explored sensory replacement/augmentation and fundamental neural encoding through delivering sensory information to the neocortex in the form of electrical stimulation [16]–[20]. However, the simultaneous use of neural recording/decoding techniques with sensory cortex intracortical microstimulation (ICMS) in real-time remains a new research arena [21], [22].

Our group has previously investigated both ensemble decoding for operant conditioning-based neuroprosthetics [3], [23] and sensory cortex ICMS [24], [25] in separate experiments. Building on our previous work, we combine these two techniques for a closed-loop preparation with the long-term goal of improving the technology of feedback control systems and identifying engineering barriers.

Three rats were trained to modulate their motor cortices in response to ICMS of the visual cortices. The animals were given feedback via visual cortex ICMS during the response periods of the behavioral task that was representative of the dynamics of the real-time motor cortex ensembles. All rats were able to modulate their motor cortex activity in response to visual cortex microstimulation, but subtle features of the feedback were either not detected or not used by the rats, thus limiting performance. We close this report by discussing the limitations of our model, technological issues, and future applications.

II. Methods

A. Electrode Arrays, Surgical Procedure, and Neural Recordings

All animal procedures were approved by the University of Michigan University Committee on the Use and Care of Animals and were in accordance with the National Institutes of Health guidelines. Three male Long-Evans Rats (400–500 g) naïve to the behavioral task were implanted with two 16-channel chronic silicon-substrate microelectrodes [26], each consisting of a single silicon shank with 1250-μm2 iridium electrode sites separated by 100 μm (catalog #3mmChron100–1250, Center for Neural Communication Technology, University of Michigan, Ann Arbor or catalog #6mmChron100–1250, NeuroNexus Technologies, Inc., Ann Arbor, MI). The visual cortex electrodes were “activated” prior to the surgery using cyclic voltammetry through the electrode sites to create iridium oxide layers to both increase charge capacity and reduce the electrode/tissue impedance [27]. We typically observed the 1 KHz impedance dropping from ~ 1 MΩ to ~ 200 kΩ after activation.

Surgery was performed as previously described [23], [26]. The rats were anesthetized with a ketamine/xylazine cocktail, the skull was exposed, and 3-mm-diameter craniotomies were made over the target cortical areas using hand drills. The silicon electrodes were inserted in the head/neck area of motor cortex (target 2.5–3.0 mm anterior to bregma, 2.5 mm lateral from bregma) [28] and visual cortex (0 mm posterior to lambda, 3–4 mm lateral from bregma) [29]. In all rats, the motor cortex electrode array was inserted in the left hemisphere and the visual cortex electrode array in the right hemisphere. The electrodes were hand-inserted using #5 Teflon coated forceps, and the electrode ribbon cables were wrapped with gelfoam before enclosing the whole electrode assembly in acrylic. Rats were given a week to recover before being trained on the behavioral task.

B. Visual Cortex Characterization

Immediately after surgery, while the animal was still anesthetized, visual and motor cortex neural responses were recorded as the house lights flashed on and off in a dark behavorial box every 20 s. The spiking responses triggered to light onset and offset were then analyzed with peri-event time histograms (PETHs) to determine the extent of visual evoked responses. The visual cortex channel with the most robust response, as determined by a maximal increase in spiking unit firing rate at light onset, was used as the stimulation channel in subsequent single channel microstimulation behavioral experiments after surgical recovery (see “pulse-based code” below).

C. Electrophysiological Recordings and Training Paradigm

After surgical recovery, rats were maintained at 85% of their free-feeding weight. For each behavioral session, the rats were plugged into unity gain impedance matching headstages and commutator cables and placed into a dark, semi-anechoic behavioral box. Rats were typically trained 4–6 days a week for 200–500 trials per day. During each experimental session, neural electrophysiological data from the electrode channels sampled at 40 kHz were simultaneously amplified and bandpass filtered (450–5000 Hz) on a Multichannel Neuronal Acquisition Processor (MNAP; Plexon Inc., Dallas, TX). Extracellular action potentials (units) were sorted by hand prior to each training session using Plexon Sortclient software (Plexon Inc., Dallas, TX). All spike times from the sorted units were then relayed via TCP/IP to a dual 1.25 GHz Dell Dimension Computer (Dell, Inc., Austin, TX) that analyzed the spike activity using in-house designed software (Mathworks, Inc., Natick, MA) running ActiveX controls that transmitted and received signals from both the behavioral box (Coulborne Instruments, Inc., Allentown, PA) and the stimulation hardware (A-M Systems 2200 Analog Stimulus Isolator, Sequim, WA, for single channel stimulation or Tucker Davis Technologies Multichannel Stimulator RX7 with MS16 Stimulator Base Station, Alachua, FL, for multichannel stimulation).

See Fig. 1 for a graphical description of the task. A rat had to maintain its motor cortex ensemble baseline firing rates, calculated in running 2 min intervals, for 450 ms to begin a trial [epoch 1, Fig. 1(a)]. The visual cortex ICMS “go” cue then occurred [epoch 2, Fig. 1(a)], and the rat had to modulate its motor cortex for 450 ms within the 4 s response period in order to receive the food reward [epoch 3, Fig. 1(a)]. Based on previous work by our group [3], rats were trained using a co-adaptive Kalman filter. During each session, the firing rates of all sorted units (both single units and multiunit clusters) were used as an input to our decoding algorithm. The average number of units, across all rats was 17.6 ± 7.5 units/session (range 8–34 units/session). These firing rate data were dimensionally reduced in real-time by the Kalman filter to a virtual cursor that the animal could thus control by modulating the firing rates of its neural ensemble. The position of this virtual cursor (x) at the time bin k was modeled by the equation

Fig. 1
A: General Behavioral Scheme. The trial began with the first epoch where the rat had to maintain the motor cortex baseline firing rates for 450 ms to proceed. The second epoch consisted of the visual cortex ICMS “go” cue signaling the ...

xk=Axk1+w
(1)

where A relates the prior cursor position to the current (A = 1 in our experiments), and w is a white noise term with a normal distribution

wN(0,W)
(2)

We modeled the firing rates of the units as an observed noisy response to the unobserved cursor position. We defined this relationship as

zk=Hxk+q
(3)

where z is a U × 1 vector of 90 ms spike bins from U units, H is a U × 1 vector that linearly relates the position of the neural cursor to the neural firing, and q is a noise term. Again, we assume that the noise has zero mean and is normally distributed

qN(0,Q)
(4)

To account for drifts in firing rate over the training session, we normalized the firing rates by dividing z with a 2-min moving average of previous bins. Also, as stimulation artifact caused an artificial drop in detected spikes, we disallowed decreases during the response window via the equation

z=max(0,z)
(5)

The firing rates were used to get a bin-by-bin estimate of the cursor position (x̂) using a co-adaptive Kalman filter (which weighted modulating units) as previously described [3].

The estimate of cursor (x̂) was provided as feedback to the subjects via visual cortex ICMS. Fig. 1(d) illustrates the cursor estimate and the feedback scheme. A trial was rewarded when the cursor was held in a bounded “target” range for 450 ms. The cursor was allowed to travel outside the bounds of the baseline and target zones. The size and location of the target range was fixed to ±17% of the range of values (xk) that the cursor could assume in the neural modulation task, and was the same for all subjects. Pilot studies determined that this 450 ms criterion window allowed subjects to acquire the target in ~30% of trials by chance. See Fig. 1(e) for an example of the baseline and target zones, a subset of rasters, and the Kalman filter output during five consecutive trials.

D. Visual Cortex ICMS Feedback

Our stimulation parameters used in the visual cortex ICMS “go” stimulus consisted of cathodic first, 200 μs/phase biphasic pulses, delivered at 150 Hz, for 0.25 s, at 20 μA. These parameters were chosen as such stimulation has been shown to be both perceivable, safe, and exhibits reversible faradaic reactions at the electrode/tissue interface [25], [30]. Periodically, impedances at 1 kHz were taken for the visual cortex electrodes with an Autolab system (Eco Chemie, Utrecht, The Netherlands) to determine if any electrode sites had failed (typically revealed by a 1 kHz impedance > 3 MΩ). If bad sites were found, the channels were dropped from the visual cortex ICMS paradigms during subsequent training. In the data reported here, such a failure only happened once, for R1, between sessions 6 and 7 (Fig. 3).

Fig. 3
Rat R1 learning curve and motor cortex spiking responses. Top: graph shows performance for R1. Chance calculation includes 95% confidence bands. Text at the top of graph indicates the percentage of total trials consisting of catch trials and the type ...

During the response window, the estimate of cursor (x̂) was provided as feedback to the subjects via visual cortex ICMS [Fig. 1(d)]. The stimulation artifact, however, would saturate the recording amplifiers, in effect preventing the recording of motor cortex spikes during the ICMS and reducing the observed firing rates of the ensemble. Amplifer recovery time to stimulus artifact on the recorded spike channels was typically 2–4 ms. To mitigate ICMS artifacts corrupting the motor cortex recordings, we used an interleaved feedback system. All of our experiments used 90 ms clock cycles, whereby the number of spike occurrences since the last query, and all software calls to hardware, were made every 90 ms. Only during the first 10 ms of each 90 ms cycle was ICMS feedback delivered to the visual cortex. This effectively resulted in a drop of ~ 11% in recordable spikes in the motor cortex recordings. As the Kalman filter would interpret a consistent ~11% drop in firing rates across the ensemble during the response period as the rat decreasing firing rates in order to be rewarded, the algorithm was modified to only allow increases in firing rate to be considered actual neural responses to the visual cortex ICMS “go” stimulus. The “spike-drop rate” of ~ 11% was minimal enough that increases in firing rates by the rats would not be overpowered and could still be observed during the response period.

An alternative way to avoid the 11% drop in firing rate affecting the decoding algorithm would be to simply ignore the 10 ms during which ICMS feedback occurred, and only feed the remaining 80 ms of each clock cycle into the Kalman filter. However, we were unable to do this in our preparation, as the acquisition system (Plexon) could only be synchronized with the microstimulation and behavioral box hardware (TDT) in 90 ms clock cycles due to performance limitations of Windows XP and Matlab. Experiments with faster clock cycles (10, 30, 60 ms) proved infeasible for system performance.

Two ICMS feedback codes were employed over the course of our hardware and software development: a pulse-based code and a spatial-based code.

In the pulse-based code, the amplitude of the feedback was invariant (identical to the amplitude of the initial visual cortex ICMS “go” stimulus), but the number of pulses was modulated in a linear transformation of distance between the baseline firing rates and reward threshold firing rates. This was based on previous work by our group whereby the frequency of a sound stimulus increased as the animal approached the threshold required for reward [3], [23]. During the first 10 ms of every 90 ms of the moving criterion window of the response period, 0–10 pulses were delivered with inter-biphasic pulse intervals of 600 μs. The real-time output of the Kalman filter was normalized and rounded to the nearest value of 0–10 to select the appropriate number of pulses for stimulation during ICMS feedback [see Fig. 1(d)]. Thus, if the animal’s firing rates were at baseline, zero pulses were delivered, and if the animal’s response firing rates were in the response band necessary to be rewarded, 10 pulses were delivered. In this stimulation scheme, putatively the volume of activation was kept the same, but the firing rates of the activated neurons in the same volume were modulated by the feedback ICMS [30]–[32].

In the spatial-based code, the top electrode in the electrode array was stimulated as the initial visual cortex ICMS “go” stimulus, initiating a trial. This feedback paradigm consisted of stimulating the electrode sites in a dorsal direction as the animal’s response firing rates approached the threshold for reward. In this spatial-based code, the real-time output of the Kalman filter was similarly normalized and rounded as the pulse-based code, though for 16 discrete values [for each electrode location, see Fig. 1(d)]. If the animal’s response firing rates were at baseline, the bottom-most site was stimulated during the sliding criterion window. If the animal’s response firing rates were in the response band necessary to be rewarded, the top-most site was stimulated. In the spatial feedback scheme, five biphasic pulse pairs were delivered to the appropriate electrode site with an inter-biphasic pulse interval of 600 μs. Only one electrode site was stimulated at any given moment in this paradigm.

During all feedback sessions, 15% of the trials were “catch trials” where either 1) feedback was randomized such that the feedback was no longer correlated to the motor ensemble activity, or 2) feedback was absent. Percentage correct between the “normal trials” and “catch trials” were then used as our metrics in determining 1) whether the rats were using the feedback signal, and 2) what features of the visual cortex ICMS feedback were salient sources of information.

E. Data Analysis

Peri-event time histograms (PETHs) of spikes about the visual cortex ICMS “go” cue were examined to see whether the spiking activity of the neurons increased in response to the stimulus. To determine if a recorded unit showed an excitatory response, 95% confidence intervals were calculated for the PETHs post hoc using the NeuroExplorer software package (Neuro-Explorer, Littleton, MA). The computations for these intervals assumed spike trains are the result of independent Poisson-point processes as described in the literature [33]. A chi-square test (with a Yates correction for continuity due to the low numbers of catch trials per training session) was used to determine whether the “catch trials” percentage correct and the “normal trials” percentage correct were significantly different from each other [34] during a given training session.

In all training paradigms, Monte Carlo simulations, in which the trial “go” cue times were shuffled while keeping the spike trains intact, were done offline to test if the animals were modulating their motor cortex ensemble above chance in response to the visual cortex ICMS “go” cue, using the identical Kalman filter as in the online version. For each session, we ran 200 simulations of shuffled “go” cue times and used the mean and standard deviation of the percentage correct in the simulations as our metric of chance performance [3].

F. Histology

Upon completion of training, the top, middle, and bottom site positions of the multielectrode arrays in the neocortex were microlesioned by passing 35 μA of constant dc current for two seconds while the animal was anesthetized [35]. Three hours after the microlesioning procedure, the rats were euthanized by intraperitoneal overdose of sodium pentobarbital. Following transcardial perfusion with 4% paraformaldehyde, the brain was removed, sectioned into coronal 100 μm slices, and stained with conventional cresyl violet Nissl stains. The sections were then analyzed using a Leica MZFLIII light microscope (Leica Microsystems, Inc., Germany) to determine probe placement. Upon extraction of the skull, we were able to keep the probes intact in situ, and thus we were able to postmortem measure implantation angles with respect to the skull in order to reconstruct the recording spans shown in Fig. 2.

Fig. 2
Histology and evoked responses to light flash for the three rats. Responses in both motor cortex and visual cortex were recorded simultaneously immediately postsurgery. Grey line above each evoked response shows the time the light was presented to the ...

III. Results

A. Visual-Evoked Responses

Fig. 2 shows the histology for both the motor and visual cortex implantation sites for the three rats used in this study, as well as the evoked responses to light flash in both the visual and motor cortices in the same rats. All rats had discernible units immediately after surgery in both the motor and visual corticies; the evoked spiking responses to light flash in the visual cortex was much greater than the motor cortex evoked responses. Spiking unit responses in visual cortex were typically very rapid with a 3–5 fold increase in firing rate with light onset and offset. Motor cortex responses were much more subtle; either there was no change in activity, a “phase-reset” of the 1–2 Hz ketamine oscillations [36], or a slight increase or decrease in firing rate (see Fig. 2 for representative evoked responses). Since the neocortex contains sparse long connections between distant brain areas both among and between hemispheres, it is not surprising that the motor cortex also responded to light flash; nonetheless, the visual cortex had a more robust response to light flash with shorter synaptic delays.

B. Visual Cortex ICMS Feedback With Motor Cortex Ensemble Operant Conditioning

Figs. 35 depict the behavioral learning curves and spiking responses for each of the three rats tested sequentially. Sessions where behavioral performance for the normal trials was significantly different from the catch trials using a chi-square test with a Yates correction (p < 0.05) [34], are indicated by asterisks. PETHs show spiking responses to the visual cortex ICMS “go” cue from two selected neurons of the ensemble used in the respective sessions. Results from these three animals are presented in sequence below.

Fig. 5
Rat R3 learning curve and motor cortex spiking responses. The format is identical to Fig. 3. In sessions 12–15, no visual cortex ICMS “go” cue was delivered. As such, time = 0 on the session 15 PETHs indicates the beginning of ...

Rat R1 (Fig. 3) was initially trained on the pulse-based feedback (sessions 1–6) and then on the spatial-based feedback (sessions 7–13). For a week prior to session one, the rat was run with developmental versions of the Kalman filter algorithm [37]. Once formal training began, the normal trials and the catch trials began to be consistently different in performance rates during sessions 3–8. In these sessions, the catch trials had no visual cortex ICMS feedback during the response window. R1 performed significantly better (p < 0.05) on the normal trials (with ICMS feedback) than the catch trials (no ICMS feedback), indicating that the feedback was providing a source of information to the rat (see Fig. 3, session 6 neural response).

We then sought to characterize the salience of the feedback by using randomized feedback in the catch trials (sessions 9–12); feedback was delivered via visual cortex ICMS during the response window, but the parameters of the feedback were no longer related to the motor cortex ensemble firing rates. The visual cortex ICMS “go” cue remained the same. In these sessions with randomized feedback catch trials, there was no significant difference in performance between the normal trials and the catch trials, even though the animal could still perform the task above chance (see evoked responses in Fig. 3, session 12, in which the neural responses in the normal trials and catch trials are essentially identical). These results argue that the rat was using the feedback only as a measure of whether a trial was still occurring; the rat was not attending to or was unable to discern changes in the spatial location of the stimulation of the feedback signal during the response window. To test this further, on the last session of rat R1 (Fig. 3, session 13), 50% of the trials had ICMS feedback, and 50% of the trials had no ICMS feedback. Though the rat performed above chance in both the feedback and non-feedback trials, the animal performed significantly better during feedback trials (88% correct) than the non-feedback trials (64% correct), and the neural responses during the feedback trials was greater than the non-feedback trials (see Fig. 3, session 13 neural responses).

Rat R2 (Fig. 4) was initially trained on the pulse-based feedback (sessions 1–7) before being trained on the spatial-based feedback (sessions 8–21). Unexpectedly, during sessions 9–17, in which the catch trials contained no visual cortex ICMS feedback, this animal performed better on the catch trials than the normal trials (see Fig. 4, session 15. Notice the neural responses on the catch trials are greater than the normal trials). We hypothesized that the rat was only attending to the initial visual cortex ICMS “go” stimulus, and the feedback effectively made the task harder because of the ~11% reduction in detectable spikes during the response window. We tested this hypothesis by training the animal for one session where the feedback was removed, and only the initial visual cortex ICMS “go” cue was delivered to the animal. The rat was then immediately able to perform the task above chance (see Fig. 4, session 18), with robust neural responses.

Fig. 4
Rat r2 learning curve and motor cortex spiking responses. The format is identical to Fig. 3. In sessions 19–21, no visual cortex ICMS “go” cue was delivered. As such, time = 0 on the session 21 PETHs indicates the beginning of ...

Following this observation, we trained the animal for three additional sessions in which the initial visual cortex ICMS “go” cue was removed, feedback during the response window was reintroduced, and the catch trials consisted of randomized feedback. In this manipulation, the rat was able to perform the task successfully above chance, but with no significant difference in performance between the randomized feedback catch trials and the normal feedback trials. The behavioral performance similarity between the normal and catch trials suggests the rat was not using, or could not discern, the information in the subtle features of the spatial-based feedback; the rat was using the feedback only as an indicator of an ongoing trial (see Fig. 4, session 21), similar to rat R1.

Rat R3 (Fig. 5) was trained in a similar progression as rat R2 (though only with spatial based feedback), and the results and interpretations were similar to rat R2; like the previous rats, R3 was able to use feedback only as an indicator of an ongoing trial.

In addition, we analyzed the reaction times from the beginning of a trial to the reward between the normal trials and catch trials. Across all sessions, there was no detectable difference in reaction times between normal and catch trials across the three rats (R1: 2.5 ± 1.1 s normal trials versus 2.4 ± 1.0 s catch trials; R2: 2.4 ± 1.2 s normal trials versus 2.3 ± 1.1 s catch trials; R3: 2.3 ± 1.1 s normal trials versus 2.2 ± 1.1 s catch trials) (note: due to the five-point smoothing (450 ms) of the Kalman filter output, the behavioral reaction time from “go” to “reward” is slightly longer than the neural modulation response time seen in the PETH’s in Figs. 35). Within specific sessions, significant differences in reaction time (students t-test) were sporadic. These data, combined with the performance rates between the catch trials and the normal trials, and the chance calculations with the Monte Carlo simulations, indicate that the feedback during the response window did not appear to provide additional information on the state of the neural ensemble that was used by the rats. The rats appeared to use the feedback only marginally as an indication of an ongoing trial.

IV. Discussion

As the technology of neuroprosthetics moves forward, the use of neocortex ICMS as a feedback signal for control of a neuromotor device is speculated to increase [10], [21]. Here, we presented initial feasibility results combining recording from motor cortex with electrical stimulation of sensory cortex in a behaving rat model of a neural control system. Previous work has demonstrated the use of simultaneous recording and stimulation techniques for neuroprosthetics, though mostly for functional electrical stimulation (FES) applications whereby neural activity from either the dorsal roots of the spinal cord, peripheral nerves, or motor cortex were used as a signal to stimulate limb musculature [38]–[42]. One recent exception is a study that combined stimulation and recording in the motor cortex to investigate plasticity of muscle representation [43], and another recent experiment has shown a monkey successfully performing a cortical control task in response to a somatosensory cortex microstimulation “go” cue [21].

In our experiments combining visual cortex microstimulation with motor cortex operant conditioning, the use of the feedback in rats R1–R3 varied across the subjects. R1 appeared to be using the feedback, as it did not perform above chance during the no-feedback catch trials from sessions 3–8 (Fig. 3). However, in sessions in which the catch trials consisted of randomized feedback, R1 achieved similar performance in the catch trials as in the normal trials. Thus, R1 was only using the feedback as an indicator of whether the trial was still occurring. Conversely, R2 and R3 performed better on the trials where feedback was not delivered (though the visual cortex ICMS “go” cue was still delivered). The visual cortex ICMS feedback, even in the interleaved stimulation paradigm, would cause a drop in the recording of spikes by our acquisition hardware by ~11%. Since decreasing motor cortex firing rates were not considered a behavioral response by the algorithms precisely for this reason, the task was more difficult with the feedback, as the animals had to increase their firing rates to a greater degree in order to be rewarded. When the ICMS feedback was removed, rats R2 and R3 were immediately able to do the task above chance (Fig. 4, session 18, and Fig. 5, session 11). Conversely, when the visual cortex ICMS “go” cue was removed, leaving the interleaved feedback as the only indication of behavioral trials for the animals, R2 and R3 were able to do the task above chance (R2: sessions 19–21, R3: sessions 12–15). The performance, however, in the randomized feedback catch trials and the normal trials was similar, suggesting that these animals were, like R1, only associating the ICMS with an ongoing trial. These observations posit the case that, for rats R2 and R3, the rats were mainly attending to the initial visual cortex ICMS “go” cue and the interleaved visual cortex ICMS feedback during the response window was distracting and limited performance, though ultimately the rats could use the interleaved ICMS feedback as a behavioral cue.

Given that the rats were trained on a one-state task, it is possible that the rats could learn to simply hold still (to maintain average firing rates) for 450 ms to begin a trial, then make a specific evoked movement, and this would drive reward. Thus, the rats would be only learning the contingency between motor behavior and reward, and the ICMS would not provide any information. Observations, however, in the experiments reported here and in pilot experiments as well [37], revealed that though the rats would indeed hold still to begin a trial, they would wait until an ICMS “go” cue occurred before making a stereotyped movement to drive reward. Indeed, in trials in which the ICMS “go” was removed and only ICMS feedback during the response window was delivered, (session 19–21, R2; sessions 12–15, R3), we observed that the rats would not start making the stereotyped movements until the ICMS feedback began, again demonstrating that rats were perceiving the ICMS as indicative of an ongoing trial. Although manipulations with the feedback catch trials indicated that the rats were not attending to and/or not detecting the subtle features (rate or electrode location) of the visual cortex ICMS feedback, more sophisticated behavioral tasks with multiple states (target locations) could more explicitly determine optimum feedback strategies for a closed-loop system.

Ideally, delivering ICMS without causing artifact in the motor cortex recordings would ensure a greater transmission of the ensemble dynamics and improved performance on the task. A challenge encountered in this study was the fact that the brain does not encode sensory features by firing action potentials for 10 ms out of every 90 ms, as was the protocol with our interleaved feedback scheme. The only way to avoid this is to find a way to excite brain tissue at fine temporal resolution without the stimulation artifact interfering with the signals on the recording electrodes. This is a common problem in neurophysiology, as neural tissue simultaneously requires high currents to be activated but itself only generates small currents that require sensitive amplifiers to detect. One solution may be via hardware development, such as faster switching amplifiers and improved artifact cancellation [22]. Indeed, if the artifact does not saturate the amplifier, the artifact has very recently shown to be removable via real time template subtraction [44].

An ideal stimulating electrode would be small enough to only stimulate small populations, but the smaller the electrode, the higher the impedance, requiring stimulation voltages that exceed the water window for the electrode and/or safe levels of charge density. A mitigating solution to this problem may be through improved electrode technology, such as reducing the impedance of electrodes via bioactive polymers conjugated to the electrode site [45]–[47]. Yet, a promising technique would be to avoid ICMS altogether and excite brain tissue with optical methods [48], [49]; this technique currently requires genetic modification of neurons and powerful bulky laser sources, but technical improvements in this field are happening remarkably fast [50].

Future work combining the novel electrode technology and the improved data acquisition hardware/software described above will no doubt improve the efficacy of using sensory cortex ICMS for feedback in closed-loop neural control systems. Such feedback may then improve performance and usability of complex neuromotor prosthetics.

Acknowledgments

The authors would like to thank H. Parikh and L. Salas for valuable assistance with electronic hardware troubleshooting, and to E. Kim for preliminary hardware fabrication and preliminary histology. Some of the silicon substrate electrodes used in these experiments were purchased from NeuroNexus Technologies, Inc., of which DRK (ownership) and TCM (consulting) have financial interest.

The work of T. C. Marzullo was supported by the National Aeronautics and Space Administration Graduate Student Research Program (NASA GSRP). The work of M. J. Lehmkuhle was supported by the National Institutes of Health post-doctoral fellowship program (NIH NRSA DC7797). The work of G. J. Gage was supported by the NIH R21 under Grant 1HD049842-02 “Cortical Control Using Multiple Signal Modalities.” The work of D. R. Kipke was supported by the NIH P41 under Grant EB002030 “Center for Neural Communications Technology.” Pilot results from these experiments were previously published in the 3rd IEEE EMBS International Neural Engineering Conference Proceedings, Kona, Hawaii.

Biographies

An external file that holds a picture, illustration, etc.
Object name is nihms221153b1.gif

Timothy Charles Marzullo (S’06–M’08) received the B.S. degree in biochemistry from the University of Texas at Austin, in 2001, studying cortical neuro-physiology, and the Ph.D. degree in neuroscience under D. R. Kipke from the University of Michigan, in 2008.

He then worked for a year with NeuroNexus Technologies as a Research Engineer, developing next-generation neural interfaces. He is currently a Kauffman Postdoctoral Entrepreneurial Fellow, bootstrapping “Backyard Brains,” a startup devoted to producing low-cost neurophysiology equipment for high school students and early undergraduates.

An external file that holds a picture, illustration, etc.
Object name is nihms221153b2.gif

Mark J. Lehmkuhle (M’06) received the B.S. degree in biomedical engineering from Case Western Reserve University, Cleveland, OH, in 1999, and the Ph.D. degree in bioengineering under R. A. Normann from the University of Utah, Salt Lake City, in 2004.

Following a post-doc with D. R. Kipke at the University of Michigan, he is currently a research fellow in the laboratory of F. E. Dudekin the Physiology Department at the University of Utah. His research interests include deep brain stimulation for Parkinson’s disease, obesity, and epilepsy, cortical electrical stimulation, and models of epilepsy. He is also working for Epitel, designing wireless EEG recording devices for epilepsy research.

An external file that holds a picture, illustration, etc.
Object name is nihms221153b3.gif

Gregory J. Gage (S’06) received the undergraduate degree in computer and electrical engineering from Michigan State University, in 1994. He is completing the Ph.D. degree in May 2010 in biomedical engineering under D. Kipke and J. Berke on the role of the striatum in motor learning and decision making at the University of Michigan, Ann Arbor.

He previously worked in industry for AT&T and NCR developing advanced cash registers. He is also a co-founder of “Backyard Brains.”

An external file that holds a picture, illustration, etc.
Object name is nihms221153b4.gif

Daryl R. Kipke (S’87–M’90) received the B.Sc. degree in engineering science, the M.Sc. degree in bioengineering, the M.S.E. degree in electrical engineering, and the Ph.D. degree in bioengineering from the University of Michigan, Ann Arbor, in 1985, 1986, 1988, and 1991, respectively.

From 1991 to 1992, he was a Research Associate with the Department of Bioengineering and the Institute for Sensory Research at Syracuse University, Syracuse, NY. In 1992, he joined the Bioengineering Faculty at Arizona State University, Tempe. In 2001, he joined the Faculty of the College of Engineering, University of Michigan, where he is currently a Professor in the Departments of Biomedical Engineering and Electrical Engineering and Computer Science, and directs the Neural Engineering Laboratory. His primary research interests include the areas of BioMEMS for neural implants, neuroprostheses, systems neuroscience, functional neurosurgery, and neurotechnologies. He teaches in the areas of biomedical instrumentation and neural engineering. He has co-founded two University spin-outs: Neural Intervention Technologies and NeuroNexus Technologies.

Contributor Information

Timothy Charles Marzullo, Neuroscience Program, University of Michigan in Ann Arbor, MI 48109 USA. He is now with Backyard Brains, Ann Arbor, MI 48105 USA.

Mark J. Lehmkuhle, Biomedical Engineering Department, University of Michigan, Ann Arbor, MI 48109 USA.

Gregory J. Gage, Biomedical Engineering Department, University of Michigan, Ann Arbor, MI 48109 USA.

Daryl R. Kipke, Biomedical Engineering Department, University of Michigan, Ann Arbor, MI 48109 USA.

References

1. Paxinos G, Watson C. The Rat Brain in Stereotaxic Coordinates. San Diego, CA: Academic; 1998.
2. Chapin JK, Moxon KA, Markowitz RS, Nicolelis MA. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat Neurosci. 1999 Jul;2(7):664–670. [PubMed]
3. Gage GJ, Ludwig KA, Otto KJ, Ionides EL, Kipke DR. Naive coadaptive cortical control. J Neural Eng. 2005 Jun;2(2):52–63. [PubMed]
4. Carmena JM, Lebedev MA, Crist RE, O’Doherty JE, Santucci DM, Dimitrov D, Patil PG, Henriquez CS, Nicolelis MA. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 2003 Nov;1(2):E42. [PMC free article] [PubMed]
5. Taylor DM, Tillery SI, Schwartz AB. Direct cortical control of 3D neuroprosthetic devices. Science. 2002 Jun 7;296(5574):1829–1832. [PubMed]
6. Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for self-feeding. Nature. 2008 Jun 19;453(7198):1098–1101. [PubMed]
7. Santhanam G, Ryu SI, Yu BM, Afshar A, Shenoy KV. A high-performance brain-computer interface. Nature. 2006 Jul 13;442(7099):195–198. [PubMed]
8. Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, Donoghue JP. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006 Jul 13;442(7099):164–171. [PubMed]
9. Kennedy PR, Bakay RA, Moore MM, Adams K, Gold-waithe J. Direct control of a computer from the human central nervous system. IEEE Trans Rehabil Eng. 2000 Jun;8(2):198–202. [PubMed]
10. Fagg AH, Hatsopoulos NG, de Lafuente V, Moxon KA, Nemati S, Rebesco JM, Romo R, Solla SA, Reimer J, Tkach D, Pohlmeyer EA, Miller LE. Biomimetic brain machine interfaces for the control of movement. J Neurosci. 2007 Oct 31;27(44):11842–11846. [PMC free article] [PubMed]
11. Abbott A. Neuroprosthetics: In search of the sixth sense. Nature. 2006 Jul 13;442(7099):125–127. [PubMed]
12. Childress DS. Closed-loop control in prosthetic systems: Historical perspective. Ann Biomed Eng. 1980;8(4–6):293–303. [PubMed]
13. Lee J. PhD dissertation. Univ. Michigan; Ann Arbor: 2008. A closed-loop deep brain stimulation device with a logarithmic pipeline ADC. [Online]. Available: http://hdl.handle.net/2027.42/60733.
14. Colpan ME, Li Y, Dwyer J, Mogul DJ. Proportional feedback stimulation for seizure control in rats. Epilepsia. 2007 Aug;48(8):1594–1603. [PubMed]
15. Fetz EE. Operant conditioning of cortical unit activity. Science. 1969 Feb 28;163(870):955–958. [PubMed]
16. Dobelle WH, Quest DO, Antunes JL, Roberts TS, Girvin JP. Artificial vision for the blind by electrical stimulation of the visual cortex. Neurosurgery. 1979 Oct;5(4):521–527. [PubMed]
17. Bradley DC, Troyk PR, Berg JA, Bak M, Cogan S, Erickson R, Kufta C, Mascaro M, McCreery D, Schmidt EM, Towle VL, Xu H. Visuotopic mapping through a multichannel stimulating implant in primate V1. J Neurophysiol. 2005 Mar;93(3):1659–1670. [PubMed]
18. Romo R, Hernandez A, Zainos A, Salinas E. Somatosensory discrimination based on cortical microstimulation. Nature. 1998 Mar 26;392(6674):387–390. [PubMed]
19. London BM, Jordan LR, Jackson CR, Miller LE. Electrical stimulation of the proprioceptive cortex (area 3a) used to instruct a behaving monkey. IEEE Trans Neural Syst Rehabil Eng. 2008 Feb;16(1):32–36. [PMC free article] [PubMed]
20. Fitzsimmons NA, Drake W, Hanson TL, Lebedev MA, Nicolelis MA. Primate reaching cued by multichannel spatiotemporal cortical microstimulation. J Neurosci. 2007 May 23;27(21):5593–5602. [PubMed]
21. O’Doherty JE, Lebedev MA, Hanson TL, Fitzsimmons NA, Nicolelis MA. A brain-machine interface instructed by direct intracortical microstimulation. Front Integr Neurosci. 2009 [PMC free article] [PubMed]
22. Venkatraman S, Elkabany K, Long JD, Yao Y, Carmena JM. A system for neural recording and closed-loop intracortical microstimulation in awake rodents. IEEE Trans Biomed Eng. 2009 Jan;56(1):15–22. [PubMed]
23. Marzullo TC, Miller CR, Kipke DR. Suitability of the cingulate cortex for neural control. IEEE Trans Neural Syst Rehabil Eng. 2006 Dec;14(4):401–409. [PubMed]
24. Otto KJ, Rousche PJ, Kipke DR. Cortical microstimulation in auditory cortex of rat elicits best-frequency dependent behaviors. J Neural Eng. 2005 Jun;2(2):42–51. [PubMed]
25. Rousche PJ, Otto KJ, Reilly MP, Kipke DR. Single electrode micro-stimulation of rat auditory cortex: An evaluation of behavioral performance. Hear Res. 2003 May;179(1–2):62–71. [PubMed]
26. Vetter RJ, Williams JC, Hetke JF, Nunamaker EA, Kipke DR. Chronic neural recording using silicon-substrate microelectrode arrays implanted in cerebral cortex. IEEE Trans Biomed Eng. 2004 Jun;51(6):896–904. [PubMed]
27. Weiland JD, Anderson DJ. Chronic neural stimulation with thin-film, iridium oxide electrodes. IEEE Trans Biomed Eng. 2000 Jul;47(7):911–918. [PubMed]
28. Sanes JN, Suner S, Donoghue JP. Dynamic organization of primary motor cortex output to target muscles in adult rats. I. Long-term patterns of reorganization following motor or mixed peripheral nerve lesions. Exp Brain Res. 1990;79(3):479–491. [PubMed]
29. Shuler MG, Bear MF. Reward timing in the primary visual cortex. Science. 2006 Mar 17;311(5767):1606–1609. [PubMed]
30. Merrill DR, Bikson M, Jefferys JG. Electrical stimulation of excitable tissue: Design of efficacious and safe protocols. J Neurosci Methods. 2005 Feb 15;141(2):171–198. [PubMed]
31. Brindley GS, Lewin WS. The sensations produced by electrical stimulation of the visual cortex. J Physiol. 1968 May;196(2):479–493. [PubMed]
32. Penfield W, Perot P. The Brain’s record of auditory and visual experience. A final summary and discussion. Brain. 1963 Dec;86:595–696. [PubMed]
33. Abeles M. Quantification, smoothing, and confidence limits for single-units’ histograms. J Neurosci Methods. 1982 May;5(4):317–325. [PubMed]
34. Pagano M, Gauvreau K. Principles of Biostatistics. Pacific Grove, CA: Duxbury; 2000.
35. Parikh H, Marzullo TC, Kipke DR. Lower layers in the motor cortex are more effective targets for penetrating microelectrodes in cortical prostheses. J Neural Eng. 2009 Apr;6(2):26004. [PMC free article] [PubMed]
36. Fontanini A, Spano P, Bower JM. Ketamine-xylazine-induced slow (< 1.5 Hz) oscillations in the rat piriform (olfactory) cortex are functionally correlated with respiration. J Neurosci. 2003 Sep 3;23(22):7993–8001. [PubMed]
37. Marzullo TC. PhD dissertation. Univ. Michigan; Ann Arbor: 2008. The accessibility and manipulation of brain signals for neuroprosthetic applications. [Online]. Available: http://deepblue.lib.umich.edu/handle/2027.42/61629#.
38. Weber DJ, Stein RB, Everaert DG, Prochazka A. Limb-state feedback from ensembles of simultaneously recorded dorsal root ganglion neurons. J Neural Eng. 2007 Sep;4(3):S168–S180. [PubMed]
39. Strange KD, Hoffer JA. Restoration of use of paralyzed limb muscles using sensory nerve signals for state control of FES-assisted walking. IEEE Trans Rehabil Eng. 1999 Sep;7(3):289–300. [PubMed]
40. Popovic DB, Stein RB, Jovanovic KL, Dai R, Kostov A, Armstrong WW. Sensory nerve recording for closed-loop control to restore motor functions. IEEE Trans Biomed Eng. 1993 Oct;40(10):1024–1031. [PubMed]
41. Pohlmeyer EA, Oby ER, Perreault EJ, Solla SA, Kilgore KL, Kirsch RF, Miller LE. Toward the restoration of hand use to a paralyzed monkey: Brain-controlled functional electrical stimulation of forearm muscles. PLoS One. 2009;4(6):e5924. [PMC free article] [PubMed]
42. Moritz CT, Perlmutter SI, Fetz EE. Direct control of paralysed muscles by cortical neurons. Nature. 2008 Dec 4;456(7222):639–642. [PMC free article] [PubMed]
43. Jackson A, Mavoori J, Fetz EE. Long-term motor cortex plasticity induced by an electronic neural implant. Nature. 2006 Nov 2;444(7115):56–60. [PubMed]
44. McCairn KW, Turner RS. Deep brain stimulation of the globus pallidus internus in the parkinsonian primate: Local entrainment and suppression of low frequency oscillations. J Neurophysiol. 2009 Apr;:1941–60. [PubMed]
45. Ludwig KA, Uram JD, Yang J, Martin DC, Kipke DR. Chronic neural recordings using silicon microelectrode arrays electrochemically deposited with a poly(3,4-ethylenedioxythiophene) (PEDOT) film. J Neural Eng. 2006 Mar;3(1):59–70. [PubMed]
46. Wilks SJ, Richardson-Burns SM, Hendricks JL, Martin DC, Otto KJ. Poly(3,4-ethylenedioxythiophene) as a micro-neural interface material for electrostimulation. FrontNeuroeng. 2009;2:7. [PMC free article] [PubMed]
47. Abidian MR, Ludwig KA, Marzullo TC, Martin DC, Kipke DR. Interfacing conducting polymer nanotubes with the central nervous system: Chronic neural recording using poly(3,4-ethylene-dioxythiophene) nanotubes. Adv Mater. 2009;21(37):3764–3770. [PMC free article] [PubMed]
48. Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K. Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci. 2005 Sep;8(9):1263–1268. [PubMed]
49. Farah N, Reutsky I, Shoham S. Proc IEEE Eng Med Biol Soc Conf. Vol. 1. 2007. Patterned optical activation of retinal ganglion cells; pp. 6368–6370. [PubMed]
50. Berndt A, Yizhar O, Gunaydin LA, Hegemann P, Deisseroth K. Bi-stable neural state switches. Nat Neurosci. 2009 Feb;12(2):229–234. [PubMed]