|Home | About | Journals | Submit | Contact Us | Français|
Motivated by the success of cochlear implants for deaf patients, we are now facing the goal of creating a visual neuroprosthesis designed to interface with the occipital cortex as a means through which a limited but useful sense of vision could be restored in profoundly blind patients. We review the most important challenges regarding this neuroprosthetic approach and emphasize the need for basic human psychophysical research on the best way of presenting complex stimulating patterns through multiple microelectrodes. Continued research will hopefully lead to the development of and design specifications for the first generation of a cortically based visual prosthesis system.
The societal consequences of profound blindness, coupled with recent advances in miniaturized electronics and improvements in our ability to selectively stimulate the neurons of the central nervous system, have motivated a number of groups worldwide to explore different approaches to restoring vision by electrically activating the remaining healthy parts of the visual pathways. One such interventional target is the retina. Unfortunately, the neurons of the retina often degenerate in many forms of retinal blindness  and the neurons in the retina and/or optic nerve are often destroyed in traumatic injuries or from surgeries necessitated by tumors in the visual pathways. As a result, a retinal based visual prosthesis would not be expected to provide a general therapeutic opportunity for all causes of blindness. Because retinal degeneration does not appear to spread to the neurons of the higher visual centers, a cortical approach for a visual prosthesis could provide an interventional site for virtually all forms of retinal blindness, trauma to the retina and optic nerve, and blindness necessitated by precortical surgical procedures.
The concept of restoring sight to individuals with profound blindness by electrical stimulation at some region in the visual pathways is easily understood, but its achievement has been an elusive goal. Blindness evoked retinal degeneration, the delicate nature of the retina, and the complex anatomical locations of the optic nerve and thalamus make these interventional sites particularly challenging. Thus, the earliest work on a visual prosthesis was targeted at the visual cortex [2–4]. A cartoon of what such a cortically based visual prosthesis might look like is shown in figure 1. A cortical visual prosthesis, like other proposed approaches to sight restoration, would consist of a video camera, concealed in a pair of eyeglasses that generates a video data stream which captures the visual scene in front of the subject. This video data stream is then transformed and processed by a bioinspired retina-like encoder. This encoder would remap visual space onto cortical space and adjust the stimulus levels delivered to the implanted cortical electrodes. The digital data stream would then be transmitted wirelessly to a number of electrode arrays implanted in the subject's visual cortex. The processed digital data stream would produce a two-dimensional spatio-temporal pattern of electrical currents that would be injected into the cortical tissues via the implanted electrode arrays, and a two-dimensional visual percept would be created in the subject. For example, in the cartoon of figure 1, the spatio-temporal stimulus pattern would create a visual perception of the letter ‘h’ in front of the subject.
This goal of restoring a useful visual sense in individuals with profound blindness using electrical stimulation of visual cortex was first pursued by Brindley , Dobelle  and Pollen . These efforts were based on the earlier observations of Penfield and Rassmussen  that electrical stimulation of visual cortex with surface electrodes evoked perceptions of points of light, termed phosphenes. These early studies demonstrate that crude spatial patterns of phosphenes could be evoked by stimulation of groups of electrodes positioned on the surface of the visual cortex. However, these early efforts did not culminate in the restoration of a useful visual sense in their subjects. Problems with this early work were associated with the surface electrodes that were used to evoke phosphenes. These electrodes were relatively large and located above the pia arachnoid that overlays the cortical tissues. Thus, relatively high electrical currents were required to evoke phosphenes, and when multiple electrodes were stimulated, these large currents could interact in a nonlinear fashion, evoking phosphenes with unpredictable spatial properties. In more recent human experiments with such surface electrodes, occasionally the currents used to evoke phosphenes were observed to induce seizures in some subjects. This body of work suggests that surface electrodes may not be a tenable approach to stimulation of the visual cortex, and this observation might be extended to other stimulation sites in the visual pathways.
The next era of work that was focused on the development of a cortically based visual prosthesis was done with small clusters of microelectrodes that were inserted into the visual cortex and which were intended to evoke much more focal excitation of cortical neurons. The work of Bak et al  and Schmidt et al  in human subjects and Bradley et al  in a non-human primate provided basic information on the nature of the percepts that were evoked with such focal stimulation, but this work also did not result in the development of a useful cortically based visual prosthesis.
Work over the past two decades has focused on the use of high electrode count microelectrode arrays designed to be implanted into the visual cortex. Because of their small dimensions, the active tips of the electrodes in these implants, like those used in the work of Bak  and Schmidt , are expected to achieve highly focal stimulation of cortical neurons located around the tips of each electrode. However, the large numbers of electrodes in these arrays could mediate spatial stimulation patterns that generate the complex spatio-temporal percepts expected to be needed in a useful visual prosthesis. Although this body of work has not yet resulted in a clinical visual prosthesis system, we report in this paper ongoing work that is focused on solving some of the challenges that such a cortical visual prosthesis must overcome.
In the following sections, we describe work that is focused on (1) the development of an optical encoder/stimulation system that encodes the visual scene in front of a subject and controls the stimulation currents that are passed through the implanted cortical electrodes, (2) the neural interfaces that will allow highly selective access to large numbers of cortical neurons, (3) behavioral experiments that are underway in non-human primates to better understand the nature of the phosphenes that would form the basis for cortical microstimulation and (4) the experimental paradigms that will be used in basic studies with human volunteers of the patterned percepts that are evoked by patterned electrical stimulation. Although some of these projects are well underway, others are just beginning, but the integrated effort of a number of laboratories has reached the stage that experiments in human volunteers is expected within the next few years.
The output of the video camera must be transformed into a spatio-temporal pattern of stimuli that are sent to the implanted cortical electrodes. This transformation must take into account many issues that are normally performed by the functioning retina such as retinotopy, phosphene thresholds, phosphene brightness and potential interactions that may exist between phosphenes. thus, it may be prudent or even necessary to incorporate some of the normal retinal functions into this processor of the video camera's output. Such a bioinspired retina-like encoder/stimulator could perform a multichannel spatio-temporal filtering to extract and enhance the most salient features of the visual scene in front of a subject, and then transform this information into a neuromorphic stream of electrode addresses and stimulus intensities. The system then will transmit this information to the implanted device through a radio-frequency link for a wireless transfer of power and data. The implanted electrode array/integrated electronics system must decode the signals, identify the target electrodes and adjust the stimulus parameters applied to the appropriate microelectrodes.
The goal of developing such a bioinspired retinal encoder is not to simply record a high-resolution image, but to transmit visual information in a meaningful way to the appropriate site(s) in the brain. Hence, the question of how the information about the external world is compressed in the retina and how this compressed representation is encoded into spike trains is of seminal importance. The full description of this problem has been discussed elsewhere [10–12] and is beyond the scope of this paper, but figure 2 summarizes the basic architecture of a bioinspired model we are currently investigating. It is based on information about the photoreceptor distribution within the human/primate retina and on electrophysiological recordings from populations of retinal ganglion cells.
To approximate spatio-temporal transfer functions in the encoder, we are using a linear combination of separate spatial and temporal filters that mimic the function of the photoreceptor, horizontal, amacrine and bipolar cells in order to enhance specific features of the captured visual information such as edge enhancement (band pass spatial filtering) and light adaptation (sensitivity control). The encoder/stimulator takes into account the irregular distribution of photoreceptors within the human retina and is able to encode the visual information in a way that is similar to the output of real retinal ganglion cells. Thus the continuously varying input video stream is converted into neuromorphic pulse-coded signals through a circuit that emulates the function of retinal neurons [10, 11].
Although the model is essentially analog, we have used a digital hardware implementation in order to have a more flexible and standardizable device. Furthermore, we have developed software tools to automatically translate the retinal models into specific digital hardware implementations useful to create portable devices [10–13].
It is expected that this bioinspired retinal encoder/stimulator will be able to extract and enhance the most relevant features of the visual scene and also to re-encode this information into a neuromorphic stream of electrode addresses. The whole system is presently able to work properly up to 40 MHz. This means that 40 000 electrodes could be stimulated with an inter-spike temporal resolution equal or lower than 1 ms.
A number of psychophysical experiments have been performed to establish rough limits on how many electrodes may be required to be implanted in the visual cortex to achieve a useful, albeit limited, visual sense in a profoundly blind subject. Early experiments were performed by Cha et al  using a portable, pixelized vision simulator where a video camera was mounted on a pair of goggles that also contained a miniaturized video monitor. The monitor was viewed monocularly and subtended an approximate 5° field of view. Perforated masks were placed on the monitor creating a pixelized view of the scene in front of the subject. The subject was trained with the system by reading text displayed on a video monitor. It was learned that with a 625 pixel view of the text, the subjects could read out loud at about 200 words min−1 . Normal out-loud reading speeds without the pixelized system were about 300 words min−1. After training, subjects were challenged by navigating a programmable maze constituted in a large room. It was further learned that with 625 pixels of vision, the subjects were able to navigate the various mazes at near normal walking speeds and made virtually no errors . These experiments suggested that 625 electrodes, appropriately placed in the visual cortex, with each electrode evoking an appropriate and independent phosphene may be sufficient to provide limited mobility for a profoundly blind individual. More recent results by Dagnelie [17, 18] have confirmed these findings and extended them with a more refined simulator. It is stressed that these simulations might not be as predictive as one might wish: the pixelized input is still processed by the retina, thalamus and visual cortex. Direct patterned stimulation of the cortex might not evoke the useful patterned percepts that characterize normal vision.
Much of the work by Cha et al was motivated by the development by Utah researchers of a 100 electrode array, termed the ‘Utah Electrode Array, or UEA’ . A scanning electron micrograph of a UEA is shown in figure 3. The length of the microelectrodes in this array was designed to be 1.5 mm, a length that would reach layer 4Cβ, the layer in the visual cortex that normally receives visual input from the lateral geniculate nucleus of sighted individuals. The 100 microelectrodes are separated from each other by 0.4 mm, and they project out from the surface of a 4 mm × 4 mm × 0.25 mm silicon substrate that rests on the surface of the cortex. The base of each microelectrode is surrounded by a moat of glass that insulates each electrode from neighboring electrodes, and the entire structure, with the exception of the electrode tips, is insulated from the neural tissues by a polymer. In the present embodiment of the UEA, the rear surface of each microelectrode contains a bond pad that mediates bonding of a lead wire that connects to a percutaneous, skull-mounted connector. The tips of the electrodes are metalized with sputtered iridium oxide to facilitate electronic to ionic transduction. The UEA is implanted into the cortical tissues using a high velocity, impact insertion technique  and when implanted, the very large surface area of the 100 microelectrodes acts to integrate the array with the cortical tissues, thereby providing a stable interface to the cortical neurons.
The UEA has been used for over a decade in many experiments to record neural responses from single units in cat visual cortex  and monkey motor cortex . A recent example of this recording stability is illustrated in figure 4 left, where the implant site was the cat motor cortex. In this example, 75 electrodes in the 100 electrode UEA were able to record identifiable single unit responses at the time of sacrifice (over 8 months). The array implanted in this animal had sputtered iridium oxide film (SIROF) tips designed for micro-stimulation of cortical tissue. The single-unit recording properties of the UEA have recently been compared to the recordings made with conventional single microelectrodes, and it was found that the UEA records these single units with comparable fidelity .
We have recently performed a set of acute implantations of UEAs in the temporal lobes of human subjects. These subjects suffered from temporal lobe epilepsy and were about to undergo a resection of a region in their temporal lobe to reduce the frequency and severity of their seizures. We received permission from the Food and Drug Administration and the University of Utah Institutional Review Board to perform short-term implantations of UEAs prior to the surgical resection. As can be seen in figure 4 (right), the UEA used in this experiment (Blackrock Microsystems, Inc., Salt Lake City, UT) was also able to record identifiable single units from these human subjects with a fidelity similar to that observed in our cat recordings.
The ability of UEAs to record single units from cortical neurons over these long periods demonstrates that the implantation of systems as complex as a UEA can be done without major complications, even though many of the implanted electrodes certainly will damage blood vessles and many neurons. We have also performed histological analysis of the regions of the cat cortex that have been implanted over long periods. We see a fibrotic proliferation around the electrodes that is typical of penetrating wounds. However, in spite of this tissue response, the presence of single unit recordings indicates that large numbers of cortical neurons are viable around the electrode tips. Work is ongoing in our Utah labs to better understand the correlations between an electrode's recording capability and the extent of the tissue insult around the electrode.
Because the tips of the penetrating microelectrodes abut the cortical neurons, the amount of electrical current that must be injected into the tissues via the electrodes to excite the neurons is much lower than that required with electrodes that are placed on the surface of the cortex. Dobelle reports  that the currents required to evoke phosphene percepts in his subjects using surface electrodes are in the range of 1–2 mA, some two to three orders of magnitude larger than the currents required to evoke phosphene percepts when injected via penetrating mircroelectrodes . We look forward to exploring electrically evoked phosphene thresholds in the monkey experiments we will be conducting in the near future and that are described in a subsequent section of this paper.
In order for a clinical embodiment of a cortically based visual prosthesis to be acceptable to individuals with profound blindness, the implant system must be cosmetically acceptable. Therefore, it should be a fully implantable system with no percutaneous connections. Colleagues at the University of Utah have been working over the past few years to develop wireless versions of the UEA . A cartoon of a working model of such a wireless UEA and an assembled prototype are shown in figure 5. The wireless version of the UEA contains an integrated circuit that performs signal processing/stimulus-controlling operations as well as power and signal telemetry . The individual components of the wireless UEA have been tested in vitro and in animal experiments, but the fully integrated wireless system is still under development.
The physiological and behavioral experiments we have performed in the cat cortex and in the peripheral nerve have motivated us to work with an animal model that is better suited to behavioral experiments and we have begun training of non-human primates. Specifically, we will be using this animal model to learn (1) how much current must be injected via electrodes of a UEA implanted in the visual cortex to evoke phosphene percepts, (2) what is the long-term stability of these thresholds, (3) how close can two injection sites be to evoke the perception of two discrete phosphenes, (4) what is the lowest stimulation frequency that evokes non-flickering phosphenes and (5) do current injections in neighboring electrodes alter phosphene thresholds in tested electrodes. Although this list of experiments is extensive and will take considerable time to complete, the results of these experiments will set the stage for future experiments in human volunteers.
A standardized paradigm is being used in these behavioral experiments. Monkeys are trained to fixate on a central small crosshair projected on a video monitor (figure 6). A test target of arbitrary size, color and intensity is briefly presented at some random location on the screen. An auditory tone then cues the monkey to indicate if it saw the test target by removing its right hand from a switch. If it did not detect the target, it indicates this by removing its left hand from a second switch. This experimental paradigm allows us to correlate visual thresholds with intensity, color, size and retinal eccentricity (figure 7). Performance data from a monkey performing a similar behavioral task are shown in figure 8.In this task, the monkey was challenged to discriminate between one or two spots projected on the monitor. As seen in this figure, our monkeys presently perform this task with a high rate of success over the 3 month period that the tests were run. These monkeys will soon be implanted with a UEA and the daily micro-stimulation training/testing will be started. The photic test targets will be interspersed with brief current injections via the electrodes in the implanted array. Ideally, the monkeys will perceive the cortical micro-stimulation as simply another visual stimulus and respond according to their training. However, some additional training may be necessary to obtain reliable responses to micro-stimulation. Using this paradigm, we will be able to perform longitudinal studies of the stability and variance in the stimulus parameters that evoke just perceptible phosphenes. A similar training paradigm will be used to examine the questions outlined above.
We will also study more complex psychophysical questions focused on patterned stimulation. Specifically, we will want to learn how different must two patterns of electrical stimulation be in order to evoke two different percepts in the monkeys. Monkeys will be trained to indicate whether two visual stimulus patterns are the same or different using a ‘two alternative-forced choice’ paradigm. As with the standard paradigm described above, when the monkeys are performing this task at acceptable levels, we will ask them if they can distinguish two differing patterns of electrical stimulation.
These sets of non-human primate experimentation will not only set the stage for human studies but they will also be used to obtain FDA and local Institutional Review Board permission for the human experimentation. These applications will be sought with physiological and histological data obtained from our ongoing cat experiments.
Although the field of visual neuroprosthesis is changing rapidly, it is often not appreciated that vision rehabilitation in a case of blindness most likely cannot be reduced simply to the production and control of phosphenes. With approved clinical protocols, we have begun preliminary investigations, mainly with passive microelectrode arrays, to establish the safety of the surgical implantation procedures and to adapt the array insertion technique to the requirements unique to the human situation. The results of these insertion tests have been encouraging. Moreover, similar electrode arrays have been implanted in the motor cortex of several quadriplegic patients and, by monitoring their neuronal activity, used to provide them with control over external devices such as wheelchairs, a computer terminal or a robotic arm . No adverse effects associated with these implantations have been reported since the beginning of the trials in the fall of 2005.
Like auditory brainstem implants in the profoundly deaf and recent work with ‘deep brain’ stimulation in those afflicted by severe Parkinson's disease, cortical visual prostheses for the blind are now on the verge of entering a phase of human experimentation. The complexity of the visual system is such that many issues remain to be answered in these human experiments before useful vision can be provided for blind patients.
Due to the special nature of human vision and because verbal feedback can provide detailed information regarding the nature of the evoked percepts, most of the core questions must be centered around the visual percepts elicited by electrical stimulation [11, 28]. These questions will hopefully be answered in experimental trials involving human subjects. We expect to obtain some preliminary observations on these questions from the non-human primate experimentation. Examples of the basic physiological and psychophysical issues that must be answered are as follows.
The answers to these and other important questions will provide design specifications for a future generation of cortical visual prosthesis that could be used in clinical trials.
However, although there are many attributes that characterize a visual scene (e.g. color, form, motion and field of view), present strategies in visual prosthesis development address only the most basic component, i.e. spatial detail. Thus, it has been suggested that the quality of visual percepts with a cortical prosthesis is likely to improve as remaining technical challenges are addressed. Some investigators have assumed, for example, that more complex perceptions could be generated by simply increasing the number of stimulating electrodes, thereby increasing the potential resolution of images being perceived [29, 30]. Obviously, because the complex relationship between patterns of electrical stimulation and the resulting visual percepts remains largely unknown, this view may be an oversimplification. At the same time, current psychophysical evidence suggests that the human visual system is able to identify and extract complex information (such as a human face) from relatively poor quality images by focusing on multiple salient visual features and cues . The assumption that visual perception will improve by concentrating on efforts geared toward increasing image resolution alone (as opposed to other visual attributes such as temporal encoding related to motion perception) may be incorrect. On the other hand, the stimulation patterns that evoke visual perception are expected to vary with a number of parameters such as pulse duration, duty cycle, amplitude, number of pulses in a train, etc. These parameters can be varied for every channel and might take on very different values for each implanted subject. Even for a given subject, these values could change over time due to electrode encapsulation or neuroplastic changes.
Clearly we need basic human psychophysical research on the best way of presenting complex stimulating patterns through multiple microelectrodes and we will need to develop new approaches to generate more recognizable patterns of visual perceptions. The work that has been described in this paper is focused on the development of a tool that can enable this basic psychophysical research. The outcome of this research will be an improved understanding of how complex spatio-temporal patterns of neural activity in the visual cortex evoke complex two-dimensional visual perceptions. This understanding will hopefully lead to the development of and design specifications for the first generation cortically based visual prosthesis systems. We hope eventually to achieve a promise that has been decades in the coming: providing useful sight restoration to those with profound blindness.