Interfacing the Brain – With Microelectronics?
Division of Engineering, Brown University, Providence, RI 02912, U.S.A.
Department of Neuroscience, Brown University, Providence, RI 02912, U.S.A.
1. Introduction
While paying appropriate homage to the technological revolution(s) empowered by modern microelectronics, and the continued march towards ever increasing sophistication of nanoscale high-density digital circuits, it maybe occasionally useful to pause and draw a reference to a computer of very different variety, namely the human brain. A product of millions of years of evolution, this non-silicon, biochemical processor features about 1015 interconnects amongst its three-dimensional (3D) network of 1011 neurons (“neuristors”), while consuming about 10 W of power. Approximately 1016 synaptic events per second occur within the sublime architecture of our “cortical computer”, whose detailed design and operational principles define the main quest of modern neuroscience.
Science fiction has frequently visited the question of direct communication with the brain, including the prospects of interfacing the brain with external machines and extrasensory communication such as telepathy. Peeling away many such ideas of “brain alchemy”, in addition to their ethics, there is a nonetheless a serious and compelling medical rationale to pursue direct retrieval of command and control signals from the brain. Although difficult to quantify and express by a single number, there are millions of individuals who suffer from serious neurological illnesses, ranging from Parkinson’s disease to complete paralysis. Spinal cord injuries and degenerative diseases, such as ALS in particular, involve very large populations of people whose quality of life is much impaired, even if the brain is cognitively functional. Enabling such individuals to translate their command thoughts from the brain to direct activation of external assistive devices offers a potential intersection with modern technologies of significant societal and health care value.
In this chapter, we outline some of the first steps in this new area of neuroengineering and neurotechnology, focusing on specifics examples of contemporary progress. We first review very recent work, where direct extraction of signals from the human brain, preceded by vast amount of animal work, is showing striking early promise for enabling severely paralyzed individuals to interface with their external surroundings. We then consider the role of modern (and future) microelectronics in creating powerful platforms for future neurotechnologies featuring brain- and body-implantable neural prosthetic microsystems.
2. “Thought-to-action” using brain-implanted cortical neurosensor arrays
Within the enormously complex human brain, a highly distributed and hierarchical biological computer, modern neuroscience has made much progress in identifying certain areas of the cortex where direct access to and extraction of electrical signal impulses at the single neuron level can provide information about the commands which enable e.g. our motor functions (arm and leg movement).1 For example, the so-called arm area can be identified within the primary motor cortex, as indicated in Fig. 1, which also shows the typical local neuronal architecture. Since all current noninvasive (i. e. external to the skull) brain “imaging” techniques from EEG to MEG to fMRI lack the spatial resolution anywhere near the single neural cell level, invasive insertion of microelectrodes to an appropriate depth is required to record the ~ms duration biphasic action potentials (or “spikes”) from single neurons, on which neural coding is mainly based – see Fig. 1. Two-dimensional arrays of up to 100 of such needle-like surgically implanted electrodes with μm size tips and inter-electrode separation on the 100 μm scale (to roughly match the neuronal density) have been employed usefully in several animal “models” to develop understanding of the neural code that drives e.g. the arm motion in a monkey.2 The decoding approaches involve correlating the rates of spike train activity recorded across the microelectrode array with the observed motion of the monkey’s arm e.g. in the x-y plane, while the monkey uses a joystick or mouse to move a cursor on a computer screen.
Figure 1. Location of the arm area in the cortex (upper left); typical action potentials of neural spikes (lower left); local neural landscape with a single needle-like microelectrode in the vicinity of a neuron (right). Cell body size ~20 μm.
Figure 2. A silicon-based cortical microelectrode array (upper left); schematic of insert of the array and percutaneous connection to a skull-mounted pedestal connector (lower left); and a patient in a human clinical trial (courtesy of Cyberkinetics Inc).
A contemporary version of this brain recording platform, connecting the cortical neuroprobe to the signal processing electronics systems, summarized in Fig. 2 has been recently used in the very first human pilot trials involving chronic implants inserted for up to two years in severely paralyzed patients by a team combining expertise from Brown University, Cyberkinetics Inc., and Harvard Medical School.3 The Cyberkinetics BraingatesTM system employs a 10×10 element silicon-based multielectrode neural array4 encased in silicone, with protruding platinized tips in physical contact with the neural tissue (Pt is chosen for impedance “matching” to the neural “electrolyte”). A 100-wire bundle (Au, 25 μm diameter) conducts the ~10–50 μV amplitude neural signals through a skull-mounted titanium-based percutaneous connector to an electronics instrumentation package which features low noise preamplifiers for each channel, signal multiplexing, A/D conversion, spike “sorting” and other signal processing tasks. Finally, the neural signals are decoded by specific algorithm approaches and, once deciphered, are connected to an external device, such as an electronic mouse, for exploring direct cortical control by the brain of a cursor on a computer screen or an artificial prosthetic device. While the process of surgically implanting the multi-electrode array places individual recording tips randomly into the neural cell tissue (within a depth of about 1 mm from the top of the cortical surface), the array geometry and dimensions are such that in general there is a high probability for a given electrode registering neural impulses from a single nearby neuron (the practical effective sampling radius of each tip is on the order of 150 μm). The occasional pick up from two nearby neurons can generally be discriminated and separated from their time-amplitude stamps. Prior animal studies in monkeys, involving the removal and reimplantation of the “bed-of-needles” arrays,5 have shown that these neuroprobes in the primary motor cortex create generally only modest tissue damage over time spans of years. Likewise, long-term tissue-electrode interactions (from immune reactions) appear to be moderate in terms of decreasing the recording performance of the array and impairment of neural circuitry. Figure 3 shows a typical multichannel recording from a monkey with most of the 100 channels in the 10×10 microelectrode array picking up useful biphasic action potential impulses of ~ms duration.
Returning back to Fig. 2, which shows a photograph of a human tetraplegic patient in the clinical test setting, we see a patient with an implanted multielectrode array connected to a head-mounted module that mainly houses preamplifiers and multiplexing circuitry, subsequently cabled to multiple electronic system modules. As summarized in Ref. 3, this clinical trial, conducted over nearly one year (and currently underway with several other paralyzed patients suffering from major impairment of motor functions) has enabled the patient, whose spinal cord is severed at the neck, to control a cursor on a computer screen for communication activities such as reading email, typing messages, drawing elementary freeform shapes, and operating a open-close prosthetic hand. Other assistive devices amenable to direct brain control that are being developed and tested include a wheelchair and a robotic arm. It is the culmination of much research and development of brain recording devices that has not only led to the these striking results in the first human trials, but is now motivating attempts to microminiaturize and enhance the performance of the neuroprobe systems by turning to advanced microelectronics.
3. Application of microelectronics to brain interfaces in neuroprosthetics
The progress in the early development of human neuroprosthetic approaches, summarized above, offers a tantalizing glimpse of a future where more complex tasks can be accomplished by neurologically impaired persons via direct retrieval of command and control signals from the brain. As noted, the present approaches to chronic multichannel cortical recording involve a passive (i.e. unpowered) implanted neural probe arrays with a percutaneous and often bulky and fragile cabling to electronic processors outside the body. The integration of a significant portion or all of the electronics onto a body-implantable platform is highly desirable for future portable and wearable prosthetics, inclusive of on-board telemetry to transmit signals to internal body sites (e.g. abdominal cavity) or external remote processors and other assistive technologies.
Figure 3. Images of the cortical array being implanted into a monkey’s brain by a pneumatic inserter (left) and “snapshot” neural recordings after repeat implants from the motor cortex, showing activity at significant majority of the 10×10 channels, and displayed to reflect the rows and columns of the multielectrode array.
Such an implantable cortical microsystem presents a multifaceted technical challenge, which includes the development and integration of ultralow-power microelectronic ICs to the neuroprobe recording platform, approaches to (broadband) on-board telemetry, either by optical (IR) or radiofrequency components, as well as the means to deliver power for the active components. The choice of an optical telemetry link, either free-space through the skin (from the top of head) or by fiber optical guides to less anatomically and physiologically demanding sites in the body is very attractive, as the expected data rates for more complex neural activity to be sampled in the future reach well beyond 1 GB/s. Finally, encapsulation and biocompatibility of such an electrically active multielement implants, in conjunction with surgical and other clinical considerations, are challenges of fundamental importance for future human applications. Here we summarize efforts to develop one approach to an implantable cortical microsystem, presently underway in the authors’ laboratories, and shown schematically as a block diagram in Fig. 4. By employing the silicon-based microelectrode array as a recording platform, we have completed a full 16-channel microsystem, and experimentally tested key elements, including in-vivo evaluation in rats.
Figure 4. Block diagram for the implantable microsystem discussed in the text, enumerating the key micro- and optoelectronic on-chip circuit elements.
Figure 5. Photograph of a 4×4 element microelectrode array flip-chip bonded to a VLSI CMOS chip as an integrated cortical neuroprobe (left) and the full implantable microsystem fabricated on a flexible LCP substrate and encapsulated by silicone (right). The “back-end” panel houses A/D and command/control CMOS chips, together with a photovoltaic converter for incoming power/clock, and a VCSEL for outgoing IR digitized data stream, both by using an optical fiber.
The full microsystem design discussed here is based on specific device layout geometry and architecture, aimed firstly at non-human primates (monkeys), and is composed of analog and digital microelectronic components as well as infrared digital optical telemetry. The microsystem is powered by a photovoltaic “optical power supply” and has a soft and flexible encapsulation. Its design emphasizes scalability to increasingly larger amounts of neural information transmitted from the cortex to assistive technologies, aiming ultimately at human neuroprosthetics.
From a purely engineering viewpoint, the concept of using the cortical microelectrode array as a platform onto which all the microelectronic and telenietric components are directly integrated as a single monolithic cortical implant module might at first appear to represent an attractive approach. However, after weighing a number of neurophysiological, biomedical, telemetric, and surgical considerations, we have chosen to pursue a spatially more distributed microsystem architecture that is shown as a fabricated prototype in Fig. 5.
The layout in the figure reflects our “dual-panel” microsystem design concept, composed of two electrically interconnected islands that are landscaped on a common flexible liquid crystal polymer (LCP) substrate. The “front-end” panel of the dual-panel system is directly implanted into the cortex. This front panel is flexibly connected to the back panel which threads through a sealable “burr” hole in the skull and resides between the skull and the skin. It houses the cortical microelectrode array (in the present prototype version, a 4×4 unit), directly flip-chip bonded onto an ultra-low power analog CMOS chip.6 The custom designed and fabricated CMOS IC includes low power preamplifier and multiplexing circuitry – see Refs. 6 and 7 for details and performance. A custom hybrid flip-chip bonding technique was developed at Brown to produce the functional, encapsulated microminiaturized front-end neuroprobe device whose separate recording characteristics have been reported in detail.7 The front end has been implanted in a rat, to show its viability in recording brain signals from the somatosensory cortex.8
A photograph of the fabricated implantable microsystem of Fig. 5 shows some of the details of the full dual-panel system layout with its micro- and optoelectronic component functional grouping, according the schematic of Fig. 4. The multiplexed analog neural signals are routed from the front panel to the peripheral circuits of the second “back-end” panel of the implant. The back end circuitry integrates a low-power A/D converter, a digital control-and-command chip, an optional microcrystal photovoltaic energy converter, and an infrared (IR) data-out transmission link employing a very low-threshold IR vertical cavity surface emitting semiconductor laser (VCSEL). The concept of an optoelectronic data link enables considerable system flexibility for future neural prosthesis applications, including very large bandwidth capability. Extraction of the digitized IR neural signals from the neural interface unit can be performed either via a free-space beam from the laser emerging directly through the skin; which is sufficiently transparent in the near infrared, or via a fiber optic strand threaded subcutaneously to other information-linking sites in the body (e.g. in the thoracic unit). Likewise, our platform enables different options for power delivery to the active components of the implant. This implementation employs a photovoltaic GaAs/AlGaAs three tandem cell photovoltaic energy converter (>50% efficiency) as the “optical power supply” of Fig. 5, but the system can be adapted to an inductively coupled RF link which we have also demonstrated in the laboratory.
To summarize the rationale for the architecture and design of our implantable, compact dual platform neural interface, we note that it is based on the convergence of engineering considerations, input from neurosurgeons and neurologists, all leading to a number of practical electronic and biomedical device choices, compromises, and solutions, with the ultimate goal of human implants providing the project motivation. Important factors include practical surgical considerations, minimizing the heat transferred by the unit directly to the cortex (temperature rise below 1°C based on thermal modeling scaled up to a full 10×10 array, i.e. a 100 channel system), mechanical stability against stress on the tether from movement of brain, the microsystem’s scalability to ever higher performance in terms of its signal bandwidth, its convenient modularity, adaptability to different wireless telemetries and powering (infrared, either free space or via optical fiber, RF), adaptability to multiple encapsulation strategies, and so on.
A performance demonstration result of the full 16 channel microsystem is shown in Fig. 6, in a benchtop evaluation test, where the microelectrode array has been excited by a series of “pseudospikes” (or artificial neurons) by applying a bipolar transient voltage through a wire to a saline solution (ACSF, which mimics closely the electrolyte background in the brain) into which the microsystem unit was immersed (labeled “electrical” in the left panel of Fig. 6).8 After detection and reconstruction of the received IR optical data stream transmitted by the full microsystem into analog form (labeled “all-optical system”), a typical series of pseudospikes at high fidelity and good signal-to-noise ration were obtained as shown. (The upper trace is acquired from electrically accessing the front end of the system only). The signals acquired from the all-optical system in the lower trace shows almost the same preamplifier gain of 42 dB as the front-end microelectronic system, with comparable input noise characteristics. In the panel on the right of Fig. 6, a detail of the digital optical data stream trace is compared with the digital sampling rate clock trace on a microsecond time scale. The pulse coded modulation (PCM) data at the photoreceiver demonstrates the robust fiber optic data link in the present microsystem. The lower trace originates from the sinusoidal waveform applied to the external diode laser power source, current modulated to generate an infrared optical input carrying a 15.36 MHz clock signal.
At the time of writing, the microsystem described above is about enter animal testing phase in monkeys. A proof-of-concept 16 channel version prototype of this microsystem has been partially implanted in a rat animal model and proven to be functional on benchtop experiments, as summarized above. Many developmental challenges remain, nonetheless. These include long term reliability of encapsulation, development of practical and safe surgical procedures, and in-vivo microsystem trouble-shooting and maintenance strategies.
4. Future application prospects for brain interface microsystems
While brain implantable biomedical devices, such as the microsystem outlined above as well as others,9-11 are still at early development stages, with many technical challenges to be solved, it is important to consider the longer-term impact and prospects for an implanted neural prosthetic system, as an extension of the technology currently being advanced. A visionary/imaginary example of a fully “closed-loop” design augmenting the human nervous system is suggested schematically in Fig. 7, synergizing three principal performance functionalities: (a) a cortical recording microsystem (BIC sensor), capable of simultaneously addressing several cortical sites; (b) the ability to stimulate both the cortex, as well as peripheral nerves (such as muscle nerves in an arm of a leg); and (c) a very wide bandwidth “artificial” neural signal distribution network.
Figure 6. (a) Comparison of an electrically recorded train of “pseudospikes”, generated in a saline solution, with those retrieved from the digital optical data stream transmitted from the full microsystem by the IR optical fiber; (b) upper trace shows a piece of the digitized outgoing optical data train while lower trace shows the incoming clock signal retrieved by photovoltaic conversion of modulated IR laser light entering the microsystem via the fiber.
Figure 7. Futuristic view of an optical fiber wiring of neural communication system where cortical recording and stimulation are combined with peripheral nerve stimulation and sensing, all part of a closed-loop interactive prosthetic system.
The cartoon of Fig. 7 suggests a scenario, where (biocompatible) optical fibers – analogous to optical telecommunication systems, – provide the information pathway for such an artificial nervous system. We note that the demand for processing large numbers of single neurons is already substantial at the present early stages of interfacing the brain, and is expected to increase significantly as the field of neuromotor prosthesis develops to mimic increasingly complex neurally driven tasks. Neural stimulation of the cortex will become essential in any closed-loop system, where feedback and sensory inputs from prosthetic devices or intact limbs are returned to the brain. For example, one would seek to evoke movement response and correlate this with spatiotemporal stimulation patterns.
As noted above, individual action potentials have duration of ~1 ms. Hence neural recording data stream from 100 electrodes at a sampling rate of only 10 kHz with a resolution of 10 bits (which is ultimately not adequate), requires a baud rate of 2 Gb/s. In the futuristic scenario of Fig. 7, the demands of implementing a closed-loop system will ultimately push ultralow-power and low-noise CMOS custom designed microelectronics to the limits, especially for analog circuits. The corresponding optoelectronic components would also need to provide ultrahigh speeds. In the scheme of Fig. 7, a cortical microsystem-on-chip is optically interconnected by a fiber to an advanced signal processing and command-control center module placed in the chest or abdominal cavity (akin to the heart pacemaker). Glass or plastic optical fibers are a lightweight, biocompatible, flexible means of routing neural signals within the body, without being subject to electromagnetic interference; they can carry very large amounts of information along a single strand, in contrast to metal wiring. This will drastically reduce the wiring burden and facilitate configurations where sensors and nerve stimulators in the limbs are also connected to a network by the optical neural information highway.
In terms of peripheral nerve stimulation, recent advances in functional electrical stimulation (FES) of muscle nerves by using electrical impulses show significant promise for restoring voluntary movement in patients with paralysis or other severe motor impairments.12 Current approaches for implantable FES systems involve multisite stimulation, often with extensive wiring, posing research issues related to the physical size, power and signal delivery of the wiring, in conjunction with surgical and safety challenges. We note that by exploiting high-efficiency epitaxially-grown solar-cell type microdevices, efficient conversion of light to electrical stimulus (and system power) can be achieved. To explore a different means for delivering the stimulus to a distant muscle nerve site, we have recently elicited in-vitro FES response using a high-efficiency microcrystalline photovoltaic device as a neurostimulator, integrated with a biocompatible glass optical fiber that forms a lossless, interference-free lightwave conduit for signal and energy transport. As a proof-of-concept demonstration, the sciatic nerve of a frog was stimulated by the microcrystalline device connected to a multimode optical fiber (core diameter of 62.5 μm). This set-up converts optical activation pulses (~100 μs) from an infrared semiconductor laser source (at 852 nm wavelength) into an FES signal.13
5. Summary
In this overview, we have attempted to give the reader an impression of the current intersection of microelectronics and brain science, in the quest for hybrid device platforms to provide a direct electronic interface with the brain. Only early steps in this direction have been taken, in what is a major technological and biomedical challenge that will have far-reaching societal impact in the future.
We have focused specifically on an approach for retrieving neural signals from the brain through a contemporary implantable microsystems, where the performance demands on the micro- and optoelectronics devices at the moment are modest by e.g. current computer industry standards (though packaging of brain implantable devices is highly nontrivial). However, if and when the integrated neuroprobe platform discussed above, or related approaches, will be shown to be practical in a human patient setting, it is clear that this emerging neurotechnology will demand the best that semiconductor technology can offer. Truly far-reaching ideas, such as implementation and/or integration of silicon “brain” components, might then also become more realistic.14
Acknowledgments
The authors acknowledge the participation of many colleagues and students in the work described above, including Leigh Hochberg, Mijail Serruya, Selim Suner, Matt Fellows, John Simeral, Kristina Davitt, Paula Petrica, Joanna Zhang, and Heng Xu. We especially thank Dr. Eric Eisenstadt for his role in catalyzing this work. Research supported by ONR, DARPA, and NIH.
References
- Among many general brain science textbooks see e.g. E. R. Kandel, J. H. Schwartz, and T. M. Jessel, Principles of Neural Science, 4th ed., New York: McGraw-Hill Medical, 2000.
- M. D. Serruya, N. G. Hatsopoulos, L. Paninski, M. R. Fellows, and J. P. Donoghue, “Instant neural control of a movement signal,” Nature 416, 141 (2002); D. M. Taylor, S. I. Tillery, and A. B. Schwartz, “Information conveyed through brain-control: Cursor versus robot,” IEEE Trans. Neural Syst. Rehabil. Eng. 11, 195 (2003).
- L. R. Hochberg, M. D. Serruya, G. M. Friehs, etc., “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature 442, 164 (2006); the website includes video clips.
- K. E. Jones, P. K. Campbell, and R. A. Normann, “A glass/silicon composite intracortical electrode array,” Ann. Biomed. Eng. 20, 423 (1992).
- S. Suner, M. R. Fellows, C. Vargas-Irwin, K. Nakata, and J. P. Donoghue, “Reliability of signals from chronically implanted, silicon-based electrode array in non-human primate primary motor cortex,” IEEE Trans Neural Syst. Rehabil. Eng. 13, 524 (2005).
- W. R. Patterson, Y.-K. Song, C. W. Bull, et al., “A microelectrode/micro-electronic hybrid device for brain implantable neuroprosthesis applications,” IEEE Trans. Biomed. Eng. 51, 1845 (2004).
- Y.-K. Song, W. R. Patterson, C. W. Bull, et al., “Development of a chipscale integrated microelectrode/microelectronic device for brain implantable neuro-engineering applications,” IEEE Trans. Neural Rehabil. Eng. 13, 220 (2005).
- Y.-K. Song, W. R. Patterson, C. W. Bull, et al., “Development of brain implantable microsystems with infrared optical telemetry for neuromotor prosthesis,” submitted to IEEE Trans. Biomed. Eng. (2006).
- M. Mojarradi, D. Binkley, B. Blalock, R. Andersen, N. Uslhoefer, T. Johnson, and L. Del Castillo, “A miniaturized neuroprosthesis suitable for implantation into the brain,” IEEE Trans. Neural Syst. Rehabil. Eng. 11, 38 (2003).
- I. Obeid, J. Morizio, K. Moxon, M. A. Nicolelis, and P. D. Wolf, “Two multichannel integrated circuits for neural recording and signal processing,” IEEE Trans. Biomed. Eng. 50, 255 (2003).
- R. Harrison, P. Watkins, R. Kier, R. Lovejoy, D. Black, R. Normann, and F. Solzbacher, “A low-power integrated circuit for a wireless 100-electrode neural recording system,” Tech. Dig. ISSCC (2006), pp. 2258–67.
- W. M. Grill and R. F. Kirsch, “Neuroprosthetic applications of electrical stimulation,” Assist. Technol. 12, 6 (2000); B. P. Heilman, M. L. Audu, R. F. Kirsch, and R. J. Triolo, “Selection of an optimal muscle set for a 16-channel standing neuroprosthesis using a human musculoskeletal model,” J. Rehabil. Res. Dev. 43, 273 (2006).
- Y.-K. Song, J. Stein, W. R. Patterson, et al., “A microscale photovoltaic neurostimulator for fiber optic delivery of functional electrical stimulation”, to appear in J. Neuroeng. (2007).
- T. W Berger, M. Baudry, R. D. Brinton, et al., “Brain-implantable biomimetic electronics as the next era in neural prosthetics", Proc. IEEE 89, 993 (2001).