Use of sensory substitution devices as a model system for investigating cross-modal neuroplasticity in humans

Use of sensory substitution devices as a model system for investigating cross-modal neuroplasticity in humans

Blindness provides an unparalleled opportunity to study plasticity of the nervous system in humans. Seminal work in this area examined the often dramatic modifi cations to the visual cortex that result when visual input is completely absent from birth or very early in life (Kupers and Ptito, 2014). More recent studies explored what happens to the visual pathways in the context of acquired blindness. This is particularly relevant as the majority of diseases that cause vision loss occur in the elderly. Our lab and others have demonstrated compromised visual pathway integrity in those with peri-natal and acquired blindness (Schoth et al., 2006; Chan et al., 2012; Li et al., 2013; Lee et al., 2014; Dietrich et al., 2015; Ho et al., 2015; Reislev et al., 2015). Additional studies have begun to examine the changes occurring with certain disease states: patients suff ering from retinitis pigmentosa, optic neuritis, and glaucoma, all so far demonstrate deterioration of the white matter tract architecture as a function of disease severity (Garaci et al., 2009; Gabilondo et al., 2014; Ohno et al., 2015). This evidence indicates that the visual system as a whole is profoundly susceptible to degeneration even with small amounts of vision loss. On the surface, these investigations appear to have negative implications for vision restoration eff orts. Yet, parallel studies which examine the phenomenon of cross-modal plasticity suggest that a remodeling of the central nervous system is possible, such that areas of the brain which have been deprived of normal aff erent input are able to reconstitute themselves to be receptive to alternative sensory channels (Merabet and Pascual-Leone, 2010; Kupers and Ptito, 2014). The literature includes several examples of investigations which show that the visual cortex will react to tactile and auditory stimuli in the blind but will be less readily recruited in sighted patients (Merabet and Pascual-Leone, 2010). Moreover, cross-modal interactions have been demonstrated well beyond the traditional “critical period” and into late adulthood, albeit perhaps in a less robust fashion (Sadato et al., 2002; Bedny et al., 2012; Collignon et al., 2013). The notion that the adult brain is still capable of signifi cant structural and functional remodeling after vision loss provides opportunities to restore vision through mechanical or biological means.

Sensory substitution is a non-invasive method for restoring a sense of the environment. The fi rst sensory substitution device was the white cane which is still widely used by the blind community. Braille is another example of tactile sensory substitution, and software programs such as JAWS substitute are auditory based. More modern attempts at sensory substitution aim to translate visual, camera-based stimuli into a non-visual tactile or auditory stimuli. Sensory substitution in this context was described by Bach-y-Rita in the 1960’s (Bach-y-Rita et al., 1969) and has gained traction in the last few years as a potentially legitimate means of providing functionality to the blind that is “beyond the reach of the cane’. The most recent devices continue to exploit tactile and auditory aff erent channels. The BrainPort (Wicab, Inc.) and the AuxDeco (EyePlus-Plus, Inc.) are tactile based sensory substitution devices using the tongue and forehead, respectively. The vOICe (Metamodal, Inc.), and the spinoff software program known as EyeMusic (Abboud et al., 2014) both translate visual stimuli into soundscapes. The EyeC-ane (Maidenbaum et al., 2014b) attempts to improve the functionality of the white cane to include positional information. Sensory substitution devices can be relatively diffi cult to master and the current resolution provided may remain relatively limited. Nevertheless, sensory substitution devices have been shown to result in improvements over baseline in a number of outcomes metrics (Lee et al., 2014; Maidenbaum et al., 2014a; Nau et al., 2015). Researchers are working on improving this nascent technology to be more user-friendly as well as developing the rehabilitation protocols necessary to properly retrain the brain to accept more complex alternative sensory input.

Beyond the potential for improving the mobility and independence for the blind, sensory substitution devices also provide an excellent tool to study cross-modal plasticity of the visual system in living humans. The primary advantage is their relatively inexpensive and non-invasive nature, which allows for large numbers of subjects with diff erent etiologies and durations of blindness to be followed. Modern sensory substitution devices are trying to enable activities of daily living such as object recognition, navigation beyond the reach of the white cane and non-text sign identifi cation (Lee et al., 2014; Maidenbaum et al., 2014a; Nau et al., 2015). These attributes are starting to allow researchers to study the cross-modal interactions of the brain in tasks that can more accurately represent daily activities than early studies which were relegated to testing very rudimentary stimuli. Moreover, sensory substitution devices exploit multiple afferent streams including tactile (hand, tongue, back) and auditory channels. This unique attribute can provide information about normal interactions between sensory subsystems and how they can be remodeled in the setting of blindness. In addition, it is possible to compare plasticity in the same subject over time, or compare training eff ects between age groups, disease severity and etiology of vision loss. It is possible that the way a brain reacts to using a sensory substitution device could be a biomarker for success with other, more invasive vision restoration technologies such as retinal implants (Sadato et al., 2002). Using sensory substitution devices as a model system for studying the mechanisms and eff ects of cross-modal interactions can improve our understanding of the plasticity of the nervous system that may be generalizable to conditions other than blindness.

In the past few years, positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have been used in combination with sensory substitution devices to investigate the mechanisms of cross-modal neuroplasticity in the blind. Early PET studies by our group and others showed that the visual cortex is metabolically active in the blind during sensory substitution tasks (Lee et al., 2014), and that the strength of this activity may be dependent on task diffi culty, the duration of blindness, or the etiology of visual impairment (Kawashima et al., 1995; Gougoux et al., 2005). However, PET off ers relatively low spatial resolution and is expensive to administer. In addition, PET studies require injection of radioactive isotopes, and limited tasks are allowed in any given experimental session. In contrast, fMRI, especially blood-oxygenation-level-dependent (BOLD) fMRI utilizes the intrinsic properties of paramagnetic deoxyhemoglobin in the brain and the hemodynamic response triggered by neuronal activities for non-invasive functional imaging. It provides a cheaper and safer alternative neuroimaging tool that allows multiple or repeated tasks for examining brain activities in the same subject under diff erent experimental conditions in the same scanning session and across time. Our recent fMRI studies using the BrainPort and The vOICe helped reveal the nature of brain responses under diff erent types of sensory substitution tasks.

Figure 1 BOLD functional MRI of multimodal sensory substitution.

As shown in Figure 1, when a typical checkerboard visual stimulus was presented to the eyes of normally sighted subjects, a positive BOLD response to the visual cortex was observed. This occurred primarily because of a corresponding increase in brain activity and oxygen-rich blood contents entering the visual brain region. Similarly, when BrainPort (tactile-vision) and The vOICe (sound-vision) sensory substitution tasks were administered to the sighted subjects and both peri-natally blind and acquired blind subjects in the fMRI scanner, positive BOLD responses were found in the somatosensory cortex and auditory cortex, respectively in both sighted and blind subject groups. This was expected because the somatosensory cortex mediates tactile stimuli on the tongue, and the auditory cortex mediates sound information. In the visual cortex, a negative BOLD response was found in the sighted subjects with the use of either BrainPort or The vOICe, suggestive of relatively decreased blood fl ow to this area. The negative BOLD response in the visual cortex of sighted subjects was likely due to cross-modal inhibitions which naturally occur because the visual part of the brain in sighted subjects is not necessarily required for interpreting non-visual sensory stimuli (Kawashima et al., 1995; Laurienti et al., 2002; Hairston et al., 2008). In contrast, a positive BOLD response was found in the visual cortex of both peri-natal and acquired blind subjects with both BrainPort and The vOICe input, suggesting that this brain region, deprived of aff erent visual input because of blindness, is being recruited to assist with the interpretation of both tactile and auditory information. The diff erent BOLD responses exhibited between blind and sighted subjects in the visual cortex demonstrated direct evidence of functional brain reorganization as a function of blindness. Not only were the brain responses diff erent between blind and sighted subjects, but the brains of the same blind subjects were also able to secondarily activate the visual cortex from two completely diff erent primary alternative senses (touch and sound). This fi nding supports the fl exibility of the visual brain to adapt to multi-sensory cross-modal inputs, at least in the context of visual deprivation.

To evaluate the possible causes of visual cortex activation with sensory substitution, a separate experiment was performed to test the minimum duration of training needed to increase activity of the visual cortex in the blind. Our initial fi ndings suggested that this can occur shortly. In naive subjects with no prior exposure to The vOICe, active interpretation of the vOICe stimulus after only 10 minutes of rudimentary training increased the BOLD signal in the visual cortex of the blind subjects compared to passive presentation of the same stimulus at baseline (Murphy et al., 2014). Notably, sighted subjects did not show apparent changes in BOLD activation in any visual cortical regions before and after the short training session.

In addition to active task-based fMRI, fMRI technology has the capability to evaluate functional connectivity between brain regions when the subjects are at rest. Our initial fi ndings suggested that in the passive, resting state when the subjects were not instructed to perform any tasks, functional connectivity of the visual cortex becomes weaker within sensory networks but stronger in those brain regions responsible for higher-order cognitive functions such as task-positive networks, where activity increases during cognitive tasks, and salience networks (Chan et al., 2014a). These fi ndings support the recent model that the visual brain is not limited to a single sensory modality but rather is highly plastic and task-fl exible (Reich et al., 2012). We speculate that an adaptive brain reorganization which shifted the visual cortex from a predominantly sensory network to that of a higher-order, top-down task-positive cognitive network was already extant in the blind group before task-based fMRI experiments, and that this adaptation enabled BOLD activation of the visual cortex during our experiments.

In addition to functional imaging, magnetic resonance enables multiparametric assessments of the metabolism and structure of the central nervous system in both humans and animal models. Future studies are envisioned that utilize magnetic resonance spectroscopy to determine the neurochemical changes in visually deprived brains such as the balance between excitatory and inhibitory neurotransmissions (Chow et al., 2011; Weaver et al., 2013; Wu et al., 2013). Advanced diffusion MRI techniques and MRI tracers can also be used to determine which neural pathways are altered in diff erent types of blindness (Schoth et al., 2006; Chan et al., 2012, 2014b; Li et al., 2013; Lee et al., 2014; Dietrich et al., 2015; Ho et al., 2015; Reislev et al., 2015), and which pathways are predominantly responsible for cross-modal brain activation by probing training-induced white matter plastic changes during sensory substitution rehabilitation over time. This data set could be co-registered with fMRI to study the structure-function relationships between white matter integrity and brain activities as well as sensory substitution performance across diff erent life fractions of blindness.

An interesting observation is that even though cross-modal plasticity is a well-established phenomenon in the blind, and that this eff ect is measurable using various methods of neuroimaging when the blind use sensory substitution devices, whether these neuronal changes confer superior abilities in the functional domain remains controversial. For example, in our laboratory, we have yet to fi nd a correlation between increased activation of the visual cortex and performance improvements on many diff erent types of outcomes measures (Lee et al., 2014). This fi nding is at odds with the generally accepted notion that the blind should be better able to discriminate auditory and tactile stimuli than their sighted counterparts. It is also inconsistent with experiments that suggest brain structure and function can predict performance with spatial hearing (Roder et al., 2002; Gougoux et al., 2005) and pitch discrimination (Voss and Zatorre, 2012; Voss et al., 2014). Future experiments with greater numbers of subjects followed for longer periods of time will be needed to shed light on this question, but are not likely to occur until sensory substitution devices are more widely available and are easier for subjects to use throughout the day.

In summary, sensory substitution devices for blindness should be considered a valuable method for studying plasticity of the central nervous system. In order for vision restoration eff orts to move forward, a better understanding of the brain changes as a result of vision loss is urgently needed. Neuroimaging in combination with sensory substitution devices off ers tremendous versatility to provide the answers needed when deciding on the appropriate candidates forvision restoration.

This work was supported by National Institutes of Health Contracts P30-EY008098 and T32-EY017271-06 (Bethesda, MD); United States Department of Defense DM090217 (Arlington, VA); Alcon Research Institute Young Investigator Grant (Fort Worth, TX); Eye and Ear Foundation (Pittsburgh, PA); Research to Prevent Blindness (New York, NY); Aging Institute Pilot Seed Grant, University of Pittsburgh (Pittsburgh, PA); and Postdoctoral Fellowship Program in Ocular Tissue Engineering and Regenerative Ophthalmology, Louis J. Fox Center for Vision Restoration, University of Pittsburgh and UPMC (Pittsburgh, PA). We thank all collaborators who contributed to our research papers upon which the present commentary is based.

Amy C. Nau*, Matthew C. Murphy, Kevin C. Chan*

UPMC Eye Center, Ophthalmology and Visual Science Research

Center, Department of Ophthalmology, School of Medicine, University of Pittsburgh, Pittsburgh, PA, USA (Nau AC, Murphy MC, Chan KC)

Department of Bioengineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA, USA (Chan KC)

McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, USA (Nau AC, Chan KC)

Louis J. Fox Center for Vision Restoration, University of Pittsburgh, Pittsburgh, PA, USA (Nau AC, Murphy MC, Chan KC)

Center for the Neural Basis of Cognition, University of Pittsburgh and Carnegie Mellon University, Pittsburgh, PA, USA (Chan KC)

*Correspondence to: Amy C. Nau, O.D. or Kevin C. Chan, Ph.D., anau@korbassociates.com or chuenwing.chan@fulbrightmail.org.

Accepted: 2015-08-15

orcid: 0000-0003-4012-7084 (Kevin C. Chan)

Abboud S, Hanassy S, Levy-Tzedek S, Maidenbaum S, Amedi A (2014) EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution. Restor Neurol Neurosci 32:247-257.

Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Nature 221:963-964.

Bedny M, Pascual-Leone A, Dravida S, Saxe R (2012) A sensitive period for language in the visual cortex: distinct patterns of plasticity in congenitally versus late blind adults. Brain Lang 122:162-170.

Chan KC, Cheng JS, Fan S, Zhou IY, Yang J, Wu EX (2012) In vivo evaluation of retinal and callosal projections in early postnatal development and plasticity using manganese-enhanced MRI and diff usion tensor imaging. Neuroimage 59:2274-2283.

Chan KC, Murphy MC, Fisher C, Kim SG, Schuman JS, Nau AC (2014a) Functional plasticity of the visual system in the blind during sensory substitution task and at rest. In: Investigative Ophthalmology and Visual Science 55:2163.

Chan KC, Fan SJ, Chan RW, Cheng JS, Zhou IY, Wu EX (2014b) In vivo visuotopic brain mapping with manganese-enhanced MRI and resting-state functional connectivity MRI. Neuroimage 90:235-245.

Chow AM, Zhou IY, Fan SJ, Chan KW, Chan KC, Wu EX (2011) Metabolic changes in visual cortex of neonatal monocular enucleated rat: a proton magnetic resonance spectroscopy study. Int J Dev Neurosci 29:25-30.

Collignon O, Dormal G, Albouy G, Vandewalle G, Voss P, Phillips C, Lepore F (2013) Impact of blindness onset on the functional organization and the connectivity of the occipital cortex. Brain 136:2769-2783.

Dietrich S, Hertrich I, Kumar V, Ackermann H (2015) Experience-related structural changes of degenerated occipital white matter in late-blind humans - a diff usion tensor imaging study. PLoS One 10:e0122863.

Gabilondo I, Martinez-Lapiscina EH, Martinez-Heras E, Fraga-Pumar E, Llufriu S, Ortiz S, Bullich S, Sepulveda M, Falcon C, Berenguer J, Saiz A, Sanchez-Dalmau B, Villoslada P (2014) Trans-synaptic axonal degeneration in the visual pathway in multiple sclerosis. Ann Neurol 75:98-107.

Garaci FG, Bolacchi F, Cerulli A, Melis M, Spano A, Cedrone C, Floris R, Simonetti G, Nucci C (2009) Optic nerve and optic radiation neurodegeneration in patients with glaucoma: in vivo analysis with 3-T diff usion-tensor MR imaging. Radiology 252:496-501.

Gougoux F, Zatorre RJ, Lassonde M, Voss P, Lepore F (2005) A functional neuroimaging study of sound localization: visual cortex activity predicts performance in early-blind individuals. PLoS Biol 3:e27.

Hairston WD, Hodges DA, Casanova R, Hayasaka S, Kraft R, Maldjian JA, Burdette JH (2008) Closing the mind’s eye: deactivation of visual cortex related to auditory task diffi culty. Neuroreport 19:151-154.

Ho LC, Wang B, Conner IP, van der Merwe Y, Bilonick RA, Kim SG, Wu EX, Sigal IA, Wollstein G, Schuman JS, Chan KC (2015) In vivo evaluation of white matter integrity and anterograde transport in visual system after excitotoxic retinal injury with multimodal MRI and OCT. Invest Ophthalmol Vis Sci 56:3788-3800.

Kawashima R, O’Sullivan BT, Roland PE (1995) Positron-emission tomography studies of cross-modality inhibition in selective attentional tasks: closing the “mind’s eye”. Proc Natl Acad Sci U S A 92:5969-5972.

Kupers R, Ptito M (2014) Compensatory plasticity and cross-modal reorganization following early visual deprivation. Neurosci Biobehav Rev 41:36-52. Laurienti PJ, Burdette JH, Wallace MT, Yen YF, Field AS, Stein BE (2002) Deactivation of sensory-specifi c cortex by cross-modal stimuli. J Cogn Neurosci 14:420-429.

Lee VK, Nau AC, Laymon C, Chan KC, Rosario BL, Fisher C (2014) Successful tactile based visual sensory substitution use functions independently of visual pathway integrity. Front Hum Neurosci 8:291.

Li J, Liu Y, Qin W, Jiang J, Qiu Z, Xu J, Yu C, Jiang T (2013) Age of onset of blindness aff ects brain anatomical networks constructed using diff usion tensor tractography. Cereb Cortex 23:542-551.

Maidenbaum S, Abboud S, Amedi A (2014a) Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation. Neurosci Biobehav Rev 41:3-15.

Maidenbaum S, Hanassy S, Abboud S, Buchs G, Chebat DR, Levy-Tzedek S, Amedi A (2014b) The “EyeCane”, a new electronic travel aid for the blind: Technology, behavior & swift learning. Restor Neurol Neurosci 32:813-824. Merabet LB, Pascual-Leone A (2010) Neural reorganization following sensory loss: the opportunity of change. Nat Rev Neurosci 11:44-52.

Murphy MC, Fisher C, Kim SG, Schuman JS, Nau AC, Chan KC (2014) Top down infl uence on the visual cortex of the blind during auditory sensory substitution. Proc Intl Soc Mag Reson Med 22:579.

Nau AC, Pintar C, Arnoldussen A, Fisher C (2015) Acquisition of visual perception in blind adults using the brainport artifi cial vision device. Am J Occup Ther 69:6901290010p6901290011-6901290018.

Ohno N, Murai H, Suzuki Y, Kiyosawa M, Tokumaru AM, Ishii K, Ohno-Matsui K (2015) Alteration of the optic radiations using diff usion-tensor MRI in patients with retinitis pigmentosa. Br J Ophthalmol 99:1051-1054.

Reich L, Maidenbaum S, Amedi A (2012) The brain as a fl exible task machine: implications for visual rehabilitation using noninvasive vs. invasive approaches. Curr Opin Neurol 25:86-95.

Reislev NL, Kupers R, Siebner HR, Ptito M, Dyrby TB (2015) Blindness alters the microstructure of the ventral but not the dorsal visual stream. Brain Struct Funct DOI:10.1007/s00429-015-1078-8.

Roder B, Stock O, Bien S, Neville H, Rosler F (2002) Speech processing activates visual cortex in congenitally blind humans. Eur J Neurosci 16:930-936.

Sadato N, Okada T, Honda M, Yonekura Y (2002) Critical period for cross-modal plasticity in blind humans: a functional MRI study. Neuroimage 16:389-400.

Schoth F, Burgel U, Dorsch R, Reinges MH, Krings T (2006) Diff usion tensor imaging in acquired blind humans. Neurosci Lett 398:178-182.

Voss P, Zatorre RJ (2012) Occipital cortical thickness predicts performance on pitch and musical tasks in blind individuals. Cereb Cortex 22:2455-2465.

Voss P, Pike BG, Zatorre RJ (2014) Evidence for both compensatory plastic and disuse atrophy-related neuroanatomical changes in the blind. Brain 137:1224-1240.

Weaver KE, Richards TL, Saenz M, Petropoulos H, Fine I (2013) Neurochemical changes within human early blind occipital cortex. Neuroscience 252:222-233.

Wu L, Tang Z, Sun X, Feng X, Qian W, Wang J, Jin L (2013) Metabolic changes in the visual cortex of binocular blindness macaque monkeys: a proton magnetic resonance spectroscopy study. PLoS One 8:e80073.

10.4103/1673-5374.169612 http://www.nrronline.org/

Nau AC, Murphy MC, Chan KC (2015) Use of sensory substitution devices as a model system for investigating cross-modal neuroplasticity in humans. Neural Regen Res 10(11):1717-1719.