1018 Full
1018 Full
Behavioral/Cognitive
Hemianopia (unilateral blindness), a common consequence of stroke and trauma to visual cortex, is a debilitating disorder
for which there are few treatments. Research in an animal model has suggested that visual-auditory stimulation therapy,
which exploits the multisensory architecture of the brain, may be effective in restoring visual sensitivity in hemianopia. It
was tested in two male human patients who were hemianopic for at least 8 months following a stroke. The patients were
repeatedly exposed to congruent visual-auditory stimuli within their blinded hemifield during 2 h sessions over several weeks.
The results were dramatic. Both recovered the ability to detect and describe visual stimuli throughout their formerly blind
field within a few weeks. They could also localize these stimuli, identify some of their features, and perceive multiple visual
stimuli simultaneously in both fields. These results indicate that the multisensory therapy is a rapid and effective method for
restoring visual function in hemianopia.
Key words: cortical blindness; cross-modal; hemianopsia; superior colliculus; training; visual-auditory
Significance Statement
Hemianopia (blindness on one side of space) is widely considered to be a permanent disorder. Here, we show that a simple
multisensory training paradigm can ameliorate this disorder in human patients.
to auditory and visual stimuli, and both show evidence of multi- strokes. While undergoing rehabilitation with PT and OT, he was
sensory plasticity (Giard and Peronnet, 1999; Foxe et al., 2000; noted to have visual impairment on the left. The patient was referred
Calvert, 2001; Calvert et al., 2001). Thus, we examined the possi- for a visual field test in August 2021, when a Humphrey visual field
bility that this multisensory rehabilitation paradigm would be test (Goldmann size 3) showed homonymous hemianopia on the left
with sparing of the bottom-left quadrant below 10°. He was seen in
effective in human stroke-induced hemianopic patients.
the neurology clinic for follow-up, recruited to participate in the
study, and consented.
Materials and Methods The results of the initial research testing at study onset, 8 months af-
All methods were conducted in compliance with an approved protocol ter the insult, matched the results of the earlier visual examination. CW
by the Institutional Review Board of the Wake Forest University School could not detect 500 ms flashes of light (;1000 lm) anywhere in the
of Medicine (IRB00074687) and a registered clinical trial on the clinical- upper-left quadrant of space (flash test) but could detect flashes in the
trials.gov website (NCT04963075). left inferior space below 10° of eye level and everywhere in right space.
Patients. The inclusion criteria for this study were patients with (1) Tests with stationary and moving objects produced similar results, total
homonymous hemianopia in the absence of hemineglect and as a result lack of any visual perception in the upper-left quadrant but intact detec-
of an ischemic or hemorrhagic stroke, who had been stable for at least tion, localization, and identification of visual stimuli everywhere else in
6 months to avoid confounding by spontaneous recovery soon after the space. Based on these findings, the upper-left quadrant was identified as
insult; (2) a lesion encompassing at least primary visual cortex but spar- the targeted blind field.
ing parietal cortex; (3) normal auditory and cognitive function (the para- Scheduling. The patients committed to 10 sessions of training and
digm required hearing and the ability to follow commands); (4) the testing, each lasting a maximum of 2 h. A session was terminated before
willingness to participate in the program; and the (5) ability to perform the 2 h limit when the patient reported eye strain or continued difficulty
visual discriminations in their intact field. in maintaining fixation. Sessions began near the beginning of October
Two patients who met these criteria were referred from the Department 2021 and continued on an approximately weekly basis until January
of Neurology by C.B., a vascular neurologist at Atrium Health Wake Forest 2022 (14 weeks). In each session, the patient was led into the experimen-
Baptist. Both patients had undergone formal visual field testing with the tal room at the Wake Forest University School of Medicine Clinical
Humphrey visual field test (Goldmann size 3) at least 3 months before pro- Research Unit (CRU) and seated in a comfortable chair facing the appa-
viding their consent. ratus. The patients were briefly interviewed and briefed by the investiga-
JM is a 64-year-old dextral male who suffered an earlier right poste- tor (B.A.R.) before interleaved blocks of visual testing and multisensory
rior cerebral artery (PCA) infarct (August 2020) followed by a left PCA training. Each session concluded with a debriefing.
infarct 2 d later. A magnetic resonance imaging (MRI) scan without gad- Training/testing room. Three rooms of similar size and layout within
olinium (October 2020) confirmed bilateral occipital lobe infarct the CRU were used (selection dependent on availability). Each was
with hemorrhagic conversion with subsequent reduction in vaso- ;8 20 meters and contained a patient bed, chairs, and tables, with
genic edema on the left. JM was referred for rehabilitation with ample room for the apparatus described below. Shades over the windows
physical therapy (PT) and occupational therapy (OT). He under- in each room produced a dimly lit environment (;10 lm), and each had
went outpatient follow-up of visual fields with Humphrey visual field low ambient background noise (;41 dB).
test (Goldmann size 3) in August 2021, which showed persistent left Apparatus. The apparatus (Fig. 1) consists of visual-auditory stimu-
homonymous hemianopia with a small right inferior quadrantanopia. lation devices affixed to the top of movable towers. Each tower was a
The patient consented and was enrolled in the study in October 2021, height-adjustable desktop stand (YIFU B07KX29CB7) with a 5.9 5.11
14 months after the initial stroke. inch base placed on a table as shown in Figure 1. The stimulation device
On an initial examination by the research team, patient navigation on top of each tower consisted of a wide-frequency speaker (Panasonic
into the room was highly compromised, and he required assistance in 4D02C0) and a 3 6 array (;1 inch wide 3 inches high) of white light
being guided into the chair because of his visual impairment. He emitting diode (LED) strips (Dephen DEP-SNW320F008W05-COB)
reported complete absence of stationary visual perception on the left attached to it. The assembly connected via a flexible adapter with a
side of space starting at midline and also in the far-right periphery, custom-printed circuit board (PCB; OSH Park) to an Arduino
with significant impacts on quality of life, particularly with navigat- Mega 2560 Rev3 microcontroller modified to run at a clock fre-
ing complex environments because of what he described as “tunnel quency of ;31 kHz. The PCB routed output ports and grounds of
vision.” He also reported subjective difficulty in thinking clearly and the Arduino to connectors for each of the towers. The Arduino was
with short-term memory, which he attributed to the effort required connected to a laptop via a USB, with which it communicated via a
to process visual scenes. virtual serial port. Custom firmware on the Arduino and software
The results of initial research testing matched the results of the earlier on the computer controlled stimulus delivery. The LED/speaker
clinical examination; the patient could not detect 500 ms flashes of light assembly atop a single (inactive) tower placed 2 m in front of the
(;1000 lm) anywhere to the left of fixation (flash test). He could, how- patient served as the fixation point. A camera (Zuodon) on tripod
ever, detect such flashes everywhere to the right of fixation, although recorded each session. Data from the camera were used post hoc to
detectability was not perfect beyond 45° of eccentricity. There was a total confirm the real-time assessments of patient visual fixation that were
lack of visual perception for stationary or moving stimuli to the left of made by the researcher. A movable LCD monitor (Acer S241HL) was
fixation and compromised visual perception in the far-right periphery. used to present virtual visual stimuli. All stimuli were presented 2 m
In right space at eccentricities ,45°, detection, localization, and identifi- from the patient’s head, at eccentricities between 90° (left) and
cation of visual stimuli were fully intact. Based on these findings, the left 190° (right) of fixation, and at elevations 30° (below) and 130°
hemifield was identified as the targeted blind field. (above) eye level.
CW is a 72-year-old dextral male who suffered a right PCA infarct Rehabilitation procedure. As in previous studies in animals (Jiang et
along with multiple scattered foci of infarction involving the cerebellum al., 2015, 2020; Dakos et al., 2019, 2020), the rehabilitation procedure
and bilateral supratentorial cerebral hemispheres following ST-segment involved repeatedly presenting identical visual-auditory stimuli at a cho-
elevation myocardial infarction and cardiac arrest in February 2021. sen location in the hemianopic field (initially at 45° of eccentricity) while
Cardiology diagnostic evaluation revealed severe diffuse coronary dis- the patient maintained fixation on the top of the central (0°, 0°) tower.
ease without an obvious culprit lesion. CT surgery was consulted to Fixation was monitored by the researcher. The visual stimulus consisted
echodensity within the pericardium adjacent to the right ventricle, of a brief (500 ms) and bright flash of an LED complex (1000 lm). The
and CW underwent subxiphoid pericardial window and evacuation auditory stimulus was in spatial and temporal congruence with the LEDs
of pericardial hematoma. At the time of discharge from the hospital and consisted of a brief (500 ms) broadband noise burst (68 dB SPL)
to inpatient rehabilitation, he was noted to have impaired mobility emitted from the speaker. Most rehabilitative training sessions contained
and activities in daily living, and cognitive deficits from the multifocal 600 of these cross-modal stimulus trials in blocks of 100–150 trials at
1020 • J. Neurosci., February 8, 2023 • 43(6):1018–1026 Rowland et al. · Ameliorating Hemianopia with Multisensory Training
Figure 1. Top: The apparatus used in multisensory training and flash detection/localization. LED and speaker assemblies were mounted on top of a set of movable and height-adjustable
towers (7 depicted). During training, visual-auditory stimuli were repeatedly presented at a location within the blind field. During testing, patients were asked to verbally report and/or point
to the location of a flashed stimulus. Middle and Bottom: MRI scans and initial testing for both patients. In both patients, a right PCA infarct created blindness on the left side of space.
Unshaded areas in the plots (Az = azimuth, El = elevation) indicate areas responsive to light flashes, black shaded regions indicate where those flashes could not be detected. Note that CW
had vision spared in left space below 10° of elevation.
interstimulus intervals of 2–6 s. These rehabilitative training stimuli lights, two bright (;2000 lm) nonflashing light sources were placed
were interleaved with fixation breaks and visual probe tests. at different disparities from one another (from 0 to 30°) and at dif-
Tests of visual function. All visual tests were performed binocularly ferent positions in the left and right hemifields. These were activated
while the patient maintained fixation, following as closely as possible the individually or together, and patients were asked to report how
paradigm developed in the cat model (Jiang et al., 2015, 2020; Dakos et many (and which) they perceived. In some testing sessions patients
al., 2019, 2020). As in the animal model, testing here included multiple wore goggles in which the central half of the left eye and peripheral
sites in both visual hemifields. half of the right eye were occluded to effectively block all visual
The principal quantitative visual test was a standard flash test in input from the right (intact) hemifield. These were used to eliminate
which the patient responded as fast as possible with a verbal report and the possibility that this input could contaminate tests in the left
pointing response to the onset of a brief (500 ms) and high-intensity field; however, they did not appear to affect test results.
(;1000 lm) flashed LED complex that was unpredictable in space and Experimental design and statistical analysis. The hypotheses guiding
time (Fig. 1). In some tests patients were also asked to report the nature this study were qualitative; subjects would either regain the ability to
of the resultant visual percept (i.e., the perceived intensity, shape, and detect and localize visual stimuli in the blinded hemifield, or they would
color of the stimulus). Interleaved with stimulus-containing trials were not. Before the training paradigm, they could not in the identified areas
occassional trials in which no stimulus was presented, but patients were of contralesional space. Binomial tests were used to evaluate the signifi-
prompted for a response. Additional qualitative tests of visual capability cance of the probabilities with which they could detect and localize stim-
were added as rehabilitation proceeded to determine whether capabilities uli after the training paradigm.
beyond the simple detection and conscious perception of flashed lights
had emerged. For example, to determine whether small discrepancies in
stimulus location could be accurately detected in the rehabilitated hemi- Results
field, the patient was asked to discriminate which of three closely spaced Pre-rehabilitation visual detection capability
(3° apart) identical LED complexes were flashed in central (centered at Baseline visual capabilities were established in the first session.
18°) or peripheral (centered at 48°) space. Additional qualitative tests The standard flashed visual stimulus was presented at multiple
probed the patients’ perception of real 3-D dimensional objects (e.g., cell eccentricities and elevations in left (-) and right (1) visual
phone, wallet, plastic toys, pliers, roll of tape, soap dispenser, water bot- space while central fixation was maintained. The patient was
tle, glasses, experimenter’s hands/fingers) or virtual geometric shapes instructed to verbally acknowledge when he detected a stimu-
(triangle, square, rectangle, parallelogram, etc.) when they were station-
lus and to point to its perceived location while maintaining
ary, moved, or (for virtual stimuli) flashed at different locations. Real
objects were first introduced in regions in which flashed visual stim- fixation. The results were consistent with the visual defects
uli were ineffective and then moved to regions in which flashed evident in clinical examination, the lesions observed with MRI
stimuli were detected. Patients were asked to identify and describe (Fig. 1), and the patients’ own reports.
the features of the object/shape and describe its motion if it was Both patients had left visual field defects. JM could not
moving. To assess the ability to simultaneously perceive multiple detect flashes anywhere left of fixation but could detect them
Rowland et al. · Ameliorating Hemianopia with Multisensory Training J. Neurosci., February 8, 2023 • 43(6):1018–1026 • 1021
Figure 2. Recovery of visual detection and localization. Depicted are each patients’ ability to detect the flashed stimulus at the beginning of select sessions in the experiment. Unshaded
regions indicate areas where responses were elicited by the flash stimulus and black shaded regions indicate where they were not elicited. The icon (a flash in a circle) indicates the location of
the visual-auditory training stimulus in the previous session. Dashed and solid circles (connected by arrows) in the last of JM’s figures (see session 6) represent his translocation of visual stimuli
in the far periphery of the rehabilitated field, which were systematically biased to a compressed, more central range.
everywhere to the right. CW could not detect flashes above 10° of occurrence and pointing to them while maintaining fixation.
elevation on the left but could detect them below this level and They never reported seeing a stimulus when one had not been
everywhere on the right (Fig. 1). Neither patient could detect sta- presented and, when asked “Did you see anything?” they always
tionary or moving stimuli where they could not detect flashes: They answered no.
were totally blind to all presented visual stimuli in these regions. Rapid improvements in visual detection and localization were
JM had also had a visual defect in his intact (right) hemifield. observed with training, initially for stimuli at the border of the
He could not detect stationary or moving stimuli in the inferior sighted field, followed by a progressive expansion of the effective
(below 0°) periphery (greater than 160°), although flashes could visual field that ultimately extended beyond the training stimulus
be detected there. This region was not subjected to rehabilitative location.
training and served as a within-subject control. JM first showed visual improvement during the pretrain-
ing evaluation period of session 3 (Fig. 2). He could for the
Rehabilitation of visual detection and localization first time reliably (5/5) detect and point to a visual stimulus
The primary assessment in tracking visual rehabilitation at 4° of eccentricity, but not to more eccentric stimuli. Post-
during weekly sessions was the ability to detect and localize training session tests revealed that his visual field expanded
flashed LEDs, a sensitive measure of visual recovery (Dakos to 8° of eccentricity. He was aware that his detection capabil-
et al., 2019, 2020). Using this measure, rapid training-induced ity had improved and commented on this. The expansion was
visual improvements were observed in both patients. The most retained at session 4 pretraining tests. At the midpoint of this
impressive changes occurred within the first 4–5 weekly sessions. session (after 300 visual-auditory exposures), visual probe tests
This is in keeping with the timeline for rehabilitation in the ani- revealed that he could now detect and point to a visual stimulus
mal model (Jiang et al., 2015, 2020; Dakos et al., 2020; Stein and at 20° of eccentricity (5/5). Once again, this visual field expan-
Rowland, 2020). As in the animal model, both patients showed sion was confirmed in postsession tests. At this point, JM could
little tendency to lose gains made over the weekly interlude also accurately discriminate which of three adjacent lights (3°
between training sessions. separation) were flashed in central space (78/128 = 61% correct
In each rehabilitative training session, patients received vs chance = 33%, binomial test, p , 3E-11). In pretraining tests
600 exposures to a spatiotemporally concordant visual-au- at the beginning of session 5 it became apparent that his visible
ditory stimulus presented well within the blind region (JM, field had again expanded, in this case without additional train-
45, 0°; CW, 45, 120°). The higher elevation of CW’s ing. He could now detect and point to flashes out to 40° at all
training stimulus ensured it did not encroach on regions of tested elevations (630°). During the midpoint break of this ses-
spared vision. Both patients were highly motivated and eagerly sion, probe tests revealed that his visual field now extended to
engaged in training and testing. Patients were noted to break 90°. In post-training tests he correctly detected the visual
fixation 3–4 times/session, typically toward the end of the 2 h stimulus at every location tested from 0 to 90° at all elevations
period. In all such instances the trial was aborted, and training tested (630°) with 100% reliability.
or testing began again when the patient had reacquired fixation. However, his ability to accurately localize stimuli beyond 45°
In each visual test trial, patients responded to detected stimuli of eccentricity was significantly compromised and would remain so.
with little delay and had no difficulty verbally reporting their Stimuli at eccentricities of 45 and 90° were all localized to
1022 • J. Neurosci., February 8, 2023 • 43(6):1018–1026 Rowland et al. · Ameliorating Hemianopia with Multisensory Training
Figure 3. Visuospatial extinction: before and after. Visual extinction was evaluated by simultaneously presented two lights within the same hemifield or in different hemifields. Depicted
above are the regions where a bright light could or could not be detected in the left hemifield when an equally-bright light was simultaneously presented in the right hemifield (depicted with
a light bulb). Different plots illustrate the observations before (blind regions shown with black shading) and after rehabilitation. Areas of space in which both lights could be perceived are
unshaded. Areas of space after rehabilitation in which individual lights could be detected, but where detection was suppressed by the simultaneous presence of a light in the right hemifield,
are shaded in gray.
approximately 45°, indicating a compression of the perceptual blind spot was eventually eliminated (session 10), as was the
field. Consistent with this observation, he never accurately discrimi- tendency for systematic mislocalizations in elevation; CW now
nated the locations of the three adjacent lights (3° separation) when accurately pointed to flashes everywhere in space. Whether addi-
individually flashed in peripheral space (6/20 = 30% correct vs tional training at the original stimulation site would have also
chance = 33%, binomial test, p = 0.51). He could, however, infer resolved the blind spot is unknown; however, it had not changed
that their locations were different based on differences in their per- in the six sessions preceding the change of the training location.
ceived intensity (see below). He was unaware that these percepts
were anomalous, and these were not observed in the seeing portion Additional observations
of the opposite hemifield (his visual perception in the anomalous Visuospatial extinction
region on the right was unchanged). The ability to perceive multiple visual objects was assessed using
CW showed even more rapid training-induced visual improve- two high-intensity (;2000 lm) and continuously illuminated
ments. These were apparent at the end of session 1 when visual light sources. These light sources were either presented individu-
stimuli were detected on 50% (5/10) of the trials at ( 45°, 0°). He ally or together at different spatial separations and locations.
was even more reliable (10/10 = 100%) at elevations below 0° (but Both patients easily detected and reported the presence of both
was unable to detect stimuli (0/10 = 0%) at elevations above 0°). lights in the right (normal) hemifield. However, the perception
By the midpoint of session 2, after 300 visual-auditory exposures, of a light in the recovered hemifield could be altered by the pres-
visual stimuli were reliably detected and localized on 100% (10/10) ence of a simultaneous competing light placed within either
of the trials at the training site ( 45, 120°; Fig. 2). Post-training hemifield (Fig. 3).
tests revealed that the visual field had expanded significantly, now This was evident in JM as soon as he detected lights on the
extending from 30 to 70° of eccentricity at elevations of 0 to left (previously blind) side of space during rehabilitative training.
120°. This improvement in contralesional visual performance The light in the recovering hemifield would disappear and reap-
was retained and evident at the beginning of session 3. He could pear when a second light was moved into and out of the intact
now reliably (21/36 = 58%, chance = 33%, binomial test, p , 5E- hemifield (he reported that the first light appeared to be “turned
4) discriminate the individual locations of three adjacent (3° sepa- off”). When two lights were presented in the left visual hemifield,
ration) lights at 48, 120° in peripheral space. This localization even when separated by up to 30°, he only perceived the more
accuracy was retained in the next session (40/69 = 58%, binomial central of them. This central stimulus appeared brighter when
test, p , 7E-6) and improved once again in session 5 (38/44 = the second light was moved closer to it. These instances of
86%, binomial test, p , 3E-14). By session 6, visual detection and competitive interactions were resolved over the central por-
localization extended to 90° of eccentricity and to 130° of eleva- tion of the hemifield during the rehabilitative period so that
tion (Fig. 2). But a blind region in upper central visual space by the final session, lights central of 30° were no longer
remained and was described by him as a “blind spot.” This region extinguished by a simultaneous light in the right or left hemi-
was roughly triangular, defined by three points: (0°, 0°), (0°, field (i.e., two lights were perceived).
130°), and ( 30°, 130°; Fig. 2). CW had no such problem. He could perceive two lights
To examine whether training within this residual blind spot within his recovered visual hemifield (as it expanded) regardless
would ameliorate this residual deficit, an additional 450 training of their spatial separation. The presence of a light in the intact
trials were conducted at ( 5°, 110°) in session 6. The training hemifield diminished the apparent intensity of one in the reha-
stimulus was very near the border of the blind spot. The strategy bilitated hemifield but did not extinguish it. The exception to
was effective and raised the lower border of the blind area by 5°. this was his previous blind spot, where perceived lights were
With additional training in session 7, the lower border reached extinguished by a light in the opposite hemifield. This defect did
110° of elevation. Although his localization of flashes in this not resolve by the end of the testing period.
area of restored vision were accurate in azimuth, they were sys-
tematically mislocalized in elevation, always appearing ;10° Perception of intensity, movement, and stationary objects
above their actual location. The training stimulus was moved Interspersed with the above quantitative evaluations were
once again to an equivalent location within the contracted blind requests for the patients to describe the intensity and color of
spot, in this case to ( 10°, 110°). With additional training, the the flashed lights and requests for them to report and describe
Rowland et al. · Ameliorating Hemianopia with Multisensory Training J. Neurosci., February 8, 2023 • 43(6):1018–1026 • 1023
the features (e.g., color, shape, size, identity) of a variety of objects or lights, high intensity flashes, etc.). Over several 2 h
real objects, as well as virtual 2-D objects illuminated on an weekly training/testing sessions they progressively regained the
LCD monitor. ability to perceive light flashes throughout their previously blind
Within and across sessions, JM reported that the flash stimu- regions. They also regained some ability to detect and describe
lus was becoming “brighter” and “sharper” and described a moving and stationary objects, and both reported significant
change in his perception of its color from yellow to white (which improvements in their quality of life and activities of daily living.
was correct). He accurately reported when the flashes changed The trajectory of visual recovery was similar to that observed
intensity and could also detect large (.5°) and flashed or station- in the cat model of hemianopia that guided the rehabilitative
ary illuminated gray 2-D shapes wherever he could detect flashes. training paradigm used here (Jiang et al., 2015, 2020; Dakos et
This capability eventually extended over the entire left hemifield. al., 2019, 2020; Stein and Rowland, 2020). In this model, a stand-
However, all visual stimuli beyond 10° of eccentricity were per- ard daily (five/week, 600 trials/session) multisensory training
ceived as large “blobs.” In central space, moving and stationary session was conducted with invariant and spatiotemporally con-
real objects could be detected and identified at 4° in session 3 gruent visual-auditory cues centered in the blind hemifield of the
and 7° in session 6. Within these areas he was able to report animal. This restored its ability to detect and localize visual stim-
the color, shape, and contour (often its identity) and general fea- uli throughout that hemifield in 2–4 weeks. Recovery was also
tures of real and virtual objects. However, object perception was obtained in anesthetized hemianopic animals given less frequent
consistently described as “blurry” compared with that of identical sessions (one/week) and between 100–2400 visual-auditory tri-
stimuli in the intact hemifield. als/session (10 min–4 h; Jiang et al., 2020). Greater numbers of
CW’s recovery was more impressive for both real and virtual trials were associated with more rapid recovery but with
stimuli. He accurately characterized the intensity and color of diminishing returns; 100 trials/session produced recovery in
detected flashes. He also accurately described stationary and 8 weeks, whereas 600 and 2400 trials/session produced recov-
moving objects wherever he could detect flashes. This rapid ery in 4 weeks. The training paradigm for human patients
improvement in his visual abilities was readily apparent to him, was similarly sparse (one/week, 600 trials/session), and re-
and he commented on it spontaneously. He could (after a few covery (4–6 weeks) was similarly short.
seconds) identify novel objects on the left by session 5. When An important earlier study also found salutary effects in hem-
queried about this delay, he reported making a conscious infer- ianopic patients presented with interleaved visual, auditory, and
ence about the identity of the object based on its detected fea- visual-auditory stimuli varying in location and timing (Bolognini
tures such as angles and/or contours. In his intact field, reports et al., 2005). In that study, patients became efficient at making
of shapes were more immediate, and he was unaware of any saccades to visual stimuli in the blind hemifield, but when required
inferential process preceding his report. The retention of a to maintain central fixation, they had no visual awareness in that
blind spot in upper-left central space provided an interesting hemifield. Their recovery was therefore interpreted as strictly visuo-
discontinuity in his stationary visual perception. He noted that motor in nature. This contrasts with the current findings, which
objects with straight lines (e.g., handrails) crossing the blind yielded both motor and perceptual recovery. The difference likely
region were not perceived as continuous, although he knew exists in the stimulation paradigm adopted which, in the pres-
they must be. ent case, strictly adhered to the paradigm developed in the
animal model. During rehabilitative training, patients were
Quality of life changes presented only with visual-auditory pairs that were invariant
Both patients reported significant quality of life improvements in their congruent spatiotemporal relationship. It is possible
that they attributed to the restoration of visual capabilities. Both that restoring visuomotor processing is dependent on looser
reported that their perception of eye strain outside the session constraints than those leading to visual awareness and may
improved within the first 3–4 sessions. After four sessions both precede its emergence. If engaged in the present paradigm, it
noted a new awareness of high contrast visual stimuli in their might have even speeded recovery. That would be consistent
rehabilitated hemifield; for example, streetlights or lights in with findings that some visual recovery can be obtained in
houses that were now perceptible. JM noted, as did his wife, that hemianopic patients trained to make progressively larger sac-
cades into the hemianopic field (Zihl, 1995; Nelles et al., 2001)
his navigation had markedly improved between the first and
or to saccade to moving targets there (Dundon et al., 2015;
third sessions, and he had a decreased sense of “tunnel vision.”
Frolov et al., 2017; Sahraie et al., 2020; Szalados et al., 2021).
At home he was able to detect illuminated lamps on his rehabili-
Other recent work has shown some recovery on visual
tated side and use them to navigate. CW reported improvements
responsiveness in hemianopic patients using visual-auditory
in reading in sessions 4 and 5 so that he no longer depending on
training in a virtual reality (VR) headset (Daibert-Nido et
using his finger to keep place. He also noted a renewed percep-
al., 2021). VR devices can be convenient therapeutic tools,
tion of leaves on trees to his left during walks and great improve-
and the patients in these studies reported some improve-
ment in navigating root-laden paths when hiking, a favorite
ments in the detectability of visual stimuli in formerly blind
hobby. He now hiked with greater confidence and at speeds
regions as well as improvements in quality of life. In another
equal to those before the stroke. These were twice his hiking
recent study, intensive visual discrimination training also
speed at the beginning of the study.
proved to have some positive visual effects in the hemia-
nopic field (Ajina et al., 2021). Other work in hemianopic
Discussion nonhuman primates has also established some recovery after
The present study demonstrated that a multisensory rehabilita- extensive visual training (Yoshida et al., 2008, 2017; Kato et
tive technique for hemianopia first developed in an animal al., 2011, 2021; Takakuwa et al., 2017; Kinoshita et al., 2019;
model can be successfully applied to human patients. Both Isa and Yoshida, 2021). However, it is notable that these dif-
patients in this study initially failed to detect any visual test stim- ferent studies used different tests for visual function evaluation
uli presented in the affected field (i.e., stationary or moving and subject populations with different lesions and different
1024 • J. Neurosci., February 8, 2023 • 43(6):1018–1026 Rowland et al. · Ameliorating Hemianopia with Multisensory Training
preintervention visual capabilities. (Here, as in the cat model, recovery than was previously available. Both regained conscious
they were totally blind to flashes in the affected field.) visual perception and the ability to describe the intensity and
It is clear from prior work in the animal model that not only color of flashes and point to them without shifting gaze.
is the multisensory nature of the stimuli presented necessary for Furthermore, neither could detect moving low-contrast stim-
the present rehabilitative paradigm to be effective (Jiang et al., uli or identify them at a given site without previously being
2015) but also their spatiotemporal congruence (Dakos et al., able to detect flashes at that site. It could be that the ability to
2020). These are the same factors that govern the mechanisms of detect a light flash is a more rudimentary capability whose re-
multisensory plasticity by which SC neurons are sensitized to covery necessarily precedes recovery of other abilities.
their visual inputs (Yu et al., 2013) and may help explain how Despite these similarities between patients, there were also
these neurons, which lose visual sensitivity after a hemianopia- substantial individual differences, especially in the visual capabil-
inducing lesion, regain this sensitivity and do so in parallel with ities observed at the end of rehabilitative training. Whereas CW
the restoration of overt visual behavior. The SC is believed to be could accurately localize flashes and describe the shape and
the nexus of the residual neural circuit through which recovery is direction of motion of visual stimuli, JM perceived peripheral
achieved, and its neurons are particularly well positioned for a visual stimuli as diffuse light sources and had a persistent defect
role in controlling sensorimotor recovery (Stein and Rowland, in localizing them. CW also had no difficulty in detecting multi-
2020). They send efferents to both brainstem and spinal tar- ple stimuli, whereas JM’s peripheral visual perception was readily
gets involved in visuomotor responses (Graham, 1977; Stein challenged when presenting a second visual stimulus elsewhere.
et al., 1982, 1984; Meredith and Stein, 1985; Moschovakis It is notable, however, that JM’s ability to perceive multiple si-
and Karabelas, 1985; Bruce and Stein, 1988; Sparks and Hartwich- multaneous lights continued to improve, and the endpoint of
Young, 1989; Paré et al., 1994; Burnett et al., 2004) and (via tha- this recovery is not clear. Ultimately, rehabilitated cats had no
lamic connections) to sensory processing cortical areas for vis- difficulty in making choices among multiple simultaneous lights
ual perception (Graham, 1977; Benedek et al., 1997; McHaffie (Dakos et al., 2019).
et al., 2005). The specific subtypes of tectopetal and tectofugal The specific factors accounting for the variability in the recov-
neurons engaged in this process and their neurobiological dy- ery of CW and JM are not easy to identify. Both patients were
namics remain to be identified. highly motivated, of similar age and time since their stroke, and
Their overlapping visual and auditory topographies also underwent the same rehabilitative training paradigm. However,
appear to be reflected in the pattern of visual recovery, which, their lesions and the initial topography of their visual deficits dif-
in both animals and humans, expands progressively from the fered. CW presented with a larger sighted visual field and a more
margin of the sighted hemifield to beyond the training location, impressive recovery, perhaps reflecting less extensive functional
ultimately encompassing the entire hemifield. Recovery does damage to critical components of the residual visual circuit (e.g.,
not begin at the stimulation location, which would be the naive certain tectopetal afferents from association cortex must be
expectation for a process that engaged a Hebbian mechanism. spared for recovery to occur; Jiang et al., 2015). JM had a less im-
At present it is not known why recovery starts from the margin pressive recovery and had suffered a second PCA affecting the
of the sighted field, and why it seems to expand in broad pat- opposite cortex. Although the visual defects associated with this
tern. It is plausible that the pattern is dependent on cooperative left-side lesion appeared minor and restricted to the far periph-
dynamics between adjacent neurons in a topographic map; for ery, it may have interfered with any facilitation of recovery it
example, along the anterior–posterior axis of the SC (represent- provided to CW. Relating lesions to the pattern of recovery is a
ing central-peripheral visual space), but this is speculative at general problem in assessing the effectiveness of any rehabilita-
present. Furthermore, it is quite possible that other circuits in tive strategy in stroke patients, underscoring the importance of
the midbrain, thalamus, and/or cortex contribute to this pro- large population studies in which patients can be categorized
cess (Baleydier, 1977; Benedek et al., 1997, 2019; Jiang et al., properly into subgroups, and treatment paradigms can be varied.
2003; McHaffie et al., 2005; Nagy et al., 2011). These possibil- Of particular interest in the present context is how the specifics
ities remain to be explored. of the rehabilitative strategy (time, number, and patterns of mul-
It is also important to note that the hemianopia in these tisensory exposures, etc.) affect recovery among different groups
patients were a consequence of stroke, rather than a direct lesion (see below).
as in the referential animal studies. This makes possible the spar- Most significant to the present patients were improvements
ing of tissue within the damaged areas of visual cortex in which in their quality of life and activities of daily living. They noted a
visual-auditory interactions may still take place (Foxe et al., 2000; reduction in what they described as “eye strain” and improve-
Foxe and Schroeder, 2005; Schroeder and Foxe, 2005; Sperdin et ments in reading, navigation, and detecting moving objects (e.g.,
al., 2009; Fiebelkorn et al., 2011). Results from the animal model falling leaves, streetlights, etc.). These benefits were noted as
have suggested that certain higher-order areas are crucial to soon as visual detection was possible but continued to improve.
recovery (Jiang et al., 2015) but that residual regions of visual The ability to detect visual events in the previously blind hemi-
cortex are not. However, they may facilitate the rehabilitative field also allowed the patients to engage a variety of strategies to
process and make possible far greater visual capabilities than compensate for their impaired vision (e.g., orienting to bring vis-
could otherwise be achieved. Whether patients with more focal ual events into the fully sighted field and/or depending on other
scotomas resulting from different lesions could be rehabilitated senses for object identification) and to gain greater confidence in
using a similar technique, whether the plasticity engaged here everyday tasks.
might degrade with time, and whether early interventions might Given the present results, it may seem surprising that hem-
yield a faster or more robust recovery are unknown at present. A ianopia is normally such a persistent disorder. Spontaneous
larger, more comprehensive study is required to answer these recovery appears limited to a short window following a stroke
questions. (,6 months; Zhang et al., 2006) despite continued exposure
By verbally reporting and describing their visual perception, to a host of cross-modal events whose number and frequency
the patients provided more detailed information about visual in the normal environment will ultimately far exceed those
Rowland et al. · Ameliorating Hemianopia with Multisensory Training J. Neurosci., February 8, 2023 • 43(6):1018–1026 • 1025
provided in the training paradigm. Based on the current and Dundon NM, Bertini C, Làdavas E, Sabel BA, Gall C (2015) Visual rehabilita-
prior studies, the answer is likely to involve the consistency, tion: visual scanning, multisensory stimulation and vision restoration
regularity, and simplicity of the training paradigm stimuli, as trainings. Front Behav Neurosci 9:192.
well as the absence of contravening sensory experience. Yet, Fiebelkorn IC, Foxe JJ, Butler JS, Molholm S (2011) Auditory facilitation of
visual-target detection persists regardless of retinal eccentricity and de-
despite the rapidity and extent of visual recovery induced by spite wide audiovisual misalignments. Exp Brain Res 213:167–174.
the current training paradigm, it is unlikely to have optimized Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory conver-
all its features. Matching the types of stimuli presented, their gence during early cortical processing. Neuroreport 16:419–423.
iterative rate, the number of trials per session, the responses Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE
required, and/or a variety of other paradigm specifics to the (2000) Multisensory auditory-somatosensory interactions in early cortical
lesion and visual defect will likely hasten the speed and extent processing revealed by high-density electrical mapping. Brain Res Cogn
of recovery. Thus, the present results may best be viewed as Brain Res 10:77–83.
Frolov A, Feuerstein J, Subramanian PS (2017) Homonymous hemianopia
proof of concept that this paradigm is a highly effective thera-
and vision restoration Therapy. Neurol Clin 35:29–43.
peutic strategy. Giard MH, Peronnet F (1999) Auditory-visual integration during multimo-
dal object recognition in humans: a behavioral and electrophysiological
References study. J Cogn Neurosci 11:473–490.
Ajina S, Jünemann K, Sahraie A, Bridge H (2021) Increased visual sensitivity Goodwin D (2014) Homonymous hemianopia: challenges and solutions.
and occipital activity in patients with hemianopia following vision reha- Clin Ophthalmol 8:1919–1927.
bilitation. J Neurosci 41:5994–6005. Graham J (1977) An autoradiographic study of the efferent connections of
Baleydier C (1977) A bilateral cortical projection to the superior colliculus in the superior colliculus in the cat. J Comp Neurol 173:629–654.
the cat. Neurosci Lett 4:9–14. Hakon J, Quattromani MJ, Sjölund C, Tomasevic G, Carey L, Lee J-M,
Benedek G, Perény J, Kovács G, Fischer-Szátmári L, Katoh YY (1997) Visual, Ruscher K, Wieloch T, Bauer AQ (2018) Multisensory stimulation
somatosensory, auditory and nociceptive modality properties in the feline improves functional recovery and resting-state functional connectivity in
suprageniculate nucleus. Neuroscience 78:179–189. the mouse brain after stroke. Neuroimage Clin 17:717–730.
Benedek G, Keri S, Nagy A, Braunitzer G, Norita M (2019) A multimodal Isa T, Yoshida M (2021) Neural mechanism of blindsight in a macaque
pathway including the basal ganglia in the feline brain. Physiol Int model. Neuroscience 469:138–161.
106:95–113. Jiang H, Stein BE, McHaffie JG (2003) Opposing basal ganglia processes
Bi G, Poo M (2001) Synaptic modification by correlated activity: Hebb’s pos- shape midbrain visuomotor activity bilaterally. Nature 423:982–986.
tulate revisited. Annu Rev Neurosci 24:139–166. Jiang H, Stein BE, McHaffie JG (2015) Multisensory training reverses mid-
Blake R (1979) The visual system of the cat. Percept Psychophys 26:423–448. brain lesion-induced changes and ameliorates haemianopia. Nat
Blake R (1988) Cat spatial vision. Trends Neurosci 11:78–83. Commun 6:7263.
Bolognini N, Rasi F, Coccia M, Làdavas E (2005) Visual search improvement Jiang H, Rowland BA, Stein BE (2020) Reversing hemianopia by multisen-
in hemianopic patients after audio-visual stimulation. Brain 128:2830– sory training under anesthesia. Front Syst Neurosci 14:4.
2842. Kato R, Takaura K, Ikeda T, Yoshida M, Isa T (2011) Contribution of the ret-
Bolognini N, Convento S, Rossetti A, Merabet LB (2013) Multisensory proc- ino-tectal pathway to visually guided saccades after lesion of the primary
essing after a brain damage: clues on post-injury crossmodal plasticity visual cortex in monkeys. Eur J Neurosci 33:1952–1960.
from neuropsychology. Neurosci Biobehav Rev 37:269–278. Kato R, Zeghbib A, Redgrave P, Isa T (2021) Visual instrumental learning in
Brett CE, Sykes C, Pires-Yfantouda R (2017) Interventions to increase blindsight monkeys. Sci Rep 11:14819.
engagement with rehabilitation in adults with acquired brain injury: a Kinoshita M, Kato R, Isa K, Kobayashi K, Kobayashi K, Onoe H, Isa T (2019)
systematic review. Neuropsychol Rehabil 27:959–982. Dissecting the circuit for blindsight to reveal the critical role of pulvinar
Bruce LL, Stein BE (1988) Transient projections from the lateral geniculate to and superior colliculus. Nat Commun 10:135.
the posteromedial lateral suprasylvian visual cortex in kittens. J Comp McHaffie JG, Stanford TR, Stein BE, Coizet V, Redgrave P (2005) Subcortical
Neurol 278:287–302. loops through the basal ganglia. Trends Neurosci 28:401–407.
Burnett LR, Stein BE, Chaponis D, Wallace MT (2004) Superior colliculus Meredith MA, Stein BE (1985) Descending efferents from the superior colli-
lesions preferentially disrupt multisensory orientation. Neuroscience culus relay integrated multisensory information. Science 227:657–659.
124:535–547.
Moschovakis AK, Karabelas AB (1985) Observations on the somatodendritic
Calvert GA (2001) Crossmodal processing in the human brain: insights from
morphology and axonal trajectory of intracellularly HRP-labeled efferent
functional neuroimaging studies. Cereb Cortex 11:1110–1123.
neurons located in the deeper layers of the superior colliculus of the cat. J
Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of
Comp Neurol 239:276–308.
audio-visual integration sites in humans by application of electrophysio-
Nagy AJ, Berényi A, Gulya K, Norita M, Benedek G, Nagy A (2011) Direct
logical criteria to the BOLD effect. Neuroimage 14:427–438.
projection from the visual associative cortex to the caudate nucleus in the
Crawford MLJ, Anderson R, Blake R, Jacobs G, Neumeyer C (1990)
feline brain. Neurosci Lett 503:52–57.
Interspecies comparisons in the understanding of human visual per-
ception. In: Visual perception: the neurophysiological foundations Nelles G, Esser J, Eckstein A, Tiede A, Gerhard H, Diener HC (2001)
(Spillmann L, Werner JS, eds), pp 23–52. San Diego: Academic Press. Compensatory visual field training for patients with hemianopia after
Cuppini C, Magosso E, Rowland B, Stein B, Ursino M (2012) Hebbian mech- stroke. Neurosci Lett 306:189–192.
anisms help explain development of multisensory integration in the supe- Paré M, Crommelinck M, Guitton D (1994) Gaze shifts evoked by stimula-
rior colliculus: a neural network model. Biol Cybern 106:691–713. tion of the superior colliculus in the head-free cat conform to the motor
Daibert-Nido M, Pyatova Y, Cheung KG, Reginald A, Garcia-Giler E, Bouffet map but also depend on stimulus strength and fixation activity. Exp
E, Markowitz SN, Reber M (2021) An audiovisual 3D-immersive stimula- Brain Res 101:123–139.
tion program in hemianopia using a connected device. Am J Case Rep Sahraie A, Cederblad AMH, Kenkel S, Romano JG (2020) Efficacy and pre-
22:e931079. dictors of recovery of function after eye movement training in 296 hemia-
Dakos AS, Walker EM, Jiang H, Stein BE, Rowland BA (2019) nopic patients. Cortex 125:149–160.
Interhemispheric visual competition after multisensory reversal of Saionz EL, Feldon SE, Huxlin KR (2021) Rehabilitation of cortically induced
hemianopia. Eur J Neurosci 50:3702–3712. visual field loss. Curr Opin Neurol 34:67–74.
Dakos AS, Jiang H, Stein BE, Rowland BA (2020) Using the principles of Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, “uni-
multisensory integration to reverse hemianopia. Cereb Cortex 30:2030– sensory” processing. Curr Opin Neurobiol 15:454–458.
2041. Sherman SM (1977) The effect of superior colliculus lesions upon the visual
Danzl MM, Etter NM, Andreatta RD, Kitzman PH (2012) Facilitating neu- fields of cats with cortical ablations. J Comp Neurol 172:211–229.
rorehabilitation through principles of engagement. J Allied Health Sparks DL, Hartwich-Young R (1989) The deep layers of the superior collicu-
41:35–41. lus. Rev Oculomot Res 3:213–255.
1026 • J. Neurosci., February 8, 2023 • 43(6):1018–1026 Rowland et al. · Ameliorating Hemianopia with Multisensory Training
Sperdin HF, Cappe C, Foxe JJ, Murray MM (2009) Early, low-level auditory- Wallace SF, Rosenquist AC, Sprague JM (1990) Ibotenic acid lesions of the
somatosensory multisensory interactions impact reaction time speed. lateral substantia nigra restore visual orientation behavior in the hemia-
Front Integr Neurosci 3:2. nopic cat. J Comp Neurol 296:222–252.
Sprague JM (1966) Interaction of cortex and superior colliculus in mediation Woods TM, Recanzone GH (2004) Visually induced plasticity of auditory
of visually guided behavior in the cat. Science 153:1544–1547. spatial perception in macaques. Curr Biol 14:1559–1564.
Sprague JM, Meikle TH (1965) The role of the superior colliculus in visually Yoshida M, Takaura K, Kato R, Ikeda T, Isa T (2008) Striate cortical lesions
guided behavior. Exp Neurol 11:115–146. affect deliberate decision and control of saccade: implication for blind-
Stein BE, Rowland BA (2020) Using superior colliculus principles of sight. J Neurosci 28:10517–10530.
multisensory integration to reverse hemianopia. Neuropsychologia Yoshida M, Hafed ZM, Isa T (2017) Informative cues facilitate saccadic local-
141:107413. ization in blindsight monkeys. Front Syst Neurosci 11:5.
Stein BE, Spencer RF, Edwards SB (1982) Efferent projections of the neonatal Yu L, Stein BE, Rowland BA (2009) Adult plasticity in multisensory neurons:
superior colliculus: extraoculomotor-related brain stem structures. Brain
short-term experience-dependent changes in the superior colliculus. J
Res 239:17–28.
Neurosci 29:15910–15922.
Stein BE, Spencer RF, Edwards SB (1984) Efferent projections of the neonatal
Yu L, Rowland BA, Stein BE (2010) Initiating the development of mul-
cat superior colliculus: facial and cerebellum-related brainstem structures.
tisensory integration by manipulating sensory experience. J Neurosci
J Comp Neurol 230:47–54.
Szalados R, Leff AP, Doogan CE (2021) The clinical effectiveness of Eye- 30:4904–4913.
Search therapy for patients with hemianopia, neglect or hemianopia and Yu L, Rowland BA, Xu J, Stein BE (2013) Multisensory plasticity in adult-
neglect. Neuropsychol Rehabil 31:971–982. hood: cross-modal experience enhances neuronal excitability and exposes
Takakuwa N, Kato R, Redgrave P, Isa T (2017) Emergence of visually-evoked silent inputs. J Neurophysiol 109:464–474.
reward expectation signals in dopamine neurons via the superior collicu- Yu L, Xu J, Rowland BA, Stein BE (2014) Multisensory plasticity in superior
lus in V1 lesioned monkeys. Elife 6:e24459. colliculus neurons is mediated by association cortex. Cereb Cortex
Taub E (2004) Harnessing brain plasticity through behavioral techniques to 26:1130–1137.
produce new treatments in neurorehabilitation. Am Psychol 59:692–704. Zhang X, Kedar S, Lynn MJ, Newman NJ, Biousse V (2006) Natural history
Wallace SF, Rosenquist AC, Sprague JM (1989) Recovery from cortical blind- of homonymous hemianopia. Neurology 66:901–905.
ness mediated by destruction of nontectotectal fibers in the commissure Zihl J (1995) Visual scanning behavior in patients with homonymous hemia-
of the superior colliculus in the cat. J Comp Neurol 284:429–450. nopia. Neuropsychologia 33:287–303.