|Home | About | Journals | Submit | Contact Us | Français|
The Gestalt psychologists were fascinated with dynamics evident in visual perception, and they theorized that these dynamics were attributable to ever-changing electrical potentials within topographically organized brain fields. Dynamic field theory, as it was called, was subsequently discredited on grounds that the brain does not comprise a unitary electrical field but, instead, a richly interconnected network of discrete computing elements. Still, this modern conceptualization of brain function faces the challenge of explaining the fact that perception is dynamic in space and in time. To pursue the question of visual perception and cortical dynamics, we have focused on spatio-temporal transitions in dominance during binocular rivalry. We have developed techniques for initiating and measuring these transitions psychophysically and for measuring their neural concomitants using functional magnetic resonance imaging (fMRI). Our findings disclose the existence of waves of cortical activity that travel across the retinotopic maps that define primary and secondary visual areas within occipital cortex, in correspondence with the subjective perception of spreading waves of dominance during binocular rivalry. This paper reviews the results from those studies.
Binocular rivalry is the alternation in perception that occurs when the two eyes view dissimilar monocular patterns (Blake & Logothetis, 2002). The phenomenon was first systematically described by C. Wheatstone (1838) in his landmark monograph on stereopsis, and it was subsequently highlighted in the writings of H. von Helmholtz (1866), William James (1890) and B. B. Breese (1899). There now exists a voluminous literature on the characteristics of binocular rivalry (see reviews by Walker, 1975; Blake, 2001), and in recent years significant advances have been made in identifying the neural concomitants of this beguiling phenomenon (Leopold & Logothetis, 1999; Tong et al., 2006).
The purpose of this paper is to summarize recent work on one particularly intriguing aspect of binocular rivalry: the transitions between alternative states of dominance during which one stimulus emerges from suppression in a wave-like fashion. The locations in time and space of these transitions are unpredictable, but once started these wave-like transitions imply the existence of spreading waves of neural activity. To set the stage for describing this aspect of rivalry, it is useful to begin with a summary of two key properties of binocular rivalry, one having to do with rivalry’s spatial extent and the other having to do with its temporal characteristics.
Upon extended viewing of binocular rivalry between dissimilar monocular targets, it is readily apparent that the incidence of complete dominance of one target or the other depends importantly on the size of those targets (compare the rivalry associated with the two pairs of rival stimuli reproduced in Figure 1). When viewed foveally, rival targets larger than a degree or so of visual angle can exhibit significant periods of mixed dominance during which portions of both targets are simultaneously visible within different locations; these periods of mixed dominance resemble an ever-changing mosaic comprising bits and pieces of both eyes’ views. When foveally viewed, rival targets are relatively small; however, one tends to experience exclusive dominance of one target or the other, with periods of mixed dominance occurring mostly during transitions from dominance of one to dominance of the other. This dependence of exclusive dominance on stimulus size suggests that rivalry transpires within local zones whose size is governed by the sizes of the receptive fields of neurons embodying those zones. In this regard, it is noteworthy that larger rival targets imaged within more peripheral regions of the visual field, away from fixation, are less susceptible to mixed dominance, implying that zone sizes scale with retinal eccentricity (Blake et al., 1992).
A second key property of binocular rivalry concerns the pattern of alternations in dominance over time (see Figure 2). Rather than switching regularly between one stimulus and the other, rivalry alternations occur unpredictably, with the successive durations of dominance being stochastically independent (Fox & Hermann, 1967; Lehky, 1988). The resulting frequency histogram of dominance durations conforms to the general shape of the gamma distribution (Levelt, 1965; but see Brascamp et al., 2005). The average rate of alternations during rivalry is dependent on stimulus variables such as luminance (Fox & Rasche, 1969), contrast (Mueller & Blake, 1989) and spatial frequency (Fahle, 1982). Observers can lengthen the average duration of dominance states by intensely focusing attention on the currently dominant rival stimulus (Chong et al., 2005), but observers cannot willfully hold one stimulus dominant indefinitely: rivalry alternations appear to be an obligatory outcome of the neural events underlying rivalry. It is commonly believed that neural adaptation plays an important role in promoting rivalry alternations, the idea being that the strength of the currently dominant stimulus wanes over time and eventually falls below some threshold level that tips the balance of strength in favor of the previously suppressed stimulus. Pointing to the involvement of adaptation in rivalry alternations are studies showing that exposure to a pattern immediately prior to experiencing rivalry biases initial dominance away from that stimulus when it is placed in rivalry against another, unadapted stimulus (e.g., Hancock et al., 2008). Also, implicating the involvement of adaptation is a study showing that rivalry alternations are substantially slowed when rival targets move smoothly around the visual field, thereby precluding neural adaptation within any particular region of the retina (Blake et al., 2003).
These spatio-temporal properties of rivalry have led to the development of models of rivalry based on local competition between neural representations of the competing rival stimuli (see Figure 3). According to these models, neurons supporting these competing representations exert reciprocal inhibition on one another, such that the activity associated with one representation temporarily outstrips the other, causing dominance of that stronger representation in a winner takes all fashion. At the same time both pools of neurons undergo self-adaptation, with the strength of adaptation dependent on the level of activity within those neurons. The dominant representation, being temporarily more active, undergoes stronger adaptation that eventually culminates in a reversal in the balance of relative activity. This causes the neural representation of the previously suppressed stimulus to exceed that of the previously dominant stimulus, thereby triggering a switch in the state of rivalry. To instantiate the irregular durations of dominance during rivalry, these models typically incorporate internal noise within some component of the neural circuitry (Wilson, 2007), and in some neural models this noise plays a preeminent role (Moreno-Bote et al., 2007).
This reciprocal inhibition model can differ in its subtleties (Lehky, 1988; Blake, 1989; Sugie, 1982; Kalarickal & Marshall, 2000; Wilson, 2005), but until recently all versions of the model have been designed to account for rivalry within a local region of the visual field, and the transitions between rivalry states have been treated as all or none phenomena. Yet we know from simple inspection of rival stimuli that these transitions are not abrupt and spatially unitary. When viewing rivalry, people readily perceive smooth transitions in dominance as one rival stimulus replaces the other in visual awareness. These orderly transitions in dominance are particularly conspicuous when viewing large rival patterns whose features have global configuration – with these kinds of stimuli, one perceives waves of dominance that originate locally and then spread rapidly over the remainder of the previously suppressed target. These waves of dominance are difficult to study, however, because they arise unpredictably anywhere within the currently suppressed rival target, and their directions of spread are typically unconstrained by the geometry of the rival targets. Still, the existence of dominance waves would seem to offer a unique opportunity for studying cortical dynamics in visual perception, and for that reason we have sought to develop techniques for triggering dominance waves and controlling their propagation. The following sections describe the details of our efforts and the resulting insights gained from studying dominance waves using brain imaging techniques.
Two of us (SHL and RB), in collaboration with Hugh Wilson, developed a novel procedure for inducing and measuring dominance waves (Wilson et al., 2001). Waves were induced with annular rival targets, essentially one-dimensional shapes that constrained the direction of motion. The rival targets were circular patterns like those illustrated in Figure 4, shown here as a spiral grating presented to one eye and a radial grating to the other. The observer always maintained fixation at the center of the circular patterns, where rivalry was not being induced. The two monocular rival gratings were surrounded by identical fusion frames to promote stable binocular alignment; these stimuli were displayed on the left and right halves of calibrated video monitor, and viewed through a mirror stereoscope. All aspects of the displays and all trial-related events were under computer control.
The observer maintained strict central fixation and depressed and held a designated key on the computer keyboard when one of the two rival targets was completely dominant, with no hint of any portion of the other, suppressed pattern. The computer then triggered the introduction of a brief, abrupt contrast increment at one small location somewhere around the annular region of the currently suppressed rival target. As expected, this contrast increment reliably initiated a change in rival state at that region of the annular pattern, causing a small wedge of the previously suppressed grating to achieve dominance. This wedge then appeared to spread from the initial trigger location in a wave-like fashion around the rest of the annular grating (even though there was no physical change to the stimulus other than the brief trigger). The observer released the key that was pressed to trigger the wave once the dominance wave reached a target point (indicated in Figure 4 by the white ticks on the inner and outer edges of the annulus), and the computer registered the time it took for the wave to travel from trigger to reference locations. By placing the trigger point at different locations relative to the reference point, we were able to estimate the average speed of dominance waves around the annular rival target.
Our results revealed several intriguing characteristics of traveling waves in rivalry. As can be seen in Figure 5, wave speeds varied somewhat from observer to observer but averaged 3.6 deg/sec (angular degrees of visual angle) for emergence of dominance within a grating comprised of radial wedges and 9.6 deg/sec for waves traveling around a rival grating comprised of concentric rings. The marked dependence of wave speed on the configuration of the rival target may be attributable to interactions among neighboring neurons forming a circuit within which neural waves propagate. It is known that neurons within primary visual cortex are interconnected, with the strength of these intracortical connections being stronger among neighboring neurons whose receptive fields have the same or similar preferred orientations. There also exist feedback signals from extrastriate cortex to orientation-selective neurons in primary visual cortex that may form part of the circuitry underlying contour integration (Angelucci et al., 2002). Together, these neural mechanisms could serve to promote stronger neural interactions among concentrically shaped contours compared to radially shaped contours.
By varying the size and eccentricity of the rival targets, we found that the wave speed was faster when traveling around larger diameter targets, where speed is expressed as degrees of polar angle per unit of time. We then used published measurements of the V1 cortical magnification factor to transform distance in the visual field to distance across the cortical surface, and discovered that wave speed was constant when expressed in terms of millimeters of cortical tissue; all wave speeds collapsed onto an estimated value of 2.24 cm/sec. This value intrigued us, for it suggested that it might be possible to observe the neural concomitants of these waves in visual cortex using brain imaging techniques, thereby setting the stage for the next phase of this project.
A number of brain imaging studies have found that the blood oxygen-level dependent (BOLD) fMRI responses to unchanging rival visual stimulation (i. e. dissimilar patterns presented to the two eyes) are modulated during binocular rivalry, in synchrony with perceptual transitions. fMRI modulations have been observed in brain areas ranging from the thalamus (Haynes et al., 2005; Wunderlich et al., 2005) to the frontal cortex (e.g., Lumer et al., 1998). Of relevance for the work described here, it has been repeatedly found that BOLD signals measured from primary visual cortex (V1) fluctuate in amplitude dependent on whether the evoking stimulus is dominant or suppressed during binocular rivalry (Polonsky et al., 2000; Tong & Engel, 2001; Lee & Blake, 2002). Is there also evidence for traveling waves of activation within V1, as the psychophysical results described in the previous section suggest there might be? To answer this question, we (Lee et al., 2005) performed an imaging study in which traveling waves were induced using the contrast increment technique described above.
In this study, observers viewed a pair of annular rival targets, one a low contrast radial grating and the other a high contrast spiral grating; the two rival stimuli were presented separately to the two eyes and observers maintained strict fixation on a small cross located in the center of the annular region. Using the flash suppression procedure (Wolfe, 1984), it was possible to cause the high contrast spiral pattern to dominate initially in rivalry. Perceptual dominance was then reversed by presenting a brief, abrupt increase in contrast in a small region at the top of the low-contrast radial grating (i. e. the same triggering technique used in the psychophysical experiments). As expected, this contrast pulse generated a perceptual traveling wave: perceptual dominance of the low-contrast image progressively spread from the top of the annular region to the bottom – observers indicated when the wave reached the bottom of the annulus by pressing a key. Because of the contrast difference between the two rival gratings, this transition was associated with a progressive change in dominance from a high contrast pattern to a low contrast pattern. Now, it is well known that neural activity in V1 (measured with fMRI or electrophysiology) increases monotonically with the stimulus contrast; when the contrast of the evoking stimulus changes over time, the magnitude of the fMRI response changes too, taking into account the BOLD signal’s hemodynamic lag. This means, therefore, that if waves of dominance during rivalry are accompanied by waves of cortical activity within V1, it should be possible to measure corresponding modulations in the fMRI response time course, as the high contrast stimulus gives way in dominance to the low contrast stimulus. Moreover, cortical regions nearest the trigger point will receive high contrast stimulation for briefer durations, and this should produce a smaller fMRI response whose peak occurs earlier in time relative to the initiation of the perceptual transition from high contrast to low contrast. Thus, by measuring fMRI responses within the retinotopically defined regions of stimulation, we set out to measure traveling waves of cortical activity, neural correlates of perceptual traveling waves. It is important to note that it is the perceptual state of the evoking stimuli that was changing over space and time; the conditions of physical stimulation remain unchanged during the measurement interval.
Results from these measurements are summarized in Figure 6. The left-hand picture shows a high resolution anatomical image of portions of the occipital cortex near and including the calcarine fissure; included in this image are retinotopically identified regions containing voxels associated with regions of the visual field in which the rival gratings were imaged (shown by the dotted white line). The white arrow indicates a region of interest (ROI) corresponding to a visual field location near the top of the vertical meridian and, therefore, near the location at which dominance of the low-contrast radial grating first emerged; the black arrow indicates a ROI closer to the bottom of the low-contrast radial grating and, therefore, near a region at which dominance emerged later in time. For voxels located sequentially along the entire path of the retinotopically identified ROIs, we estimated the latency of the peak of the fMRI response: the middle in Figure 6 shows those average latency values as the function of distance from the upper vertical meridian where dominance waves were initiated. (These values represent the averages from three observers, all of whom showed the same monotonic pattern of results as those evident in these average data). The time elapsing between the onset of the dominance trigger and the peak in the fMRI response increased distance around the retinotopic representation of the annuli.
From trial to trial, the speed of the perceptual waves varied, and this allowed us to determine whether the peak in the fMRI response was correlated with the latency of the perceptual waves. To carry out this analysis, we divided all the trials on which waves were successfully instigated into three categories: slow, medium and fast wave trials. For each category, we calculated the average delay in the fMRI responses, and the results from that analysis are shown in panel C of Figure 6. fMRI response latency was indeed correlated positively with the speed of traveling waves. As an aside, our results confirm that variations in the BOLD signal can reveal very brief timing differences over distances covering just a few millimeters of cortical tissue.
In a subsequent study, S. H. Lee et al. (2007) also performed fMRI response latency analyses on retinotopically identified regions in visual areas V2 and V3 and found comparable dependence of response latency on retinotopic distance. These wave-like responses in extra-striate visual areas could result from feed-forward connections from V1, or they could reflect the generation of waves within extra-striate cortex that are then passed to V1 through feedback connections. We are currently inclined to favor the former interpretation (waves originating in V1 and propagating to higher visual areas), in part because of the differential effect of diverted attention on neural waves in V1, where waves survive diverted attention, compared to V2 and V3, where waves disappear with diverted attention (Lee et al., 2007). It remains for future work to settle this question definitively.
These observations concerning traveling waves of dominance suggest needed refinements in neural models of binocular rivalry. The scheme presented in Figure 3 could be modified along the lines shown in Figure 7, so that local rival circuits are replicated and sequentially interconnected, to represent neighboring retinal areas or zones. With this modification, neighboring local circuits would be interconnected by excitatory interactions between neurons representing the stimulus presented to the left eye (EL) and between neurons representing the stimulus presented to the right eye (ER); in addition, excitatory neurons associated with one stimulus would exert spatially graded inhibition on excitatory neurons representing the other stimulus (shown in this schematic by IL and IR). What this schematic does not incorporate are possible top-down influences from other cortical areas, influences that would be required to account for effects of attention (Chong et al., 2005; Chong & Blake, 2006; Lee et al., 2007), global context (Sobel & Blake, 2002), emotional connotation (Alpers & Gerdes, 2007) and motor control (Maruya et al., 2007) on binocular rivalry dynamics.
Also missing is circuitry to account for other forms of visual rivalry that are not dependent on eye of origin and, instead, are thought to transpire at levels of processing where competition is based purely on stimulus representations (Logothetis et al., 1996; Wilson, 2005). F. Tong et al. (2006) provide an overview of these various forms of visual rivalry and their possible instantiation within a single neural model.
Binocular rivalry provides an effective means for studying several questions central within contemporary psychology and neuroscience. Those questions include the neural concomitants of visual awareness (Crick & Koch, 2003), the role of meaning and experience in resolution of perceptual ambiguity (Leopold & Logothetis, 1999) and, as illustrated by the work summarized here, the contributions of cortical dynamics in perception. It is ironic – and gratifying – to realize that the field of perceptual psychology has come full circle, returning to ideas that were advanced by the Gestalt psychologists last century when experimental psychology was getting off the ground:
The looks of things are determined by the field organization to which the proximal stimulus distribution gives rise. (Koffka, 1935)
The principle of isomorphism demands that in a given case the organization of experience and the underlying physiological facts have the same structure. (Köhler, 1920/1938)
This work was supported by grants from the U.S. National Institutes of Health (EY13358 and EY016752) and a grant from the Brain Research Center of the 21st Century sponsored by the Frontier Research Program Ministry of the Office of Science and Technology of the Republic of Korea (M103KV010018-07K2201-01810). The paper is based on an invited talk given by RB at the 2008 International Congress of Psychology, held in Berlin, Germany.
*The paper was presented at the Symposium “Visual Perception: From Experiments to Modeling” which formed part of the ICP2008 conference held in Berlin in July 20–25, 2008.
Randolph Blake, Centennial Professor of Psychology, Department of Psychology, Vanderbilt University, Nashville TN 37212, USA.
Sang-Hun Lee, Department of Psychology, Seoul National University, Gwanak-ro, Gwanak-gu, Seoul, 151–742, Korea.
David Heeger, Department of Psychology and Center for Neural Science, New York University, 6 Washington Place, 8th floor, New York NY, 10003, USA.