|Home | About | Journals | Submit | Contact Us | Français|
Although there are many well-known forms of visual cues specifying absolute and relative distance, little is known about how visual space perception develops at small temporal scales. How much time does the visual system require to extract the information in the various absolute and relative distance cues? In this article, we describe a system that may be used to address this issue by presenting brief exposures of real, three-dimensional scenes, followed by a masking stimulus. The system is composed of an electronic shutter (a liquid crystal smart window) for exposing the stimulus scene, and a liquid crystal projector coupled with an electromechanical shutter for presenting the masking stimulus. This system can be used in both full- and reduced-cue viewing conditions, under monocular and binocular viewing, and at distances limited only by the testing space. We describe a configuration that may be used for studying the microgenesis of visual space perception in the context of visually directed walking.
Visual space perception research has identified a variety of stimulus cues that specify absolute distance (between an object and an observer) and relative distance (between two objects). This research has populated a list of cues that includes binocular parallax (the stimulus to convergence), binocular disparity, relative motion parallax, angular declination, linear perspective, texture gradients, and so forth (Sedgwick, 1986). More recently, research has been aimed at understanding how visual space perception is used to control actions—behaviors directed at objects and locations in the nearby environment. For example, in the visually directed walking paradigm (or, simply, blind walking), participants view a target, then cover their eyes and attempt to walk to the remembered target location without further visual input (Thomson, 1980, 1983). In well-lit, natural viewing conditions, performance in this task is typically quite accurate and precise out to at least 22 m (e.g., Loomis, Da Silva, Fujita, & Fukusima, 1992). This good performance has been interpreted by several authors as indicating that the initial target location was perceived accurately (Loomis, Da Silva, Philbeck, & Fukusima, 1996; Philbeck, Loomis, & Beall, 1997).
How long does it take the visual system to extract the spatial information needed to control spatially directed behaviors? Does extraction of information proceed at the same speed for all visual cues, or are some cue combinations processed faster than others? These questions are crucial for understanding how vision is used to control behavior, but, as yet, the answers are largely unknown. In this article, we describe a system that can be used to address a wide variety of fundamental issues surrounding the microgenesis of visual space perception.
One specification for such a system is that it provides brief, precisely timed (tachistoscopic) exposures of the stimulus scene, so that the time available for extracting information from a given glimpse can be finely controlled. Limited-viewing-time tasks have long been used to study the speeded visual processing of a variety of stimuli, ranging from words (e.g., Adams, 1979) to depictions of objects (e.g., Ryan & Schwartz, 1956) and pictures of scenes (e.g., Potter, 1976). When the stimuli are letters, line drawings, or pictures, it is relatively easy to present brief exposures using a computer monitor. However, computer monitors are currently incapable of reproducing the full richness of spatial information that is typically available in real, three-dimensional objects or scenes. Several studies have circumvented this limitation by using a liquid crystal shutter (“smart window”), which can transition from opaque to translucent, and vice versa, very rapidly under computer control. Klatzky and colleagues (Fikes, Klatzky, Pellegrino, Hebert, & Murdock, 1990; Klatzky, Lederman, & Matula, 1993), for example, have used a smart window to precisely control the exposure onset of real, manipulable objects, which allowed them to study the planning and execution of reaching and grasping. Precisely controlling the total duration of stimulus exposure was less important in these studies, however. Philbeck (2000) used a smart window to study the effect of fixation location on egocentric distance perception using the blind walking task. By limiting exposures of a real indoor scene to 150 msec, the smart window did not allow observers sufficient time to move their eyes to fixate other locations in the scene. In this study, observers were able to walk accurately without vision to targets glimpsed for only 150 msec, even if they did not directly fixate the target. Furthermore, accuracy was equally high for 150-msec and 5-sec exposures. Although equivalent performance in the limited- and extended-viewing-time conditions might be considered striking, there is very little empirical basis for making more detailed predictions in this kind of paradigm. The aim to further investigate this outcome was the impetus behind the development of the system described here.
A second specification for a system that can probe the time course of visual space perception is that it provides a visual masking stimulus following the brief exposure of the stimulus scene. Visual masking (backward masking, in particular) refers to the ability of one stimulus—the mask— to reduce the visibility of a preceding briefly presented stimulus—the target. Although the masking literature is rich, and there are ongoing attempts to better characterize the mechanisms that underlie the phenomenon (see Breitmeyer & Öğmen, 2000, 2006), there is general agreement that the mask interacts with the processing of the target in a way that degrades the emerging representation and/or interrupts its continued processing. Given the goal of characterizing the time course of absolute-distance perception, a masking stimulus is crucial for precisely controlling how much time participants have to extract information from their glimpse of the scene containing the target object. It is generally held that a highly detailed visual sensory memory representation persists for some time after stimulus availability is terminated, and that this memory representation is susceptible to masking (e.g., Irwin & Yeomans, 1986). The use of a masking stimulus is thought to impose a strict deadline, beyond which no further information can be extracted and converted to a more durable kind of memory (e.g., Gegenfurtner & Sperling, 1993; Vogel, Woodman, & Luck, 2006). Although methods for controlling the duration of viewing of real three-dimensional scenes have been developed and implemented (Fikes et al., 1990; Klatzky et al., 1993; Philbeck, 2000), the addition of a masking stimulus to the method appears to be novel. Masking a real scene presents several technical challenges, however. The masking stimulus must be arranged in such a way that it is visible in the same direction as the stimulus scene (straight ahead), yet does not obscure the scene itself during the brief stimulus exposure. In addition, if an action-based response toward the briefly glimpsed object or scene is required (e.g., reaching or walking), the smart window and masking apparatus must not interfere with execution of the action. Finally, the smart window and masking stimulus presentation must be carefully coordinated in a way that takes into account the temporal response properties of all components of the system.
The system we describe below meets these specifications. The primary components of the system are (1) a smart window, used to control the exposure duration; (2) a semisilvered mirror (or beamsplitter), used to allow vision of both the stimulus scene and a virtual image of a masking stimulus (via mirror reflection) in the same direction (i.e., straight ahead relative to the observer); (3) a liquid crystal (LC) projector, used to project the masking stimulus; and (4) an electromechanical shutter mounted on the LC projector lens, used to control the onset of the masking stimulus. Control and coordination of the smart window and mechanical shutter may be accomplished using a computer equipped with a digital data acquisition board and appropriate software (e.g., LabVIEW, National Instruments, Austin, TX).
Figure 1 shows a basic configuration that can be used to study the exposure duration required to control blind walking to a target seen in a well-lit environment. The system is configured to progress through three states. In the initial state, the smart window and the mechanical shutter are both closed. The participant directs his or her gaze through the semisilvered mirror and toward the center of the opaque window. (For additional experimental control, a chinrest may be used to stabilize the head.) When the participant is ready, a buttonpress by the experimenter initiates the exposure sequence. In the second state, the smart window goes transparent, while the mechanical shutter remains closed. At this point in time, the participant will have a clear view of the target in the laboratory scene. The availability of various absolute- and relative-distance cues during the glimpse may be manipulated by controlling the viewing conditions during the glimpse. For example, the system we describe affords manipulation of background visibility (well-lit background vs. glowing target in the dark), monocular versus binocular viewing, floor-level versus eye-level target position, stationary head versus moving head, and so on. Thus, the time course for cues and interactions between cues can be examined by comparing the viewing durations required for accurate performance in these varied contexts. In the third state, the smart window goes opaque while the mechanical shutter opens. When the smart window is closed and the electromechanical shutter is open, the LC projector displays the masking stimulus on a projection screen to the participant’s left, which is reflected to the viewer by way of the semisilvered mirror (35.6 cm wide × 25.4 cm tall; Edmund Optics, model NT72-500). The mirror is mounted at an approximately 45° angle relative to the observer’s line of sight. Ideally, the reflectance and transmittance properties of the mirror should maximize the transmittance of direct light from the stimulus scene, while maintaining sufficient reflectivity to provide a reasonably bright reflected image of the masking stimulus. Our mirror reflects 30% and transmits 70% of the incident light, but other mirrors may work just as well or better. The projection screen and the area surrounding it are kept dark when the smart window is open, thereby minimizing the visibility of reflections in the semisilvered mirror during the stimulus presentation. When the masking stimulus is displayed on the projection screen, its reflected image is bright and easily visible.
In addition to controlling the duration of viewing, the system is designed so that participants can walk straight ahead without vision to the remembered target location. That is, the chinrest, smart window, and beamsplitter are each bolted to a platform mounted on drawer rails, which can be slid out of the way before the beginning of the walking response. The sliding platform supporting the chinrest, mirror, and smart window is, itself, mounted on a stationary stage. This is a structural component (62 cm wide × 92 cm long × 122 cm tall) that provides a stable base for the moving platform. The sliding platform is also supported by a vertical piece of wood that terminates in a caster resting on the floor. This allows smooth motion of the platform in and out across the floor.
The stationary stage may be used to house devices such as an oscilloscope for monitoring the smart window timing and/or the computer that controls the smart window and electromechanical shutter timing. The masking image is stored on a computer and sent to the projector through a VGA cable; opening or closing the electromechanical shutter mounted on the LC projector determines when the masking image is cast onto the projection screen. To project the masking stimulus, our configuration uses a Toshiba Data Projector model TLP470, which has an LC matrix illuminated by a high-pressure mercury-vapor arc lamp. Because the projector receives its images from a nearby laptop computer, any computer software capable of displaying image files can be used to present masking images. In our initial testing, we used Microsoft Power-Point to present an unchanging masking image—a scrambled photograph of the testing space. In principle, the LC projector and laptop computer system could be used to generate and display unique, randomized masking images on each trial of an experiment or to present a masking image in only one part of the visual field. Masking images that are in motion could also be projected. Note that because the projector lamp is continuously illuminated and is shuttered by the electromechanical shutter, the response characteristics of the projector itself are not important.
The smart window, shown in Figure 2, is an LC window that has the ability to cycle rapidly between opaque and translucent states. When unpowered, liquid crystals in the window adopt a random alignment, which scatters light and makes the window virtually opaque (Figure 2, left panel). When current is applied, the crystals align and allow light to pass through the window (Figure 2, right panel). The model described in this article was made by LC-TEC Displays, AB (Borlänge, Sweden), model number FOS-307x406-PSCT, and measures 30.7 × 40.6 cm.
Accurate knowledge of the smart window’s temporal characteristics is crucial. Figure 2 shows the setup used to characterize the smart window. A 4.8-mW battery-powered infrared (IR) laser was placed on one side of the smart window and a phototransistor on the other. Phototransistors produce a voltage output as a function of the light input, thus providing a relative measure of the light passing through the smart window. The IR laser is advantageous, because its relatively high intensity facilitates detection by the phototransistor, whereas its relatively long wavelength minimizes the visibility of the light source if the laser is used to monitor smart window operation online during an experiment. The phototransistor was a Fairchild Optoelectronic Group QSE113 880-nm (IR) side-looking NPN phototransistor. As we will describe below, for our arrangement, the smart window had a minimum transition time (closed-to-open and open-to-closed) of 2.6 msec. The phototransistor has a published response time of 16 µsec. This means that the phototransistor can respond over 100 times faster than the smart window, giving us confidence in our measurements. The output of the phototransistor was wired to a Tektronix TDS 2014 100-MHz digital oscilloscope.
The smart window is a DC-powered device operating with a minimum voltage of 50 V to a maximum of 160 V. We converted 120 VAC 60-Hz line voltage to 100 VDC using a Micro Commercial MP502W bridge rectifier, which has a 500-msec response time. The smart window’s opening response time is strongly dependent on the input voltage, ranging from 4.5 msec using 90 VDC (the lowest value given in the manufacturer’s technical specifications) to as fast as 0.5 msec using 160 VDC. The manufacturer’s specifications show that our 100 VDC yields a closed-to-open time of 2.5 msec and an open-to-closed (relaxation) time of 0.5 msec. Thus, the fastest time for a full closed-to-open-to-closed cycle is 3.0 msec. We verified this value using the IR laser and oscilloscope setup described above.
The signal required to send the 100-VDC power to the smart window is controlled by a mixture of a laptop-mounted data acquisition system and integrated circuits. The process starts at the laptop when the researcher clicks one of a set of virtual trigger buttons (denoting possible smart window durations) on the laptop’s screen. Using code generated in LabVIEW, the computer sends a 5-VDC digital signal through a National Instruments PC slot-mounted DAQ 516 data acquisition card to an op amp on an external circuit board. The op amp (National Semiconductor, model LN324N) is wired as a voltage follower, which is standard procedure for most data acquisition interfaces. A voltage follower isolates impedance and protects the computer in case of an overvoltage on the circuit board.
The op amp then triggers a Fairchild Semiconductor LM556 timer chip, which is configured to send out a one-shot (monostable mode) on/off signal. This on/off signal goes to an International Rectifier PVA30N solid-state relay (with a published off-to-on-to-off response time of 160 µsec), and this relay controls DC power to the smart window.
The LM556 is a dual-timer chip (the equivalent of two LM555 timer chips). The square wave sent to the relay is also used as input to the second timer. This second timer is essentially a flip-flop, used to trigger the mechanical shutter as soon as the smart window is turned off. The output timing signal of any 555 or 556 timer chip is controlled by a combination of a capacitor and a resistor. To control timing, we used a 2.0-µF capacitor and an Analog Devices AN 5220 100-kΩ digital potentiometer. The digital potentiometer produces resistance values from 1,000 to 100,000 Ω in 780-Ω increments. The resistance is set by putting a square wave on the CLK pin of the AN 5220 and by the chip counting the number of pulses. On each dropping edge of the square wave, the output resistance increases by one increment (780 Ω). Resetting is accomplished by placing a high signal to a separate pin (the U/D pin), which causes the potentiometer to increment backward. With this arrangement, we are able to control timing of the smart window to within 2 msec.
This configuration uses four digital output lines from the digital acquisition card. Two are used to set the timing by setting the digital potentiometer. One deactivates the mechanical shutter, if we should wish to prevent masking. The fourth triggers the shutter. Thus, ultimately, the shutter timing and power are handled by a simple, reliable external circuit based on a well-known integrated circuit, the 555 timer.
In the configuration described here, an electromechanical shutter (Melles Griot 48-V IES 211) is placed directly over the lens of the LC projector. The shutter’s adjustable aperture control is left in the maximum aperture setting (24-mm in diameter). The projector is continuously illuminated and is positioned so that it casts the masking image onto the projection screen to the participant’s left when the mechanical shutter is electronically opened. The electromechanical shutter system is composed of two components: an iris-style mechanical shutter device and a control box (a Lafayette Instrument model 43016 electronic shutter controller). The control box has several functions: It provides power to the iris mechanism, controls how long the shutter remains open, and allows the iris to be controlled by an external trigger. External triggering to open and close the shutter is accomplished by grounding two of the four pins on the back of the controller. The controlling electronics use another PVA30N solid-state solenoid that grounds these two pins.
Characterization of the temporal characteristics of the mechanical shutter was accomplished with a similar arrangement used to characterize the smart window. A phototransistor and a 5-W incandescent light source were aligned on either side of the center of the mechanical shutter’s iris.
The time required to fully open the shutter iris was 2.3 msec, and the time required to fully close the iris was 3 msec. These values were obtained with the shutter control set for a 5-msec exposure. In the configuration we describe, accuracy and precision in closing the electromechanical shutter (which terminates the masking image) are not critical. The signal that opens the shutter iris can be adjusted via the software graphical interface so that the iris becomes fully open as soon as possible after the smart window goes fully opaque.
Although additional steps could be taken to control participants’ fixation prior to triggering their glimpse of the stimulus scene, there are no provisions for this in the configuration we have described. This means that participants’ eyes may not be appropriately accommodated and converged to the target during the glimpse. How might this affect target localization during the glimpse? Importantly, inappropriate accommodation and convergence do not preclude egocentric distance perception. Under unrestricted viewing, the stimulus cues that elicit accommodation and convergence—namely, monocular and binocular parallax, respectively—are available and potentially useful as egocentric distance cues, regardless of the oculomotor posture of the eyes (Foley, 1978). These cues remain available and potentially useful under tachistoscopic presentation, unless steps are taken to eliminate them. Eye movements executed after a brief glimpse might exert an additional effect on egocentric distance perception, above and beyond any monocular and binocular parallax information. That is, even if a masking stimulus follows a brief glimpse of the stimulus scene, changes in accommodation and convergence could still be initiated during the glimpse, perhaps on the basis of monocular and binocular parallax. These changes might then reach completion (and influence distance perception) after the visual stimulus has been replaced by the masker (Foley & Richards, 1972).
If no steps are taken to control fixation prior to the glimpse, one possibility is that participants may begin a trial fixating on the opaque smart window, which is less than 50 cm from the eyes, and then see a scene with a target more than 2 or 3 m away during the glimpse. Another possibility is that prior to the glimpse, participants may converge to a distance somewhat beyond the smart window; in its opaque state, the window is relatively featureless and provides a poor fixation target. In any case, the influence of eye position prior to the glimpse is poorly understood in this paradigm. In future work, we plan to monitor eye position and eye movements before and during glimpses of the stimulus scene to investigate this issue.
Another currently unresolved issue concerns the effectiveness of a masking stimulus that appears at a depth plane different from that of either the fixated location during a glimpse or the target location during the glimpse. Although the center of the beamsplitter is only a few inches from the viewer, the virtual distance of the mask is greater and depends on the distance from the projection screen to the beamsplitter. As a result, the masker distance could be made to correspond to or diverge from the actual distance of the target. There is some evidence that a masking stimulus can indeed reduce the visibility of a target offset in depth via binocular disparity (Lehmkuhle & Fox, 1980), but the full effect of the masker distance in our paradigm remains to be explored.
The system described here is extremely flexible and may be used to study a variety of issues in visual space perception and the visual control of action. It can be used under both multicue and reduced-cue viewing conditions, under monocular or binocular vision, and with target distances limited only by the size of the testing space. It can be used to study not only reaching and grasping, but also visually directed walking. Because observers view real three-dimensional objects and scenes, the system is capable of providing a rich variety of visual space perception cues, in a way that allows the cues to vary together as they do in the real world. The system could also be augmented by equipment to study movement kinematics after brief stimulus exposures.
This work was supported in part by NIH Grant RO1 NS052137 to J.P.