Stimuli consisted of random dot kinematograms (RDK) generated on an Apple Macintosh G4 computer and displayed on a 17” Apple CRT monitor. RDK motion sequences were displayed at 75Hz in a calibrated gray-scale mode at a screen resolution of 832 × 624 pixels. Each RDK was displayed in an imaginary square aperture subtending 44.5° × 44.5° at a viewing distance of 30 cm. The dots were distributed in a virtual trapezoidal volume whose bases were located 400 cm and 1500 cm from the observer. Dots were 2 × 2 pixels (4 × 4 arcmin) and were placed with a density of 2 dots/deg2. The motion of the dots within this volume simulated observer’s self-motion along a straight line trajectory at a speed of 100 cm/sec. Dots moving outside the trapezoidal volume were randomly assigned to new locations such that the density of dots inside the 3D volume was held constant. In each trial, the direction of self-motion, defined by the FOE, was randomized along an imaginary horizontal line extending throughout the center of the display, so that the FOE could be located at a range of positions within ± 22.25° of the screen center. The RDK stimulus was presented for 480 msec (12 frames), with each frame updated every 3 screen refreshes resulting in effective frame duration of 40 msec. At the end of the motion a new static random dot pattern, with the same spatial statistics, was displayed together with a vertical target line (8.96° long) that intersected the horizontal midline of the display. In all experimental conditions, the psychophysical variable of interest was the distance between the target and FOE, which provided a measure of heading accuracy, and is referred to as target offset. Target offset levels ranged between 0.0159° and 14.83°. In each stimulus, the dots and target were white (79.55 cd/m2) and displayed against a gray background (10.22 cd/m2).
The 3D trajectories of each dot were randomly perturbed between successive frames in three different experimental conditions: 1- Direction noise from constrained random-walk (); 2- Direction noise from constrained fixed-random-trajectory () and 3- Random heading direction noise ().
Figure 1 Schematic view of dots’ displacements for the three types of direction noise examined in experimental conditions 1–3. a) Experimental condition 1: Random-Walk Direction Noise. The direction of each dot is perturbed in each stimulus frame (more ...)
In experimental condition 1 (random-walk), the solid angle (θ) of each dot’s 3D displacement vector was randomly selected from a normal probability distribution, with a specified standard deviation (σnoise), centered around the dot’s unperturbed motion (). The 3D displacement vector’s magnitude was constant between each pair of frames. A random direction perturbation was applied to each dot in each frame independent of its perturbation in the previous frame (random-walk noise) (), such that the direction noise was spatially uncorrelated across dots within each frame and temporally uncorrelated across frames. Since this noise was spatially and temporally uncorrelated, motion vectors could be averaged over space and time, thus reducing the effect of external stimulus noise. This 3D noise resulted in perturbations of the local 2D displacement vectors as illustrated in that were well characterized by an offset exponential. In experimental condition 2 (fixed-random-trajectory), the perturbed trajectory of each dot was held constant after the first stimulus frame regardless of its change in position within the display (fixed-random-trajectory noise) (), creating noise that was spatially uncorrelated across dots within each frame, but temporally correlated across frames since the same deviation was applied to each dot throughout its lifetime. In experimental condition 3 (random-heading), direction noise was applied by shifting the direction of heading, characterized by the focus of expansion (FOE), in each stimulus frame (random-heading noise) (). The amount of FOE-shift was randomly selected from a normal distribution, with a specified standard deviation (σFOE-shift), centered around the actual heading. All dots moved coherently according to the heading direction of the current frame; thus creating a frame-wise coherent global motion percept. Random-heading direction noise was spatially correlated within each frame (i.e. each dot had the same 3D perturbation) but temporally uncorrelated across frames. The spatial correlation between direction vector perturbations limited the use of spatial integration for the accurate estimation of dot trajectories and heading location. Throughout the duration of the stimulus, shifting the FOE location had the effect of introducing local motion noise with respect to the unperturbed heading angle, thus providing a common measure of direction noise across all three experimental conditions. In the subsequent analysis of the random-heading results, we simulated shifted FOE locations within the heading stimulus to determine the relationship between the global perturbation and the local direction noise.
Figure 2 Illustration of direction noise. a) In this scheme, each dot’s trajectory can be visualized as falling on an imaginary sphere characterized by a 3D normal distribution of solid angles (θ) between the unperturbed and perturbed motion vectors. (more ...)
2.2. Experimental procedure
Prior to the start of an experimental session, observers adapted for 5 minutes to the background luminance of the monitor display in a quiet, darkened room. Each trial was preceded by an auditory cue, immediately followed by the RDK stimulus (480 msec) and then by the presentation of a static random dot pattern containing the target line. During the psychophysical task, observers were required to fixate a small central cross (0.75° × 0.75°). Stimuli were presented binocularly in a two alternative forced choice (2AFC) paradigm with no feedback and the observers’ task was to determine whether their heading was to the left or to the right of the target line. Responses were entered by pressing a predetermined button on the computer keyboard.
Observers’ target offset thresholds (79% correct) were estimated as the average over the last six reversals of the 3-down/1-up constant step size portion of an adaptive staircase procedure (Vaina et al., 2003
). In all experimental conditions observers’ performance was reported as the mean threshold averaged across at least three staircases. Two additional staircase thresholds were collected for each test condition containing a measured threshold greater than two standard deviations from the mean.
For experimental conditions 1 and 2, target offset thresholds were obtained for σnoise of 6.32°, 15.09°, 31.58°, 37.54°, 43.51°, 56.84°, 69.47°, 82.11°, 94.74°, 107.37° and 120°. For experimental condition 3, target offset thresholds were obtained for σFOE-shift of 3°, 6°, 9°, 12° and 15°.
To determine observers’ ability to perform the heading discrimination task, we first conducted a screening experiment using the test described in sections 2.1 and 2.2 and replicated from the work of Royden and Vaina (2004)
. Direction perturbation was not used during this screening period. Observers practiced the task for one hour. At the end of the hour each observer’s target offset threshold was approximately 2°, consistent with the performance of normal observers previously reported for this task (Vaina and Soloviev, 2004
; Warren et al., 1988
shows discrimination thresholds for heading direction (mean target offset threshold ± SE) for each observer plotted against standard deviation for the random-walk noise distribution (σnoise). Thresholds were averaged across the five observers (± SE) are indicated by the shaded region. In order to illustrate the effect of an increase in random-walk direction noise on the accuracy of heading judgments, a repeated measures of ANOVA was performed on the observers’ data. There was a significant effect of noise on heading perception across observers (F(1,49)=217.18, p<0.0001). Due to the set-up of the experiment, the maximum measurable target offset was 12°, therefore in the statistical analysis we considered only those noise ranges whose thresholds were less than 12° for all observers. For the five observers tested, this corresponded to maximum σnoise of 107.37° and at this level the mean target offset threshold across observers was approximately 8°.
Figure 3 Heading discrimination thresholds (random-walk direction noise), expressed in degrees of target offset for five observers a) AC, b) AT, c) ES, d) KC, e) YC as a function of the standard deviation (σnoise) of the random-walk noise distribution. (more ...)
In a similar heading stimulus, Warren et al. (1991)
applies random-walk noise with uniform distributions characterized by direction bandwidths of 45°, 90°, and 135° to 2D flow fields. In order to compare our results with theirs, we determined the best-fit distribution of projected 2D flow vectors perturbed with random-walk noise. A least squares fit using uniform distributions with different bandwidth values was used to determine the bandwidth value with highest correlation and lowest Kullback-Leibler (KL) distance values. For σnoise
of 37.54°, the bandwidth for the distribution of 2D perturbations was approximately 140°. At this level the average threshold was roughly 4° for the random-walk condition. This is similar to the performance reported by Warren et al. (1991)
, i.e. average threshold of roughly 3.5° for bandwidth of 135°.
Since random-walk direction noise was chosen independently for each dot and for each frame, the noise vectors present in each stimulus were uncorrelated in space and time. Accurate estimates of heading direction could utilize spatiotemporal integration of local dot trajectories, providing a performance baseline. Using fixed-random-trajectory direction noise, we investigated the case where the direction noise applied in 3D was spatially uncorrelated but temporally correlated. In this case, the noise applied to each dot was constant for the duration of each trial. Since the noise was temporally correlated, the temporal integration mechanisms should not be able to reduce stimulus noise. Thus, the purpose of using fixed-random-trajectory direction noise was to determine the effect of impairing the temporal integration mechanisms’ ability to reduce the external noise, thus allowing isolation of spatial integration mechanisms.
shows discrimination thresholds for heading direction (mean target offset thresholds ± SE) for each observer plotted against standard deviation for the fixed-random-trajectory noise distribution (σnoise). Heading discrimination across observers significantly worsened with increasing fixed-random-trajectory noise as indicated by a repeated measures of ANOVA (F(1,34)=92.51, p<0.0001). As in the random-walk condition, we considered only those values of σnoise, which resulted in measurable thresholds for all observers. In the case of fixed-random-trajectory noise, this corresponded to a σnoise of 69.47° (compared to 107.37° in random-walk noise) and a mean target offset threshold of 8.5°, for the observers tested. We attribute this increased sensitivity to direction perturbations to the inability of temporal mechanisms to reduce stimulus noise.
Figure 4 Heading discrimination thresholds (fixed-random-trajectory direction noise) expressed in degrees of target offset for five observers as a function of the standard deviation (σnoise) of the fixed-random-trajectory noise distribution. Error bars (more ...)
To compare the thresholds from random-walk and fixed-trajectory direction noise, a generalized linear model (GLM) was fit to the thresholds averaged across subjects,
where η was a binary classifier indicating the type of direction noise, (i.e. for random-walk (experimental condition 1) η = 1 and for fixed-random-trajectory (experimental condition 2) η = 0), A
was the offset for the threshold versus σnoise
was the slope term of the threshold versus σnoise
fit and C
was the interaction term between the type and amount of direction noise. The interaction term denoted whether the slopes were the same or different for two types of direction noise conditions. If the interaction term was not significant, then the slopes for different types of direction noise were the same. Note that the slope of the fit for the random-walk condition alone corresponded to the sum of B
, while B
gave the slope for the fixed-random-trajectory condition.
Comparison of the results from random-walk and fixed-random-trajectory direction noise conditions revealed that heading perception under the effect of fixed-random-trajectory noise was significantly worse than heading perception under the effect of random-walk noise (GLM analysis’ (df=92) interaction term: t
<0.0001). In a 2D direction discrimination task, Watamaniuk et al. (1989)
performed a similar comparison and found that there was no significant difference between thresholds obtained when the direction noise resulted from random-walk versus fixed-random-trajectory. The difference between our results and those of Watamaniuk et al. (1989)
may indicate a difference in the nature of the integration mechanisms employed in heading perception and 2D direction discrimination. The results obtained with fixed-random-trajectory noise suggest that observers temporally integrate across frames and can make use of the acquired temporal information in a heading discrimination task, but this information is not used in 2D direction discrimination.
In order to address quantitatively the properties of the temporal integration employed by the observers, we computed the cumulative direction vector for each dot from the first to last frame in the random-walk condition and then calculated the average noise (deviation of the 12-frame motion vector from the no-noise vector) over a 12-frame window for random-walk stimuli, which resulted in lower effective noise levels (σnoise-effective). For example, a σnoise of 56.84° with random-walk noise corresponded to a σnoise-effective of approximately 17° for a 12-frame window (480 msec) since over time the dot regressed towards its unperturbed motion vector. This allowed us to understand what performance on the random-walk task should be if observers simply averaged direction vectors over the entire stimulus.
The effective noise calculation results in a decrease in the cumulative noise with time by a factor of
, where (n-1
) is the number of frame-pairs. We scaled the noise values for the random-walk experimental condition to simulate temporal integration in an n
-frame time window. For each time window, we performed a GLM fit on the average target offset thresholds from the scaled random-walk and fixed-random-trajectory direction noise data. In the case of fixed-random-trajectory noise, the magnitudes of the noise vectors did not change with the duration of the integration, since the perturbed directions for each dot remained identical through the duration of the stimulus. The interaction terms of the GLM fits were not significant (p>0.05) for durations of 4, 5 and 6 frames, suggesting that the differences between thresholds in the two conditions were not statistically different (GLM analysis’ (df=16) interaction term for 4 frames: t=−0.9468, p=0.3578; 5 frames: t=0.6322, p=0.5362; 6 frames: t=1.7960, p=0.0914). The Pearson correlation (R2
) values obtained from the pooled data fits provide a measurement of how closely the threshold values from both noise conditions clustered around the fitted line. Therefore large values of R2
indicate that thresholds showed a high degree of similarity between the random-walk and fixed-random-trajectory conditions’ datasets. The best fit was obtained for a stimulus duration of 5 frames (R2
=0.91), meaning that thresholds for random-walk were indistinguishable from thresholds for fixed-random-trajectory conditions when scaled noise values were based on less than half the actual stimulus duration (200 msec).
illustrates thresholds for both conditions as a function of σnoise-effective
in the case of the 5-frame window (200 msec), for which the performance under the random-walk and fixed-random-trajectory noise conditions were most overlapping. The implication of this result is that observers were not performing the task by averaging over a full 12-frame window of the stimulus. Instead, they appear to use temporal integration, which can be explained as a dot-by-dot reduction in noise (i.e. a local process) averaged over a 5-frame window. This time window may reflect a limitation in the duration available to the temporal integration mechanisms, an initial latency before temporal integration began, or a combination of both. For the straight-trajectory heading, it has been reported that observers can compute translational heading by employing spatial integration over two frames (Warren et al., 1991
). Here, we show (in experimental conditions 1 and 2) that, in addition to the spatial integration, temporal integration has a beneficial role in the perception of straight-trajectory heading. Our result is similar to Watamaniuk et al.’s (1989)
findings that temporal integration leads to an improvement on direction discrimination in 2D RDK stimuli, especially in the presence of motion noise.
Figure 5 Average heading discrimination thresholds expressed in degrees of target offset for five observers as a function of effective noise (σnoise-effective) over a 5-frame stimulus duration (200msec). Circles denote thresholds for random-walk noise (more ...)
In experimental conditions 1 and 2, the noise in the heading stimuli resulted from local perturbations of direction. In experimental condition 3 (random-heading), we investigated the effect of a global direction perturbation on the accuracy of heading perception. The purpose of the random-heading condition was to determine the specific contributions of temporal integration mechanisms to the perception of heading direction, by reducing the involvement of spatial integration mechanisms.
shows discrimination thresholds for heading direction (mean target offset threshold ± SE) for each observer plotted against standard deviation for the random-heading noise distribution (σFOE-shift). Across observers, the heading discrimination accuracy dropped significantly with increasing noise as shown by a repeated measures of ANOVA (F(1,19)=470.82, p<0.0001). As in the other two locally applied direction noise types, we considered only the levels of direction noise that resulted in thresholds less than 12°, the maximum measurable target offset. In the case of random-heading direction noise (experimental condition 3) this corresponded to σFOE-shift of 12° with a target offset threshold of 12° across all observers.
Figure 6 Heading discrimination thresholds (random-heading direction noise), expressed in degrees of target offset for five observers as a function of the standard deviation (σFOE-shift) of the random-heading noise distribution. Error bars correspond to (more ...)
The shifts in heading angle due to the global noise introduced spatially structured perturbations in the local dot movements. To compare the local effects of the global noise condition with the random-walk and fixed-random-trajectory noise conditions, we derived a common metric by studying the 2D projected local noise levels for all direction noise conditions. We performed simulations for specific 3D noise distributions, i.e. σnoise for random-walk and fixed-random-trajectory direction noise and σFOE-shift for random-heading direction noise. Because the flow vector projections varied with eccentricity, we simulated heading for eccentricities of 0° (fixation) and 22.5° (edge of our aperture). In all noise conditions, σprojected-noise decreased with FOE eccentricity, so the 2D noise distributions corresponding to a central FOE and the most eccentric FOE constituted the upper and lower bounds for the resultant noise distributions (σnoise). We projected the 3D perturbed translation vectors onto the 2D plane and calculated the difference between the perturbed and unperturbed polar angles. Then we fit Gaussian curves to the projected noise distributions to quantify the resulting spread (σprojected-noise), for each FOE location and for each level of 3D direction noise. For all curve fits, the minimum R2 value was 0.87 and the maximum KL distance was 7.61. Since we used translation vectors between frames, there was no difference in σprojected-noise between random-walk and fixed-random-trajectory noise conditions. illustrate the effective range of the projected 2D direction noise (σprojected-noise) corresponding to local perturbations (σnoise) and global FOE perturbations (σFOE-shift), respectively.
Figure 7 a) Illustration of the equivalent projected 2D local perturbations (σprojected-noise) resulting from 3D local perturbations (σnoise) used in random-walk and fixed-random-trajectory noise. b) Illustration of equivalent 2D local perturbations (more ...)
Globally applied random-heading noise had a greater effect on heading perception than either random-walk or fixed-random-trajectory noise (as illustrated in ) even for the least effective situation (when FOE was at the periphery – ). For example, a random-heading noise (σFOE-shift) of 9° corresponded to local motion perturbations (σprojected-noise) of 14.50° and 6.51° when the heading angle was at the center or at the edge of the screen respectively, which leads to a σprojected-noise of about 10.50°. A similar level of σprojected-noise (11.39°) was obtained for σnoise of 6.32°. At this level, the average threshold across observers was approximately 3.04° for random-walk noise and 4.05° for fixed-random-trajectory noise, while the corresponding threshold for random-heading noise (i.e., for a σFOE-shift of 9°) was 8.23° (). As discussed previously, in random-heading direction noise, spatial integration mechanisms alone cannot be used to reduce noise. Thus, when comparing on the basis of projected 2D noise distributions, the increased thresholds for the random-heading noise condition suggest that, for perception of heading, the human visual system is more sensitive to noise affecting spatial integration mechanisms than to temporal integration mechanisms.
Using the equivalent local noise values for all the experimental conditions, we showed a progressive drop in heading accuracy from random-walk noise (spatiotemporal integration) to fixed-random-trajectory noise (spatial integration) to random-heading direction noise (temporal integration). The difference between random-walk and fixed-random-trajectory direction noises was fully accounted for by a temporal windowing of the random-walk noise, limiting the temporal integration of motion vectors to a 5-frame (200 msec) window (). Furthermore, the dramatic drop in performance on random-heading noise compared to random-walk and fixed-random-trajectory noise conditions indicates that when spatial integration was not available to improve task performance, subjects were significantly impaired on the task, demonstrating the importance of spatial over temporal integration mechanisms.