|Home | About | Journals | Submit | Contact Us | Français|
The effect of sampling irregularity and window size on orientation discrimination was investigated using discretely sampled gratings as stimuli. For regular sampling arrays, visual performance could be accounted for by a theoretical analysis of aliasing produced by undersampling. For irregular arrays produced by adding noise to the location of individual samples, the incidence of perceived orientation reversal declined and the spatial frequency range of flawless performance expanded well beyond the nominal Nyquist frequency. These results provide a psychophysical method to estimate the spatial density and the degree of irregularity in the neural sampling arrays that limit human visual resolution.
Vision begins with the neural sampling of a continuous retinal image, a process of fundamental importance that imposes an upper limit to the spatial resolving power of the visual system. According to the sampling theory of visual resolution, when other limiting factors are avoided (e.g. filtering, noise) spatial acuity for extended gratings is set by the spatial density of neural sampling elements (Bergmann, 1858, Geisler & Hamilton, 1986, Helmholtz, 1911, Hughes, 1981, Merchant, 1965, Thibos, 1998, Williams & Coletta, 1987, Yellott, 1988). In this sampling-limited domain, resolution acuity is equal to the highest spatial frequency that can be represented veridically by the neural sampling array, the so-called Nyquist frequency. Theory predicts that retinal image components with spatial frequencies higher than the Nyquist limit may still be signaled by the array, but will be mis-perceived as “aliases” of the physical stimulus. Numerous experimental studies have confirmed this prediction in peripheral vision, where the relatively high optical bandwidth of a well-focused retinal image greatly exceeds the Nyquist frequency of the retinal mosaic (Anderson, Evans & Thibos, 1996, Anderson & Thibos, 1997, Anderson, Drasdo & Thompson, 1995, Anderson & Hess, 1990, Anderson, Mullen & Hess, 1991, Artal, Derrington & Colombo, 1995, Coletta & Williams, 1987, Smith & Cass, 1987, Thibos, Cheney & Walsh, 1987a, Thibos, Still & Bradley, 1996, Thibos, Walsh & Cheney, 1987b, Wang, Bradley & Thibos, 1997a, Wang, Bradley & Thibos, 1997b, Williams & Coletta, 1987). Although the eye’s optical system normally serves as an effective anti-alias filter in the foveal region of the retina, thereby preventing the attainment of sampling-limited performance for central vision, aliasing has been reported when this optical limitation has been circumvented by stimulating the retina with interference fringes (Coletta & Williams, 1987, He & MacLeod, 1996, Thibos et al., 1987a, Thibos et al., 1987b, Williams, 1985, Williams & Coletta, 1987, Williams & Collier, 1983). Within on this body of work, the transition spatial frequency that separates the domain of veridical perception (supported by well-sampled retinal images) from the domain of non-veridical perception (supported by under-sampled retinal images) has been used as a non-invasive measure of the functional density of retinal neurons in the living eye.
This paper is concerned with three issues that complicate the estimation of neural sampling density from psychophysical performance when the neural sampling mosaic is irregular. First, the theoretical formulae which link the Nyquist frequency of the array to sampling density assume that density is a fixed parameter, which is strictly true only for a regular lattice. For irregular arrays, sampling density and Nyquist frequency are random variables subject to statistical variability. Taking this statistical variability into account, it might seem reasonable to suppose that visual resolution limits are set by the average sampling density of the array. However, Geller et al. (Geller, Sieving & Green, 1992) have argued that psychophysical judgments are more likely based on isolated pockets of high sampling density, while the remainder of the array is ignored. If this be true, then psychophysical estimates would overestimate the mean sampling density, reflecting instead the maximum local density.
The second issue relates to the size of the window used to limit a grating stimulus to a finite patch. For a regular sampling array, enlarging a patch of grating to recruit more sample points does not help to remove the ambiguity of aliasing caused by undersampling. Thus, stimulus size should be irrelevant for experimental measurements of the Nyquist limit of regular arrays. However, if the sampling array is irregular then expanding the stimulus would be expected to aid visual resolution because larger grating patches are more likely to include a portion of retina which happens to have, by chance, a locally elevated sampling density. Psychophysical experiments in central and peripheral vision (Anderson et al., 1996, Pokorny, 1968) have demonstrated that visual resolution of gratings increases with the number of cycles contained within a patch of sinusoidal grating. Although that result could be accounted for by spectral analysis of the stimulus, an alternative hypothesis of irregularity neural sampling could not be excluded and therefore will be reconsidered here. Such considerations are also important for reconciling sampling theory with experiments employing sampled optotypes (Carkeet, Gerasimou, Parsonson, Biffin & Fredericksen, 2008).
The third issue is the criterion for identifying the Nyquist frequency of a sampling array, which is relevant to clinical applications such as determining functional density of neurons in diseased eyes (Chui, Thibos, Bradley & Burns, 2009). Previously we have argued that the onset of aliasing, as revealed subjectively or by the appearance of less-than-perfect performance in objective, orientation-identification tasks, is a reliable indicator of the transition from veridical to non-veridical perception and therefore is a reasonable estimate of the neural Nyquist limit (Anderson et al., 1996, Anderson & Thibos, 1999a, Anderson & Thibos, 1999b, Thibos et al., 1987b). Others have preferred to estimate the Nyquist frequency as half the stimulus frequency that causes an orientation reversal phenomenon predicted by 2-dimensional sampling theory in which the perceived orientation of gratings is orthogonal to the physical orientation (Coletta & Williams, 1987). Unfortunately, orientation reversals are rarely reported in studies of peripheral vision, which seems to obviate this technique for routine use. The reasons for this failure to observe orientation reversal in the peripheral field are unclear, but the possibility investigated here is that increased irregularity in the sampling array is the cause (Hirsch & Miller, 1987, Yellot, 1982).
Two experimental methods have been used previously for studying the consequences of spatial sampling on visual resolution. In the observer method, the critical sampling stage is located in the subject’s retina. This is the method used by most of the studies quoted above. In the source method, the critical sampling stage is transferred to the visual stimulus by using discretely sampled visual stimuli displayed on a computer monitor and viewed foveally. (The terms “source method” and “observer method” are used here in the same way they are used in the study of optical limits to vision (Smith, Jacobs & Chan, 1989).) In a previous study using this latter paradigm, Geller et al. (Geller et al., 1992) found that when individual pixels in a computer display of a grating pattern were randomly deleted, performance on an orientation discrimination task did not suffer, even though the average sampling density was significantly reduced. This observation led them to conclude that psychophysical performance on a resolution task is determined by that region of the stimulus with highest local sampling density. Alexander et al. (Alexander, Xie, Derlacki & Szlyk, 1995) used a similar paradigm to study letter identification and found that random deletion of pixels on a computer monitor hampered letter identification by an amount predicted by the resulting loss of stimulus contrast. Unfortunately, the random deletion paradigm confounds the three parameters of irregularity, sampling density, and contrast. Therefore, we developed an alternative approach that allowed us to control the degree of sampling irregularity while holding constant the average sampling density and average contrast of stimuli.
Our principle aim in the present study was to evaluate current methods for estimating the density and degree of irregularity in a neural sampling array based on psychophysical measurements of performance on an orientation identification task. A secondary aim was to delineate conditions that prevent the estimation of neural sampling density based on the method of orientation reversal. We pursed these aims with the source method that allowed systematic variation of the degree of sampling irregularity and window size on psychometric functions for the orientation discrimination task.
Two of the authors (DWE and YZW) served as subjects. The stimulus was viewed foveally by the right eye from a distance of 1 m and the left eye was occluded. Refractive errors for the experimental viewing distance were corrected with spectacle lenses. The experiments were approved by the Indiana University Committee for Protection of Human Subjects and was undertaken with the understanding and written consent of each subject.
The stimulus was an array of dots displayed in the center of a gamma-corrected monochrome monitor (1152 (H) × 882 (V) pixels, 8 bit luminance resolution, 82 dpi, Radius, Inc.) controlled by a Macintosh computer. As illustrated schematically in Fig. 1, the dots represented sample points obtained from patches of high contrast (80%) sine wave gratings as follows. A square patch of grating surrounded by a uniform area of the same mean luminance as the grating (40 cd/m2) was represented in computer memory by a two-dimensional table of luminance values corresponding to the pixels of the display. For a given experimental session, the grating patch contained a fixed number of cycles (n = 1, 2, 3, 4, 5, 6, 8, 10, 12, or 14) and the size of the patch, which we will call the window size, was made smaller or larger from trial to trial in order to vary the grating’s spatial frequency while maintaining a constant number of cycles. Anderson et al. (Anderson et al., 1996) provide a detailed account of this experimental paradigm and the advantages of co-varying window size with spatial frequency to maintain a fixed number of cycles. In the present series of experiments we modified the Anderson protocol by displaying not the grating itself, but a sampled version of the grating produced by first creating a sampling array used to extract corresponding values from the two-dimensional table of pixel luminances. Thus the output of the sampling process was a collection of grating samples the size of individual pixels on a uniform background with the same luminance as the surround. To improve the visibility of this array of samples on the computer monitor, each sample point was expanded to become a uniform, circular dot 4 pixels (1.2 mm) in diameter. The displayed dots were relatively small in comparison with their separation and were easily visible at a viewing distance of 1 m, for which the angular subtense of each dot was 4.3 arcmin. Examples of stimuli for n=4 cycles are shown in Fig. 1.
As described in detail in the Appendix, the sampling array was based on a triangular lattice with center-to-center spacing S between points. For such an array the sampling density is D = 2/(S2√3) samples per unit area and the Nyquist frequency ranges from a minimum of 1/(S√3) = 0.58/S to a maximum of 2/(3S) = 0.67/S, depending on stimulus orientation. All sampling arrays were based on the same lattice, for which S = 2.7 mm (i.e. D = 16 samples/cm2) on the display. Irregularity was introduced into the sampling array by displacing each point vertically and horizontally by a random amount. This spatial jitter was achieved by adding independent samples of Gaussian noise (mean = 0; S.D. = σS) to the x- and y-coordinates of each point. Fresh samples from the Gaussian noise source were taken for each stimulus trial. The degree of irregularity in the sampling array was determined by parameter σ, which is the standard deviation of the noise in units of the spacing constant S. Parameter σ was assigned values of 0 (regular array), 9.25%, 18.5%, 37.0%, 55.5% or 74.0% of S. In order to place sampling arrays with different degrees of irregularity on a common basis for comparison, we define the nominal Nyquist frequency of an irregular array to be 0.5√D. For a regular hexagonal array, the nominal Nyquist frequency is therefore 0.54/S, which is slightly less than the minimum Nyquist frequency value of 0.58/S and slightly greater than the familiar value of 0.5/S for 1-dimensional regular sampling. Adding noise to the sample coordinates had negligible effect on mean sampling density and therefore the nominal Nyquist frequency was the same for all experiments being reported. At the 1 m viewing distance of this study, the nominal Nyquist frequency of 0.54/(0.27 cm) = 2 cyc/cm on the display corresponded to an angular frequency of 3.5 cyc/deg. This experimental design ensured high visibility of the test stimuli because they were in a range of spatial frequencies for which foveal vision has high contrast sensitivity.
We anticipated that the orientation of the sampling lattice relative to the grating would be an important factor for sampling arrays with low degree of irregularity. Therefore, we repeated all experiments for 3 orientations (0°, 15°, and 45°) of the underlying triangular lattice. Orientation was set by rotating the lattice anti-clockwise from the zero orientation configuration as defined by Appendix Eqn. A1. By this convention, a triangular sampling lattice in the 0° orientation has natural axes lying at 0°, 60° and 120° relative to the horizontal. Example stimuli illustrated in Fig. 1 are for sampling arrays rotated 45 degrees, for which the natural axes lie at 45°, 105° and 165° relative to the horizontal.. When generating irregular sampling arrays, this rotation was performed prior to adding noise to x- and y-coordinates.
The Fourier spectra of the visual stimuli were computed as the convolution of the spectrum of the grating (a pair of delta functions) with the spectrum of the sampling array (the inverse lattice (Petersen & Middleton, 1962), see Appendix). If the sampling lattice is triangular with spacing S between samples, then the lattice spectrum is triangular with spacing constant 2/(S√3), but transposed and rotated with respect to the sampling lattice. The result was then weighted by the Fourier transform of a disk the same size as a display spot, which dampens very high frequency components of the stimulus.
Examples of frequency spectra are shown in Fig. 2 for three stimulus configurations. Part A depicts the case of sampling a vertical and horizontal grating of spatial frequency slightly higher than the nominal Nyquist frequency fN. The spectrum of the continuous sinusoidal grating is represented in the left panel by the pair of circles (vertical grating) and the pair of crosses (horizontal grating), plus another delta function at the origin representing the mean luminance of the grating. The center panel shows the central portion of the spectrum of a sampling lattice in the 0° orientation, which puts the spectral lattice in the 30° orientation. The circle centered on the origin is a 2-dimensional extension of the concept of a Nyquist limit. We call this circle a Nyquist ring because the radius of the ring indicates the highest spatial frequency of the continuous input that can be faithfully represented by the sampled output. Strictly, the Nyquist ring for a triangular lattice is a hexagon defined by a nearest-neighbor rule: all points inside the hexagon are closer to the origin than to any other lattice node. Here we make the simplifying assumption that the Nyquist ring is circular with radius equal to the nominal Nyquist frequency (0.54/S for a regular lattice). This choice of radius is convenient because it applies also to irregular arrays.
The right hand panel of Fig. 2A shows the result of convolving the left and middle panels to compute the spatial frequency spectrum of the sampled stimulus. Convolution creates multiple copies of the source spectrum, one copy centered on each point in the array spectrum. To use this panel as a graphical method for predicting the alias patterns produced by undersampling, we concentrate our attention on the interior of the Nyquist ring since this is the domain of spatial frequencies that satisfy the sampling theorem (Petersen & Middleton, 1962). When the source grating exceeds the Nyquist frequency of the array, the spectrum of the sampled stimulus will fall outside the Nyquist ring. However, other copies of the source spectrum centered on nearby lattice points may fall inside the Nyquist ring, thus masquerading as low-frequency gratings below the Nyquist limit. This process by which high-frequency components masquerade as low-frequency components when undersampled, sometimes called leaking or folding of the spectrum, is the essence of aliasing. The stimulus spectrum depicted in Fig. 2B is for the same grating frequency as in 2A, but rotated 15 degrees. The spectrum depicted in Fig. 2C is also for a rotated grating, but a higher spatial frequency.
Psychometric functions for grating resolution as a function of spatial frequency were measured with an orientation discrimination task using the method of constant stimuli. A two-alternative forced-choice (2AFC) paradigm was used in which each stimulus trial contained a grating chosen randomly to be oriented either vertically or horizontally. The subject’s task was to indicate by pressing a key which orientation was present on each trial. The stimulus was visible until the key press initiated the next trial. Each session randomly interleaved 10 horizontal and 10 vertical targets with a fixed number of cycles displayed at each of 10 different stimulus sizes for a total of 200 trials. Subject’s responses for the two grating orientations were pooled to determine percent correct. Window sizes were selected so that the spatial frequencies presented in each session would range from approximately 30% below to 300% above the nominal Nyquist frequency of the sampling array.
To provide a theoretical framework for understanding the effects of sampling irregularity, we developed a probability summation model of psychophysical performance for orientation identification based on the fraction of the stimulus that is well sampled. For any given point in the sampling array, the distance d to the nearest neighboring point is equal to the half-period of the finest grating that is adequately sampled by that particular pair of points. Similarly, the inverse quantity f=0.5/d is the spatial frequency of the finest grating that is adequately sampled by that particular pair of points. In an irregular sampling array, the normalized local frequency = f/fN for nearest neighbors is a random variable that is characterized statistically by a probability density function, p(). The corresponding cumulative distribution can therefore be interpreted as the fraction of point pairs in the sampling array that will inadequately sample a grating of normalized frequency and therefore contribute to aliasing. Conversely, 1-P() is the fraction of point pairs in the sampling array that will adequately sample a grating of normalized frequency and therefore contribute to veridical perception.
Representative examples of psychometric functions for the orientation discrimination task are shown in Fig. 3. The sampled grating contained 6 cycles and the results are shown separately for three different orientations of the sampling array. Consider first the psychometric functions for the regular sampling array (SD=0). For two of the three examples shown (Figs. 3B,C), performance was flawless for low spatial frequencies, began to fall when the stimulus frequency reached the nominal Nyquist frequency, became worse than chance (50% correct) for frequencies higher than about 150% of the nominal Nyquist frequency, and achieved flawlessly incorrect performance (i.e. orientation reversal) at 200% of the nominal Nyquist frequency. Further increases in spatial frequency caused performance to recover to chance levels at 250% of the nominal Nyquist frequency and somewhat exceed chance at 300% of nominal Nyquist. As noted in Discussion, all of these features of the results can be accounted for by theoretical analysis of aliasing produced by undersampling with a regular lattice. The same analysis also predicts that when the lattice is in the zero degree orientation, undersampling of horizontal or vertical gratings will not produce orientation reversal of the alias. This prediction is confirmed by the psychometric function of Fig. 3A for the lattice (SD=0) condition, in which performance does not fall significantly below chance levels.
When noise was added to the sampling arrays to produce increasing amounts of irregularity, the psychometric functions in Fig. 3 changed in a characteristic way. Worse-than-chance performance became less frequent, resulting in a flatter function that also shifted laterally towards higher spatial frequencies. In other words, sampling irregularity reduced the incidence of orientation reversal and expanded the spatial frequency range of flawless performance beyond the nominal Nyquist frequency.
In order to quantify the effect of sampling irregularity on resolution acuity, we adopted 90% correct as a criterion for estimating acuity from the experimental psychometric functions of Fig. 3. This criterion was selected as an indicator of the transition from veridical to non-veridical perception, and the onset of aliasing, as revealed by the appearance of less-than-perfect performance in the orientation-identification tasks. The results, shown in Fig. 4 for both subjects, were normalized by the nominal Nyquist frequency (0.5√D) calculated from the array density. Except for the special case when the lattice is in the zero degree orientation (Fig. 4A), all the results showed the same trend towards improved resolution acuity as irregularity increased up to a limit of σ/S = 0.6 (subject DWE) or 0.4 (subject YZW) and then declined slightly. The maximum resolution acuity measured for irregular arrays was nearly double the nominal Nyquist frequency for this particular target (n=6 cycles).
All of the results illustrated in Figs. 3, ,44 were for the case of n=6 cycles in the target grating. In order to characterize the effect of number of cycles per window on resolution acuity, the experiments and data analysis described above were repeated for 9 other window sizes. As shown in Fig. 5, the effect of varying the number of grating cycles from 1 to 3 was the same for regular and irregular samping arrays. However, as the number of cycles increased beyond 3, resolution depended strongly on the degree of irregularity. For the regular sampling array, resolution remained equal to the nominal Nyquist frequency of the array, regardless of the number of grating cycles present. In contrast, resolution for a moderate level of irregularity (σ/S = 0.14) continued to improve up to 1.6 times the nominal Nyquist frequency. For a high level of irregularity (σ/S = 0.37) resolution continued to improve as additional cycles were added to the stimulus grating. For the largest number of cycles tested (n=14), resolution was 2.7 times the nominal Nyquist frequency.
To provide a theoretical comparison with the psychophysical results, we performed Monte-Carlo simulations of triangular sampling lattices containing approximately 1000 points that were jittered with various amount of noise (see Appendix). We then performed a nearest-neighbor analysis of these jittered arrays to estimate the cumulative probability distribution 1-P() that describes the performance of the probability summation model as described in Methods. The results, shown in Fig. 6, demonstrate that as the spatial frequency of a grating stimulus increases, fewer points in the array are able to adequately sample the stimulus. Therefore the probability of correct identification of stimulus orientation should decline, a prediction verified psychophysically (Fig. 3). The rate of this decline, indicated by the slopes of the curves in Fig. 6, is greater when the array is more regular. However, the corner frequency where performance first begins to decline is approximately equal to the nominal Nyquist frequency, regardless of the degree of irregularity in the array. Taken together, these two results imply that as the degree of irregularity in the sampling array increases, a larger fraction of the stimulus will be adequately sampled even when the stimulus frequency is greater than the nominal Nyquist frequency.
The principle aim of the present study was to determine the quantitative relationship between the nominal Nyquist frequency of an irregular sampling array and the visual resolution limit determined psychophysically. Our results show that psychophysical resolution closely matches the nominal Nyquist frequency when the sampling array is a perfect lattice (except for the special case of the 0° lattice aligned with one axis parallel to the stimulus), but increases beyond the nominal Nyquist frequency when the sampling array is irregular. The extent to which psychophysical resolution overestimates the nominal Nyquist limit depends upon the degree of irregularity in the sampling array and upon the number of cycles displayed in the sampled grating. Our data also reveal an interaction between the level of array irregularity and the occurrence of orientation reversal that suggests a method for psychophysical estimation of the degree of irregularity in the neural sampling mosaics of the observer’s retina. To pursue this possibility we begin with an account of the main features of the experimental results based on the sampling theory of visual resolution.
Inspection of Fig. 2A reveals that the sub-Nyquist aliases of a horizontal grating remain horizontal, regardless of their spatial frequency, when the sampling array is in the 0° orientation. This explains why subjects in our experiments were able to perform the orientation discrimination task without error even for frequencies beyond the Nyquist limit (Fig. 3A). The reason the orientation of the alias is the same as the target is that the original sampling lattice has certain natural axes (0°, 60°, 120°). As may be appreciated from the graphical analysis in Fig 2A, if the orientation of the source grating matches one of the natural axes of the sampling array, the alias will have the same orientation as the original target. By contrast, undersampling of a vertical (90°) grating just beyond the Nyquist frequency produces multiple aliases of different orientations near the Nyquist frequency. Thus, psychophysical performance could remain high simply by adopting the strategy: if the stimulus clearly looks horizontal, say horizontal, otherwise say vertical. Comments from our two observers indicated that this was indeed the strategy they developed with practice. Such a strategy begins to fail when the frequency of the target approaches double the nominal Nyquist frequency because the orientation of all the low-frequency aliases are roughly horizontal, regardless of the grating’s actual orientation. These expectations are reflected in Fig. 3A by the decline of psychophysical performance to chance levels when a regular lattice in the 0° orientation samples a grating with frequency twice the nominal Nyquist frequency.
When the sampling array is rotated into the 15° or 45° orientation, neither the vertical nor the horizontal targets are aligned with any of the natural axes of the sampling lattice. Consequently, low-frequency aliases of both targets rapidly invade the Nyquist ring and may have nearly the same orientation, as shown in Fig. 2B. Thus we should expect to see performance for orientation discrimination fall quickly as the target frequency grows beyond the nominal Nyquist frequency, which was confirmed experimentally (Fig. 3B). When the frequency of the target grating is increased to twice the nominal Nyquist frequency, the aliases are found by the graphical analysis of Fig. 2C to be nearly orthogonal to the original target. This leads to the prediction of orientation reversal, which was a prominent feature of results in Fig. 3B, C. Thus all of the main features of the psychometric functions for regular sampling arrays at a variety of orientations can be accounted for by the foregoing theoretical analysis of aliasing produced by undersampling with a regular lattice.
The graphical analysis described above applies also to the case of sampling by irregular arrays, provided that the spectrum of the array is known. We have not attempted such an analysis for the many different arrays used in the present series of experiments. However, Yellott has shown using anatomical data for the sampling mosaic of extra-foveal cones in monkey retina (and also for theoretical models of randomly packed disks subject to a constraint on minimum spacing) that the spectrum of the sampling array contains a “desert island” devoid of signal energy at spatial frequencies below the dominant frequency (1/S, or twice the Nyquist frequency in Fig. 2B) of the array (Yellott, 1983). Convolution of such a spectrum with that of a grating beyond the Nyquist limit produces a complicated spectrum with aliased energy at many locations inside the desert island, but with a strong concentration of energy at the original input frequency just outside the Nyquist ring. Williams and Coletta have argued that observers in their experiments made use of this supra-Nyquist energy to perform orientation discrimination in the parafovea (Williams & Coletta, 1987).
Present experiments support and extend the above arguments by externalizing the sampling process and placing it on the computer display where it can be controlled. Both of our observers agreed that, although aliasing was evident for gratings beyond the nominal Nquist frequency, traces of the original, grating were also clearly visible at those places in the irregular array where sample points happened by chance to be closer together than usual. Thus, although the onset of aliasing was well correlated with the nominal Nyquist frequency, performance on the orientation discrimination task continued above chance levels for frequencies well beyond the nominal Nyquist limit. Evidently this “supra-nominal Nyquist” performance occurs because jitter in the array causes some sample points to become closer together than the spacing of the original lattice. Thus, the local Nyquist frequency is greater than the nominal Nyquist and, apparently, even a small patch of adequately sampled grating is sufficient for observers to identify the stimulus orientation correctly as suggested previously (Geller et al., 1992). In short, the term supra-Nyquist performance refers to performance beyond the sampling limit computed for the average spacing of the sampling elements, but not beyond the maximum limit computed for local regions where (because of jitter in the array) sample points happen to be more closely packed.
Probability summation provides a foundation for earlier suggestions (Coletta & Williams, 1987, Coletta et al., 1990, Williams & Coletta, 1987) that array irregularity could account for supra-Nyquist visual performance without violating the sampling theorem at the local level. Visual resolution can exceed the nominal Nyquist limit without violating sampling theory (Fig. 6) because localized areas of relatively high sampling density present in an irregular sampling array provide adequate clues for orientation identification of grating stimuli.
Present results confirm that, for regular sampling arrays, the Nyquist frequency of the array can be deduced accurately from psychophysical performance on an orientation discrimination task. Conversely, psychophysical performance significantly overestimates the nominal Nyquist frequency for irregular arrays. Measured resolution acuity grew with increasing number of cycles in the target and with increasing degree of irregularity in the sampling array. The highest resolution measured was more than double the nominal Nyquist frequency. These results support previous suggestions that supra- Nyquist resolution in parafoveal vision can be caused by irregularity in the retinal cone mosaic, and support also the corollary suggestion that retinal irregularity may cause large errors when estimating the density of the retinal sampling array when using an orientation discrimination task (Geller et al., 1992, Williams & Coletta, 1987). This is true regardless of whether the experimenter selects the traditional criterion of 75% correct performance or the corner of the psychometric function where performance first begins to fall below 100% (Fig. 3). Our probability summation model suggests that stimulus size should be a critical parameter affecting performance, and inferred estimates of sampling density, because recruitment of additional sample points increases the probability that some of them will be sufficiently closely spaced to allow the observer to identify the stimulus orientation. We pursue this idea in the next section.
The analysis of undersampling by regular arrays presented above does not depend strongly upon the number of cycles present in the target grating. Consequently, when a patch of grating is sampled by a regular array, enlarging the patch to recruit more sample points will not help remove the ambiguity of aliasing caused by undersampling. Thus, window size should not affect psychophysical performance for the case of regular sampling arrays and it doesn’t, judging from the flat curve shown in Fig. 5. If the sampling array is irregular, however, then expanding the stimulus would be expected to improve performance because a larger patch of grating is more likely to include a region of locally elevated sampling density. As shown in the Appendix (Fig. A2), if the regularity of a sampling lattice is perturbed by adding Gaussian noise to the location of individual sample points, the average spacing between sample points decreases roughly in proportion to the standard deviation σ of the noise, leveling off at the value S/2 for σ ≥ 0.4. The minimum local spacing can be even lower than S/2, with a probability determined by the left-hand tail of a Rayleigh probability function. Thus it is not surprising that experimental measurements of the resolution limit measured experimentally for highly irregular sampling arrays (Fig. 5) increased steadily with increasing number of grating cycles, achieving levels more than double the nominal Nyquist frequency for gratings containing 8 or more cycles.
Present results suggest that the shape of the psychometric function for orientation discrimination can be used to estimate the degree of irregularity in the mosaic of retinal neurons of human observers. Our experimental results indicate that the orientation reversal phenomenon, in which performance falls well below chance levels, disappears when the level of array irregularity (as measured by noise parameter σ) exceeds 10–20% of dot separation (Fig. 3). Since orientation reversal has been demonstrated for human observers viewing gratings in the parafoveal (3.8 degrees) retina where sampling-limited performance is determined by cone density (Coletta & Williams, 1987), present results would suggest that the irregularity in the cone mosaic is less than 20% of inter-cone separation at that retinal location. This conclusion is consistent with anatomical data showing a high degree of regularity in the parafoveal cone mosaic in macaque monkeys (Hirsch & Miller, 1987) and humans (Curcio & Sloan, 1992). Both studies reported that the standard deviation of nearest neighbor distances in the cone array at 3.8 deg eccentricity is 0.15 times the mean of nearest neighbor distances. From this result we infer that σ/S = 0.13 based on the results of Monte Carlo simulation described in Appendix (see Fig. A2). Although we did not use this specific value of array irregularity in our experiments, interpolation of the results in Figs. 3B and 3C suggest that orientation reversals probably would have occurred at a rate greater than chance.
The results in Fig. 3 also suggest that the dissolution of orientation reversal in the peripheral retina beyond 20 to 25 degrees eccentricity (Coletta & Williams, 1987), where sampling-limited performance is determined by ganglion cell spacing (Anderson & Hess, 1990, Thibos et al., 1987a), is due to increased irregularity of the mosaic of ganglion cell receptive fields as eccentricity increases. This prediction is seemingly at odds with anatomical studies of peripheral human retina show a remarkable degree of regularity in the mosaic of ganglion cell soma for the midget (Dacey, 1993), parasol (Field, Sher, Gauthier, Greschner, Shlens, Litke & Chichilnisky, 2007), and small-bistratified (Field et al., 2007) classes. The conformity ratio (i.e. the inverse ratio of standard deviation to mean of nearest neighbor distances) is between 7–8 for these three classes of ganglion cells, which makes them slightly more regular than cone photoreceptors at 3.8° eccentricity (conformity ration = 1/0.15 = 6.7). However, the irregularity of functional sampling of the retinal image by ganglion cells may be greater than these anatomical data indicate because of variability in the decentration of the dendritic field from the cell body (Dacey, Fig. 12) and in the mapping of anatomical dendritic fields to functional receptive fields (Cleland, Levick & Wassle, 1975).
The variation of visual acuity with window size is also characteristic of the degree of irregularity in the sampling array. Anderson et al. explored the effects of window size on visual acuity in central and peripheral vision (30° eccentricity) using the same experimental paradigm as in the present study, except that the stimulus was a continuous grating and the sampling operation was performed by the observer’s retina (Anderson et al., 1996). They found that resolution acuity increases as the number-of-cycles in the visual stimulus increased up to about 6 cycles and then remained constant as more cycles were added. Those results are similar to ours shown in Fig. 5 for the case of moderate irregularity in the sampling array. To draw a quantitative comparison, the mean resolution acuity for Anderson’s three subjects reported in their Fig. 7 was scaled by 1.5/6.5 to bring those data into agreement with the present results in the range 1–3 cycles/window. We justify this scaling by our finding that the rate of change of resolution acuity with number of cycles per window is independent of the degree of irregularity when the window exposes less than 4 cycles of the grating stimulus. The resulting function, shown by the dashed curve in Fig. 5 suggests two new conclusions from the Anderson data. First, the resolution limit measured for 6 cycles/window or more overestimated the nominal Nyquist frequency of their subjects’ neural arrays by 50%. Second, the degree of irregularity is moderate, with a value of σ/S somewhat less than 0.14, which corresponds to a conformity ratio of 8, which is the same value reported for midget retinal ganglion cells in peripheral retina (Dacey, 1993). Thus we conclude that although Anderson et al. were able to account for their results by a quantitative model of the spectral dispersion of energy caused by windowing, present results provide support for an alternative explanation based on sampling irregularity.
In this study we used sampled visual stimuli as a model of neural sampling of the retinal image. However, there is an important limitation in such a model that is potentially misleading. The visual world is spatially continuous, which requires that samples of visual targets be presented on a continuous background with visible gaps between spots used to represent the sample points. To the contrary, neural images carried by discrete arrays of visual neurons are spatially discrete. This difference between the spatially continuous domains of visual objects (and their retinal images) and the spatially discrete domain of neural images has important consequences for describing their frequency spectra because continuous functions have infinite bandwidth, whereas discrete functions have finite bandwidth (Bracewell, 1978).
For our experimental stimuli, the frequency spectra in Fig. 2 include a broad range of spatial frequency components outside the Nyquist ring. As explained in Appendix, the portion of the spectrum outside the Nyquist ring exists because the visual display is continuous, with zero values (i.e. neutral grey pixels) in between the dots that render the sampled grating. Spatial filtering by the eye’s optical system will prevent the very high spatial frequencies from appearing the retinal image, but the retinal image is nevertheless continuous with a frequency spectrum similar to that shown in Fig. 2. However, the spectrum of Fig. 2 is an inappropriate description for the discrete neural image produced when a continuous grating imaged on the retina is sampled by an array of photoreceptors. The discrete nature of the neural image (i.e. the array of cone responses) places an upper limit (i.e. the Nyquist frequency) on the highest spatial frequency (in cycles per neuron) represented by the neural array.
For these reasons, a neural analog to Fig. 2 depicting the frequency spectrum of a discrete neural image would contain only those spatial frequencies inside the Nyquist ring. Subsequent processing of the neural image by post-receptoral neurons is based solely on this discrete neural image provided by the photoreceptors. Thus, there is no need to postulate a physiological mechanism to impose a “window of visibility” (Coletta, Williams & Tiana, 1990) in the post-receptoral visual system to remove high-frequency portions of the spectrum outside the Nyquist ring because those frequencies won’t exist in the neural image produced by the photoreceptors. Nevertheless, is often a useful artifice to assume the neural image is continuous for the purpose of visualizing how aliasing arises as a result of neural undersampling (Thibos, 1995). Continuity is also a useful artifice for computing the spatial frequencies and orientations of aliases optically (Coletta & Williams, 1987, Yellott, 1983). Such computations are valid because, although the high frequencies don’t exist in the discrete neural image, the low frequencies inside the Nyquist ring (including aliases) are exactly the same for a discrete sampler as they are for a continuous sampler that assigns zero weight to all points between samples.
We thank Roger Anderson and Arthur Bradley for helpful discussions in the early phase of this research. This research was supported by NIH grant EY05109 to L. N. Thibos.
In the experiments reported in this paper we explored the consequences of irregularity in the sampling mosaic by adding noise to the positions of points in a perfectly regular lattice. This initial lattice had equilateral triangular packing (i.e. an hexagonal array) generated by the vector equation (Petersen & Middleton, 1962)
This equation specifies the location of an arbitrary point on the lattice as the sum of integer multiples of the two basis vectors and . where
Two different interpretations may be given to this lattice when modeling the visual process. In a neruophysiological context, the lattice points represent the locations of neural receptive fields (e.g. cone photoreceptors or retinal ganglion cells) that sample the retinal image. In this case the space between neighboring lattice points has no meaning since the neural image exists as a discrete set of neural responses. The alternative context, which includes our simulation experiments, interprets the lattice as a continuous function that has unit value at the lattice points and zero value at all points between the lattice points. Since this function is defined over the entire domain of the input image, it can be multiplied point-by-point with the input to produce a sampled image that has value zero between samples.
In both of these modeling applications the Fourier spectra of the sampled images will be discrete because of the finite dimensions of any physical image. However the bandwidths of these two spectra are very different. The spectrum of a discrete neural image is band-limited because, in general, N sample points can be described exactly by the weighted sum of N harmonic terms in a Fourier series. This bandwidth is equal to the Nyquist frequency of the lattice, which implies that frequency components beyond the Nyquist limit do not exist in the spectrum. To the contrary, the spectrum of a continuous sampled image has infinite bandwidth because an infinite number of harmonic terms are needed in a Fourier series to reconstruct the zero values between sample points. Although expanding the diameter of the sample points will dampen the high frequencies, the bandwidth of the continuous image will still exceed the Nyquist limit of the underlying lattice.
This equation specifies the location of an arbitrary point on the frequency-domain lattice as the sum of integer multiples of the two basis vectors and . where
The basis vectors for the reciprocal lattice are orthogonal to the basis vectors of the original lattice (i.e. vectors and are orthogonal, as are vectors and ). Thus the spectrum of a triangular lattice will be another triangular lattice, transposed and rotated with respect to the sampling lattice. Reciprocity also requires that the inner product of vectors and be unity, as is the inner product of vectors and . This results in a spectral lattice with spacing 2/(S√3), which implies that the density of the spectral lattice is the inverse of the density of the spatial lattice.
To introduce irregularity into a lattice, we added independent samples of Gaussian noise (mean = 0, variance = σ2) to the x- and y- coordinates of each point in the array. Such a jittered array is defined by the equation
where is a noise vector consisting of two independent samples of noise. The degree of irregularity in the jittered array is determined by parameter σ, which raises the following questions: (1) What is the relationship between the variance of the Gaussian noise source and the degree of irregularity in the array? (2) Can the process defined by Eqn. [A5] produce arrays which span the full range of behaviors from perfectly regular lattices to the totally random arrays produced by a Poisson point process?
In the context of sampling theory, the most important parameter of a sampling array is the spacing S between adjacent points. Spacing is constant for a regular, hexagonal array but is a random variable characteristic of the degree of randomness in an irregular array. Therefore, to approach the above questions empirically, we computed the coordinates of an array of points by the above method and then calculated the spacing between nearest neighbors for each point in the array. It was convenient to start with an hexagonal array for which S = 1 so that σ would be in units of this spatial dimension.
The statistical distribution of nearest-neighbor distances pooled across three separate realizations of jittered arrays (N ≈ 400 points each, for 1200 total) is illustrated in Fig. A1 for three different noise levels (σ/S = 0.2, 0.4, 1.0). Summary statistics were computed separately for each of these arrays, and for others generated with different amounts of jitter, and the results are shown in Fig. A2. The close agreement between the three datasets indicates good repeatability of the results. We found that as σ increases up to about 0.4, the mean separation between nearest neighbors decreases to about S/2, the standard deviation of separation increases to about S/4, and the ratio stdev/mean increases to about 1/2. This latter ratio is the inverse of the conformity ratio that has been widely used in neuroscience to quantify the regularity of mosaics of retinal neurons (Cook, 1996). No further changes in these statistics occurred for values of σ > 0.4.
To examine whether the method described above for generating irregular arrays may be treated as a Poisson random process, we apply the line of reasoning developed by Feller (p. 10) (Feller, 1971). Consider a Poisson ensemble of sample points in the plane with density D. The probability that a domain of area A contains no point equals e−DA. Saying that the nearest neighbor to the origin has a distance > r amounts to saying that a circle of radius r contains no sample point in its interior. The area of such a circle equals π r2, and hence in a Poisson ensemble of points the probability that the nearest neighbor has a distance > r is given by e−Dπr2. Conversely, the probability that the nearest neighbor has a distance < r is given by 1 e−Dπr2. This latter expression represents the (cumulative) probability distribution function for r, and therefore to obtain the corresponding probability density function we must differentiate with respect to r, which yields p(r) =2πrDe−Dπr2. To put this result in textbook form, we introduce the change of variables (Wassle & Riemann, 1978) α2 = 1/(2πD) to give p(r) = (r/α2)e−r2/2α2 which is recognized as the Rayleigh density function (Papoulis, p. 104) (Papoulis, 1965). Evaluating the known moments of the Rayleigh density (Papoulis, p. 148) we conclude that
If a putative Poisson array is produced by jittering a triangular lattice of spacing S, for which the density is D = 2/(S2√3), then the above results indicate the mean should equal 0.465S, the standard deviation should equal 0.243S, and the ratio of the two should equal 0.52. These conclusions follow from the observation that although jitter will affect the location of individual points, it won’t affect their overall density.
We note from Fig. A2 that the asymptotic values of the statistics of empirical arrays agree with theoretical predictions, provided σ > 0.4. We conclude from these results that jittering a triangular lattice is a valid technique for producing experimental arrays varying from a perfectly regular lattice to a random array with the same statistics as a theoretical Poisson array. A more stringent test for total randomness involves the comparison of experimental histograms with expected Rayleigh densities as in Fig. A1. Such comparisons indicate an acceptable level of agreement (χ2 goodness-of-fit test, p = 0.05) only for σ ≥ 1.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.