Separating region-derived components and their interaction
Statistically significant steady-state-evoked responses were present at harmonics of the distinct region-tag frequencies in all stimulus conditions and for all observers, but the magnitude and presence of the inter-region interaction terms, particularly the 1f1 +1f2 term, depended on stimulus condition and border arrangement. shows the temporal frequency spectrum for the phase-defined form stimulus as an overlay of all 128 locations for the 13-observer sensor-space average. Prominent responses are visible at the harmonics of the region tag frequencies (2f1,2f2,4f1,4f2, etc.), with the amplitude of these responses (and the background EEG noise) decreasing with increasing frequency. The second harmonics (2f1; 6.0 and 2f2; 7.2 Hz) are the dominant region-response components. Since the image updating procedure produces two temporal transients for each cycle, it is not surprising that a large response would be evoked at the second harmonic. The observation that the responses are dominated by even harmonics of the tag frequencies indicates that each stimulus transition evoked similar responses in the population.
Figure 3 EEG spectra derived from the phase-defined form are shown for (A) the 13 observer average with all 128 sensors superimposed and (B) for a single sensor from one observer. Figure responses are indicted in blue, background responses in cyan, and nonlinear (more ...)
The individual observer, single sensor spectrum in , illustrates the smaller region interaction components in relationship to the region responses. The responses evoked by the figure region are shown in dark blue (nf1), while those evoked by the background are shown in light blue (nf2). Responses that reflect nonlinear interaction between regions (e.g., the mutualor intermodualtion terms) occurred at frequencies equal to low-order sums and differences of the two tags and are highlighted in red (nIM, where IM stands for intermodulation). Those frequency components colored in gray did not meet the statistical criterion of p > 0.05 (T2circ test). This single sensor spectrum illustrates the presence of multiple, statistically significant interaction terms, with the largest coupling occurring at the sum frequency (1f1 +1f2; 6.6 Hz).
There are many possible frequencies at which nonlinear figure/ground interactions could occur (e.g., all frequencies equal to nf1
, where n and m are integers). In practice, only small integer combinations are observed, and among these only responses with even parity (e.g., m and n are both 1, or both even) are substantial. Tags at f1
were chosen to be close together so as to minimize the effects that differences in temporal frequency might have on region processes, and we have shown previously (Appelbaum et al., 2006
) that the pattern of region responses did not depend on the whether the 3.0- or 3.6-Hz tag was applied to the figure or the background. This experimental design choice however has the consequence that second- and fourth-order difference frequencies (1f2
; 0.6 Hz and 2f2
; 1.2 Hz) are located in an unfavorable part of the EEG spectrum where spontaneous EEG activity (noise) is high. In order to determine which interaction terms could be used for further analysis, we first determined which of these frequencies evoked responses that could be observed above the spontaneous EEG activity in most of the observers. To compute a relative signal-to-noise ratio, we compared the response at each low-order nonlinear combination frequency for each cue type to that measured in the figure only condition. Because the figure only condition has only a single input, there can be no nonlinear interaction at the combination frequencies, and it can thus be used to establish the noise level.
shows average peak voltages (maximum sensor amplitude) at the second- and fourth-order interaction terms for each cue type (column 2) and for the figure only condition (column 3) at the same sensor locations. The ratio of amplitudes in these conditions is shown on the right. Signal-to-noise ratios are low for both difference terms across conditions and are near 1 except for the 4thorder difference of the orientation-defined form, which appears to have some residual signal. Signal-to-noise ratios for the sum terms, however, are considerably larger for all conditions (>4.5) except for the second-order sum of the temporally defined form. In light of these ratios, we restrict our analysis of the figure/ground interaction to the sum terms, separately considering the second- and fourth-order sum terms.
Table 1 Summary of interaction term signal-to-noise ratios. Peak amplitudes (maximum sensor in group average) are shown in the second column for each cue type and the figure only condition in the third column. The ratio of these responses is shown in the fourth (more ...)
Distribution of region and interaction terms in sensor and source space
We previously found (Appelbaum et al., 2006
) that the response topography and source distribution of figurerelated activity differed from that of background-related activity. Here we compare these profiles to those of the most prominent region interaction terms (1f1
). shows the average response distributions for the second harmonics of the figure and background regions, and the second- and fourth-order sum terms for the phase-, orientation-, and temporally-defined forms. The figure is organized with response components as rows and stimulus types as columns. Within a subpanel, the spline interpolated scalp topography is shown at the upper left, and several views of the average CCDs are shown on the right. The topographic maps have units of microvolts (2V) and the current density distributions have units of pA/mm2
. The maximum voltage in each panel is shown below the corresponding spline topographic map and the maximum current density is shown above and to the right of the corresponding cortical maps. Color bars for each data type are shown below and the CCD averages are thresholded to gray at one-third of the maximum current density within each panel.
As we reported previously, the second-harmonic source distribution of the figure-region extends laterally across the occipital cortex (top row). This bilateral pattern activity is consistent across the three cue types and reflects a specialized network for the processing of the figure region. Background responses (second row) are also consistent across cue types but show a strikingly different distribution than figure responses. Activity at the second harmonic of the background tag is maximal at the occipital pole, extends dorsally rather than laterally, and also shows considerable activity on the medial aspect of the occipital cortex consistent with responses arising from the peripheral visual field (for details and additional controls, see Appelbaum et al., 2006
Figure/background region interaction at 6.6 Hz (1f1 + 1f2; third row) is substantial for both the phase-and orientation-defined conditions but is absent for the temporally defined stimulus. When present, this term shows two maxima: one at the occipital pole and another that lies dorsal and anterior to the maximum of the figure- region second harmonic. The secondary maximum is present in both phase- and orientation-defined form conditions but is less apparent in the false-color maps due to the greater magnitude of this response at the occipital pole in the orientation-defined form condition. The secondary maximum occurred at average Talairach coordinates of -39.5, -76.9, 16 on the left and 37, -76.9, 12.7 on the right. lists the Talairach coordinates of this ROI, as well as that of hMT+, LOC, and V3A.
Mean Talairach coordinates for left and right ROIs.
Figure/background region interaction at the fourth-order sum frequency (13.2 Hz; 2f1 +2f2) is of similar peak magnitude in all three stimulus conditions, as shown in the fourth row of . This component has a lower response magnitude than the 1f1 +1f2 component, but it is statistically significant in most of the individual observers and in each cue condition. This activity displays multiple maxima that are largely coincident with those seen for the 1f1 +1f2 term in the phase- and orientation-defined form condition. However, the distribution of the 2f1 +2f2 term peaks at the occipital pole and in left lateral cortex for the temporally defined form condition and is of lower magnitude in the TOPJ region. In short, figure/background interactions are present for all three cue types but show a high degree of cue dependency, in particular, second-order temporally defined form does not elicit a response at the second-order sum frequency.
The effects of cue type on interaction strength: ROI-based analysis
In order to quantitatively assess the effects of varying the cues used to define the figure/ground segmentation, we performed ROI-based analyses on the response magnitudes and their cortical distribution for the orientation-, phase-, and temporally-defined form stimuli, focusing on the second- and fourth-order sum terms. The projected response magnitude across observers is shown for 8 regions-of-interest (V1, V2d, V3d, V3a, V4, LOC, hMT +, and TOPJ) in . We had noted previously that the ventral divisions of V2 and V3 had lower figure- and background-region response magnitudes than the corresponding dorsal divisions (Appelbaum et al., 2006
). In the present analysis, we also considered the phase of the response (see Timing differences across the cortical surface: Surface-based averages section) and found a 180° phase difference between in dorsal and ventral divisions of V2 and V3 in addition to the amplitude differences we had previously reported (see Supplementary materials, Section 3
). These differences are likely due to source geometry effects or to properties of the inverse, rather than physiological differences between dorsal (lower visual field) and ventral (upper visual field) divisions of these two early visual areas. We have therefore performed the quantitative analysis using the larger and statistically more reliable dorsal-division responses. The locations of all ROIs are shown on one observer's cortical surface, along with the color-coding convention below. For comparison purposes, we plot the projected magnitude of the response at 1f1
as measured in the figure only condition (bottom row histograms and solid black line overlays for each of the different cue-types). These values reflect the average noise level for each ROI because no interaction terms are expected with only a single active input.
We first compared current density estimates for the three different cue types (orientation, phase, and temporal frequency) over the eight ROIs for the 1f1 +1f2 and 2f1 + 2f2 interaction terms. As expected from the average current density distributions of , the temporally defined form condition produced less response than the orientation or phase conditions (CUE: F(2,7) = 9.813, p = 0.009) at the 1f1 +1f2 term. Response magnitudes in the temporally defined form condition are at the noise level in all ROIs. The magnitude of the 2f1 +2f2 intermodulation component, in contrast, did not depend on cue type (CUE: F(2,7) = 1.663, p = 0.256). In the two cases (orientation and alignment) where the 1f1 +1f2 term was large, we compared the magnitude profiles over the ROIs and found them to be the same for the two cues for both the 1f1 +1f2 and 2f1 +2f2 terms (CUE(90v180)*ROI, 1f1 +1f2: F(7,2) = 18.059, p = 0.053; CUE(90v180)*ROI, 2f1 +2f2: F(7,2) = 0.129, p = 0.983).
The effect texture discontinuity on interaction strength: ROI-based analysis
As a second step in understanding the object segmentation process, we sought to isolate mechanisms that can detect continuous versus discontinuous images. These mechanisms must be able to compare texture properties such as orientation, spatial frequency, and contrast polarity across space. We used the nonlinear interaction components as a direct index of this comparison process. Based on previous VEP studies of nonlinear lateral interactions (Hou et al., 2003
; Norcia et al., 1999
; Victor & Conte, 2000
; Zemon & Ratliff, 1982
), we expected that the interaction terms would be sensitive to the relative alignment of the textures making up the figure and background regions. In the following experiment, we compared responses to different image sequences that consisted of identical modulations within their figure and background regions but in which the textures within the two regions either matched or did not match when they had the same orientation. In the changing segmentation
case, the global image structure was either entirely uniform or was segmented, either on the basis of an orientation difference between the figure and the background or on the basis of a relative alignment/phase cue. In the constant segmentation
cases, the texture in the figure region was drawn from a different random sample than that used to generate a continuous texture. These image sequences always contained a discontinuity, but the modulations within the two regions were the same as those in the changing segmentation condition. In the 90°constant segmentation case, the stimulus progressed through two different orientation-defined forms and two different phase-defined forms, depending on whether the figure and background regions had different or identical orientations respectively. In this case, the figure region was always visible, and it was supported by time-varying differences in the nature of the orientation cue. In the other constant segmentation condition, the 180° case, the stimulus alternated between four different, but indiscriminable, phase-defined forms. This stimulus also was continuously segmented, but there was no time variation in the orientation cue defining the segmentation and there were no visible configuration changes.
Responses at the harmonics of the figure and background region tagging frequencies were not affected by the constant segmentation, but the responses at the sum frequency were affected in both the orientation and phase-defined cue conditions. Sum frequency responses to the continuously segmented orientation-defined form stimulus were reduced by a factor of 2-3 across all ROIs ( middle, upper two panels) and the sum component interaction was reduced to the noise level ( left, upper two panels) in the phase-defined form case. The elimination of the uniform field states from the stimulus sequences resulted in a reduction of the 1f1 +1f2 term that was larger for the phase condition than for the orientation condition (CONFIG*CUE: F(1,8) = 6.275, p = 0.037) and independent of ROI (CONFIG*CUE*ROI: F(7,2) = 5.39, p = 0.165).
In contrast to what is observed at 1f1
, the segmentation status of the stimuli did not affect the projected amplitudes at 2f1
(none of the main effects or interactions were significant). Here again, the lack of significance may be partly due to the low SNR of this component even though projected amplitudes were consistently above the noise level. Nonetheless, the averages of suggest that the fourth-order term (2f1
)is present given that its distribution over the cortical surface has a similar appearance, especially in the orientation- and phase-defined form conditions. Fourth-order interaction terms have been reported previously by (Victor & Conte, 2000
) who used iso-oriented line targets. This term has also been described to be insensitive to the location and relative orientation of small grating patches, unlike the 1f1
term that was orientation tuned and largest for collinear vs. noncollinear but iso-oriented stimuli (Hou et al., 2003
). The presence of interaction at 2f1
was confirmed independently in the next experiment that used the phase-defined form. We thus conclude that at least some of the nonlinear interaction between figure and background, specifically the 1f1
component, depends on the continuity of the image.
The role of region separation in figure ground interactions: ROI-based analysis
Another way of assessing the role of discontinuities in determining the figure/ground interaction is to introduce static, untextured gaps between the figure and background regions. The addition of mean luminance gaps removes local spatio-temporal discontinuities in the image and by varying the size of the gap, we can ask how critical local features such as junctions are to the generation of figure- ground interaction. Previous low-channel count studies have shown that the spacing between the inputs is a critical determinant of the strength of the second-order nonlinear interactions between textured regions (Norcia et al., 1999
; Victor & Conte, 2000
; Zemon & Ratliff, 1982
). Here we asked whether region interactions of different nonlinear orders have a similar, or different, ability to span untextured gaps between regions in the different ROIs. To do this, we used a 3- phase-defined form that was either contiguous with the background or was separated by one of 5 mean gray gap sizes (see Spatial separation (gap) variations section). We also included a continuously segmented condition (identical to that the phase-defined condition described above) that was constructed with the same temporal structure and local textures and differed only in the particular conjunctions of features across the figure/background border. The effect of region separation was evaluated at the second-harmonics of both tags and their second- and fourth-order sums.
shows projected current density magnitudes as a function of gap size (arcmin) in the different ROIs. Each column shows the gap function for a different response component. Error bars indicating 1 SEM across observers are included. Plotted on the far right of each graph are data for the continuously segmented conditions. Effects of gap size were tested by comparing the zero gap and 60 arcmin gap conditions. Figure-region responses (2f1) across all ROIs show flat functions of gap size (gap effect: 3.457, p = 0.1) whose asymptotic levels (between gap sizes of 10-60 arcmin) are similar to those estimated when the figure remains continuously segmented. Background gap functions decline gradually (gap effect: F(1,8) = 11.77, p = 0.009), but this is expected because the gap deletes texture from the background region. The introduction of gaps between the figure and background region reduced the level of nonlinear interaction between regions ( third and fourth columns). This effect was larger for the 1f1 +1f2 term (F(1,8) = 26.642, p < 0.001) than for the 2f1 +2f2 term (F(1,8) = 3.526, p = 0.097 1.525, p = 0.252). Although there appears to be some level of gap tuning for the 2f1 +2f2, this effect did not reach statistical reliability due, in part, to the lower signal-to-noise ratio of this term (note differences in scale max at bottom of each row).
Figure 6 Gap Functions are shown for each ROI at the second harmonic of each tag, and at their second- and fourth-order sums. In each panel, ROI projected amplitude is plotted as a function of gap size. Data points for the constant segmentation stimuli are indicated (more ...)
The gap tuning functions were similar in the different ROIs (1f1 +1f2 ROI*GAP: F(7,2) = 2.402, p = 0.325; 2f1 +2f2 ROI*GAP: F(1,8) = 2.045, p = 0.367), as they were for the figure (ROI*GAP: f(7,2) = 0.4, p = 0.848) and background region responses (ROI: ROI*GAP: F(7,2) = 2.439, p = 0.321).
The asymptotic levels of response measured at the largest gap size were similar to those measured in the constantly segmented condition run in the same session. There was no measurable response for either the 60-min gap or the constantly segmented condition (open circles) for the 1f1 +1f2 term, but there was a measurable response for both of these conditions at the 2f1 +2f2 term. Thus, although both interaction terms depend on gap size, the fourth-order term remains present for large gaps and is less affected by the segmentation status of the figure and background regions than is the 1f1 +1f2 term, as was seen in (4th row) and 4 (right column).
Timing differences across the cortical surface: Surface-based averages
The SSVEP, as we measure it, is a complex-valued quantity with both a magnitude and phase. Response phase is related to the relative delay of the response over the cortex and thus provides indirect information about response timing. Just as response amplitude can be visualized as a map at the sensors or on the cortical surface, so too can phase information, although the phase variable is circular (modulus 2pi).
We visualized the average cortical surface phase distributions and complex-valued ROI responses of the phase-defined form stimulus in . Phase maps were computed from the 13 observer, sensor-space averages of for the figure (A), background (B), and low-order sum (C and D) components. These maps are displayed as unthresholded phase distributions shown from posterior and lateral perspectives. The color wheel codes the response phase according to the following convention: the time origin (0 phase lag with respect to the stimulus) is coded as purple. Increasing delay is coded as positively increasing values (purple to blue to green at +180°) and decreasing lag is coded in the direction of purple to red to green at -180°. Note that phase values of many regions of the maps are indeterminate: in the absence of a driven response, phase at these map locations will vary randomly. To provide a visual indication of which regions of the phase maps are interpretable, we provide the thresholded current density maps (from ) to the right for reference.
Differences in the phase distributions are present across the cortical surface at each frequency. Phase delays for the figure response show a continuous gradient of increasing delay extending from the occipital pole (green) to the lateral aspects of the occipital cortex (orange). The phase map for the background response has a relatively homogenous distribution over the medial and dorsal aspects of the occipital cortex (red) with a steep gradient extending laterally (red to blue). Phase distributions for the second-order sum term are also continuous, with the lateral cortex responding at an increased delay (orange) with respect to the occipital pole (blue). We also examined response phase in the individual observer averages for the V1, V2d, V3d, V3a, V4, MT+, LOC, and TOPJ ROIs. Here we plotted the magnitude and phase of the vector average of the individual observer responses for each ROI. The same phase convention described by the color wheel was used for the ROI response phase and the dispersion ellipses represented the 95% confidence interval of the mean.
From this analysis, we see that the response phases of V1, V2, V3, and V3A figure region responses at 2f1
are similar and that the phases of the figure responses in LOC, MT, and TOPJ are lagged by approximately 45-, or about 20 ms in equivalent latency. The 2f2
background responses of the first-tier areas also cluster at similar values. Background responses are present in the MT, but not in LOC or TOPJ ROIs, as we reported previously (Appelbaum et al., 2006
). The phase of the background f2
response in the MT ROI is nearly 180° different from that in the first-tier areas and is thus difficult to interpret as difference in delay because 180° phase flips can occur due to tissue orientation effects or due to properties of the inverse method. Interaction terms show a clustering of first-tier areas at one phase with responses in the LOC, MT, and TOPJ ROIs at a range of phases that differs by about 145°.
Beyond providing information about the propagation of responses through the cortical network, phase information allows us to assess potential artifacts in the source inverse. Early studies of the minimum norm method reported that “ghost” images of a single source were sometimes observed (Valdes-Sosa, Marti, Garcia, & Casanova, 1996
). The phase of the ghost image of a single source would necessarily have a direct relationship to the phase of the primary source. This is not observed in our data. Distinct phase values are seen in the different ROIs, indicating that multiple sources are present in first-tier and lateral cortical areas. A detailed analysis is provided in Supplementary materials, Section 4
Suppressive effects of differing background contexts
The stimuli we use resemble those that have been used to study center-surround interaction in visual cortex. These interactions have generally been found to be suppressive (Allman, Miezin, & McGuinness, 1985
; Blakemore & Tobin, 1972
; Nelson & Frost, 1978
) but can be facilitative under a more limited range of stimulus conditions (for a review, see Angelucci & Bressloff, 2006
). By comparing the figure only responses to the orientation-defined form response at harmonics of the figure frequency, we can obtain another measure of figure-ground interaction; surround suppression (the figure only version of the phase-defined form stimulus was not tested). Similar analyses have been performed previously for different two input stimulus configurations using the SSVEP (Hou et al., 2003
; Zemon & Ratliff, 1982
displays the topographic distributions and peak amplitudes for the second and fourth harmonics of the figure tag (2f1 and 4f1) for each condition. For each harmonic, maps across conditions are on the same scale and peak amplitudes in microvolts are indicated. Adding a changing orientation surround of either type (changing segmentation or constant segmentation) reduced the figure-region response by about 40% at the second harmonic and by about 10% at the fourth harmonic.
Figure 8 Figure region response distribution at the second (top) and fourth harmonic (bottom) under three background contexts. Response maxima are indicated above each map, and maps for each row are on the same scale. Two stimulus frames for each condition are (more ...)