Behavioral Performance in Experiment 1 (Natural Saccades)
Seven healthy volunteers performed a one-back word recognition task while reading words presented foveally after horizontal eye movements 10 degrees apart (). Both novel words (50% of trials) and one repeated word were presented for 250 ms either early (76 ms) or late (643 ms) after saccade detection in real time using the electrooculogram (see Methods). The selection of these time windows allowed us to control the duration of word viewing at the fixation following a saccade, ensuring that, in each individual, words appeared either immediately after the end of the saccade (range 2 – 72 ms, 31.5 ± 3.3 ms), when we expected postsaccadic influences, or ~600 ms later, when we expected these influences to fade. Subjects’ task was to determine whether the word at fixation was the same as that in the previous trial (10% match-trials) or different (90% non-match trials). High overall accuracy scores were observed in both match (average percent correct = 80.06%) and non-match trials (99.79%). All further analyses were based on the large number of non-match correct trials that were free of blinks and other artifacts not related to eye movements.
Percent correct scores and reaction times (RTs) were analyzed with repeated-measure three-way ANOVAs with within-subject factors of word presentation time (early or late after saccades), word type (novel or repeats), and saccade direction (right or left). There were no main effects or interactions on accuracy (all F(1, 6)s < 4.1, p’s > 0.05). illustrates average RTs for individual conditions. RTs were slower when words appeared early vs. late after a saccade, F(1, 6) = 22.01, p < 0.004, suggesting that word recognition is overall temporarily impaired after an eye movement. Slower RTs were also observed for right vs. left saccades, F(1, 6) = 19.41, p < 0.005, with an additional interaction between saccade direction and word presentation time, F(1, 6) = 8.9, p < 0.007. This stronger modulation for right saccades is likely to have arisen because words appeared on average 21 ms earlier after the end of right than left saccades (range 14 – 30 ms, see Methods). In addition, slower RTs were found for repeated vs. novel words, F(1, 6) = 50.32, p < 0.0004.
Behavioral Performance in Experiment 2 (Background Movement)
In parallel experiments, the same subjects read words presented foveally during constant fixation, either early (59 ms) or late (626 ms) after the end of background movement that mimicked saccades (). As above, we found high overall accuracy scores for both one-back match (80.16%) and non-match trials (99.61%). Further analyses were again based on non-match correct trials (90% of all trials) that were free of artifacts. Accuracy scores and RTs were analyzed with repeated-measure three-way ANOVAs with within-subject factors of word presentation time (early or late after background movement), word type (novel or repeats), and movement direction (right or left). We found that both accuracy and RTs varied across conditions. Accuracy was worse to words presented early vs. late after movement, F(1, 6) = 28.01, p < 0.002 (trials with incorrect responses < 2.5% for individual conditions). Similar to the saccade task, RTs were slower when words appeared early vs. late after movement (), F(1, 6) = 50.92, p < 0.0004, indicating that saccade-like retinal motion disrupts subsequent word processing. Here too, slower RTs were found for repeated vs. novel words, F(1, 6) = 24.01, p = 0.0027. An additional interaction was found between word type and word presentation time, F(1, 6) = 8.39, p < 0.008, reflecting larger background movement effects on repeated vs. novel words. There was no main effect of movement direction (p > 0.05).
Comparison of behavioral performance in Experiment 1 and Experiment 2
To directly compare the behavioral effects of natural saccades and background movement (measured as changes in RTs for early minus late word presentation conditions), we performed a three-way repeated-measure ANOVA with factors of experiment, word type and image movement direction. There was no main effect of experiment (p = 0.49), indicating similar increases in RTs to words presented early vs. late after saccades (74.8 ms) and background movement (65.3 ms). Consistent with the results reported above, we found a significant interaction between experiment and movement direction (F(1, 6) = 10.34, p < 0.004) reflecting the slower RTs after right vs. left natural saccades, but no such effects of direction in Experiment 2. This analysis also confirmed the slower RTs on repeated vs. novel words, F(1, 6) = 6.73, p < 0.041.
Previous studies report changes in spontaneous activity and visual responses related to central effects around the time of saccades in early visual areas (Sylvester et al, 2005
; Royal et al., 2006
; Rajkai et al., 2008
; Cloherty et al., 2010
), which could modulate the variability in RT to words following saccades, rendering it different from that following background movement (Burr et al., 1994
; Diamond et al., 2000
; Watson and Krekelberg, 2011
). We compared the RT standard deviations using a repeated-measure four-way ANOVA with within-subject factors of experiment, latency, word type and image movement direction. There was no main effect of experiment (F(1, 6) = 2.19, p = 0.19) or interactions (p’s > 0.05), indicating similar variability in RTs after saccades and background movement.
Estimated Cortical Activity Patterns in Experiment 1 (Natural Saccades)
Average MEG waveforms were computed for each condition in each subject. illustrates a representative example of responses to novel words presented early and late after the end of right saccades. Averages from no-stimulus trials illustrated activity generated by saccades alone as well as the eye-movement related artifacts (see Methods). This latter waveform was subtracted from the waveform in each condition (), resulting in a differential signal that reflects responses to words alone. Due to the stereotypical nature of eye movements across trials, this approach effectively eliminated the eye-movement related artifacts.
This differential MEG signal was further analyzed to estimate the patterns of cortical activity across locations and time using a distributed source modeling approach that constrained current sources to the cortical surface of each participant reconstructed from structural MRI (Dale et al. 1993
). Noise-normalized dynamic statistical parametric maps (dSPMs) (Dale et al., 2000
) were computed for individual conditions to evaluate the statistical significance of estimated responses relative to pre-stimulus baseline activity. illustrates snapshots of average dSPMs across subjects, at selected latencies after word presentation. The evolution of the activity patterns was consistent with that reported in previous MEG studies of visual word processing (Dhond et al., 2001
; Marinkovic et al., 2003
). Briefly, activity began in occipital pole (peak at ~95 ms), and subsequently spread anteriorly within the ventral visual stream, recruiting the occipitotemporal and posterior superior temporal regions, followed by ventral and anterior temporal cortices, and prefrontal cortex (early peaks between ~140–155 ms). Subsequent peaks of left lateralized activity were found in occipitotemporal cortex at ~170 ms, in anterior temporal cortex at ~210 ms, and within the 300–500 ms (N400) time window, distributed across anterior occipitotemporal, anterior temporal and prefrontal cortices. For words presented early vs. late after saccades, we observed a prominent reduction in the response within multiple regions, at multiple stages of cortical processing, beginning with the earliest stage in occipital pole. These reduced responses, as well as repetition effects, were further quantified in regions of interest (ROIs), as described below.
Average Dynamic Statistical Parametric Maps (dSPMs) in response to novel words presented after right saccades in Experiment 1, and after left background movement in Experiment 2
Regions of Interest Analysis: Postsaccadic Effects (Early vs. Late Word Presentation)
ROIs were selected on the basis of previous studies of visual word processing and they all exhibited large activity values here (see Methods). Regional time-courses of estimated currents were computed in each ROI and individual subject, by averaging values across all voxels. illustrates regional time-courses averaged across subjects for early vs. late word presentation time. For each ROI, effects were evaluated in response windows selected a priori based on previous MEG and intracranial studies of the time course of visual word recognition (see Methods). Relevant time windows included 80–120 ms for early visual processing in occipital pole, 120–160 ms for the earliest response phase in downstream ROIs, 165–215 ms for orthographic processing and word-form access in occipitotemporal cortex, 190–240 ms for lexical processing and 300–500 ms (N400) for semantic processing in distributed language networks including inferior and superior temporal and prefrontal cortices. For an individual ROI and time range of interest, fixed across subjects, a single repeated-measure three-way ANOVA with within-subject factors of word presentation time (early or late), word type (novel or repeated) and saccade direction (right or left) was performed to compare currents across conditions. The detailed comparisons of all individual tests for the main effects of word presentation time and word repetition are presented in . Effects of word presentation time, i.e. post-saccadic effects, are summarized in the text below.
Average time courses of estimated currents in selected cortical regions, in response to words presented early vs. late after saccades (A), and after background movement (B)
Postsaccadic effects were similar for right and left saccades, and also for novel and repeated words (e.g., ). For the repeated word condition, we repeatedly presented a single word either early or late after right and left saccades in order to reliably assess postsaccadic effects in early visual areas that are sensitive to the visual attributes of the stimulus.
Differential effects of saccades and background movement
In occipital pole, saccades diminished both the earliest (80–120 ms) as well as later phases of the response (e.g., 120–160), suggesting an overall suppression of activity to words entering the fovea via saccades (, ). Attenuated activity from 120–160 ms was found in bilateral ventral occipitotemporal (vOT) junction, bilateral occipitotemporal (OT), right anterior inferior temporal (IT), bilateral anterior superior temporal sulcus (STS), right posterior STS, and bilateral inferior Sylvian fissure (SF); a similar trend was observed in left anterior IT, left posterior STS, right planum temporale, and left inferior prefrontal cortex (IPFC).
Unlike the strong effects at earlier processing stages, activity in OT cortex (both fusiform area and lateral occipitotemporal sulcus) from 165–215 ms was reduced only at trend level (p < 0.1). In downstream areas, reduced activity from 190–240 ms occurred in the left hemisphere in anterior STS, inferior SF, and at trend level in IPFC; and in the right hemisphere in posterior STS and planum temporale. Reduced activity from 250–350 ms was found at trend level in left planum temporale, during a pronounced, left-lateralized response component; and also in right posterior STS. Activity from 300–500 ms (N400) was attenuated within a distributed temporofrontal network including bilateral OT and anterior IT, right anterior STS, right posterior STS, right inferior SF and right IPFC. Together, these results demonstrate postsaccadic effects of variable strength in both visual and higher cortical areas during early and late phases of the word-evoked response.
Regions of Interest Analysis: Repetition Effects (Novel vs. Repeated Words)
To evaluate effects of repetition, we focused on response windows that have been previously associated with repetition priming, from 190–240 ms, 240–300 ms and 300–500 ms (see Methods); we also report results at earlier spatiotemporal stages of the response described above (). Average responses across subjects to novel vs. repeated words in illustrate robust repetition effects left-lateralized in occipitotemporal, anterior temporal, superior temporal and prefrontal regions, within the same ROIs that revealed significant postsaccadic modulation. Below we evaluate early (< 300 ms) and late (300–500 ms) repetition effects.
Average time courses of estimated currents in selected cortical regions, in response to novel vs. repeated words presented after saccades (A), and after background movement (B)
We found early repetition enhancement (repeats > novel) from ~120–160 ms in left OT cortex, with a similar tendency in occipital pole, left vOT junction and anterior temporal regions. This was followed by repetition suppression (novel > repeats) from 190–240 ms in bilateral planum temporale and left inferior SF. Robust left-lateralized repetition suppression from ~240–300 ms occurred in left OT, left anterior IT, left anterior STS, and left inferior SF, as well as bilaterally in planum temporale.
Repetition suppression extended to 300–500 ms response range in anterior temporal and inferior prefrontal regions. These late effects were of smaller magnitude and occurred relatively earlier (e.g., peaks at ~350 ms) compared with previous reports. Significance was evaluated within four 50-ms windows from 300–500 ms, and was found in left anterior STS (300–400 ms), left inferior SF (300–400), and left IPFC (300–400 ms). In addition, from 400–450 ms repetition enhancement (repeat > novel) was found in right anterior IT. No significant interactions were found between postsaccadic and repetition effects. Overall, these results indicate co-localization of postsaccadic and repetition effects previously associated with lexical and semantic processing.
Estimated Cortical Activity Patterns in Experiment 2 (Background Movement)
illustrates average MEG waveforms to novel words presented early and late after background movement for the same subject as in 2B. Similar to the analysis in Experiment 1, waveforms from no-stimulus trials that reflect activity generated by movement alone were subtracted from the waveform in each condition; this resulted in differential signals that reflect responses to words alone (). Estimated activity patterns were qualitatively similar to those in the saccade task (). Again, we found reduced activity for words presented early vs. late after background movement, consistent with a contribution of retinal image motion to the overall suppressive postsaccadic effects.
Regions of Interest Analysis: Background Movement Effects (Early vs. Late Word Presentation)
Similar to the saccade task, diminished responses, although varying in magnitude, were observed for words presented early vs. late after background movement (, ). In occipital pole, background movement attenuated the earliest phase of the response from 80–120 ms although, unlike saccades, not later activity. From 120–160 ms we found reduced activity in bilateral vOT junction, left OT, left anterior IT, bilateral posterior STS, and right inferior SF, with a similar trend in left anterior STS.
Significant attenuation from ~165–215 ms was found in left OT cortex. In downstream areas, reduced activity from 190–240 ms was seen in left anterior IT, bilateral anterior STS, right inferior SF, and bilateral IPFC; reduced activity in this time range also occurred in bilateral posterior STS. Effects were found also from ~250–350 ms in the left planum temporale and posterior STS, as well as in the right posterior STS. Similar to the saccade task, attenuated responses from 300–500 ms occurred within a distributed temporofrontal network including bilateral OT and anterior IT, bilateral anterior STS, right posterior STS, bilateral planum temporale, right inferior SF, right IPFC. Overall, these results indicate that the visual effects of image motion are carried over from early visual to language areas, and likely contribute to postsaccadic suppression.
Regions of Interest Analysis: Repetition Effects (Novel vs. Repeated Words)
The spatiotemporal pattern of repetition effects in the background movement task was similar to that in the saccade task. illustrates left lateralized repetition effects that co-localized with background movement effects in occipitotemporal, anterior inferior and superior temporal, posterior temporal and planum temporale, and prefrontal regions. The detailed comparisons of all individual tests are presented in .
Comparison of Effects on Word Responses between Experiment 1 and Experiment 2
While diminished responses were found after both saccades and background movement, the degrees of response change varied significantly across experiments and cortical regions. We compared effects of saccades and background movement in three selected ROIs, including occipital pole, ventral occipitotemporal junction and occipitotemporal cortex (). Response attenuation was significantly larger in the saccade than background movement experiment in occipital pole from 80–120 ms (early/late responses, 65.9% vs. 83.4%, F(1, 6) = 16.7, p < 0.007), and in vOT junction from 120–160 ms (66.6% vs. 83.4%, F(1, 6) = 19.83, p < 0.0043). Unlike these earlier effects, the degrees of response modulation in left OT cortex were similar in the two experiments.