PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of scirepAboutEditorial BoardFor AuthorsScientific Reports
 
Sci Rep. 2017; 7: 43573.
Published online 2017 March 6. doi:  10.1038/srep43573
PMCID: PMC5337927

The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments

Abstract

Predictive coding is an influential model emphasizing interactions between feedforward and feedback signals. Here, we investigated the temporal dynamics of these interactions. Two gray disks with different versions of the same stimulus, one enabling predictive feedback (a 3D-shape) and one impeding it (random-lines), were simultaneously presented on the left and right of fixation. Human subjects judged the luminance of the two disks while EEG was recorded. The choice of 3D-shape or random-lines as the brighter disk was used to assess the influence of feedback signals on sensory processing in each trial (i.e., as a measure of post-stimulus predictive coding efficiency). Independently of the spatial response (left/right), we found that this choice fluctuated along with the pre-stimulus phase of two spontaneous oscillations: a ~5 Hz oscillation in contralateral frontal electrodes and a ~16 Hz oscillation in contralateral occipital electrodes. This pattern of results demonstrates that predictive coding is a rhythmic process, and suggests that it could take advantage of faster oscillations in low-level areas and slower oscillations in high-level areas.

The outside world provides us only the light, but our visual system is capable of extracting the basic features in low-level areas and understanding them as meaningful concepts in high-level areas. Interactions between these different areas involve both feedforward and feedback processes. However, the underlying mechanisms in these dynamical processes, especially in the temporal domain, are not fully understood.

Predictive coding is a model emphasizing interactions between feedforward and feedback signals. Specifically, it suggests that the brain employs an efficient coding strategy for these interactions by generating predictions in higher-level areas and comparing them with the incoming sensory signals in the lower-level areas1. Previous neuroimaging evidence revealed the existence of such two-way communication2; however, different computational models of predictive coding have suggested very different details of implementation1,3,4,5.

It has been proposed that the feed-forward and feedback in predictive coding could take advantage of oscillations for information processing6,7. On the one hand, recent neurophysiological evidence on laminar-specific oscillations and the functional roles of different layers suggested a faster oscillation for the feed-forward pathway and a slower oscillation for the feedback pathway6,8,9,10,11. On the other hand, recent studies showed a link between behavioral performance and cortical oscillations in perception12,13 and reaction time14,15. Since neural oscillations can reflect the cyclic fluctuations of excitability in a network, investigation of the relationship between trial-to-trial variability and the phase of ongoing oscillations could link specific oscillations to cognitive functions (e.g. attention).

Here, we used this approach to investigate the specific influence of ongoing oscillations on the feed-forward/feedback interactions in predictive coding, by measuring their effect on perception for different pre-stimulus oscillatory phases. A minimal definition of “predictive coding” encompasses any stimulus-driven higher-level activity that modulates, in a feedback manner, the lower-level sensory processing of the same stimulus. In a typical predictive coding experiment, therefore, two conditions must be compared, which differ in the amount or the type of predictive feedback signals. Here, we chose one of the first experimental manipulations used in predictive coding to generate different amounts of predictive feedback16: shape perception.

Specifically, 3D-shape outlines and random-lines versions of the same stimuli, similar to the stimuli used in a previous influential study16, were used in this experiment. It has been shown that 3D-shape outlines can be easily recognized (presumably activating higher-level areas) and thus produce more (or at least, different) predictive feedback than the random-lines versions16. To measure the effect of different amounts of predictive feedback, we asked the subjects to judge the luminance (report the side of the brighter disk) of two gray disks simultaneously displayed on the left and right of fixation on a black background, one containing the 3D-shape outlines and the other containing the random-lines version. Our previous study showed that the 3D-shape (which generates more predictive feedback16) increases the subjective luminance of the disk relative to the random-lines version17. Thus, perceived luminance can be used as a marker for the efficiency of predictive feedback on every trial. Here, the luminance of the disks was adjusted to achieve about 50% choice rate for 3D-shape vs. random-lines disks for all trials (this was achieved by slightly increasing the luminance of the random-lines disk). With our knowledge of the predictive feedback’s effect on luminance judgements17, we can predict that in trials where predictive feedback efficiency is higher than average, subjects would be more likely to report the 3D-shape disk as brighter; and vice versa, when predictive feedback efficiency is lower than average, subjects would more likely report the random-lines disk as brighter. We recorded EEG signals and analyzed the relationship between pre-stimulus oscillation phase and the post-stimulus judgment. If predictive feedback mechanisms during the post-stimulus stage involved one or more periodic processes, then the oscillatory state of the system just before stimulus onset (as reflected in pre-stimulus oscillatory phase) should have an influence on the outcome of predictive feedback, and thus on the perceptual judgment. We found that, independent from the spatial choice (left/right side), the phase of 5 Hz contralateral frontal and 16 Hz contralateral occipital pre-stimulus oscillations modulated the subject’s choice of a brighter 3D-shape disk (more effective predictive feedback) or a brighter random-lines disk (less effective predictive feedback). Since in the predictive coding framework, higher hierarchical level areas are assumed to send predictive feedback and lower hierarchical level areas to send prediction error signals, our results could imply that in the post-stimulus stage, the brain sends predictive feedback periodically at a preferred phase of a theta frequency oscillation in the frontal region, and sends prediction errors periodically at a preferred phase of a beta frequency oscillation in the occipital region.

Results

Human observers judged the luminance of two disks that were presented for 150 ms on the left and right of a central fixation point. The disks contained different versions of the same stimulus, one with a 3D-shape enabling predictive feedback, and the other with a random-lines version of the same shape which impeded predictive feedback (Fig. 1). Before the stimulus onset there was a random period of time (1000–1500 ms) with only the fixation point on the screen. After the stimulus offset a question mark appeared at the center and the subjects were instructed to report the side with the brighter disk. In the main experimental trials, the luminance of the disks was adjusted, such that observers reported the 3D-shape disk as brighter in half of the trials. 15% of trials were catch trials: extreme luminance values were assigned to one disk to monitor the subject’s ability to judge the luminance difference.

Figure 1
Illustration of the experimental paradigm.

Behavioral Results

On average, subjects judged the 3D-shape disk as brighter in half of the trials (49.24% ± 1.57%, mean ± standard error of the mean, SEM) in the main experimental condition, as expected. The luminance judgment correct rate in the catch trials (subjects judged the disk with higher luminance value as brighter or judged the disk with lower luminance value as darker) was high (93.98% ± 1.83%, mean ± SEM), indicating that subjects were adequately engaged in the luminance judgment task.

Electrophysiological Results

We focused on the relationship between the oscillatory phase and the trial-by-trial variations in the efficiency of post-stimulus predictive coding. EEG was recorded during the experiment. Regarding the expected time window of such a relationship, the post-stimulus phase information is driven to a large extent by stimulus-locked activity (e.g. evoked potentials) and is thus further removed from spontaneous activity. On the other hand, since a spontaneous oscillation is a continuous process, pre-stimulus phase information is likely to be related to the oscillatory phase at the (post-stimulus) time point at which predictive coding actually happens. Therefore, the pre-stimulus phase can be a better measurement of the relevant state of a spontaneous oscillation than the post-stimulus phase. Thus, we expected the relation between oscillatory phase and behavior to be most visible in the pre-stimulus time window, where phase information reflects spontaneous fluctuations in neuronal excitability. We used classical stimuli16 for inducing different amounts of predictive feedback on the left and right of the screen: 3D-shape and random-lines versions of the same stimuli (Fig. 1; the 3D-shape version enabling predictive feedback, the random-lines simultaneously impeding it, as described in Murray et al.16). To measure the effective amount of post-stimulus predictive feedback on each trial, we probed the perceived luminance of the disks under the stimuli. We have previously demonstrated that the 3D-shape (producing more predictive feedback, as described by ref. 16) results in a relatively higher perceived luminance than the random-lines17. Here, this net effect was compensated on each trial by slightly lowering the luminance of that disk so that the average likelihood of perceiving either disk brighter was about 50% (see Methods). Therefore, residual fluctuations of luminance perception on every trial can be thought to arise from trial-by-trial fluctuations in the efficiency of post-stimulus predictive coding: the 3D-shape disk may still be perceived brighter on trials where predictive coding was more efficient than average, and darker on trials where it was less efficient than average. Of course, spatial bias and/or trial-by-trial fluctuations in the direction of spatial attention may well also contribute to the choice of which disk appears brighter on a given trial. Thus, for each subject we divided all trials into two datasets based on their spatial choice (left-side choice vs. right-side choice), and we only investigated the relation between pre-stimulus EEG phase and 3D-shape/random-lines choice within each dataset. As the correlates of choosing the prediction-consistent stimulus (3D-shape) were expected to be strongest on electrodes contralateral to that stimulus, which would map onto opposite hemispheres for the two datasets, before plotting any scalp topographies we permuted the electrode positions (symmetrically across the midline axis) of all right-side choice trials. This procedure resulted in a mapping of ipsilateral effects to the spatial choice onto left electrodes, and contralateral effects onto right electrodes.

We estimated the relation between EEG phase and predictive coding via the phase opposition product (POP, see Methods). This measure should be maximal when 3D-shape choice trials (prediction-consistent) and random-lines choice trials (prediction-inconsistent) tend to have opposite phase values. For each subject, dataset, electrode, time point and oscillatory frequency, we obtained surrogate POP values (80,000,000 surrogates) by randomly permuting the trial outcomes, keeping the number of trials constant. Both real and surrogate POP values were averaged across datasets and subjects. The significance was determined as the proportion of surrogate POP values that were more extreme than the observed value. P-values were corrected for multiple comparisons across time points, frequencies and electrodes (100 × 30 × 64) using the FDR method (FDR α = 0.05, corresponding to a P value threshold of 9.53 × 10−6). To show the overall POP in the time-frequency domain, a z-score was computed by comparing the real POP values (combined across all subjects, datasets, and electrodes) to the mean and standard deviation of a null-hypothesis distribution with 10,000 surrogate POP values (generated using the same procedure described before, and also combined across all subjects, datasets, and electrodes). This analysis revealed a significant phase-behavior relation between the post-stimulus 3D-shape/random-lines choice and two pre-stimulus oscillations (Fig. 2A): one theta-frequency oscillation (~3.1 Hz to 7.6 Hz) in the time window from −545 ms to −268 ms, and one beta-frequency oscillation (~13.2 Hz to 25.7 Hz) in the time window from −107 ms to −25 ms. Green outlines mark the significant time-frequency regions (at least one significant electrode) after FDR correction.

Figure 2
Pre-stimulus EEG phase predicts luminance judgment of 3D-shape disk vs. random-lines disk.

Scalp topographies of the z-score show that the two oscillatory effects involve distinct electrode groups and presumably distinct brain regions (Fig. 2B and C): the theta-frequency effect is maximal over frontal regions and the beta-frequency effect over occipital regions. In both cases, these effects are contralateral to the side that subjects chose as “brighter” (i.e., the right side of the topographies, due to our electrode permutation procedure). Electrodes with at least one significant time-frequency point (after FDR correction) inside the corresponding time-frequency window are highlighted in green.

To quantify the influence of pre-stimulus oscillations on post-stimulus choice, we binned single trials according to the phase at the optimal time-frequency point (for the theta oscillation: −397 ms, 5.4 Hz; for the beta oscillation: −68 ms, 16.5 Hz). Single trials were thus sorted in 13 phase bins based on the average phase of the significant electrodes for each oscillation (four frontal electrodes for the theta oscillation, three occipital electrodes for the beta oscillation). For each phase bin we then computed the post-stimulus choice probability of the 3D-shape disk. These choice probabilities were normalized by dividing them by the overall 3D-shape choice probability across all phase bins. For each experimental dataset (left- vs. right-side choice), phase bins were rotated such that the phase at which 3D-shape disk choice probability was largest was aligned to a phase angle of zero. As a result of this alignment, the 3D-shape choice probability is necessarily maximal at a phase angle of zero; therefore, the zero-phase bin was discarded from further analyses. For both frequencies, the 3D-shape disk choice probability monotonically decreased to a minimum at the opposite phase angle, confirming that pre-stimulus phase affected post-stimulus judgment (Fig. 3). A one-way ANOVA showed that both pre-stimulus theta phase and pre-stimulus beta phase significantly modulated the 3D-shape disk choice probability (for theta oscillation, F(11, 27) = 3.95, p = 2.23 × 10−5; for beta oscillation F(11, 27) = 6.17, p = 3.86 × 10−9). The magnitude of each effect was determined as the difference between the maximum and minimum 3D-shape disk choice probabilities across all phase bins. The frontal theta oscillation accounted for a difference of ~14% of the 3D-shape disk choice probability between phase bins, and the occipital beta oscillation accounted for a difference of ~19%.

Figure 3
Normalized choice probability of 3D-shape disks as a function of pre-stimulus phase.

Because the time-frequency analysis relies on signal convolution with wavelet filters whose duration is non-negligible, one might wonder whether the observed pre-stimulus phase differences could actually be driven by stimulus-evoked activity. For example, at 16 Hz the above time-frequency analysis used a 250 ms time window (4 cycles, 125 ms from the past and 125 ms into the future); thus, significant phase effects observed at −67 ms pre-stimulus may be contaminated by post-stimulus activity. To rule out such contamination, we repeated the POP time-frequency analysis with one-cycle wavelets at all frequencies, and compared the timing of pre-stimulus phase effects with the time-frequency region of possible post-stimulus contamination, determined using the wavelet window length at each frequency (Fig. 4). Both theta- and beta-frequency phase effects were replicated in this analysis, and were found to lie outside of the possible post-stimulus contamination zone.

Figure 4
Significance of POP values in a one-cycle wavelet analysis.

We also ascertained that phase effects were not caused by any eye movement artifacts that may have survived our artifact rejection procedure. For example, the observed pre-stimulus phase differences could be thought to reflect different patterns of eye blink or saccades for different perceptual outcomes. Therefore, we applied our POP time-frequency analysis to the horizontal and vertical EOG (HEOG and VEOG) signals. P-value maps (obtained by comparison of POP values against 80,000,000 surrogates) did not reveal any signs of systematic eye movements in either the theta- or the beta-frequency bands (Fig. 5), ruling out an explanation of our pre-stimulus phase effects in terms of ocular artifacts.

Figure 5
Significance of the POP values of VEOG and HEOG signals.

Discussion

We investigated the temporal dynamics of predictive coding by exploring the relation between pre-stimulus oscillatory phase and the presumed trial-by-trial variations in post-stimulus predictive feedback. We used 3D-shape outlines and random-lines versions of the same stimuli (as in one of the seminal predictive coding studies16) to induce different amounts of predictive feedback (Fig. 1), and measured the corresponding effects on luminance judgment as trial-by-trial markers of the efficiency of predictive coding. (To reiterate our experimental logic: Feedforward processing of the 3D shape likely induced predictive feedback, which increased the perceived brightness of the 3D-shape disk. The more efficient this predictive coding, the larger the effect of predictive feedback, and the brighter the 3D-shape disk should be perceived.). Using a similar analysis method as in well-established studies of the relationship between pre-stimulus phase and post-stimulus behavior18,19, we found that two pre-stimulus ongoing oscillations from different regions and frequencies could strongly influence the post-stimulus luminance judgment, and thus the post-stimulus predictive coding efficiency: a contralateral frontal theta oscillation and a contralateral occipital beta oscillation (Fig. 2). The phase of the theta oscillation before stimulus onset could explain 14% of the luminance judgment difference while the phase of the beta oscillation could explain 19% (Fig. 3). Control analyses ruled out contamination of the phase-behavior relationship by post-stimulus activity (Fig. 4) or ocular artifacts (Fig. 5). These results not only imply that the feedforward/feedback interactions in predictive coding involve periodic processes, but also reveal two periodicities with different sources. Since the occipital and frontal signals likely reflect activity from hierarchically lower and higher areas, respectively, and since predictive coding theory suggests that the brain sends back predictions from higher areas and sends prediction errors from lower areas, our results allow us to speculate on a possible temporal dynamic for predictive coding: predictions sent periodically at a theta frequency, prediction errors sent periodically at a beta frequency. Of course, more direct connectivity studies would be needed in the future to confirm these postulated roles for occipital and frontal signals.

The experimental paradigm used in this study takes advantage of the relationship between shape perception and predictive coding: 3D-shape outlines are assumed to generate more predictive feedback than the random-lines version of the same stimulus. Murray et al.16 used similar stimuli to provide one of the first evidence of predictive coding: compared to the random-lines, 3D-shape outlines increased activity in the lateral occipital complex (LOC), but decreased it in primary visual cortex (V1), suggesting an increase of predictive feedback accompanied by a decrease in prediction errors16,20. Here, we used the same paired stimuli as in the original study, placed them on two gray disks and asked subjects to judge the luminance of the disks. This luminance judgment, associated with perceived contrast, is likely to have a positive and monotonic relationship with neural activity in early visual cortex21,22,23,24. Thus, the variability in luminance judgment associated with the 3D-shape vs. random-lines stimuli could reflect trial-by-trial changes in the effect of predictive feedback on neural activity in early visual cortex.

Previous fMRI studies showed that shape perception could not only reduce16, but also up-regulate neural activity in V125. Our own previous study found that 3D-shape disks were generally perceived brighter than random-lines disks, and that this effect could be attributed to predictive coding rather than attentional biases17. In the present study, we compensated for this net effect by adjusting the disks’ luminance to obtain a 50% selection probability of 3D-shape/random-lines disks. (This operation suggested that predictive feedback has an excitatory effect on lower level activity, which is in contrast to many predictive coding models. This apparent discrepancy was already extensively discussed in our previous paper17. For the present purposes, it is sufficient to note that, if predictive coding is a rhythmic process, both an excitatory and a suppressive main effect of predictive coding would produce the same outcome: a periodic modulation of contrast judgments.) We then focused on the remaining variability in luminance judgement as a trial-by-trial marker of the efficiency of predictive coding. On the other hand, systematic spatial biases (e.g. a general tendency to respond to the left or right stimulus) and/or trial-by-trial fluctuations in the direction of spatial attention can also be expected to affect the luminance judgement26. As a matter of fact, spatial attention itself appears to involve a periodic process12,18,27,28 which could potentially influence the luminance judgement. We carefully avoided these potential confounding factors by dividing the trials into two datasets based on the post-stimulus spatial response (left/right) and performing the analysis within each dataset. If spatial attention biases, for example, were the only cause of the perceived luminance changes, the left-response dataset would pool all trials with a left-side attention bias (and similarly for the right-response dataset), and within each dataset pre-stimulus oscillatory phases would not bear any relation to post-stimulus luminance judgments. Of course, this analysis controls for spatial biases and changes in spatial attention, but cannot rule out object-based attention related to the shape of the object; however, it could be argued that this type of object-based attention is indistinguishable from (or even, identical to) the predictive feedback arising from the predictive coding framework. To sum up, the existence of significant phase-behavior relationships in our analysis can therefore be safely attributed to predictive coding mechanisms, rather than spatial attention or other spatial biases.

Neurophysiological recordings have shown that feedforward and feedback may take advantage of oscillations in different frequencies. Laminar recordings showed that high-frequency oscillations are prominently generated in superficial layers and low-frequency oscillations in deep layers9,10,29. Since superficial and deep layers correspond respectively to the main sources of forward and feedback projections30,31,32, it follows that feedforward communication takes advantage of high-frequency oscillations and feedback takes advantage of low-frequency oscillations. A recent study with simultaneous recordings and micro-stimulation in different layers in V1 and V4 confirmed this notion8. Several authors have independently proposed that low-frequency oscillations send predictions via feedback, while high-frequency oscillations send prediction errors via feedforward6,11,33,34,35,36. Even though the exact frequency bands (theta, beta) are not entirely consistent with those described in the previous studies (e.g. beta, gamma in ref. 11), our results still provide support for distinct functional roles of low and high frequency oscillations in predictive coding at the EEG and behavioral level.

Our results also provide supportive evidence for the hypothesized functions of frontal theta-band and occipital beta-band oscillations. Our conclusions are in line with the notion that 5–10 Hz oscillations could contribute to “top-down” control37,38, which has already been suggested based on attentional phase effects on perception12,18, reaction time14,15,39 and perceptual variability in TMS-induced effects13,40. We found the origin of such theta periodicity in contralateral frontal electrodes, compatible with the involvement of frontal areas in the top-down controlling process2,41,42 and with the involvement of 5–10 Hz oscillations in this region43. On the other hand, local field potential (LFP) recordings showed that, in mammalian visual cortex, beta frequency oscillations are also prominent during the deployment of top-down control44,45,46,47,48. Our findings of beta frequency phase effects on predictive feedback in the occipital area are concordant with such LFP results and suggest a valuable role for the beta frequency oscillations in predictive coding.

In summary, we measured the relation between pre-stimulus oscillations and a predictive feedback-induced effect to investigate the neural oscillations involved in predictive coding. We found that the pre-stimulus phases of frontal theta-frequency oscillations and occipital beta-frequency oscillations jointly determine post-stimulus subjective judgments. These results shed light on the temporal dynamics of predictive coding, and suggest a periodic predictive coding process with faster oscillations in lower areas and slower oscillations in higher areas.

Materials and Methods

Subjects

Fifteen volunteers participated in the experiment. One participant was excluded from the analysis due to the poor behavioral performance in catch trials (<60% trials were correctly reported, with a chance level of 50%, see below). Fourteen participants remained in the sample (8 females, mean age 28.01 ± 4.81 years, four left-handed, four with left eye dominance). All subjects had normal or corrected to normal vision. The study was approved by the local ethics committee “Sud-Ouest et Outre-Mer I” and followed the Code of Ethics of the World Medical Association (Declaration of Helsinki). All subjects provided signed informed consent before starting the experiments.

Apparatus

Stimuli were presented at 57 cm distance using a desktop computer (2.09 GHz Intel processor, Windows XP) with a cathode ray monitor (resolution: 800 × 600 pixels; refresh rate: 140 Hz, Gamma corrected luminance function). Stimuli were designed and presented via the Psychophysics Toolbox (Brainard, 1997) running in MATLAB (MathWorks).

Stimuli and tasks

Stimuli consisted of a central white fixation point (diameter: 0.2 degrees of visual angle) and two circular gray disks (diameter: 4 degrees each) presented randomly to the left and right of fixation (3 degrees eccentricity). One 3D-shape stimulus was in the center of one disk (3D-shape disk) and one random-lines version of the same stimulus in the other (random-lines disk). The 3D-shape and random-lines stimulus pair was randomly chosen from twenty pairs of stimuli generated beforehand using a method similar to Murray et al.16: 3D-shapes were generated by randomly selecting 4–6 vertices, connecting the vertices and adding small extensions to render perceived depth; random-lines stimuli were created by breaking the 3D-shape at its intersections and randomly shifting the lines within the display16. The diameter of both 3D-shape and random-lines stimuli was 3 degrees. The stimulus outlines were black.

Before stimulus onset, there was a blank screen with only the fixation point that lasted from 1000 to 1500 ms (random uniform distribution). Then the two disks and the fixation point appeared for 150 ms. After that, a question mark appeared in the center of the screen. There were two kinds of randomly mixed experimental trials: the main experimental trials and the catch trials. In main experimental trials, the luminance of the disks was adjusted (i.e. the random-line disks were set 1.45% brighter than the 3D-shape disks) based on a previous study17 to obtain an average 50% selection rate of 3D-shape/random-lines disks. In catch trials, one of the disks had its luminance value changed up or down by 20% compared to the luminance used in the main experimental trials, while the other disk kept the same luminance as in the main experimental trials. Subjects were presented with 4 or 8 blocks of 200 trials with 85% main experimental trials and 15% catch trials (the first 6 of the 14 subjects performed only 4 blocks of the present experiment, together with 4 blocks of another experiment that was eventually canceled and whose data were not analyzed). Subjects were instructed to fixate the fixation point all the time, judge the luminance of the disks and respond using the arrow keys (left arrow to indicate that left disk is brighter, right arrow for right disk brighter) on a standard 105 key keyboard when the question mark appeared. There was no feedback after the response.

EEG data acquisition and analysis

EEG was recorded at 1024 Hz using a Biosemi system (64 active electrodes). Horizontal and vertical electro-oculograms (EOG) were recorded by three additional electrodes around the subjects’ eyes. For data pre-processing, the EEG and EOG data were downsampled offline to 256 Hz, re-referenced to average reference and epoched around the stimulus onset in each trial for data analysis via the MATLAB (MathWorks) and EEGLAB toolbox49. Individual electrode data were visually inspected, and channel data containing artifacts were interpolated by the mean of adjacent electrodes (three subjects had one electrode containing artifacts, one subject had two; the positions of the interpolated electrodes were different across subjects).

As the post-stimulus spatial choice was lateralized on each trial to the left or right side, the pre-stimulus oscillatory correlates of the post-stimulus luminance judgment may not only reflect the oscillation’s influence on shape perception and predictive coding, but also its influence on spatial choice (i.e., pre-stimulus oscillations may bias the left/right spatial choice independently of the 3D-shape/random-lines content inside of the disk). To avoid any contribution from the spatial choice, we first divided the trials for each subject into two trial datasets based on the post-stimulus spatial choice, and performed the time-frequency analysis (described below) within each dataset. We reasoned that this analysis would lead to shape perception correlates not on a given fixed set of electrodes, but rather on different electrode groups depending on the side of choice (i.e., electrodes “contralateral” or “ipsilateral” to the spatial choice). Therefore, we arbitrarily chose to permute the electrode locations for the dataset corresponding to a right-side choice: we replaced the left-hemisphere electrodes by the symmetric ones from the right and vice versa (midline electrodes were unaffected). With this new electrode assignment, left-hemisphere electrodes would thus always correspond to those ipsilateral to the spatial choice, and right-hemisphere electrodes to contralateral ones.

For the time-frequency analysis, time-frequency transformations were first generated over all channels using EEGLAB with a function akin to a wavelet transform, starting with 3 cycles at 2 Hz and increasing to 5 cycles at 50 Hz in the multiple-cycle analysis, and with 1 cycle from 2 Hz to 50 Hz in the one-cycle analysis. This yields a complex representation of the amplitude, A, and the phase, [var phi], for trial j at time t and frequency f:

An external file that holds a picture, illustration, etc.
Object name is srep43573-m1.jpg

The phase of this representation can be extracted by normalizing the complex vector to the unit length:

An external file that holds a picture, illustration, etc.
Object name is srep43573-m2.jpg

Inter-trial phase coherence (ITPC) measures the phase consistency across trials. We calculated the ITPC using the method described previously50:

An external file that holds a picture, illustration, etc.
Object name is srep43573-m3.jpg

where N is the number of trials in one group of trials. Note that the ITPC is a scalar rather than a vector and it is always non-negative.

Here, we wanted to evaluate the relation between the pre-stimulus oscillatory phase and the influence of shape perception on luminance judgment (our measure of the efficiency of predictive coding). Would a particular pre-stimulus phase occur more frequently for trials with post-stimulus 3D-shape disk choice, and the opposite phase for trials with post-stimulus random-lines disk choice ? In the pre-stimulus period, because intertrial intervals are randomized and unpredictable, the phase of the spontaneous EEG signal at a given pre-stimulus time should follow a uniform distribution over all trials. However, if (and only if) there is a systematic relation between EEG phase and behavioral outcome, higher-than-chance inter-trial phase coherence should be observed in each of the trial subgroups. Indeed, if EEG phases of two subgroups corresponding to different behavioral outcomes tend to oppose each other, the phases of one subgroup will concentrate around one phase angle and the phases of the other subgroup will concentrate around the other phase angle; thus, high ITPC values will be observed for both subgroups. Vice versa, if EEG phases of two subgroups do not oppose each other, the phases of the two subgroups will distribute uniformly across different phase angles; thus, low ITPC values will be observed. Therefore, the ITPC values of the two subgroups are reliable indicators of phase opposition. The product or the sum of the two subgroup inter-trial phase coherence values can therefore summarize, in a single variable, the strength of the phase-behavior relation18,19,51. Here we chose to use the product of the inter-trial phase coherence values, a measure we call Phase Opposition Product (POP):

An external file that holds a picture, illustration, etc.
Object name is srep43573-m4.jpg

Again, high ITPC values for the two subgroups (suggesting different phase angles for the different trial subgroups) will result in a high POP value; and two low ITPC values (suggesting random/uniformly distributed phases for the different trial subgroups) will result in a low POP value. Thus, the measure of POP will be maximal when two subgroups show strong inter-trial phase coherence, in other words, strong phase opposition between two subgroups.

To accurately assess the significance of the phase-behavior relation without any assumption about the probability distribution of the POP values, we performed a nonparametric permutation test: We first computed the POP values for each point in the time-frequency plane from −650 to 150 ms, from 2 to 50 Hz for each electrode, dataset, and subject and then averaged across all datasets and all subjects. Surrogate POP values were obtained by randomly assigning the trials to one or the other condition for each subject (keeping the number of trials in each condition constant) and recalculating the grand-average POP values. We computed the P value by simply and directly counting the number of surrogate POP values that were more extreme than the observed value. Here, we used 80,000,000 surrogates and thus assigned the P value of 1.25 × 10−8 to the points without any more extreme POP values in the surrogates. The P values were corrected for multiple comparisons over time points, frequencies and electrodes using the FDR method (FDR α = 0.05, corresponding to a P value threshold of 9.53 × 10−6). To show the overall POP in the time-frequency domain, we computed a z-score by combining the observed POPs across all datasets, subjects, and electrodes and comparing the value with the mean and SD of a null-hypothesis distribution with 10,000 surrogate POP values (generated using the procedure described before, and also combined across all electrodes, subjects, and datasets).

Additional Information

How to cite this article: Han, B. and VanRullen, R. The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments. Sci. Rep. 7, 43573; doi: 10.1038/srep43573 (2017).

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Acknowledgments

Biao Han is supported by China Scholarship Council; Rufin VanRullen is supported by an ERC Consolidator Grant P-CYCLES number 614244.

Footnotes

The authors declare no competing financial interests.

Author Contributions B.H. and R.V. conceived the experiments, B.H. conducted the experiments, B.H. analysed the results under supervision of R.V. B.H. and R.V. wrote the manuscript.

References

  • Rao R. P. & Ballard D. H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999). [PubMed]
  • Summerfield C. et al. . Predictive codes for forthcoming perception in the frontal cortex. Science (80-.). 314, 1311–1314 (2006). [PubMed]
  • Friston K. A theory of cortical responses. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 360, 815–836 (2005). [PMC free article] [PubMed]
  • Spratling M. W. Reconciling predictive coding and biased competition models of cortical function. Front. Comput. Neurosci. 2, 4 (2008). [PMC free article] [PubMed]
  • Spratling M. W. Predictive Coding as a Model of Response Properties in Cortical Area V1. J. Neurosci. 30, 3531–3543 (2010). [PubMed]
  • Fontolan L., Morillon B., Liegeois-Chauvel C. & Giraud A.-L. The contribution of frequency-specific activity to hierarchical information processing in the human auditory cortex. Nat. Commun. 5, 4694 (2014). [PMC free article] [PubMed]
  • Bastos A. M. et al. . A DCM study of spectral asymmetries in feedforward and feedback connections between visual areas V1 and V4 in the monkey. Neuroimage 108, 460–475 (2015). [PMC free article] [PubMed]
  • van Kerkoerle T. et al. . Alpha and gamma oscillations characterize feedback and feedforward processing in monkey visual cortex. Proc. Natl. Acad. Sci. USA 111, 14332–14341 (2014). [PubMed]
  • Buffalo E. a, Fries P., Landman R., Buschman T. J. & Desimone R. Laminar differences in gamma and alpha coherence in the ventral stream. Proc. Natl. Acad. Sci. USA 108, 11262–11267 (2011). [PubMed]
  • Maier A., Adams G. K., Aura C. & Leopold D. A. Distinct superficial and deep laminar domains of activity in the visual cortex during rest and stimulation. Front. Syst. Neurosci. 4, 1–11 (2010). [PMC free article] [PubMed]
  • Bastos A. M. et al. . Canonical microcircuits for predictive coding. Neuron 76, 695–711 (2012). [PMC free article] [PubMed]
  • Busch N. A. & VanRullen R. Spontaneous EEG oscillations reveal periodic sampling of visual attention. Proc. Natl. Acad. Sci. USA 107, 16048–16053 (2010). [PubMed]
  • Dugué L., Marque P. & VanRullen R. The Phase of Ongoing Oscillations Mediates the Causal Relation between Brain Excitation and Visual Perception. J. Neurosci. 31, 11889–11893 (2011). [PubMed]
  • Drewes J. & VanRullen R. This Is the Rhythm of Your Eyes: The Phase of Ongoing Electroencephalogram Oscillations Modulates Saccadic Reaction Time. J Neurosci 31, 4698–4708 (2011). [PubMed]
  • Song K., Meng M., Chen L., Zhou K. & Luo H. Behavioral Oscillations in Attention: Rhythmic Alpha Pulses Mediated through Theta Band. J Neurosci 34, 4837–4844 (2014). [PubMed]
  • Murray S. O., Kersten D., Olshausen B. A., Schrater P. & Woods D. L. Shape perception reduces activity in human primary visual cortex. Proc. Natl. Acad. Sci. USA 99, 15164–15169 (2002). [PubMed]
  • Han B. & VanRullen R. Shape perception enhances perceived contrast: evidence for excitatory predictive feedback? Sci. Rep. 6, 22944 (2016). [PMC free article] [PubMed]
  • Busch N. a, Dubois J. & VanRullen R. The phase of ongoing EEG oscillations predicts visual perception. J. Neurosci. 29, 7869–7876 (2009). [PubMed]
  • VanRullen R., Busch N. A., Drewes J. & Dubois J. Ongoing EEG Phase as a Trial-by-Trial Predictor of Perceptual and Attentional Variability. Front. Psychol. 2, 60 (2011). [PMC free article] [PubMed]
  • Clark A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sci. 36, 181–204 (2013). [PubMed]
  • Dean A. F. The relationship between response amplitude and contrast for cat striate cortical neurones. J. Physiol. 318, 413–427 (1981). [PubMed]
  • Albrecht D. G. & Hamilton D. B. Striate cortex of monkey and cat: contrast response function. J. Neurophysiol. 48, 217–237 (1982). [PubMed]
  • Goodyear B. G. & Menon R. S. Effect of luminance contrast on BOLD fMRI response in human primary visual areas. J. Neurophysiol. 79, 2204–2207 (1998). [PubMed]
  • Boynton G. M., Demb J. B., Glover G. H. & Heeger D. J. Neuronal basis of contrast discrimination. Vision Res. 39, 257–269 (1999). [PubMed]
  • Kok P. & de Lange F. P. Shape Perception Simultaneously Up- and Downregulates Neural Activity in the Primary Visual Cortex. Curr. Biol. 24, 1531–1535 (2014). [PubMed]
  • Carrasco M. M. et al. . Attention alters appearance. Nat. Neurosci. 7, 308–313 (2004). [PMC free article] [PubMed]
  • Landau A. N. & Fries P. Attention samples stimuli rhythmically. Curr. Biol. 22, 1000–1004 (2012). [PubMed]
  • Fiebelkorn I. C., Saalmann Y. B. & Kastner S. Rhythmic sampling within and between objects despite sustained attention at a cued location. Curr. Biol. 23, 2553–8 (2013). [PMC free article] [PubMed]
  • Roopun A. K. et al. . A beta2-frequency (20–30 Hz) oscillation in nonsynaptic networks of somatosensory cortex. Proc. Natl. Acad. Sci. USA 103, 15646–15650 (2006). [PubMed]
  • Douglas R. J. & Martin K. A. C. Neuronal circuits of the neocortex. Annu. Rev. Neurosci. 27, 419–451 (2004). [PubMed]
  • Barbas H. & Rempel-Clower N. Cortical structure predicts the pattern of corticocortical connections. Cereb. Cortex 7, 635–646 (1997). [PubMed]
  • Wang X.-J. Neurophysiological and computational principles of cortical rhythms in cognition. Physiol. Rev. 90, 1195–1268 (2010). [PMC free article] [PubMed]
  • Yordanova J., Kolev V. & Kirov R. Brain oscillations and predictive processing. Front. Psychol. 3, 1–2 (2012). [PubMed]
  • Arnal L. H. & Giraud A. L. Cortical oscillations and sensory predictions. Trends Cogn. Sci. 16, 390–398 (2012). [PubMed]
  • Bauer M., Stenner M. P., Friston K. J. & Dolan R. J. Attentional Modulation of Alpha/Beta and Gamma Oscillations Reflect Functionally Distinct Processes. J Neurosci 34, 16117–16125 (2014). [PMC free article] [PubMed]
  • Todorovic A., van Ede F., Maris E. & de Lange F. P. Prior Expectation Mediates Neural Adaptation to Repeated Sounds in the Auditory Cortex: An MEG Study. J. Neurosci. 31, 9118–9123 (2011). [PubMed]
  • Jensen O., Bonnefond M. & VanRullen R. An oscillatory mechanism for prioritizing salient unattended stimuli. Trends Cogn. Sci. 16, 200–206 (2012). [PubMed]
  • VanRullen R. Visual attention: a rhythmic process? Curr. Biol. 23, R1110–2 (2013). [PubMed]
  • Huang Y., Chen L. & Luo H. Behavioral Oscillation in Priming: Competing Perceptual Predictions Conveyed in Alternating Theta-Band Rhythms. J Neurosci 35, 2830–2837 (2015). [PubMed]
  • Dugué L., Marque P. & VanRullen R. Theta Oscillations Modulate Attentional Search Performance Periodically. J. Cogn. Neurosci. 945–958, doi: (2015).10.1162/jocn [PubMed] [Cross Ref]
  • Summerfield C. & Egner T. Expectation (and attention) in visual cognition. Trends Cogn. Sci. 13, 403–409 (2009). [PubMed]
  • Summerfield C. & de Lange F. P. Expectation in perceptual decision making: neural and computational mechanisms. Nat. Rev. Neurosci (2014). [PubMed]
  • Phillips J. M., Vinck M., Everling S. & Womelsdorf T. A long-range fronto-parietal 5- to 10-Hz network predicts ‘top-down’ controlled guidance in a task-switch paradigm. Cereb. Cortex 24, 1996–2008 (2014). [PMC free article] [PubMed]
  • Lopes da Silva F. H., van Rotterdam A., Storm van Leeuwen W. & Tielen A. M. Dynamic characteristics of visual evoked potentials in the dog. II. Beta frequency selectivity in evoked potentials and background activity. Electroencephalogr. Clin. Neurophysiol. 29, 260–268 (1970). [PubMed]
  • Bekisz M. & Wróbel A. Attention-dependent coupling between beta activities recorded in the cat’s thalamic and cortical representations of the central visual field. Eur. J. Neurosci. 17, 421–426 (2003). [PubMed]
  • Buschman T. J. & Miller E. K. Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices. Science (80-.). 315, 1860–1862 (2007). [PubMed]
  • Bosman C. A. et al. . Attentional Stimulus Selection through Selective Synchronization between Monkey Visual Areas. Neuron 75, 875–888 (2012). [PMC free article] [PubMed]
  • Grothe I., Neitzel S. D., Mandon S. & Kreiter A. K. Switching neuronal inputs by differential modulations of gamma-band phase-coherence. J. Neurosci. 32, 16172–80 (2012). [PubMed]
  • Delorme A. & Makeig S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004). [PubMed]
  • Lachaux J. P., Rodriguez E., Martinerie J. & Varela F. J. Measuring phase synchrony in brain signals. Hum. Brain Mapp. 208, 194–208 (1999). [PubMed]
  • VanRullen R. How to Evaluate Phase Differences between Trial Groups in Ongoing Electrophysiological Signals. Front. Neurosci. 10, 426 (2016). [PMC free article] [PubMed]

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group