|Home | About | Journals | Submit | Contact Us | Français|
Conceived and designed the experiments: CR UH TK IN. Performed the experiments: CR AF CM TK. Analyzed the data: CR MDV SD BIT. Contributed reagents/materials/analysis tools: BT UH IN. Wrote the paper: CR MDV SD.
Cognitive task demands in one sensory modality (T1) can have beneficial effects on a secondary task (T2) in a different modality, due to reduced top-down control needed to inhibit the secondary task, as well as crossmodal spread of attention. This contrasts findings of cognitive load compromising a secondary modality’s processing. We manipulated cognitive load within one modality (visual) and studied the consequences of cognitive demands on secondary (auditory) processing. 15 healthy participants underwent a simultaneous EEG-fMRI experiment. Data from 8 participants were obtained outside the scanner for validation purposes. The primary task (T1) was to respond to a visual working memory (WM) task with four conditions, while the secondary task (T2) consisted of an auditory oddball stream, which participants were asked to ignore. The fMRI results revealed fronto-parietal WM network activations in response to T1 task manipulation. This was accompanied by significantly higher reaction times and lower hit rates with increasing task difficulty which confirmed successful manipulation of WM load. Amplitudes of auditory evoked potentials, representing fundamental auditory processing showed a continuous augmentation which demonstrated a systematic relation to cross-modal cognitive load. With increasing WM load, primary auditory cortices were increasingly deactivated while psychophysiological interaction results suggested the emergence of auditory cortices connectivity with visual WM regions. These results suggest differential effects of crossmodal attention on fundamental auditory processing. We suggest a continuous allocation of resources to brain regions processing primary tasks when challenging the central executive under high cognitive load.
The brain’s capacity to re-allocate resources and to deal with its attentional capacities is relevant for survival and serves adaptive functioning , . How limited processing resources are managed between sensory modalities which are implicated simultaneously via two or more different tasks is however not fully understood. Cross-modal processing has been subject to several experimental investigations , , . The results can be subsumed under different theoretical frameworks: The ‘automaticity theory’  states automatic processing to be present in the unattended secondary task and immunity to cross-modal influences , . Several studies have found evidence for an absence of crossmodal effects on secondary task processing . In contrast, the ‘gain-load theory’  suggests that the primarily engaged modality uses the limited capacities which causes inhibition and thereby decreased processing of secondary input. This was also supported by others  and moderated by the assumption of differential effects on undistractable and distractable components of crossmodal attention (e.g. reduced distraction effect but intact automatic change-detection mechanisms, ).
Recently, Haroush and colleagues  have reported evidence for yet another alternative. In a perceptually demanding visual attentional blink paradigm healthy young participants showed cross-modal augmentation processing of unattended sounds. This was interpreted as a consequence of executive control due to cognitive overload resulting from the attended task. The decrease in executive control challenged the otherwise effective suppression of irrelevant input . In contrast to the gain-load theory, the effects expected here on secondary task processing are beneficial rather than detrimental.
Another alternative explanation for advantageous crossmodal effects may include generalized attention, a concept attributed to a spread of cognitive alertness , . This may be caused by the challenging task in the primary modality, which supports the notion of attention being a general, modality-independent cognitive resource serving beneficial purposes for other modalities.
One family of crossmodal effects are primary visual load effects on secondary auditor processing. Most studies focused on the auditory change effects . Results show inconsistencies regarding the directionality of crossmodal effects. Some report decreased MMN amplitudes in the secondary task , , others find the opposite ,  and there also exist null-findings on potential crossmodal influences , , ). While the MMN reflects active sensory memory processing , the N1 as its prerequisite contributes to encoding the sensory memory trace. It acts out stimulus perception as well as feature-detection mechanisms and represents fundamental auditory processing . However, it was usually not distinguished whether standard or deviant processing was affected and which of the two was responsible for the decrease in auditory change detection . It remained open whether the observable effects would already be present during basic tone processing. SanMiguel and colleagues  made an exception to this reporting an effect of visual working memory (WM) on the auditory N1. However, memory load was manipulated only on one level and the directionality of this effect (decreasing/increasing) could not be determined. Haroush and colleagues  also reported auditory evoked responses (AEPs), however, they also focused on the MMN and the significance of the effect specifically of N1 or P2 amplitudes could not be evaluated.
The aim of the present study was to investigate how stepwise increases in a four-level visual WM design would influence basic auditory processing. Rather than audiotry change effects we wanted to specifically analyze standard tones, representing sensory encoding for the memory trace which is the prerequisite for further higher-level processes such as the auditory change effect.
We used FMRI in order to assess WM manipulation and concurrently recorded event-related potentials (ERP) to measure auditory processing. Although a simultaneous recording of both modalities is not strictly required, it has several advantages: It enables disentangling modality-specific effects but guarantees inferring direct relations due to the measurement simultaneity and the time-point stable task manipulation of both modalities.
Often-replicated fronto-parietal network activations are well-established fMRI correlates of visual WM , , . Electrophysiological tone responses are characterized by the auditory N1-P2 vertex potential. These ERP components can also be reliably obtained when recorded in the MR scanner and the potential coupling between both measures is a matter of ongoing research .
Our hypotheses were based on a successful manipulation of visual WM load, which would result in uni-modally enhanced fMRI activation patterns in WM-related areas. Cross-modal effects in fundamental auditory processing were investigated via AEPs simultaneously measured, as well as measured outside the scanner in a different sample. Our simultaneous measurement setup would further help us to provide a more refined answer to how the spatial and temporal correlates of potential crossmodal load effects would manifest themselves by reciprocally informing one measurement modality by the results of the other , , .
The experimental set-up conformed to the Code of Ethics of the World Medical Association and the study was approved by the ethics board of the medical faculty, RWTH Aachen University. Participants gave written informed consent on the study protocol.
The group of participants consisted of 15 healthy adults (7 females, M age=25.60 yrs., SD=2.87) for simultaneous EEG-fMRI (inside) measurements and 8 healthy adults (5 females, M age=24.25 yrs., SD=3.20) for EEG-only (outside) measurements (5 subjects were measured inside and outside, with 12 months in between measurements,). Participants were recruited through local advertisements, followed by a detailed screening which confirmed a negative history of psychiatric disorder, neurological illness or current substance abuse. All participants were right-handed , had normal or corrected-to-normal vision and fulfilled MR scanning inclusion criteria. Two participants were excluded from the EEG analysis and subsequent integration of fMRI and EEG due to low EEG signal quality. This reduced the final sample for EEG and EEG-informed analyses to 13 participants.
The experiment (Figure 1) consisted of an attended visual n-back task (T1) with four conditions for parametric modulation of WM load. The baseline condition (‘fixation’) included watching a presented series of letters, the ‘0-back’ condition required subjects to respond via a button-press to the target letter ‘X’, in ‘1-back’, they had to respond to two consecutive identical letters, and in ‘2-back’, they had to respond to letters identical to the one presented two trials before. Letters in 1-back and 2-back could be any of the alphabet, except 'X'. Each of the 3 condition-blocks (0-back, 1-back, 2-back) appeared five times, interspersed with 15 baseline blocks. Stimuli were presented for 500 ms every 1.4 s for block duration of 27 s in the n-back conditions and 15 s in the baseline. Every block was initiated by a 3 s task instruction. A fixed order was repeated five times.
The unattended task (T2) consisted of frequent standard tones (1000Hz, ~60 dB) and infrequent deviant tones (1300Hz, ~60 dB), continuously presented every 1.4 s with a standard oddball ratio of 91. This resulted in approximately 14–16 standards and 1–2 deviants per experimental block (9–11 standards and 1 deviant during fixation) and a total of 360 standards and 45 deviants. Tones and visual stimuli were time-locked to each other: tones were presented first and always followed after 500 ms by visual stimuli. The presentation of the tones was not directly triggered to the MR pulse to prevent from time-locking of the MR artifact signal and the EEG signal. The very first three standards and off-timed tones (exceeding ±30 ms deviance from inter-stimulus-interval) were excluded from further analysis. This was the same for standards following deviants and first standards of each block.
Participants were prepared for the simultaneous measurement and comfortably placed in the MR scanner with the right hand’s index finger placed on a response button (LUMItouch™, Lightwave Technologies, Richmond, Canada). The experimental stimuli were presented using Presentation® (Neurobehavioral Systems Inc., San Francisco, CA). Tones were presented at ~60 dB via MR-compatible heaphones (Behringer ®).
In order to guarantee that EEG quality of auditory evoked potential was reliable inside the scanner, eight control participants were prepared for an EEG-only measurement and measured in a dimly-lit room, in a supine position, wearing goggles and headphone to ensure comparability of experimental influence.
Subjects were instructed to engage in the visual n-back task and to ignore all tones.
In order to investigate cross-modal effects caused by increasing visual WM load on basic auditory processing, we followed a reciprocal analysis strategy of informing the EEG analysis by effects found in the fMRI and vice versa.
Behavioral responses (hit rates and reaction times) were analyzed one-way repeated measures ANOVAs in IBM® SPSS® (version 20). These models included the within-subjects factor ‘n-back’ with three levels (leaving out fixation). Post-hoc tests were performed using paired t-tests and Bonferroni correction.
fMRI data were obtained on a 3 Tesla Tim Trio® MR scanner (Siemens Medical Systems, Erlangen, Germany) during one run with an echo-planar imaging (EPI) T2* contrast sequence sensitive to blood oxygenation level dependent (BOLD) changes (3.125×3.125×3.4 mm3 voxel size, 64×64 matrix, 200×200 mm2 FOV, 33 3.4 mm-thick axial (AC-PC) slices with whole brain coverage (0.51 mm gap), ascending acquisition, TR/TE=2000/30 ms, 76° flip angle , 360 volumes). Data analysis was performed using SPM8 (Wellcome Department of Cognitive Neurology, London, UK). Preprocessing of data included realignment of data to correct for head movement, coregistration of the mean EPI scan onto the SPM8 grey matter tissue probability map, and normalization using the unified segmentation approach . Images were resampled to a voxel size of 1.5×1.5×1.5 mm3 and smoothed with an isotropic 8 mm FWHM (full width at half maximum) Gaussian kernel.
On a single-subject level, four regressors (one for each WM condition) were created by convolving the respective box-car function with the canonical double-gamma hemodynamic response function (HRF) . Realignment parameters were included as covariates of no interest and the session mean was regressed on a constant term. Prior to parameter estimation, a 128 s high-pass filter was applied. Serial auto-correlations were accounted for by including a first order autoregressive model (AR-1). Simple main contrasts (fixation, 0-back, 1-back, 2-back) were further used for the group-level analysis. A first step was to validate visual WM load by analyzing fMRI activations corresponding to the visual letter presentation. A mixed-effects GLM was used for group-level inference with subjects as random effects and WM conditions as fixed effects. Departures from sphericity were corrected for by variance components assuming a compound symmetry structure for within-subjects (correlated) measures and heteroscedasticity between subjects and conditions.
In order to reveal neural activation corresponding to visual WM load, we carried out an F-test, testing for general difference between the four conditions. The statistical parametric map was thresholded at p<.05, corrected for multiple comparisons at the voxel-level using Gaussian random field theory (family-wise-error, FWE) and a cluster-extent threshold of 20 voxels. A T-contrast was carried out, testing for a parametric increase of WM-load (conjunction analysis 2-back>1-back ∩ 1-back>0-back ∩ 0-back>fixation). This contrast was thresholded with a combined height and extent threshold technique based on Monte-Carlo (MC) simulations calculated with AlphaSim . Based on an uncorrected threshold of p<.001 and the spatial properties of the residual image an extent threshold of 125 voxels was estimated using 100000 and complied with a family wise error of p<.05.
Activation maxima are reported as MNI-coordinates and anatomical locations are based on the Talairach Client (Lancaster & Fox, Research Imaging Center, University of Texas Health Science Center San Antonio) and the Anatomy Toolbox .
We used a 64-channel MR-compatible EEG system (two BrainAmp MR plus 32-channel amplifiers, BrainProducts GmbH, Gilching, Germany), connected to a MR-compatible electrode cap (Easycap GmbH, Herrsching-Breitbrunn, Germany) with 64 Ag–AgCl electrodes (5 kΩ resistors), 63 of which covered the 10–20 system and one electrocardiogram (ECG) electrode placed ~10 cm below the left scapula. Electrodes at positions FCz and AFz served as the recording reference and ground electrode, respectively. The online sampling rate was set to 5000 Hz (0.01–250 Hz analog band-pass filter), and electrode impedances were below 20 kΩ. To improve EEG artifact attenuation a sync box (BrainProducts GmbH, Gilching, Germany) was used for optimal synchronization of EEG acquisition with the clock controlling MRI slice acquisition. At the start of each volume acquisition, an event marker was sent to the recording device (Brain Vision Recorder 1.0, BrainProducts GmbH, Gilching, Germany) to enable identification of gradient onsets and to create a template for artifact subtraction.
Offline analysis of EEG data was accomplished using Brain Vision Analyzer software, version 2.0 (BVA 2.0, Brainproducts, Gilching, Germany) and EEGLAB, version 188.8.131.52b . Continuous EEG data underwent gradient artifact removal using the template matching algorithm in BVA . After gradient artifact removal, the data were low-pass-filtered with a digital infinite impulse response filter (IIR, 70 Hz, 48 dB slope) and downsampled to 500 Hz. Cardiac pulse correction was carried out based on an automatically detected pulse template in the ECG channel. Markers were set at highly correlated (>0.7) and above-threshold amplitude (0.4–1.4) time-points. Cardiac pulse markers were visually confirmed and the data subsequently exported to EEGLAB in order to apply a channel-wise optimal basis set procedure ,  as implemented in the EEGLAB-plugin FMRIB 1.2. Data was then re-referenced to linked mastoids (mean TP9-TP10). Independent component analysis (ICA, extended infomax) ,  revealed components relating to eye movement which were removed from the data (maximally two components were removed). Data were filtered (1–30 Hz) and epochs exceeding 125 µV were rejected from further analysis.
Fundamental auditory processing was assessed by analyzing saturated standard tones (Figure 1) which were conceptualized as any tone not appearing as first tones of a block or following a deviant, leaving 372 trials total. Since blocks were uniformly distributed across the experiment a direct comparison of conditions did not include general adaptation effects across time.
Cross-modal effects on event-related auditory potentials (AEPs) were assessed by extracting auditory events in different WM conditions and analyzing condition-specific peak amplitudes and latencies. AEP epochs included 700 ms around the tones (−100 to 600 ms post-stimulus onset, baseline-corrected) and were averaged within each of the four visual WM load condition. Based on other work , ,  we extracted peak amplitudes from electrode position Fz. The absolute N1-P2 peak-to-peak values were extracted using an N1 search window between 80–140 ms and a P2 search window between 170–210 ms for each of the four conditions. This yielded four values per subject (fixation, 0-back, 1back, 2-back). N1-P2 amplitudes and the latencies of N1 and P2 peaks were statistically analyzed within generalized linear estimating equations (GEE) in IBM® SPSS® (version 20). The statistical models included main effect of the factors 'n-back' (four levels). Post-hoc tests were performed using paired t-tests and Bonferroni correction.
We carried out three ANCOVA models, each testing for the effect of the mean N1, P2, and N1-P2 amplitude of standards, respectively on a group level. Values (four per subject) were entered into the statistical design as a covariate explaining inter-individual BOLD variance after mean-centering. This approach assumes a correspondence of event-related potentials to neural activity measured via changes in BOLD , , . While allowing for an interaction between the covariate and the main task, we were interested in two contrasts: A T-contrast testing for the average effect of the covariate regressors which would represent a general effect of the auditory response on brain activation, as well as an F-contrast testing for the effect of the covariate interacting with the task effect (differences between WM-load conditions). Both contrasts were masked with the effects of interest of the F-contrast testing for unsigned differences between visual WM-conditions of the four BOLD regressors (inclusive mask, thresholded at p<.05 uncorrected) and thresholded at p<.001 (MC-cluster-corrected, p<.05).
Cross-modal effects caused by visual working memory load on basic auditory processing motivated a subsequent individual region-of-interest analysis in primary auditory cortex (AC). Since auditory processing of the tones was not experimentally manipulated (only visual working memory load was), we would have expected null findings in both auditory cortices. However, primary and non-primary AC are the primary generator regions of AEPs , , . Therefore it was investigated whether the BOLD signal measured in AC complemented the effects of cross-modal manipulation of AEP amplitudes.
For this region-of interest analysis, anatomical masks of left and right Heschl’s gyrus were created using the AAL database  in WFU Pickatlas , . Using MarsBaR , the same single subject analysis was performed as described above for the whole brain analysis (four regressors modeling the BOLD response of each WM load condition) separately for each AC. The condition-wise averaged time-series of AC activation were analyzed in IBM® SPSS® (version 20) using generalized linear models with the within-subject factor condition (four levels).
While the region-of-interest analysis would allow specific insight into the effect of experimental conditions in a specific region it was also of interest how functional connectivity patterns of this region and others would emerge in different WM conditions.
In the subsequent PPI analysis we therefore extracted the individual time-series of left and right primary AC (same masks as used for the ROI analysis). After deconvolution, data vectors were multiplied with the respective box-car functions representing one WM condition each and reconvolved with the canonical HRF . On a single subject level, the data vector (representing one of four conditions) was implemented as a PPI regressor. The convolved main effect of each condition, the seed region’s time-course, and six realignment parameters as well as an intercept were entered as covariates of no interest into the analysis.
After model estimation, parameter estimates of the PPI regressors from each subject’s four first-level analyses were entered into a mixed-effects GLM for group-level inference with subjects as random effects and four PPI regressors as fixed effects. As in the BOLD-GLM described above, departures from sphericity were corrected for by variance components assuming a compound symmetry structure for within-subjects measures and heteroscedasticity between subjects and conditions. Simple main effects, representing task-related connectivity of AC with in each WM-load condition were thresholded at p<0.05 (FWE-corrected), the conjunction analysis testing for effects correlating with parametrically increasing visual WM load was thresholded at p<.05 (MC-cluster-corrected).
The main effect of the WM task (F-contrast testing for differences between all conditions, Table 1A, Figure 2) revealed activations in bilateral inferior frontal and prefrontal cortex, supplementary motor area (SMA) and ventromedial prefrontal cortex (vmPFC), middle cingulate gyrus, and precuneus, as well as activations in parietal areas (intraparietal sulcus, inferior parietal lobe and angular gyrus) extending to temporal areas, and several cerebellar clusters.
The parametric effect of WM load (T-contrast conjunction 2-back>1-back ∩ 1-back>0-back ∩ 0-back>fixation, Table 1B) showed activation in Area 6 (precentral gyrus) and supplementary premotor area (SMA).
The GLM analyzing participants’ hit rates and reaction times each showed a significant main-effect of 'n-back' (hit rates: Wald χ2(2)=17.74, p<.001; RTs: Wald χ2(2)=35.47, p<.001). Post-hoc tests of hit rates showed a significant decrease from condition 0-back to 2-back (t(14)=3.92, p<.001) and from condition 1-back to 2-back (t(14)=3.84, p=.013). Post-hoc tests of RTs showed a significant increase condition 0-back to conditions 1-back (t(14)=−4.11) and 2-back (t(14)=−4.10) (Table 2 and Figure 2).
In control participants (EEG-only), hit rates also significantly increased (main effect of ‘n-back’, Wald χ2(2)=16.09, p<.001). Post-hoc tests showed a significant decrease in hit rates from condition 0-back to 2-back (t(4)=3.50, p<.001) and from condition 1-back to 2-back (t(4)=2.73, p=.006). Although on a descriptive level, the effects were comparable to those observed in inside data the main effect of ‘n-back’ on RTs was not significant (Wald χ2(2)=3.95, p<.139).
Generally, EEG data quality was similar after correction of MR- and CB-artifacts of EEG data measured inside the scanner, and signal-to-noise ratios ('noise' being defined as the difference of an odd-even split) were not significantly different between EEG-fMRI and EEG-only data (t(21)=−1.26, p=.13). This supports former reports of valid data recorded from simultaneous continuous measurements compared to interleaved gap measurements  or measurements inside the MR scanner without applying HF pulses .
N1-P2 amplitude of standard tones of EEG data from inside the scanner showed a significant main effect of ‘n-back’ (Table 3, Figure 3; Wald χ2(3)=15.66, p<.001). Post-hoc pairwise comparisons showed that with increasing WM load, AEP amplitudes continuously increased from fixation to all subsequent WM-conditions. The difference was significant (Bonferroni-corrected for all possible comparisons) between fixation and 2-back (t(12)=−2.62, p=.002) as well as between 0-back and 2-back (t(12)=−3.03, p=.01). The comparison between fixation and 1-back (t(12)=−2.41, p=.03) was only significant if not Bonferroni-corrected and the other comparisons (fixation compared to 0-back, 0-back compared to 1-back and 1-back compared to 2-back) were not significant.
In EEG-only data, crossmodal effects on auditory processing replicated the effects of the data recorded inside the scanner. We found a significant main effect of ‘n-back’ on AEPs (Wald χ2(3)=9.49, p<.023). Post-hoc tests showed a significant increase from fixation to 1-back (t(7)=−2.83, p=.025). No other post-hoc test showed significant differences between conditions.
No significant main effect of ‘n-back’ was found for N1 or P2 latency in EEGfMRI or EEG-only data.
Generally, the N1 peak values of standard tones explained inter-individual subject variance of the BOLD signal in bilateral anterior and posterior cingulate cortex, in left inferior and middle frontal gyrus, as well as inferior temporal gyrus and superior parietal lobe. Subcortical activation was located in the amygdala, caudate nucleus, and hippocampus (T-contrast testing for the average effect of the covariate, Figure 4, Table 4).
When taking into account the cross-modal condition effect and testing for unsigned differences between any of the N1 peak amplitude regressors with an F-Test, activation patterns consisted of focal activations in the left DLPFC and superior medial gyrus, inferior parietal cortex and precuneus.
Neither P2 peak amplitudes, nor N1-P2 absolute peaks explained inter-individual subject variance above the set Monte-carlo corrected threshold of p<.05.
The GLM analyzing averaged time-series of each WM condition BOLD response from a region-of-interest analysis of left AC (main effect ‘n-back’, Wald χ2(3)=8.52, p=.04) revealed a significant increase in deactivation from the fixation condition to the different WM load conditions. Post-hoc tests showed significant deactivation increases between 0-back and 2-back (t(14)=2.82, p=.014) and between 1-back and2-back (t()14)=2.59, p=.21) only if uncorrected for multiple comparisons (Table 5).
This effect was replicated in right AC (Wald χ2(3)=12.56, p=.006). Post-hoc t-tests tests indicated significantly higher deactivation in 2-back compared to fixation (t(14)=4.68, p<.001), 2-back compared to 0-back (t(14)=4.57, p<.001) and 2-back compared to 1-back (t(14)=3.224, p=.006).
Testing task-related functional connectivity of left AC in each WM-condition yielded the following results: during fixation, the AC showed significant functional connectivity with the right primary AC, precuneus, fusiform gyrus, and ventromedial PFC (Table 6, Figure 5). This changed with increasing WM-load to a connectivity pattern increasingly representing the fronto-parietal WM network (SMA, DLPFC, inferior parietal lobe, thalamus, and cerebellum). The parametric effect of WM-load (T-contrast conjunction combining 2-back>1-back ∩ 1-back>0-back ∩ 0-back>fixation) revealed functional connectivity with bilateral inferior parietal lobes and sulci, several frontal areas (IFG, middle and superior frontal gyrus, SMA) as well as thalamus and lobules VI and VIIa crus I of the cerebellum (Table 6). When masking this contrast with the parametric effect of the initial nback GLM (inclusively masking, p<.05 uncorrected) we found one cluster in right DLPFC (MC-cluster-corrected, p<.05).
The results of the PPI analysis of right AC task-related connectivity yielded similar results. Functional connectivity with the left AC under fixation condition appeared slightly weaker but the parametrically increasing functional connectivity with the fronto-parietal WM network was replicated.
The present study investigated how increasing visual WM-load (T1) affected secondary fundamental auditory processing (T2). EEG-fMRI enabled us to identify WM load activation representing the primary task manipulation as well as precision in the temporal domain to analyze neural action of simultaneously ongoing auditory processing.
The primary task manipulation, a visual WM task, was validated by the fMRI results showing bilateral fronto-parietal and subcortical neural activation patterns corresponding to WM-load  and participants' behavior (increasing RTs and decreasing hit rates with increasing WM load, Figure 6).
Basic auditory processing, represented by saturated standard tones revealed a simultaneously happening stepwise increase of the AEP corresponding to a gradual increase in crossmodal WM load. Cross-modal attention effects have been repeatedly reported on auditory processing , , . Nevertheless, first, results are contradictory and range from an automaticity assumption regarding the basic analysis of auditory perception , , to proposing differential effects of the primary task on several outcome parameters of the secondary task , . Secondly, to our knowledge, fundamental processing, as expressed by the N1-P2 complex, had not been explicitly studied in a comparable design.
Our results contradict the automaticity assumption and show a clear susceptibility of fundamental auditory processing to cross-modal WM load manipulation, systematically investigated by using four modulations. A susceptibility to crossmodal cognitive load influences supports and extends Haroush’s  findings. They argued that the enhanced processing of T2 was due to a lack of executive control which usually causes an attenuation of second-modality input. This lack was present because the system, being busy with T1 consolidation, was challenged to a point of cognitive overflow where the effective suppression of inputs other than the attended one could not be guaranteed anymore. While the attentional blink paradigm challenges participants to the limits of conscious perception via temporal manipulation of sensory perception , , ,  and working memory consolidation in a short time period  the applied n-back task employed several levels of working memory load. Here, under low T1-load, the crossmodal processing of unattended T2-processing was smaller (smaller AEP amplitudes) than when compared to high T1-load (higher AEP amplitudes). Because an intuitive cause of this would have to be found in generators of this response, we carried out a region-of-interest analyses in the primary auditory cortices in which active suppression of secondary input is either not possible or not intended due to mechanisms of generalizing attentional resources as described next.
Potential effects of decreasing central executive control inhibition on secondary task processing were investigated with region-of-interest analyses of the BOLD response in both primary auditory cortices (ACs) as important auditory response generators , . The results revealed AC deactivation associated with cross-modal increasing task demands in T1. This, at first, questioned the presence of break-down of cognitive control on primary sensory areas because the latter should have intuitively caused an increase of the (uninhibited) primary AC. Instead, this processing decrease of a secondary task supported 'gain theory’ assumptions of resource allocation to the primary attended modality , .
However, the auditory cortices are not the only generators of late AEPs  and attentional effects seem to play a crucial role here , . This promotes the possibility of differential regional contributions to the AEP  when cognitive crossmodal load comes into play. Indeed, our data showed correlations of N1-amplitudes with BOLD activation in potential frontal contributors of AEPs . When taking into account the cross-modal manipulation (main effect of covariate, F-contrast, Table 4), activation was present in DLPFC, superior medial gyrus and inferior parietal gyrus (Figure 4, yellow coloring). This hints to an involvement of the so-called ‘core’ network of WM-load activation  covarying with the electrophysiological response specifically when considering crossmodally manipulated WM load rather than if simply considering general effects the N1 amplitudes have on BOLD variance (green coloring).
Hence, the AEP seems to be associated with visual WM nodes. We further carried out a psychophysiological interaction analysis (PPI) which resulted in a parametric effect of task-dependent functional connectivity of AC with the WM network. This might be due to a stepwise re-allocation of cognitive resources to regions associated with processing primary cognitive load. Contrary, during fixation, where no cognitive load was imputed left AC connectivity patterns were present in its contralateral counterpart, precuneus and vmPFC.
We propose that low WM load did not intervene much with the limits of executive control and there was no necessity to re-allocate attentional resources. High load, however, subtracted attention from uni-modal sensory processing areas and allocated the available resources to the relevant neural structures which may contribute to a more increased joint generation of responses in the secondary modality , , , , . This strongly suggests that the AEP is associated with nodes in a network, which may or may not biophysically contribute or modulate to its appearance.
Our findings may finally help to explain the often reported decreases of the auditory change-effects which is represented in a smaller difference between deviant and standard tone processing under high load. The current investigation of fundamental tone processing might be an important step in a sensitive approach for evaluating cognitive load effects on continuous stimulus processing in a different modality. It remains speculative if deviant processing which recruits automatic, bottom-up attentional resources [59,19 63] appears less sensitive to cognitive load manipulations compared to standard processing (the datasets included only 45 deviant tones in total and we refrained from a condition-wise analysis).
Using a simultaneous EEG-fMRI measurement protocol we demonstrated that basic auditory processing is systematically related to cross-modal cognitive load. We extend existing findings and show that increasing cognitive load impacts secondary task sensory processing. While a beneficial effect of crossmodal task load was found in elctrocortical responses of basic auditory processing by the vertex potential a deactivations of primary auditory cortices contradict a break down of executive control and rather points to a reallocation of attentional resources and spread of attention. However, the region-of-interest analysis was based on an anatomical template of the complete Heschl’s gyrus, which neglects to pay tribute to potential differential effects within this region.
A potential caveat that should be considered is the mixing of effects in the fMRI analyses (main and PPI) in which one block consisted of multiple letter presentations, but also tone presentations. While this should be generally considered, a condition-effect is likely to be caused by the manipulated letter presentation (n-back), however, an interaction with the (stable) tone presentation cannot be ruled out.
Another aspect refers to trial-by-trial fluctuations as they have been shown to be of predictive value in simultaneous EEG-fMRI designs . While regarding auditory trial-by-trial coupling is a matter of ongoing debate , our design with an inter-trial interval of 1.4 s of the tones did not allow for an event-related investigation of the BOLD response because of its inertness.
Summarized, we could show that auditory cortices are increasingly connected to exactly those regions, which are up-regulated during increasing demands of cognitive/attentional control. We further demonstrate that cognitive load crossmodally manipulates auditory-cortex functional connectivity patterns via mechanisms of spread of attention. This causes a re-allocation of neural networks associated with the generation of a secondary sensory memory signal. To what extent the identified nodes actually represent neural generators of the AEP remains to be explicitly tested.
This work was supported by the Deutsche Forschungsgemeinschaft (DFG: IRTG1328, KFO-112/TP9: Ha3202/2-2), the Interdisciplinary Center for Clinical Research of the Medical Faculty of the RWTH Aachen University (N2-6, N4-4), the Research Council KUL (GOA MaNet), and the Flemish Government (FWO G.0427.10N, Integrated EEG-fMRI).CR is supported by a start-up grant of the IRTG1328 (DFG). MDV is supported by an Alexander von Humboldt stipend. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.