We have shown that applying Lagrange Multiplier diagnostic tests to fMRI data consistently identifies violations in model assumptions. This confirms the necessity of rigorous model criticism for fMRI analysis.
The FMRIDC data sets contain multiple scanning sessions for each subject performing identical experiments. Examining the patterns of model violations detected reveals strong consistency in results between repeated sessions; such that exactly replicating an experiment in separate scanning sessions usually gives the same decision on whether the basic fMRI model has been violated somewhere within the data volume. This consistency within results between sessions reaffirms the necessity of performing further analysis even in cases where only a few voxels are detected as violating model assumptions for a given LM-anomaly test.
The pattern of detected anomaly varies between experiments, subjects and stimulus conditions but is empirically consistent within each of those factors for these particular data. In particular the anomaly maps from experiments 119AG and 1198T show particularly clustered patterns of anomaly corresponding variously with those areas of brain identified in the published work as well as other areas of cortex. Anomaly maps from the 119FM and NeuRA experiments show qualitatively sparse anomaly patterns distributed throughout the particular brain volumes with less clustering of voxels. The model violation appearing to be induced from physical motion artefact in the NeuRA data is equally important and relevant as those anomalies from a physiological origin, since this source of error may not have been visually apparent in subsequent BOLD activation maps.
The discrepancy in observing apparent motion artefact between different experiments could be explained by the large differences in scanning time per session () between the NeuRA and FMRIDC data. We expect a longer cumulative scanning time to give rise to more movement artefact in fMRI data. This is observed in the data analysed herein.
Detailed results from the single data volume in Section V-B demonstrate a typical diagnostic analysis.
The experiment involved various cognitive activities from the time of stimulus (observing numbers) to making a decision (button press). Previous analyses for BOLD signal within this data [15
] had shown cognitive activation in expected areas of visual cortex, motor cortex and medial frontal lobe. The results presented herein demonstrate the violation of three common fMRI model assumptions in parts of this particular data volume, indicating a need for further analysis using more appropriate models.
It is particularly interesting that a conventional test for BOLD activation using the nonparametric continuous Laguerre polynomial basis set for the HRF model detects activation in almost exactly the same areas as the LM-anomaly map for violation of the SPM Double Gamma model. In contrast, the conventional BOLD activation test using the SPM Double Gamma HRF detects almost no activation at all. Again the point here is that with almost no extra computation the anomaly map is showing that the SPM Double Gamma fails badly on this data. The Laguerre-zis happening in this case.
It is important to reiterate that the Double Gamma anomaly test is not affected by temporal drift artefact within the data, since that effect is included in the null hypothesis model. Other possible confounding factors such as Time-to-Onset delay of the HRF (see [15
]) are assumed to be zero in both the Laguerre and Double Gamma models, so the model comparison is not affected. We note that this analysis does not use the exact same processing pipeline as SPM software, due to our slightly different noise modelling and test statistic threshold selection. However the analysis is consistent since we use the same noise model throughout all our analyses. Therefore the Double Gamma LM test results only indicate model violation for the exact SPM Double Gamma HRF specification and do not reflect on the efficacy of other aspects of SPM software analysis. Equivalent LM model violation tests can easily be designed for other Double Gamma HRF specifications.
An obvious interpretation of LM-test model violation results is to show possible areas of false-positive activation detection, when LM-anomaly maps overlap with activation maps. However it is equally important to consider the indications of false-negative results, arising when a region of anomaly does not overlap with detected BOLD activation regions; lack of activation detection may mean some anomaly suppresses it as in the Double Gamma case above.
Our test for BOLD signal non-linearity is based on a physiologically motivated non-linear model of behaviour within the brain, estimating BOLD signal as an interaction between blood volume and blood flow. Although this is a viable model, further research has since extended and refined understanding of non-linear BOLD behaviour (e.g. [39
]) as well as the non-linear adaptation of raw neural signal components [41
]. It would be interesting to test and compare the appropriateness of such advanced non-linear models using Lagrange Multiplier anomaly tests, further exploring the relationship between multiple stimuli, study design, scan parameters and inter-session variability.
To the best of our knowledge there are no existing theoretical tests for the presence of BOLD signal non-linearity. Volterra characterisation of the non-linear Balloon model [39
] treats the hemodynamic system as a black box, hence only judging the non-linearity fit through empirical comparison of the model and associated Volterra series expansion. In contrast, the LM method directly and quantitatively tests specific model characteristics, without needing to actually fit the alternative model.
Other future development of Lagrange Multiplier methods could include testing for fMRI model violation in such assumptions as spatial independence, validity of the ‘Double Gamma with temporal derivative’ HRF specification and stationary noise distributions.
In the examples presented, the model assumptions are not violated at every ‘active’ voxel which supposedly contains relevant BOLD signal. Nevertheless we suggest that data showing any model violation should be re-analysed using more advanced methodology appropriate to the types of model violations found. As outlined in Section III-D, the alternative model is specified already as part of the anomaly test derivation.
Fitting that alternative model (e.g. for subsequent BOLD activation testing if that is the aim of the experiment) can present challenges depending on the circumstance. Mainstream fMRI software packages offer some choice in model selection, e.g. using a FIR model for the HRF estimation; however many suitable model choices are not straightforward to implement or are available only from in-house software development. This presents an ongoing challenge to provide suitable, accessible software for the fMRI community.