No differences were detected between repeated single runs nor between single runs and parallel runs on the same workstation and for the same FreeSurfer and OS version. For the same OS version, all Mac workstations produced identical results. However, differences were revealed between:
- Mac and HP workstations
- FreeSurfer versions v4.3.1, v4.5.0, and v5.0.0
- OSX 10.5.8 and OSX 10.6.4/5
Since we did not find any differences between OSX 10.6.4 and OSX 10.6.5, we will use the terms OSX 10.5 and OSX 10.6 henceforth for OSX 10.5.8 and OSX 10.6.4/5, respectively.
The differences are presented in more detail below, starting with an overview and subsequently zooming in on specific structures and data processing comparisons. For the volume measurements, only voxel volumes are considered since the results for tabulated volumes were very similar.
Significance of result differences
A complete overview in the form of colored cells for all comparisons is illustrated in (voxel volume) and (CT). With such reproduction, reminiscent of a DNA microarray, one can get a good impression of the results at a glance. By far the most colored cells were found for the cross-version comparisons (v4.3.1 vs. v5.0.0 and v4.5.0 vs. v5.0.0).
Overview of the statistical significance of voxel volume comparisons for all considered structures.
The same as , but now for the cortical thickness comparisons.
As described above, the level of statistical significance after correction for multiple comparisons depends on the data processing contrast being considered, see . It turned out that for the volumes, significant differences were derived only for the cross-version contrasts v5.0.0 vs. the two earlier versions (both Mac and HP). However, for CT, significant differences were found also for some other data processing comparisons. If FDR correction was applied to the volume results, then almost all the colored cells for the cross-version contrasts v5.0.0 vs. v4.3.1/v4.5.0 (both Mac and HP) in represent significant differences. In fact, about half of the 178 structures were significant. With a conservative Bonferroni correction, about a quarter of the structures were significant. For CT rather similar results were obtained: about half/a quarter of the 68 cortical structures were significant after FDR/Bonferroni correction in case of the cross-version contrasts v5.0.0 vs. v4.3.1/v4.5.0 (both Mac and HP). Furthermore, in case of the HP vs. Mac and Mac OSX 10.6 vs. OSX 10.5 contrasts, significant CT differences were found for versions v4.3.1 and v4.5.0, whereas no significant CT differences were present at all for version v5.0.0.
Correction for multiple comparisons on volume and cortical thickness differences.
Strength of result differences
A summary of a subset of descriptive statistics for all structures is given in (voxel volume) and (CT). More details can be found in the Supplementary material, e.g., Table S1
for voxel volumes, Table S2
for tabulated volumes, and Table S3
for cortical thickness. The largest differences (mean as well as range) were found for the cross-version contrasts v4.3.1/v4.5.0 vs. v5.0.0 (both Mac and HP). Generally, the mean (signed or absolute) differences and the COVs for these contrasts were about a factor of two larger than for the other contrasts. The differences manifested a large variation across the structures. The largest absolute volume difference was found in the 5th ventricle and Mac v4.5.0 vs. v5.0.0 contrast: about 64%. However, for the other contrasts some absolute volume differences were large too (about 40%). These findings translated to correspondingly low ICC values and overlap measures (SI). For CT, the largest absolute thickness difference was found in the right isthmus cingulate cortex and Mac v4.3.1 vs. v5.0.0 contrast: about 7.7%. Note that generally the ICC values for CT were larger than for the volume: they were all above 0.5547 compared to 0.0000 for volume measures. Of particular note is that the results of the HP vs. Mac contrast are almost identical to those of the OSX 10.6 vs. OSX 10.5 contrast, both for volume and CT values.
Summary of voxel volume differences.
Summary of cortical thickness differences.
Overlays of differences for version v4.3.1 vs. v5.0.0 (Mac)
To zoom in on the largest differences observed and to show also the corresponding statistical significances, overlays were produced on the inflated pial surfaces of an average brain (so-called “fsaverage”, supplied by Freesurfer) for the comparison between Mac version v4.3.1 and v5.0.0. displays the results for the grey matter (GM) cortical structures. Again, these overlays demonstrate a non-uniformity in difference across the cortex. Note the highly significant (−log10(p)≥4; p≤0.0001) differences for the left and right frontal pole, left and right insula, left and right isthmus cingulate cortex, left and right medial orbital frontal cortex, left and right rostral anterior cingulate cortex, left inferior temporal gyrus, left rostral middle frontal cortex, left temporal pole, right fusiform gyrus, right lateral orbital frontal cortex, and right parahippocampal gyrus.
The differences in cortical grey matter volumes between FreeSurfer version v4.3.1 and v5.0.0 on a Mac (OSX 10.5).
The results for the cortical white matter (WM) structures (i.e., cortically associated gyral WM structures) are depicted in . Although it may not be anatomically correct, these structures were overlaid also on the pial surfaces for visualisation purposes. The pattern of WM structures showing highly significant differences was rather similar to the pattern found for GM. Note that almost all of the largest differences are associated with the smallest p-values for both GM and WM.
The same as , but now for cortical white matter.
For the subcortical structures we generated overlays on coronal, sagittal and transversal slices of T1 data of a single participant, transformed to the MNI305 standard space (). Structures showing highly significant differences were the brain stem, the right amygdala, the right accumbens area, and the anterior, mid-posterior, and posterior partitions of the corpus callosum, left cerebellar white matter, and finally, the left lateral ventricle. Note that the fragmentation of some structures (e.g., cerebellum) is due to the application of a nearest neighbor interpolation in the transformation to the MNI305 template.
The differences in subcortical grey matter volumes between FreeSurfer version v4.3.1 and v5.0.0 on a Mac (OSX 10.5).
The pial surface overlays for the CT values are displayed in . In this case highly significant differences were found for the structures left and right rostral anterior cingulate cortex, left and right isthmus cingulate cortex, left postcentral gyrus, left superior parietal cortex, left superior frontal gyrus, right insula, right pars triangularis, and right posterior cingulate cortex.
The same as , but now for cortical thickness.
Volume differences for specific set of structures
The percentage voxel volume absolute differences for a number of structures frequently used in the literature (c.f., 
) are shown in . Examination of the contrasts Mac vs. HP (panel A) and OSX 10.5 vs. OSX 10.6 (panel C) illustrates the large volume differences for the left and right entorhinal cortex and left and right parahippocampal gyrus (up to about 10% and 5%, respectively). Also the left and right pallidum showed relatively large volume differences for v4.3.1. There was a slight tendency for a more robust segmentation in case of v5.0.0. The largest differences (up to about 17%) were observed between Mac v4.3.1 and v5.0.0 and between v4.5.0 and v5.0.0 (panel B). The structures found to be sensitive were: the left (especially) and right accumbens, left and right amygdala, entorhinal cortex, and parahippocampal gyrus. Moreover, large differences for the left and right pallidal volumes were present between v4.3.1 and v4.5.0 (about 7%), between v4.3.1 and v5.0.0 (about 10%), and between v4.5.0 and v5.0.0 (about 5%). The cross-version results for the HP were very similar. For almost all shown structures, the differences were significant in case of the v4.3.1/v4.5.0 vs. v5.0.0 contrast. See Figure S1
for similar graphs for CT.
Effects of data processing conditions on the voxel volumes for a subsample of (sub)cortical structures.
regarding the determinant of the Talairach transformation matrix and variability under Mac OSX 10.6 can be found in Document S1
and Table S4