This study highlights demographic differences in subjects recruited into the convenience-sample ADNI cohort compared with subjects recruited into the population-based MCSA cohort, and demonstrates that imaging biomarkers from these two different recruitment mechanisms differ.
The most striking difference was that rates of decline in hippocampal volume were greater in ADNI compared to the MCSA, for both CN and aMCI subjects. This difference was observed even after matching for key demographic and cognitive variables. Increased rates of decline in hippocampal volume in CN subjects predict a faster rate of progression to dementia28
, suggesting that the ADNI CN population includes a larger proportion of subjects on the path to AD dementia. While it was somewhat unexpected that the proportion of APOE e4 carriers was not higher in the ADNI CN subjects, our findings are consistent with the unusually high proportion (50%) of ADNI controls who showed amyloid pathology as measured by Pittsburgh Compound B (PiB)29
. By contrast, the proportion of PiB positive controls in the MCSA was only 30%30
. The pathological diagnosis of AD was also more common in controls from a clinic versus a community setting in a previous study31
. The ADNI CN subjects were more highly educated than the MCSA CN subjects; therefore, cognitive reserve mechanisms may have protected them from clinical decline even though they are on a steeper downward trajectory of brain atrophy. Similarly, the higher rates of atrophy suggest that the aMCI group in ADNI consists of a higher proportion of subjects with a more aggressive disease than the MCSA. Indeed, the aMCI subjects in ADNI had a higher proportion of APOE e4 carriers than those in the MCSA in sample 1. Once again, the aMCI subjects in ADNI were more highly educated than those in the MCSA suggesting that cognitive reserve mechanisms may have protected them from decline on the MMSE and progression to a clinical diagnosis of AD.
We hypothesize that this bias in ADNI is due to the recruitment mechanism. We can speculate that CN subjects who are worried about their cognition would be more likely to attend memory clinics and be more motivated to answer advertisements for the study. Both CN and aMCI subjects with higher education are also more likely to seek medical help at a memory clinic and become involved in observational studies. These highly educated subjects could have a more aggressive underlying disease but are able to compensate cognitively. Amnestic MCI subjects recruited through a population-based study are less likely to have sought medical care at a memory clinic and may have a broader spectrum of cognitive function. In addition, an important motivator for participation in ADNI, and other convenience studies, could be the presence of a family history of dementia. Indeed, ADNI did show a higher proportion of family history compared to MCSA. Although one may assume that similar biases would be observed in the MCSA subjects that agreed to imaging, we have demonstrated that this is not the case; likely because less effort was required to agree for imaging than seek out participation in ADNI. The clinical inclusion criteria for both CN and aMCI differed slightly across the two cohorts. A diagnosis of CN in the MCSA was made by multidisciplinary consensus, which may be more conservative than the method employed in ADNI. Similarly, the diagnosis of aMCI in the MCSA is based on clinical grounds, whereas ADNI relied more on a specific cut-point on a memory test. The ADNI approach is likely to result in the recruitment of more impaired subjects. The reason that this is not reflected in the MMSE scores could be because higher education is providing cognitive reserve, and the MMSE may be insensitive to subtle cognitive impairment. The ADNI also recruited younger subjects than the MCSA, which could also have resulted in the recruitment of subjects with more aggressive disease. Rates of atrophy have been found to be greater in younger aMCI subjects32
, possibly because they have a more pure, and hence aggressive, AD pathology compared to older subjects. Older subjects are more likely to have a mixture of pathologies33
, including cerebrovascular disease34, 35
. However, the trend for higher APOE e4 carrier frequency, younger age, and higher education in convenience samples compared to population-based samples has been observed in other cohorts36–39
, suggesting that this bias may be due to the general recruitment mechanism rather than the specific inclusion criteria employed in ADNI. Our findings suggest that CN and aMCI subjects in ADNI are not representative of the general population, and, importantly, suggest that subjects included in future pre-clinical prevention trials using the same recruitment mechanisms will also not be representative of the population. Finally, our results indicate that even rigorous demographic matching efforts are insufficient to correct for the selection bias.
The only difference observed in baseline hippocampal volumes between ADNI and MCSA was in the CN subjects in sample 1, with larger hippocampal volumes observed in ADNI. This difference is likely being driven by the younger age of the ADNI cohort, since hippocampal volume has been shown to decrease with age40
. After matching for demographic features no differences in hippocampal volume were observed across cohorts. Cross-sectional hippocampal volumes also did not differ across ADNI and the MCSA within the aMCI subjects in sample 1, despite the observed differences in age, education, APOE genotype, and MMSE score. This could suggest that rates of decline in hippocampal volume are more sensitive markers of incident AD than cross-sectional hippocampal volume, perhaps because of the large degree of inter-subject variability in hippocampal volume. TIV also differed between MCSA and ADNI. We suspect that MCSA subjects have larger TIVs because of the northern European heritage of many Minnesotan residents, and the link between these nationalities and tall height41
The strengths of this study are the large numbers of subjects and the use of two samples with and without restrictive correction for major demographic or cognitive confounders. A limitation however, is that while matching was performed on the major demographic factors, it may not eliminate other potential differences, such as in other comorbidities, medication, family history, race and ethnicity, that may influence the imaging findings. ADNI had a higher proportion of minorities than MCSA. The MCSA and ADNI cohorts underwent imaging at different field strengths; however, we demonstrated excellent agreement between hippocampal volumes measured across field strengths (). Scan intervals also differed between ADNI and MCSA, although we adjusted for these differences. While atrophy rates have been shown to accelerate over time in AD32
, the trajectory of change is likely to be approximately linear over these relatively short intervals. Lastly, while the MCSA is a population-based study there may also be some inherent participation biases12
, as is the case with any survey. The MCSA is however representative of Olmsted County in Minnesota, and of US Caucasians in general. The incidence of MCI and the demographic predictors of incident MCI in the MCSA are also similar to those reported in other population-based studies42–44
, including studies that have assessed other racial groups45
Overall, our findings show that subjects recruited into ADNI are not representative of the general population, and instead more closely resemble clinical populations. The imaging findings all point towards ADNI including more CN subjects that are on the path to AD dementia and more aMCI subjects that have a pure and aggressive disease phenotype. Therefore, convenience clinical series may be limited by selection biases. These findings have important implications for the design of future treatment trials. If studies that assess power calculations and sample size estimates are performed in biased convenience samples, the high rates of atrophy will lead to smaller than appropriate sample size estimates and therefore trials could be underpowered to detect treatment effects in the population. In addition, treatment trials that utilize a convenience sample will include a higher proportion of subjects with a pure and aggressive disease, and hence are more likely to detect a treatment effect. However, the magnitude of the treatment effect is likely to be less than expected when the treatment is applied to an unbiased population in which subjects are less likely to have pure AD. Care should also be taken when interpreting imaging studies from convenience samples, like ADNI. Biomarkers identified from these highly selected convenience-samples may not perfectly translate to the general population, and will need to be validated in a population-based sample. This will be particularly important for studies seeking to validate new diagnostic criteria for AD in its various stages, in which imaging biomarkers play an important role.