PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of neurologyNeurologyAmerican Academy of Neurology
 
Neurology. 2012 May 1; 78(18): 1376–1382.
PMCID: PMC3345787

Predicting missing biomarker data in a longitudinal study of Alzheimer disease

Raymond Y. Lo, MD, MScorresponding author

Scientific Advisory Boards:

  1. NONE

Gifts:

  1. NONE

Funding for Travel or Speaker Honoraria:

  1. NONE

Editorial Boards:

  1. NONE

Patents:

  1. NONE

Publishing Royalties:

  1. NONE

Employment, Commercial Entity:

  1. NONE

Consultancies:

  1. NONE

Speakers’ Bureaus:

  1. NONE

Other Activities:

  1. NONE

Clinical Procedures or Imaging Studies:

  1. NONE

Research Support, Commercial Entities:

  1. NONE

Research Support, Government Entities:

  1. NONE

Research Support, Academic Entities:

  1. NONE

Research Support, Foundations and Societies:

  1. NONE

Stock/Stock Options/Board of Directors Compensation:

  1. NONE

License Fee Payments, Technology or Inventions:

  1. NONE

Royalty Payments, Technology or Inventions:

  1. NONE

Stock/Stock Options, Research Sponsor:

  1. NONE

Stock/Stock Options, Medical Equipment & Materials:

  1. NONE

Legal Proceedings:

  1. NONE
and William J. Jagust, MD, For the Alzheimer's Disease Neuroimaging Initiative

Scientific Advisory Boards:

  1. Genentech 2009, 2011

Gifts:

  1. NONE

Funding for Travel or Speaker Honoraria:

  1. NONE

Editorial Boards:

  1. Associate Editor, Frontiers in Human Neuroscience (current). Editorial Board Annals of Neurology, Brain Imaging and Behavior. Alzheimer’s Disease and Associaed Disorders (current)

Patents:

  1. NONE

Publishing Royalties:

  1. Imaging the Aging Brain, Oxford University Press 2009, Jagust and D’Esposito, eds

Employment, Commercial Entity:

  1. NONE

Consultancies:

  1. Consultant Synarc 2008, 2009, 2010, 2011 Consultant Elan/Janssen Alzheimer Immunotherapy 2008, 2009, 2010, 2011 Consultant Genentech 2009, 2011 Consultant Abbott Pharmaceuticals 2010 Consultant GE Healthcare 2010 Consultant Bayer Healthcare 2010 Consultant TauRx 2010 Consultant Otsuka Pharmaceuticals 2010 COnsultant Merck & Co, 2009

Speakers’ Bureaus:

  1. NONE

Other Activities:

  1. NONE

Clinical Procedures or Imaging Studies:

  1. NONE

Research Support, Commercial Entities:

  1. NONE

Research Support, Government Entities:

  1. NIH, Principal Investigator on the following grants: AG034570 AG027859 AG027984 Alzheimer’s Association ZEN-08-87090 NIH Co-Investigator on the following grants: AG036535 AG032306 AG031563 AG030048 AG024904

Research Support, Academic Entities:

  1. NONE

Research Support, Foundations and Societies:

  1. Alzheimer’s Association 2008-2011

Stock/Stock Options/Board of Directors Compensation:

  1. NONE

License Fee Payments, Technology or Inventions:

  1. NONE

Royalty Payments, Technology or Inventions:

  1. NONE

Stock/Stock Options, Research Sponsor:

  1. NONE

Stock/Stock Options, Medical Equipment & Materials:

  1. NONE

Legal Proceedings:

  1. NONE
From the Division of Epidemiology (R.Y.L., W.J.J.) and Helen Wills Neuroscience Institute (W.J.J.), University of California, Berkeley; and Department of Neurology (R.Y.L.), Buddhist Tzu Chi General Hospital, Hualien, Taiwan.
Paul Aisen, MD, Clifford R. Jack, Jr., MD, Arthur W. Toga, PhD, Laurel Beckett, PhD, Anthony Gamst, PhD, Holly Soares, PhD, Robert C. Green, MD, MPH, Tom Montine, MD, PhD, Ronald G. Thomas, PhD, Michael Donohue, PhD, Sarah Walter, MSc, Anders Dale, PhD, Matthew Bernstein, PhD, Joel Felmlee, PhD, Nick Fox, MD, Paul Thompson, PhD, Norbert Schuff, PhD, Gene Alexander, PhD, Charles DeCarli, MD, Dan Bandy, MS,, Kewei Chen, PhD,, John Morris, MD, Virginia M.-Y. Lee, PhD, MBA, Magdalena Korecka, PhD, Karen Crawford, Scott Neu, PhD, Danielle Harvey, PhD, John Kornak, PhD, Andrew J. Saykin, PsyD, Tatiana M. Foroud, PhD, Steven Potkin, MD, Li Shen, PhD, Neil Buckholtz, PhD, Jeffrey Kaye, MD, Sara Dolen, BS, Joseph Quinn, MD, Lon Schneider, MD, Sonia Pawluczyk, MD , Bryan M. Spann, DO, PhD, James Brewer, MD, PhD, Helen Vanderswag, RN, Judith L. Heidebrink, MD, MS, Joanne L. Lord, LPN, BA, CCRC, Ronald Petersen, MD, PhD, Kris Johnson, RN, Rachelle S. Doody, MD, PhD, Javier Villanueva-Meyer, MD, Munir Chowdhury, MS, Yaakov Stern, PhD, Lawrence S. Honig, MD, PhD, Karen L. Bell, MD, John C. Morris, MD, Mark A. Mintun, MD, Stacy Schneider, APRN, BC, GNP, Daniel Marson, JD, PhD, Randall Griffith, PhD, ABPP, David Clark, MD, Hillel Grossman, MD, Cheuk Tang, PhD, George Marzloff, BS, Leylade Toledo-Morrell, PhD, Raj C. Shah, MD, Ranjan Duara, MD, Daniel Varon, MD, Peggy Roberts, CNA, Marilyn S. Albert, PhD, Julia Pedroso, MA, Jaimie Toroney, BA, Henry Rusinek, PhD, Mony J de Leon, EdD, Susan M De Santi, PhD, P. Murali Doraiswamy, MD, Jeffrey R. Petrella, MD, Marilyn Aiello, BS, Christopher M. Clark, MD, Cassie Pham, BS, Jessica Nunez, Charles D. Smith, MD, Curtis A. Given, II, MD, Peter Hardy, PhD, Oscar L. Lopez, MD, MaryAnn Oakley, MA, Donna M. Simpson, CRNP, MPH, M. Saleem Ismail, MD, Connie Brand, RN, Jennifer Richard, BA, Ruth A. Mulnard, DNSc, RN, FAAN, Gaby Thai, MD, Catherine Mc-Adams-Ortiz, MSN, RN, A/GNP, Ramon Diaz-Arrastia, MD, PhD, Kristen Martin-Cook, MA, Michael DeVous, PhD, Allan I. Levey, MD, PhD, James J. Lah, MD, PhD, Janet S. Cellar, RN, MSN, Jeffrey M. Burns, MD, Heather S. Anderson, MD, Mary M. Laubinger, MPA, BSN, George Bartzokis, MD, Daniel H.S. Silverman, MD, PhD, Po H. Lu, PsyD, Neill R Graff-Radford MBBCH, FRCP, Francine Parfitt, MSH, CCRC, Heather Johnson, MLS, CCRP, Martin Farlow, MD, Scott Herring, RN, Ann M. Hake, MD, Christopher H. van Dyck, MD, Martha G. MacAvoy, PhD, Amanda L. Benincasa, BA, Howard Chertkow, MD, Howard Bergman, MD, Chris Hosein, M.Ed, Sandra Black, MD FRCP(C), Simon Graham, PhD, Curtis Caldwell, PhD, Ging-Yuek Robin Hsiung, MD, MHSc, FRCPC, Howard Feldman, MD FRCP(C), Michele Assaly, MA, Andrew Kertesz, MD, John Rogers, MD, Dick Trost, PhD, Charles Bernick, MD, Donna Munic, PhD, Chuang-Kuo Wu, MD PhD, Nancy Johnson, PhD, Marsel Mesulam, MD, Carl Sadowsky, MD, Walter Martinez, MD, Teresa Villena, MD, Scott Turner, M.D., Kathleen B. Johnson, ANP, Kelly E. Behan, B.A., Reisa A. Sperling, MD, Dorene M. Rentz, PsyD, Keith A. Johnson, MD, Allyson Rosen, PhD, Jared Tinklenberg, MD, Wes Ashford, MD, PhD, Marwan Sabbagh, MD, FAAN, CCRI, Donald Connor, PhD, PhD, Sandra Jacobson, MD, Ronald Killiany, PhD, Alexander Norbash, MD, Anil Nair, MD, Thomas O. Obisesan, MD, MPH, Annapurni Jayam-Trouth, MD, Paul Wang, PhD, Alan Lerner, MD, Leon Hudson, MPH, Paula Ogrocki, PhD, Charles DeCarli, MD, Evan Fletcher, PhD, Owen Carmichael, PhD, Smita Kittur, MD, Seema Mirje, MBBS, Michael Borrie, MD, T-Y Lee, PhD, Dr Rob Bartha, PhD, Sterling Johnson, PhD, Sanjay Asthana, MD, Cynthia M. Carlsson, MD, Steven G. Potkin, MD, Adrian Preda, MD, Dana Nguyen, PhD, Pierre Tariot, MD, Adam Fleisher, MD, Stephanie Reeder, BA, Vernice Bates, MD, Horacio Capote, MD, Michelle Rainka, PhD, Barry A. Hendin, MD, Douglas W. Scharre, MD, Maria Kataki, MD, PhD, Earl A. Zimmerman, MD, Dzintra Celmins, MD, Alice D. Brown, FNP, Sam Gandy, MD, PhD, PhD, Marjorie E. Marenberg, MD, Barry W. Rovner, MD, Godfrey Pearlson, MD, Karen Anderson, RN, Andrew J. Saykin, PsyD, Robert B. Santulli, MD, Jessica Englert, PhD, Jeff D. Williamson, MD, MHS, Kaycee M. Sink, MD, MS, Franklin Watkins, MD, Brian R. Ott, MD, Chuang-Kuo Wu, MD, PhD, Ronald Cohen, PhD, Stephen Salloway, MD, MS, Paul Malloy, PhD, Stephen Correia, PhD, Howard J. Rosen, MD, Bruce L. Miller, MD, and Jacobo Mintzer, MD

Abstract

Objective:

To investigate predictors of missing data in a longitudinal study of Alzheimer disease (AD).

Methods:

The Alzheimer's Disease Neuroimaging Initiative (ADNI) is a clinic-based, multicenter, longitudinal study with blood, CSF, PET, and MRI scans repeatedly measured in 229 participants with normal cognition (NC), 397 with mild cognitive impairment (MCI), and 193 with mild AD during 2005–2007. We used univariate and multivariable logistic regression models to examine the associations between baseline demographic/clinical features and loss of biomarker follow-ups in ADNI.

Results:

CSF studies tended to recruit and retain patients with MCI with more AD-like features, including lower levels of baseline CSF Aβ42. Depression was the major predictor for MCI dropouts, while family history of AD kept more patients with AD enrolled in PET and MRI studies. Poor cognitive performance was associated with loss of follow-up in most biomarker studies, even among NC participants. The presence of vascular risk factors seemed more critical than cognitive function for predicting dropouts in AD.

Conclusion:

The missing data are not missing completely at random in ADNI and likely conditional on certain features in addition to cognitive function. Missing data predictors vary across biomarkers and even MCI and AD groups do not share the same missing data pattern. Understanding the missing data structure may help in the design of future longitudinal studies and clinical trials in AD.

Missing data are common in cohort studies, particularly in Alzheimer disease (AD) research.1 Higher mortality risk and cognitive impairment hinder older adults from staying in studies requiring multiple visits and thus result in incomplete data.2 Although statistical methods have been developed to handle missing data in repeated-measures studies,35 the underlying mechanism for missing data is rarely examined in actual studies.

Most longitudinal studies of AD use complete data for analysis and ignore missing data, assuming the complete data are a random sample drawn from the entire study population, so-called missing completely at random (MCAR).6 A less stringent assumption, missing at random (MAR),6 may be satisfied if missingness does not depend on the variable itself, conditional on observed covariates. If missingness does depend on the variable itself, even after accounting for observed covariates, then data are said to be missing not at random (MNAR).6 Analysis methods should be used which are appropriate to the type of missingness at work. However, it is important to note that it is not possible to distinguish between MAR and MNAR based on observed data, suggesting sensitivity analyses ought to ideally be performed.

In this study we examined the missing data structure of the Alzheimer's Disease Neuroimaging Initiative (ADNI), an AD longitudinal study with multiple biomarkers repeatedly measured, in an attempt to understand the direction of bias due to dropouts, which we believe is essential to developing strategies to retain cases in future longitudinal studies and to inform how the ADNI data themselves are analyzed.

METHODS

Study population.

This is a cohort study with 3 subgroups. A total of 819 research participants (NC: 229; MCI: 397; AD: 193) were enrolled in the ADNI study from 59 centers in the United States and Canada during 2005–2007. Full inclusion/exclusion criteria are detailed at www.adni-info.org. Briefly, screening criteria for entry into the study included the Mini-Mental State Examination score (MMSE), Clinical Dementia Rating scale (CDR), and an education-adjusted cutoff score on delayed recall of 1 paragraph from the Logical Memory subtest of the Wechsler Memory Scale–Revised.7 All participants were recruited between the ages of 55 and 90, and had at least 6 years of education and a study partner able to provide an independent evaluation of functioning. Specific psychoactive medications or other neurologic disorders were excluded.

Standard protocol approvals, registrations, and patient consents.

The study procedures were approved by institutional review boards of all participating institutions. Written informed consents to blood sampling, lumbar puncture, neuropsychological testing, and neuroimaging were obtained from all research participants or their representatives.

Follow-up timeline.

Detailed schedules of assessment for NC, MCI, and AD are posted in the general procedure manual on the ADNI Web site (http://adni.loni.ucla.edu/wpcontent/uploads/2010/09/ADNI_GeneralProceduresManual.pdf).

Briefly, after the baseline visit, subsequent visits took place at 6- or 12-month intervals in person. Participants with NC or MCI were followed up for 3 years, while those with AD for 2 years at maximum. The visit schedules for collecting biomarkers were similar but not the same for NC, MCI, and AD groups. Participants might visit the research clinic for other assessments without consenting or completing certain biomarker tests. We used the data from ADNI up to April 19, 2011.

Biomarkers.

Missing data for blood homocysteine, CSF Aβ42 and tau proteins, [18F]fluorodeoxyglucose PET (FDG-PET), and volumetric MRI were examined in ADNI.

Biofluids.

Serum and plasma samples from blood were prepared separately for all participants at each visit. Blood and CSF samples were collected and analyzed using a standardized protocol.8 Biochemical profiles including homocysteine in blood samples and Aβ42, total-tau, and phosphorylated-tau in CSF were measured. The study was targeted to acquire baseline CSF samples for at least 20% of total participants by approaching any potential subject who might be interested.

PET.

The protocol to acquire ADNI PET data at sites nationwide is detailed at www.loni.ucla.edu/ADNI/Data/ADNI_Data.shtml, and methods for FDG-PET analysis have been described previously.9 The study was targeted to acquire baseline PET scans for 50% of total participants. While inclusion in the PET protocol was randomly assigned, participants were free to decline to enter this arm of the study.

MRI.

The 1.5-T MRI protocol was described elsewhere,10 which was standardized across all sites and the acquisition time was approximately 30 minutes. The analyses we report here used FreeSurfer software (http://surfer.nmr.mgh.harvard.edu) to obtain bilateral hippocampal volumes in mm3. The study was targeted to acquire baseline MRI scans for all participants; individuals who refused MRI could not enroll.

Predictors of missing biomarkers.

Predictors of interest were baseline demographic and clinical features that were likely associated with both cognitive impairment (study outcome) and loss of follow-up (missingness).

Demographic features.

Age, sex, years of formal education, smoking, and family history of AD were recorded at enrollment. Occupation types were recorded and classified into 3 levels: 1) professional or managerial; 2) skilled; 3) partly skilled or unskilled occupations according to The National Statistics Socioeconomic Classification.11 APOE genotyping was carried out at the University of Pennsylvania AD Biomarker Laboratory. APOE4 gene carriers were participants who had at least 1 APOE 4 allele. Premorbid intelligence indicated by number of errors (range 0–50) in American National Adult Reading Test was evaluated at baseline as part of the neuropsychological battery.12

Clinical assessments.

Body mass index was measured at baseline. The number of comorbid illnesses was documented regardless of severity or chronicity. Cardiovascular risk score was calculated using the office-based cardiovascular risk profile prediction function from the Framingham Heart Study13; higher scores indicate higher risks of cardiovascular events. Gait function was assessed as part of the neurologic examination. Functional Assessment Questionnaire,14 Geriatric Depression Scale,15 and Neuropsychiatric Inventory Questionnaire16 were all included to reflect the global function and behavior of participants.

Cognitive measures.

CDR scale,17 MMSE score, and Alzheimer's Disease Assessment Scale–Cognitive Subscale (ADAS-Cog) were used to evaluate cognitive performance at enrollment.

Statistical analysis.

Predictors were treated as continuous variables, except sex, smoking, family history of AD, APOE4 carrier, and gait, which were dichotomous. We first examined factors that influenced whether biomarkers were obtained at baseline. The outcome was the indicator (missing = 1; nonmissing = 0) of missing data for biomarkers (blood, CSF, PET, and MRI) in each diagnostic group (NC, MCI, and AD) and the aforementioned demographic, clinical, and cognitive predictors were entered into the logistic regression model one at a time for univariate analyses. Odds ratios (ORs) were calculated; ORs >1 indicated increased probability of missingness and ORs <1 indicated increased probability of remaining in the study for each unit increase of predictors. Significant predictors in univariate models were subsequently pooled into a multivariable model to test the robustness as some of these predictors might correlate with one another. MCAR assumptions would be violated if the missingness was associated with any of these predictors.

Secondly, we were interested in factors associated with loss to follow-up once participants enrolled in biomarker studies. For participants who had baseline biomarkers, we defined longitudinal missingness as having only baseline without further lumbar puncture for CSF biomarkers and having only measures within the first year for blood, PET, and MRI biomarkers without longer follow-ups. In addition to the predictors above, we included baseline biomarker values (blood homocysteine, CSF Aβ42 and tau, FDG-PET ROIs, MRI hippocampal volume) in these longitudinal analyses.

All statistical analyses and graphics were performed in R version 2.11.1. All tests of statistical significance were conducted at the 2-tailed α level of 0.05.

RESULTS

Baseline demographic and clinical features, biomarker values, and year of last visit in ADNI are shown in table 1. Regardless of whether biomarkers were obtained at the visit, most participants were followed up for over a year (NC: 93%, MCI: 85%, AD: 81%); there were 8 participants (NC: 1; MCI: 4; AD: 3) who died during the first year and 23 who died during the 3-year observation. All participants had at least 1 blood test (819/819, 100%) with the majority having a MRI scan (814/819, 99%), and more than half of participants in each diagnostic group had at least 1 CSF study (418/819, 51%) or 1 PET scan (455/819, 56%). Although the sample size in general shrank over time, the majority of participants who had baseline tests had biomarkers repeatedly measured longer than a year.

Table 1
Baseline characteristics of 819 participants in ADNI

In CSF studies, a family history of AD was associated with having CSF measured at baseline for participants with MCI or AD, but no evidence was found against MCAR for the NC group at enrollment (table e-1 on the Neurology® Web site at www.neurology.org). During follow-ups for CSF biomarkers, higher baseline ADAS-Cog scores (worse cognitive performance) predicted dropouts for NC and higher levels of baseline β-amyloid in CSF predicted dropouts for MCI (table 2). Thus the NC group tended to keep cognitively normal participants while the MCI group tended to recruit individuals with an AD family history and retain those who were more AD-like in the longitudinal CSF study.

Table 2
Univariate association with missing CSF during follow-upa

In PET studies, we found no evidence against MCAR for the NC group at enrollment. MCI participants with lower ADAS-Cog scores (better cognitive performance) as opposed to AD participants with more neuropsychiatric complaints and higher CDR scores were more likely to be included in PET studies (table e-2). During follow-ups for PET, female normal participants were more likely to drop out, depression and lower cognitive performance predicted missing data in the MCI group, while family history of AD, APOE4 carrier, and higher cardiovascular risk scores were associated with dropouts in the AD group (table 3). Baseline FDG-PET results did not predict missing data in subsequent visits for all 3 groups.

Table 3
Univariate association with missing PET during follow-upa

During follow-ups for MRI after the first year, poor cognitive performance (lower MMSE scores and higher ADAS-Cog scores) was predictive of missing data even for the NC group; depression stood out among all other factors in a multivariable model to be associated with dropouts in MCI; and a family history of AD and higher CDR scores characterized AD participants who stayed in the study. Baseline MRI hippocampal volume was not predictive of missing data during follow-ups (table 4).

Table 4
Univariate association with missing MRI during follow-upa

For blood tests, lower cognitive performance predicted missing data for NC and MCI during follow-ups. Higher cardiovascular risk scores and higher baseline levels of serum homocysteine were associated with dropouts in AD (table 5).

Table 5
Univariate association with missing blood sample during follow-upa

DISCUSSION

The missing data structure varied across different biomarkers that were repeatedly measured in ADNI. For at least some of the measured parameters we show that missingness is not MCAR, although whether it is MAR or MNAR cannot be determined based on the observed data. Our findings indicate that using complete data analysis may result in biased estimates and that handling missing data must be tailored to the target biomarker.

MCI participants with positive family histories of AD and lower premorbid verbal intelligence were more likely to be included in CSF studies and a similar pattern was also seen in AD; these findings suggest that MCI/AD recruitment for CSF donation likely captured people with more AD characteristics. Subjects with positive family histories of AD may have learned about AD from family experience and thus be more motivated to participate in AD studies even though the study procedure is invasive. The motivation may be further enhanced when subjects themselves are cognitively impaired, have hopes of finding effective treatments, or in the case of MCI are apprehensive about converting to dementia. During CSF follow-ups, poor cognitive performance in NC and higher baseline CSF Aβ42 in MCI predicted missingness, suggesting the NC group tended to retain relatively normal subjects and the MCI group would retain subjects with lower CSF Aβ42 who have a higher likelihood of converting to AD. Thus using CSF biomarkers to track clinical progression in MCI would be predicted to result in an overestimation of the proportion of converters in longitudinal studies or clinical trials.

Better cognitive function was associated with PET enrollment in MCI. This association, however, did not extend to the AD group who were more likely to enroll if more impaired. The AD group tended to retain APOE4 positive individuals, those with positive family histories, and those with lower cardiovascular risk, suggesting that following up patients with AD using PET scans may capture more purely AD than those with more vascular risk factors. This demonstrates that the missing data structure in MCI and AD should not be assumed to be the same.

Cognitive impairment, particularly decision-making impairment, may reduce the willingness to participate in research18; this may explain our observations in the MCI group. But for patients with AD who have overt dementia, surrogates may have more involvement in the decision-making process,19 which would explain the associations between greater impairment and participation and retention in the PET and MRI components. However, for patients with comorbid illnesses, such as cardiovascular diseases, surrogates may be concerned that the overall benefit/risk ratio does not favor longer participation20 or such subjects may be more likely to drop out due to medical illness. We cannot confirm these explanations without interviewing both patients and study partners, but our observation at least demonstrates that retained patients with MCI and patients with AD in a follow-up study belong to 2 selected groups. These data suggest that caution is required when assuming that MCI and AD represent the same cognitive spectrum, especially when using PET scans to track disease progression.

Loss of follow-up in MRI studies was conditional on poor cognitive performance in both NC and MCI but not in AD, which again suggests that cognitive impairment may have differential influence on following participants with MCI and AD. In line with CSF studies, baseline cognitive performance despite the limited variability among people considered cognitively normal is still associated with long-term dropouts in MRI studies. Similar to PET studies, depression was also associated with missingness in follow-up MRI scans, suggesting that depression is the major factor driving longitudinal missingness of imaging markers among all covariates considered in the study.

Since repeated blood tests are the standard source of biomarkers in population health studies, blood biomarkers can serve as a control variable to compare missing data patterns across different biomarkers. Poor cognitive function seemed to affect participation in long-term follow-ups in NC and MCI groups. After a diagnosis of AD, cognitive function was no longer critical in determining the missingness. Interestingly, similar to the results from the PET studies, higher baseline homocysteine and higher cardiovascular risk in AD were associated with loss of follow-up, suggesting that patients with AD with vascular risk factors may be more likely to drop out of longitudinal studies per se.

Our study has several strengths. First, the design of ADNI emulates a typical clinical trial in terms of case enrollment criteria, multicenter setting, standardized outcome measures, and follow-up protocols, making our results generalizable to other AD clinical trials. However, we recognize that ADNI is not a clinical trial; missingness related to adverse drug effects or hope of improvement cannot be addressed in this observational study. Second, biomarkers in ADNI have been demonstrated to be useful in tracking AD progression. Future clinical trials for AD will likely incorporate these biomarkers to track cognitive decline and similar missing data challenges may be encountered; therefore our ADNI case study is of high reference value. Third, the ADNI study provides comprehensive data on demographic features, laboratory tests, and clinical assessments, allowing us to systematically examine the missing data structure and plausibly test MCAR and MAR assumptions.

There are also several limitations in the study. First, despite the comprehensive approach taken in ADNI, we can never be certain whether missing data are MAR or MNAR based on the observed data. Second, we acknowledge that some ORs were just barely statistically significant and results might be due to multiple comparisons as we included more than a dozen potential predictors in the models. However, all of these predictors were selected based on a priori hypotheses and most of these significant predictors were coherent with the missingness across biomarkers and diagnostic groups rather than reflecting a random set of variables. Third, although one can hypothesize plausible reasons why certain predictors might predict dropout, we could not confirm these, being neither able to interview the individuals nor to collect information on the reasons for missingness. Fourth, 3 diagnostic groups had different visit schedules, making the missing data structures of NC, MCI, and AD less comparable. Thus we should be conservative in making inferences about intergroup difference.

How best to handle missing data is the subject of considerable interest and debate. Ideally the method chosen should be based on the assumptions one is willing to make regarding missingness. For example, popular methods such as multiple imputation, maximum likelihood, or weighted estimating equation methods are typically based on the missing at random assumption.3,6,21 A possible alternative is to stratify by biomarker-specific missingness predictors and perform a complete case analysis, although this increases the complexity of trial design, and assumes that predictors of missingness are consistent across studies.

Longitudinal missingness in ADNI is not completely at random and CSF and imaging markers may bias longitudinal parameters in different directions. Poor cognitive performance at baseline is predictive of missingness even for cognitively normal participants but may be less critical for patients with AD. Depression is a strong predictor for missingness of imaging biomarkers. Patterns of longitudinal missingness may reflect their different levels of accessibility, invasiveness, public awareness, and surrogate decision-making in relation to dementia. Dealing with the missing data in a cohort study or clinical trial for dementia should be tailored to the target biomarker and cognitive stage.

Supplementary Material

Data Supplement:
Coinvestigators:
Accompanying Editorial:

GLOSSARY

AD
Alzheimer disease
ADAS-Cog
Alzheimer's Disease Assessment Scale–Cognitive Subscale
ADNI
Alzheimer's Disease Neuroimaging Initiative
CDR
Clinical Dementia Rating
MAR
missing at random
MCAR
missing completely at random
MCI
mild cognitive impairment
MMSE
Mini-Mental State Examination
MNAR
missing not at random
NC
normal cognition
OR
odds ratio

Footnotes

Alzheimer's Disease Neuroimaging Initiative coinvestigators are listed on the Neurology® Web site at www.neurology.org.

Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (www.loni.ucla.edu\ADNI). As such, the investigators within the ADNI contributed to the design and implementation of ADNI or provided data but did not participate in analysis or writing of this report.

Editorial, page 1370

Supplemental data at www.neurology.org

Contributor Information

Paul Aisen, UC San Diego, ADCS PI and Director of Coordinating Center Clinical Core, Executive Committee.

Clifford R. Jack, Jr., Mayo Clinic, Rochester, Core PI MRI, Executive Committee.

Arthur W. Toga, UCLA, Core PI Informatics, Executive Committee.

Laurel Beckett, UC Davis, Core PI, Biostatistics, Executive Committee.

Anthony Gamst, UC San Diego, Executive Committee.

Holly Soares, Pfizer, Chair, Industry Scientific Advisory Board [iSAB]

Robert C. Green, Boston University, Chair, Data and Publication Committee [DPC]

Tom Montine, University of Washington, Chair, Resource Allocation Review Committee.

Ronald G. Thomas, UC San Diego, Clinical Informatics and Operations.

Michael Donohue, UC San Diego, Clinical Informatics and Operations.

Sarah Walter, UC San Diego, Clinical Informatics and Operations.

Anders Dale, UC San Diego, MRI.

Matthew Bernstein, Mayo Clinic, Rochester, MRI.

Joel Felmlee, Mayo Clinic, Rochester, MRI.

Nick Fox, University of London, MRI.

Paul Thompson, UCLA School of Medicine, MRI.

Norbert Schuff, UCSF MRI, MRI.

Gene Alexander, Banner Alzheimer's Institute, MRI.

Charles DeCarli, UC Davis, MRI.

Dan Bandy, CNMT Banner Alzheimer's Institute, PET.

Kewei Chen, Banner Alzheimer's Institute, PET.

John Morris, Washington University, St. Louis, Neuropathology Core.

Virginia M.-Y. Lee, UPenn School of Medicine, Biomarkers.

Magdalena Korecka, UPenn School of Medicine, Biomarkers.

Karen Crawford, UCLA, Informatics.

Scott Neu, UCLA, Informatics.

Danielle Harvey, UC Davis, Biostatistics.

John Kornak, UC Davis, Biostatistics.

Andrew J. Saykin, Indiana University Genetics.

Tatiana M. Foroud, Indiana University, Genetics.

Steven Potkin, UC Irvine, Genetics.

Li Shen, Indiana University, Genetics.

Neil Buckholtz, National Institute on Aging/National Institutes of Health.

Jeffrey Kaye, Oregon Health and Science University, site investigator.

Sara Dolen, Oregon Health and Science University, site investigator.

Joseph Quinn, Oregon Health and Science University, site investigator.

Lon Schneider, University of Southern California, site investigator.

Sonia Pawluczyk, University of Southern California, site investigator.

Bryan M. Spann, University of Southern California, site investigator.

James Brewer, University of California-San Diego, site investigator.

Helen Vanderswag, University of California-San Diego, site investigator.

Judith L. Heidebrink, University of Michigan, site investigator.

Joanne L. Lord, University of Michigan, site investigator.

Ronald Petersen, Mayo Clinic, Rochester , site investigator.

Kris Johnson, Mayo Clinic, Rochester , site investigator.

Rachelle S. Doody, Baylor College of Medicine, site investigator.

Javier Villanueva-Meyer, Baylor College of Medicine, site investigator.

Munir Chowdhury, Baylor College of Medicine, site investigator.

Yaakov Stern, Columbia University Medical Center, site investigator.

Lawrence S. Honig, Columbia University Medical Center, site investigator.

Karen L. Bell, Columbia University Medical Center, site investigator.

John C. Morris, Washington University, St. Louis, site investigator.

Mark A. Mintun, Washington University, St. Louis, site investigator.

Stacy Schneider, Washington University, St. Louis, site investigator.

Daniel Marson, University of Alabama - Birmingham, site investigator.

Randall Griffith, University of Alabama - Birmingham, site investigator.

David Clark, University of Alabama - Birmingham, site investigator.

Hillel Grossman, Mount Sinai School of Medicine, site investigator.

Cheuk Tang, Mount Sinai School of Medicine, site investigator.

George Marzloff, Mount Sinai School of Medicine, site investigator.

Leylade Toledo-Morrell, Rush University Medical Center, site investigator.

Raj C. Shah, Rush University Medical Center, site investigator.

Ranjan Duara, Wein Center, site investigator.

Daniel Varon, Wein Center, site investigator.

Peggy Roberts, Wein Center, site investigator.

Marilyn S. Albert, Johns Hopkins University, site investigator.

Julia Pedroso, Johns Hopkins University, site investigator.

Jaimie Toroney, Johns Hopkins University, site investigator.

Henry Rusinek, New York University, site investigator.

Mony J de Leon, New York University, site investigator.

Susan M De Santi, New York University, site investigator.

P. Murali Doraiswamy, Duke University Medical Center, site investigator.

Jeffrey R. Petrella, Duke University Medical Center, site investigator.

Marilyn Aiello, Duke University Medical Center, site investigator.

Christopher M. Clark, University of Pennsylvania, site investigator.

Cassie Pham, University of Pennsylvania, site investigator.

Jessica Nunez, University of Pennsylvania, site investigator.

Charles D. Smith, University of Kentucky, site investigator.

Curtis A. Given, II, University of Kentucky, site investigator.

Peter Hardy, University of Kentucky, site investigator.

Oscar L. Lopez, University of Pittsburgh, site investigator.

MaryAnn Oakley, University of Pittsburgh, site investigator.

Donna M. Simpson, University of Pittsburgh, site investigator.

M. Saleem Ismail, University of Rochester Medical Center, site investigator.

Connie Brand, University of Rochester Medical Center, site investigator.

Jennifer Richard, University of Rochester Medical Center, site investigator.

Ruth A. Mulnard, University of California, Irvine, site investigator.

Gaby Thai, University of California, Irvine, site investigator.

Catherine Mc-Adams-Ortiz, University of California, Irvine, site investigator.

Ramon Diaz-Arrastia, University of Texas Southwestern Medical School, site investigator.

Kristen Martin-Cook, University of Texas Southwestern Medical School, site investigator.

Michael DeVous, University of Texas Southwestern Medical School, site investigator.

Allan I. Levey, Emory University, site investigator.

James J. Lah, Emory University, site investigator.

Janet S. Cellar, Emory University, site investigator.

Jeffrey M. Burns, University of Kansas, Medical Center, site investigator.

Heather S. Anderson, University of Kansas, Medical Center, site investigator.

Mary M. Laubinger, University of Kansas, Medical Center, site investigator.

George Bartzokis, University of California, Los Angeles, site investigator.

Daniel H.S. Silverman, University of California, Los Angeles, site investigator.

Po H. Lu, University of California, Los Angeles, site investigator.

Neill R Graff-Radford MBBCH, London- Mayo Clinic, Jacksonville, site investigator.

Francine Parfitt, Mayo Clinic, Jacksonville, site investigator.

Heather Johnson, Mayo Clinic, Jacksonville, site investigator.

Martin Farlow, Indiana University, site investigator.

Scott Herring, Indiana University, site investigator.

Ann M. Hake, Indiana University, site investigator.

Christopher H. van Dyck, Yale University School of Medicine, site investigator.

Martha G. MacAvoy, Yale University School of Medicine, site investigator.

Amanda L. Benincasa, Yale University School of Medicine, site investigator.

Howard Chertkow, McGill Univ., Montreal-Jewish General Hospital, site investigator.

Howard Bergman, McGill Univ., Montreal-Jewish General Hospital, site investigator.

Chris Hosein, McGill Univ., Montreal-Jewish General Hospital, site investigator.

Sandra Black, Sunnybrook Health Sciences, Ontario, site investigator.

Simon Graham, Sunnybrook Health Sciences, Ontario, site investigator.

Curtis Caldwell, Sunnybrook Health Sciences, Ontario, site investigator.

Ging-Yuek Robin Hsiung, U.B.C. Clinic for AD & Related, B.C., site investigator.

Howard Feldman, U.B.C. Clinic for AD & Related, B.C., site investigator.

Michele Assaly, U.B.C. Clinic for AD & Related, B.C., site investigator.

Andrew Kertesz, Cognitive Neurology-St. Joseph's, Ontario, site investigator.

John Rogers, Cognitive Neurology-St. Joseph's, Ontario, site investigator.

Dick Trost, Cognitive Neurology-St. Joseph's, Ontario, site investigator.

Charles Bernick, Cleveland Clinic Lou Ruvo Center for Brain Health, site investigator.

Donna Munic, Cleveland Clinic Lou Ruvo Center for Brain Health, site investigator.

Chuang-Kuo Wu, Northwestern University, site investigator.

Nancy Johnson, Northwestern University, site investigator.

Marsel Mesulam, Northwestern University, site investigator.

Carl Sadowsky, Premiere Research Inst [Palm Beach Neurology], site investigator.

Walter Martinez, Premiere Research Inst [Palm Beach Neurology], site investigator.

Teresa Villena, Premiere Research Inst [Palm Beach Neurology], site investigator.

Scott Turner, Georgetown University Medical Center, site investigator.

Kathleen B. Johnson, Georgetown University Medical Center, site investigator.

Kelly E. Behan, Georgetown University Medical Center, site investigator.

Reisa A. Sperling, Brigham and Women's Hospital, site investigator.

Dorene M. Rentz, Brigham and Women's Hospital, site investigator.

Keith A. Johnson, Brigham and Women's Hospital, site investigator.

Allyson Rosen, Stanford University, site investigator.

Jared Tinklenberg, Stanford University, site investigator.

Wes Ashford, Stanford University, site investigator.

Marwan Sabbagh, Sun Health Research Institute, site investigator.

Donald Connor, Sun Health Research Institute, site investigator.

Sandra Jacobson, Sun Health Research Institute, site investigator.

Ronald Killiany, Boston University, site investigator.

Alexander Norbash, Boston University, site investigator.

Anil Nair, Boston University, site investigator.

Thomas O. Obisesan, Howard University, site investigator.

Annapurni Jayam-Trouth, Howard University, site investigator.

Paul Wang, Howard University, site investigator.

Alan Lerner, Case Western Reserve University, site investigator.

Leon Hudson, Case Western Reserve University, site investigator.

Paula Ogrocki, Case Western Reserve University, site investigator.

Charles DeCarli, University of California, Davis-Sacramento , site investigator.

Evan Fletcher, University of California, Davis-Sacramento , site investigator.

Owen Carmichael, University of California, Davis-Sacramento , site investigator.

Smita Kittur, Neurological Care of CNY, site investigator.

Seema Mirje, Neurological Care of CNY, site investigator.

Michael Borrie, Parkwood Hospital, site investigator.

T-Y Lee, Parkwood Hospital, site investigator.

Dr Rob Bartha, Parkwood Hospital, site investigator.

Sterling Johnson, University of Wisconsin, site investigator.

Sanjay Asthana, University of Wisconsin, site investigator.

Cynthia M. Carlsson, University of Wisconsin, site investigator.

Steven G. Potkin, University of California, Irvine-BIC, site investigator.

Adrian Preda, University of California, Irvine-BIC, site investigator.

Dana Nguyen, University of California, Irvine-BIC, site investigator.

Pierre Tariot, Banner Alzheimer's Institute, site investigator.

Adam Fleisher, Banner Alzheimer's Institute, site investigator.

Stephanie Reeder, Banner Alzheimer's Institute, site investigator.

Vernice Bates, Dent Neurologic Institute, site investigator.

Horacio Capote, Dent Neurologic Institute, site investigator.

Michelle Rainka, Dent Neurologic Institute, site investigator.

Barry A. Hendin, Dent Neurologic Institute, site investigator.

Douglas W. Scharre, Ohio State University, site investigator.

Maria Kataki, Ohio State University, site investigator.

Earl A. Zimmerman, Albany Medical College, site investigator.

Dzintra Celmins, Albany Medical College, site investigator.

Alice D. Brown, Albany Medical College, site investigator.

Sam Gandy, Thomas Jefferson University, site investigator.

Marjorie E. Marenberg, Thomas Jefferson University, site investigator.

Barry W. Rovner, Thomas Jefferson University, site investigator.

Godfrey Pearlson, Hartford Hosp, Olin Neuropsychiatry Research Center) Karen Blank, MD (Hartford Hosp, Olin Neuropsychiatry Research Center.

Karen Anderson, Hartford Hosp, Olin Neuropsychiatry Research Center.

Andrew J. Saykin, Dartmouth-Hitchcock Medical Center, site investigator.

Robert B. Santulli, Dartmouth-Hitchcock Medical Center, site investigator.

Jessica Englert, Dartmouth-Hitchcock Medical Center, site investigator.

Jeff D. Williamson, Wake Forest University Health Sciences, site investigator.

Kaycee M. Sink, Wake Forest University Health Sciences, site investigator.

Franklin Watkins, Wake Forest University Health Sciences, site investigator.

Brian R. Ott, Rhode Island Hospital, site investigator.

Chuang-Kuo Wu, Rhode Island Hospital, site investigator.

Ronald Cohen, Rhode Island Hospital, site investigator.

Stephen Salloway, Butler Hospital, site investigator.

Paul Malloy, Butler Hospital, site investigator.

Stephen Correia, Butler Hospital, site investigator.

Howard J. Rosen, UC San Francisco, site investigator.

Bruce L. Miller, UC San Francisco, site investigator.

Jacobo Mintzer, Medical University South Carolina, site investigator.

AUTHOR CONTRIBUTIONS

Study concept and design: Dr. Lo and Dr. Jagust. Data interpretation: Dr. Lo and Dr. Jagust. Drafting of the manuscript: Dr. Lo. Statistical analysis: Dr. Lo. Critical revision of the manuscript: Dr. Jagust.

STUDY FUNDING

Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (NIH grant U01 AG024904). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through contributions from the following: Abbott, AstraZeneca AB, Bayer Schering Pharma AG, Bristol-Myers Squibb, Eisai Global Clinical Development, Elan Corporation, Genentech, GE Healthcare, GlaxoSmithKline, Innogenetics, Johnson and Johnson, Eli Lilly and Co., Medpace, Inc., Merck and Co., Inc., Novartis AG, Pfizer Inc, F. Hoffman-La Roche, Schering-Plough, Synarc, Inc., as well as nonprofit partners the Alzheimer's Association and Alzheimer's Drug Discovery Foundation, with participation from the US Food and Drug Administration. Private sector contributions to ADNI are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Disease Cooperative Study at the University of California, San Diego. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of California, Los Angeles. This research was also supported by NIH grants P30 AG010129, K01 AG030514, and the Dana Foundation.

DISCLOSURE

Dr. Lo reports no disclosures. Dr. Jagust has served on a scientific advisory board for Genentech, Inc.; has served as a consultant for Bayer Healthcare, GE Healthcare, Synarc, Janssen Alzheimer Immunotherapy, Genentech, Inc., TauRx, and Merck & Co; and receives research support from the NIH (AG027859 [PI], AG027984 [PI], and AG 024904 [Co-I]) and from the Alzheimer's Association. Go to Neurology.org for full disclosures.

REFERENCES

1. Hardy SE, Allore H, Studenski SA. Missing data: a special challenge in aging research. J Am Geriatr Soc 2009;57:722–729. [PMC free article] [PubMed]
2. Atkinson HH, Rosano C, Simonsick EM, et al. Cognitive function, gait speed decline, and comorbidities: the health, aging and body composition study. J Gerontol A Biol Sci Med Sci 2007;62: 844–850. [PubMed]
3. Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol 1995;142:1255–1264. [PubMed]
4. Little RJ. Modeling the drop-out mechanism in repeated-measures studies. J Am Stat Assoc 1995;90:1112–1121.
5. Twisk J, de Vente W. Attrition in longitudinal studies: how to deal with missing data. J Clin Epidemiol 2002;55:329–337. [PubMed]
6. Little RJA, Rubin DB. Statistical Analysis With Missing Data, 2nd ed. Hoboken, NJ: Wiley; 2002.
7. Petersen RC, Aisen PS, Beckett LA, et al. Alzheimer's Disease Neuroimaging Initiative (ADNI): clinical characterization. Neurology 2010;74:201–209. [PMC free article] [PubMed]
8. Shaw LM. PENN biomarker core of the Alzheimer's Disease Neuroimaging Initiative. Neurosignals 2008;16:19–23. [PMC free article] [PubMed]
9. Landau SM, Harvey D, Madison CM, et al. Associations between cognitive, functional, and FDG-PET measures of decline in AD and MCI. Neurobiol Aging 2011;32:1207–1218. [PMC free article] [PubMed]
10. Jack CR, Jr, Bernstein MA, Fox NC, et al. The Alzheimer's Disease Neuroimaging Initiative (ADNI): MRI methods. J Magn Reson Imaging 2008;27:685–691. [PMC free article] [PubMed]
11. Chandola T, Jenkinson C. The new UK National Statistics Socio-Economic Classification (NS-SEC): investigating social class differences in self-reported health status. J Public Health Med 2000;22:182–190. [PubMed]
12. Grober E, Sliwinski M. Development and validation of a model for estimating premorbid verbal intelligence in the elderly. J Clin Exp Neuropsychol 1991;13:933–949. [PubMed]
13. D'Agostino RB, Sr, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743–753. [PubMed]
14. Pfeffer RI, Kurosaki TT, Harrah CH, Jr, Chance JM, Filos S. Measurement of functional activities in older adults in the community. J Gerontol 1982;37:323–329. [PubMed]
15. Yesavage JA. Geriatric Depression Scale. Psychopharmacol Bull 1988;24:709–711. [PubMed]
16. Kaufer DI, Cummings JL, Ketchel P, et al. Validation of the NPI-Q, a brief clinical form of the Neuropsychiatric Inventory. J Neuropsychiatry Clin Neurosci 2000;12:233–239. [PubMed]
17. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology 1993;43:2412–2414. [PubMed]
18. Kim SY, Cox C, Caine ED. Impaired decision-making ability in subjects with Alzheimer's disease and willingness to participate in research. Am J Psychiatry 2002;159:797–802. [PubMed]
19. Kim SY, Kim HM, McCallum C, Tariot PN. What do people at risk for Alzheimer disease think about surrogate consent for research? Neurology 2005;65:1395–1401. [PubMed]
20. Beck C, Shue V. Surrogate decision-making and related issues. Alzheimer Dis Assoc Disord 2003;17(suppl 1):S12–S16. [PubMed]
21. Rubin DB, Schenker N. Multiple imputation in health-care databases: an overview and some applications. Stat Med 1991;10:585–598. [PubMed]

Articles from Neurology are provided here courtesy of American Academy of Neurology