Demographic and FAQ Variables
Comparisons of demographic variables between the MCI and AD groups () indicated that AD subjects were significantly more likely to be non-Hispanic Whites (p<0.001), older (p=0.015), and have lower MMSE scores (p<0.001) than MCI subjects. Informants for AD subjects provided information on fewer FAQ items (p<0.001) and were less likely to provide data for all FAQ items (p<0.001) than informants for MCI subjects.
As expected, AD subjects had significantly higher total FAQ scores [; Z(1185)=−22.99, p<0.001] and mean FAQ item scores [; Z(1801)=−26.27, p<0.001] than MCI subjects. Significantly higher scores in the AD group were also seen for each individual FAQ item (all p’s<0.001; Bonferroni-corrected critical p=0.005), both in the subgroup with complete FAQ data and in the overall cohort (). Higher scores on each of these indices indicate greater functional impairment.
Figure 1 FAQ indices in the MCI and AD groups. A) Total FAQ scores for the subset of subjects with valid data for all items. B) Mean FAQ item score for all subjects. Mean individual FAQ item scores for C) participants with complete FAQ data and D) all participants. (more ...)
Subjects with complete versus incomplete FAQ responses demonstrated significant differences in demographic and FAQ variables. MCI subjects with complete FAQ data were more likely to be male [56.6% vs. 43.8%, χ2(1,1108)=15.09, p<0.001], have higher MMSE scores [27.91 vs. 27.28; t(1106)=5.36, p<0.001], and have lower mean FAQ item scores [0.24 vs. 0.38; t(1106)=4.56, p<0.001] than those with incomplete data. There was no difference in the distribution of MCI subtypes between subjects with complete versus incomplete responses [χ2(3,1108)=5.06, p=0.17]. AD subjects with complete FAQ data were more likely to be male [63.3% vs. 38.0%; χ2(1,693)=43.48, p<0.001] and have higher mean FAQ item scores [1.32 vs. 1.00, Z(691)=−5.78, p<0.001], but marginally less likely to meet NINDS-ADRDA criteria for probable AD [82.8% vs. 88.0%; χ2(1,693)=3.61, p=0.057] than those with incomplete data.
ROC Analyses of Global FAQ Indices
Demographic variables, FAQ scores, proportion of subjects with complete FAQ data, and distributions of MCI subtype and AD diagnostic categories did not differ between the development and test sets (data not shown, all p’s >0.05). Separate ROC curves were generated from the development set to determine the optimal cut-points using total FAQ scores or mean FAQ item scores for distinguishing between AD and MCI ().
Receiver operating characteristic (ROC) curves for A) total FAQ scores and B) mean FAQ item scores for distinguishing AD from MCI. AUC: area under the ROC curve.
ROC analysis of total FAQ scores produced an area under the curve (AUC) of 0.903 [; 95% confidence interval (CI): 0.876–0.930, p<0.001] and d’ of 1.80. The optimal cut-point was between 5 and 6, which yielded 82.9% sensitivity, 83.9% specificity, and 83.6% classification accuracy when applied to the development set, and 80.3% sensitivity, 87.0% specificity, and 84.7% classification accuracy when applied to the test set. Slightly poorer discrimination was seen between probable AD and multiple-domain amnestic MCI when this cut-off was used with the test set: 80.3% sensitivity, 81.3% specificity, and 80.7% classification accuracy. These findings indicate that total FAQ scores < 6 were most consistent with a clinical diagnosis of MCI, and total FAQ scores ≥ 6 were most consistent with a clinical diagnosis of AD.
Using the test set, logistic regression analysis of the total FAQ score cut-point versus clinical diagnosis was conducted. After adjusting for age, race, and MMSE score, this analysis yielded a Nagelkerke R2 value of 0.579. A total FAQ score ≥ 6 was independently associated with a diagnosis of AD vs. MCI [β=2.97, S.E.=0.24, Wald χ2=150.06, odds ratio (OR)=19.51, CI=12.13–31.38, p<0.001] and with a diagnosis of probable AD vs. multiple-domain amnestic MCI (β=2.65, S.E.=0.34, Wald χ2=60.12, OR=14.15, CI=7.24–27.64, p<0.001).
ROC analysis of mean FAQ item scores produced an AUC of 0.864 (; CI: 0.840–0.889, p<0.001) and d’ of 1.49. The optimal cut-point was between 0.436 and 0.437, which yielded 82.4% sensitivity, 76.5% specificity, and 78.8% classification accuracy for distinguishing AD from MCI in the development set and 81.8% sensitivity, 77.4% specificity, and 79.1% classification accuracy in the test set. Similar results were obtained when this cut-point was used to distinguish between probable AD and multiple-domain amnestic MCI in the test set: 82.4% sensitivity, 70.9% specificity, and 78.3% classification accuracy. The use of mean FAQ item score allowed for the inclusion of a greater number of subjects than the use of the total FAQ score but resulted in poorer discrimination between groups.
ROC Analyses of Individual FAQ Items
ROC data for the diagnostic value of individual FAQ items were separately derived from the development set for subjects with valid data for all items and all subjects with valid data for each item. The optimal cut-off point for each item was a score ≥ 1 (i.e. presence of any impairment). For subjects with complete FAQ data, the items that yielded the best discriminative power between AD and MCI included: paying bills (86% sensitivity, 77.5% specificity, and 80.3% classification accuracy), assembling tax records (88.6% sensitivity, 71.9% specificity, and 77.4% classification accuracy), and traveling outside the neighborhood (80.3% sensitivity, 77.5% specificity, and 78.4% classification accuracy). Discriminative indices were consistently higher for the subset of subjects with valid data for all FAQ items than for the overall cohort.
In order to determine which individual FAQ items were independently associated with a clinical diagnosis of AD, stepwise logistic regression analysis was performed using the development set and adjusted for age, race, and MMSE score. This analysis yielded a Nagelkerke R2 value of 0.618 and indicated that subjects with any impairment on paying bills (β=1.28, S.E.=0.33, Wald χ2=14.76, OR=3.60, CI=1.87–6.91, p<0.001); shopping alone (β=0.83, S.E.=0.33, Wald χ2=6.52, OR=2.29, CI=1.21–4.34, p=0.011); tracking current events (β=0.85, S.E.=0.30, Wald χ2=8.15, OR=2.34, CI=1.31–4.20, p=0.004); traveling outside the neighborhood (β=0.87, S.E.=0.30, Wald χ2=8.59, OR=2.39, CI=1.33–4.28, p=0.003); or playing a game of skill (β=0.70, S.E.=0.31, Waldχ2=4.97, OR=2.01, CI=1.09–3.72, p=0.026) were more likely to be diagnosed with AD. When cut-offs on these individual items were applied to the test set, their discriminative power for identifying AD remained poorer than that obtained using global FAQ indices: 60.6–84.6% sensitivity, 78.9–88.5% specificity, and 77.9–80.9% classification accuracy for subjects with valid data for all items and 56.1–81.1% sensitivity, 75.0–84.3% specificity, and 75.3–77.3% classification accuracy for all subjects.
Use of FAQ in Diagnosis Across ADCs
Of the 29 ADCs actively collecting UDS data, 28 responded to the survey regarding the use of FAQ for diagnosis. These centers contributed data for 93.0% of the subjects included in our analyses. Nineteen centers (comprising 77.4% of subjects) do not use the FAQ for diagnosis and 9 centers (comprising 22.6% of subjects) use FAQ data only as supporting information. None of the ADCs implement a specific cut-point on FAQ scores for distinguishing between MCI and AD. Sensitivity, specificity, and classification accuracy of optimal cut-points for total FAQ and mean FAQ item scores did not differ between centers that considered FAQ scores during diagnosis and those that did not (all p’s>0.1; ).
Mean sensitivity, specificity, and classification accuracy of optimal total FAQ and mean FAQ item score cut-points dichotomized by diagnostic use of FAQ scores in individual ADCs.