|Home | About | Journals | Submit | Contact Us | Français|
Due to the high prevalence of mild cognitive impairment (MCI) and dementia in Parkinson disease (PD), routine cognitive screening is important for the optimal management of patients with PD. The Montreal Cognitive Assessment (MoCA) is more sensitive than the commonly used Mini-Mental State Examination (MMSE) in detecting MCI and dementia in patients without PD, but its validity in PD has not been established.
A representative sample of 132 patients with PD at 2 movement disorders centers was administered the MoCA, MMSE, and a neuropsychological battery with operationalized criteria for deficits. MCI and PD dementia (PDD) criteria were applied by an investigator blinded to the MoCA and MMSE results. The discriminant validity of the MoCA and MMSE as screening and diagnostic instruments was ascertained.
Approximately one third of the sample met diagnostic criteria for a cognitive disorder (12.9% PDD and 17.4% MCI). Mean (SD) MoCA and MMSE scores were 25.0 (3.8) and 28.1 (2.0). The overall discriminant validity for detection of any cognitive disorder was similar for the MoCA and the MMSE (receiver operating characteristic area under the curve [95% confidence interval]): MoCA (0.79 [0.72, 0.87]) and MMSE (0.76 [0.67, 0.85]), but as a screening instrument the MoCA (optimal cutoff point = 26/27, 64% correctly diagnosed, lack of ceiling effect) was superior to the MMSE (optimal cutoff point = 29/30, 54% correctly diagnosed, presence of ceiling effect).
The Montreal Cognitive Assessment, but not the Mini-Mental State Examination, has adequate psychometric properties as a screening instrument for the detection of mild cognitive impairment or dementia in Parkinson disease. However, a positive screen using either instrument requires additional assessment due to suboptimal specificity at the recommended screening cutoff point.
The long-term, cumulative prevalence of dementia in Parkinson disease (PDD) is as high as 80%,1 and impairment not meeting criteria for dementia (i.e., mild cognitive impairment [MCI]) has been reported to occur in 20%–30% of patients with PD,2–4 even among those patients newly diagnosed,2,5 and appears to be twice as common in PD patients without dementia as in healthy elders.6 Cognitive impairment in PD patients without dementia has been found to predict future cognitive decline, including development of PDD.3,4,7 Additionally, the presence of cognitive impairment in PD patients without dementia is associated with worse health-related quality of life (QOL)8 and functional impairment.9
For these reasons and to assist with clinical management, recognition of cognitive disorders in PD is important. However, few screening instruments for global cognition are brief, appropriate for use in routine clinical care, and validated in PD. The Mini-Mental State Examination (MMSE) is the most commonly used instrument in PD, despite its lack of validation in this population. Previous research has called into question its accuracy and sensitivity in PD,10–15 yet it is still recommended and used as the primary screening instrument for dementia in PD.16
The Montreal Cognitive Assessment (MoCA)17 was developed as a brief screening instrument for MCI and mild Alzheimer disease (AD) to address limitations of the MMSE. The MoCA is divided into 7 subscores: visuospatial/executive (5 points); naming (3 points); memory (5 points for delayed recall); attention (6 points); language (3 points); abstraction (2 points); and orientation (6 points). One point is added if the subject has ≤12 years of education.
The MoCA has been shown to be more sensitive than the MMSE for the detection of MCI and mild AD in the general population, and a score ≤25 was found to be the optimal cutoff point for a diagnosis of cognitive impairment.17 A study of MoCA performance in patients with PD with normal MMSE scores found 52% of subjects had cognitive impairment using this cutoff point.13 One study that compared MoCA and MMSE performance in PD found the MoCA to be more sensitive than the MMSE in detecting cognitive impairment, but a MMSE score <26 was used to classify patients as having cognitive impairment, and this MMSE cutoff point has not been validated in PD.12 Finally, another study found that the MoCA has good test-retest reliability, interrater reliability, and convergent validity with a neuropsychological battery in a small sample of patients with PD.18
Given the promising preliminary results on the utility of the MoCA in PD as a screening instrument for cognitive impairment compared with the MMSE, the aim of this study was to assess the discriminant validity of both the MoCA and the MMSE to detect MCI and dementia in PD using established diagnostic criteria as well as operationalized criteria for defining cognitive deficits on the basis of a neuropsychological test battery.
A convenience sample of 132 patients (no cognitive disorder = 92; MCI = 23; PDD = 17) with idiopathic PD at 2 movement disorders clinics was assessed between August 2006 and April 2009. The diagnosis of possible or probable PD was confirmed by the patient’s movement disorder neurologist according to established criteria.19 Patients who had undergone deep brain stimulation (DBS) within the previous 6 months were excluded from the study.
The Institutional Review Board at each participating institution approved the study, and written informed consent was obtained from subjects prior to study participation.
Trained research staff administered the MoCA and MMSE in counterbalanced fashion. The neuropsychological battery included measures in the following 4 cognitive domains: memory (Hopkins Verbal Learning Test [HVLT]20), executive abilities (Tower of London-Drexel [TOLDX],21 Stroop Color-Word Test,22 and Semantic Verbal Fluency23), attention (Backward Digit Span), and visuospatial (Cube Copying, which was extracted from the MoCA and rescored for this purpose using a more detailed methodology outlined below). When standardized scores were available, a score ≥1.5 SD below the published normative data mean was considered to represent a “deficit,” which is consistent with previous PD research.3,6 Memory deficit was defined as ≥1.5 SD below the published normative data mean on at least 1 of 2 HVLT measures (immediate free recall or recognition discrimination). Executive deficit was ≥1.5 SD below the normative data mean on 1 of 2 TOLDX measures (total moves or total correct scores), the Stroop Color-Word mismatch, or Verbal Fluency. Attention deficit was a score ≤4 on the Backward Digit Span, as recommended for the elderly.24 Finally, visuospatial deficit was a score of 0–2 on a 5-point scale scoring method for Cube Copying.25 Due to impaired color discrimination, approximately 10% of subjects were unable to perform the Stroop. MoCA and MMSE questionnaires with incomplete questions that constituted ≤3 points (10% of total points) were included with a prorated score.
The median time interval between administration of the MoCA or MMSE and the neuropsychological battery was 5 weeks. Subjects who completed the battery over 6 months after the index tests were excluded from analysis.
A Movement Disorder Society task force recommended diagnostic criteria for probable PDD16,26 that included cognitive deficits in at least 2 of the 4 core cognitive domains (attention, executive functions, visuospatial, and memory), as well as cognitive deficiency severe enough to impair daily life (e.g., inability to manage finances and cope in social situations). Therefore, our dementia criteria were 1) ≥1.5 SD below the normative data mean on tests in at least 2 cognitive domains, 2) self-report of cognitive decline, and 3) impairment of instrumental activities of daily living (IADLs).
A modification of the Peterson criteria27 that allows for impairments in a range of cognitive domains, called the Winblad criteria,28 was used to diagnose MCI. Our MCI criteria were 1) ≥1.5 SD below the normative data mean on tests in at least 1 cognitive domain, 2) self-report of cognitive decline, and 3) preserved IADLs. Subjective reports of cognitive decline and impairment in IADLs were obtained from subjects by the study PI (D.W.) during an unstructured interview (in person or by telephone, asking participants if they had noted any meaningful change over the course of PD in their ability to plan, remember, pay attention, or complete tasks), and when available the input of informed others was solicited. Formal diagnostic criteria for MCI and PDD were applied by the study PI, who was blinded to MoCA and MMSE scores.
Five subjects failed to fit into 1 of the 3 diagnostic categories (PDD, MCI, or normal), as they had deficits in only 1 cognitive domain but reported functional impairment. These 5 subjects were excluded from the validation process.
The 15-item Geriatric Depression Scale (GDS-15) was administered to measure severity of depression symptomatology (scores ranging from 0 to 15, higher scores indicating greater depression severity).29 The Unified Parkinson’s Disease Rating Scale (UPDRS) motor scores and disease severity as measured by Hoehn & Yahr stage (scores ranging from 1 to 5, higher scores indicating greater disease severity)30 were obtained from the subjects or by chart review. Patients were encouraged to take their regularly scheduled PD medications during the study visit so that they would be evaluated in their “on” state.
Between-group comparisons of demographic and clinical characteristics, including MoCA and MMSE scores, between cognitively impaired (i.e., those meeting criteria for either MCI or PDD) and unimpaired samples were performed using either an independent-sample t test with Levene’s test for equality of variances or a Pearson χ2 test for dichotomous variables.
A receiver operating characteristic (ROC) with area under the curve (AUC) (95% CI) was plotted for each of the instruments’ discriminant validity for detecting any cognitive disorder (MCI or PDD) vs absence of a cognitive disorder, as this is often the primary comparison when assessing the validity of cognitive screening instruments.31 Secondary analyses included examining each questionnaire’s discriminant validity for detecting MCI alone (vs no cognitive disorder), PDD alone (vs no cognitive disorder), and deficits on neuropsychological testing (≥2 domains, without the requirement of self-report of cognitive decline) vs no neuropsychological deficits.
The AUC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and percent correctly diagnosed were calculated for each questionnaire. The optimal screening cutoff point was defined as the lowest value that achieved >80% sensitivity and NPV; the optimal diagnostic cutoff point was defined as the highest value that achieved >80% specificity and PPV.
All statistical procedures were performed with SPSS 15.0 for Windows.32
The figure is a flow diagram of study participants,33 and characteristics of the sample population are listed in table 1. Mean (SD) MoCA and MMSE scores were 25.0 (3.8) and 28.1 (2.0). The sample was 75.8% male and 94.7% white, and mean (SD) age = 65.1 (9.7) years, PD duration = 6.3 (5.3) years, and education = 16.4 (3.1) years. The study cohort was reflective of the overall clinic populations in terms of sociodemographic characteristics, clinical status, and global cognition, as compared to a larger sample of 884 subjects from the 2 centers (data not shown). Approximately 30% of the sample met diagnostic criteria for a cognitive disorder (12.9% PDD and 17.4% MCI). Regardless of diagnosis, 37.1% of the study population had deficits in ≥2 domains, 22.7% had deficits in 1 domain, and 40.2% had no domain deficits.
Psychometric properties for the detection of any cognitive disorder are listed in table 2. The ROC AUCs (95% CI) were 0.79 (0.72–0.87) for the MoCA and 0.76 (0.67–0.85) for the MMSE. The optimal screening cutoff points were 26/27 (sensitivity = 0.90, specificity = 0.53) for the MoCA and 29/30 (sensitivity = 0.90, specificity = 0.38) for the MMSE. The optimal diagnostic cutoff points for the MoCA and MMSE were 17/18 and 24/25.
After excluding subjects with an MCI diagnosis, each questionnaire’s discriminant validity for the detection of dementia was examined (table 3). The AUCs (95% CI) were 0.87 (0.79–0.95) for the MoCA and 0.80 (0.67–0.93) for the MMSE. The optimal screening cutoff points were 24/25 (sensitivity = 0.82, specificity = 0.75) for the MoCA and 28/29 (sensitivity = 0.82, specificity = 0.63) for the MMSE. The optimal diagnostic cutoff points for the MoCA and MMSE were 17/18 and 24/25.
After excluding patients with a dementia diagnosis, each questionnaire’s discriminant validity for the detection of MCI was examined. The AUCs (95% CI) were 0.74 (0.64–0.85) for the MoCA and 0.72 (0.61–0.83) for the MMSE. The optimal screening cutoff points were 26/27 (sensitivity = 0.83, specificity = 0.53) for the MoCA and 29/30 (sensitivity = 0.91, specificity = 0.38) for the MMSE. The optimal diagnostic cutoff points for the MoCA and MMSE were 16/17 and 23/24.
A secondary analysis was performed to determine the discriminant validity of the MoCA and MMSE for the detection of deficits on neuropsychological testing in ≥2 domains regardless of MCI or dementia diagnosis (n = 49). The MoCA ROC AUC (95% CI) = 0.87 (0.80–0.93) and MMSE ROC AUC (95% CI) = 0.77 (0.69–0.85). The optimal screening cutoff points were 25/26 (sensitivity = 0.86, specificity = 0.72) for the MoCA and 29/30 (sensitivity = 0.92, specificity = 0.42) for the MMSE. The MoCA and MMSE optimal diagnostic cutoff points were 20/21 and 25/26.
MoCA and MMSE performance for the subjects who had cognitive deficits on neuropsychological testing (≥1 domain) but did not report cognitive decline (n = 39) was compared with the MCI group (n = 23), who also had to report cognitive decline in order to meet MCI diagnostic criteria. Controlling for age, sex, and educational level, there were no significant between-group differences in MoCA or MMSE scores (data not shown). Likewise, MoCA and MMSE performance for MCI subjects with impairments on ≥2 cognitive domains but who did not report functional impairment (n = 13) was compared with PDD patients (n = 17), who also had to report functional impairment in order to meet PDD diagnostic criteria. Again, there were no significant between-group differences in MoCA or MMSE scores (data not shown).
The group with a diagnosis of any cognitive disorder had lower total MoCA scores than the group with no cognitive disorder diagnosis (22.2 vs 26.15, t = 5.5 [df = 56.8], p < 0.001). Examining by MoCA subscore, patients with MCI or PDD had significantly lower visuospatial/executive, attention, language, delayed recall, and orientation subscores (data not shown).
We found that the MoCA has good overall discriminant validity as a global cognition assessment instrument for the detection of MCI or dementia in PD. It performed similarly to the MMSE overall, but was superior as a screening instrument. Given suboptimal specificity and PPV for both instruments at the recommended screening cutoff points, additional evaluation of screen-positive patients is required to determine if patients meet clinical criteria for a cognitive disorder. Lower cutoff points for both instruments are recommended if they are to be used as diagnostic instruments instead of screening instruments.
The MMSE has been recommended as a useful tool to identify cognitively impaired patients with PD,16 but our results suggest several factors that recommend the MoCA for use over the MMSE as a screening instrument. The optimal screening cutoff point for detection of any cognitive disorder for the MoCA had greater specificity (0.53), PPV (0.46), and percent correctly diagnosed (64%) than the optimal MMSE screening cutoff point (specificity = 0.38; PPV = 0.39; percent correctly diagnosed = 54%). In addition, the MoCA produced a larger range of scores (12–30; range = 19 points) compared to the MMSE (22–30; range = 9 points). Finally, the optimal MMSE screening cutoff point of 29/30 means that only patients scoring a perfect 30 are considered to have a negative screen for the detection of a cognitive disorder.
Given that cognitive impairment 1) is common throughout the course of PD; 2) typically progresses to PDD long-term; 3) adversely impacts function, QOL, and caregiver burden; and 4) should inform clinical decision-making, including use of cognitive enhancing agents, it is important that patients with PD at all stages of the disease undergo routine screening of global cognitive abilities in the context of clinical care. Our research is consistent with most research to date that the MMSE does not perform well as a screening instrument for MCI and PDD,34 due in part to lack of sensitivity to milder cognitive deficits (i.e., instrument ceiling effect,35 with 29.5% of patients achieving a perfect score on the MMSE, compared with 6.8% on the MoCA). Both the Mattis Dementia Rating Scale36 and the Cambridge Cognitive Assessment have been validated against a DSM-IV37 diagnosis of dementia,38 but their completion times are too long to be appropriate for use in routine clinical care (approximately 20–45 minutes for these instruments compared with less than 10 minutes for the MoCA). Other global cognitive instruments recently developed for use in PD with preliminary evidence to support their validity include the Parkinson Neuropsychometric Dementia Assessment11 and the PD-Cognitive Rating Scale,39 but neither validation study operationalized the definition of impairment on neuropsychological testing to support a cognitive disorder diagnosis, and the latter instrument takes 17–26 minutes to complete.
An interesting finding of our study regards apparent limitations in the value of self-report of cognitive decline and functional impairment in the context of a clinical research interview. Sixty-two of the 115 subjects without dementia (53.9%) had impaired cognitive testing in ≥1 domain, and 39 of the 62 (62.9%) did not self-report cognitive decline that would have led to a diagnosis of MCI. Similarly, of the 30 patients who had impairments in ≥2 cognitive domains and were eligible for a diagnosis of dementia, 13 of the 30 (43.3%) denied any functional impairment that would have led to a diagnosis of PDD. However, there were no significant between-group differences in MoCA or MMSE scores for patient groups distinguished by these self-reports. Additional study is needed to determine if self-report of cognitive decline or functional impairment is of clinical or prognostic significance in this population.
Regarding study limitations, our results may not be generalizable, as the majority of our patients were male, white, and highly educated, and all were recruited from specialty care centers. Second, while the MoCA includes a minor correction for lower educational levels, the MMSE does not, which affects the diagnostic accuracy of the latter in highly educated individuals.40 Third, the majority of patients had mild to moderate PD (i.e., Hoehn & Yahr stages 1–3). Fourth, we did not have a matched non-PD control group, although the goal of the study was not to compare cognitive functioning in patients with PD and non-PD patients. Furthermore, our neuropsychological battery had more detailed testing of memory and executive functioning compared with attention and visuospatial functioning, and there was no assessment of language abilities. Although there is no consensus regarding the ideal neuropsychological battery to detect cognitive deficits in PD, our battery may have led to an underestimation of PDD and MCI frequencies in our study population. Finally, the number of MCI and PDD cases relative to unimpaired patients was relatively low, and including more patients with a cognitive disorder would lend more certainty to our findings and improve the PPV of both instruments. However, our study sample seemed to be representative of the overall population at these 2 movement disorders centers. While the reported point prevalence of PDD and MCI is typically higher than in our study population, this would not affect our findings regarding the sensitivities and specificities of the 2 screening instruments. Due to the sample size limitations, we were not able to validate the optimal cutoff points in a separate sample, so additional studies of the MoCA and MMSE in PD are needed.
Given the high prevalence of dementia in PD and the high conversion rate of MCI to PDD, early and routine screening for cognitive impairment with a brief, sensitive instrument is warranted. Our study recommends the MoCA over the widely used MMSE and suggests the need for further validation of the MoCA and MMSE in a larger sample of patients with PD, against other screening instruments, and using a more detailed neuropsychological battery.
Statistical analysis was conducted by Dr. Daniel Weintraub and Dr. Sharon X. Xie.
S. Hoops and S. Nazem report no disclosures. Dr. Siderowf serves on the editorial advisory board of Neura; serves on a speakers’ bureau for Teva Pharmaceutical Industries Ltd.; serves as a consultant to Supernus Pharmaceuticals, Inc., Teva Pharmaceutical Industries Ltd., Merck Serono, and Schering-Plough Corp.; and receives research support from the NIH [U10 NS044451 (Principal Investigator), P50 NS053488 (Project Leader), R43 NS0636071(Co-Investigator)]. Dr. Duda serves/has served on a scientific advisory boards for Boehringer Ingelheim, the Lewy Body Dementia Association, and the Lewy Body Society; has received honoraria for lectures or educational activities not funded by industry; and receives research support from the Department of Veterans Affairs [Biomedical Laboratory Research and Development Service Merit Award (PI), Cooperative Studies Program 468 (Site PI)], the NIH [RO1 NS41265-01 (Coinvestigator), RO1 NS44266 (Coinvestigator)], the Michael J. Fox Foundation, and the Samueli Foundation. Dr. Xie receives research support from the NIH/NINDS [NS053488 (Coinvestigator and Core Director)]. Dr. Stern receives royalties for publishing Deep Brain Stimulation for Parkinson’s Disease (Taylor and Francis Group, 2007); has received honoraria from Teva Pharmaceutical Industries Ltd., Boehringer Ingelheim, and Novartis; serves as a consultant to Teva Pharmaceutical Industries Ltd., Boehringer Ingelheim, Novartis, Vernalis plc, Ipsen, Schering-Plough Corp., and NuPathe Inc.; and receives research support from Ceregene, and a foundation grant from the Institute of Neurodegeneration, New Haven, CT, and the Department of Defense [USAMRAA810002 (Coinvestigator)]. Dr. Weintraub has served on a scientific advisory board for Boehringer Ingelheim; serves on the editorial board of Movement Disorders; has received honoraria from Boehringer Ingelheim, ACADIA Pharmaceuticals Inc., Novartis, Osmotica Pharmaceutical Corp., BrainCells Inc., Merck Serono, Sanofi-Aventis, and Pfizer Inc.; and receives research support from Boehringer Ingelheim and the NIH [K23 MH067894 (Principal Investigator)].
Address correspondence and reprint requests to Dr. Daniel Weintraub, 3615 Chestnut St., Room 330, Philadelphia, PA 19104-2676 firstname.lastname@example.org
Disclosure: Author disclosures are provided at the end of the article.
Received May 14, 2009. Accepted in final form August 20, 2009.