|Home | About | Journals | Submit | Contact Us | Français|
To evaluate the utility of Mini-Mental State Examination (MMSE) scores in detecting cognitive dysfunction in a sample of highly educated individuals.
Archival data were reviewed on 4248 participants enrolled in the Mayo Clinic Alzheimer's Disease Research Center (ADRC) and Alzheimer's Disease Patient Registry (ADPR).
1141 primarily Caucasian (93%) individuals with 16 or more years of self-reported education were identified. These included 307 (164 males and 143 females) dementia cases (any type), 176 patients with Mild Cognitive Impairment (106 males and 70 females), and 658 nondemented controls (242 males and 416 females).
Mayo Clinic ADRC and ADPR cohort.
Diagnostic accuracy estimates (sensitivity, specificity, positive and negative predictive power) of MMSE cut-scores in detecting cognitive dysfunction.
In this sample of highly educated, largely Caucasian older adults, the standard MMSE cut-score of 24 (23 or below) yielded a sensitivity of .66, specificity of .99 and an overall correct classification rate of 89% in detecting dementia. A cut score to 27 (26 or below) resulted in an optimal balance of sensitivity and specificity (.89 and .91, respectively) with an overall correct classification rate of 90%. In a cognitively impaired group (dementia and MCI), a cut-score of 27 (sensitivity = .69, specificity = .91) or 28 (sensitivity and specificity = .78) might be more appropriate.
Elderly patients with college education who present with complaints of cognitive decline (self- or other-report) and score below 27 on the MMSE are at greater risk of being diagnosed with dementia and should be referred for a comprehensive dementia evaluation, including formal neuropsychological testing.
The Mini-Mental State Examination (MMSE)1 is the most commonly administered psychometric screening assessment of cognitive functioning. The MMSE is used to screen patients for cognitive impairment, track changes in cognitive functioning over time, and oftentimes to assess the effects of therapeutic agents on cognitive function2. Since its development, there has been a wealth of literature published on the MMSE demonstrating it to be a relatively sensitive marker of overt dementia3-5. Its utility decreases, however, when patients with mild cognitive decline and psychiatric conditions are assessed.6-8
Performance on the MMSE is moderated by demographic variables, with scores decreasing with advanced age and lower levels of education9. Although normative data stratified by age and education have been published10-12, those studies have focused almost exclusively on the impact of lower levels of education, whereas there remains relatively little information available regarding appropriate cut-scores or interpretive strategies for highly educated individuals. This gap is particularly problematic given implications in the literature regarding cognitive reserve13. This literature demonstrates that, once diagnosed, patients with probable Alzheimer's disease who have higher levels of education tend to demonstrate a steeper slope of decline14, 15 and earlier mortality rates15. Identifying cognitive dysfunction in these individuals as early as possible is desirable so that appropriate treatment strategies can be implemented earlier in the course of the disease. To date, however, the authors are unaware of any published investigations that have specifically examined the utility of the MMSE in detecting cognitive dysfunction in highly educated individuals. The current investigation explored this question in individuals with at least 16 years of education. It was hypothesized that in highly educated patients, the frequently implemented MMSE cut-score of 249 would not yield an adequate balance between sensitivity and specificity and that a higher cut-score would need to be utilized to achieve optimal estimates of diagnostic accuracy.
Archival data were reviewed from 4248 consecutive participants recruited into the Mayo Clinic Alzheimer's Disease Research Center (ADRC) and Alzheimer's Disease Patient Registry (ADPR) database. The Rochester Mayo ADPR is responsible for recruiting dementia patients and non-demented control subjects for studies on the progression of Alzheimer's disease through the Department of Community and Internal Medicine and does not operate in Jacksonville. The Rochester and Jacksonville ADRC sites acquire dementia patients from Behavioral Neurology. The Jacksonville ADRC site also recruits community controls via churches and community agencies. The same inclusion/exclusion criteria are applied for normal controls across both recruitment sites and has been published extensively through analyses of the MOANS16-19 and MOAANS20-22 data. Patients with memory concerns raised by either the patient themselves, a family member, or a physician undergo a comprehensive neurological evaluation and neuropsychological testing to confirm or rule out dementia and Alzheimer disease.
A total of 1141 individuals with 16 or more self-reported years of education were identified. The sample included 1064 (93%) individuals who self-identified as Caucasian and 77 (7%) who self-identified as African-American. Of the 1141 participants, 658 individuals (242 males and 416 females) had no dementia and were considered cognitively normal (see Ivnik et al.19 for full criteria used to define normal cognition). The remaining 307 (164 males and 143 females) carried diagnoses of dementia established via consensus among ADRC investigators and based on published diagnostic criteria. Diagnoses included 202 (66%) patients with probable Alzheimer's disease, 48 (16%) with dementia with Lewy bodies, 18 (6%) with frontotemporal dementia, 13 (4%) with vascular dementia, and 25 (8%) with other dementia etiologies. A sample of 176 patients (106 males and 70 females) diagnosed with Mild Cognitive Impairment (MCI) was also included for comparison purposes.
The total sample included 512 (45%) males and 629 (55%) females, with a mean age of 75.9 (SD=7.2) years and a mean self-reported education of 17.1 (SD=1.5) years. There were no significant between-group differences (dementia vs. no dementia) in terms of age, gender, or education.
While the MMSE was available in diagnostic meetings, the diagnosis of dementia (and particular subtype) was arrived at via consensus-based judgment taking into account information from the neurological examination, clinical interview, lab results, imaging, informant ratings of activities of daily living (ADLs), as well as neuropsychological test data. Therefore, the MMSE had minimal impact on diagnostic decisions in the dementia cohort and was not considered at all as part of the determination of control status.
Estimates of sensitivity and specificity were calculated for MMSE cut-scores from 16 (i.e., 15 and below) to 30 (i.e., 29 and below). Results comparing non-demented controls to those diagnosed with some form of dementia are presented in Table 1 and illustrated via receiver operating characteristic (ROC) plot in Figure 1. The traditional cut-score of 24 (23 or below) yielded a moderate estimate of sensitivity (.66) with very high specificity (.99) and an overall correct classification rate of 88.9%. The modest test sensitivity reflects the failure of the traditional cut score to identify a sizeable number of dementia patients in this highly educated sample. Specifically, 104 (34%) dementia cases in this sample were misclassified as normal.
An optimal balance between sensitivity (.89) and specificity (.91) was obtained with a cut-score of 27 (26 or below). This yielded only slight improvement in the overall correct classification rate (90.1%) but identified 70 of the 104 dementia patients who were missed using the traditional cutoff. The cut score of 27 yields a likelihood ratio of 9.6, indicating that college graduates with an MMSE score of 26 and with complaints of cognitive decline (self- or other-report) are nearly 10 times more likely to have dementia that those who obtain a score of 27 or better.
As expected, the improved sensitivity obtained when the cut score is raised to 27 is achieved at the sacrifice of specificity. As a result 61 (9%) non-demented patients fall below the higher cutoff, as compared to only 3 (<1%) false positive identifications with the traditional cut-score of 24.
Next, because clinicians regularly evaluate patients with cognitive dysfunction with and without dementia, the above-mentioned analyses were calculated on a cognitively impaired group (MCI and dementia) versus non-demented controls to determine if an appropriate cut-score could be obtained. Estimates of sensitivity and specificity are presented in Table 2. As can be seen from Table 2, the traditional cut-score of 24 yields a very poor sensitivity (.45) but perfect specificity (1.0). Raising the cut-score to 27 yields an increased sensitivity (.69) with a concomitant decline, though still impressive, specificity (.91).
Although sensitivity and specificity measures are important to establish the diagnostic validity of test measures such as the MMSE, the diagnostic utility of a particular score earned by a particular patient is represented by the test's predictive values. Positive predictive values (PPV) represent the probability that a patient with a score below cutoff actually has the condition of interest. Conversely, negative predictive values (NPV) represent the probability that a patient with a score above cutoff does not have the condition of interest. Unlike sensitivity and specificity, PPV and NPV are influenced by the base rate of the condition of interest in the target population. In the current study, where the base rate of dementia (dementia only group) was 32%, the PPV and NPV for the traditional cutoff of 24 were 96.9% and 86.2%, respectively. Using a cutoff of 27 yielded a lower PPV (81.7%) but a higher NPV (94.5%). When looking at the cognitively impaired group (MCI + dementia), the standard cut-score of 24 yields a very low SN (.45), but perfect SP (1.0). The optimal balances SN and SP were found at cut-scores of 27 (PPV=.78, NPV = .86) or 28 (PPV=.63, NPV=.88). Table 3 presents predictive value calculations from the both groups for clinicians who wish to apply these data in settings where base rates of cognitive impairment and/or dementia differ from that of the current study.
The current findings suggest that the traditional MMSE cut-score of 24 does not yield optimal classification accuracy in highly educated Caucasian dementia patients. Instead, a more stringent cut-score of 27 yields greater clinical utility with regard to identifying dementia in well-educated individuals. Although there is an expected concomitant increase in false-positive identifications using the higher cut score, a sacrifice in specificity in exchange for a significant gain in sensitivity is preferred when the goal of the mental status screen is early detection of possible dementia.
The current analyses also demonstrate that, when MCI is entered into the equation, obtaining an optimal balance between SN and SP is very difficult indeed. Table 2 demonstrates that optimal balances between SN and SP are found at cut-scores of either 27 (SN=.69, SP=.91) or 28 (SN and SP = .78). One might note that the NPV and PPV for the cognitively impaired group using the traditional cut-score of 24 is quite impressive even at low base rates; however, this is a function of a perfect SP and the low base rates. What this translates to for practicing clinicians is a very high false negative rate (often 50% or more) meaning that, because of the small number of true cases in low base rate settings, a large portion of those individuals actually suffering from cognitive dysfunction will not be detected and referred on for a comprehensive evaluation and/or treatment. Table 3 allows the individual clinician to make the determination as to what cut-score(s) s/he wishes to implement given the nature of the clinic population (e.g., demographics, appropriate base rate), additional information obtained in the medical examination (i.e., screening for cognitive impairment versus dementia if information regarding functional change is obtained), as well as his/her preferences for potential diagnostic error (i.e., false negative and false positive rates).
The vast majority of the published literature examining the relationship between cognitive test performance and education focuses on lower educated populations without consideration to individuals who have obtained high levels of education. In fact, research suggests that lower cut-scores on the MMSE are appropriate when evaluating populations obtaining lower levels of education11 and correction formulas have been published5. Educational attainment is often considered one manifestation of cognitive reserve, with higher education levels associated with greater reserve and greater ability to withstand neuropathological burden before exhibiting detectable signs of disease (see Stern13 for review). Individuals with greater cognitive reserve are believed to maintain higher levels of cognitive functioning in the early stages of degenerative dementia. By the time cognitive symptoms are first identified, these patients are believed to have significantly greater disease burden and faster subsequent decline. Identifying such individuals at an earlier stage of disease development and progression is desirable for both treatment and research purposes.
There was not enough data in the current sample to test the comparative accuracy of individual cut-scores among highly educated individuals across ethnic groups. Therefore, the current findings with Caucasian individuals must be tested within ethnic minority populations before generalizations can be made. Additionally, the sample is an English-speaking sample and caution must be used when attempting to generalize to English as a second language or non-English speaking individuals. It should also be noted that the MMSE was administered as part of the clinical examination and was not used as part of the inclusion/exclusion criteria for the study database. Therefore, the MMSE was not used as a screening measure of cognitive functioning in this sample and might perform differently when used in this context (e.g., epidemiological studies).
The current findings are not intended to encourage the diagnosis of cognitive impairment or dementia based on total MMSE scores alone. Instead these results provide practitioners with revised criteria for appropriate management of highly educated Caucasian elders. Specifically, older patients who present with memory complaints (self- or other-report) that have attained a college degree or higher level of education and who score below 27 on the MMSE are at increased risk of cognitive dysfunction and dementia and should be referred for a comprehensive evaluation, including formal neuropsychological studies. When early identification is the primary goal of screening, the cost associated with performing further evaluation on individuals subsequently found to have no dementia is outweighed by the benefit of identifying a considerably larger number of individuals who are in the earliest stages of dementia, where early intervention and/or participation in clinical trials may provide maximum benefit.
This study was supported by grants P50 AG16574 and U01 AG06786 from the National Institute on Aging, and by the Robert and Clarice Smith and Abigail Van Buren Alzheimer's Disease Research Program. Dr. O'Bryant has had full access to the data analyzed in this study and takes responsibility for the integrity of the data and accuracy of the analyses.