|Home | About | Journals | Submit | Contact Us | Français|
The Montreal Cognitive Assessment (MoCA) and Addenbrooke’s cognitive examination-revised (ACE-R) are proposed as short cognitive tests for use after stroke but there are few published validations against neuropsychological battery. We studied MoCA, ACE-R and mini-mental-state-examination(MMSE) in patients with cerebrovascular disease and mild cognitive impairment (MCI)
100 consecutive patients had the MMSE, MoCA, ACE-R and NINDS-CSN VCI Harmonisation Standards Neuropsychological Battery ≥1 year after TIA or stroke in a population-based study. MCI was diagnosed using modified Petersen criteria in which subjective cognitive complaint is not required (equivalent to cognitive- impairment-no-dementia (CIND)) and sub-typed by number and type of cognitive domains affected.
Among 91 non-demented subjects completing neuropsychological testing (mean/sd age 73.4/11.6 years, 44% female, 56% stroke), 39 (42%) had MCI (amnestic multiple domain=10, non-amnestic multiple domain=9, non-amnestic single domain=19, amnestic single domain=1). Sensitivity and specificity for MCI were optimal with MoCA<25 (sensitivity=77%, specificity=83%) and ACE-R<94 (sensitivity=83%, specificity=73%). Both tests detected amnestic MCI better than non-amnestic single domain impairment. MMSE only achieved sensitivity>70% at a cut-off of<29, mainly due to relative insensitivity to single domain impairment.
The MoCA and ACE-R had good sensitivity and specificity for MCI defined using the NINDS-CSN VCI Battery ≥1 year after TIA and stroke whereas the MMSE showed a ceiling effect. However, optimal cut-offs will depend on use for screening (high sensitivity) or diagnosis (high specificity). Lack of timed measures of processing speed may explain the relative insensitivity of the MoCA and ACE-R to single non-memory domain impairment.
Stroke doubles the risk of dementia in epidemiological studies and rates of dementia in the first year after stroke are high particularly after recurrent stroke.1,2 Mild cognitive impairment (MCI) is also common after stroke and is associated with increased risk of dementia.1,3,4 However, lengthy neuropsychological batteries are often not feasible in routine practice or large-scale studies and there is thus a need for short tests of cognition that are sensitive to MCI and to the frontal/executive deficits that are prominent in vascular cognitive impairment.
Two recently developed short tests of cognition, the Montreal Cognitive Assessment (MoCA-30 point test)5 and the Addenbrooke’s cognitive examination revised6 (ACE-R-100 point test in which the mini-mental state examination (MMSE)7 is embedded) include tests of frontal lobe function (executive and attentional tasks) and were designed to be sensitive to MCI in a non-vascular setting. Preliminary studies in stable cerebrovascular disease suggest that the MoCA is more sensitive to MCI than the MMSE8 but the MoCA cut-off of <26/30 for MCI derived from a memory clinic population,5 may not be appropriate in a population with cerebrovascular disease. The ACE-R has also been proposed as a useful cognitive outcome measure in stroke but there are no published data.
There is no consensus on the definition or the criteria for MCI since there are uncertainties in delineating the boundaries between normal cognitive function and MCI and between MCI and dementia, with at least 18 different terms and definitions in current use.9–12 We used the modified Petersen criteria9,11 (in which subjective memory complaint is not required as in Cognitive Impairment No Dementia (CIND)1,3,4) which allow sub-typing of MCI by number and types of cognitive domains affected.
Our hypothesis was that the MoCA and the ACE-R would be more sensitive for MCI than the MMSE but that the MoCA and ACE-R would perform similarly in view of the cognitive domains covered and level of detail. We therefore aimed to determine the sensitivities and specificities of the MoCA, ACE-R and MMSE at ≥1 year after TIA or stroke for detection of MCI identified with the neuropsychological battery recommended in the National Institute of Neurological Disorders and Stroke-Canadian Stroke Network (NINDS-CSN) Vascular Cognitive Impairment (VCI) Harmonization Standards working group.13
Patients were participants in the Oxford Vascular Study (OXVASC 2002-), a prospective population-based cohort study of all acute vascular events occurring within a defined population of about 91 000.14,15 The study was approved by the local ethics committee and informed consent was obtained. Between August 2009 and November 2010, consecutive patients attending the OXVASC clinic were invited at their routine 1 or 5 year follow-up to undergo further cognitive testing with the ACE-R6 and the NINDS-CSN Harmonisation Standards Neuropsychological Battery13 in addition to the MMSE,7 MoCA,5 Barthel16 and modified Rankin score (mRS)17 which were done routinely at the follow-up appointment. Further cognitive testing was performed by investigators (SP, JM, LB) who did not undertake the routine follow-up and were blinded to the MMSE and MoCA results. Nursing home residents and patients who had problems that interfered with testing such as poor vision, severe hearing impairment, inability to use the right arm, dysphasia, poor English or acute illness, were excluded.
The neuropsychological battery tests frontal/executive, attentional, language, visuospatial and memory domains and took around 50-60 minutes to administer. Specific tests were:
Depression was assessed using the short-form Geriatric Depression Score (GDS).28
For MCI diagnosis,9,11 the subject had to be impaired (≥1.5 sd) on at least one cognitive domain compared to age and education matched published norms,19,20,23,25 had no impairment of basic functional activities of daily living as measured by the Barthel index, and did not fulfil the DSM-IV dementia diagnostic criteria.29 Four sub-types of MCI were distinguished:11
Rates of MCI were also determined using the original Petersen criteria in which presence of a subjective memory complaint is required, using the question: “Do you think you have more problems with your memory than most?” The original Petersen criteria are widely used in memory clinics and Alzheimer’s disease in which memory impairment is prominent but are likely to underestimate cognitive impairment where non-memory domains are preferentially affected.9,10
For MoCA, ACE-R and MMSE, mean raw and % (as % of the maximum possible score) subtest scores were calculated and z scores were derived by converting the mean raw score and standard deviation to the standard normal distribution with mean 0 and standard deviation 1 (lower z scores indicating greater discrimination between subjects).8
Years of education, Barthel and mRS scores were dichotomised as follows: < versus ≥12 years of education, Barthel<20 versus ≥20 and mRS <3 (non dependent) versus ≥3 (dependent).
Level of agreement between MCI and MoCA, ACE-R and MMSE and scores were assessed using the area under the ROC curves (c statistic). Sensitivities, specificities, positive predictive values (ppv) and negative predictive values (npv) for various MoCA and ACE-R cut-offs for identifying MCI were determined. Significance levels for odds ratios (OR) were calculated using Chi square test.
Among 100 consecutive patients (mean age 73.4/11.6 years, 44% female) 9 subjects had incomplete neuropsychology data (5=dementia, 2=poor vision, 1=severe residual hemiparesis, 1=declined part of testing) and a further 3 did not have the ACE-R. Of the 91 subjects with complete neuropsychology data, there were 63% (n=57) with<12 years education, 56% (n=51) stroke (40 first and 11 recurrent), 52% (n=47) at 1 year follow-up, 89% (n=81) with mRS <3 with mean MMSE 27/3.3 and mean MoCA 22.7/4.9.
MMSE and ACE-R scores were skewed towards higher values [median and interquartile range 28 (26-29) and 93 (86-96)] whereas MoCA scores were normally distributed [23 (20-26)]. The MoCA and ACE-R were strongly correlated (Spearman r2=0.87, p<0.01; figure 1). Individual subtests of the MoCA and ACE-R are described in table 1 together with the mean and Z scores. All MoCA subtests and most ACE-R subtests discriminated well between subjects.
In the neuropsychological battery, more patients performed below the cut-off (≥1.5 sd below published norms) on visuospatial and executive/attentional tasks than on memory, language and naming tasks (table 2). TIA and stroke patients were similar in age, education level and sex distribution but compared to those with TIA, stroke patients had lower mean MMSE, MoCA, ACE-R and memory (HVLT) scores with a trend to worse performance on the SDMT, trails B and verbal fluency (table 2).
39 /91 (43%) non-demented subjects who completed neuropsychological testing, had a diagnosis of MCI. Half of the MCI cases (20/39) had single domain impairment, the vast majority of which was in a non memory domain: non-amnestic single-domain=19, amnestic single-domain=1, non-amnestic multiple-domain=9, amnestic multiple-domain=10. C statistics for MCI were: MoCA=0.85 (95% CI 0.78-0.93), ACE-R=0.90 (0.83-0.96) and MMSE=0.83 (0.75-0.92).
Optimal sensitivities and specificities for MCI were achieved for MoCA cut-offs around 25-26 (MoCA<25, sensitivity=77%, specificity=83%; MoCA<26, sensitivity=87%, specificity=63%) and ACE-R cut-offs between 92-94 (ACE<92, sensitivity=72%, specificity=79%; ACE-R<94, sensitivity=83%, specificity=73%) (table 3). Sensitivity of the MMSE for MCI was relatively low, being >70% only at a cut-off of <29. Restricting analyses to multiple-domain MCI, showed similar predictive values for the MoCA, ACE-R and MMSE: MoCA<25, sensitivity= 89%,95%CI 67-99, specificity=69,57-80, ppv=0.44,0.28-0.60, npv=0.96,0.88-1.0; ACE-R<92, sensitivity=88,64-99, specificity=69,56-79, ppv=0.42,0.26-0.59, npv=0.96,0.86-0.99; MMSE<28, sensitivity=79,54-94, specificity=78,66-87, ppv=0.48,0.30-0.67, npv=0.93,0.84-0.98.
The MoCA detected all cases of amnestic MCI but missed 9 and 5 cases of non-amnestic MCI at cut-offs of <25 and <26 respectively (table 4). Most cases of non-amnestic MCI that were missed by the MoCA were of single rather than multiple-domain impairment. Results were qualitatively similar for ACE-R <94 and <92 (table 4). Nine MCI subjects had MMSE≥29 and 14 had MMSE≥28 of whom 4 had multiple-domain impairment (table 4).
13/86 (16%) subjects completing the GDS had abnormal scores indicating possible depression. There was a trend towards increased likelihood of abnormal GDS in subjects with MCI vs those without: 8/34 vs 5/52, OR=2.89, 0.87-9.76; p=0.087. MCI associated with high GDS score was nearly always multiple-domain (7/8) (table 4).
Agreement between objective cognitive impairment and subjective memory complaint was poor (kappa=0.27, 0.08-0.47; p=0.006): only 17 of the 39 patients with objective cognitive impairment had a subjective memory complaint and therefore met criteria for MCI by the original Petersen criteria (table 5). Nine subjects with subjective memory complaint did not have any objective cognitive impairment.
The NINDS-CSN Harmonization Standards Neuropsychological Battery was feasible in the majority of OXVASC community dwelling patients tested at least 1 year after TIA or stroke, although around 10% were unable to complete all tests. Rates of MCI defined using modified Petersen criteria (subjective memory impairment not required) were high with non-amnestic single-domain and multiple-domain impairment most common. Both the MoCA and the ACE-R had good sensitivity and specificity for MCI thus defined.
The pattern of cognitive deficits in our study with rarity of isolated memory impairment and prominence of slowed processing speed and visuoexecutive deficits is characteristic of vascular cognitive impairment3 and suggests that the NINDS-CSN Neuropsychological Battery, although relatively short, covers the relevant cognitive domains effectively. Cognitive profiles were qualitatively similar in TIA and stroke patients although more abnormalities were seen after stroke, particularly in memory. There was a trend towards more abnormal depression scores in the MCI group, particularly in those with multi-domain impairment, consistent with the recognized association between depression and vascular cognitive impairment.30
It is important to note that although neuropsychological testing is considered the “Gold Standard” for identifying cognitive impairment, there is no consensus on how such data are used to diagnose MCI.9–12 The sensitivities and specificities obtained for different MoCA , ACE-R and MMSE cut-offs will therefore be highly dependent on how MCI is defined as well as on other factors such as case-mix. The Petersen criteria class single-domain impairment as MCI whereas other methods require at least 2 domains to be impaired and the threshold for abnormal cognitive domain function ranges from ≥1 to ≥2 sd below normal. Further, the requirement for subjective memory decline affects estimates9,10 as seen also in our study: less than half of those with objective deficits had subjective memory complaint and were thus classed as MCI by the standard Petersen criteria. Finally, in distinguishing MCI from dementia, the Barthel index may underestimate functional impairment owing to ceiling effects and thus some MCI patients in our study may have been classed as having dementia using alternative functional criteria. Distinguishing MCI from dementia is of less relevance where the aim is to detect any cognitive impairment regardless of severity.
Our data confirm the previously observed ceiling effect for the MMSE31 only cut-offs of <29 or greater had sensitivities for MCI of >70%, with MMSE<27 having a sensitivity of only 50%, although MMSE sensitivity was greater for multi-domain impairment. Both the MoCA and ACE-R performed well in detecting MCI including single domain impairment. Optimal MoCA cut-off was lower than seen in our study when measured at a mean of 6 days after stroke32 against neuropsychological testing performed 2-3 weeks later, probably due in part to different definitions of cognitive impairment and/or to effects of delirium and acute illness. Our study used similar criteria for MCI (Petersen criteria) to the original MoCA study on a memory clinic cohort5 and our results are broadly consistent despite the different clinical characteristics of the patients.
Our results suggest that the MoCA and the ACE-R are similarly useful in measuring cognitive outcomes in stable cerebrovascular disease, most items on both tests discriminated well between subjects, although our study was not powered to detect small differences in performance. Our data should inform such calculations in future studies. Although both the MoCA and ACE-R had excellent sensitivity for amnestic impairment, sensitivity to single domain non-amnestic impairment was less good, possibly because of the lack of timed tasks needed to measure reduced information processing speed. Both the MoCA and ACE-R contain similar visuoexecutive tasks although the MoCA also includes abstraction and has more attentional tests whereas the ACE-R contains more language and memory items. The ACE-R takes a few minutes longer to administer than the MoCA alone but includes the MMSE within it. Choice of cut-off will depend on whether the test is being used as a screen (high sensitivity required) or as a diagnostic tool (high specificity required).
In conclusion, the MoCA and the ACE-R had good sensitivity and specificity for MCI, defined using modified Petersen criteria with the NINDS-CSN Harmonisation Standards Neuropsychological Battery, in patients with stable cerebrovascular disease, but the MMSE had lower sensitivity for single domain MCI. However, all three tests performed similarly in detecting multi-domain impairments, within the limits imposed by the relatively small numbers in our study. Both the MoCA and ACE-R are short, feasible tests suitable for routine clinical practice and for large studies of stroke outcome. Further longitudinal studies are required to determine the prognostic value of the MoCA and ACE-R for the development of dementia after TIA and stroke.
The Oxford vascular study is funded by the UK Stroke Association, the Dunhill Medical Trust, the National Institute of Health Research (NIHR), the Medical Research Council, the NIHR Biomedical Research Centre,Oxford, and the Wellcome Trust. Dr Pendlebury is supported by the NIHR Biomedical Research Centre, Oxford. Professor Rothwell is an NIHR Senior investigator and a Wellcome Trust Senior Investigator.
Ethical approval: The Oxford Vascular Study was approved by the Oxfordshire clinical research ethics committee (CO.043).
Author contributionsSarah Pendlebury planned this substudy and analyses, performed neuropsychological assessments, collected data and wrote the manuscript. Jose Mariz and Linda Bull performed neuropsychological assessments, Ziyah Mehta performed analyses and provided statistical expertise, and Peter Rothwell planned and directs the OXVASC study, co-wrote the manuscript and advised on analyses.
Disclosures/ Competing interests: None declared.