|Home | About | Journals | Submit | Contact Us | Français|
The validity of a self-reported stroke remains inconclusive. The objective of the present study was to validate the diagnosis of self-reported stroke using stroke identified by magnetic resonance imaging (MRI) as the standard.
Community-based cohort study of non-demented, ethnically diverse elderly in northern Manhattan.
High-resolution quantitative MRI was acquired on 717 participants without dementia. Sensitivity and specificity of stroke by self-report were examined using cross-sectional analyses and the χ2-test. Putative relations between factors potentially influencing the reporting of stroke, including memory performance, cognitive function and vascular risk factors were assessed using logistic regression models. Subsequently all analyses were repeated stratified by age, sex, ethnic group and level of education.
In analyses for the whole sample, sensitivity of stroke self-report for a diagnosis of stroke on MRI was 32.4% and specificity was 78.9%. In analyses stratified by median of age (80.1 years), the validity between reported stroke and detection of stroke on MRI was significantly better in the younger than the older age group (for all vascular territories: sensitivity: 36.7% (specificity 81.3%) vs. sensitivity 27.6% (specificity: 26.2%), p=0.02). Impaired memory, cognitive or language ability, and the presence of hypertension or myocardial infarction were associated with higher false-negatives.
Using brain MRI as the standard, specificity and sensitivity of stroke self-report are low. Accuracy of self-report is influenced by age, presence of vascular disease and cognitive function. In stroke research, sensitive neuroimaging techniques rather than stroke self-report should be used to determine stroke history.
Self-administered questionnaires are frequently used to obtain information on a previous history of stroke, but the validity of a self-reported stroke remains inconclusive. In general, self-reports on medical conditions that are well defined and relatively easy to diagnose often have a high positive predictive value in contrast to conditions characterized by complex symptoms.1 Stroke is associated with motor impairment but can also be accompanied by impairments in memory, sensation and speech or language diminishing the ability of an individual to accurately report a history of stroke. Although the importance of being aware of these difficulties, particularly in studies among the elderly, has been emphasized,2 most previous studies assessing validity of stroke self-report either used neurological examination or review of medical records as the standard or performed brain imaging only on persons reporting to have had a stroke.3-9 This likely produces underreporting of patients with ambiguous symptoms or silent strokes, and consequently increased false-negatives and diminished sensitivity estimates.
The Washington/Hamilton Heights-Inwood Columbia Aging Project (WHICAP) is an ongoing, community-based study of aging and dementia that comprises elderly participants from an urban community. A unique aspect of the cohort is its multiethnic composition with inclusion of Caucasians, Caribbean Hispanics and African American participants which allows for the examination of diverse cultural, educational, medical, and genetic factors as possible modifiers in aging diseases. We previously observed a higher prevalence and incidence of cerebrovascular disease and white matter hyperintensities as well as a larger relative brain volume in African Americans and Hispanics than in Caucasians.10, 11 These observations strongly suggest that it is important to take ethnic differences in brain morphology and vascular disease into account when assessing the validity of self-reported vascular events.
The objective of the present study was to examine the validity of the self-reported history of stroke across ethnic groups in the large multiethnic WHICAP cohort by calculating sensitivity and specificity of self-reported stroke using magnetic resonance imaging (MRI) as the standard. We also explored whether the validity of stroke self-report differed by age and whether it is influenced by cognitive function, educational level or specific concomitant diseases.
Participants were part of an original cohort participating in a prospective study of aging and dementia in Medicare recipients, 65 years and older and residing in northern Manhattan.11 These participants were recruited at two time points (1992 and 1999) and followed up at regular intervals of 18 months. The sampling strategies and recruitment outcomes have been described in detail.10 Recruitment, informed consent and study procedures were approved by the Institutional Review Boards of Columbia Presbyterian Medical Center and Columbia University Health Sciences and the New York State Psychiatric Institute.
The WHICAP MRI imaging project was concurrent with the second follow-up visit of the cohort recruited in 1999 and the 6th follow-up of the cohort recruited in 1992. Participants were deemed eligible for MRI if they did not meet criteria for dementia at their last research assessment (Figure 1). At the conclusion of the first follow-up, a total of 2,113 participants were considered for MRI eligibility; 2,053 of these individuals had been seen at the first follow-up, and for 60 of these participants, their most recent visit was baseline (i.e., they were not seen during the 1st follow-up wave). Dementia was diagnosed in 272 (12.9%) participants of these 2,113 participants. Of the remaining 1,841 participants, 769 (41.8%) received MRI scans. Of the 1,072 participants who did not receive MRI scans, 407 (38.0%) refused participation, 166 (15.4%) died before they were able to be scheduled for imaging, 191 (17.8%) were lost to follow-up, 283 (26.3%) had MRI contraindications, and 25 (2.3%) were unable to be scheduled. Compared to persons who received MRI scans, those who refused participation in the MRI study but otherwise met inclusion criteria were a year older, more likely to be women and less likely to be African Americans. There were no differences in education level between the two groups.
At each evaluation, participants underwent an in-person interview of general health and function, medical history, a physical and neurological examination and a neuropsychological battery that included measures of memory, orientation, language, abstract reasoning, and visuospatial ability.12 The neuropsychological test battery and its validity in the diagnosis of dementia has been described previously.12 The diagnosis of dementia was based on standard research criteria13 and was established using all available information (except the MRI results) gathered from the initial and follow-up assessments and medical records at a consensus conference of physicians, neurologists, neuropsychologists and psychiatrists.
Stroke was defined according to the WHO criteria.14 The presence of stroke was ascertained from an interview with participants and/or their informants. Positive response(s) to any 1 of the 8 questions shown in Figure 2 was considered as suggestive of a history of stroke. Persons who answered yes to any of the 8 questions were referred to see a board certified neurologist. In addition, 80% of self-reported strokes were confirmed by review of medical records, as described previously in detail.15 Then all persons, independent on stroke self-report status, underwent MRI.
Scan acquisition was performed on a 1.5T Philips Intera scanner at Columbia University Medical Center and transferred electronically to the university of California at Davis for morphometric analysis in the Imaging of Dementia and Aging Laboratory. For measures of total brain volume, ventricular volume, and WMH volume, fluid attenuated inverse recovery (FLAIR) weighted images (TR=11,000 ms, TE=144.0 ms, 2800 inversion time, FOV 25 cm, 2 nex, 256×192 matrix with 3 mm slice thickness) were acquired in the axial orientation. T1-weighted images acquired in the axial plane and re-sectioned coronally were used to quantify hippocampus and entorhinal cortex volumes (TR=20 ms, TE = 2.1 ms, FOV 240 cm, 256×160 matrix with 1.3 mm slice thickness). The presence or absence of brain infarction on MRI was determined using all available images, including T1-weighted images, FLAIR weighted images and proton density and T2-weighted double echo images. Only lesions 3 mm or larger qualified for consideration as brain infarcts. Signal void, best seen on the T2 weighted images was interpreted to indicate a vessel. Other necessary imaging characteristics included CSF density on the T1 weighted image and if the stroke was in the basal ganglia area, distinct separation from the circle of Willis vessels and perivascular spaces. Scans were further interpreted for number of infarcts, their location (right or left hemisphere, cortical or subcortical, and specific region) and size. Infarcts ≤1 cm were defined as “small infarct”, infarcts >1 cm were defined as “large infarct”. Two raters determined the presence of cerebral infarction on MRI. Previously published kappa values for agreement among raters has been generally good, ranging from 0.73 to 0.90.16
WMH volumes were derived on FLAIR-weighted images following a two-step process. First, an operator manually traced the dura mater within the cranial vault, including the middle cranial fossa but not the posterior fossa and cerebellum. Intracranial volume was defined as the number of voxels contained within the manual tracings, multiplied by voxel dimensions and slice thickness. These manual tracings also defined the border between brain and non-brain elements and permitted for the removal of the latter. Nonuniformities in image intensity were removed and two Gaussian probability functions, representing brain matter and cerebrospinal fluid (CSF), were fitted to the skull-stripped image. Once brain matter was isolated, a single Gaussian distribution was fitted to image data and a segmentation threshold for WMH was set a priori at 3.5 SDs in pixel intensity above the mean of the fitted distribution of brain matter. Erosion of two exterior image pixels was applied to the brain matter image before modeling to remove partial volume effects and ventricular ependyma on WMH determination. WMH volume was calculated as the sum of voxels greater to or equal to 3.5 SD above the mean intensity value of the image and multiplied by voxel dimensions and slice thickness and adjusted by intracranial volume.
The presence of diabetes mellitus and hypertension were defined as a history of either disorder at any time during life. At baseline, all participants were asked whether or not they had a history of diabetes or hypertension. If affirmed, they were asked whether or not they were under treatment and the specific type of medication. Heart disease was defined as a history of atrial fibrillation and other arrythmias, myocardial infarction, congestive heart failure or angina pectoris at any time during life. A trigger question asked whether or not the individual ever smoked at least one cigarette per day for a period of one year or more. If the answer to the trigger question was no, the subject was classified as non-smoker and no further questions were asked. Participants who answered the question affirmatively were classified as current smokers when they were still smoking, or past smokers when they had quit smoking. Current and past smokers were additionally asked at what age they began smoking and how many cigarettes on average they had smoked or still smoked per day. Past smokers were also asked at what age they had stopped smoking.
First, differences in demographic and clinical characteristics between persons reporting stroke correctly and falsely were explored using ANOVA and χ2 test. Then self-report of stroke was classified as follows: a) True-positive (TP): The person reported a stroke and had a stroke on MRI; b) False-positive (FP): A stroke was reported although no stroke was detected on MRI; c) False-negative (FN): The person reported to have no history of stroke but a stroke was detected on MRI; d) True-negative (TN): This classification was used when there was no infarct on MRI and the person reported correctly to have no history of stroke. Sensitivity, specificity, and positive and negative predictive values were calculated using the following formulas: Sensitivity=TP/(TP+FN); Specificity=TN/(FP+TN); Positive Predictive Value=TP/(TP+FP); and Negative Predictive Value=TN/(FN+TN). We first classified a self reported stroke as a positive answer to any question. Then, to minimize the possibility that we misclassified TIA as stroke, we performed a series of subanalyses: first we defined self-reported stroke as a positive answer to question 7 or 8 but negative answer to questions 1-6, then we defined positive self-report as a positive answer to question 1-6 but negative answer to both questions 7 and 8. The reasoning for this approach is that positive answers to questions 1-6 may include TIA, while positive answers to questions 7 and 8 likely assess manifest stroke.
Finally, the relation between correct self report of stroke and demographic and clinical characteristics potentially influencing accurate self-report of stroke, such as age, ethnic group, cognitive function, memory function, language function, educational level or cardiovascular risk factors was assessed using logistic regression models. All analyses were first performed as crude analyses and subsequently stratified by median age (80.1 years), ethnic group and education. A value of P<0.05 was considered statistically significant. All data analysis was performed using SPSS version 15.0 software (SPSS Inc, Chicago, Ill).
Of the 769 participants with available structural MRI data, 52 met diagnostic criteria for dementia at the clinical evaluation closest to the neuroimaging study. These individuals were excluded from the current analyses, leaving 717 subjects in the final analytic sample. The characteristics of this sample are shown in table 1. In the total MRI sample were 484 (67.5%) women, the mean age was 80.1±5.5 years and 22.2% had a history of diabetes, 11.7% a history of myocardial infarction and 66.5% a history of hypertension. Eighty-five persons (11.9%) reported to have had a stroke, and 86 persons (12.0%) of the sample were current smokers. On the MRI, a stroke was observed in 225 persons (31.4%), and among these 68 persons (30.2%) had a large infarct and 186 persons (82.7%) a small infarct. Twenty-nine persons (4.0%) had both large and small infarcts. Individuals with a stroke on MRI but who failed to report a history of stroke, were more likely to be women, older and less likely to be African American than persons reporting stroke correctly.
Sensitivity and specificity of stroke self-report for a diagnosis of stroke on MRI were low (for stroke in any vascular territory: sensitivity: 32.4%; specificity: 78.9%; table 2). The corresponding positive and negative predictive values were 41.5% and 71.6%. Among the 225 persons who had a brain infarct on MRI, 73 (32.4%) reported correctly to have had a stroke, while 152 persons (67.6%) underreported stroke. When the analyses were stratified by stroke size, sensitivity of stroke self-report was better for large than small strokes (51.5% vs. 28.5%) and better for cortical than subcortical infarcts (40.0% vs. 33.1%). There were no significant differences in sensitivity or specificity between right or left hemispheric infarcts. In analyses stratified by ethnic group, there was a higher sensitivity of stroke self-report in African Americans than Caucasians or Hispanics in all analyses performed that was close to statistical significance, except cortical strokes that Caucasians reported slightly more accurate (table 2; p-value for differences in sensitivity of stroke self-report for all vascular territories between ethnic group = 0.08). In analyses stratified by the median age, sensitivity and specificity of self-report were significantly better in the younger than the older age group (p-value for differences in sensitivity of stroke self-report for all vascular territories between ages=0.02). Exclusion of questions that specifically included TIA (questions 1-3) increased specificity slightly but did not affect sensitivity (specificity for all vascular territories: 82.5%; table 2). Restriction of “stroke self-report” to a positive answer to questions 7 or 8 but negative answers to questions 1-6 lead to a decrease in sensitivity (for all vascular territories: 14.1%) and increase in specificity (for all vascular territories: 95.6%). Restriction of “self-report” to a positive response to question 1-6 (along with negative responses to questions 7 and 8) yielded similar results (table 2), suggesting that we captured true stroke rather than TIA when using all 8 questions.
In analyses relating demographic and clinical characteristics with accuracy of stroke self-report, memory, cognitive and language function were inversely related with underreporting of stroke (table 3). Presence of hypertension or myocardial infarction increased frequency of false-negative reporting. These relations did not change in models adjusting for age, sex, education and ethnic group. There was no association between sex, educational level or smoking and accuracy of self-report of stroke.
In this study sensitivity of stroke self-report for a diagnosis of stroke on MRI in the total sample was 32.4% and specificity was 78.9%. The corresponding positive and negative predictive values were 41.5% and 71.6%. When the analyses were stratified by stroke size or stroke location, sensitivity of self-report was better for large than for small strokes, better for cortical than subcortical infarcts, and highest for strokes in the MCA territory. In analyses stratified by age, sensitivity and specificity of self-report were significantly higher in younger than older persons. In analyses stratified by ethnic group, sensitivity was slightly higher in African Americans than Caucasians or Hispanics. Lower memory, cognitive or language ability or presence of hypertension or myocardial infarction were associated with an increased frequency of false-negatives. Exclusion of self-report questions 1-3 that included TIA led to a slight increase in specificity. Restriction of “stroke self-report” to a positive answer to questions 7 and 8 but negative answers to questions 1-6 lead to a decrease in sensitivity and increase in specificity.
Few studies have assessed the validity of stroke self-report, all were performed in Caucasians, and most had only the ability to assess positive predictive value but not sensitivity.1, 4-9, 17-19 Most studies reported sensitivity estimates or positive predictive values that were higher than ours. In the Tromsø Study,19 self-reported history of stroke had a positive predictive value of 79% and a sensitivity of ~80%. A population based study from Rotterdam,4 the Italian Longitudinal Study on Aging,18 the American National Health Survey3, and the American prospective study of nurses1 found positive predictive values of self-reported stroke of ~66%. In the Copenhagen stroke study,8 the true positive predictive value of self-reported stroke via questionnaire was 50% and thus lower than in our study.
A potential explanation for the differences in false-positive or false-negative rates among studies are differences in validation of stroke. Most of the previous studies did not use brain imaging but neurological examination or review of medical records to confirm stroke.1, 3, 5-8 Although stroke is mainly a clinical diagnosis, brain imaging data facilitate the validation of stroke, particularly in patients with ambiguous or no obvious symptoms. Some studies used cerebral CT which is less sensitive to subtle cerebrovascular lesions than MRI which was used in the current study.20, 21 One of the studiesthat had CT assessment, used imaging solely to confirm stroke in persons who responded affirmative when asked for a history of stroke but had an unclear diagnosis on medical records or neurological examination. The study did not reconfirm negative responses on stroke self-report.4 The use of imaging methods that are not able to detect subtle lesions and the omission of scans in persons without a history of stroke, can lead to a potential of misclassification of patients with subtle symptoms or silent strokes. This in turn can lead to an increase of the false-negative rate and a higher estimation of positive predictive value and sensitivity estimates compared to our study. The failure to separate TIA from stroke in studies lacking (sensitive) brain imaging is a common contributor to a false-positive rate of self-reported stroke.
Another explanation for the inconsistencies between studies is the difference in factors that can influence accurate report of stroke in the study populations. The mean age of our population was on average approximately 15-20 years higher than that of most previous studies assessing self-report of stroke.3-5, 7, 8, 19 It is likely that unreported strokes in survey populations occur mainly among elderly participants6 who are also frequently the victims of stroke. Approximately 75% of strokes occur in persons over age 65 years.22 Our findings of an inverse association between cognitive ability, memory function and language function suggest that impaired cognition contributes to a higher false-negative rate of self-reported stroke in the elderly compared with younger persons. Recall problems in the elderly when asked about prior events would lead to lower estimates of sensitivity and positive predictive value. However, it is likely that some of the strokes that were falsely not reported were silent strokes and thus are truly negative. In their review, Vermeer et al.23 summarized the results of 105 original papers that provided data on frequency, risk factors, or consequences of silent brain infarcts detected by MRI in various adult populations. According to these studies, silent strokes have an age-dependent frequency of 8-28% in the general adult population and are 5 times more frequent than clinical strokes. In the large population-based Cardiovascular Health Study 89% of lacunar strokes were silent but associated with neurological symptoms.24 In the Rotterdam Study, 20% of the population had silent infarcts, while 2.4% had symptomatic infarcts and 1.5% had both.25
In our study, sensitivity was slightly higher in African Americans than Caucasians or Hispanics. A possible explanation for this observation is that prevalence and incidence of vascular risk factors and cerebrovascular disease are higher in African Americans.26 It is likely, that individuals who have had previous contact with health services or physicians at which vascular risk factors as a risk factor for stroke were discussed or who have more contact with persons suffering from a stroke, are more aware of signs and symptoms of stroke.
Our study has important strengths. High-resolution quantitative MRI was available in all persons, independent of affirmative or negative self-report. This allowed us to determine not only positive predictive values but also sensitivity, specificity and negative predictive values of self-report. The use of a well-characterized multiethnic cohort allowed us to determine the validity of self-report among a diverse group of older adults whose susceptibility for vascular disease is widely variable. Finally, the neuropsychological test battery used allowed us to assess several cognitive domains and take into account the impact of these functions on accuracy of self-report.
An important consideration in the interpretation of our results is that the study was based on survivors that were able to undergo MRI, and that it was conducted in an urban elderly community with a high prevalence of risk factors for mortality and vascular disease. Thus, our results, including positive and negative predictive values, may not be generalizable to cohorts with younger individuals or a lower morbidity burden. Also, despite using highly sensitive MRI, it is possible that we missed subtle vascular lesions, and that true sensitivity and positive predictive values are slightly lower than reported.
Our results indicate that sensitivity and specificity of stroke self-report are low when using MRI scans as validation. It further suggests that accuracy of self-report is influenced by ethnicity, age, cognitive function and diagnosis of vascular disease. In stroke research, sensitive neuroimaging techniques rather than stroke self-report should be used to determine stroke history.
This work was supported by National Institutes of Health grants AG007232 and AG029949. The authors had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.