|Home | About | Journals | Submit | Contact Us | Français|
The telephone assessment of cognitive functions may reduce the cost and burden of epidemiological studies.
We validated the Telephone Interview for Cognitive Status-modified (TICS-m) using an extensive in-person assessment as the standard for comparison. Clinical diagnoses of normal cognition, mild cognitive impairment (MCI), or dementia were established by consensus of physician, nurse, and neuropsychological assessments.
The extensive in-person assessment classified 83 persons with normal cognition, 42 persons with MCI, and 42 persons with dementia. There was considerable overlap in TICS-m scores among the three groups. Receiver operating characteristic curves identified ≤31 as the optimal cutoff score to separate subjects with MCI from subjects with normal cognition (sensitivity = 71.4%; subjects with dementia excluded), and ≤27 to separate subjects with dementia from subjects with MCI (sensitivity = 69.0%; subjects with normal cognition excluded). The TICS-m performed well when subjects with MCI were pooled either with subjects with dementia (sensitivity = 83.3%) or with subjects with normal cognition (sensitivity = 83.3%).
Although the TICS-m performed well when using a dichotomous classification of cognitive status, it performed only fairly in separating MCI from either normal cognition or dementia. The TICS-m should not be used as a free-standing tool to identify subjects with MCI, and it should be used with caution as a tool to detect dementia.
There is much interest in developing brief instruments for the telephone assessments of cognitive function to be used in epidemiological studies or in clinical trials. Telephone interviews offer great flexibility and efficiency for conducting cognitive assessments when the number of subjects to study is very large or when subjects are geographically dispersed. In addition, telephone assessments are a reasonable alternative for subjects who are unwilling or unable to leave their homes because of advanced age or disability. Cognitive assessment by telephone is necessarily constrained by the medium and only cognitive tests with auditory queries and auditory responses (or button pushing) are feasible. The domains of memory, language, attention, and some aspects of executive function can be assessed reasonably well on the telephone.
The Telephone Interview for Cognitive Status-modified (TICS-m) is a widely used telephone assessment instrument with good reliability and validity to screen for dementia [1,2,3,4]. Some studies have proposed cutoff scores to screen for dementia or cognitive impairment; however, these proposed cutoff scores have not been replicated [1,5,6,7]. In addition, there is limited evidence on the use of the TICS-m to separate mild cognitive impairment (MCI) from normal cognition or dementia. We are aware of only one report on the validity of the TICS-m across the full range of normal cognition, MCI, and dementia . Thus, we conducted a validity study that focused on MCI.
The sample for this study included a total of 167 subjects drawn from either the Mayo Clinic Study of Aging (n = 132) or the Mayo Clinic Alzheimer's Disease Research Center (ADRC; n = 35). The Mayo Clinic Study of Aging is a population-based cohort study from Olmsted County, Minn., USA, that includes over 2,000 persons between the ages of 70 and 89 years at the time of recruitment. Extensive details about this cohort were reported elsewhere . The ADRC recruits and follows series of patients with MCI, Alzheimer's disease, and other types of dementia. Subjects in the ADRC were recruited from the referral practice of the Behavioral Neurology Section, Department of Neurology, Mayo Clinic . Both source studies were approved by the Mayo Clinic and Olmsted Medical Center Institutional Review Boards, and written informed consent was obtained from all subjects. For patients with dementia, we also required written informed consent from a family informant.
The telephone interview occurred within 3 months before, or within 1 year after, the in-person assessments. All research assistants who administered the TICS-m were specifically trained on the procedure and were kept unaware of the results from the in-person assessment. The TICS-m was administered according to published procedures and followed a standardized script . The TICS-m includes the following items: (1) name, date, age, phone number (worth 9 points); (2) counting backward (worth 2 points); (3) first, a 10-word list learning exercise and then a delayed recall of that word list (both worth 10 points each); (4) subtractions (worth 5 points); (5) responsive naming (worth 4 points); (6) repetition (worth 2 points); (7) current President and Vice President (worth 4 points); (8) finger tapping (worth 2 points), and (9) word opposites (worth 2 points). The total score is 50 points. We followed the published procedures for scoring the TICS-m and we adjusted the total scores for years of education according to the algorithm developed by Gallo and Breitner [1,2,3,7]. In particular, we added 5 points to the raw TICS-m score for subjects with less than 8 years of education, we added 2 points for 8–10 years of education, we did not add any points for 11–15 years of education, and we subtracted 2 points for subjects with 16 or more years of education .
Independent of the telephone administration of the TICS-m, all participants underwent a full in-person assessment that included a neuropsychological test battery, a physician examination, and an interview with an informant. The battery was designed using subtests of the Wechsler Adult Intelligence Scale-Revised , the Wechsler Memory Scale-Revised , and other tests to assess four cognitive domains. The domains evaluated were: (1) executive function (Trail Making Test Part B  and Digit Symbol Substitution Test ); (2) language (Boston Naming Test  and Category Fluency ); (3) memory (Logical Memory II [delayed recall], Visual Reproduction II [delayed recall] , and Auditory Verbal Learning Test [delayed recall] ); and (4) visuospatial (Picture Completion and Block Design ). Further details about the battery of tests were reported elsewhere .
All subjects were examined by a physician who completed a neurological examination and administered the Short Test of Mental Status (STMS) [16,17,18]. In addition, a nurse skilled in the clinical assessment of dementia patients interviewed an informant for each subject. Informants were almost always family members.
Clinical diagnoses of normal cognition, MCI, or dementia were established by a consensus conference that involved all of the clinicians who had participated in the evaluations plus several other neurologists, physicians, and neuropsychologists. The diagnoses were based on consensus criteria for normal cognition [19,20] and MCI , and on the criteria for dementia in the DSM-IV . Dementia patients were required to have a Clinical Dementia Rating score of 1 or higher .
We compared the TICS-m scores adjusted for education to the consensus clinical diagnoses that served as the standard. For each cutoff score, we computed sensitivity and specificity. Receiver operating characteristic (ROC) curves were drawn, and the area under the ROC curve along with its corresponding 95% confidence interval provided a measure of overall validity.
We conducted four sets of validation analyses. First, we compared subjects with MCI to subjects with normal cognition, excluding subjects with dementia from the sample. Second, we compared subjects with dementia to subjects with MCI, excluding subjects with normal cognition from the sample. Third, we conducted analyses on the complete sample pooling subjects with MCI with subjects with dementia (cognitively impaired vs. cognitively normal contrast). Fourth, we conducted analyses on the complete sample pooling subjects with MCI with subjects with normal cognition (demented vs. nondemented contrast).
The extensive in-person examination identified 83 cognitively normal subjects, 42 persons with MCI, and 42 persons with dementia. Table Table11 shows the demographic characteristics of the three groups and the distribution of times between the phone call and the in-person visit. There was considerable overlap among the three groups on education-adjusted TICS-m scores (fig. (fig.1).1). In the overall group, the education-adjusted TICS-m and the STMS were strongly correlated (r = 0.65; 95% CI = 0.55–0.73; p < 0.001). Among cognitively normal patients, the unadjusted TICS-m scores were only minimally correlated with education (r = 0.13; 95% CI = −0.09–0.34; p = 0.23).
Tables Tables22 and and33 show sensitivity and specificity results for MCI compared to normal cognition (excluding subjects with dementia) and for dementia compared to MCI (excluding subjects with normal cognition). ROC analyses identified ≤31 as the optimal cutoff score on education-adjusted TICS-m to separate subjects with MCI from normal subjects (fig. (fig.2a),2a), and ≤27 as the optimal cutoff score to separate subjects with dementia from subjects with MCI (fig. (fig.2b).2b). The area under the ROC curve was 0.83 (95% CI = 0.74–0.91) for MCI versus normal and 0.79 (95% CI = 0.70–0.89) for dementia versus MCI.
Tables Tables44 and and55 show sensitivity and specificity results for analyses pooling MCI with either dementia or normal cognition (dichotomous classification of the complete sample). The education-adjusted TICS-m performed well in the dichotomous classification of cognitively impaired (MCI + dementia) versus cognitively normal subjects (area under the ROC curve = 0.89; 95% CI = 0.83–0.94). At the optimal cutoff score of ≤31, sensitivity was 83.3% and specificity was 78.3% (fig. (fig.3a).3a). The TICS-m also performed well in a similar dichotomous analysis of demented subjects versus nondemented (normal cognition + MCI; area under the ROC curve = 0.89; 95% CI = 0.83–0.96). At an optimal cutoff score of ≤28, the sensitivity was 83.3% and the specificity was 81.6% (fig. (fig.3b3b).
When using the education-adjusted TICS-m as a diagnostic tool (rather than as a screening tool), our findings suggest to classify subjects scoring ≤27 as demented, subjects scoring from 28 through 31 as MCI, and those scoring ≥32 as normal (fig. (fig.1).1). These cutoff points correctly classified 78.3% of cognitively normal subjects, and 69.0% of demented subjects. However, only 42.9% of MCI subjects were correctly classified, and subjects who were misclassified were equally likely to be classified as normal or demented. Similarly, most misclassified cognitively normal subjects (18.1%) and demented subjects (26.2%) were misclassified as MCI.
The fair performance of the TICS-m when used as a diagnostic tool for MCI may be caused by the limited discriminatory power of the learning and recall tasks. There is only one exposure to the 10-word list during the learning phase of the TICS-m, and not surprisingly, the number of words recalled immediately is rather low in the normal subjects (fig. (fig.4a).4a). The low performance in the normal subjects was even more apparent in the delayed recall task (fig. (fig.4b).4b). Therefore, scores on both immediate and delayed recall demonstrated considerable overlap across the three cognitive performance groups.
Although the education-adjusted TICS-m performed well when subjects with MCI were pooled with subjects with normal cognition or with subjects with dementia, it performed only fairly in a three-group diagnostic classification in our elderly group of subjects. Consistent with the results from a previous smaller study , the TICS-m performed only fairly in separating persons with MCI from persons with either normal cognition or dementia. In particular, persons scoring from 28 through 31 on the TICS-m cannot be classified with certainty.
Diagnoses of cognitive impairment require information from knowledgeable informants and from direct patient examination. To make a diagnosis of MCI or mild dementia, neuropsychological testing is often required. Compared to consensus clinical diagnoses that include information from an informant, neuropsychological testing, and a mental status examination, the TICS-m alone will necessarily be imperfect. Thus, the TICS-m is best suited for screening rather than to assign a stand-alone diagnosis.
The TICS-m is suitable to identify subjects with normal cognition or those with dementia if we accept some limited specificity. For example, if the goal of screening with the TICS-m is to identify persons with a high probability of being cognitively normal, a cutoff score of ≥32 can be used. However, this score would necessarily fail to include a moderately large number of cognitively normal subjects who scored <32. Indeed, a recent study found that 93% of persons over age 85 years who scored >28 were cognitively normal . It was not stated in that report whether an education adjustment was applied or not . In the present study, a cutoff score of ≥28 identified 96.4% of the cognitively normal subjects, but also 71.4% of subjects with MCI and 31.0% of subjects with dementia.
In contrast, if the goal of screening is to identify persons likely to be demented, a cutoff score of ≤31 would identify a high proportion of demented subjects, but it would include a few cognitively normal subjects and many MCI subjects. Prior studies using the TICS-m to detect dementia have proposed cutoff scores for TICS-m of <31 , <29 , or ≤27 [6,7]. Only one of these studies used education adjustments . The education-adjusted cutoff score of ≤27 was also used in the Mayo Clinic Family Study of Parkinson's Disease and in the Mayo Clinic Cohort Study of Oophorectomy and Aging [24,25].
TICS-m scores between 19 and 37 have been used to identify subjects with possible MCI to be recruited for a clinical trial . The use of such a broad range of scores may have insured the exclusion of severely demented subjects and of undoubtedly normal subjects; however, that study probably included a large group of cognitively normal and demented subjects.
Compared to standard mental status examination such as the STMS or the Mini-Mental State Examination , the TICS-m should be superior because it contains a delayed recall task that uses 10 words. A longer list should allow greater separation between persons with preserved learning ability and patients with cognitive disorders that impair learning. However, the learning phase of the 10-word recall item on the TICS-m includes only one repetition. As our observations confirmed, a single exposure to a 10-item list resulted in poor initial learning among our normal elderly. With limited initial learning, subsequent delayed recall was also compromised in the cognitively normal subjects. Therefore, the delayed recall activity in the TICS-m did not successfully distinguish subjects with normal cognition from those with cognitive impairment. A method to increase learning of the word list that would benefit those who are cognitively normal would be preferable. The two options for enhancing initial learning would be multiple learning trials or some means of elaborative encoding such as generating sentences. Both of these techniques are suitable for telephone administration. We have applied similar methods with an alternative telephone assessment as described elsewhere .
Strengths of our study include the relatively large sample size and the independence of the clinical assessment used as standard from the TICS-m administration. Limitations of our study include the lack of longitudinal follow-up that might have shed further light on the diagnostic accuracy of the TICS-m. The majority of our patients were whites of European ancestry and well educated. Therefore, the validity of the TICS-m observed in this study may not be generalizable to more heterogeneous populations. In addition, there was an imbalance in the order of testing between the patients with dementia and other subjects, and in the lag time between in-person assessments and telephone assessments. However, this delay should not have compromised diagnostic accuracy, because the subjects with the longer delays were all in the dementia group. The use of both referral patients and randomly selected patients from a defined population could have introduced some biases into the sample. However, the majority of subjects were recruited from the random sample (132 of 167). Finally, because of variability in the use of education adjustments in other studies, it was sometimes difficult to compare our results to those of others.
This work is supported by grants from the National Institute on Aging (P50 AG016574 and U01 AG006786), the National Institute of Mental Health (K01 MH068351), the National Institute of Aging (K01 AG028573), the National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01 AR030582), and by the Robert H. and Clarice Smith and Abigail van Buren Alzheimer's Disease Research Program.
Conflict of Interest: Dr. Knopman has served on a Data Safety Monitoring Board for Sanofi-Aventis and serves on a Data Safety Monitoring Board for Lilly. He served as a consultant to GlaxoSmithKline. He is also an investigator in a clinical trial sponsored by Elan Pharmaceuticals. Dr. Petersen has been a consultant to GE HealthCare, Servier, and Elan Pharmaceuticals.