PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nedKargerHomeAlertsResources
 
Neuroepidemiology. Jan 2010; 34(1): 34–42.
Published online Nov 5, 2009. doi:  10.1159/000255464
PMCID: PMC2857622

Validation of the Telephone Interview for Cognitive Status-modified in Subjects with Normal Cognition, Mild Cognitive Impairment, or Dementia

Abstract

Background

The telephone assessment of cognitive functions may reduce the cost and burden of epidemiological studies.

Methods

We validated the Telephone Interview for Cognitive Status-modified (TICS-m) using an extensive in-person assessment as the standard for comparison. Clinical diagnoses of normal cognition, mild cognitive impairment (MCI), or dementia were established by consensus of physician, nurse, and neuropsychological assessments.

Results

The extensive in-person assessment classified 83 persons with normal cognition, 42 persons with MCI, and 42 persons with dementia. There was considerable overlap in TICS-m scores among the three groups. Receiver operating characteristic curves identified ≤31 as the optimal cutoff score to separate subjects with MCI from subjects with normal cognition (sensitivity = 71.4%; subjects with dementia excluded), and ≤27 to separate subjects with dementia from subjects with MCI (sensitivity = 69.0%; subjects with normal cognition excluded). The TICS-m performed well when subjects with MCI were pooled either with subjects with dementia (sensitivity = 83.3%) or with subjects with normal cognition (sensitivity = 83.3%).

Conclusions

Although the TICS-m performed well when using a dichotomous classification of cognitive status, it performed only fairly in separating MCI from either normal cognition or dementia. The TICS-m should not be used as a free-standing tool to identify subjects with MCI, and it should be used with caution as a tool to detect dementia.

Key Words: Dementia, Mild cognitive impairment, Telephone Interview for Cognitive Status-modified

Introduction

There is much interest in developing brief instruments for the telephone assessments of cognitive function to be used in epidemiological studies or in clinical trials. Telephone interviews offer great flexibility and efficiency for conducting cognitive assessments when the number of subjects to study is very large or when subjects are geographically dispersed. In addition, telephone assessments are a reasonable alternative for subjects who are unwilling or unable to leave their homes because of advanced age or disability. Cognitive assessment by telephone is necessarily constrained by the medium and only cognitive tests with auditory queries and auditory responses (or button pushing) are feasible. The domains of memory, language, attention, and some aspects of executive function can be assessed reasonably well on the telephone.

The Telephone Interview for Cognitive Status-modified (TICS-m) is a widely used telephone assessment instrument with good reliability and validity to screen for dementia [1,2,3,4]. Some studies have proposed cutoff scores to screen for dementia or cognitive impairment; however, these proposed cutoff scores have not been replicated [1,5,6,7]. In addition, there is limited evidence on the use of the TICS-m to separate mild cognitive impairment (MCI) from normal cognition or dementia. We are aware of only one report on the validity of the TICS-m across the full range of normal cognition, MCI, and dementia [5]. Thus, we conducted a validity study that focused on MCI.

Methods

Study Sample

The sample for this study included a total of 167 subjects drawn from either the Mayo Clinic Study of Aging (n = 132) or the Mayo Clinic Alzheimer's Disease Research Center (ADRC; n = 35). The Mayo Clinic Study of Aging is a population-based cohort study from Olmsted County, Minn., USA, that includes over 2,000 persons between the ages of 70 and 89 years at the time of recruitment. Extensive details about this cohort were reported elsewhere [8]. The ADRC recruits and follows series of patients with MCI, Alzheimer's disease, and other types of dementia. Subjects in the ADRC were recruited from the referral practice of the Behavioral Neurology Section, Department of Neurology, Mayo Clinic [9]. Both source studies were approved by the Mayo Clinic and Olmsted Medical Center Institutional Review Boards, and written informed consent was obtained from all subjects. For patients with dementia, we also required written informed consent from a family informant.

Telephone Interview for Cognitive Status-modified

The telephone interview occurred within 3 months before, or within 1 year after, the in-person assessments. All research assistants who administered the TICS-m were specifically trained on the procedure and were kept unaware of the results from the in-person assessment. The TICS-m was administered according to published procedures and followed a standardized script [1]. The TICS-m includes the following items: (1) name, date, age, phone number (worth 9 points); (2) counting backward (worth 2 points); (3) first, a 10-word list learning exercise and then a delayed recall of that word list (both worth 10 points each); (4) subtractions (worth 5 points); (5) responsive naming (worth 4 points); (6) repetition (worth 2 points); (7) current President and Vice President (worth 4 points); (8) finger tapping (worth 2 points), and (9) word opposites (worth 2 points). The total score is 50 points. We followed the published procedures for scoring the TICS-m and we adjusted the total scores for years of education according to the algorithm developed by Gallo and Breitner [1,2,3,7]. In particular, we added 5 points to the raw TICS-m score for subjects with less than 8 years of education, we added 2 points for 8–10 years of education, we did not add any points for 11–15 years of education, and we subtracted 2 points for subjects with 16 or more years of education [7].

Standard for Comparison

Independent of the telephone administration of the TICS-m, all participants underwent a full in-person assessment that included a neuropsychological test battery, a physician examination, and an interview with an informant. The battery was designed using subtests of the Wechsler Adult Intelligence Scale-Revised [10], the Wechsler Memory Scale-Revised [11], and other tests to assess four cognitive domains. The domains evaluated were: (1) executive function (Trail Making Test Part B [12] and Digit Symbol Substitution Test [10]); (2) language (Boston Naming Test [13] and Category Fluency [14]); (3) memory (Logical Memory II [delayed recall], Visual Reproduction II [delayed recall] [11], and Auditory Verbal Learning Test [delayed recall] [15]); and (4) visuospatial (Picture Completion and Block Design [10]). Further details about the battery of tests were reported elsewhere [8].

All subjects were examined by a physician who completed a neurological examination and administered the Short Test of Mental Status (STMS) [16,17,18]. In addition, a nurse skilled in the clinical assessment of dementia patients interviewed an informant for each subject. Informants were almost always family members.

Clinical diagnoses of normal cognition, MCI, or dementia were established by a consensus conference that involved all of the clinicians who had participated in the evaluations plus several other neurologists, physicians, and neuropsychologists. The diagnoses were based on consensus criteria for normal cognition [19,20] and MCI [20], and on the criteria for dementia in the DSM-IV [21]. Dementia patients were required to have a Clinical Dementia Rating score of 1 or higher [22].

Statistical Analyses

We compared the TICS-m scores adjusted for education to the consensus clinical diagnoses that served as the standard. For each cutoff score, we computed sensitivity and specificity. Receiver operating characteristic (ROC) curves were drawn, and the area under the ROC curve along with its corresponding 95% confidence interval provided a measure of overall validity.

We conducted four sets of validation analyses. First, we compared subjects with MCI to subjects with normal cognition, excluding subjects with dementia from the sample. Second, we compared subjects with dementia to subjects with MCI, excluding subjects with normal cognition from the sample. Third, we conducted analyses on the complete sample pooling subjects with MCI with subjects with dementia (cognitively impaired vs. cognitively normal contrast). Fourth, we conducted analyses on the complete sample pooling subjects with MCI with subjects with normal cognition (demented vs. nondemented contrast).

Results

The extensive in-person examination identified 83 cognitively normal subjects, 42 persons with MCI, and 42 persons with dementia. Table Table11 shows the demographic characteristics of the three groups and the distribution of times between the phone call and the in-person visit. There was considerable overlap among the three groups on education-adjusted TICS-m scores (fig. (fig.1).1). In the overall group, the education-adjusted TICS-m and the STMS were strongly correlated (r = 0.65; 95% CI = 0.55–0.73; p < 0.001). Among cognitively normal patients, the unadjusted TICS-m scores were only minimally correlated with education (r = 0.13; 95% CI = −0.09–0.34; p = 0.23).

Fig. 1
Scatterplot of TICS-m scores for subjects with normal cognition, with MCI, and with dementia. The horizontal line at ≤31 shows the optimal cutoff score to separate subjects with normal cognition from subjects with MCI. The horizontal line at ≤27 ...
Table 1
Description of participants and their cognitive assessments (total n = 167)

Tables Tables22 and and33 show sensitivity and specificity results for MCI compared to normal cognition (excluding subjects with dementia) and for dementia compared to MCI (excluding subjects with normal cognition). ROC analyses identified ≤31 as the optimal cutoff score on education-adjusted TICS-m to separate subjects with MCI from normal subjects (fig. (fig.2a),2a), and ≤27 as the optimal cutoff score to separate subjects with dementia from subjects with MCI (fig. (fig.2b).2b). The area under the ROC curve was 0.83 (95% CI = 0.74–0.91) for MCI versus normal and 0.79 (95% CI = 0.70–0.89) for dementia versus MCI.

Fig. 2
a ROC curve of the TICS-m for comparison of subjects with MCI versus subjects with normal cognition (subjects with dementia are excluded). The area under the ROC curve was 0.83 (95% CI = 0.74–0.91). b ROC curve of the TICS-m for subjects with ...
Table 2
Sensitivity and specificity of the TICS-m to separate subjects with MCI (n = 42) from subjects with normal cognition (n = 83), excluding subjects with dementia (n = 42) from the sample (total n = 125)
Table 3
Sensitivity and specificity of the TICS-m to separate subjects with dementia (n = 42) from subjects with MCI (n = 42), excluding subjects with normal cognition (n = 83) from the sample (total n = 84)

Tables Tables44 and and55 show sensitivity and specificity results for analyses pooling MCI with either dementia or normal cognition (dichotomous classification of the complete sample). The education-adjusted TICS-m performed well in the dichotomous classification of cognitively impaired (MCI + dementia) versus cognitively normal subjects (area under the ROC curve = 0.89; 95% CI = 0.83–0.94). At the optimal cutoff score of ≤31, sensitivity was 83.3% and specificity was 78.3% (fig. (fig.3a).3a). The TICS-m also performed well in a similar dichotomous analysis of demented subjects versus nondemented (normal cognition + MCI; area under the ROC curve = 0.89; 95% CI = 0.83–0.96). At an optimal cutoff score of ≤28, the sensitivity was 83.3% and the specificity was 81.6% (fig. (fig.3b3b).

Fig. 3
a ROC curve of the TICS-m for comparison of subjects with cognitive impairment (MCI + dementia) versus subjects with normal cognition in the complete sample. The area under the ROC curve was 0.89 (95% CI = 0.83–0.94). b ROC curve of the TICS-m ...
Table 4
Sensitivity and specificity of the TICS-m to separate subjects with either MCI or dementia (n = 84) from cognitively normal subjects (n = 83; dichotomous classification; total n = 167)
Table 5
Sensitivity and specificity of the TICS-m to separate subjects with dementia (n = 42) from subjects with either normal cognition or MCI (n = 125; dichotomous classification; total n = 167)

When using the education-adjusted TICS-m as a diagnostic tool (rather than as a screening tool), our findings suggest to classify subjects scoring ≤27 as demented, subjects scoring from 28 through 31 as MCI, and those scoring ≥32 as normal (fig. (fig.1).1). These cutoff points correctly classified 78.3% of cognitively normal subjects, and 69.0% of demented subjects. However, only 42.9% of MCI subjects were correctly classified, and subjects who were misclassified were equally likely to be classified as normal or demented. Similarly, most misclassified cognitively normal subjects (18.1%) and demented subjects (26.2%) were misclassified as MCI.

The fair performance of the TICS-m when used as a diagnostic tool for MCI may be caused by the limited discriminatory power of the learning and recall tasks. There is only one exposure to the 10-word list during the learning phase of the TICS-m, and not surprisingly, the number of words recalled immediately is rather low in the normal subjects (fig. (fig.4a).4a). The low performance in the normal subjects was even more apparent in the delayed recall task (fig. (fig.4b).4b). Therefore, scores on both immediate and delayed recall demonstrated considerable overlap across the three cognitive performance groups.

Fig. 4
Score distributions on the immediate (a) and delayed (b) 10-word recall item of the TICS-m. Note the substantial overlap among groups. White bars = Subjects with normal cognition; gray bars = subjects with MCI; and black bars = subjects with dementia. ...

Discussion

Discussion of Findings

Although the education-adjusted TICS-m performed well when subjects with MCI were pooled with subjects with normal cognition or with subjects with dementia, it performed only fairly in a three-group diagnostic classification in our elderly group of subjects. Consistent with the results from a previous smaller study [5], the TICS-m performed only fairly in separating persons with MCI from persons with either normal cognition or dementia. In particular, persons scoring from 28 through 31 on the TICS-m cannot be classified with certainty.

Diagnoses of cognitive impairment require information from knowledgeable informants and from direct patient examination. To make a diagnosis of MCI or mild dementia, neuropsychological testing is often required. Compared to consensus clinical diagnoses that include information from an informant, neuropsychological testing, and a mental status examination, the TICS-m alone will necessarily be imperfect. Thus, the TICS-m is best suited for screening rather than to assign a stand-alone diagnosis.

The TICS-m is suitable to identify subjects with normal cognition or those with dementia if we accept some limited specificity. For example, if the goal of screening with the TICS-m is to identify persons with a high probability of being cognitively normal, a cutoff score of ≥32 can be used. However, this score would necessarily fail to include a moderately large number of cognitively normal subjects who scored <32. Indeed, a recent study found that 93% of persons over age 85 years who scored >28 were cognitively normal [23]. It was not stated in that report whether an education adjustment was applied or not [23]. In the present study, a cutoff score of ≥28 identified 96.4% of the cognitively normal subjects, but also 71.4% of subjects with MCI and 31.0% of subjects with dementia.

In contrast, if the goal of screening is to identify persons likely to be demented, a cutoff score of ≤31 would identify a high proportion of demented subjects, but it would include a few cognitively normal subjects and many MCI subjects. Prior studies using the TICS-m to detect dementia have proposed cutoff scores for TICS-m of <31 [1], <29 [5], or ≤27 [6,7]. Only one of these studies used education adjustments [7]. The education-adjusted cutoff score of ≤27 was also used in the Mayo Clinic Family Study of Parkinson's Disease and in the Mayo Clinic Cohort Study of Oophorectomy and Aging [24,25].

TICS-m scores between 19 and 37 have been used to identify subjects with possible MCI to be recruited for a clinical trial [26]. The use of such a broad range of scores may have insured the exclusion of severely demented subjects and of undoubtedly normal subjects; however, that study probably included a large group of cognitively normal and demented subjects.

Comparison with Other Instruments

Compared to standard mental status examination such as the STMS or the Mini-Mental State Examination [27], the TICS-m should be superior because it contains a delayed recall task that uses 10 words. A longer list should allow greater separation between persons with preserved learning ability and patients with cognitive disorders that impair learning. However, the learning phase of the 10-word recall item on the TICS-m includes only one repetition. As our observations confirmed, a single exposure to a 10-item list resulted in poor initial learning among our normal elderly. With limited initial learning, subsequent delayed recall was also compromised in the cognitively normal subjects. Therefore, the delayed recall activity in the TICS-m did not successfully distinguish subjects with normal cognition from those with cognitive impairment. A method to increase learning of the word list that would benefit those who are cognitively normal would be preferable. The two options for enhancing initial learning would be multiple learning trials or some means of elaborative encoding such as generating sentences. Both of these techniques are suitable for telephone administration. We have applied similar methods with an alternative telephone assessment as described elsewhere [28].

Strengths and Limitations

Strengths of our study include the relatively large sample size and the independence of the clinical assessment used as standard from the TICS-m administration. Limitations of our study include the lack of longitudinal follow-up that might have shed further light on the diagnostic accuracy of the TICS-m. The majority of our patients were whites of European ancestry and well educated. Therefore, the validity of the TICS-m observed in this study may not be generalizable to more heterogeneous populations. In addition, there was an imbalance in the order of testing between the patients with dementia and other subjects, and in the lag time between in-person assessments and telephone assessments. However, this delay should not have compromised diagnostic accuracy, because the subjects with the longer delays were all in the dementia group. The use of both referral patients and randomly selected patients from a defined population could have introduced some biases into the sample. However, the majority of subjects were recruited from the random sample (132 of 167). Finally, because of variability in the use of education adjustments in other studies, it was sometimes difficult to compare our results to those of others.

Acknowledgements

This work is supported by grants from the National Institute on Aging (P50 AG016574 and U01 AG006786), the National Institute of Mental Health (K01 MH068351), the National Institute of Aging (K01 AG028573), the National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01 AR030582), and by the Robert H. and Clarice Smith and Abigail van Buren Alzheimer's Disease Research Program.

Footnotes

Conflict of Interest: Dr. Knopman has served on a Data Safety Monitoring Board for Sanofi-Aventis and serves on a Data Safety Monitoring Board for Lilly. He served as a consultant to GlaxoSmithKline. He is also an investigator in a clinical trial sponsored by Elan Pharmaceuticals. Dr. Petersen has been a consultant to GE HealthCare, Servier, and Elan Pharmaceuticals.

References

1. Welsh KA, Breitner JCS, Magruder-Habib KM. Detection of dementia in the elderly using telephone screening of cognitive status. Neuropsychiatry Neuropsychol Behav Neurol. 1993;6:103–110.
2. Brandt J, Spencer M, Folstein M. The telephone interview for cognitive status. Neuropsychiatry Neuropsychol Behav Neurol. 1988;1:111–117.
3. Plassman BL, Newman TT, Welsh KA, Helms M, Breitner JCS. Properties of the telephone interview for cognitive status. Application in epidemiological and longitudinal studies. Neuropsychiatry Neuropsychol Behav Neurol. 1994;7:235–241.
4. De Jager CA, Budge MM, Clarke R. Utility of TICS-m for the assessment of cognitive function in older adults. Int J Geriatr Psychiatry. 2003;18:318–324. [PubMed]
5. Crooks V, Clark L, Petitti D, Chui H, Chiu V. Validation of multi-stage telephone-based identification of cognitive impairment and dementia. BMC Neurol. 2005;5:8. [PMC free article] [PubMed]
6. Beeri MS, Werner P, Davidson M, Schmidler J, Silverman J. Validation of the modified telephone interview for cognitive status (TICS-m) in Hebrew. Int J Geriatr Psychiatry. 2003;18:381–386. [PubMed]
7. Gallo JJ, Breitner JC. Alzheimer's disease in the NAS-NRC Registry of aging twin veterans. 4. Performance characteristics of a two-stage telephone screening procedure for Alzheimer's dementia. Psychol Med. 1995;25:1211–1219. [PubMed]
8. Roberts RO, Geda YE, Knopman DS, Cha RH, Pankratz VS, Boeve BF, Ivnik RJ, Tangalos EG, Petersen RC, Rocca WA. The Mayo Clinic Study of Aging: design and sampling, participation, baseline measures and sample characteristics. Neuroepidemiology. 2008;30:58–69. [PMC free article] [PubMed]
9. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999;56:303–308. [PubMed]
10. Wechsler DA. Wechsler Adult Intelligence Scale-Revised. New York: Psychological Corporation; 1987.
11. Wechsler DA. Wechsler Memory Scale-Revised. New York: Psychological Corporation; 1987.
12. Reitan RM. Validity of the Trail Making Test as an indicator of organic brain damage. Percept Motor Skills. 1958;8:271–276.
13. Kaplan EF, Goodglass H, Weintraub S. The Boston Naming Test. ed 2. Philadelphia: Lea & Febiger; 1982.
14. Lucas JA, Ivnik RJ, Smith GE, Bohac DL, Tangalos EG, Graff-Radford NR, Petersen RC. Mayo's Older Americans Normative Studies: category fluency norms. J Clin Exp Neuropsychol. 1998;20:194–200. [PubMed]
15. Ivnik RJ, Malec JF, Smith GE. WAIS-R, WMS-R and AVLT norms for ages 56 through 97. Clin Neuropsychol. 1992;6:1–104.
16. Kokmen E, Naessens JM, Offord KP. A short test of mental status: description and preliminary results. Mayo Clin Proc. 1987;62:281–288. [PubMed]
17. Kokmen E, Smith GE, Petersen RC, Tangalos E, Ivnik RC. The Short Test of Mental Status. Correlations with standardized psychometric testing. Arch Neurol. 1991;48:725–728. [PubMed]
18. Tang-Wai DF, Knopman DS, Geda YE, Edland SD, Smith GE, Ivnik RJ, Tangalos EG, Boeve BF, Petersen RC. Comparison of the Short Test of Mental Status and the Mini-Mental State Examination in mild cognitive impairment. Arch Neurol. 2003;60:1777–1781. [PubMed]
19. Ivnik RJ, Malec JF, Smith GE, Tangalos EG, Petersen RC, Kokmen E, Kurland LT. Mayo's Older Americans Normative Studies: WAIS-R norms for ages 56 to 97. Clin Neuropsychol. 1992;65:1–30.
20. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256:183–194. [PubMed]
21. American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders. ed 4. Washington: American Psychiatric Association; 1994.
22. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43:2412–2414. [PubMed]
23. Graff-Radford NR, Ferman TJ, Lucas JA, Johnson HK, Parfitt FC, Heckman MG, Todd M, Sadowsky C, Epstein DE, Crook JE. A cost effective method of identifying and recruiting persons over 80 free of dementia or mild cognitive impairment. Alzheimer Dis Assoc Disord. 2006;20:101–104. [PubMed]
24. Rocca WA, Bower JH, Maraganore DM, Ahlskog JE, Grossardt BR, de Andrade M, Melton LJ., 3rd Increased risk of cognitive impairment or dementia in women who underwent oophorectomy before menopause. Neurology. 2007;69:1074–1083. [PubMed]
25. Rocca WA, Bower JH, Ahlskog JE, Elbaz A, Grossardt BR, McDonnell SK, Schaid DJ, Maraganore DM. Risk of cognitive impairment or dementia in relatives of patients with Parkinson disease. Arch Neurol. 2007;64:1458–1464. [PubMed]
26. Lines CR, McCarroll KA, Lipton RB, Block GA. Telephone screening for amnestic mild cognitive impairment. Neurology. 2003;60:261–266. [PubMed]
27. Folstein MF, Folstein SE, McHugh PR. ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. [PubMed]
28. Knopman DS, Knudson D, Yoes ME, Weiss DJ. Development and standardization of a new telephonic cognitive screening test: the Minnesota Cognitive Acuity Screen (MCAS) Neuropsychiatry Neuropsychol Behav Neurol. 2000;13:286–296. [PubMed]

Articles from Neuroepidemiology are provided here courtesy of Karger Publishers