PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
AIDS Behav. Author manuscript; available in PMC Mar 12, 2013.
Published in final edited form as:
PMCID: PMC3594991
NIHMSID: NIHMS439853
Performances on the CogState and Standard Neuropsychological Batteries Among HIV Patients Without Dementia
Edgar Turner Overton,1 John S.K. Kauwe,2 Rob Paul,3 Karen Tashima,4 David F. Tate,4 Pragna Patel,5 Chuck Carpenter,4 David Patty,2 John T. Brooks,5 and David B Clifford1
1Washington University School of Medicine, St Louis MO
2Department of Biology, Brigham Young University, Provo, UT
3University of Missouri, St. Louis, Department of Psychology, St. Louis, MO
4Brown University School of Medicine, Providence, RI
5Division of HIV/AIDS Prevention, Centers for Disease Control and Prevention, Atlanta GA
CORRESPONDING AUTHOR: E. Turner Overton, MD, Washington University School of Medicine, St Louis, MO, Phone: 314-454-8225, Fax: 314-454-5392, toverton/at/dom.wustl.edu
HIV-associated neurocognitive disorders (HAND) remain prevalent but challenging to diagnose particularly among non-demented individuals. To determine whether a brief computerized battery correlates with formal neurocognitive testing, we identified 46 HIV-infected persons who had undergone both formal neurocognitive testing and a brief computerized battery. Simple detection tests correlated best with formal neuropsychological testing. By multivariable regression model, 53% of the variance in the composite Global Deficit Score was accounted for by elements from the brief computerized tool (p<0.01). These data confirm previous correlation data with the computerized battery, yet illustrate remaining challenges for neurocognitive screening.
With the development of combination antiretroviral therapy (cART), the prevalence of HIV-associated dementia (HAD) has declined but less severe HIV-associated neurocognitive disorders (HAND) have become prominent [1]. With the transformation of HIV into a chronic medical illness, advanced age and HIV infection may act synergistically to increase the prevalence of HAND [2]. Recent cohort studies have reported the prevalence of HAND to range from 39% to 69% of subjects on cART [3, 4]. The need for brief and psychometrically sound methods to evaluate neurocognitive function is important to identify HAND and limit progressive impairment in one’s capacity to adhere to medical regimens, safely operate a motor vehicle, complete basic activities of daily life, and maintain employment [58].
Some available screening tools used to diagnose neurocognitive impairment, such as the International AIDS Dementia Screen or the Mini-Mental Status Exam, are insensitive to early manifestations of functional impairment [911]. Conversely, formal neurocognitive testing is time-consuming and requires special training and thus, cannot realistically be used in the typical outpatient setting, where the necessary time, staffing, space, and funding are generally not available. CogState, a commercially available product (http://www.cogstate.com), is a computerized cognitive test battery designed to measure psychomotor performance, attention, memory, and executive functioning: domains frequently impaired in persons with early neurocognitive disorders [12, 13].The battery consists of brief tasks in the form of card games to minimize language and cultural differences. It has previously been validated in persons with HAD [13] and been used in clinical cohort studies to measure change in neurocognitive function and performance [14]. In this analysis, we sought to determine the correlation between CogState and formal neuropsychological testing to detect neurocognitive impairment in a subset of healthy, autonomous HIV-infected persons who are followed in two cohort studies at the Washington University Outpatient HIV Clinic.
Washington University in St. Louis is a site for two prospective cohort studies evaluating complications of HIV and HIV therapy including cognitive function, one with a validated traditional neurocognitive testing battery and the other with CogState. To determine the utility of the CogState computerized battery as a screening tool, subjects enrolled in both studies were identified to compare the two neurocognitive batteries. Subjects were eligible for participation in this study if they completed an assessment in CHARTER within 6 months of the baseline SUN assessment. Enrollment criteria for the studies have been outlined previously [15, 16]. Both studies were approved by the Washington University IRB and all subjects provided written informed consent.
SUN Study
The Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (The SUN Study) is a CDC-funded multi-site prospective observational cohort monitoring complications of antiretroviral therapy and HIV [15]. At baseline and each six-month visit thereafter, participants are evaluated for neurocognitive ability using CogHealth© CogState Ltd., Melbourne, Australia [13]. The computerized battery requires 12–15 minutes to complete and consists of 6 individual tests: two tests of detection evaluating simple reaction times (DET1 and DET2) assess psychomotor function and speed of processing domains; a test of identification as a choice reaction time (IDN) assesses the domains of visual attention and vigilance; the one back test (ONB) assesses the domains of attention and working memory; the monitoring test is a measure of divided attention (MON) that assesses the domain of attention; and the associate learning test (ASSL) assesses the domain of visual learning and memory. Each test is scored based on time to complete the task (speed) and error rate (accuracy).
CHARTER Study
The CHARTER study is an NIH-funded multi-site cohort to explore HIV neurological complications in the context of emerging antiviral treatments such as cART [16, 17]. Participants receive comprehensive neuromedical, neurocognitive, and laboratory examinations. The neurocognitive battery performed in CHARTER requires approximately 1 hour to assess the following domains: attention/psychomotor speed (Trailmaking Test Part A, Symbol Digit Test and Symbol Search Test from WAIS-III, Paced Auditory Serial Addition Task); fine motor speed skills (Dominant and Non-dominant Hand Pegboard Test); learning and memory (Brief Visuospatial Memory Test–Revised, Hopkins Verbal Learning Test–Revised, Figure Memory Learning Test and Story Memory Learning test); executive functioning including working memory (Wisconsin Card Sorting Test, Letter-Number Sequence from WAIS-III), fluency (Controlled Oral Word Association Test and Animal Category Test), and set shifting/response inhibition (Trailmaking Test Part B) [1830].
Statistical Analysis
Data from the computerized battery were evaluated for normality of data distribution; reaction time measures were log10 transformed due to a positive skew of the distribution and accuracy measures were transformed using Arcsine-root transformation [31]. The raw scores from the neurocognitive tests from CHARTER were converted into T scores corrected for demographic data to minimize the impact of education, age, race/ethnicity, and sex. The Global Deficit Score (GDS) is a composite score calculated by converting the T scores from the CHARTER battery into one summary deficit score ranging from 0 (normal) to 5 (severe neurocognitive impairment) [32]. For the evaluation of construct validity, a correlation matrix was created to determine the correlation between the CogState measures and the CHARTER Study standard neuropsychological measures, including the GDS, using Pearson’s correlation coefficient. To determine the potential value of the CogState battery as a screening tool, we performed a stepwise linear regression analysis with GDS (log transformed to approximate a normal distribution) as the dependent variable and the CogState measures as independent variables. The stepwise regression was performed using the default selection method (forward in, backward out) in SAS. The stepwise regression was validated using 1,000 bootstrapping replicates. The significant Cogstate variables from the stepwise regression were used as independent variables in a multivariable regression model with GDS as the dependent variable to determine the model R2.
Forty-six subjects enrolled in both cohort studies were eligible for the present analysis. Table 1 outlines the clinical parameters of the subjects at the time of baseline CogState evaluation between 2004 and 2006. Median CD4 count was 424.5 c/mm3 with 74% of the cohort on cART and 61% with HIV viral load < 400 cp/mL. The median GDS, as calculated from the CHARTER study, for the cohort was 0.47 (range 0.00–2.79), with 24 subjects (52%) having normal function (GDS<0.5) and 22 (48%) having mild to moderate impairment (GDS≥0.5).
Table 1
Table 1
Baseline Demographics of Study Participants, the Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (The SUN Study) and the CNS HIV Antiretroviral Therapy Effects Research (CHARTER) study.
The correlation matrix showing the Pearson product-moment correlation coefficient between each COGSTATE index and the dependent variables from the CHARTER clinical exam, including GDS is shown in Table 2. The speed measures for both simple detection tests (DET1, DET2) and the identification task (IDN), a more complex reaction time task, correlated with the GDS and had the highest levels of correlation across the tests performed in the CHARTER battery. The accuracy components of the CogState battery generally correlated poorly with the individual neurocognitive tests in the CHARTER battery. The accuracy measures for the complex tasks of One-Back Memory (ONB) and the Measure of Divided Attention (MON) were the only accuracy measures that correlated with the GDS derived from the CHARTER testing.
Table 2
Table 2
Correlation Matrix between CogState Tests and CHARTER Neurocognitive Tests§
The stepwise regression analysis identified the following potential independent correlates with GDS: the accuracy and speed of the two simple detection tests (DET1lmn, DET2acc) and 3 more complex tasks: the accuracy of the associate learning (ASSLacc), the accuracy of monitoring tasks (MONacc), and the accuracy of the One Back test (ONBacc). A regression model using GDS as the dependent variable and these measures as independent variables yielded a model R2 of 0.53 (p<0.0001) indicating that approximately 53% of the variance in the GDS is explained by these five CogState variables. In the validation analyses using results from the stepwise regression analysis of 1,000 bootstrap replicates MONacc appeared in 96% of the replicates, DET1lmn appeared in 76% of the replicates, DET2acc appeared in 71% of the replicates, ASSLacc appeared in 57% of the replicates and ONBacc appeared in 51% of the replicates. A regression model using GDS as the dependent variable and MONacc, DETlmn and DET2acc (the three variables that appear in more than 70% of the replicates) yields a model R2 of 0.39 (p=0.0002).
In this study, we compared a brief, self-administered computerized screening battery with formal neurocognitive assessment. With the exception of reaction tests that evaluate functional speed, individual tests in the computerized battery correlated poorly with formal neurocognitive testing. Cognitive slowing is a prominent feature of HAND, and thus, these results are consistent with previous comparisons of the CogState battery to formal neurocognitive functioning although the associations are not as robust for less severe cognitive impairment [13].
While these results illustrate that a single brief screening test may be insensitive to identifying people with neurocognitive impairment, particularly those with mild impairment, the modeling of several parameters from the CogState battery against the Global Deficit Score highlights that there is utility to a brief computerized neurocognitive screening tool. Future prospective studies are needed to determine the sensitivity of the CogState battery to identify HAND. If it proves to be a sensitive tool with established valid cutpoints, this battery could be used to identify persons with mild cognitive impairment. However, in the absence of cutpoints that denote clinical significance, the battery will not achieve optimal clinical utility.
We recognize the limitations of this analysis. It consists of a small group of individuals and lacks a control group. No persons with advanced cognitive imparirment were included. Another limitation to these findings was the lack of correlation between the Associate Memory test and the measures of learning and memory. The CogState battery evaluated here was insensitive to these measures and thus may impact the ability of Cogstate to identify persons with impaired learning ability. Additionally, there was a lack of correlation between CogState indices and the Trailmaking Part B test, which is widely used to reflect executive function with important implications regarding one’s ability to perform activities of daily living independently [33]. Our data fail to confirm the previous work by Cysique et al, in which the strongest correlations were with the Trail Making tests. However, their work focused on persons with HAD. While the Trailmaking Part B test does not necessarily capture all of the components of executive control important for daily living, our findings suggest that CogState may not provide critical information related to early decline in functional independence, though it should be noted that executive function is a heterogeneous construct.
In summary, we found that a compilation of the tests from a brief computerized screening tool for neurocognitive function was correlated to traditional neurocognitive testing among HIV-infected persons. These findings confirm previous reports of correlation between brief computerized CogState battery and standard neuropsychological examination [13], especially for identifying cognitive slowing, a central feature of HIV-associated neurocognitive disorders, although there were differences regarding different domains potentially related to differences in the severity of impairment in the studies [34]. Nevertheless, additional research is needed to fully evaluate the utility of this battery.
Acknowledgments
The authors would like to express our appreciation to all the SUN study and CHARTER participants. No authors have conflicts of interest regarding this research.
The SUN Study Investigators are: John T. Brooks, Pragna Patel, Lois Conley, and Tim Bush, Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, CDC, Atlanta, Georgia; Kathleen Wood, Rose Baker, and Cheryl Akridge, Cerner Corporation, Vienna, Virginia; John Hammer, Tara Kennedy, Barbara Widick and Billie Thomas, Denver Infectious Disease Consultants, Inc., Denver, Colorado; Ken Lichtenstein and Cheryl Stewart, National Jewish Medical and Research Center, Denver, Colorado; Keith Henry, Jason Baker, Rachel Prosser, Edie Gunderson, Miki Olson, and John Hall, Hennepin County Medical Center, Minneapolis, Minnesota; Frank Rhame, Mark Olson, and Eve Austad, Abbott-Northwestern Hospital, Minneapolis, Minnesota; Hal Martin, Meaghan Morton, and Cheri Murch, Park-Nicollet Institute, Minneapolis, Minnesota; Charles Carpenter, Susan Cu-Uvin, Kenneth Mayer, Erna Milunka Kojic, Jennifer Florczyk, Sara Metzler, and Patricia D’Aiello, The Miriam Hospital, Providence, Rhode Island; and E. Turner Overton, Don Connor, Lisa Kessels, Mariea Snell, Sara Hubert, Dorothea Dedeaux-Turner, and Dave Coughlan, Washington University School of Medicine, St. Louis, Missouri.
The CNS HIV Anti-Retroviral Therapy Effects Research (CHARTER) is supported by award N01 MH22005 from the National Institutes of Health. The CNS HIV Anti-Retroviral Therapy Effects Research (CHARTER) group is affiliated with the Johns Hopkins University, Mount Sinai School of Medicine, University of California, San Diego, University of Texas, Galveston, University of Washington, Seattle, Washington University, St. Louis and is headquartered at the University of California, San Diego and includes: Director: Igor Grant, M.D.; Co-Directors: J. Allen McCutchan, M.D., Ronald J. Ellis, M.D., Ph.D., Thomas D. Marcotte, Ph.D.; Center Manager: Donald Franklin, Jr.; Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Terry Alexander, R.N.; Laboratory, Pharmacology and Immunology Component: Scott Letendre, M.D. (P.I.), Edmund Capparelli, Pharm.D.; Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), J. Hampton Atkinson, M.D., Steven Paul Woods, Psy.D., Matthew Dawson; Virology Component: Joseph K. Wong, M.D. (P.I.); Imaging Component: Christine Fennema-Notestine, Ph.D. (Co-P.I.), Michael J. Taylor, Ph.D. (Co-P.I.), Rebecca Theilmann, Ph.D.; Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman,; Statistics Unit: Ian Abramson, Ph.D. (P.I.), Florin Vaida, Ph.D.; Protocol Coordinating Component: Thomas D. Marcotte, Ph.D. (P.I.), Rodney von Jaeger, M.P.H.; Johns Hopkins University Site: Justin McArthur (P.I.), Mary Smith; Mount Sinai School of Medicine Site: Susan Morgello, M.D. (Co-P.I.) and David Simpson, M.D. (Co-P.I.), Letty Mintz, N.P.; University of California, San Diego Site: J. Allen McCutchan, M.D. (P.I.), Will Toperoff, N.P..; University of Washington, Seattle Site: Ann Collier, M.D. (Co-P.I.) and Christina Marra, M.D. (Co-P.I.), Trudy Jones, M.N., A.R.N.P.; University of Texas, Galveston Site: Benjamin Gelman, M.D., Ph.D. (P.I.), Eleanor Head, R.N., B.S.N.; and Washington University, St. Louis Site: David Clifford, M.D. (P.I.), Muhammad Al-Lozi, M.D., Mengesha Teshome, M.D.
Footnotes
DISCLAIMER: The findings and conclusions from this review are those of the authors and do not necessarily represent the views of the United States Government or the Centers for Disease Control and Prevention.
1. Antinori A, Arendt G, Becker JT, Brew BJ, Byrd DA, Cherner M, et al. Updated research nosology for HIV-associated neurocognitive disorders. Neurology. 2007;69:1789–1799. [PubMed]
2. Effros RB, Fletcher CV, Gebo K, Halter JB, Hazzard WR, Horne FM, et al. Aging and infectious diseases: workshop on HIV infection and aging: what is known and future research directions. Clin Infect Dis. 2008;15:542–553. [PMC free article] [PubMed]
3. Robertson KR, Smurzynski M, Parsons TD, Wu K, Bosch RJ, Wu J, et al. The prevalence and incidence of neurocognitive impairment in the HAART era. AIDS. 2007;12:1915–1921. [PubMed]
4. Simioni S, Cavassini M, Annoni J, Rimbault Abraham A, Bourquin I, Schiffer V, et al. Cognitive dysfunction in HIV patients despite long-standing suppression of viremia. AIDS. 2010;24:1243–1250. [PubMed]
5. Woods SP, Moran LM, Carey CL, Dawson MS, Iudicello JE, Gibson S, et al. HIV Neurobehavioral Research Center Group. Prospective memory in HIV infection: is "remembering to remember" a unique predictor of self-reported medication management? Arch Clin Neuropsychol. 2008;23:257–270. [PMC free article] [PubMed]
6. Marcotte TD, Lazzaretto D, Scott JC, Roberts E, Woods SP, Letendre S, et al. Visual attention deficits are associated with driving accidents in cognitively-impaired HIV-infected individuals. J Clin Exp Neuropsychol. 2006;28:13–28. [PubMed]
7. Heaton RK, Velin RA, McCutchan JA, Gulevich SJ, Atkinson JH, Wallace MR, et al. Neuropsychological impairment in human immunodeficiency virus-infection: implications for employment. HNRC Group. HIV Neurobehavioral Research Center. Psychosom Med. 1994;56:8–17. [PubMed]
8. Sevigny JJ, Albert SM, McDermott MP, Schifitto G, McArthur JC, Sacktor N, et al. An evaluation of neurocognitive status and markers of immune activation as predictors of time to death in advanced HIV infection. Arch Neurol. 2007;64:97–102. [PubMed]
9. Sacktor NC, Wong M, Nakasujja N, Skolasky RL, Selnes OA, Musisi S, et al. The International HIV Dementia Scale: a new rapid screening test for HIV dementia. AIDS. 2005;2:1367–1374. [PubMed]
10. Bottiggi KA, Chang JJ, Schmitt FA, Avison MJ, Mootoor Y, Nath A, et al. The HIV Dementia Scale: predictive power in mild dementia and HAART. J Neurol Sci. 2007;260:11–15. [PubMed]
11. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state” A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. [PubMed]
12. U.S. Department of Health and Human Services Agency for Health Care Policy and Research. Clinical Practice Guidelines, Number 19. (1996). Recognition and initial assessment of Alzheimer's disease and related dementias. AHCPR Publication No. 97-0702. http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=hstat6.chapter.30948.
13. Cysique LA, Maruff P, Darby D, Brew BJ. The assessment of cognitive function in advanced HIV-1 infection and AIDS dementia complex using a new computerised cognitive test battery. Arch Clin Neuropsychol. 2006;21:85–94. [PubMed]
14. Winston A, Garvey L, Scotney E, Yerrakalva D, Allsop JM, Thomson EC, et al. Does acute hepatitis C infection affect the central nervous system in HIV-1 infected individuals? J Viral Hepat. 2010;17:419–426. [PubMed]
15. Vellozzi C, Brooks JT, Bush TJ, Conley LJ, Henry K, Carpenter CC, et al. The Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (SUN Study) Am J Epidemiol. 2009;169:642–652. [PubMed]
16. Heaton RK, Clifford DB, Franklin DR, Woods SP, Ake C, Vaida F, et al. HIV-associated Neurocogntive Disorders Persisint in the Era od Poetent Antiretroviral Therapy. Neurology. 2010;75:2087–2096. [PMC free article] [PubMed]
17. Letendre SL, Marquie-Beck J, Ellis RJ, Woods SP, Best B, Clifford DB, et al. The role of cohort studies in drug development: clinical evidence of antiviral activity of serotonin reuptake inhibitors and HMG-CoA reductase inhibitors in the central nervous system. J Neuroimmune Pharmacol. 2007;2:120–127. [PubMed]
18. Army Individual Test Battery. Manual of directions and scoring. Washington, DC: War Department, Adjutant General’s Office; 1994.
19. Gladsjo JA, Schuman CC, Evans JD, Peavy GM, Miller SW, Heaton RK. Norms for letter and category fluency: Demographic corrections for, age, education, and ethnicity. Assessment. 1999;6:147–178. [PubMed]
20. Heaton RK, Grant I, Matthews CG. Comprehensive norms for an expanded Halstead-Reitan Battery: Demographic corrections, research findings, and clinical applications. Odessa, FL: Psychological Assessment Resources, Inc.; 1991.
21. Heaton RK, Taylor MJ, Manly JJ. Demographic effects and use of demographically corrected norms with the WAIS-III and WMS-III. In: Tulsky DS, Heaton RK, Chelune G, Ivnik R, Bornstein RA, Prifitera A, Ledbetter M, editors. Clinical Interpretation of the WAIS-III and WMSIII. San Diego, CA: Academic Press; 2002.
22. The Psychological Corporation. Wechsler Adult Intelligence Scale – Third Edition (WAIS-III) San Antonio, TX: Author; 1997.
23. Diehr MC, Heaton RK, Miller W, Grant I. The Paced Auditory Serial Addition Task (PASAT): Norms for age, education, and ethnicity. Assessment. 1998;5:375–387. [PubMed]
24. Gronwall DM. Paced auditory serial-addition task: A measure of recovery from concussion. Perceptual and Motor Skills. 1977;44:367–373. [PubMed]
25. Kløve H. Clinical neuropsychology. In: Forster FM, editor. The medical clinics of North America. New York: Saunders; 1963.
26. Benedict RH. Brief Visuospatial Memory Test – Revised. Odessa, FL: Psychological Assessment Resources, Inc; 1997.
27. Benedict RHB, Schretlen D, Groninger L, Brandt J. Hopkins Verbal Learning Test –Revised: Normative data and analysis of inter-formand test-retest reliability. The Clinical Neuropsychologist. 1998;12:43–55.
28. Kongs SK, Thompson LL, Iverson GL, Heaton RK. Wisconsin Card Sorting Test – 64 card computerized version. Odessa, FL: Psychological Assessment Resources; 2000.
29. Benton AL, Hamsher K, Sivan AB. Multilingual Aphasia Examination. Iowa City: AJA Associates; 1994.
30. Reitan RM, Wolfson D. The Halstead–Reitan Neuropsychological Test Battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press; 1993.
31. Howell DC. Statistical Methods for psychology. (5th edition) Pacific Grove: Thomas Learning; 2002.
32. Carey CL, Woods SP, Gonzalez R, Conover E, Marcotte TD, Grant I., HNRC Group Predictive validity of global deficit scores in detecting neuropsychological impairment in HIV infection. J Clin Exp Neuropsychol. 2004;26:307–319. [PubMed]
33. Kortte KB, Horner MD, Windham WK. The trail making test, part B: cognitive flexibility or ability to maintain set? Appl Neuropsychol. 2002;9:106–109. [PubMed]
34. Hardy DJ, Hinkin CH. Reaction time performance in adults with HIV/AIDS. J Clin Exp Neuropsychol. 2002;24:912–929. [PubMed]