|Home | About | Journals | Submit | Contact Us | Français|
HIV-associated neurocognitive disorders (HAND) remain prevalent but challenging to diagnose particularly among non-demented individuals. To determine whether a brief computerized battery correlates with formal neurocognitive testing, we identified 46 HIV-infected persons who had undergone both formal neurocognitive testing and a brief computerized battery. Simple detection tests correlated best with formal neuropsychological testing. By multivariable regression model, 53% of the variance in the composite Global Deficit Score was accounted for by elements from the brief computerized tool (p<0.01). These data confirm previous correlation data with the computerized battery, yet illustrate remaining challenges for neurocognitive screening.
With the development of combination antiretroviral therapy (cART), the prevalence of HIV-associated dementia (HAD) has declined but less severe HIV-associated neurocognitive disorders (HAND) have become prominent . With the transformation of HIV into a chronic medical illness, advanced age and HIV infection may act synergistically to increase the prevalence of HAND . Recent cohort studies have reported the prevalence of HAND to range from 39% to 69% of subjects on cART [3, 4]. The need for brief and psychometrically sound methods to evaluate neurocognitive function is important to identify HAND and limit progressive impairment in one’s capacity to adhere to medical regimens, safely operate a motor vehicle, complete basic activities of daily life, and maintain employment [5–8].
Some available screening tools used to diagnose neurocognitive impairment, such as the International AIDS Dementia Screen or the Mini-Mental Status Exam, are insensitive to early manifestations of functional impairment [9–11]. Conversely, formal neurocognitive testing is time-consuming and requires special training and thus, cannot realistically be used in the typical outpatient setting, where the necessary time, staffing, space, and funding are generally not available. CogState, a commercially available product (http://www.cogstate.com), is a computerized cognitive test battery designed to measure psychomotor performance, attention, memory, and executive functioning: domains frequently impaired in persons with early neurocognitive disorders [12, 13].The battery consists of brief tasks in the form of card games to minimize language and cultural differences. It has previously been validated in persons with HAD  and been used in clinical cohort studies to measure change in neurocognitive function and performance . In this analysis, we sought to determine the correlation between CogState and formal neuropsychological testing to detect neurocognitive impairment in a subset of healthy, autonomous HIV-infected persons who are followed in two cohort studies at the Washington University Outpatient HIV Clinic.
Washington University in St. Louis is a site for two prospective cohort studies evaluating complications of HIV and HIV therapy including cognitive function, one with a validated traditional neurocognitive testing battery and the other with CogState. To determine the utility of the CogState computerized battery as a screening tool, subjects enrolled in both studies were identified to compare the two neurocognitive batteries. Subjects were eligible for participation in this study if they completed an assessment in CHARTER within 6 months of the baseline SUN assessment. Enrollment criteria for the studies have been outlined previously [15, 16]. Both studies were approved by the Washington University IRB and all subjects provided written informed consent.
The Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (The SUN Study) is a CDC-funded multi-site prospective observational cohort monitoring complications of antiretroviral therapy and HIV . At baseline and each six-month visit thereafter, participants are evaluated for neurocognitive ability using CogHealth© CogState Ltd., Melbourne, Australia . The computerized battery requires 12–15 minutes to complete and consists of 6 individual tests: two tests of detection evaluating simple reaction times (DET1 and DET2) assess psychomotor function and speed of processing domains; a test of identification as a choice reaction time (IDN) assesses the domains of visual attention and vigilance; the one back test (ONB) assesses the domains of attention and working memory; the monitoring test is a measure of divided attention (MON) that assesses the domain of attention; and the associate learning test (ASSL) assesses the domain of visual learning and memory. Each test is scored based on time to complete the task (speed) and error rate (accuracy).
The CHARTER study is an NIH-funded multi-site cohort to explore HIV neurological complications in the context of emerging antiviral treatments such as cART [16, 17]. Participants receive comprehensive neuromedical, neurocognitive, and laboratory examinations. The neurocognitive battery performed in CHARTER requires approximately 1 hour to assess the following domains: attention/psychomotor speed (Trailmaking Test Part A, Symbol Digit Test and Symbol Search Test from WAIS-III, Paced Auditory Serial Addition Task); fine motor speed skills (Dominant and Non-dominant Hand Pegboard Test); learning and memory (Brief Visuospatial Memory Test–Revised, Hopkins Verbal Learning Test–Revised, Figure Memory Learning Test and Story Memory Learning test); executive functioning including working memory (Wisconsin Card Sorting Test, Letter-Number Sequence from WAIS-III), fluency (Controlled Oral Word Association Test and Animal Category Test), and set shifting/response inhibition (Trailmaking Test Part B) [18–30].
Data from the computerized battery were evaluated for normality of data distribution; reaction time measures were log10 transformed due to a positive skew of the distribution and accuracy measures were transformed using Arcsine-root transformation . The raw scores from the neurocognitive tests from CHARTER were converted into T scores corrected for demographic data to minimize the impact of education, age, race/ethnicity, and sex. The Global Deficit Score (GDS) is a composite score calculated by converting the T scores from the CHARTER battery into one summary deficit score ranging from 0 (normal) to 5 (severe neurocognitive impairment) . For the evaluation of construct validity, a correlation matrix was created to determine the correlation between the CogState measures and the CHARTER Study standard neuropsychological measures, including the GDS, using Pearson’s correlation coefficient. To determine the potential value of the CogState battery as a screening tool, we performed a stepwise linear regression analysis with GDS (log transformed to approximate a normal distribution) as the dependent variable and the CogState measures as independent variables. The stepwise regression was performed using the default selection method (forward in, backward out) in SAS. The stepwise regression was validated using 1,000 bootstrapping replicates. The significant Cogstate variables from the stepwise regression were used as independent variables in a multivariable regression model with GDS as the dependent variable to determine the model R2.
Forty-six subjects enrolled in both cohort studies were eligible for the present analysis. Table 1 outlines the clinical parameters of the subjects at the time of baseline CogState evaluation between 2004 and 2006. Median CD4 count was 424.5 c/mm3 with 74% of the cohort on cART and 61% with HIV viral load < 400 cp/mL. The median GDS, as calculated from the CHARTER study, for the cohort was 0.47 (range 0.00–2.79), with 24 subjects (52%) having normal function (GDS<0.5) and 22 (48%) having mild to moderate impairment (GDS≥0.5).
The correlation matrix showing the Pearson product-moment correlation coefficient between each COGSTATE index and the dependent variables from the CHARTER clinical exam, including GDS is shown in Table 2. The speed measures for both simple detection tests (DET1, DET2) and the identification task (IDN), a more complex reaction time task, correlated with the GDS and had the highest levels of correlation across the tests performed in the CHARTER battery. The accuracy components of the CogState battery generally correlated poorly with the individual neurocognitive tests in the CHARTER battery. The accuracy measures for the complex tasks of One-Back Memory (ONB) and the Measure of Divided Attention (MON) were the only accuracy measures that correlated with the GDS derived from the CHARTER testing.
The stepwise regression analysis identified the following potential independent correlates with GDS: the accuracy and speed of the two simple detection tests (DET1lmn, DET2acc) and 3 more complex tasks: the accuracy of the associate learning (ASSLacc), the accuracy of monitoring tasks (MONacc), and the accuracy of the One Back test (ONBacc). A regression model using GDS as the dependent variable and these measures as independent variables yielded a model R2 of 0.53 (p<0.0001) indicating that approximately 53% of the variance in the GDS is explained by these five CogState variables. In the validation analyses using results from the stepwise regression analysis of 1,000 bootstrap replicates MONacc appeared in 96% of the replicates, DET1lmn appeared in 76% of the replicates, DET2acc appeared in 71% of the replicates, ASSLacc appeared in 57% of the replicates and ONBacc appeared in 51% of the replicates. A regression model using GDS as the dependent variable and MONacc, DETlmn and DET2acc (the three variables that appear in more than 70% of the replicates) yields a model R2 of 0.39 (p=0.0002).
In this study, we compared a brief, self-administered computerized screening battery with formal neurocognitive assessment. With the exception of reaction tests that evaluate functional speed, individual tests in the computerized battery correlated poorly with formal neurocognitive testing. Cognitive slowing is a prominent feature of HAND, and thus, these results are consistent with previous comparisons of the CogState battery to formal neurocognitive functioning although the associations are not as robust for less severe cognitive impairment .
While these results illustrate that a single brief screening test may be insensitive to identifying people with neurocognitive impairment, particularly those with mild impairment, the modeling of several parameters from the CogState battery against the Global Deficit Score highlights that there is utility to a brief computerized neurocognitive screening tool. Future prospective studies are needed to determine the sensitivity of the CogState battery to identify HAND. If it proves to be a sensitive tool with established valid cutpoints, this battery could be used to identify persons with mild cognitive impairment. However, in the absence of cutpoints that denote clinical significance, the battery will not achieve optimal clinical utility.
We recognize the limitations of this analysis. It consists of a small group of individuals and lacks a control group. No persons with advanced cognitive imparirment were included. Another limitation to these findings was the lack of correlation between the Associate Memory test and the measures of learning and memory. The CogState battery evaluated here was insensitive to these measures and thus may impact the ability of Cogstate to identify persons with impaired learning ability. Additionally, there was a lack of correlation between CogState indices and the Trailmaking Part B test, which is widely used to reflect executive function with important implications regarding one’s ability to perform activities of daily living independently . Our data fail to confirm the previous work by Cysique et al, in which the strongest correlations were with the Trail Making tests. However, their work focused on persons with HAD. While the Trailmaking Part B test does not necessarily capture all of the components of executive control important for daily living, our findings suggest that CogState may not provide critical information related to early decline in functional independence, though it should be noted that executive function is a heterogeneous construct.
In summary, we found that a compilation of the tests from a brief computerized screening tool for neurocognitive function was correlated to traditional neurocognitive testing among HIV-infected persons. These findings confirm previous reports of correlation between brief computerized CogState battery and standard neuropsychological examination , especially for identifying cognitive slowing, a central feature of HIV-associated neurocognitive disorders, although there were differences regarding different domains potentially related to differences in the severity of impairment in the studies . Nevertheless, additional research is needed to fully evaluate the utility of this battery.
The authors would like to express our appreciation to all the SUN study and CHARTER participants. No authors have conflicts of interest regarding this research.
The SUN Study Investigators are: John T. Brooks, Pragna Patel, Lois Conley, and Tim Bush, Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, CDC, Atlanta, Georgia; Kathleen Wood, Rose Baker, and Cheryl Akridge, Cerner Corporation, Vienna, Virginia; John Hammer, Tara Kennedy, Barbara Widick and Billie Thomas, Denver Infectious Disease Consultants, Inc., Denver, Colorado; Ken Lichtenstein and Cheryl Stewart, National Jewish Medical and Research Center, Denver, Colorado; Keith Henry, Jason Baker, Rachel Prosser, Edie Gunderson, Miki Olson, and John Hall, Hennepin County Medical Center, Minneapolis, Minnesota; Frank Rhame, Mark Olson, and Eve Austad, Abbott-Northwestern Hospital, Minneapolis, Minnesota; Hal Martin, Meaghan Morton, and Cheri Murch, Park-Nicollet Institute, Minneapolis, Minnesota; Charles Carpenter, Susan Cu-Uvin, Kenneth Mayer, Erna Milunka Kojic, Jennifer Florczyk, Sara Metzler, and Patricia D’Aiello, The Miriam Hospital, Providence, Rhode Island; and E. Turner Overton, Don Connor, Lisa Kessels, Mariea Snell, Sara Hubert, Dorothea Dedeaux-Turner, and Dave Coughlan, Washington University School of Medicine, St. Louis, Missouri.
The CNS HIV Anti-Retroviral Therapy Effects Research (CHARTER) is supported by award N01 MH22005 from the National Institutes of Health. The CNS HIV Anti-Retroviral Therapy Effects Research (CHARTER) group is affiliated with the Johns Hopkins University, Mount Sinai School of Medicine, University of California, San Diego, University of Texas, Galveston, University of Washington, Seattle, Washington University, St. Louis and is headquartered at the University of California, San Diego and includes: Director: Igor Grant, M.D.; Co-Directors: J. Allen McCutchan, M.D., Ronald J. Ellis, M.D., Ph.D., Thomas D. Marcotte, Ph.D.; Center Manager: Donald Franklin, Jr.; Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Terry Alexander, R.N.; Laboratory, Pharmacology and Immunology Component: Scott Letendre, M.D. (P.I.), Edmund Capparelli, Pharm.D.; Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), J. Hampton Atkinson, M.D., Steven Paul Woods, Psy.D., Matthew Dawson; Virology Component: Joseph K. Wong, M.D. (P.I.); Imaging Component: Christine Fennema-Notestine, Ph.D. (Co-P.I.), Michael J. Taylor, Ph.D. (Co-P.I.), Rebecca Theilmann, Ph.D.; Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman,; Statistics Unit: Ian Abramson, Ph.D. (P.I.), Florin Vaida, Ph.D.; Protocol Coordinating Component: Thomas D. Marcotte, Ph.D. (P.I.), Rodney von Jaeger, M.P.H.; Johns Hopkins University Site: Justin McArthur (P.I.), Mary Smith; Mount Sinai School of Medicine Site: Susan Morgello, M.D. (Co-P.I.) and David Simpson, M.D. (Co-P.I.), Letty Mintz, N.P.; University of California, San Diego Site: J. Allen McCutchan, M.D. (P.I.), Will Toperoff, N.P..; University of Washington, Seattle Site: Ann Collier, M.D. (Co-P.I.) and Christina Marra, M.D. (Co-P.I.), Trudy Jones, M.N., A.R.N.P.; University of Texas, Galveston Site: Benjamin Gelman, M.D., Ph.D. (P.I.), Eleanor Head, R.N., B.S.N.; and Washington University, St. Louis Site: David Clifford, M.D. (P.I.), Muhammad Al-Lozi, M.D., Mengesha Teshome, M.D.
DISCLAIMER: The findings and conclusions from this review are those of the authors and do not necessarily represent the views of the United States Government or the Centers for Disease Control and Prevention.