PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Psychiatr Serv. Author manuscript; available in PMC 2010 April 12.
Published in final edited form as:
PMCID: PMC2853242
NIHMSID: NIHMS124491

Are We Ready for Computerized Adaptive Testing?

Gibbons and colleagues’ article, “Mental Health Computerized Adaptive Testing,” highlights an emerging methodology of considerable value to psychiatric services. Computerized adaptive testing (CAT) has multiple advantages over standard paper-and-pencil assessment tools (1). CAT uses existing data to streamline and individualize the measurement process. By selecting items of particular relevance to an individual respondent, CAT applications simultaneously reduce the number of needed questions, increase measurement precision, and decrease respondent burden. The applicability of CAT has been demonstrated in the assessment of depression (2, 3), anxiety (4), and quality of life (5), but CAT’s full potential for psychiatric services has yet to be realized.

Streamlining and individualizing diagnostic assessment and symptom monitoring offer many potential benefits to psychiatric services research and practice. Asking fewer questions improves efficiency, freeing time for clinical care or more comprehensive assessments. Computerized assessment can provide clinicians with real-time data that can be put to immediate use. Individualized assessment, that takes an individual’s prior symptoms and functioning into account, can increase the precision of ongoing outcome monitoring and enhance the personalization of disease management programs. The multidimensional approach to assessment that is reflected in the bifactor model can improve the evaluation of comorbid conditions and integrate dimensional and categorical diagnostic paradigms. Creating item banks that aggregate data across populations and settings makes it possible to equate and compare assessments conducted with different sets of items to enrich available evidence about disorder prevalence, illness course and intervention impact.

One can easily envision a variety of useful, innovative applications of CAT technology in clinical and research settings. A computer kiosk in a primary care waiting room could be used to screen patients for common mood and anxiety disorders. After a five minute interaction with a touch screen monitor, each patient’s screening results could be available to their primary care providers by the time they reach the exam room. Longitudinal follow-up assessments in a psychoactive drug trial could be tailored to each patient’s previously reported symptoms and side effects and be administered and recorded by an interviewer using a handheld device communicating with a distant item bank. Similarly, a web-based depression management program could efficiently elicit patient reported assessments individualized to each participant’s prior experiences and convert responses to the metrics of multiple standardized depression measures.

Unfortunately, technology and infrastructure needed to implement the many potentially useful applications of CAT methodology are currently beyond the reach of most clinicians and researchers. Access to basic computer equipment is often limited. Assessments are frequently conducted in settings where computer use is impractical. Existing CAT software is highly technical and there is limited facility for the creation of specialized user interfaces. Importantly, successful implementation of CAT relies on large banks of previously collected and calibrated responses to multiple measures. Banks of items relevant to the range of disorders treated in specialty mental health settings are not widely available, and, when they exist, they are not readily accessible to the majority of clinicians or researchers.

Considerable collaborative effort is needed to develop the infrastructure that would make mental health focused CAT widely available and accessible. The NIH is currently sponsoring a large collaborative endeavor, the Patient-Reported Outcomes Measurement Information System (PROMIS) (6), that provides a promising start toward building a CAT infrastructure for measuring health outcomes. PROMIS includes an item bank and web-based interface for selection, administration and management of both CAT and associated paper-and-pencil outcome measures, all of which are in the public domain. Currently, PROMIS focuses on five general outcome domains applicable across a broad range of chronic diseases. However, the PROMIS project’s strategies and methods could be applied to the range of measures relevant to specialty mental health services. Investment in flexible, accessible technology that would make CAT a reality in psychiatric services could yield multiple benefits for our field.

References

1. Wainer H, Dorans NJ. Computerized adaptive testing : a primer. 2nd ed. xxiii. Mahwah, N.J: Lawrence Erlbaum Associates; 2000. p. 335.
2. Fliege H, Becker J, Walter OB, et al. Development of a computer-adaptive test for depression (D-CAT) Qual Life Res. 2005;14(10):2277–2291. [PubMed]
3. Gardner W, Shear K, Kelleher KJ, et al. Computerized adaptive measurement of depression: a simulation study. BMC Psychiatry. 2004;4:13. [PMC free article] [PubMed]
4. Walter OB, Becker J, Bjorner JB, et al. Development and evaluation of a computer adaptive test for 'Anxiety' (Anxiety-CAT) Qual Life Res. 2007;16 Suppl 1:143–155. [PubMed]
5. Petersen MA, Groenvold M, Aaronson N, et al. Multidimensional computerized adaptive testing of the EORTC QLQ-C30: basic developments and evaluations. Qual Life Res. 2006;15(3):315–329. [PubMed]
6. Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS) Med Care. 2007;45(5) Suppl 1:S22–S31. [PubMed]