|Home | About | Journals | Submit | Contact Us | Français|
We wanted to test the psychometric reliability and validity of self-reported information on psychological and functional status gathered by computer in a sample of primary care outpatients. Persons aged 65 years and older visiting a primary care medical practice in Baltimore (n = 240) were approached. Complete baseline data were obtained for 54 patients and 34 patients completed 1-week retest follow-up. Standard instruments were administered by computer and also given as paper and pencil tests. Test–retest reliability estimates were calculated and comparisons across mode of administration were made. Separately, an interviewer administered a questionnaire to gauge patient attitudes and feelings after using the computer. Most participants (72%) reported no previous computer use. Nevertheless, inter-method reliability of the GDS15 at baseline (0.719, n = 47), intra-method reliability of the computer in time (0.797, n = 31), inter-method reliability of the CESDR20 at baseline (0.740, n = 53), and the correlation between the CESDR20 computer version at baseline and follow-up (0.849, n = 34) were all excellent. The inter-method reliability of the CESDR20 at follow-up (0.615, n = 37) was lower but still acceptable. Although 28% were anxious prior to using the computer testing system, that percent decreased to 19% while using the system. The efficiency and reliability in comparison to the paper instruments were good or better. Even though most participants had not ever used a computer prior to participating in the study, they had generally favorable attitudes toward the use of computers, and also reported having favorable experience with the computer testing system.
The primary care physician occupies a strategic position to be a case-finder and coordinator for the care of elderly persons. Nevertheless primary care physicians often do not have the time or means for carrying out a complete geriatric assessment, and an effective computer-assisted assessment tool might facilitate the ability to evaluate patients and assess change. The development of practical assessment tools is important because certain aspects of geriatric assessment, especially depression, are not commonly addressed in primary care settings in any standardized way.
Several factors make primary health care services relevant as a location for automated assessment for mental health care of older persons. First, older persons usually make several visits per year to their doctor. Second, chronic medical illnesses (e.g. cardiovascular disease, arthritis, and diabetes mellitus) often co-occur with depression among older persons presenting in primary care, complicating the recognition and treatment of major depression and other mental disturbances. Third, the majority of patients with depression may be treatable by the primary care physician, and innovative methods can facilitate the treatment process. Although primary care physicians have a low recognition rate of alcohol, drug abuse, and mental health disorders, and are pessimistic about patients' outcomes even with treatment, primary care physicians prescribe the majority of tricyclic and anxiolytic medications . Not only does depression appear to result in over-utilization of health care resources, but depression may worsen the outcome of other medical conditions such as myocardial infarction [2,3].
Recent research suggests that identification and treatment of depressive disorders in primary care are more complex than previously assumed. Competing demands vie for the attention of the clinician, with insufficient time to address all demands . The relatively poor detection and treatment of depression in the elderly is not due to lack of concern on the part of the physician but rather lack of time resources. Integration of efficient computerized systems for detection of depression into the practice ecosystem has considerable potential to effectively provide more time allocated to patient assessment, without requiring as much staff time . We developed a computer-assisted assessment device, which could be used to efficiently screen elderly outpatients for changes in psychological and functional status. The purpose of this study was to assess the reliability of our instrument, comparing computer administration to paper and pencil among older primary care patients.
The CESD-R is a revised version of the Center for Epidemiologic Studies of Depression Scale (CESD), a tool for self-report of depressive symptoms. It was revised by William W. Eaton, Professor in the Department of Mental Hygiene in the Johns Hopkins University School of Public Health . The MDLogix computerized version of the CESD-R is a Windows program for self-report to determine risk for depression (Fig. 1). Subjects can take either the 20- or 35-item versions, and each version includes available digital audio of the questions and responses for persons with poor literacy or visual impairment. The self-report data can be stored on the local computer or transmitted over the Internet and stored on a server, for instance at a physician's office. The diagnostic and total and sub-scale score results and clinical recommendation are presented to the subjects each time after they complete the questionnaire. This report is printable, so it could be taken to a health care provider for further evaluation and treatment if indicated. Features of the MDLogix computerized CESD-R include: optional digital audio, 20-item short form, 35-item long form, DSM-IV-based diagnostic scoring, dimensional depression sub-scale scores, and summary printable graphic chart and clinical recommendations. The CESD-R client program is free, and can be downloaded from the MDLogix website (http://www.mdlogix.com).
We implemented the computer application as a client-server system, using Microsoft NT Server 4.0 and SQL Server 7.0 running on a server and two client computers. Borland Delphi Client-Server 4.0 was used for programming the client computers. To limit difficulties caused by poor vision or literacy we developed and linked digitized audio files for all the text of the assessment interface. The voice reading this text was that of an older woman. The participant could turn the audio off or on as desired. When the audio is on, the participant can touch any panel on the interface to replay the audio for that text. Headphones with volume control were used to maintain privacy. To facilitate use of the assessment by older persons we used touch screens for test items and response choices.
The study was carried out in a convenience sample of elderly of the Center for Primary Care at Good Samaritan Hospital in Baltimore. The practice has three physicians. The patient population is comprised of approximately 40% persons over the age of 65 years, 60% African–American, and 60% female. To be eligible for the study, participants needed to be aged 65 years and older and to have visited the doctor for outpatient care during the study period. Upon the recommendation of the clinic staff some patients were not approached for enrollment if the staff thought the patient would be unlikely to use a computer due to extreme fragility, poor cognitive status, or other factors. About 240 patients were approached for recruitment into the study.
The instruments employed were the Center for Epidemiologic Studies Depression Scale, Revised (CESDR20, 20 items ; Geriatric Depression Scale (GDS15, 15 items) ; activities of daily living (ADL, four items) ; and instrumental activities of daily living (IADL, six items) . All instruments were implemented in the computer system but were also assembled in paper form. An interviewer administered a separate questionnaire to assess patient experiences, attitudes, and comments about the computer.
The inter-method variation of the means of the results of all instruments at baseline and at 1-week retest follow-up were measured. Reliability comparisons were made by calculating the Pearson correlation coefficients measuring the correlation between all bivariate depression and function assessment measures. These included the test–retest reliability for both modes (computer and paper and pencil forms), the concurrent validity at each time point across modes of administration, and the predictive validity for computer at baseline and for paper at baseline. We also report descriptive analysis of the experiences, attitudes, and comments about the computer system from the users.
Out of the sample of 240 persons approached a total of 68 patients agreed to participate. We were unable to systematically ask the patients who refused about their reasons for refusal. We did, however, conversationally inquire, and also listened to their statements and concerns. Our impression was that most of the patients refusing did not want to do additional work that was not part of the clinic routine. One concern that many patients expressed was worry that they would miss seeing their doctor or that the doctors' time would somehow be reduced. Complete baseline data, both paper and computer assessments, were obtained on 54 patients, and 34 of these patients completed 1-week retest follow-up assessment. Of 54 participants, 35 were women and 42 self-identified their ethnicity as African–American.
Descriptive statistics of the computer and paper versions of CESDR20, GDS15, ADL, and IADL are provided in Table 1. The CESDR20 inter-method variation of the means at baseline and at 1-week retest follow-up were less than 10%. In other words, the computer-administered baseline CESDR20 mean was within 10% of the paper-administered baseline CESDR20. Similarly, the GDS15, ADL, and IADL inter-method variation of the means at baseline and at 1-week retest were less than 5%.
Table 2 provides the Pearson correlation coefficients for the computer and paper versions of the GDS15 and CESDR20 at baseline and follow-up assessments. The inter-method reliability of the GDS15 at baseline was good (0.719, n = 47), as was the intra-method reliability of the computer in time (0.697, n = 31). The correlation between the computer and paper versions of the CESDR20 at baseline was good (0.740, n = 53), and the correlation between the CESDR20 computer version at baseline and follow-up was high (0.849, n = 34). The correlation between the computer and paper versions of the CESDR20 at follow-up was not as high (0.605, n = 34), but still acceptable. The correlation between the baseline and follow-up administration of the paper CESDR20 was lower (0.615, n = 37) than that of the computerized version. Although the correlation of CESDR20 across time when administered by paper was still good, the computer CESDR20 appeared somewhat more stable over time than the paper version.
Few participants had experience with computers: 72% reported never having used a computer, and 90% said they did not have a computer in their home. Only 6% had ever used the Internet, and 94% had never used e-mail. The participants as a group did not express anxiety or frustration with computers in general (Table 3). Although 28% were anxious prior to using the computer testing system, that percent decreased to 19% after using the system. About half the patients (53%) said their feelings changed during the study, and comments suggest changes were for the better. Some participants were anxious and frustrated by the thought of using computers, but for most persons anxiety diminished with use.
We were encouraged by the results of the use of the software system for assessment of depression in older persons. The efficiency and reliability of computer assessment in comparison to the paper instruments were as good or better. Even though most patients had not ever used a computer prior to participating in the study, patients had generally favorable attitudes toward the use of computers, and also reported having a favorable experience with the computer testing system.
Before placing our results in the context of other investigations of computer applications in mental health care, the limitations of our work demand comment. First, the sample for this preliminary study was small. Second, we did not systematically ask about reasons for refusal to participate. As a result the non-participants are not well characterized. However, our impression was that most of the patients refusing did not want to do additional work that was not part of the clinic routine and were not refusing specifically to avoid using computers. Third, depression and functional assessments in this study were based on self-report, so there is the potential for sources of error associated with imperfect recall and response bias (e.g. socially desirable responding).
Experience with computers is not widespread among older people. Five percent of people older than 65 years use computers, compared with about 45% of young and middle-aged adults . A recent study in primary care suggests 54.3% of patients have e-mail access . The number of older patients with computer experience will increase in coming decades. Education level, previous experience with computers, and attitudes toward computers are related to successful patient–computer interaction . Although most participants in our study had no previous computer experience patients generally had favorable attitudes toward computers. This finding supports other studies suggesting that electronic data-capture methods were preferred over traditional paper-and-pen methods by elderly volunteers and patients [12–14]. In our study computer anxiety reported by some patients before the assessment appeared to decrease with experience with the system. Computer administration resulted in scores that were highly correlated to paper and pencil testing and would seem to be a viable alternative. Computer-assisted interviews also resulted in highly stable results of questionnaires over a 1-week follow-up interval.
The care of depressive disorders may be our most useful barometer for the quality of mental health services in the primary care sector . Older adults with depression commonly present to primary care physicians in the context of physical illness and not to specialists in mental health . Physical illness and depression contribute to overall disability [17,18], but physicians may miss depression because of the competing demands of care . Patients with depression may have medical illness or somatic complaints and may deny depression [19,20], making dealing with depression even more difficult. Among patients whom the physician has identified and is managing with depression, availability of a computerized assessment would facilitate follow-up care because change in depressive symptoms could be documented, complementing the clinical assessment. The computer-assisted interview may be viewed as any other clinical test for which we would send the patient for evaluation and follow-up.
Automated methods for collection of mental health data are needed that are easy, quick, inexpensive, and reliable and that can be integrated into practice routine. Computer-administered rating scales offer a reliable, inexpensive, accessible and time-efficient means of assessing psychiatric symptoms. Patients often find it easier to disclose sensitive information to a computer system, despite knowing that humans will see their answers, particularly regarding sexual behavior , HIV risk factors , and suicidal ideas [23,24]. Computer-assisted interviews for evaluation of depression, function and cognition can complement clinical activities and free clinicians to do what they do best—form a therapeutic relationship with patients.
This work was supported by SNUF and FMW, Katholieke Universiteit Nijmegen, The Netherlands and an NIMH Small Business Innovation Research grant to Dr Tien (R43 MH56315). Data analysis was supported by an American Academy of Family Physicians Advanced Research Training Grant (Dr Bogner).