Overview of the technology
Computer-based technology to collect data requires responders to interact directly with a stand-alone computer through a touch screen or keypad input detection. Like IVR technology, options for computer-based technology include choice of languages, ability to format delivery of questions in various ways, and systems to alert clinicians of abnormal patient reports. Most computerized devices will collate, organize, and analyze according to the researchers' or clinicians' predetermined criteria, if programmed to do so. The capacity for collecting data 24 hours a day/7 days a week is also available, however, this may be limited by the choice of the computerized device. For instance, stand-alone computers in a clinic setting will only be accessible during clinic hours compared to a hand held PDA which, if given to the patient, can be accessible 24 hours a day.
Multiple computer formatting options are available and may be incorporated into computerized collection of self-report data. Usability should be the primary concern, and formatting options, such as user interface and operation, need to be developed in response to the unique needs of the population of interest. Large font sizes and mid-tone colors are generally preferred.
17 Questions may be displayed one question at a time on the screen, or several questions displayed concurrently. One question per screen minimizes scrolling and increases focus on the individual question;
18 however, multiple items per screen reduces completion time and the number of keystrokes.
19 While most computerized questionnaires are visual in nature (displayed on the computer screen), an audio component may be added so that users can see as well as hear the questions and responses. Adding an audio component is particularly helpful for those with low literacy rates, vision difficulties or inexperience with computers.
20 Another available formatting option is to install an alert system to notify people completing the questionnaires of unanswered questions, giving responders an additional opportunity to answer missed questions. This type of alert system potentially reduces the amount of missing data, a particularly appealing option for improving the quality of data collection processes.
21, 22Using computer-based technology for QOL data collection
Health care providers and researchers interested in patient-reported outcomes, such as QOL, are now applying computer technologies to obtain this much-needed information. Like IVR systems, a psychometrically sound paper version of a QOL instrument is typically adapted for use on a computer device. A number of reliable and valid paper-and-pencil QOL questionnaires have been converted for use in a computerized format. The PROQOLID© lists a total of 10 generic and disease-specific instruments.
1 In people with cancer, other computerized QOL instruments have been reported in the literature, such as the EORTC QLQ-C30,
23, 24 University of Washington Quality of Life Questionnaire,
25 and Functional Assessment of Quality of Life (FACT).
26, 27 As more researchers and clinician shift to computerized collection of data, the number of QOL instruments converted from a paper-and-pencil format to a computerized version will continue to grow.
QOL questionnaires may be installed on a variety of computerized devices, such as stationary desktops, notebooks/laptops, PDAs, cellular phones or even subjective markers of wrist actigraphs. Because of the mobility of at least some of these computerized devices, QOL data may be gathered in a variety of settings, including the user’s natural environment. Choosing a device depends on the needs and ability of the users as well as the purposes of QOL data collection. For instance, a stationary desktop may be the most appropriate choice in a clinic setting when QOL information is being collected to monitor the patient’s response to treatment. Conversely, researchers interested in evaluating changes in QOL over time may chose a smaller device, such as a PDA, to gather data over multiple time points in the user’s natural environment.
The ability to capture data in real time in the patient’s natural environment, as opposed to only in the health care setting, is one of the major advantages of using a computerized approach. The subjective event marker () of a wrist actigraph is one example of a computer device that can be used to collect real-time data in the patient’s natural environment. We have successfully used this subjective event marker as a 1-item symptom rating scale to collect repeated self-reported measurements of fatigue throughout the day.
28, 29 Hematopoietic stem cell transplant patients rated their fatigue intensity three times a day over five days before admission to the hospital for the transplant and during the immediate post-transplant period, when they were experiencing the acute toxicities associated with the high-dose chemotherapy. Even though their fatigue increased substantially as they became acutely ill, the majority of patients continued to provide the fatigue ratings. Real-time data collection of quality of life outcomes is particularly attractive in the clinical setting, if the data can be used to guide clinical decision making.
Although data collection in real time can be accomplished using paper versions, improved patient compliance has been documented with computerized compared to paper diaries.
30 Computerized real-time data collection may be preferable over paper diaries due to the ability to date- and time-stamp data entries in an objective manner. Paper diaries may not be a reliable source of information as patients may hoard the diaries, completing them in one sitting on the basis of their memory as opposed to the real-time experience.
The time it takes to complete computerized QOL questionnaires is an important consideration, particularly if it is to be adopted in busy clinical settings. A study of head and neck cancer patients (n = 196) reported that it took an average of 9 minutes to complete three questionnaires (EORTC QLQ-C30, EORTC-HN, and the Hospital Anxiety and Depression Scale). The investigators considered the time to completion adequate, as head and neck health care providers previously reported that QOL questionnaires should take no more than 10 minutes.
31 A more recent study of 342 cancer patients examined the acceptability of asking symptom and quality-of-life questions using the Electronic Self Report Assessment-Cancer program on touch screen, wireless laptops.
18 Patients completed questionnaires in the clinic or examination rooms within an average of 15 minutes. Time and resource constraints have been cited as potential barriers for implementing computerized collection of quality of life data, therefore it is imperative that potential stakeholders, such as patients, nurses and physicians, have adequate opportunity to evaluate new data collection processes.
32One of the more novel developments associated with computer-based technology is the ability to use computer-adaptive testing for QOL assessments. Unlike static paper-and-pencil QOL questionnaires, computerized assessments of QOL permit branching of questions, generally referred to as computer-adaptive testing, so that the types and/or order of questions depend on responses to previous questions. This individualized approach allows respondents to answer only those questions considered appropriate, given their health status and personal situation. Most computer-adaptive testing programs for QOL assessment rely on item response theory to model patient-reported outcomes, such as QOL. While item response theory and computer-adaptive testing are exciting developments, issues such as the need for research into their limitations still remain.
33 An in-depth discussion of item response theory and patient-reported outcomes is beyond the scope of this article. The reader is referred to a series of articles published in the August (2007) supplement of
Quality of Life Research for a more extensive review.
Reliability and validity issues
The mode of administration (paper vs. computer) potentially impacts the psychometric properties of the QOL instrument; therefore, it is vitally important that reliability and validity testing occurs. Several studies compared paper-and-pencil administration of QOL instruments to computer administration.
34–39 One early study evaluated the acceptability, reliability and consistency of a computerized version of the EORTC QLQ-C30.
39 The patients reported that the computerized version of the EORTC was acceptable in terms of providing QOL information to oncology health care providers. Data from the computer administration correlated highly with the paper-and-pencil administration, and internal consistencies for both modes of administration were similar.
Similar results for other QOL instruments have been reported when comparing paper administration to computer administration. In a study of 152 prostate cancer patients examining the reliability, validity, and feasibility of using PDAs compared to paper versions of the International Prostate Symptom Score (IPSS), the Patient Oriented Prostate Cancer Utility Survey (PORPUS), and the International Index of Erectile Function-5 (IIEF-5), similar internal consistencies were demonstrated for both the PDA and paper-and-pencil versions of the instruments.
35 The computerized versions also demonstrated adequate test-retest reliability. Data from both modes of administration were highly correlated. However, the PDA version provided the added advantage of lower missing response rates compared to the paper version. In addition, 82% of patients preferred either PDAs or had no preference for PDA or paper-and-pencil versions.
Not all studies found equivalence when comparing computerized QOL instruments to paper versions; therefore, a mode of administration effect may be present. In a study evaluating the use of the computer versus paper administration of the Center for Epidemiologic Studies Depression Scale (CES-D), higher scores were reported on the paper version when the paper version was administered before the computer version.
34 This suggests that reliability and validity testing of newly converted computerized questionnaires must be undertaken to ensure adequate psychometric properties.
Cost
Initiating computerized QOL data collection can be a costly endeavor, specifically the upfront costs during the implementation phase. Factors that influence overall costs include choice of a computerized device choice, number of devices needed, and amount of data to be stored. Like IVR technology, all of the computerized devices have the capacity to store data, although some devices have limited storage capabilities, requiring more frequent downloading of data and/or shorter collection times. PDAs, generally considered less costly compared to standalone desktops, may be more cost-effective for collecting data long-term in the subject’s natural environment. However, these costs may be offset by the number of PDA devices needed if each patient is given their own device. On the other hand, stand-alone laptops may be more cost-effective for a clinic setting if only a few devices are required to adequately capture the data.