A working group was assembled in 2003 to select a technology platform and content for a patient symptom-reporting portal; to facilitate design and implementation of the system; and to oversee a research program for evaluating the potential clinical and administrative benefits of the system. This group included physicians, nurses, behavioral scientists, and representatives from hospital administration and information services. The group outlined a work flow and timeline based on a set of identified considerations for the creation and scientific evaluation of an electronic PRO platform (). The underlying hypothesis was that patient self-reporting may improve the quality and efficiency of symptom toxicity data collection, and may ultimately benefit the well-being and satisfaction of patients.
Table 1 Questions and Considerations When Developing an Electronic PRO Questionnaire Platform
Toxicity-related symptom burden was identified as the most pertinent information that can be collected from chemotherapy patients via a PRO approach. Such information is frequently the basis for dose modification or supportive medications, or in the case of clinical trials for regulatory reporting. The mandated instrument for this purpose in oncology trials is the National Cancer Institute’s Common Terminology Criteria for Adverse Events (CTCAE). 20,21
The CTCAE is comprised of over 1000 individual items, each graded on a 5-point ordinal scale, including toxicity-related phenomena such as laboratory values (e.g., anemia) as well as symptoms (e.g., nausea). Each numeric grade is associated with a descriptor, such that grade 1 = mild; grade 2 = moderate; grade 3 = severe; grade 4 = disabling; and grade 5 = death. Grades of 3 and greater are reportable during trials, and may be the grounds for management changes. Although the CTCAE was designed for clinician reporting, the rating scale was readily adaptable for patient self-reporting, 22
and a paper correlation study was conducted to assess the relationship between patient and clinician responses to similar items. 23
The CTCAE items developed to test PRO were added to a master item bank from which electronic questionnaires could be constructed based on the characteristics of a patient population of interest.
For the design of the online patient portal, flexibility was desired to integrate advanced questionnaire techniques, including item response theory and computerized adaptive testing. 24–27
An administrator interface was necessary to control patient and clinician users, as well as a patient interface to securely enter symptom data, view past entered data, and dynamically generate longitudinal symptom reports.
Ease of use for those without prior computer experience was considered essential. Based on user acceptance testing involving 20 patients, a touchscreen design with one item per page was adopted (). The longitudinal report was designed to display individual symptom grades for any given day of data entry, as well as temporal relationships to chemotherapy administration. An automated rapid reporting function was included for potentially serious toxicities whereby a designated clinician would be informed electronically in real-time by e-mail whenever a concerning toxicity was self-reported by a patient.
A custom form building application was developed to generate XML-encoded questionnaires using symptoms derived from the item bank, dynamically generate Oracle tables for each questionnaire, and to accommodate integration of item response theory and computerized adaptive testing techniques. To assure security, the data entry interface can be accessed via the Internet outside of the institution’s external firewall, the Web server resides inside an external firewall, the application server resides behind an internal firewall, and all protected health information is stored in a secure database server behind a second internal firewall. Medical record numbers are the only personally identifying information associated with reported symptoms. An automated audit trail records user access to patients’ reported data. We were required by institutional policy to institute a 6-minute time-out for lack of mouse movement and 20-minute absolute system time-out for all users. Touchscreen computers were installed in clinic waiting areas for patients to enter data at appointments.
User acceptance testing with 30 patients found that all participants were able to login, complete, and submit questionnaire forms successfully. Mean duration to complete a questionnaire including 19 items and an open-ended data entry field was 4.7 minutes (median 4.2 minutes). CTCAE items often include lengthy text and take longer per item to read than simple multiple choice items.