|Home | About | Journals | Submit | Contact Us | Français|
To determine whether e/Tablets (wireless tablet computers used in community oncology clinics to collect review of systems information at point of care) are feasible, acceptable, and valid for collecting research-quality data in academic oncology.
Primary/Duke Breast Cancer Clinic.
Pilot study enrolling sample of 66 breast cancer patients.
Data were collected using paper- and e/Tablet-based surveys: Functional Assessment of Cancer Therapy General, Functional Assessment of Cancer Therapy-Breast, MD Anderson Symptom Inventory, Functional Assessment of Chronic Illness Therapy (FACIT), Self-Efficacy; and two questionnaires: feasibility, satisfaction.
Patients supported e/Tablets as: easy to read (94 percent), easy to respond to (98 percent), comfortable weight (87 percent). Generally, electronic responses validly reflected responses provided by standard paper data collection on nearly all subscales tested.
e/Tablets offer a valid, feasible, acceptable method for collecting research-quality, patient-reported outcomes data in outpatient academic oncology.
Patient-reported outcomes (PROs) have increasingly gained acceptance as important and valid measures of symptoms, experiences, and health-related quality of life (HRQOL), and thus as critical metrics for evaluating the value of health care services to patients (Ganz 1994; Cella et al. 1997; Coates, Porzsolt, and Osoba 1997; Dancey et al. 1997). Major policy making entities which have emphasized the importance of incorporating PROs into cancer research and policy formation (Lipscomb, Gotay, and Snyder 2007) include the National Cancer Institute (2008), American Cancer Society (2007), U.S. Food and Drug Administration ( Johnson, Williams, and Pazdur 2003; U.S. Department of Health and Human Services 2007), U.S. Centers for Medicare & Medicaid Services (2004), and National Institutes of Health (2007). These agencies' interest in PROs reflects a growing national recognition that traditional medical outcomes (i.e., survival, disease progression) do not fully capture the patient's experience of health, an acknowledgement that patients' self-perceived well-being is an important outcome, and a new definition of “value” of health care which includes improvement in subjective outcomes of importance to patients.
The two principal, if not only, methods of gathering PROs before the advent of electronic methodologies are the paper survey and clinician interview, both labor and time-intensive methodologies. In order for PROs to be routinely integrated into clinical practice, PRO data collection methods must be efficient, that is, they must be easy, rapid, convenient, inexpensive, reliable, and clinically feasible (Cella 1995). Newly introduced technologies have revolutionized the data collection methods available to researchers and offer new, more efficient tools for clinical management.
Studies in diverse populations have found that technology-based methods of PRO data capture are well received by patients, if not preferred over paper-based instruments (Skinner and Allen 1983; O'Connor, Hallam, and Hinchcliffe 1989; Drummond et al. 1995; Lewis et al. 1996; Yarnold et al. 1996; Velikova et al. 1999; Ruland et al. 2003). A recent study also found that regular, repeated, PRO collection and feedback to physicians, in addition to facilitating physician–patient communication, have a positive impact on HRQOL and emotional functioning (Velikova et al. 2004). A recent meta-analysis of 65 studies directly assessed the equivalence of computer versus paper versions of PROs used in clinical trials and found that computer and paper-administered PROs are equivalent data collection methodologies, with very small mean differences that were neither statistically or clinically significant (Gwaltney, Shields, and Shiffman 2008). These findings directly compare the differences between assessments moved from paper to electronic versions, and cannot be generalized to all forms of electronic administration of PROs (Gwaltney et al. 2008).
The e/Tablet technology utilized in the present study is a validated, widely implemented, HIPAA-compliant wireless tablet computer which patients use at the point of care to complete PRO surveys (Figure 1). The e/Tablet feeds PROs into a database system, which longitudinally warehouses data locally and/or across multiple sites. A network of >110 community oncology practices in the United States currently uses e/Tablets to administer a validated, medical review of systems (ROS) instrument, the Patient Care Monitor™ (PCM), which collects symptom, QOL, and performance status information (Fortner et al. 2003). Doctors from participating practices use these PROs to enhance clinical care; the ongoing use of e/Tablets by these practices suggests positive reception by community oncologists.
To date, the validity, acceptability, and clinical benefit of the use of the e/Tablets technology have not been investigated in the academic oncology setting. It cannot be assumed that an instrument validated in one clinical setting will perform equally well in another, nor that a system validated for one instrument (e.g., PCM) will prove similarly capable of delivering other instruments (e.g., research surveys). It is therefore important to study the validity and acceptability of e/Tablets in academic oncology to confirm previous findings in community settings and other patient populations.
The primary purpose of this pilot study was to determine whether e/Tablets, a PRO data-collection method successfully implemented in community oncology, can also be used in academic medicine to collect research-quality data while integrating clinical functions with research, thereby potentially improving efficiency and quality. Its principal objectives were:
This pilot feasibility study enrolled 66 patients from the Duke Breast Cancer Clinic.
Eligible participants were: (1) adults diagnosed with breast cancer; (2) expecting at least four clinical visits over the ensuing 6 months; and (3) able to speak and read English.
e/Tablets were programmed with a standard ROS survey, the PCM. The PCM asks female patients 86 questions (80 for men) reflecting symptoms plus psychological distress and performance status. The PCM has seven subscales: General Physical Symptoms, Treatment Side Effects, Distress, Despair, Impaired Performance, Impaired Ambulation, and QOL. PCM subscale scores are normalized into T scores.
e/Tablet software was modified to deliver multiple PRO surveys. Individual survey instruments are selected using a modular survey selection screen, and electronically concatenated into a single stream of questions that are presented to the patient with appropriate instructions corresponding to the individual instrument. The Functional Assessment of Cancer Therapy General scale (FACT-G) (Cella et al. 1993), Functional Assessment of Cancer Therapy-Breast scale (FACT-B; disease-specific subscale that combines with the FACT-G) (Brady et al. 1997), MD Anderson Symptom Inventory (MDASI) (Cleeland, Mendoza, and Wang 2000), Functional Assessment of Chronic Illness Therapy-Fatigue (FACIT-Fatigue; symptom-specific subscale that combines with the FACT-G) (Yellen et al. 1997), and Self-Efficacy (Lorig et al. 1989) scales were selected because they: (a) capture data of greatest importance to comprehensive, patient-centered, oncology care; and (b) represent well-recognized, frequently used, PRO instruments. Approval was obtained from the author of each survey. An eight-item feasibility and a 19-item satisfaction with care questionnaire were also programmed onto e/Tablets. Description of completion of electronic and paper instruments appears under “Procedures.”
A wireless frequency site survey was conducted in the Duke University Medical Center (Durham, NC) cancer clinics. Duke has a secure WEP-enabled WiFi Protected Access system, across which data travel wirelessly from e/Tablets to a fully dedicated local server. The vendor's software was reconciled with Duke's wireless system.
An ANSI-compliant database, which uses Microsoft SQL Server (version 08.00.0760, Seattle, WA), warehouses the Duke e/Tablet data behind the Duke firewall. The database is populated by information entered into the e/Tablets by patients, and is backed up nightly. Data can be viewed or extracted using a Microsoft Access 2003 front-end (Seattle, WA), or extracted and downloaded for analysis using a standard statistical package (e.g., SAS; SAS, Cary, NC).
Attending physicians or mid-level providers suggested the study to patients during clinic visits. The research nurse approached potential participants in the clinic waiting area, described the e/Tablets and study purpose, and obtained informed consent. A “Patient Flow Protocol” for handing out e/Tablets, identifying appropriate instruments for each participant at each timepoint, collecting e/Tablets, and getting reports to clinicians was approved by all senior clinical staff. A “Clinical Response Thresholds Protocol” was developed to standardize response to urgent symptom scores.
At each of four study visits within 6 months, participants initially completed a paper version of one of four instruments (FACT-B, MDASI, FACIT-Fatigue, Self-Efficacy), and then completed electronic versions of PCM, FACT-B, MDASI, FACIT-Fatigue, and Self-Efficacy surveys. For each patient, the visit during which each of these paper instruments were administered was randomly ordered. Participants completed an electronically administered demographics questionnaire at the first visit. An eight-item satisfaction with e/Tablet questionnaire and a 19-item satisfaction with care questionnaire were also completed via e/Tablet at each visit.
Analyses included all study surveys and questionnaires completed between March 19, 2006 and October 31, 2006.
All analyses were conducted using SAS 9.1. Results were summarized using descriptive statistics. Cronbach's α's were calculated to ensure internal consistency of the subscales. Mean subscale scores were calculated using published algorithms, visually inspected, and compared with published norms. Paper and electronic mean subscale scores were compared using paired Student's t-tests and verified with Sign tests. A two-sided p<.05 constituted statistical significance.
The main hypothesis tested, and on which the sample size estimate was calculated, was that there would no significant difference between subscale scores derived from data collected via paper and those same scores derived from data collected electronically via e/Tablets. If the true mean difference between paper and e/Tablet scores was greater than half the standard deviation of the mean difference, then e/Tablet data collection would not be considered equally accurate as paper data collection. The statistical hypothesis tested was as follows: H0; μd=0, versus H1; μd≠0, where μd is the unknown mean difference of the subscale score calculated from data collected via paper and e/Tablets. The study's sample size goal of 60 patients allowed the test for each subscale to have the type I error rate and power of 0.05 and 0.968, respectively, without accounting for multiple testing.
This study was funded through an Outcomes Research service agreement with Pfizer Inc. and approved by the Duke University Health System Institutional Review Board.
Of 73 referred and eligible patients, 66 provided informed consent. Baseline characteristics for participants (Table 1) were: non-Caucasian, 22 percent; ≥60 years of age, 38 percent; no college degree, 49 percent. Of the seven eligible women who declined to participate, three (43 percent) were non-Caucasian and mean age was 59 (SD 6; range 52–68).
An initial indication of e/Tablets' acceptability lies in the fact that 90 percent of screened, eligible patients enrolled in the study.
Feasibility and acceptability of e/Tablets were explored through analysis of participants' responses to an eight-item questionnaire (Table 2) delivered via e/Tablet at each of four study visits. Responses to this questionnaire have previously been presented (Abernethy et al. 2007). Highlights were that:
To evaluate the utility of e/Tablets for research purposes, subscale scores were calculated for each of the surveys collected in both electronic and paper versions (FACT-G, FACT-B, MDASI, FACIT-Fatigue, Self-Efficacy). Cronbach's α coefficients were calculated to verify internal consistency (Table 3); all coefficients were acceptable and generally better than those previously published. Electronic scores were matched to paper scores for the timepoint at which the patient completed both the paper and the electronic version of the survey in question. Differences between electronic and paper scores were compared using raw scores and paired Student's t-tests; results were verified with the Sign test (Table 3). Paper and electronic versions of the FACT-B, MDASI, FACIT-Fatigue, and three of four FACT-G subscales were not significantly different. FACT-G Social Well Being and all Self-Efficacy subscales were significantly dissimilar.
This pilot study demonstrated that: (1) e/Tablets can collect data of comparable validity to data collected by well-recognized paper-based PRO surveys; and (2) e/Tablets are feasible and acceptable to patients in an academic oncology clinic. Data yielded by paper and electronic format of certain subscales—specifically, the FACT-G Social Well-Being subscale and all Self-Efficacy subscales—were not statistically equivalent and thus we cannot recommend interchangeability of these two data collection methods for these subscales. Potential reasons for paper versus electronic discrepancies on these subscales are explored elsewhere (Abernethy et al. 2007).
e/Tablets functionally integrate the collection of clinical data and research data. They thus may exert a positive process impact; future studies might explore whether e/Tablets (or similar technology-based systems) enhance practice efficiency, support improved patient/provider communication, more expediently identify patient needs, and promote patient-centered care. In community oncology, the e/Tablet system has reduced dictation times, resulted in more accurate charting and billing, facilitated Joint Commission-compliant patient education, and enabled real-time quality monitoring (B. F., Fortner personal communication, August 2007). In academia, demonstration that e/Tablets can collect and longitudinally warehouse research-quality PROs, which can be efficiently matched with clinical trials, will be a strong motivator for their adoption. Linking clinical and research data with administrative data will cement their appeal to institutional leaders and administrators. Administrative data have been linked with e/Tablet data in community oncology; linkages at Duke are underway.
Patients today, through models such as consumer-directed health plans, are being asked to play an increasing role in medical decision making (Hibbard et al. 2004). Patients' effectiveness in this role hinges upon “patient activation,” i.e. patients' acquisition of the skills, knowledge, and motivation to participate in their own health care (von Korff et al. 1997). This pilot study suggests that e/Tablets may promote patient activation. Participants reported that e/Tablets helped them remember symptoms, and encouraged them to discuss medical issues with their doctor. At least one-third of participants felt that the e/Tablet enhanced the patient/physician dialogue. A study designed to more definitively evaluated e/Tablets' impact on patient activation might shed light on the technology's ability to facilitate patient activation. Current directions in software development, such as creation of Spanish and low-literacy versions, raise hopes that e/Tablets might help engage patients who are otherwise challenging to activate.
The data collection, integration, and storage capabilities of e/Tablets, and of similar validated technologies, could help advance initiatives designed to use PROs to enhance quality and value in health care. Nationally, the Patient Reported Outcomes Measures Information System (PROMIS), part of the NIH Roadmap, Re-engineering Clinical Research Initiative, is developing sophisticated methods of PRO collection, coordination, warehousing, and analysis. Internationally, countries such as Australia (Jackson 2007), England (Walley 2007), and Canada (Hailey 2007) are using health technology assessment (HTA) to evaluate the effectiveness, costs, and broad impact of medical interventions; evaluations include patient-centered metrics such as quality-adjusted life years and health state.
e/Tablets represent a possible mechanism for gathering, storing, and making available for analysis the PRO data that will become part of large-scale health services research initiatives, and that will support the national effort to optimize efficiency and value in health care. Ultimately, the success of initiatives such as PROMIS and HTA may hinge on their ability to integrate data collected through many new technology-based systems into large-scale repositories that leverage the strengths of local clinical/research data collection and management systems in service of quality, integrity, efficiency, and value goals.
Several limitations of this e/Tablets study warrant noting. This was a pilot study conducted at a single institution; it enrolled a small convenience sample; participants comprised only women; all participants had the same disease.
The authors thank all of the study participants who generously donated their time and personal information in an effort to improve the care of others. We are also deeply indebted to the clinical teams in the Duke Breast Cancer Clinic, without whom this study would not have been possible.
Disclosures: This pilot study was funded through an Outcomes Research service agreement with Pfizer, Inc. Pfizer does not have access to the individual data collected through this study. Supportive Oncology Services (SOS), Inc., as vendor, provided the hardware (i.e., the e/Tablets), which was not funded through the Pfizer agreement. Duke University Medical Center provided the wireless system and associated technical support.
Roles of contributors on this project are as follows: Amy Abernethy served as Principal Investigator and was involved in all aspects of the project. James Herndon served as lead statistician and contributed significantly to the statistical design and analysis of data. Meenal Patwardhan, Heather Shaw, H. Kim Lyerly, and Kevin Weinfurt contributed expert guidance, review, and content input during the study itself, the analysis of data, and the final presentation of results. Jane Wheeler served as lead writer who helped to frame the analysis and discussion, and contributed significantly to the content of the final manuscript. Krista Rowe served as Clinical Research Manager who oversaw all research nurse activity in conduct of the study. Laura Criscione provided critical administrative and data support.
Dr. Abernethy has a nominal consulting arrangement with SOS, Inc., developed after this study was completed; this consulting arrangement was declared to and is monitored by the Duke Conflict of Interest Committee. PROMIS is an NIH Roadmap Initiative. Dr. Weinfurt is Principal Investigator of the Duke PROMIS primary research site with a cancer supplement grant (1 U01AR052186-01); Dr. Abernethy is a co-investigator on the Duke PROMIS cancer supplement grant. No other authors or contributors have conflicts of interest to report.
Additional supporting information may be found in the online version of this article:
Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.