|Home | About | Journals | Submit | Contact Us | Français|
Although there is broad consensus that careful content vetting and user testing is important in the development of technology-based educational interventions, often these steps are overlooked. This paper highlights the development of a theory-guided, web-based communication aid (CONNECT™), designed to facilitate treatment decision making among patients with advanced cancer.
The communication aid included an online survey, patient skills training module and an automated physician report. Development steps included: 1) evidence-based content development, 2) usability testing, 3) pilot testing, and 4) patient utilization and satisfaction.
Usability testing identified some confusing directions and navigation for the on-line survey and validated the relevance of the “patient testimonials” in the skills module. Preliminary satisfaction from the implementation of the communication aid showed that 66% found the survey length reasonable and 70% found it helpful in talking with the physician. Seventy percent reported the skills module helpful and about half found it affected the consultation.
Designing patient education interventions for translation into practice requires the integration of health communication best practice including user feedback along the developmental process.
This developmental process can be translated to a broad array of community based patient and provider educational interventions.
Patients with advanced cancer exist in a challenging decision making context, as they are often facing mortality, with treatment options characterized by uncertain outcomes and significant potential for toxicity. In general, physicians serve as the primary source of medical information [1, 2] and the communication between doctor and patient is of critical importance to quality decision making [3–5], even although it can be challenging for both parties. Our previous research raised concerns about the adequacy of physician-patient communication regarding information relevant to patient decision making and values among patients with advanced cancer . For example, we found that although 95% of patients valued quality of life at least as much as length of life, only 28% reported that the impact of treatment on quality of life was discussed with their oncologist. In contrast, their oncologists reported that this topic was discussed in 73% of the consultations. Furthermore, we observed significant discordance in the expectations of cancer patients and their oncologists regarding the outcomes from standard and experimental therapies. We thus sought to develop an intervention to improve communication between cancer patients and providers that, if proven effective, could be translated to a variety of health care settings. In addition, we believed that integrating theory and health communication best practice into the development and testing was critical to ensure that the intervention was relevant and effective in improving informed decision making. We selected a web-based approach for several reasons, including 1) widespread and increasing Internet use among patients seeking health-related information; 2) applicability to implementation at home or any location with Internet access before the physician visit; 4) ability to automate production and delivery of summary information; and 4) potential for broad dissemination to oncology community-based practice. This paper highlights the developmental process and user testing of a web-based communication aid based on the current literature in patient/provider communication and health communications best practices.
Communication is a two-way exchange between the patient and provider. On the provider side, physicians have established communication styles and patterns that are difficult to change. Brief interventions in the cancer context have been successful in affecting behavior immediately following the intervention and have a greater potential for translation into practice . For example, several studies have demonstrated that when results of quality of life (QOL) patient questionnaires were provided to physicians of cancer patients before their visits, physicians asked more questions about QOL and discussed more issues related to QOL [8–10]. In addition, physicians reported that they believed the intervention enhanced communication . Prompting of physicians immediately before an office visit can also improve cancer screening behaviors [7, 11]. While the physician's communication style is important, recent studies on patient participation, especially those focusing on communication skills training, identify patient behaviors that are directly relevant to physician-patient communication about cancer treatment [12, 13].
Patient participation in medical consultations is a function of several factors, such as knowledge of one's disease, personality and motivation, and provider communication , as well as the patient's repertoire of communication skills and routines. It is likely that patient’s communication competence in talking with physicians is of critical importance to the information they receive from physicians  and their decision making about cancer treatment and participation in clinical trials. There is a limited, but growing literature on the effects of patient communication skills training [12, 16–28] that bears on the issue of cancer treatment communication. Several studies have shown that with relatively modest effort, communication skills interventions significantly enhance patients' question-asking, provision of detailed information about symptoms, elicitation of information from the physician, and overall sense of control [29–35]. Other studies have shown that patient communication skills training promotes adherence [34, 36] to treatment regimens and improvement in health outcomes [37–39].
With recent advances in computer technology and access to the Internet, increasingly patients and physicians are turning to the use of computers to obtain and deliver patient-centered health care information [40–45]. In addition, the use of web-based interventions broadens the reach and increases the translation of these patient education tools into practice in all settings. As the use of the Internet increases, (recent estimates of over 73% of Americans are using the Internet  and eight in ten of these users indicate that they have looked online for health information), the opportunity for broad dissemination of proven interventions is ripe. Moreover, web-based approaches provide a communications channel that is easily accessible for most patients, through their home, work or local libraries. The Science Panel on Interactive Communication and Health [47, 48] suggested the value of interactive health communication and those that use structured psycho-educational approaches can improve knowledge and health behavior skills . Specifically, patients with cancer commonly use the Internet to access disease and treatment-related information [50–56] and report receiving some benefit of finding health information and support online . Further, there is evidence for the high acceptability of multi-media interactive programs to enhance treatment decision making among prostate cancer patients , and for the effectiveness of web-based interventions on behavioral outcomes .
Health communication as “the way we seek, process, and share health information,”  is central to the quality of cancer care, from primary prevention to survivorship . Health communications can raise awareness of cancer issues, support informed decisions, and motivate and improve health behaviors [62, 63]. Research and program evaluation results indicate key themes and approaches for best practices in health communication including: understanding the needs and perceptions of the intended audience; planning and strategy development; development and pre-testing of concepts, message and materials; program implementation; assessing effectiveness; and making refinements . Successful health communication programs involve more than the production of messages and materials. They are based on a combination of health education, social marketing, adult learning, program planning/evaluation, and communications design.
Based on the recognition of the importance of enhancing provider-patient communication and the opportunity to utilize web-based technologies, we developed an interactive web-based communication aid for cancer patients and their oncologists that could be used in a variety of settings. We understood that in designing web-based interactive interventions, it is important to incorporate not only design principles but also health communication principles to optimize the users’ experience. As stated in the US Institute of Medicine Report,  the message is still as important as the medium. Thus, attention needs to be provided to both the content and functionality of web-based interventions to assure quality. We also recognized the importance of developing interventions that are theory driven. The Cognitive-Social Health Information Processing Model (C-SHIP) [66–68] provides a theoretical framework for the design of interventions to enhance decision-making and facilitate cancer risk communication. Specifically relevant to understanding behavior change and breast cancer risk, the C-SHIP framework includes variables such as cancer-related knowledge and perceived vulnerability, self-efficacy beliefs, distress, values and goals and self-regulatory strategies which have been shown to characterize and predict health behaviors among patients across the cancer continuum . This multi-center randomized controlled trial was designed to evaluate the efficacy of a web-based communication aid for patients with advanced cancer. Patients with advanced cancer were recruited to participate in the trial prior to their initial consultation with a medical oncologist. Eligibility criteria included: 1) first outpatient consultation with a medical oncologist; 2) documented metastatic malignancy; 3) 18 years of age or older; 4) able to read and verbally communicate in English; and 5) ability to provide written informed consent to participate. Potential study participants were recruited between January 2005 and January 2007. This study was approved by the Fox Chase Cancer Center Institutional Review Board.
A structured development phase of the project was completed over the prior year. This paper focuses on the developmental process including user feedback of the various components of the communication aid as well as the preliminary usage and patient satisfaction results from the randomized trial.
The complete CONNECT™ communication aid includes three key components: 1) a web-based patient survey to assess values, goals and communication preferences (hereafter referred to as survey): 2) a web-based communication skills training module for patients (referred to as the skills module); and 3) a summary of the responses that are included in the automated computer-generated physician report (referred to as the physician report) to provide physician prompting in an effort to maximize the communication of issues concordant with patient values.
The development and user testing of the communication aid included: 1) a carefully planned, multi-disciplinary approach of content vetting and formatting of the web-based patient survey and communication skills module, 2) usability testing of these two components with selected patients, and 3) pilot testing of the intervention, including physician feedback. In addition, tracking usage and patient satisfaction were collected as part of the randomized trial.
Content for the survey and the physician report were developed to optimize the matching of the patient-physician exchange with individual patient values and information needs. The survey includes 125 questions and 38 pages. Web designers and health communications professionals collaborated to design a layout easy to navigate, where scrolling vertically and horizontally was minimized.
An evidence-based communication skills training module was the next to be developed. It was modeled in part on the PACE System, available in booklet form and on a free-access web site (http://patcom.jcomm.ohio-state.edu), developed by Cegala et al [29, 30, 36]. PACE is an acronym for Presenting information, Asking questions, Checking understanding, Expressing concerns. It also addressed key components of the C-SHIP model, including cancer-related distress and values and goals.
The web-based skills module developed for this study is an interactive patient training system that presents patients with a selection of menu choices, including: general educational items, instruction for presenting information and concerns, and asking questions. Patients are instructed in providing detailed information about their symptoms, including psychosocial issues that may be related to their medical condition.
The research team included experts in medical oncology, medical decision making, psychology, medical ethics, and cancer education and communication. The team applied best practice approaches [64, 69, 70] in a number of ways: use of personal stories (vicarious modeling), active voice, simple, easy-to-read layout, and 7th grade reading level. Initial scripts and photographs were reviewed by cancer educators working directly with patients to elicit feedback regarding relevance and acceptability of content. We employed recommendations [71, 72] for the development of web-based formats, including using short sentences, chunking information and providing visual cues and photographs. The communication skills training module features four diverse cancer patients that “talk” (through the use of short written narratives) to the participant about four subjects: 1) how to prepare for the visit, 2) how to talk to the doctor, 3) what to ask the doctor about medical conditions, and 4) what to ask the doctor about lifestyle needs. The cancer patients model the recommended behaviors and provide real life examples of communication challenges. In addition to content development, the research team worked closely with the software programmers to ensure the functionality and “look and feel” of each of the pages of the module.
Once the development of the CONNECT communication aid was complete, we conducted usability testing to determine whether patients found the tools easy to use and navigate. A usability protocol, based on NCI guidelines , was developed and included “talk alouds” as patients reviewed the site, staff observations and additional interview questions at the end of the session. Participants were cancer patients recruited through Fox Chase Cancer Center, a United States National Cancer Institute-designated Comprehensive Cancer Center in Philadelphia, Pennsylvania. The communication aid was accessed from the Fox Chase Cancer Center’s Resource and Education Center (REC), a multimedia patient education program.
We also conducted a pilot test including all the components of communication aid and the study process with patients who met the eligibility for the study and with their physicians. Participants received all elements of the intervention, including survey, module, and physician report. Participating physicians completed a survey assessing opinions about readability and usefulness of the physician report.
Modifications to CONNECT™ communication aid (on-line survey, skills module and physician report) were implemented iteratively as feedback was collected during both usability and pilot testing. The randomized study has three arms: 1) control condition without relevant survey, module, or physician report; 2) on-line survey, skills module and physician prompt; and 3) on-line survey and skills module without automated physician report prompting. A secure web-interface permits participants to access the survey on their home personal computers. Following verbal consent to participate, patients are given the option to complete the survey at home or arrive one hour before their appointment to complete in Fox Chase Cancer Center’s Resource Education Center.
Measures included in the online survey (pre-consultation) and follow-up post-consultation paper surveys are shown in Table 1. A number of measures focus on the participant’s perceived usage of and satisfaction with the communication aid. In addition, the website was designed to gather usage data from each patient, including the number of warning messages received and corrected, total time spent on the survey and the module, time spent on each page of the aid, number of times a participant clicked the “Help” button, logged into the aid, and requested a definition of a word on the module.
Overall, 742 patients were enrolled in the study. The two experimental groups (n=470) received both the on-line pre-consultation survey and skills module. Here we focus on the data and feedback from patients who used these two components of the communication aid.
Six patients (five males and 1 female age 50 to 54) were recruited for initial usability testing. All patients had gastrointestinal cancer and were non-Hispanic, white patients. Participants indicated that some directions were confusing, however navigational problems were minimal.
Feedback from participants on the survey component ranged from specific problems with instructions to formatting. Examples of comments and resulting changes are shown in Table 2. Changes in the format, including removing shading, re-writing directions, and adding next arrows, improved the overall functionality. Participants took between 19 and 51 minutes to complete the survey. Figure 1 shows two of the revised survey pages with added help links, navigation buttons and a thermometer, which cues to indicate progress.
The module (communication skills training) required about 5–10 minutes to complete. Participants found it easy to understand. Patient “testimonials” were believable and helpful. They reported that the amount of information was appropriate and there were few navigational problems. Participants provided important feedback to refine the tool. They recommended that introductory instructions for the module include a suggestion that patients have a pen and paper ready to take notes, and that a statement should be added to the question list explaining that questions are for patients to share with doctors. Figure 2 shows the first page of the communication aid with the patient testimonials.
After revisions were made, a pilot test was conducted to provide feedback on the entire study process and usage data from the on-line survey. Twenty-two patients agreed to participate in a pilot test of the survey and the skills module. Table 3 shows demographics of pilot test participants. Of these, 18 began the communication aid; 15 completed it. The average time for completing the entire program was 65 minutes, 52 minutes spent on the survey and 13 on the module. The 18 participants clicked a help link a total of 12 times. Participants most frequently used the help for the open-ended questions page, “What are the most important things that you would like to gain from your consultation with your physician?” and “What are the most important things that you would like your physician to know about you?” (Table 3). Patients received 19 warnings (1.3 per participant) that they either skipped a question or incorrectly answered it. For 12 of these 19 warnings (63.2%), participants corrected their previous answers. All but 3 participants completed the survey and module in one session. Two people left the module to finish later, and one participant completed the survey in two sittings.
Feedback from the open-ended question included, “less questions in each section”, “In some cases none of the answers suited me”, and, “When typing questions 1–10, make the top answers follow along with the questions. We had to keep scrolling up and down for the headings of the radio buttons . . . A few questions were not clear”. Changes to the look of the survey were made based on recommendations, but content of the directions and questions were not changed because they were from validated instruments critical to the research.
Attending physicians and oncology fellows who agreed to participate in the pilot study (N=14) completed a brief questionnaire assessing the readability and usefulness of the physician report. All physicians reported that it was easy to understand, 13 reported it was easy to read, and 11 reported that it was useful to the consultation. They reported that most informative sections were: “How do you like to make decisions?” (N=6) , “Amount of Discussion: Diagnosis and Treatment” (N=5), and “Format of Discussion” (N=4). “Amount of Discussion: Diagnosis and Treatment” was noted to be the least informative (N=4) along with “Your Patient’s Mood” . One physician explained, “Maybe the sample size was too small, but [it] seemed [that] my patients wanted everything, and it became clear early on [in the consultation] what they wanted.”
The pre-consultation survey included general questions regarding Internet use. Twenty three percent stated they didn’t use a computer and another 11% used it less than one hour per week. (Table 4) Nineteen percent were heavy uses stating they used the computer 14 hours or more each week. Although the majority (65%) of the participants used their home computer to complete the survey, 24% completed the survey on site at Fox Chase’s Resource Education Center.
After completing the on-line survey, 85% of patients reported it was easy to read and understand (4 or 5 on a five point scale ranging from “not at all” to “extremely”). Sixty-six percent felt the survey length was “reasonable” (Table 4). The mean time to complete the survey was 37 minutes (SD = 17.8, Range 14 – 153 minutes). The overwhelming majority of participants (97%) were “moderately” or “extremely” satisfied with the survey format.
Usability data collected from a tracking program of the communication aid in the randomized trial shows that 23% of participants are used the “Help” buttons on the survey and the skills module (Mean = 1.24, SD = .51, range 1–3). Seventy percent of participants received at least one warning message indicating that something was either incorrectly completed or missing on each page of the survey (mean: 2.77 SD: 3.6, range 1 – 31). Ninety-five participants did not start (16%) or did not complete (8%) the module before their visit.
Several questions on the post-consultation survey addressed perceived impact of the survey and module, with 5 options from 0, “not at all” to 4, “extremely”. Seventy percent of patients reported that the survey was moderately to extremely helpful with the consultation and 49% stated that it affected how they communicated (Table 5). Similarly, 70% reported that the module was moderately to extremely helpful and 51% reported that it affected the consultation. Over half (53%) of the patients reported that taking part in the overall program or study was moderately to extremely helpful to their consultation with their physician.
This web-based communication aid was developed based upon empirical research on patient-provider communication, cognitive-affective theory, and health communication best practices, including careful planning and user feedback. The development of the aid relied on a multi-disciplinary team of behavioral and medical researchers, health communication experts and web design programmers and utilized an iterative process to gather user feedback.
The usability testing did identify a number of needed changes. These changes were not substantial, but improved the navigation. Our usability findings indicated some changes in survey format, specifically removing shading, adding next arrows and providing clearer instructions, which improved the overall functionality. Given that this usability was conducted prior to the final production, we were able to address the concerns raised by patients. We made as many changes as possible without compromising the integrity of the questions that were needed for the physician report and the study outcomes. Given the length of the survey (average of 40 minutes), the addition of a cue (e.g. thermometer) was important to help patients judge their progress. Even with the amount of time required for the survey, results from the randomized trial indicated that participants found the survey easy to read and understand, with a minority indicating that the survey was too long. The added help button was an additional feature that was used by patients in the randomized trial. The Communications Skills Training module required much less time. On average, it took about 11 minutes for patients to complete this portion of the communication aid, which seems reasonable for dissemination into routine care. The module relied on a number of health communications best practices, such as vicarious modeling, and was well received in the usability testing and the majority of patients in the randomized trial found it helpful.
Our preliminary usage and satisfaction data are quite promising. Over half indicated that the skills training had been helpful, although a small minority disagreed. Additional analyses will provide insight into patient characteristics that impact the value of these tools and the randomized trial will provide more objective results on the impact on the consultation.
We did note a number of challenges that might impact utilization in practice. Almost 25% of patients did not complete the module before their visit which may have limited the extent of use. Another important observation related to broader dissemination of web-based interventions is that almost one in five participants did not use the Internet: many of the participants accessed the survey and module at clinical sites and not at home. It is important to explore these barriers especially as more web-based resources are used to support patient decision making.
There are a number of limitations of this study. The sample size for the usability testing was small. Although the recommendation is 6–8 subjects to determine navigational problems, more subjects might be required for more complex web-based tools. Our patient population was somewhat homogenous and did not provide a largely diverse population which would have been more representative.
In conclusion, based on our formative evaluation, the tools were improved in response to patient feedback during the usability and pilot testing. The integration of health communication approaches and principles, behavioral theory and informatics design was achieved using a multidisciplinary team sharing different perspectives and approaches. Moreover, the process of iterative testing and refinement was quite feasible and is an important component of designing electronic communication aids that are easy to use and relevant to those patients we are trying to reach.
The implicit goal of intervention research is to develop effective interventions that have value to the broader population from which the efficacy study sample is drawn. The integration of health communication best practices, including user testing and feedback, can be accomplished with minimal time and resources. These steps should be built into the planning process so that the multidisciplinary team is in place and resources for conducting testing are secured. A small but significant number of patients were not using the Internet and needed to access these types of web-based tools on site. Finally, the impact of CONNECT™ on patient outcomes will be determined when data collection from the randomized trial is complete. If proven effective, it can be easily translated into community-based practice with the potential of improving patient-provider communication and patient decision making.
We confirm all patient/personal identifiers have been removed or disguised so the patient/person(s) described are not identifiable and cannot be identified through the details of the story.
We thank our collaborators, Fox Chase Cancer Center’s Behavioral Research Core Facility and the staff of Fox Chase Cancer Center’s Resource Education Center for their support and efforts on this project. The following personnel were instrumental in recruiting patients: Fox Chase Cancer Center, Neal J. Meropol, PI; Northwestern University, Al B. Benson III, PI; and Meharry Medical College, Stephen Wolff, PI.
Supported by NIH R01 CA82085; Fox Chase Cancer Center’s CCSG Population Studies Facility and Behavioral Research Core Facility P30 CA006927 (NIH)
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest: None (see comment about this above at the beginning of the Discussion section)
Linda Fleisher, Fox Chase Cancer Center, Philadelphia, PA.
Joanne Buzaglo, Fox Chase Cancer Center, Philadelphia, PA.
Michael Collins, Fox Chase Cancer Center, Philadelphia, PA.
Jennifer Millard, Fox Chase Cancer Center, Philadelphia, PA.
Suzanne M. Miller, Fox Chase Cancer Center, Philadelphia, PA.
Brian L. Egleston, Fox Chase Cancer Center, Philadelphia, PA.
Nicholas Solarino, Fox Chase Cancer Center, Philadelphia, PA.
Jonathan Trinastic, Fox Chase Cancer Center, Philadelphia, PA.
Donald J. Cegala, Ohio State University, Columbus, OH.
Al B. Benson, III, Northwestern University, Evanston, IL.
Kevin A. Schulman, Duke University, Durham, NC.
Kevin P. Weinfurt, Duke University, Durham, NC.
Daniel Sulmasy, St Vincent Catholic Medical Centers, New York, NY.
Michael A. Diefenbach, Mount Sinai School of Medicine, New York, NY.
Neal J. Meropol, Fox Chase Cancer Center, Philadelphia, PA.