Feedback we received during each phase resulted in numerous iterative refinements of both the video and the handbook. We iteratively refined the video 17 times (11 script iterations in Stage 1, three initial video iterations in Stage 2, and three final film iterations in Stage 3). We iteratively refined the handbook 26 times (three iterations in Stage 1, 22 iterations in Stage 2, and one final iteration in Stage 3). Examples of preliminary handbook presentations, reflecting major iterative design approaches from Stages 1 and 2, are presented in Figure
Representative sample iterations from stages 1 and 2 of the handbook development process.
Results: findings from mixed methods pilot study
Of the 12 participants in pilot study Phase 1, 67% were male, 92% were African American, and 75% had at most a high school degree or GED. The majority of participants were on hemodialysis (n=9, 75%), while the remaining participants were classified as pre-ESRD (n=3, 25%). Of the 36 participants in Phase 2, 44% were male, 83% were African American, and 47% had at most a high school degree or GED. Fifty-four percent of phase 2 participants were on hemodialysis, 33% were considered pre-ESRD, 29% on peritoneal dialysis, and 17% on home hemodialysis. Participants in both phases were diverse with respect to their knowledge of kidney disease, literacy, numeracy, and cognitive function (Table
Participant characteristics for handbook development Stage 2 (Phases 1 and 2)
Qualitative feedback obtained during cognitive interviews (during Phase 1 and Phase 2) focused on three key areas: (1) difficulties identifying the multiple treatment types presented in the handbook, (2) the handbook containing an intimidating amount of complex scientific information, and (3) their desires to have numerical or quantitative information from scientific studies presented in a way that was clear and understandable. Through a total of 22 iterations (16 iterations in Phase 1 and six iterations in Phase 2) we addressed these concerns through numerous adjustments (including the use of color coding to consistently identify RRT options when discussed throughout the handbook, providing explanatory guides for how to read pages, eliminating the use of abbreviations, and supplementing graphical numerical presentations with text written at a 4th to 6th grade reading level) (Table
Qualitative feedback from handbook development Stage 2 (Phases 1 and 2)
During study Phase 2, we measured gist comprehension of materials among a total of 36 participants. Twenty-four (24) participants provided feedback on iteration 20 of the handbook and 12 participants provided feedback on iteration 21 of the handbook. Among all items viewed by participants, participants demonstrated median gist comprehension of 78% (range: 15%–100%) in iteration 20. Participants’ median overall gist comprehension rose to a median of 91% (range: 9%–100%) in iteration 21. Participants demonstrated greatest gist comprehension on aspects of pages reflecting the topic being discussed, the treatment types (modalities) being compared, and the direction of scientific findings discussing treatment differences (i.e., they could correctly identify one treatment was better than the other). Participants demonstrated less gist comprehension regarding the magnitude of treatment differences (i.e., they had difficulty understanding how much better one treatment was than another) and information about the research study quality. However, gist comprehension scores improved in these areas between iterations 20 and 21 (Table
Median percentage** of times participants correctly comprehended key aspects of scientific evidence during pilot testing (handbook development Stage 2, Phase 2)
Results: feedback from decision Aid screening
Five patients and seven family members participated in two different screening groups. Patient participants were 80% female, 60% African American, and 60% had at least two years of college education. All family member participants were female, 71% were African American, and 43% had at least two years of college education.
Patients and family members provided mostly positive feedback after screening both the video and handbook. (Table
) Participants often commented that they liked the organization of the video and the breadth of topics it covered. Participants also commented on the complementary nature of the two materials. Screening group participants constructively criticized the large amount of information presented in the handbook. To address this concern, we created a mini-book entitled, “All of the Facts,” to accompany the handbook, which provides readers with a brief summary of key information presented in the longer handbook.
Positive and constructive feedback obtained from screening the decision aid
Results: final decision Aid
The final decision aid consists of a 50-minute video and a comprehensive, 159-page handbook accompanied by a 14-page mini-book (“All of the Facts”) which summarizes the key information in the handbook. Together, the video and handbook satisfied 92% of all quality criteria outlined in the three subject areas designated by IPDAS (satisfied 96% or 22 of 23 content items, 87% or 20 of 23 development process items, and 100% or 6 of 6 effectiveness items) (Additional file
The video provides testimonials from minority and non-minority (male and female) patients, their family members, and health care professionals (physicians and social workers) about “concerns” patients and their families might have which could influence their RRT selection decisions. These “concerns” correspond to the seven key content areas patients and their families identified as important to include in the decision aid during our foundational research studies. Patients and family members receiving different RRTs share subjective testimonials describing their positive and negative experiences with these “concerns” in the video. Health care providers discuss additional information regarding “concerns” to balance patient and family views and to contribute additional information.
The handbook provides evidence-based information regarding risks and benefits of different treatment options of peritoneal dialysis, in-center hemodialysis, home hemodialysis, kidney transplant, and conservative management (treatment with no transplant or dialysis) as they pertain to these same patient concerns. The introduction includes a section that defines kidney disease (“What Is Kidney Disease?”), describes and color codes the different treatment options (“What are the Treatments?”), presents a value clarification exercise to help patients determine which treatment is best for them (“How Do I Choose a Treatment?”), and orients the reader to the page layout (“What is on Each Page?”). Subsequent sections of the handbook describe summaries of scientific studies relevant to each “concern.” Handbook pages are 8.5” by 11” and oriented toward a 4th to 6th grade reading level. The handbook delivers content on a double-page spread layout to appeal to diverse learning preferences: the left-hand page includes brief statements summarizing key messages from scientific studies while each right-hand page presents a more detailed summary of study findings and a graphical presentation of data (Figure
Sample pages from final handbook.