While decision support tools such as Adjuvant! use graphical displays to communicate the mortality risks that patients face with different adjuvant therapy options, our research shows that women had difficulty interpreting the 4-option horizontal bar format currently used by Adjuvant!. Two simple changes, displaying only risk information related to treatment options that included hormonal therapy (since the scenario described an ER+ tumor) and using pictographs instead of horizontal bars, resulted in significant improvements in both comprehension accuracy and speed of use in our demographically diverse sample. Furthermore, respondents showed strong preferences for pictograph formats over the currently used horizontal bar format. It is important to note, however, that sizeable knowledge deficits were still observed even when risk information was presented using the best format tested in this study, the 2-option pictograph. Further research is clearly needed to explore even more simplified formats to determine whether we can further improve patient understanding of the risk tradeoffs associated with adjuvant therapy decisions.
We draw particular attention to the fact that the participants who viewed the 2-option pictographs not only took the least time to complete knowledge tasks () but also had the lowest error rates (). Together, these data demonstrate that reading and interpreting the 2-option pictographs required less cognitive effort than the 4-option bar graphs did. The task of making complex treatment decisions is both cognitively demanding and emotionally stressful, and evidence suggests that people’s decision making performance is often degraded under such conditions.[22
] Moreover, studies have shown that cognitive effort induces negative emotions in many people, and that these emotions can cause them to withdraw from making decisions.[24
] Thus, even if patients could figure out more complex graphics given time and support from their clinicians, their ability to use this information in their decision making would be impeded by the cognitive effort required to obtain it. Using simpler graphical formats may help to offset this unwanted effect.[25
While individual numeracy levels were strongly predictive of risk knowledge, the design of the risk graphic affected both high numerate and low numerate individuals similarly. Such findings reinforce our belief that optimal design of risk graphics is essential for all users, not just those less educated or less numerate.
Our research has several
limitations. First, although our Internet sample contained significant demographic diversity, we did experience some significant dropout during the survey. Those individuals who failed to complete the survey (and hence did not provide comparable demographic information) may have had different characteristics than those who completed it. Our participants may also
be non-representative in unidentified ways (for example, because they enjoy taking surveys). However, we ensured internal validity by using an experimental design. Furthermore, our previous research using this panel has shown that Internet survey responses from this panel closely match those of representative samples.[27
] Second, most participants (75%) reported having had at least some education beyond High School, a trait which may limit our ability to generalize these findings to a less educated population. Third, our scenario was entirely hypothetical, and actual cancer patients may be more motivated to correctly interpret risk graphics presented to them by clinicians. Patients also have the opportunity to discuss such graphs in face-to-face consultations with their oncologist, which undoubtedly leads to better comprehension than we observed. Nevertheless, our experimental results suggest that the use of non-optimal risk communication graphics can significantly inhibit comprehension of key statistics, whereas simpler graphics may enable clinicians to spend less time explaining risk information to patients and more time discussing its implications for each patient’s adjuvant therapy decision.
The results presented here support the concept that simpler information displays can make it easier for decision makers to implement optimal decision strategies.[22
] Specifically, focusing patients’ attention on those treatment options currently under consideration while removing information related to options which have been already eliminated from consideration (for medically appropriate reasons) may be particularly beneficial.[24
] In the context of adjuvant therapy decisions, such an approach would imply that clinicians should discuss the decision in two stages: a first stage in which hormonal therapy is considered and a second stage in which the incremental benefit of chemotherapy is evaluated. The 2-option pictograph tested here would be highly appropriate for the second stage of this discussion, and a similar graphic showing no therapy vs. hormonal therapy outcomes could be used to improve patient comprehension of the first stage.
Adjuvant! and other online risk calculators enable oncologists and patients to receive individually tailored estimates of mortality and recurrence risks, information that is essential to informed decision making about adjuvant therapy options. Yet, the full potential of these modeling applications cannot be realized if users misinterpret the statistics provided.[13
] Our results show that using certain graphical formats with patients can preclude comprehension, and clinicians may face similar difficulties when using statistics presented in these formats for clinical decision making. Developers of risk communication and decision support tools should incorporate evidence-based, simplifying design elements, such as removal of information not required for the current decision and the use of pictograph formats, into both existing and future tools.