In our randomized experiments, women who viewed risk graphics that only displayed survival information comprehended these graphs at least as well, and sometimes better, than women who viewed analogous graphics that displayed both mortality and survival statistics. Instead of being upset by the lack of mortality information, women who viewed survival-only graphs provided significantly higher graph evaluation ratings than women who saw multi-outcome graphs.
One potential concern about the use of survival-only graphs is the possibility that removal of the mortality statistics might obscure the fact that treatment is not fully effective and thus many women will die even with intensive treatments. Such a bias would tend to increase women’s interest in taking both chemotherapy and hormonal therapy. However, we instead found the opposite pattern. Participants who viewed survival-only graphs were less likely to add chemotherapy to hormonal therapy in a hypothetical adjuvant therapy decision scenario than women who viewed multi-outcome graphs. A mediation analysis (omitted for brevity) showed that this effect was essentially unaffected in both the primary and replication studies by controlling for the impact of graph format on risk comprehension, suggesting that the main impact of graph type is to change people’s gist impression of the size of the treatment benefit rather than affecting their specific comprehension of the statistics.
The findings reported here extend our previous work and are consistent with previous research that has argued that simpler information displays can make it easier for decision makers to use information in their decision making.(16
) They are also consistent with the growing evidence that “less is more” in many decision contexts.(15
) For example, Peters et al. showed that removing information about non-critical dimensions and highlighting essential information resulted in significant improvements in people’s ability to identify the best hospital out of a set.(25
) Contrary to the Peters et al.
study, we did not find interactions of simpler information presentation formats with individual numeracy as predictive of either comprehension or treatment intentions. Numeracy did moderate the effect of format on graph evaluation ratings, however, with more numerate participants much more likely to prefer the survival-only graphs.
Our participants’ preferences for survival-only graphics are also consistent with research that shows that people tend to avoid medical treatments that include small risks of unpleasant complications, even when doing so implies choosing a treatment option with a lower overall chance of survival.(26
) In that research, the presence of information about multiple possible outcomes steered people away from choosing the survival-maximizing option, whereas the therapeutic option that only had a binary cure vs. death outcome (but had decreased overall survival rates) seemed less threatening.
In the particular case scenarios used to create the risk statistics for our vignettes, adding chemotherapy to hormonal therapy only results in 1–2 additional women out of 100 being alive after 10 years. While individual preferences and values mean that we cannot define a “correct” choice for any particular woman, this incremental benefit is small enough that many women perceive it as insufficient to justify adjuvant chemotherapy.(21
) In this study, the contrast between the small increment in survival and the large total survival rate may have been heightened among viewers of survival-only graphs, thereby explaining their reduced interest in chemotherapy. Although we did not substantially vary the risk statistics in our research to test this proposition, we speculate that survival-only graphs may have different effects in situations where the baseline survival rate is relatively low. In such situations, the (newly heightened) contrast between incremental and total survival would make even a small increase in survival seem more valuable, a finding that would be consistent with prior research that shows that people attend more to the relative magnitude of change than to absolute differences.(27
Our research has several limitations. First, we used a hypothetical scenario administered to a general public sample rather than an actual cancer patient population. This approach enabled us both to control the risk statistics presented and also to recruit a sample large enough to compare people’s reactions to these different graphs. However, it is possible that mortality risk information would be more salient to actual cancer patients or be perceived as more critical by them than by our respondents. Patients may also place particular value on knowing what proportion of mortality risk is due to cancer versus other causes, a fact that is omitted in survival-only graphs. We also focused exclusively on survival risk information; actual patients may expect or require risk communications to address the likelihood of cancer recurrence.
Second, we reiterate that we made an error in the legend of our multi-outcome pictograph in the primary study-- the legend reported the wrong number of women who would die from cancer (the graphic itself was correct). Although none of the comprehension questions were directly related to this number (they all asked about the number of women alive – statistics which were accurately reported in the legend), the difference in knowledge accuracy rates did not fully replicate in our follow-up study and thus needs to be interpreted with caution. This error did not, however, affect participants’ significantly higher graph evaluation ratings for survival-only graphs and cannot explain why use of the survival-only graph tended to reduce participants’ interest in chemotherapy (because the mediation analysis showed that the effect of graph format on comprehension had no substantial mediational relationship to the larger impact on treatment intentions). Lastly, of course, we note that the findings of primary study were replicated in direction, if not to equal levels significance, in our replication study, which supports our belief that survival-only risk presentations are certainly no worse than multi-outcome presentations and may often be better.
Third, we experienced a moderate degree of survey discontinuation and dropout. Since individuals who failed to complete the survey did not provide any demographic information, we cannot assess whether these individuals had systematically different backgrounds than our final participant pool. Members of Internet survey panels such as the one we used are also non-representative in their demonstrated preference for taking surveys and may differ in other ways as well. Nevertheless, our randomized experimental design ensured internal study validity, and our previous research using this panel has found close correspondence between Internet survey responses and those of representative samples.(29
Adjuvant! and other online risk calculators provide individually tailored estimates of prognosis statistics, information that is essential to informed decision making about adjuvant therapy options, and it is clear that such decision support tools can facilitate better decision making by both clinicians and patients. (2
) Yet, our results suggest that even the information provided by Adjuvant! may be more than many patients can effectively absorb and process.(11
) By using a “less is more” approach and stripping the risk graphics shown to our participants down to the bare minimum, the chance of survival, we increased participants’ satisfaction with the materials while simultaneously supporting risk comprehension that was at least as good as that achieved with more complex graphics. While effective decision making about medical treatments often requires consideration of multiple possible outcomes simultaneously, many risk communications do incorporate redundant information such as separate presentations of mortality and survival risks. Further research is clearly needed in actual patient populations to assess whether removal of duplicate information may enable patients to focus on the likelihood of a single critical outcome (either mortality or survival) and potentially therefore be more cognizant of the relationship between their choice of treatment option and their health outcomes.