|Home | About | Journals | Submit | Contact Us | Français|
Participants in early-phase clinical trials have reported high expectations of benefit from their participation. There is concern that participants misunderstand the trials to which they have consented. Such concern is based on assumptions about what patients mean when they respond to questions about likelihood of benefit.
Participants were 27 women and 18 men in early-phase oncology trials at 2 academic medical centers in the United States. To determine whether expectations of benefit differ depending on how patients are queried, we randomly assigned participants to 1 of 3 interviews corresponding to 3 questions about likelihood of benefit: frequency-type, belief-type, and vague. In semistructured interviews, we queried participants about how they understood and answered the question. Participants then answered and discussed one of the other questions.
Expectations of benefit in response to the belief-type question were significantly greater than expectations in response to the frequency-type and vague questions (P = .02). The most common justifications involved positive attitude (n = 27 [60%]) and references to physical health (n = 23 [51%]). References to positive attitude were most common among participants with higher (> 70%) expectations (n = 11 [85%]) and least common among those with lower (< 50%) expectations (n = 3 [27%]).
The wording of questions about likelihood of benefit shapes the expectations that patients express. Also, patients who express high expectations may not do so to communicate understanding, but rather to register optimism. Ongoing research will clarify the meaning of high expectations and examine methods for assessing understanding in this context.
The ethics of early-phase clinical research continues to be a subject of much debate in oncology. One of the major questions concerns the degree to which participants in early-phase oncology trials have given truly informed consent.1,2 A principal worry is that patients may misunderstand the likelihood of benefit from participating in early-phase trials.1,2,4–7 This concern is based, in part, on research in which clinical trial participants have reported very high expectations of benefit.7–9 For example, Meropol et al7 found that the median expectation of benefit among patients who agreed to participate in phase 1 trials was 60%.
Whether high expectations of benefit should be interpreted as misunderstanding in every case is not clear.1,10,11 Among the many complicating factors are two assumptions that some researchers and commentators have made when interpreting patients’ reports of high expectations as failures of understanding.10,12,13 One assumption is that all types of patient expressions of expectation of benefit are the same, regardless of how the questions and answers are phrased. However, this assumption might not be valid if, for example, some patients are queried about the chance that they will benefit personally, whereas others are queried in terms of the proportion of patients who will benefit. On careful analysis, these two queries entail different conceptions of probability, so patients’ responses to them should be evaluated for correctness using different criteria.10 Another assumption is that patients are attempting to demonstrate their understanding of the likelihood of benefit when they construct their responses to questions about likelihood of benefit. It is possible, however, that patients are attempting to achieve other ends besides communicating their understanding, such as influencing their outcomes by expressing optimism.10
As an initial step in investigating both of these assumptions, we designed a pilot study in which we randomly assigned patients to different question types to determine whether expectations of benefit would differ depending on how the patient was queried. We used the qualitative method of cognitive interviewing14 to explore how patients arrived at their answers and what information they intended to convey with their responses to questions about expectations of benefit.
Eligible participants were English-speaking adults with cancer who had agreed to participate in a phase 1 or 2 clinical trial at one of two academic medical centers but had not yet begun the investigational therapy. Typical sample sizes in cognitive interview studies are between 6 and 10 people.14 To ensure that we reached conceptual saturation regarding patients’ justifications for their responses, we recruited 15 patients per question type, as described below, for a total sample size of 45. The institutional review boards of the Duke University Health System and Fox Chase Cancer Center approved this study.
We randomly assigned each participant to one of three interview protocols corresponding to three “target questions” about likelihood of benefit. We adapted Hacking’s typology of probabilities15 to develop the following target questions:
In a semistructured, audio-recorded interview, the interviewer presented the target question to the participant and asked for his or her response. Following a modified cognitive interviewing methodology,14 the interviewer then queried the participant about how he or she understood and responded to the target question. The participant then answered and discussed one of the other target questions. Participants who answered the frequency-type question first were asked the belief-type question second, and vice versa. Half of the patients who were asked the vague question first were asked the frequency-type question second; the other half were asked the belief-type question second.
The interview guide for one of the three protocols is provided as supplemental material to this paper. The two trained interviewers developed a consistent approach to the interviews through multiple mock interviews. Throughout the study, members of the study team reviewed audio recordings of the interviews to ensure the consistency and quality of the interviews, and the study team held regular meetings with the interviewers to identify emerging issues and challenges and to discuss consistent responses for handling them. The interviews lasted 30 to 45 minutes.
Expressing expectations of benefit as medians with interquartile ranges, we used Kruskal-Wallis tests followed by Wilcoxon rank sum tests to compare distributions of expectations among the three groups of participants. We also used Wilcoxon signed rank tests to compare expectations of benefit within subjects for participants who were randomly assigned to receive the frequency-type or belief-type question first.
We analyzed qualitative data from the transcripts using codes that we developed from themes identified a priori by the study team and modified in an inductive fashion to reflect themes that emerged from the data. Two members of the study team coded the interview transcripts independently and reconciled differences through consultation with the study team. The data presented here are limited to the justifications that participants gave for their responses.
Although each participant provided justifications for responses to two of the target questions, we restricted the analysis to justifications given after the first target question. To determine whether justification type was related to the magnitude of the expectation of benefit, we categorized participants into three groups on the basis of their expectation of benefit as lower (< 50), medium (50 to 70), and higher (> 70). These cutoffs were developed after the data were collected to generate groups with approximately equal numbers of participants and to reflect the special importance of the value of 50 to many participants (eg, answers below 50 were considered pessimistic and answers above 50 were considered optimistic for some participants in the study). We then performed cross-tabulations between category of expectation and dichotomous indicators of each justification type, using Fisher exact tests to evaluate whether category of expectation and each justification type were independent. Due to the small sample size, these analyses were considered exploratory.
For all analyses, we conducted sensitivity analyses in which we removed the small number of phase 2 trial participants from the sample. In all cases, the pattern of findings was the same. Therefore, we present results for the full sample.
We used SAS version 9.1.3 (SAS Institute Inc., Cary, NC) for all analyses.
Seventy-two patients were invited to participate in the study, and 45 (63%) consented. Forty participants (86%) had recently consented to participate in a phase 1 trial, and 5 participants (14%) had consented to participate in a phase 2 trial. Table 1 summarizes the characteristics of the study population.
Figure 1 shows the distribution of responses to the target questions regarding likelihood of benefit, based on the first question given. Participants expressed a wide range of expectations in response to all three question types (range, 5 to 100). Responses differed significantly depending on the question type (Kruskal-Wallis chi-square [2 df] = 7.5; P = .02). Expectations of benefit in response to the belief-type question were significantly greater than expectations of benefit in response to the frequency-type and vague questions (Wilcoxon P < .05 for all comparisons). Expectations in response to the frequency-type and vague questions did not differ from each other.
In within-subject comparisons, responses to the belief-type question were higher than responses to the frequency-type question. When the frequency-type question was asked first, the median difference was −10 (interquartile range, −20 to −10; P = .001). When the belief-type question was asked first, the median difference was 20 (interquartile range, 5 to 30; P = .008).
Some patients answered “don’t know” to one or both of the target questions about likelihood of benefit (see Table 2 for reasons for answering “don’t know”). Most of the “don’t know” responses (10/14) were given for the frequency-type question when it followed either the belief-type question or the vague question.
Participants employed a variety of justifications for their responses to the target questions (Table 2). Two collections of justifications were most frequent. The first set of justifications, given by 60% of the participants, involved expressions of positive thoughts. The second most frequent set of justifications, given by 51% of the participants, involved references to the participant’s physical health (including the participant’s health history or health relative to others). Only 7 participants (16%) said that their expectations of benefit were based on what they learned from their doctor or from the informed consent process.
The majority of the participants (64%) provided more than one type of justification. Of participants who provided more than one justification, the most frequently occurring combination (occurring among 11% of the participants) was of positive expressions and references to physical health.
Table 2 shows the distribution of justifications by the magnitude of expectation. The occurrence of positive expressions varied by magnitude, such that 85% of participants in the higher expectations group used positive expressions when asked for a justification, compared to 63% of participants in the medium expectations group and 27% of participants in the lower expectations group. As one would expect, the occurrence of ”fifty-fifty” justifications was more frequent for the category that included 50%. All other justifications, including references to physical health, were distributed equally across all magnitudes of expectation.
Consistent with previous research,6–9 we observed that many participants reported expectations of benefit that were much higher than the likelihood of benefit suggested by historical data from early-phase oncology trials. In past research, interpreting such findings has been challenging. In this study, however, we identified two important factors that should be considered in determining whether high expectations of benefit are signs of misunderstanding: (1) the type of expectation being queried and (2) what the patient intends to accomplish by providing a response.
Among participants who were asked to indicate their confidence that the experimental therapy would control their cancer, the median expectation was 80 (out of 100), compared to a median expectation of 50 among participants who were queried in terms of relative frequency. This pattern was the same for between-subject and within-subject comparisons. Seriously ill patients may have more invested in believing in the success of their personal outcomes compared to a population outcome. This finding may also reflect the “better-than-average effect” in social psychology, which refers to the tendency of people to rate themselves better than most others.16 Thus, anyone concerned about early-phase trial participants’ high expectations of benefit should be careful to determine how patients were asked about their expectations.
Differences in the way the target questions were worded corresponded to important differences between two types of expectations, known in the philosophy of probability as belief-type and frequency-type. (Kahneman and Tversky use the terms “singular” and “distributional.”17) High frequency-type expectations should generate greater ethical concern about patient misunderstanding than high belief-type expectations. Consider, for example, a patient who says, “I know that fewer than 5 out of 100 patients will benefit on average; furthermore, I am 90% confident that I will be one of the people who will benefit.” One could judge the correctness of the first half of this statement by comparing what the patient said to what the patient was told regarding aggregate response rates in past early-phase trials. It is less clear how one would judge whether the patient is correct regarding the second half of the statement. Therefore, care should be taken during the consent process to communicate probability information in terms of relative frequencies and to query patients’ understanding in frequency-type terms when assessing understanding.
Participants’ intuitive understanding of what types of responses could be justified for each question type may have led to expressions of greater expectation of benefit in response to the belief-type question. In other words, the belief-type question seems to ask patients for a feeling about their confidence, whereas the frequency-type question seems to ask patients to make a knowledge claim. Expressions of feeling or opinion might have appeared less vulnerable than knowledge claims to charges of inaccuracy, thus allowing some participants to express greater belief-type than frequency-type expectations. Additional research is needed to identify the role that belief-type expectations play in patients’ decision making and how these expectations correspond to the frequency-type information provided to patients.
The second major finding from this study concerns the justifications that participants gave for their responses to questions about likelihood of benefit. Typically, someone who asks patients to estimate the likelihood of benefit would like to assess the patients’ understanding. It is hoped that patients know this and so use their responses for the purpose of demonstrating their understanding. The justifications participants gave for their answers in our study suggest that this assumption is not always correct. If participants were trying to express their understanding of the likelihood of benefit, they might have justified their answers by, for example, citing some information conveyed during the informed consent process. We found, however, that very few participants cited such information to justify their responses.
This finding could be due to one of three possibilities: (1) participants did not receive information about the likelihood of benefit during the clinical trial consent process; (2) participants received information about the likelihood of benefit, but they did not understand what it meant and so generated their responses using other strategies; or (3) participants did receive and understand information about the likelihood of benefit, but they believed the interviewer was asking about something else or they elected not to incorporate that information into their responses. Regardless of the reason, it appears that few of the participants provided answers for the purpose of indicating their understanding of the clinical trial to which they had consented.
Rather than indicating their understanding, most participants provided answers that served as expressions of hope, positive attitude, and faith. Not surprisingly, this was especially the case with participants who expressed higher expectations of benefit. Participants explained their responses in terms of the utility and even the moral imperative of being optimistic. This finding strongly suggests that many participants used a question intended to elicit their understanding of the likelihood of benefit as an opportunity to give a rallying cry—to reassure themselves and others that they would succeed. The finding is consistent with an ethos common among patients with cancer and their care providers to cultivate a positive attitude and minimize doubt.18,19
Given that 85% of the participants who reported higher expectations (> 70) appeared to be fulfilling an obligation or desire to express a positive attitude, it would be unwise to interpret all expressions of high expectations of benefit as a misunderstanding about the nature of the trial, and hence a failure of the informed consent process. Our findings suggest that it is difficult to assess patients’ understanding of the likelihood of benefit without triggering patients’ need or perceived obligation to express something positive. Our research team is now conducting a prospective randomized trial based on the data presented here to evaluate strategies for assessing understanding in this context.
This study has several limitations. First, the participants were from two large academic cancer centers, so findings might not generalize to other settings. Compared to patients in other clinical settings, more participants in this study described themselves as white and non-Hispanic and reported higher levels of education and greater household income. It is unclear whether patients with other socioeconomic characteristics would express equally high expectations of benefit. Second, although 63% of the patients we contacted agreed to participate in the study, it is possible that the results do not reflect the expectations or the reasons for those expectations among patients who declined to participate. Nevertheless, our data can speak to the group of patients about whom bioethicists are most concerned—those who express high expectations of benefit. Third, we obtained the data through relatively brief interviews using interviewers who were not known to the participants. Participants might have expressed different expectations of benefit if the interviews had taken place over a longer period of time or if the interviewers had been people with whom the participants already had some rapport. However, any effort to enact standardized querying of the consent process would likely involve relatively brief assessments of patients’ understanding, possibly by someone unknown to the patients. Thus, while we might have obtained different data using different methods, we believe our approach is similar to what would be used to query patient understanding during an informed consent auditing process. Finally, the findings are based on a relatively small sample from a pilot study designed to inform a large-scale randomized trial that is currently underway.
These data represent the first in-depth exploration of expressions of high expectations of benefit by participants in early-phase clinical trials. Some researchers have assumed that patients are describing their understanding of the likelihood of benefit when expressing such expectations. Our findings suggest, however, that the wording of questions about likelihood of benefit shapes the expectations that patients express. Thus, investigators should disclose likelihood of benefit and query understanding in terms of relative frequency rather than belief-type probabilities. The findings also suggest that patients who provide the highest expectations of benefit may not do so to communicate their understanding, but rather to register optimism in the service of achieving good outcomes. Although this conclusion suggests that patients’ expressions of high expectations of benefit might not always be a significant ethical concern, more empirical work is needed. We are currently collecting data to clarify the meaning of high expectations in a larger sample and to examine methods for assessing understanding in this context. Future research is also needed to elucidate the consequences to patients of expressing high belief-type expectations in the presence of low frequency-type probabilities.
Financial support for this study was provided entirely by grants 1R01CA100771-01A2, 5R01CA100771-02, and 5R01CA100771-03 from the National Cancer Institute to Duke University (Dr Weinfurt, principal investigator).
We are grateful to the participants for sharing invaluable insights into their experiences with cancer and the clinical research process. We also thank Jennifer Millard of Fox Chase Cancer Center for assistance with project management; and Roger B. Cohen, MD, of Fox Chase Cancer Center for facilitating participant recruitment.
Presented in part at the 28th Annual Meeting of the Society for Medical Decision Making, October 16, 2006, Cambridge, Massachusetts; Society for General Internal Medicine 30th Annual Meeting, April 27, 2007, Toronto, Ontario; American Society of Clinical Oncology Annual Meeting, June 2, 2007, Chicago, Illinois; AcademyHealth Annual Research Meeting, June 3, 2007, Orlando, Florida; and 21st European Conference on Philosophy of Medicine and Health Care, August 18, 2007, Cardiff, Wales.