We conducted two randomized mail surveys that included a hypothetical scenario among doctors and among patients recently discharged from hospital. The doctor survey primarily explored doctors’ opinions about policy issue and was approved by the Research Ethics Committee of the University Hospitals of Geneva19,20
; the patient survey, primarily a patient satisfaction survey, was exempted from full review21,22
The doctor survey included all active clinicians in canton Geneva, Switzerland, and was conducted between November 2007 and February 2008. Doctors were identified through the registries of the Geneva Medical Association and University Hospitals of Geneva. Duplicate records, invalid addresses and doctors who did not work with patients were excluded. This left 2746 eligible doctors. The survey response rate was of 56.3% (1546/2746). Participation was not related to age, setting of practice and source data base, but differed by sex (58.0% in men vs. 53.7% in women, p
0.027) and specialty (from 52.6% in technical specialists to 62.2% in primary care doctors, p
The patient survey took place between November 2005 and February 2006. It included all adult patients discharged to their home during a one month period. No clinical data were available, nor any information about the relevance of the study scenario for the patient. We excluded patients who were transferred to another facility, did not live in Switzerland, or reported not speaking French or being incapable of filling a questionnaire. The core of the questionnaire was the Picker patient opinion survey. The response rate was 65.0% (1432/2204).
The scenario described a hypothetical clinical trial in which a new treatment provided a survival benefit over the old treatment, but caused more digestive side-effects (Box). Four basic risk formats were used in both surveys: 1) survival proportions, 2) mortality proportions, 3) relative mortality reduction, and 4) all three presentations of risk. The latter was considered to be the fully informed condition. The respondent was asked how the new treatment compared with the old treatment.
We tested two additional risk formats in doctors only: 5) the number needed to treat, and 6) the relative survival extension. These formats did not work well with patients during pretests, and we renounced their use in this sample. Furthermore, we called the viral disease HIV infection in the doctor scenario, but not in the patient scenario, to avoid singling out a specific patient group. The study was called a “multicenter clinical trial” in the doctor version, but just “study” in the patient version. We mentioned statistical significance in the doctor version because that question arose during pretests, but not in the patient version.
Presentations of treatment benefits as absolute survival, absolute mortality, relative mortality reduction, and number needed to treat do not require elaboration. Computation of the relative survival extension was based on the assumption of a constant mortality rate (i.e., exponential model). Under this assumption the expectation of survival time equals the inverse of the mortality rate23
; thus if the mortality rate with the new drug is 2/3 of the mortality rate with the old drug, the expected survival on the new drug will be 3/2 the expected survival on the old drug, i.e., an increase of 50%. Of note, in the other versions of the scenario, we reported risks rather than rates, but because risks were low, the ratios of hazards and risks were practically equivalent (an exact computation yields a survival extension of 51.2% for this case; we used the rounded figure of 50% for simplicity).
Sample Size Determination
We sought to detect a difference in positive assessments of 60% versus 75%. With a type 1 error at 5% and a desired power of 90%, 220 observations per group were necessary.
We compared the distributions of the 5-level assessment across versions of the scenario, and compared them using a chi-square test for linear trend, separately in patients and in doctors. For most analyses, we dichotomized the assessment as favorable (“much better” or “somewhat better”) versus other, and used chi-square tests to compare version of the scenario, and doctors to patients within each version. Multivariate modeling was conducted with logistic regression. The reference level was the comprehensive information condition, and differences between this and other risk frames were interpreted as framing bias. The models were replicated using ordinal logistic regression with the original 5-level assessment as dependent variable, but because the results were virtually identical, we report binary logistic regression results. To compare the effects of risk framing across subgroups we used interaction terms; e.g., to compare men and women, the model predictors included risk formats , sex, and the sex*formats interaction.