|Home | About | Journals | Submit | Contact Us | Français|
Annual lung cancer screening using low dose computed tomography (LDCT) scans is associated with a survival benefit, but is also associated with potential harm. Unlike descriptive probability formats, experienced tasks have been shown to decrease perceptions of rare events. The objective of this study was to compare descriptive versus experienced probability formats on patients’ knowledge, beliefs, endorsement of screening for heavy smokers, and preference (choice predisposition) to undergo screening.
276 patients attending an outpatient pulmonary practice were randomized to learn about screening using one of three formats: numbers only, numbers + icon arrays, numbers + a set of slides illustrating LDCT scans of 250 people in random order which displayed the number of normal scans, false positive lung nodules, cancers found leading to a life saved, and cancers found leading to death despite treatment.
Knowledge differed between the three formats (p= 0.001) with participants randomized to the numbers + icon array format having the highest knowledge score. Beliefs were more favorable among participants randomized to the numbers + experienced format compared to the numbers + icon array format [Difference between means (95% CI)= 1.6 (0.4–2.8)]. Differences in participants’ endorsement of screening (p= 0.4) and choice predisposition (p= 0.6) across probability format mirrored those of beliefs, but were not statistically significant.
Contrary to what we expected, the experienced format increased propensity toward screening compared to the numbers + icon array format as indicated by more favorable beliefs and nonsignificant trends towards stronger choice predisposition and endorsement. Experienced risk formats may not be a practical approach to improve risk communication for patients deciding whether or not to undergo annual lung cancer screening.
Lung cancer screening with annual low dose computed tomography (LDCT) scans are now recommended for persons 55 to 80 years of age who have a 30 or more pack-year smoking history and currently smoke or have quit within the past 15 years (1). Annual screening is associated with a survival benefit, but is also associated with potential harms, most notably overdiagnosis (2) and a high false positive rate (1–3). Therefore, effective communication of the probabilities associated with both the expected benefits and the harms of screening is essential to ensure that patients understand possible downstream effects, such as additional imaging tests and invasive procedures for benign lesions. In the Center for Medicare and Medicaid Services (CMS) memo addressing lung cancer screening for appropriate beneficiaries with LDCT published in February 2015 (4), a shared decision making visit is cited as a required criterion for coverage and must include at least one decision aid; however, CMS has not recommended a specific tool.
A large body of research has documented the difficulties associated with effectively communicating probabilistic information (5–7). In medicine, one of the most significant barriers to informed decision making is the tendency to overweight small probabilities. As a result, many studies have sought to develop improved methods to improve patients’ understanding of risk (8). Notably, Fagerlin et al. (9) recently outlined a series of recommendations to improve risk communication which included presenting absolute risks instead of relative risks and using icon arrays.
To date, all approaches used to improve patients’ understanding of probabilities use descriptive formats; however, a growing literature suggests that risk might also be effectively communicated though experience. Shaffer et al. (10) found, for example, that narratives focused on describing other patients’ experiences resulted in improved decision-related measures including confidence in the ability to make an informed choice and satisfaction with the decision process. However, whether testimonials result in an increased likelihood of patients making informed choices that are concordant with their values is not yet known (11). More recently, investigators have investigated the effects of experiencing probability distributions on decision making. Hertwig et al. (12) demonstrated that decisions made based on descriptions versus those made based on personal experiences lead to significantly different choices. Moreover, whereas description leads to overestimation of rare risks, experience generally results in the opposite effect as rare risks are seldom encountered.
The majority of the experiments documenting this effect have asked students to engage in gambling tasks where participants infer the expected probabilities of competing outcomes by performing repeated sampling tasks (13). Conversely, Tyszka and Sawicki (14) compared the impact of descriptive and experienced-based probability formats in a healthcare scenario. They asked female students to consider prenatal genetic testing, where they experienced risk by viewing a series of photographs of children with and without Down’s Syndrome. The authors found that worry about the genetic disease was lower among those exposed to the experienced versus description format. This effect may have been due to superior learning of risks or lack of exposure to infrequent risks.
Given this background, we sought to examine the effect of descriptive versus experienced risk formats among patients. Specifically, we compared the effects of three modes of risk presentation on patients’ objective knowledge, beliefs, endorsement of screening for heavy smokers, and personal preference to undergo lung cancer screening. Based on previous research, we expected that participants randomized to the experienced format would have greater knowledge scores, less favorable beliefs, and be less likely to endorse screening or to choose screening for themselves compared to those randomized to the descriptive formats.
Participants were recruited from an outpatient pulmonary practice during their reminder telephone call. All English-speaking participants, who were not being scheduled for follow-up of a pulmonary nodule and who did not have a history of lung cancer, were asked if they were willing to come to their follow-up appointment 30 minutes early in order to participate in the study. Upon arrival, the study was described in detail and verbal consent was obtained.
Participants were subsequently randomized to learn about lung cancer screening using one of three probability formats: numbers only, numbers and corresponding icon arrays, numbers and a set of slides illustrating LDCT scans (presented in the Appendix). The LDCT scans displayed the number of normal scans, false positive scans with benign lung nodules found (referred to as "false alarms"), cancers found leading to a life saved, and cancers found leading to death despite early identification and treatment, for 250 people in random order. Each slide was shown for one second for a total viewing time of four minutes. The transition time between slides was chosen based on feedback obtained from participants during pilot testing. While the National Cancer Institute’s “Patient and Physician Guide” related to lung cancer screening presents outcomes for a population of 1,000 persons (15) we chose to use 250 as the denominator for this study in order to decrease the time required for the experience task. We could not use “100” as the denominator because of the rarity of expected events and the need to present whole numbers. During the viewing period, Participants were seated in a quiet private room in front of a computer screen and directly observed by a medical assistant.
We measured objective knowledge using three questions: 1) If all 250 people get CT scans to screen for lung cancer: A. No one will die of lung cancer, B. A few people will still die of lung cancer, but most will be saved, C. Equal numbers of people will die of lung cancer as those who will be saved, D. Most people will still die of lung cancer, but someone will be saved (Correct response = D); 2) Compared to the number of people with lung cancers found by screening, how many people will have a false alarm: A. Many more, B. A few more, C. A few less, D. A lot less (Correct response = A); 3) Of all the people screened, about how many will have a false alarm? A. 5 in 100 (5%), B. 20 in 100 (20%), C. 35 in 100 (35%), D. 50 in 100 (50%) (Correct response = C). Forty-two participants (16.5%) answered the first knowledge question correctly, 143 (56.3%) answered the second question correctly and 98 (38.6%) answered the third question correctly. Given the ambiguity of the first knowledge question, only the latter two were used in the analyses. Scores were summed and treated as a continuous. Beliefs related to lung cancer screening were measured by summing the following four items measured on 5-point scales ranging from 1= strongly agree to 5= strongly disagree: “Screening is always the right thing to do because it catches cancer early” (reverse scored); “Screening may not be the right thing to do if there are too many false alarms”; “Screening is worth it even if it saves 1 in a 1,000,000 lives” (reverse scored); “Screening may not be the right thing to do if it leads to over diagnosis of spots on the lung that would not have caused any harm during a person’s lifetime”. Higher scores indicate more favorable beliefs towards screening.
We measured two types of preferences: 1) Endorsement of screening was measured by asking participants: Based on the information given to you, do you think heavy smokers should have screening LDCT scans? (Yes, No, Not Sure). No (n= 12) and Not sure (n= 23) were collapsed into a single group because the numbers of participants per category was too small to analyze separately. 2) Personal choice predisposition was measured using the following question rated on an 11-point numeric rating scale “What do you think right now about having a CT scan to screen for lung cancer each year?” anchored by “I am certain that I don’t want to be screened” and “I am certain that I do want to be screened”, with “Unsure” in the middle (16). Among participants who did not previously smoke, the question was preceded by: “Imagine that you were a heavy smoker”.
Perceived chance of developing lung cancer and worry related to developing lung cancer were also measured on 11-point numeric rating scales. Overall health status was measured using a 5-point scale ranging from Excellent to Poor. Objective numeracy was measured by using four items from a validated numeracy scale (17).
Analyses were conducted using SAS version 9.3. We used an analysis of variance to detect differences in continuous variable (knowledge, beliefs, choice predisposition) across the three probability formats with Tukey’s procedure to examine specific pairwise comparisons. The chi-square test was used to detect differences in endorsement across the three formats. T tests and Pearson’s correlation coefficients were used to detect associations between choice predisposition with categorical and continuous variables, respectively. We also performed exploratory analyses to examine whether probability format modified the relationship between knowledge, beliefs, worry about developing lung cancer and perceived chances of developing lung cancer with choice predisposition. Interactions where p <0.1 are reported. Choice predisposition was highly skewed and therefore log transformed.
We calculated that 246 participants were needed to detect an effect size of 0.20 or greater assuming, alpha= 0.05, beta= 0.80. A greater number of participants were enrolled (N= 254) because we did not cancel interviews once patients had agreed to participate over the phone prior to their interview.
Of the 431 invited to participate, 276 participants agreed to be interviewed. Two hundred and fifty-four arrived for their appointment, and all consented to participate. Eighty-four were randomized to view the numbers only format, 86 to view the numbers + icon array format, and 83 to view the numbers + slide presentation. The mean (SD) age of the study sample was 60.9 (8.8) years; 137 (54.2%) were woman, 88 (34.7%) were currently employed, and 143 (56.3%) were current or past smokers of whom 73 (28.7%) had a 30 or greater pack-year smoking history. Mean (SD) numeracy was 1.8 (1.0) (maximum possible score= 4). The demographic and clinical characteristics by probability format are described in Table 1. We found no statistically significant differences in patient characteristics across the three groups; however, the proportion of never smokers was higher among participants randomized to the numbers + icon array format.
Differences in knowledge, beliefs, endorsement of screening for heavy smokers, and personal choice predisposition by probability format are illustrated in Table 2. Average knowledge differed between the three formats (overall difference between means, p= 0.001). Knowledge was greater in the numbers + icon array and the numbers + experience formats when compared to the numbers only format [Difference between means (95% CI)= 0.5 (0.2–0.7) and 0.3 (0.01–0.6), respectively].
The percent of participants responding correctly to the second knowledge question was higher for those randomized to the numbers + icon array and numbers + experienced format compared to the numbers only format (Table 3). In contrast, the percent of participants responding correctly to the third knowledge question was higher only for those randomized to the numbers + icon array format compared to the numbers only format. Objective numeracy did not modify the relationship between probability format and knowledge overall or knowledge for any specific item.
Beliefs related to screening also differed by format (overall difference between means, p= 0.01), with the least favorable attitude towards screening observed in participants randomized to the numbers + icon array format (Table 2). Beliefs were more favorable among participants randomized to the numbers + experienced format compared to the numbers + icon array format [Difference between means (95% CI)= 1.6 (0.4–2.8)]. Objective numeracy did not modify the relationship between probability format and beliefs.
Smoking (whether measured as current, past or never smokers using dummy variables or by whether they met criteria for screening by pack years) was unrelated to either knowledge or beliefs and did not alter the associations between probability format and either dependent variable.
Differences in participants’ endorsement of screening (p= 0.4) and choice predisposition (p= 0.6) across probability format mirrored those of beliefs, but were not statistically significant (Table 2).
On average across probability formats, greater choice predisposition was correlated with more favorable beliefs (r= 0.4, p< 0.0001), increased worry about developing lung cancer (r= 0.2, p= 0.003), and greater perceived chances of developing lung cancer (r= 0.2, p= 0.0002); it was not associated with knowledge (r= −0.1, p= 0.3). The interaction between beliefs and probability format was borderline significant (p= 0.06) and that between worry and probability format was significant (p= 0.04). The correlation between beliefs and worry with choice predisposition within each probability format are described in Table 4. More favorable beliefs were associated with stronger choice predisposition in each format, but the association was somewhat weaker in participants randomized to the experienced format compared to the other two groups. Increased worry about developing lung cancer, on the other hand, was associated with greater choice predisposition among participants randomized to the numbers + icon array and numbers + experienced formats, but not among those randomized to the numbers only format. Probability format did not modify the positive relationship between perceived chance of developing cancer and choice predisposition.
In this study we compared the effect of probability format on patients’ knowledge, beliefs, and preferences related to lung cancer screening and found that the numbers + icon array and numbers + experienced formats were associated with higher knowledge scores compared to the numbers only format. The improvement in knowledge among participants presented with the addition of an icon array is consistent with prior studies demonstrating the benefits of using this format (9). Improved knowledge, however, was not associated with choice predisposition. Unlike some previous studies, we found no interaction between objective numeracy and format, suggesting that participants’ knowledge improved to a similar degree regardless of numeracy levels. We did not measure graphic literacy, which theoretically may also interact with probability format (18).
Contrary to what we expected, the experienced format increased propensity toward screening compared to the numbers + icon array format as indicated by more favorable beliefs, as well as nonsignificant trends towards stronger choice predisposition and endorsement. Previous studies have found that participants have lower estimates of rare events when presented with experienced risk compared to descriptive risk formats. Thus, we were surprised to find that propensity towards screening was more favorable among those randomized to the experienced condition compared to the numbers + icon array format, since we expected that these participants would better appreciate the very small number of patients who actually benefit from screening compared to the number who have normal or false positive LDCT scans. There are several possible reasons for our findings. First, the use of an advancing series of images may have actually lowered patients’ ability to appreciate the proportion of patients whose lives are saved by screening. This explanation is consistent with previous research demonstrating that interactive risk presentation formats can impede understanding of probabilistic information (19). Second, we assumed that normal LDCT scans would be associated with disutility (i.e. a wasted resource), however, patients may be reassured by having a normal test, and therefore may have viewed each normal LDCT slide as a positive outcome. If true, given that images are known to have a greater impact on decision making compared to descriptive text, the experienced task might have increased propensity for screening by reinforcing the positive reaction to a normal test result at least among some individuals. This possibility is supported by a recent qualitative study describing smokers’ misperceptions related to lung cancer screening including the belief that normal test results indicate that they are among the fortunate minority who will not experience harm from smoking (20). It would be interesting to examine how experienced formats affect patients’ perceptions related to the adverse events and benefits associated with treatment, because unlike in screening, the positive or negative utility of each outcome is clear.
We also assumed that the experienced format would enable patients to better appreciate the large number of false positives associated with LDCT. While the experienced task did increase the number of correct responses to the false alarm knowledge question (as compared to the numbers only format), this knowledge did not affect propensity towards screening. This finding may reflect a patient perspective that false positives are not important; this interpretation is consistent with the lack of anxiety, distress, worry, or health-related quality of life found in patients receiving false positive results after having undergone screening (21, 22).
In exploratory analysis, we found that worry about developing lung cancer was correlated with choice predisposition among participants randomized to the numbers + icon array and numbers + experienced formats, but not among those in the numbers only format. These results suggest that, although viewing the icon array or set of slides did not increase the mean level of worry among subjects relative to the presumably more abstract numbers only format (23), viewing either format may have motivated the use of affect in subsequent evaluations (24).
We chose to present participants with pictures based on our need to effectively communicate distinct outcomes while minimizing cognitive burden. Tyszka and Sawicki (14) used a similar approach, but showed participants familiar images (babies’ faces). In this study, we showed pictures of LDCT scans since these are the direct results of the screening process. All participants were familiarized with the images and their meaning prior to engaging in the experienced task. However, the present images were certainly less familiar than baby faces and may not have conveyed the meaning we intended. Although images can arouse greater emotional meaning than text (or numbers) (14, 25) and this greater arousal can promote greater learning (26), our unfamiliar scans may not have done so. Alternatively, the complex images may simply have increased the cognitive burden on participants and reduced their overall comprehension of the situation, only part of which was captured by our knowledge items (27).
It is difficult to compare our experiment with previous experienced risk studies since the latter have all included two possible outcomes, whereas in this study we asked patients to consider four different LDCT scans (depicted in the Appendix). It is possible that a simpler experienced condition may have led to different results. In addition, most previous experiments have required participants to actively sample from a baseline distribution as many times as they wished, whereas our task, like the one used by Tyszka and Sawicki (14), forced participants to view a single random distribution of all possible events, thus ensuring that each participant viewed the expected number of each outcome.
There are several limitations of this study which can be addressed in future research projects. We chose to run the study prior to patients seeing their physicians so as to avoid the influence of the visit on participants’ responses; thus, the length of the survey had to be limited in order not to interrupt clinic flow. Because there are no known validated knowledge or endorsement measures, both were developed for this study. Moreover, a greater number of knowledge questions may have increased the distribution of this variable and resulted in greater power to detect associations between knowledge and preferences. Our experience task, while mirroring the distribution of actual outcomes associated with lung cancer screening, may have been overly complex, and our results may not generalize to simpler tasks asking participants to consider fewer discrete events. Simpler experiments, however, may be of limited use in screening decisions where multiple outcomes usually need to be considered. The value of the present approach may also be constrained by respondents’ ability to attend to an experienced condition with a large denominator. For this reason, we presented patients with a distribution of LDCT outcomes in a population of 250 (as opposed to a population of 1000 used in the National Cancer Institute’s Patient and Physician Guide from the National Lung Cancer Screening Trial (15)) (1, 3); however, our task still required participants to attend to 250 power point slides over four minutes. Participants in the numbers only and numbers + icon array groups did not view LDCT scan images and thus we did not control for the influence of seeing the images on participants’ responses or of the longer time required to view the slides compared to see information in the other two formats. In addition, because our primary hypothesis was to compare experienced versus descriptive formats, we included participants who were not eligible for lung cancer screening; however, patients’ approach to even a hypothetical scenario may differ depending on their eligibility for screening and the perceived relevance of the task.
In this study, an experienced risk format was not superior to an icon array in improving participants’ knowledge about a subset of outcomes related to lung cancer screening. Moreover, we found that contrary to expectations, the numbers + experienced format may have increased propensity towards screening compared to the numbers + icon array format and particularly with respect to positive beliefs about screening. Given what we have learned from this study, future research examining the impact of experienced formats should 1) focus on health-related decisions involving more frequent outcomes to avoid large denominators requiring lengthy experience tasks, 2) consider choices for which patients do not hold strong beliefs (as they do for decisions such as cancer screening and vaccines), and 3) examine choices in which the direction of utility related to each outcome is clear-cut.
The authors would like to thank Amanda Kahn, Darly George and Adam Oelberg for their time and effort screening potential subjects, performing data acquisition and data entry.
Financial support for this study was provided in part by NSF grant SES-1047757 to the second author and the National Institute of Arthritis and Musculoskeletal and Skin Diseases, part of the National Institutes of Health, under Award Number AR060231-01 (Fraenkel). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
Information Sheet Shown to Subjects
Some people choose to be screened for lung cancer. They do this by having a chest CT scan each year.
A chest CT scan is a type of x ray. It creates more detailed pictures of your lungs compared to standard x-rays.
There are pros and cons to getting screened for lung cancer.
Of every 250 people (who smoke at least 1 pack per day for 20 years) who get screened for lung cancer:
Information Sheet Shown to Subjects Randomized to the Numbers + Icon Array
Following the initial page describing numbers, subjects were provided with the following information:
If these 250 people undergo screening, this is a chart of what happens (each square represents one person):
Information Sheet Shown to Subjects Randomized to the Numbers + Experience
Following the initial page describing numbers, subjects were provided with the following information. They then viewed the power point presentation.
We will now show you what happens when these 250 people get CT scans for lung cancer screening.
You will see 4 types of CT scans:
Liana Fraenkel, Department of Medicine, Yale University School of Medicine, New Haven, CT.
Ellen Peters, Department of Psychology, Ohio State University, Columbus, OH.
Shea Tyra, Department of Medicine, Western Connecticut Health Network, Danbury, CT.
David Oelberg, Department of Medicine, Western Connecticut Health Network, Danbury, CT.