|Home | About | Journals | Submit | Contact Us | Français|
The foundation of precision medicine is the ability to tailor therapy based upon the expected risks and benefits of treatment for each individual patient. In a prior study, we implemented a software platform, ePRISM, to execute validated risk-stratification models for patients undergoing percutaneous coronary intervention and found substantial variability in the use of the personalized estimates to tailor care. A better understanding of physicians’ perspectives about the use of individualized risk-estimates is needed to overcome barriers to their adoption.
In a qualitative research study, we conducted interviews, in-person or by telephone, with 27 physicians at 8 centers that used ePRISM until thematic saturation occurred. Data were coded using descriptive content analyses.
Three major themes emerged amongst physicians who did not use ePRISM to support decisions-making: 1) “Experience versus Evidence,” physicians’ preference to rely upon personal experience and subjective assessments rather than objective risk estimates; 2) “Omission of Therapy,” the perception that the use of risk models leads to unacceptable omission of potentially beneficial therapy; and 3) ”Unnecessary Information,” the opinion that information derived from risk models is not needed because physicians’ decision-making is already sound and they already know the information.
Barriers to the use of risk models in clinical practice include physicians’ perceptions that their experience is sufficient, that models may lead to omission of therapy in patients that may benefit from therapy, and that they already provide good care. Anticipating and overcoming these barriers may improve the adoption of precision medicine.
The practice of personalized, evidence-based medicine requires tailoring treatment to the risk of individual patients.1, 2 Risk prediction models are an efficient mechanism for prospectively defining an individual patient’s risks and benefits of a specific treatment. For example, the TIMI and CHADS2 risk scores3–5 are commonly used to guide decision-making in patients with acute coronary syndrome and atrial fibrillation, respectively, and have been incorporated into national guidelines as standards of care.6, 7 Despite their potential to improve healthcare, the routine incorporation of risk-stratification tools in clinical care and medical decision-making is the exception, rather than the rule.8–12 For example, while validated individualized risk models for bleeding after percutaneous coronary intervention (PCI)13 are available, a “risk-treatment paradox” exists, whereby bleeding avoidance strategies (BAS) are used least often in patients at high bleeding risk, who are most likely to benefit.14 Similarly, the risk of restenosis and the need for target vessel revascularization (TVR) following PCI can be predicted,15 yet evidence suggests that the use of drug eluting stents (DES), which reduce the risk of TVR, has little correlation with patients’ underlying restenosis risks.16
We recently reported the results of a 9-Center study providing individualized mortality, bleeding and restenosis risk estimates to physicians prior to PCI. The ePRISM tool had a favorable impact on patients’ satisfaction with the informed consent process and was associated with more rational bleeding avoidance strategies and reduced bleeding, but had no impact on DES use.17–19 Importantly, marked variability in BAS and DES use was observed across physicians, suggesting that many physicians did not incorporate the risk models into their medical decision-making, even when they were routinely available. To better understand the potential barriers to integrating risk models into routine clinical care, we invited physicians at the 9 sites to participate in a qualitative research study to describe physician-level barriers to the use of individualized risk estimates in medical decision-making. Understanding these barriers can inform the design of future interventions to support the use of precision medicine in routine practice.
A multidisciplinary team comprised of an interventional cardiologist, a nurse researcher, a cardiac nurse, an anthropologist, and a psychologist conducted this qualitative study between July 2011 and February 2012 using semi-structured interviews with interventional cardiologists. The study was approved by Saint Luke’s Hospital institutional review board.
As one of the nine centers participating in the original study opted-out of this qualitative study, invitations to participate in this study were sent to every interventional cardiologist at eight U.S. PCI centers with access to ePRISM (Appendix A). A comprehensive sampling strategy was used to seek the broadest range of physician experience and opinions regarding the barriers to adoption of risk models into clinical practice. Accordingly, interventional cardiology fellows were also invited to participate because, in many institutions, these trainees are primarily responsible for the PCI informed consent process.
To prepare for these interviews, a literature search was conducted to review studies exploring factors documented to influence physicians’ practice patterns and clinical decision-making. From this review, a 7-item semi-structured interview guide was developed to explore physicians’ experience with the ePRISM-generated risk estimates, their incorporation of risk-stratification into their decision-making process, their perceptions of the barriers to using risk-stratification models, and their perception of the value of the risk model outputs (Appendix B). Five initial interviews were conducted in-person and were transcribed, coded, and analyzed for a preliminary evaluation of the interview guide. Minor adjustments were made to the guide and all subsequent interviews were conducted by telephone.
A single experienced interviewer and anthropologist (BG) conducted all interviews, which were recorded and transcribed verbatim. Interviewees were offered a modest honorarium for participation. Each transcript was reviewed for accuracy while listening to the recorded interview. The research team coded transcripts and determined that thematic saturation20 and redundancy,21 the point in data review and analysis when no new information emerges with additional data collection, were achieved after 25 interviews.21, 22 Two additional interviews were conducted to confirm that no further information was gained,23 resulting in 27 completed interviews averaging 26 minutes (range 13–48 minutes) in duration.
Each transcribed interview was coded by at least two members of the research team. Coder order and pairings were varied to reduce the potential for inter-rater bias and improve reliability and trustworthiness.24, 25 Through consensus, a coding taxonomy was agreed upon, further reducing the data to the most meaningful insights. Descriptive content analysis was used to search for patterns and themes that occurred frequently in a single interview or across interviews and to develop categorical codes.26
We followed the following five-stage data analytic framework to ensure study soundness:27 1) thorough familiarization/immersion with the interview transcripts, 2) identifying a thematic framework distinguishing the critical issues, concepts, and themes derived from the transcripts, 3) indexing the thematic framework to each transcript, as applicable, 4) charting the raw data based on the index categories and thematic framework, and 5) mapping and interpreting the charts of organized data and applying it to the original research objectives to examine divergence and convergence. Importantly, an interventional cardiologist was part of the coding team (member checking) to obtain feedback and reduce the possibility of researcher bias or faulty clinical logic, thereby improving the trustworthiness of the data.
The Outcomes of Percutaneous Coronary Intervention Study (OPS) was supported by an American Heart Association Outcomes Research Center grant (0875149 N), and the Personalized Risk Information Services Manager (PRISM) study was supported by a grant from the National Heart Lung and Blood Institute (R01-HL096624). In addition, Drs. Chhatriwalla and Spertus receive research support from the Patient-Centered Outcomes Research Institute [CE-1304-6448]. Financial support was also provided by Saint Luke’s Hospital Foundation. The funding agencies had no role in data collection, analysis, interpretation, or the decision to submit the results. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the paper and its final content, which does not necessarily represent the official views of the funding agencies.
All physicians interviewed confirmed that they had clinical experience with PRISM informed consent documents. Twenty-seven of fifty-seven physicians consented to participate in this study, and were interviewed, either in-person (n=5) or by telephone (n=22). Of these, 3 were fellows-in-training, 2 were female and their average interventional experience was 13 (range = 0.5 – 30) years.
Qualitative thematic analysis revealed three salient themes regarding barriers to the use of personalized risk estimates in clinical decision-making: 1) Experience versus Evidence, 2) Omission of Therapy, and 3) Unnecessary Information. Physicians did not readily identify facilitators to the use of ePRISM. Representative quotes for each theme (Table 1) are identified by the site letter and individual study identification number, followed by the physician’s years of experience performing PCI.
Many physicians reported a reluctance to incorporate risk estimates into their decision-making and preferred relying upon their clinical judgment, based upon their experience and training: “Some physicians think that they’ve been doing this for years and years and years and they don’t need someone else’s tool …” (Physician #20, Hospital F). Some physicians suggested that more experienced physicians would rely on their own judgment, while less experienced or less competent physicians might benefit from using risk estimates: “My feeling is that these numbers are more clung to by the insecure or the inexperienced looking for justification of doing X rather than Y. But the more experienced and knowledgeable you are, you may choose to do X or Y partly based on the risk predictor, but also based on all of those other variables that unconsciously or consciously work their way into your clinical decision-making process” (Physician #23, Hospital H).
Several physicians also questioned the accuracy and validity of the models. One concern was the limited number of variables incorporated by the risk models: “I want to be rational, I really do, but I don’t think it captures all the nuance that’s required in stent selection.” (Physician #2, Hospital B). Physicians also questioned the generalizability of risk estimates to a specific patient being treated: “because the models are based on these populations of patients which don’t necessarily apply to the patient that you’re taking care of … That’s for a patient population, and I think I can do better tailoring than that for this particular person.” (Physician #3, Hospital B)
The perception that risk modeling supports rationing of medical care, or the avoidance of therapy in low-risk patients, was a second important theme. The use of risk models could thus lead to omission of potentially beneficial therapy: “I think physicians have a harder time withdrawing therapies than adding an effective therapy…So I guess what I’m telling you is that the biggest barrier to not using the restenosis [model] is I’m not aligned with the strategy – and that is to withdraw therapy and cause a certain number of renarrowing of the vessels.” (Physician #4, Hospital B). Another physician explained, “Restenosis is never higher with a drug eluting stent, never. So … why wouldn’t you put the Cadillac in everybody?” (Physician #3, Hospital B). Other physicians said, “At the end of the day, you want to do the best thing you can do for each and every patient, not just the high risk patients,” and “… my take on it is why wouldn’t you just do everything you can to avoid a bleed in everyone. And then the same thing with restenosis.” (Physician #24, Hospital H).
Many physicians in our study did not perceive the use of risk estimates as valuable (to physicians) because they felt that they already knew the information. In contrast to the first theme, in which they questioned the accuracy of the risk estimates, this concept reflected their perceptions that the information, even if accurate, just wasn’t valuable. One physician noted, “I don’t need [those] data to tell me what I already know … to me this is more for the patient’s education, not for me. I already know this.” (Physician #18, Hospital A). Many physicians also believed that there was no room for improvement in their medical decision-making: “I always try to minimize the bleeding risk regardless of what the person’s risk is up front … And to see a number spelled out doesn’t really help me much in terms of what I would do.” (Physician #19, Hospital D). Another physician stated: “The typical phrase you hear from operators is that I already know that information. That information is already in my head. Why do I need that form to tell me what to do?” (Physician #6, Hospital C). Some physicians further volunteered that their egos prevented them from using the risk models. “We, as physicians, obviously are notorious for being egotistical.” (Physician #21, Hospital A).
Clinical risk models have the potential to support personalized, or precision, medicine by exploiting the heterogeneity of treatment benefit to show the expected benefit of therapy for a specific patient1, 2 and can serve as the foundation for value-based healthcare.28, 29 While prospective risk-stratification is a rational strategy, little effort has been expended to evaluate how clinicians might use such data in their routine clinical practice. In this qualitative study, we found that some physicians are reluctant to incorporate statistical risk estimates into clinical decision-making, and prefer to rely on their clinical assessment and experience to make decisions about treatment. We identified three major physician-level barriers to the incorporation of individualized risk estimates into clinical decision-making for patients undergoing PCI that deserve deeper consideration if they are to inform future interventions to improve the adoption of personalized, evidence-based medicine into routine clinical care.
The “Experience versus Evidence” barrier suggests that physicians believe that they are better able to estimate risks and treatment benefits than risk prediction models. Dawes, et al., describe this phenomenon as a conflict between clinical and actuarial decision-making.30, 31 Clinical decision-making involves processing information mentally, while the actuarial method minimizes the human judgment factor and emphasizes empirically established relationships between data and the outcomes of interest.32 A large body of research documents clinicians’ preference for relying on personal experience over statistical risk estimates to make treatment decisions.33 However, without evidence-based or research-driven probabilities available, clinicians can have difficulty distinguishing among valid and invalid associations between variables and the outcome of interest. In fact, comparative studies of the two approaches have consistently suggested greater overall accuracy when clinicians rely on actuarial information over anecdotal evidence in treatment decisions.33–35 Kahneman, the Noble Prize-winning psychologist, describes decision-making as being comprised of 2 systems, a fast-thinking, intuitive system that rapidly makes decisions based on associations with past experiences, and a slow-working, logical, reasoning system that incorporates evidence to make a more rational decision.36 The first, fast-thinking system is much easier and less stressful to use and is vastly preferred over the more deliberate second system. Similarly, our study suggests that even when risk estimates derived from validated prediction models are provided at the time of clinical decision-making, physicians do not feel that this information should supersede their intuitive assessment of patients’ risk and benefits from treatment.
Physicians also voiced concerns regarding the accuracy and generalizability of risk prediction models. There is legitimacy to these concerns, and they underscore the need to continually develop, refine and prospectively validate risk prediction models, especially as therapeutic options change over time. Furthermore, assurance that a risk prediction model is both statistically significant and clinically relevant is essential to its integration. When constructed from large datasets, risk models can be informative when applied to individual patients. However, a model need not be able to predict each patient’s outcome with certainty to be clinically useful, especially when considering that such tools are meant to support decision-making and not replace physician judgment. In fact, the use of the bleeding risk prediction model in this 9-center study was associated with a substantial reduction in bleeding as compared with care delivered at the same institutions prior to the risk model results being available.17 Drs. Groopman and Hartzband have suggested that "Intuition is powerful and necessary, but if you just rely on that, you're going to get it wrong."37 Physicians might be more open to prospective risk-stratification if they recognize the benefit of prospective risk-stratification as a complementary tool to aid decision-making, as opposed to serving as a substitute for physician judgment and view the incorporation of risk prediction models as “Experience plus Evidence” instead of “Experience versus Evidence.”
The “Omission of Therapy” barrier refers to the sentiment that the omission of a potentially beneficial medical therapy is not acceptable, even in low-risk patients, and that over-treatment is preferable to under-treatment. Indeed, cardiologists’ overestimation of the benefits of PCI and their regret for not performing PCI if an adverse event could potentially have been avoided have been previously reported.38 Minimizing the potential risks of a therapy may also contribute to this theme. For example, while DES reduce the risk of restenosis and TVR following PCI, the magnitude of this benefit and the cost-effectiveness of DES are highly dependent on patients’ underlying TVR risk.16 Furthermore, the use of DES necessitates prolonged dual anti-platelet therapy (DAPT),39 which can increase patients’ drug costs and the risk of major and minor bleeding complications.40 The need for DAPT can also result in delay of unforeseen procedures or surgeries until DAPT can be safely discontinued, given the fact that premature discontinuation of (or noncompliance with) DAPT is associated with increased risk of stent thrombosis, myocardial infarction, and death.41 In the United States, while DES are used in approximately 83% of patients at high risk for TVR, they are also used in approximately 74% of low-risk patients. We have previously reported that a 50% reduction of DES use in low-TVR-risk patients could lower US health care costs by $205 million annually, while increasing the absolute rate of TVR by only 0.5%.16 Given the competing benefits and costs of DES, some patients may logically prefer a BMS over a DES, especially if their risk of TVR is low. Currently, however, only 16% of patients treated with a DES report being asked about their treatment preferences.42 The best treatment is the one that best suits the individual patient, and that may not necessarily be the “Cadillac” treatment. Physicians might be more open to using prospective risk-stratification tools if they were viewed as supporting “Precision Medicine” rather than “Omission of Therapy.”
The “Unnecessary Information” barrier refers to the belief that individualized risk estimates lack value (to physicians) because their decision-making is already sound and they know patients’ risks without having them explicitly calculated. However, the use of bleeding avoidance strategies during PCI represents a specific example in which prospective risk-stratification can serve as an aid to decision-making and support more rational use of BAS in the patients most likely to benefit, thus improving the safety and cost-effectiveness of PCI. Bleeding complications following PCI are both predictable13 and modifiable,14 and it has been shown that BAS are most effective (and most cost-effective) in patients at the highest risk for bleeding.43 Yet, a risk-treatment paradox exists, whereby BAS are most often used in patients at low risk for bleeding complications, and least often in patients at high risk, suggesting that clinicians do not intuitively understand the risks of bleeding in their patients.14 The value of prospective risk-stratification is apparent in several recent studies in which the incorporation of bleeding risk estimates into clinical practice was associated with significant changes in the use of BAS during PCI, and a reduction in bleeding.17, 44, 45 However, despite numerous reports documenting logical inconsistencies between usual care and the logical applications of clinical evidence,14, 46 there does not seem to be a sense by clinicians that this applies to their practice. In fact, prior studies have noted that while cardiologists have a good knowledge of the medical literature, their use of PCI is not always evidence-based.38, 47 These findings suggest a pressing need to alter the culture of medical practice. If physicians feel more responsible for providing the most evidence-based and cost-effective care to each of their patients, they might embrace tools, such as ePRISM, to help them achieve this aim.
The physician-level barriers identified in our study are consistent with prior insights into the challenges of improving healthcare. What is particularly compelling is that these themes appear to be broadly applicable to physicians’ strategies and approaches to medical decision-making in general, not just during PCI.48 A better understanding of evidence-based medicine may help to overcome the barrier of “Experience versus Evidence,” and therefore, these findings may identify a need to begin education about systematic approaches to risk-stratification and decision-making during the formative medical school and residency years. The “Omission of Therapy” barrier suggests that some physicians have little or no interest in cost effectiveness, which can be a major impediment to improving the value of health care. Given the current climate of increasing healthcare costs and the evolving focus on improving quality, optimizing outcomes and reducing costs, the lack of participants' value-sensitivity is notable and physician barriers to cost-effective health care need to be proactively addressed. However, if physicians do not believe that allocation of resources to provide cost-effective care is their responsibility, then such allocation decisions may need to be made on an administrative level. To overcome the barrier of “Unnecessary Information,” it may be beneficial to provide physicians with evidence-based treatment protocols focused on patient-level risk, along with feedback reports and financial or other incentives to reinforce evidence-based practice. Continued research is needed to identify effective strategies to promote evidence-based decision-making, and improve the quality and cost-effectiveness of health care in the United States.
Our findings should be interpreted in the context of the following potential limitations. First, qualitative research is intended to provide deeper insights and contextual understanding of observed behavior. The generalizability of the themes elicited in our study is uncertain; however, this study recruited physicians from 8 centers throughout the country and who had a broad range of clinical experience. Second, participation in this study was limited to volunteers and as such, the possibility of selection bias must be considered. There may be additional themes that could have been elicited from providers who were unwilling to be interviewed for this study. While we achieved saturation of elicited content from these volunteers, the inclusion of more physicians may have provided alternative perspectives. Third, this study primarily involved physicians at large U.S. medical centers, which may differ from other hospital settings in terms of their use of and receptiveness to actuarial risk estimates in treatment decisions. Finally, due to the nature of this qualitative research study, we are unable to quantify the relative importance of the identified barriers to the use of risk models in clinical care.
Many physicians prefer to rely upon subjective assessments, based upon recall and experience, to guide treatment decisions, rather than objective risk estimates. Novel strategies to enhance the integration of predictive risk estimates into clinical care are needed to support evidence-driven, precision medicine for the US population. Persuading clinicians that risk models are not only statistically significant, but also clinically relevant, may support the incorporation of risk estimates in clinical decision-making.
|Institution||Number of Respondents||Duration of Experience|
|Approximate Annual PCI|
|Bay State Medical Center, Springfield MA||3||1 year||1300|
|Henry Ford Hospital, Detroit MI||2||6 months||800|
|Integris Heart Hospital, Oklahoma City OK||2||1 year||1200|
|Kaiser Permanente, San Francisco CA||5||6 months||1300|
|Prairie Heart St John’s Hospital, Springfield IL||5||1 year||1600|
|Saint Luke’s Mid America Heart Institute, Kansas City MO||5||4 years||800|
|Washington University Barnes-Jewish Hospital, Saint Louis MO||2||1 year||1700|
|Yale New Haven Hospital, New Haven CT||3||11 months||1400|
The following author disclosures are provided for the previous 3 years and the immediate future.
Drs. Decker, L. Garavalia, B. Garavalia, Yeh, and Chhatriwalla – none
Dr. Spertus – equity interest in Health Outcomes Sciences
Ms. Gialde – consulting honoraria from Health Outcomes Sciences