|Home | About | Journals | Submit | Contact Us | Français|
We hypothesized that recent experience and specialty choice would affect physician compliance with evidence-based guidelines.
In a series of computer-simulated encounters, participants weighed the risk of spontaneous abdominal aortic aneurysm (AAA) rupture against the risk of perioperative death to determine timing for elective repair. Guideline recommendations and statistical information on the risks of rupture and surgical death were provided.
Physicians at the annual meetings of the American Geriatrics Society, American College of Surgeons, and American Society of Anesthesiologists.
Before the actual simulation, each participant was randomly exposed to one of three experiences: death during watchful waiting (WWD), perioperative death (PD), or successful outcome (SO).
Compliance with recommended guidelines for AAA treatment.
Against guideline recommendations, 67% of geriatricians, 74% of anesthesiologists, and 77% of surgeons, chose surgery when the rupture risk was lower than the risk of perioperative death(p<0.05). Surgeons exposed to the WWD experience chose surgery significantly earlier than if they were exposed to a PD or SO experience (p<0.001). Anesthesiologist choices did not differ with recent experience.
Geriatrician decisions more closely followed guideline recommendations for AAA management than those of two other specialties typically involved in AAA care. Surgeons were most affected by a prior WWD, geriatricians next, and anesthesiologists least. Geriatricians referring patients for AAA surgery should be aware of specialty-specific differences in perioperative decision behavior.
Recent attempts to improve patient care and decision accountability have focused attention on clinical trials as the most appropriate basis for clinical decisions1. Physicians, however, frequently do not make decisions consistent with evidence-based guidelines2. Examples of guideline noncompliance include prescribing asthma medications3, managing ICU sedation4, and screening for diabetes5.
Although the reasons why physicians deviate from guideline-recommended behavior are incompletely understood, cognitive biases such as framing (treating gains and losses differently), and representativeness (overestimating the likelihood of highly memorable events) likely contribute6,7. Physician emotions may also play a role. We have previously demonstrated that a simulated exposure to spontaneous abdominal aortic aneurysm (AAA) rupture and death caused both vascular surgeons and older laypeople to choose surgical repair earlier in subsequent simulations8,9. Such a finding suggests that physicians may partly base clinical decisions on outcomes of previous patient care experiences.
Physician specialty training may also affect decision behavior. For example, resource utilization and clinical outcomes in the care of joint pain differ between rheumatologists, orthopedists, and general internists10. To examine the effect of recent experience and specialty training on clinical decision making, we recruited geriatricians, surgeons, and anesthesiologists to participate in a computer simulation focused on the timing of elective AAA surgery. These specialties were chosen because of their different roles in AAA care. Geriatricians typically participate in diagnosis and surgical referral. Surgeons decide when to operate, perform the surgery, and contribute to postoperative care. Anesthesiologists participate heavily in both intra- and postoperative care. We hypothesized that all specialties would conform to provided guideline recommendations regarding operative timing, and that no effect of specialty or recent experience on decision behavior would exist.
Physician participants were recruited during annual meetings of the American College of Surgeons (October 8–12, 2006; Chicago IL, attendance: 8,372), the American Society of Anesthesiologists (October 13–17, 2007; San Francisco, CA, attendance: 8,029), and the American Geriatrics Society (April 30–May 4, 2008; Washington DC, attendance: ~1,300). At each meeting, the experiment was conducted in the exhibition hall or next to the presentation rooms. All societies consented to our presence. In exchange for participating, physicians were offered refreshments. Simulation performance was incentivized by a charitable donation made by the research team. The Social & Behavioral Sciences Institutional Review Board at the University of Chicago approved the study protocol.
To simulate a slowly expanding AAA, we adapted the Balloon Analog Risk Task, which was originally designed to assess the propensity for risk-taking11. To account for specialty-specific differences in the physician-patient relationship, case descriptions differed slightly. Geriatricians “referred” patients for surgery, surgeons “chose” surgery for the patient, and anesthesiologists “recommended” that patients undergo surgery. All other wording was identical (Appendix S1).
Participants began the simulation by reading a brief synopsis of current Society for Vascular Surgery guidelines regarding the timing of open surgical repair of an asymptomatic AAA12. The program then simulated an encounter with a patient recently diagnosed with an asymptomatic, 3.7cm AAA via CT scan. Participants could not proceed until they had correctly answered a question verifying that the synopsis had been understood (Appendix S1).
After presenting a brief history and physical, the computer then displayed a black circle simulating an axial CT image of the aneurysm. In addition, the AAA size (cm), updated statistical risk of spontaneous rupture over the next 6 months (%), and risk of perioperative mortality (5%) were provided (Appendix S2). Two decisions were offered: “continue watchful waiting” and “go to surgery”. Clicking the “watchful waiting” button simulated sending the patient home with instructions to return in 6 months. Clicking the “surgery” button simulated a decision to operate and produced a surgical outcome statistically consistent with published literature (5% mortality and 95% success12).
Each time the participant chose watchful waiting, the black circle on the screen would enlarge (to simulate AAA expansion over the 6 month interval), and all statistical information would update. If the AAA ruptured, the words “AAA RUPTURE! PATIENT DIED” would appear. If the participant chose operation and a perioperative death occurred, the words “PATIENT DIED” would appear. If the repair was successful, the words “SUCCESSFUL SURGERY” would appear (Appendix S3). The simulation ended when either the participant chose to operate or the AAA ruptured. Participants thus made a series of “watchful waiting for 6 more months” versus “surgery now” decisions telescoped into several minutes of simulation.
The AAA expansion rate, rupture risk statistics, and decision algorithm used in the simulation were based on current Society for Vascular Surgery guidelines12–15. The guidelines recommend delaying surgical repair until the AAA expands to a diameter of 5.5cm when the risk of rupture surpasses the average operative mortality risk (5%).
In exchange for participation, a small donation was made on behalf of the participants to the American Geriatrics Society Foundation for Health and Aging (geriatricians) or Doctors without Borders (surgeons and anesthesiologists). $1.00 was guaranteed, and $1.00 was added for each decision to continue watchful waiting that did not result in death. A donation beyond $1.00 was made only if the patient successfully underwent surgery. The donation structure was described during subject recruitment and the simulation.
To test the effect of recent experience, we asked participants to practice the simulation first. During this “practice round”, the program randomized participants to one of three possible outcomes: premature AAA rupture during the 6-month watchful waiting period (WWD, programmed to occur within the first 5 visits), perioperative death (PD), or successful surgical outcome (SO). Participants in the WWD group who chose surgery before the rupture occurred (n=2) were removed from analysis. All subjects completed the second simulation under identical conditions, except that outcomes followed the evidence-based statistical distribution12.
Participants also completed a survey on attitudes towards anxiety, uncertainty and risk. The first part, administered before the simulation, consisted of the “state anxiety” portion of the State-Trait Anxiety Inventory (STAI-S)16, the Intolerance of Uncertainty Scale (IUS)17, and six items from the Domain specific Risk attitude Scale (DOSPERT)18. The second part, administered after the simulation, recorded demographic information, clinical training, experience with AAA management and recent poor surgical outcomes. After the simulation, participants were “debriefed” about the study goals and asked not to discuss the simulation with others.
Between and within-group comparisons of specialty groups were performed using Chi-square and Analysis of Variance (ANOVA). The Mantel-Cox test was used to perform pair wise comparisons of Kaplan-Meier curves. The effect of simulated recent experience was assessed using time-to-event analysis where the “event” was the decision to operate and time was measured by the number of decisions to continue watchful waiting. The statistical optimum for perioperative survival was 10 “visits”. A spontaneous AAA rupture occurring at any point during the simulation was treated as a censored event. Statistical analyses were conducted using the Statistical Package for the Social Sciences, version 19 (SPSS 19.0).
Sixty-nine geriatricians, 63 surgeons, and 92 anesthesiologists participated (Table 1). All groups were comparable with respect to age, years of clinical experience, ethnicity, country of education, and scores on the pre-simulation STAI, IUS, and DOSPERT. In all three specialties, most respondents reported clinical experience with AAA patients. Geriatricians and anesthesiologists were more likely than surgeons to recall a poor outcome from AAA surgery in the past 6 months (36.6% and 29.9% vs 8.3%; respectively, p < 0.01). Analysis of physician respondents by recent experience showed no differences between groups in age, experience, gender, risk, uncertainty tolerance and anxiety (data not shown).
For each group, the timing of the decision to operate differed from guideline recommendations. In our simulation, perioperative survival would be maximized by choosing surgery during the 10th” visit” when the AAA diameter exceeded 5.5cm and the rupture risk (5.5%) had surpassed the risk of peri-operative death (5.0%). Overall, all groups on average chose surgery earlier than appropriate for optimal perioperative survival (Figure 1). Geriatricians waited 7.7 ± 0.4 (mean ± SD) visits, (95% CI 6.79 – 8.58), corresponding to an AAA diameter of 5.2cm and rupture risk of 1.8%). In contrast, anesthesiologists chose to operate sooner (7.3 ± 0.4 visits; 95% CI: 6.5 – 8.1, rupture risk 1.8%). Surgeons were the least willing to wait, choosing to operate after 7.0 ± 0.4 visits (95% CI: 6.3 – 7.7, rupture risk 1.6%). The percentage of physicians selecting surgery at 9 or fewer visits was 66.7% (geriatricians), 74.7% (anesthesiologists), and 77.0% (surgeons).
When the three groups were compared, a specialty-specific difference emerged (Figure 1a–c). Surgeons exposed to a WWD during their practice round chose to operate earlier than those exposed to the PD or SO experiences (5.5 ± 0.6 visits in the WWD group vs 8.1 ± 0.6 visits in the PD and 7.4 ± 0.6 visits in the SO group, p<0.01). Geriatricians exposed to WWD demonstrated this effect to a lesser degree, (6.6 ±0.8 visits vs 8.0 ± 0.73 (PD) or 8.0 ± 0.75 visits (SO), p=0.12). For anesthesiologists, recent experience did not alter subsequent decision behavior (7.0 ± 0.7 visits in the WWD group vs 7.2 ± 0.69 visits in the PD and 7.2 ± 0.55 visits in the SO group (p = 0.71).
In a computer simulation focusing on asymptomatic AAA management, physicians from three different specialties chose on average to operate sooner than recommended by current guidelines. The magnitude of this deviation from guideline-recommended behavior was similar for all three specialties. By selecting the 5.0% risk of perioperative mortality when the risk of spontaneous rupture was 1.6%, participants appeared on average to choose a 3.4% increased risk of perioperative mortality.
It is unclear why physicians in our study chose not to maximize perioperative survival. One possibility is the difference between our computer simulation and the more complex real-life clinical decision. For example, participants may have considered surgery inevitable and integrated the 5% perioperative mortality rate into their decision. Additionally, participants may have focused on years of life after surgery, choosing surgery earlier to “get it over with”19. However, no participant identified either inevitability of surgery or years of life as relevant decision factors.
Most participants in our study identified “burst rate” as the most important decision factor. An asymmetric focus on the risk of rupture, instead of the 5% risk of perioperative death, may thus have played a role. In our previous work, lay adults found AAA rupture as upsetting as images of a person being forcefully abducted20 (Hemmerich, JA in press). Because AAA rupture is typically lethal, anxiety from waiting for the AAA to reach the optimal size for surgery may also have led to earlier operation. Aversion to uncertainty21, and preferences among physicians and patients for action over inaction22, are other possibilities. Although open AAA repair carries significant mortality risk, choosing surgery in our scenario may have appeared more certain than risking a potential rupture. Surgeons and anesthesiologists in our study may also have considered their clinical skills above average23, leading them to undervalue the risk of perioperative death.
We also found a specialty-specific effect of recent experience. Surgeons exposed to the WWD condition chose surgery significantly sooner for a subsequent patient than did those exposed to either the PD or SO conditions. This tendency was consistent with our previous findings 8, 9. Geriatricians exhibited this behavior to a lesser degree, and anesthesiologists exposed to the WWD condition did not behave differently from those exposed to the other two conditions. Why anesthesiologists and surgeons behaved differently with respect to recent experience is unclear. Self-reported experience with AAA management, age, and attitudes towards risk and anxiety did not differ between specialties. One possibility is divergent levels of regret24,25 Because of a greater sense of control over the outcome, surgeons may have felt more regret than anesthesiologists at ‘failing” to prevent a WWD. Experience with practice standards26 and quality improvement initiatives27 may also have made anesthesiologists more willing than surgeons to follow guidelines. Finally, anesthesiologists frequently begin their relationship with the patient after the decision to operate has been made. This greater degree of emotional distance may have insulated them from cognitive factors affecting operative decisions such as regret, anxiety, or uncertainty.
Our study has limitations. Because participants may have behaved differently in our simulation than in actual practice, the real-world relevance of our findings is unclear. Conflicts between maximizing survival vs. the financial donation, and unspecified decision factors such as quality of life, may also have influenced participants. To counter these possibilities, we took precautions to ensure that participants understood the scenario and decision algorithm, and were continuously provided with updated statistical information. Our debriefing experience suggested that participants were not confused about decisional factors. Rather, many noted (often ruefully) that they knew they had acted sub-optimally. In addition, participants choosing to maximize the financial donation (rather than perioperative survival) would have waited longer than 10 visits to operate28. While we cannot say whether study subjects would have behaved differently in real life, discrepancies between physician practice and evidence-based guidelines are common2. A second limitation is the composition of our specialty groups. Most participants were not experts at AAA management. However, in light of our previous work demonstrating that vascular surgeons are affected by negative prior experience9, and evidence showing less guideline compliance with increased experience29, it is not obvious that experts or those with more experience would have performed better. In addition, the geriatrician group had more women than the other two groups. While this discrepancy might have affected our results, female gender does not clearly predict better guideline compliance30. Our scenario included no complex clinical details, required no advanced diagnostic assessment and continuously provided all relevant statistical data. To adhere to published guidelines, participants needed only to delay surgery until the risk of AAA rupture equaled the risk of perioperative death. That three separate specialty groups, studied at three different locations and times, did not do so suggests a decision preference worthy of further research.
In summary, in a realistic computer simulation of AAA management, we found two notable patterns of decision behavior. The first, shared by surgeons and anesthesiologists (and to a lesser degree geriatricians), was a preference to choose surgery when the risk of AAA rupture was lower than the risk of perioperative mortality. This preference differed from published guidelines. The second, primarily in surgeons, was a tendency to (over)react to a watchful waiting death by operating significantly sooner in a subsequent patient.
Our results raise important policy issues. That decision preferences may differ based on recent experience or specialty suggests differences in clinical training or baseline characteristics that may distort physician decision-making. That physicians with different responsibilities in multispecialty care may decide differently raises issues about how such decisions should be made, what patients should be told, and by whom. Without understanding how non-statistical factors influence decision behavior, policies targeted at increasing healthcare decision consistency through guideline dissemination may be ineffective. Identifying these factors under rigorous experimental conditions represents a first step towards bridging the gap between guideline recommendations and non-guideline-based physician behavior.
Funding: This work was supported by the University of Chicago Departments of Medicine (Section of Geriatrics & Palliative Medicine) and Anesthesia and Critical Care, Booth Graduate School of Business, and a Paul Beeson Career Development Award (K23 AG24812) to Dr. Dale.
Sponsor’s Role: None
Conflict of Interest: The editor in chief has reviewed the conflict of interest checklist provided by the authors and has determined that the authors have no financial or any other kind of personal conflicts with this paper.
Author Contributions: Indicate authors’ role in study concept and design, acquisition of subjects and/or data, analysis and interpretation of data, and preparation of manuscript. (See section on “Authorship and Duplicate Publication”).
Study concept and design:
Drs. Dale, Schwarze, Tung, Hemmerich, and Moliski
Drs. Dale and Hemmerich (Geriatrics), Schwarze (Surgery), Tung and Moliski (Anesthesia)
Analysis and interpretation of data:
Drs. Hemmerich, Dale, Tung, Schwarze and Moliski
Preparation of manuscript
Drs. Tung, Dale, Hemmerich, Schwarze, and Moliski
This report was previously presented, in part, at the American Society of Anesthesiologists Annual meeting in 2008 and at the annual meeting of the Society of Medical Decision Making in 2009