In response in part to the Institute of Medicine's reports "Crossing the Quality Chasm" [1
] and "To Err is Human" [2
], and the American Medical Informatics Association's position paper on the use of clinical decision support in electronic prescribing [3
] there is increased pressure to implement state of the art clinical information systems (CISs) with real-time clinical decision support capabilities. Unfortunately, several recent reports have documented that a disturbingly high percentage (i.e., 54 – 91%) of real-time clinical decision support suggestions are being over-ridden, or ignored, by clinicians [4
Granted, there are certainly cases in which "overriding" the computer-generated alert is the correct action on the part of the clinician including: the benefits of the action outweigh the risks, there is no good alternative, this is an "expected" side-effect of a particular therapy or procedure, or that the medication was previously or currently tolerated, to name just a few. On the other hand in all of the studies cited above, in which the clinicians overrode a very high percentage of all alerts, the authors found that in almost all cases the computer-generated alerts were "true-positives" meaning that most observers would consider the clinicians actions to ignore the alert to be contrary to "best clinical practices".
We are in the process of designing, implementing, and evaluating many new clinical decision support features and interventions [7
]. Based on our knowledge of the literature and extensive clinical informatics experience, we recognize that there are a myriad of factors associated with clinicians' refusal to accept, or follow, computer-generated, care suggestions based on clinical guidelines including lack of: awareness that the guideline even exists, familiarity with the recommendation, agreement with the suggestion, belief that they could even perform the expected behavior (often referred to as: self-efficacy), belief that the expected improvement in outcome will occur, ability to overcome the inertia of previous practice, and the existence of external barriers to the performance of the recommendations (e.g., no time or no reminder system) [9
]. In addition to these mostly internal, provider-related factors, there are also many computer-related hypotheses for why clinicians refuse to follow these suggestions including: failure to provide patient-specific information (which was not shown to be a factor in this study) [10
], specific aspects of the human-computer interaction surrounding the presentation of the reminders, for example, presenting fully-completed orders that follow the guideline on the same screen as the reminder, rather than placing them "one click away", using a distinctive color scheme to "highlight" the recommendation, disabling the escape key which made it more difficult to override the suggestion, setting the default value of the suggestion to "order" rather than "not to order", and presenting the same reminder over and over to all clinicians who viewed a particular patient's data (i.e., until the suggestion was accepted) [11
]. While we were not able to follow all of these "best practices" for the design of interactive clinical decision support features due to inherent limitations of our commercially available EMR and some institutional resistance on the part of clinical and information system administrators, we are doing our best to remove as many potential barriers as possible.
We hypothesized that there are other factors that may account for clinicians' refusal to follow computer-generated clinical suggestions with the intended action of removing, or at least reducing, as many of the identified barriers as we can. Therefore, we undertook this study to begin exploring these other potential factors affecting clinician acceptance of clinical decision support at the point of care.
Clinical computing environment
We conducted the survey within Northwest Permanente, the physicians' group associated with Kaiser Permanente, Northwest (KPNW) in Portland, OR. Briefly, KPNW is a large, group model health maintenance organization serving northwestern Oregon and southwestern Washington. KPNW is a pre-paid medical plan that is responsible for the health of over 455,000 patients. KPNW implemented a commercially available ambulatory medical record product from Epic Systems (Madison, WI) beginning in 1994 and was fully implemented in 1997. In 1998, they won the Nicholas E. Davies Award for CIS implementations [12
]. In 2003 and again in 2005, KNPW was voted the best HMO by survey respondents of a leading consumer magazine [13
Clinical decision support within KPNW
Over the past several years, a number of careful assessments of the effects of various clinical decision support features have been made using the clinical information system within KPNW including: 1) using an off-line data analysis technique to identify patients eligible for a specific alert that could be presented to the clinician [15
]; 2) the effect of alerts that remind clinicians about medications contraindicated in the elderly [16
]; 3) the effect of alerts that recommend dose changes in patients with various levels of renal insufficiency [17
]; and 4) the effect of alerts that notify clinicians in the event that a patient is on, or being prescribed two medications that may have a serious interaction [18
]. All of these studies showed significant and sustained benefits to patients. That is, the percentage of patients receiving the contraindicated medications decreased by approximately 10–20% relative to the baseline measurements after 12 months of continuous usage. While this decrease was statistically, as well as, clinically significant, there were still patients who continued to receive these contraindicated medications which means that the clinicians, ignored or overrode many of the alerts. These findings led us to begin asking clinicians questions about the clinical decision support that we were providing.
Based on the work of several investigators [19
] we hypothesized that clinicians' acceptance of clinical decision support could be explained by one, or a combination of, factors from the following categories:
■ Patient: reason for visit, severity of illness – estimated based on the number of medications the patient was taking and the number of chronic conditions they had, or age.
■ Provider: age, gender, or number of years with Kaiser Permanente.
■ Alert: type of alert or number of alerts received.
■ Environment: examination room set-up including presence of a computer or estimated number of minutes the clinician is behind schedule.
We defined clinical decision support as "clinical information that is either provided to you or accessible by you, from the EpicCare clinical workstation". We consider enhanced information displays such as flow sheets, health maintenance reminders, alternative medication suggestions, order sets or smart sets, alerts, and access to any internet-based information resources like the KPNW Clinical Library as clinical decision support.