Our model to improve reliability focuses on the rate-based measures of safety; how often do we harm patients and how often do we use evidence-based medicine. Rate-based measures are specific to a clinical area or discipline. Using the objective of eliminating CRBSI as an example, we describe the model below
1. Identify interventions associated with an improved outcome in a specific patient population
. To a large extent, this has been accomplished with practice guidelines or summaries of clinical research evidence. For example, the Centers for Disease Control (CDC) and others have published guidelines for preventing CRBSIs (CDC 2004
2. Select interventions that have the biggest impact on outcomes and convert these into behaviors
(Grimshaw et al. 2001b
; Michie and Johnston 2004
). The team should focus on approximately five interventions that are supported by strong evidence, have the greatest potential benefit, and reflect patients' values and preferences. Recent recommendations to grade evidence into “do or do not do” will greatly facilitate this step (Atkins et al. 2004
). In selecting interventions, it may be helpful, if not done in the evidence review, to make a table of each potential intervention with the strength of the evidence supporting its use, the strength of the relationship (e.g., a risk ratio) between the intervention and the outcome, and the barriers in implementing the intervention (Gordis 2004
3. Develop measures to evaluate reliability
. Here, we seek a scientifically sound and feasible rate-based measure that can either be an outcome or process element of safety. The measure(s) selected should be carefully considered. Both types of measures have strengths and weaknesses that have been published (Rubin,Pronovost, and Diette 2001
; Lilford et al. 2004
; Pronovost, Nolan et al. 2004
). For example, if the intervention is a medication, we could measure if it was given, or what medication, dose, and/or when it was given. Several principles guide us in deciding which of these to measure. First, choose measures that are scientifically sound or supported by the evidence. If timing or dose of antibiotic administration is important, measure when the medication was given and dose given as two separate variables. Second, measure what is feasible, or easily collected with available resources. Third, if possible, measure where defects most commonly occurred. To do this, review each step in the process for a sample of patients and identify where defects most commonly occurred. For example, evidence suggests that steroids reduce mortality in septic shock patients (Annane et al. 2002
). When we monitored use of steroids for this patient population, we found that failure to prescribe the medication was the most common defect. As a result, we developed a measure to evaluate whether patients with septic shock received steroids.
Development of measures typically requires significant resources and expertise in developing measures and specific clinical content (Garber 2005
), which few health care organizations will likely have available. As such, national measures should be developed and broadly shared among health care organizations.
In this case, the National Nosocomial Infection Surveillance System (NNIS) has standardized measures for CRBSI that are valid, reliable, and widely used, which prompted us to measure the outcome rather than the process. Attempts to measure the process proved neither valid nor feasible. Such a measure would require additional ICU staff—not available to us—to independently monitor the placement of all central venous catheters.
4. Measure baseline performance. This is the best test of whether the proposed measures can be feasibly collected. If baseline data cannot be collected with minimal bias, it is unlikely that these data can be collected after the intervention has been implemented. Moreover, without baseline data, an organization cannot assess if safety has improved. In addition to collecting data, a health care organization should create a database to evaluate data quality and missing data, store and analyze data, and produce reports. In our experience, few quality improvement projects create such a database.
5. Ensure patients receive evidence-based interventions
. This effort is the biggest challenge. While steps 1–4 are generally performed by a team of researchers and clinicians with sufficient resources who may or may not personally implement the interventions, step 5 involves teams from the participating health care organization who will actually implement the interventions. These interventions must be tailored to address each participant's current system, culture, resources, and commitment. While there is no formula for system redesign, there are many tactics that appear effective for improving care (Grol et al. 1998
; Cabana et al. 1999
; Grol 2001
; Pronovost, Wu et al. 2002
; Pronovost, Weast et al. 2004
; Pronovost and Berenholtz 2002
; Bradley et al. 2005
The change model we used to improve reliability (outlined in ) was designed as a practical application of theories related to diffusion of innovation and behavior change (Grimshaw et al. 2001a
; Greenhalgh et al. 2004
; Michie et al. 2005
). The change model includes four components: engage, educate, execute, and evaluate. Each component targets senior leaders, team leaders, and front-line staff.
Engaging and educating front-line staff is challenging and resource intensive. The execute component encourages staff to use HRO theory (i.e., standardize, create independent checks, and learn from mistakes) to ensure patients receive evidence-based interventions. Here, we encourage teams to first consider how they can standardize (including reducing complexity) what they do to reduce the risk of failure. Often this step includes creating a standard order set or protocol. Next, teams create independent checks (i.e., two or more persons recheck independent of the other[s]) for key processes. Finally, when defects occur, teams are encouraged to evaluate or learn the causes.