In this single-site, preliminary study, we demonstrate that prompting physicians on one MICU team to discuss care practice parameters on a checklist improves multiple processes of care compared with a similar team that received checklists but no prompting. Prompting was associated with shortened duration of mechanical ventilation, empirical antibiotic use and central venous catheter use, and increased use of DVT and stress ulcer prophylaxis. Using the checklist without ongoing prompting did not result in improvement compared with baseline measurement of outcomes. The prompted group, however, had lower severity-adjusted mortality and length of stay.
Our study contrasts active checklist use versus passive implementation, the latter representing a quality improvement paradigm in which a tool or process is initiated through regulatory or administrative mandate (22
). Indeed, a comparison of checklist-only control group outcomes with the preintervention cohort suggests that the checklist itself had little effect. Instead, our study suggests that effective quality improvement requires a robust implementation and adherence strategy. Constant attentiveness to the care practices under investigation, driven by prompting, achieved improvements in processes of care. We do not believe the checklist is superfluous: prompting by memory would be prone to similar errors of omission that it attempts to prevent.
Overlooked in the enthusiasm for checklists is the fact that the most prominent examples enforced changes in behavior. Bedside nurses were empowered to stop CVC insertion with overt support from hospital administrators in the Keystone Project (9
). Surgeons were not allowed to begin an operation or transfer a patient to the recovery room without checklist completion in the World Health Organization Surgical Safety Checklist Study (10
). Even in the oft-cited airplane pilot illustration (23
), participation by a copilot is compulsory. We describe our intervention as prompting rather than enforcing, to reflect that the checklist addressed more nuanced decisions (e.g., when to discontinue empirical antibiotics or wean mechanical ventilation) for which a prompted discussion is more appropriate than a mandatory decision. Our study suggests that checklists require an accountability strategy to change behavior and ultimately progress to culture change. This finding is critical to quality improvement strategies based on checklists.
Our study has several strengths that build on other successful quality improvement interventions (5
). First, prompting was face-to-face and repetitive, which strongly encouraged physicians to change their management behavior, similar to prior studies that employed local quality champions and close monitoring (9
). In contrast, checklist implementation without repetitive prompting in the control group had no mechanism of accountability, and did not yield improvement in outcomes.
Second, our intervention targeted multiple care practices. This may have led to more clinical benefit than any single practice alone (10
). Previous strategies that individually targeted our prompting topics—including empirical antibiotics (18
), mechanical ventilation weaning (26
), and CVCs (8
)—are each associated with improved outcomes. Also, duration of mechanical ventilation, prior antibiotic use, and prior broad-spectrum antibiotic use are all independent predictors of ventilator-associated pneumonia due to potentially drug-resistant bacteria (28
), which itself increases mortality (29
). The improvement of several factors in combination may have plausibly led to the improvement of outcomes observed in our study.
Third, an individual prompt may impact patient care or alter provider behavior beyond that single instance of prompting. In the example of mechanical ventilation, prompting for a weaning trial occurred on only 15% of patient-days, yet a substantial 6-day increase in ventilator-free days was observed. A single prompt for mechanical ventilation weaning could have prevented multiple future days of ventilation if weaning was successful on that prompted day. Prompting at one time also may have led the prompted physician to order spontaneous breathing trials on other patients or the same patient on subsequent days, even when no prompter was present. In addition, the prompt may have elicited a discussion of the rationale for the prompt, resulting in an enhanced educational effect that would carry over to other patients on other days.
Although we hypothesized that prompting would improve quality of care, mortality and LOS benefits were unanticipated. The association of prompting with reduced mortality was observed for both severity-adjusted ICU and hospital mortality, suggesting that the effects of prompting in the ICU had a direct impact on ICU outcome and that this effect increased by hospital discharge. However, this was a small study, with a 95% confidence interval for the adjusted odds ratio of mortality that was wide (0.15–0.76), which precludes a definitive analysis of the factors that may have contributed to the mortality reduction.
Some of the mortality difference may have been due to chance, as small studies may be prone to variation. Nevertheless, even a conservative estimate based on the upper 95% confidence interval limit for adjusted mortality suggests that the intervention may have led to a reduction in deaths. Several findings may support this. There was no difference in severity of illness, baseline characteristics, or discharge disposition. The results suggest lower mortality in the prompted group for patients with sepsis, pneumonia, and other respiratory conditions, diagnoses directly related to the processes of care targeted for prompting. We found no difference in mortality in the lowest patient quartile or highest range of predicted mortality (Figure E2), as expected because ICU interventions would not be expected to affect mortality when risk of death is very high or very low. We observed no early difference in LOS; an early difference would suggest that repetitive prompting was unlikely to be responsible. Last, neither seasonal nor overall variation in hospital mortality or ICU LOS between the preintervention and control groups was demonstrated, and control group SMR matched that of a prior 1-year prospective quality project in the same ICU (1,619 patients admitted in 2006–2007; SMR; 0.96; our unpublished data). Despite these findings, our intervention deserves further exploratory analyses and study in a large, multicenter study.
Our study has several potential limitations. First, It was a single-site preliminary study, potentially limiting the generalizability of the results, particularly the mortality effect, of prompting based on our specific checklist to other settings. However, the benefit of a prompter or other similar accountability strategy is likely generalizable to improvements in different processes of care addressed by a different unit-specific checklist. Reproduction of the clinical outcome benefits in larger studies will depend on the linkage between the process of care issues addressed by the checklist and disease-specific outcomes, as well as baseline compliance.
Second, the study cohorts were relatively small. We purposefully limited the length of the study to minimize crossover of attending physicians and fellows from one team to the other, which could have reduced the apparent influence of prompting. Frequent physician rotation within teams could have impacted patients with longer ICU lengths of stay. Third, no research personnel were attached to the control team. As a result, the degree of checklist use by the control team was not directly observed, which also limited our ability to ascertain other differences in team characteristics. Last, although baseline characteristics, severity of illness, discharge disposition, and night or weekend ICU admission were not confounders, other residual confounders could exist.
Using a resident physician as a prompter is clearly an artificial construct of this study. To test effectiveness and widespread implementation, future studies should focus on determining the optimal approach to prompting, such as an electronic decision-support tool (30
) or virtual prompting (11
). Alternative forms of prompting may increase the feasibility of future multicenter studies; however, the benefit of face-to-face prompting in our study may not be reproduced electronically.
In summary, this single-site study suggests that simply having a checklist available for reference without consideration of a robust implementation and adherence strategy is unlikely to maximize desired patient outcomes. The complexity of critical care medicine may benefit from, and provide an opportunity to investigate, novel approaches to reduce errors of omission.