The Evidence-based Practice for Improving Quality method builds on existing continuous quality-improvement methods,27
which usually advocate adoption of an entire package of practices from hospitals with good outcomes. In contrast, our method enables hospitals to select practice changes pertinent to them for targeted intervention. This is potentially more efficient and cost-effective.
We found that interventions aimed at one outcome may affect other outcomes. We speculate that the decrease in the incidence of nosocomial infections in the pulmonary group was related to improved lung status and a reduced need for assisted respiration, invasive interventions, improved feeding and growth, and better overall health. Because quality improvement is about transforming behaviours, spill-over from one outcome to another may be expected. Although it may be advantageous to target multiple outcomes simultaneously because of potential synergies, the possibility that improving one outcome may worsen another should be considered.
It is possible that the Hawthorne effect28
(i.e., the effect on individuals of being observed for the purposes of research) or that case findings changed once a hospital knew what outcome they were targeting could account for the study results. Because these effects were probably larger for nosocomial infection than bronchopulmonary dysplasia, this may explain why nosocomial infection decreased in both the infection and pulmonary groups. It is also possible that the larger proportion of outborn infants affected the magnitude of the reduction of the incidence of nosocomial infections in the infection group. However, because the incidence of bronchopulmonary dysplasia declined only in the pulmonary group, even though there was a lower baseline incidence in this group, the results were more likely because of the study interventions rather Hawthorne or case-finding effects.
We used a combination of qualitative and quantitative methods, as advocated by Pawson and Tilley.29
Qualitative methods were especially useful for identifying hospital-specific organizational issues, barriers to change and potential change strategies.17
Interviews also generated greater staff awareness, made individuals feel that their opinions mattered, enhanced staff buy-in and facilitated practice change.17
Quantitative methods provided feedback of results that reinforced behavioural change.
The Evidence-based Practice for Improving Quality method is a multidimensional approach aimed at changing organizational culture and sustaining behavioural change. Consequently, it may work best when it is adopted in its entirety rather than in parts. Although we cannot explain the difference in results reported by Walsh and colleagues4
and our study without more detailed information about the study by Walsh and colleagues, it is possible that the multidimensional processes for organizational and behavioural change were more rigorously implemented in our study.
In this study, not all aspects of the Evidence-based Practice for Improving Quality process were consistently implemented across all units. Thus, it is difficult to identify exactly which interventions were most effective. Because our model is a method rather than a prescription of interventions, it is potentially generalizable to other neonatal ICUs, but it will need to be tested by others. Finally, we caution that this method is not a substitute for randomized controlled trials, but it may be more appropriate when investigating the effects of change in complex adaptive systems.30
The limitations of our study included the subjective definition of bronchopulmonary dysplasia; concerns about anonymity that may have precluded feedback of site-specific qualitative results; logistical delays in reporting that sometimes impaired our ability to provide control charts in a timely manner; the lack of uniform criteria for routine screening, which may have decreased the incidence of retinopathy of prematurity in the infection group; and the withdrawal of a large neonatal ICU from the infection group. The study was stopped early even though we were about 500 patients short of the estimated sample size because we felt that there was sufficient power to analyze the results.
Our study suggests that the Evidence-based Practice for Improving Quality method is effective in reducing the incidence of nosocomial infection and bronchopulmonary dysplasia in neonatal ICUs. The method is potentially generalizable to other areas of health care and may improve the efficiency and reduce the costs of quality improvement efforts.
Presented in part at the Pediatric Academic Societies Annual Meeting, San Francisco, California, 2006.