Improving the quality of care for infants: a cluster randomized controlled trial

doi:10.1503/cmaj.081727

CMAJ. 2009 October 13; 181(8): 469–476.

PMCID: PMC2761437

Improving the quality of care for infants: a cluster randomized controlled trial

Shoo K. Lee, MBBS PhD, Khalid Aziz, MBBS, Nalini Singhal, MBBS, Catherine M. Cronin, MD MBA, Andrew James, MBChB MBI, David S.C. Lee, MBBS, Derek Matthew, MBBS, Arne Ohlsson, MD MSc, Koravangattu Sankaran, MBBS, Mary Seshia, MBChB, Anne Synnes, MDCM MHSc, Robin Walker, MBChB, Robin Whyte, MBBS, Joanne Langley, MD MSc, Ying C. MacNab, PhD, Bonnie Stevens, PhD, and Peter von Dadelszen, MBChB DPhil

From the Departments of Paediatrics (Lee, James, Ohlsson), University of Toronto, Toronto, Ont; Memorial University (Aziz), St John’s, NL; University of Calgary (Singhal), Calgary, Alta.; University of Manitoba (Cronin, Seshia), Winnipeg, Man.; University of Western Ontario (Lee), London, Ont.; University of British Columbia (Matthew, Synnes), Vancouver, BC; University of Saskatchewan (Sankaran), Saskatoon, Sask.; University of Ottawa (Walker), Ottawa, Ont.; Dalhousie University (Whyte, Langley), Halifax, NS; Department of Health Care and Epidemiology (MacNab), University of British Columbia, Vancouver, BC; Department of Nursing (Stevens), University of Toronto, Toronto, Ont.; and the Department of Obstetrics and Gynecology (von Dadelszen), University of British Columbia, Vancouver, BC

Correspondence to: Dr. Shoo K. Lee, Department of Paediatrics, Mt. Sinai Hospital, 782-600 University Ave., Toronto ON M5G 1X5; fax 416 586-8745;Email: sklee/at/mtsinai.on.ca

See commentary "Bridging the gaps: getting evidence into practice" on page 457.

This article has been cited by other articles in PMC.

Abstract

Background

We developed and tested a new method, called the Evidence-based Practice for Improving Quality method, for continuous quality improvement.

Methods

We used cluster randomization to assign 6 neonatal intensive care units (ICUs) to reduce nosocomial infection (infection group) and 6 ICUs to reduce bronchopulmonary dysplasia (pulmonary group). We included all infants born at 32 or fewer weeks gestation. We collected baseline data for 1 year. Practice change interventions were implemented using rapid-change cycles for 2 years.

Results

The difference in incidence trends (slopes of trend lines) between the ICUs in the infection and pulmonary groups was − 0.0020 (95% confidence interval [CI] − 0.0007 to 0.0004) for nosocomial infection and − 0.0006 (95% CI − 0.0011 to − 0.0001) for bronchopulmonary dysplasia.

Interpretation

The results suggest that the Evidence-based Practice for Improving Quality method reduced bronchopulmonary dysplasia in the neonatal ICU and that it may reduce nosocomial infection.

Although methods for continuous quality improvement have been used to improve outcomes,¹^–³ some, such as the National Institutes of Child Health and Human Development Quality Collaborative,⁴ have reported little or no effect in neonatal intensive care units (ICUs). These methods have been criticized for being based on intuition and anecdotes rather than on evidence.⁵ To address these concerns, researchers have developed methods aimed at improving the use of evidence in quality improvement. Tarnow-Mordi and colleagues,⁶ Sankaran and colleagues⁷ and others⁸^–¹⁰ have used benchmarking instruments⁶^,⁸^,¹¹ to show risk-adjusted variations in outcomes in neonatal ICUs. Synnes and colleagues¹² reported that variations in the rates of intraventricular hemorrhage could be attributed to practice differences. MacNab and colleagues¹³ showed how multilevel modelling methods can be used to identify practice differences associated with variations in outcomes for targeted interventions and to quantify their attributable risks.

Building on these results, we developed the Evidence-based Practice for Improving Quality method for continuous quality improvement. This method is based on 3 pillars: the use of evidence from published literature; the use of data from participating hospitals to identify hospital-specific practices for targeted intervention; and the use of a national network to share expertise. By selectively targeting hospital-specific practices for intervention, this method reduces the reliance on intuition and anecdotes that are associated with existing quality-improvement methods.

Our objective was to evaluate the efficacy of the Evidence-based Practice for Improving Quality method by conducting a prospective cluster randomized controlled trial to reduce nosocomial infection and bronchopulmonary dysplasia among infants born at 32 or fewer weeks’ gestation and admitted to 12 Canadian Neonatal Network hospitals¹⁴ over a 36-month period. We hypothesized that the incidence of nosocomial infection would be reduced among infants in ICUs randomized to reduce infection but not among those in ICUs randomized to reduce bronchopulmonary dysplasia. We also hypothesized that the incidence of bronchopulmonary dysplasia would be reduced among infants in the ICUs randomized to reduce this outcome but not among those in ICUs randomized to reduce infections.

Methods

Study design

The Canadian Neonatal Network includes all 30 tertiary-level regional neonatal ICUs across Canada. The network maintains a prospective national database of all infants admitted to any neonatal ICU. In Canada, neonatal care is highly regionalized. Tertiary-level care is provided by 30 neonatal ICUs, which serve distinct geographic regions and coordinate care with a network of primary- and secondary-level facilities. Twelve tertiary-level regional neonatal ICUs from 4 geographic regions (2 in British Columbia, 4 in the Prairie provinces [Alberta, Saskatchewan, Manitoba], 4 in Central Canada, 2 in Atlantic Canada [Nova Scotia, Newfoundland and Labrador]) participated in this trial from October 2002 to September 2005. We selected 6 units by concealed randomization to reduce nosocomial infection (infection group) and 6 to reduce bronchopulmonary dysplasia (pulmonary group). We used a complete randomization procedure with neonatal ICU as the unit. Each group was blinded to the study discussions and interventions of the other group and served as a control for the other group. One hospital in the infection group dropped out of the study during phase 1; we excluded this hospital from the analyses. We used 5 smaller tertiary-level neonatal ICUs that were not participating in the trial to provide additional comparisons (comparison group) to observe general trends not associated with the study interventions. Hospitals in the comparison group did not participate in any of the study activities, and information was not shared with them during the study.

The study was approved by the institutional ethics review boards at all of the participating hospitals. Data were reviewed by an independent data monitoring and safety committee in June 2004, and trial continuation was recommended.

Phase 1

Phase 1 was conducted during year 1 of the study and was a preparation phase. We collected data about baseline population characteristics for all infants born at 32 or fewer weeks’ gestation who were admitted to a participating neonatal ICU. We selected outcomes and practices. The pooled data were analyzed by use of mixed effects logistic modelling techniques to examine variations in the incidence of nosocomial infection and bronchopulmonary dysplasia and associated practices.

Early during phase 1, site investigators attended a 3-day critical appraisal workshop conducted by the research methods committee. The topics included systematic reviews, qualitative methods, principles of continuous quality improvement, neonatal infections, bronchpulmonary dysplasia and data interpretation. The site investigators from the infection and pulmonary groups separately defined the questions to be critically evaluated, assigned topics for systematic review and shared their findings at a face-to-face meeting.

Each hospital established a multidisciplinary team comprising a site investigator, neonatologist, nurse manager, nurse educator, quality-improvement officer, executive sponsor and an infection control nurse or respiratory therapist as appropriate. At a 2-day skills workshop conducted by the research methods committee, team representatives learned about techniques of continuous quality improvement, including the use of a clinical value compass, aims identification, satisfaction and behaviour, team building, clinical process analysis, flow charting, understanding variation, use of Pareto charts, histograms and control charts, benchmarking, use of the rapid cycle improvement models of Alemi and colleagues¹⁵ and Plsek,¹⁶ communication strategies, and change management. The team representatives then trained other members of their hospital team and examined clinical processes in their hospitals. Staff and parents of infants in the neonatal ICUs were interviewed to identify barriers to change.¹⁷ At the end of phase 1, representatives from the infection and pulmonary groups met separately to share information and materials. They divided into 3 categories (definitive, nondefinitive and unsubstantiated) the accumulated evidence from systematic reviews, phase 1 data analysis, clinical process analyses and qualitative study reports. Within each group, consensus was reached about potentially useful practices and priorities and potential strategies for implementing practice change. Each hospital used the group consensus report, together with the results of their own hospital-specific data analysis and consideration of local organizational and cultural issues to develop a hospital-specific list of recommended practice changes and priorities.

Phase 2

Phase 2, the intervention phase, comprised years 2 and 3 of the study. Hospital teams implemented rapid cycles (1–3 months) of practice change.¹⁵^,¹⁶ Single or multiple changes could be implemented during each cycle, and compliance was monitored at random intervals. Communication strategies for practice change included information sessions, focus groups, order sheet prompts, posters, feedback boxes, computer-based learning resources and training sessions. Lessons learned and training resources were shared at regular group teleconferences. Selected outcome indicators and control chart feedback were available at 3-month intervals.¹⁸ Results were shared within each of the infection and pulmonary groups.

Members of the research methods committee visited the hospital sites to trouble shoot and provide advice as needed. Halfway through phase 2, members visited other hospitals to observe practices and discuss change strategies. Site selection for visits was based on the perceived potential for learning about both good and poor practices. Neonatal ICUs in the infection and pulmonary groups were discouraged from participating in other quality improvement collaboratives that focused on nosocomial infections and bronchopulmonary dysplasia; however, local quality-improvement efforts were permitted.

Outcomes

Our primary outcomes were change in the incidence of nosocomial infections in the infection group and change in the incidence of bronchopulmonary dysplasia in the pulmonary group. We also examined death and bronchopulmonary dysplasia as a composite outcome to distinguish real change in the incidence of bronchopulmonary dysplasia from the effect of change in the incidence of death.

Nosocomial infection was defined as the growth of 1 or more organisms, including coagulase-negative staphylococcus, in blood or cerebrospinal fluid cultures taken after 48 hours of admission of an infant with clinical suspicion of infection, according to the Center for Disease Control definitions.¹⁹ To differentiate between nosocomial and primary (maternal origin) infections, we defined nosocomial infections as those that involved isolates that were different from maternal isolates (when available) or that occurred at least 7 days after initiation of treatment for a positive blood culture obtained within 48 hours after birth. Cerebrospinal fluid samples were obtained when indicated according to local practices. We defined an absence of a nosocomial infection as no culture performed or a negative blood culture or cerebrospinal fluid culture. Bronchopulmonary dysplasia was defined as oxygen dependency at 36 weeks’ corrected gestational age.²⁰ Infants who were discharged from the neonatal ICU on oxygen before they reached 36 weeks’ corrected gestational age were classified as having bronchopulmonary dysplasia. Infants who died before they reached 36 weeks’ corrected gestational age were excluded from the analysis for bronchopulmonary dysplasia.

Our secondary outcomes included death before hospital discharge, abnormalities on a cranial ultrasound (defined according to the Canadian Pediatric Society criteria²¹ for head ultrasonography performed within 14 days after birth), retinopathy of prematurity (defined according to the International Classification for Retinopathy of Prematurity²² and the Reese Classification of cicatricial disease²³) and necrotizing enterocolitis (defined according to Bell’s criteria stage 2 or higher²⁴).

Sample size estimation

We based our sample size estimation on a type 1 error of 0.05 and a power of 80% or greater to detect a 30% decrease in both the rates of nosocomial infection and bronchopulmonary dysplasia between baseline and the end of the study. We calculated the power to detect a difference in trends of incidence (β_g) between the 2 groups following the procedure of Stroup²⁵ (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.081727/DC2). For nosocomial infection, this yielded a power of 0.89 for between-group comparisons (i.e., differences in the incidence trends of nosocomial infection between infants in the infection and pulmonary groups) and a power of 0.91 for within-group comparisons (i.e., comparing the incidence of nosocomial infection before and after interventions in the infection group). For bronchopulmonary dysplasia, this yielded a power of 0.83 for between-group comparisons and 0.96 for within-group comparisons. We did not calculate power for the comparison group because it was not part of the cluster randomized controlled trial design. We determined that we needed to recruit 5083 infants for this study.

Our study had the ability to detect an incidence difference of 6.8% between the intervention and control groups (based on a type 1 error of 0.05, power of 0.80 and estimated sample size of 6000). Although this provides some idea of the detectable incidence difference, its significance may vary at different points during the cluster randomized study because of the changing sample size, population characteristics and outcome incidences. Consequently, comparisons of incidence trends rather than incidence differences are more appropriate for detecting differences between the 2 groups.

Data collection

Data were abstracted from patient charts and electronically transmitted to the Coordinating Centre in Edmonton, Alberta. This system has been reported previously.¹⁴

Data analysis

The data analysis plan was defined before commencement of the study and used the intention-to-treat principle. The characteristics of the study population were described. For the primary and secondary outcomes, we plotted the incidence and confidence intervals (CIs) quarterly and compared the incidence trends both within groups and between groups. Random-intercept multilevel logistic-regression models with random hospital-site effects were used to compare the incidence trends of nosocomial infection (or bronchopulmonary dysplasia) between the infection and pulmonary groups (primary comparison), adjusted for potential risk factors correlated with outcomes (birth weight, gestational age, sex, 5-minute Apgar, Score for Neonatal Acute Physiology version 2, outborn status [born at a hospital different from the admitting neonatal ICU], length of stay) and to examine the general trends of the incidence of nosocomial infection and bronchopulmonary dysplasia within the infection and pulmonary groups, respectively (secondary comparisons) (Appendix 2, available at www.cmaj.ca/cgi/content/full/cmaj.081727/DC2).²^,⁴^,¹⁴ In addition, we examined the comparison group for temporal trends unrelated to the study for observational comparison only.

Role of the funding source

The sponsors of the study had no role in the study design, data collection, data analysis, data interpretation or writing of the report. The corresponding author had full access to all data and had final responsibility for the decision to submit for publication.

Results

Figure 1 shows the trial profile. The study included 2465 infants (phase 1, 769 infants; phase 2, 1696 infants) in the infection group, 3070 infants (phase 1, 918 infants; phase 2, 2152 infants) in the pulmonary group, and 984 infants in the comparison group. The characteristics of the infants and the participating hospitals are shown in Table 1 and Table 2, respectively.

Figure 1

Consort flow chart for the cluster randomized trial of the Evidence-based Practice for Improving Quality method.

Table 1

Characteristics of infants included in the cluster randomized trial of the Evidence-based Practice for Improving Quality method for reducing nosocomial infections and bronchopulmonary dysplasia

Table 2

Characteristics of neonatal intensive care units that participated in the cluster randomized trial of the Evidence-based Practice for Improving Quality method for reducing nosocomial infections and bronchopulmonary dysplasia

The consensus lists of practice change strategies developed by the infection and pulmonary groups are presented in Appendices 3 and 4 (available at www.cmaj.ca/cgi/content/full/cmaj.081727/DC2). Each site used this list to select and prioritize practice change strategies for adoption. The following example illustrates how this was done. Using pooled hospital data, we previously showed that 40% of nosocomial infections in neonatal ICUs were associated with central lines and that percutaneously inserted central catheters carried the highest risk (odds ratio [OR] 3.5).²⁶

We used individual hospital data to provide a more refined look at whether these risk factors behave in the expected manner at individual sites. Kaplan–Meier survival analysis of individual hospital data during the baseline period revealed different patterns of nosocomial infection associated with percutaneously inserted central catheters at different hospitals. For example, Hospital A had an increase in the incidence of nosocomial infections with prolonged use of percutaneously inserted central catheters. Hospital B had no change in the incidence of nosocomial infections, and Hospital C had a lower incidence of nosocomial infections for all durations of percutaneously inserted central catheters (Figure 2). Thus, Hospital A should pay attention to maintenence protocols for percutaneously inserted central catheters, and Hospital B should re-examine both its catheter insertion and maintenance procedures. Although Hospital C had a low incidence of infections associated with percutaneously inserted central catheters, it had a high overall incidence of nosocomial infections. Closer scrutiny revealed that Hospital C had implemented strict protocols for the care of percutaneously inserted central catheters 2 years before this study, but other aspects of infection control (e.g., skin care, skin breaks, hand hygiene, antibiotic use) were neglected and may have accounted for the high overall rate of nosocomial infections. Consequently, we asked Hospital C to share with the other hospitals their protocols for care of percutaneously inserted central catheters and to learn about other aspects of infection control from the other hospitals. Thus, this process allowed us to design interventions in a more targeted way.

Figure 2

Probability of nosocomial infection at individual hospitals over time when percutaneously inserted central catheters are used (thick lines) compared to when they are not used (thin lines).

Primary outcomes

Between group comparisons

The incidence trends for bronchopulmonary dysplasia and for death and bronchopulmonary dysplasia in the infection and pulmonary groups were significantly (p < 0.05) different, but the incidence trends for nosocomial infection were similar (Table 3). The full model specification for comparison of incidence trends is given in Appendix 5 (available at www.cmaj.ca/cgi/content/full/cmaj.081727/DC2).

Table 3

Primary outcomes in the fourth quarter of year 1 and year 2 in the cluster randomized trial of the Evidence-based Practice for Improving Quality method for reducing nosocomial infections and bronchopulmonary dysplasia

Within group comparisons

In the infection group, there was a significant (p < 0.01) decrease in the incidence of nosocomial infection (32% decrease from baseline [25.4%] to last quarter [17.4%]; OR for 1-year incidence change 0.82; 95% confidence interval [CI] 0.72–0.93) (Table 3). There was no significant change in the incidence of bronchopulmonary dysplasia (4% decrease from baseline [31.8%] to last quarter [30.6%]; OR for 1-year incidence change 0.98; 95% CI 0.85–1.12) or death and bronchopulmonary dysplasia (6% decrease from baseline [37.8%] to last quarter [35.6%]; OR for 1-year incidence change 0.90; 95% CI 0.72–1.14).

In the pulmonary group, there was a significant (p < 0.01) decrease in the incidence of nosocomial infection (45% decrease from baseline [16.0%] to last quarter [8.8%]; OR for 1-year incidence change 0.77; 95% CI 0.66–0.90), bronchopulmonary dysplasia (15% decrease from baseline [29.4%] to last quarter [24.9%]; OR for 1-year incidence change 0.70; 95% CI 0.70–0.91), and death and bronchopulmonary dysplasia (12% decrease from baseline [35.0%] to last quarter [30.7%]; OR for 1-year incidence change 0.80; 95% CI 0.70–0.92). The p value for the Hosmer–Lemeshow goodness of fit test for the trend analysis to examine change in incidence of outcomes was 0.83 for the nosocomial infection model and 0.64 for the bronchopulmonary dysplasia model, which indicate good model fit.

Secondary outcomes

Trend analysis showed a significant (p < 0.01) decrease in the incidence of retinopathy of prematurity in the infection group (baseline 30% v. last quarter 13.9%; OR for 1-year incidence change 0.74; 95% CI 0.59–0.93) (Table 4). There were no other significant changes in the incidence trends of mortality, cranial ultrasound abnormalities and necrotizing enterocolitis in the infection or pulmonary groups. There was no change in the incidence trend for retinopathy of prematurity in the pulmonary group.

Table 4

Incidence of secondary outcomes at baseline, midway through the intervention and at the end of the study (fourth quarter of year 1 and 2) and incidence trends from generalized linear mixed effects regression models

There were no significant changes in the incidence trends of nosocomial infection, bronchopulmonary dysplasia, death and bronchopulmonary dysplasia, mortality, retinopathy of prematurity, necrotizing enterocolitis or cranial ultrasound abnormalities in the comparison group.

Interpretation

The Evidence-based Practice for Improving Quality method builds on existing continuous quality-improvement methods,²⁷ which usually advocate adoption of an entire package of practices from hospitals with good outcomes. In contrast, our method enables hospitals to select practice changes pertinent to them for targeted intervention. This is potentially more efficient and cost-effective.

We found that interventions aimed at one outcome may affect other outcomes. We speculate that the decrease in the incidence of nosocomial infections in the pulmonary group was related to improved lung status and a reduced need for assisted respiration, invasive interventions, improved feeding and growth, and better overall health. Because quality improvement is about transforming behaviours, spill-over from one outcome to another may be expected. Although it may be advantageous to target multiple outcomes simultaneously because of potential synergies, the possibility that improving one outcome may worsen another should be considered.

It is possible that the Hawthorne effect²⁸ (i.e., the effect on individuals of being observed for the purposes of research) or that case findings changed once a hospital knew what outcome they were targeting could account for the study results. Because these effects were probably larger for nosocomial infection than bronchopulmonary dysplasia, this may explain why nosocomial infection decreased in both the infection and pulmonary groups. It is also possible that the larger proportion of outborn infants affected the magnitude of the reduction of the incidence of nosocomial infections in the infection group. However, because the incidence of bronchopulmonary dysplasia declined only in the pulmonary group, even though there was a lower baseline incidence in this group, the results were more likely because of the study interventions rather Hawthorne or case-finding effects.

We used a combination of qualitative and quantitative methods, as advocated by Pawson and Tilley.²⁹ Qualitative methods were especially useful for identifying hospital-specific organizational issues, barriers to change and potential change strategies.¹⁷ Interviews also generated greater staff awareness, made individuals feel that their opinions mattered, enhanced staff buy-in and facilitated practice change.¹⁷ Quantitative methods provided feedback of results that reinforced behavioural change.

The Evidence-based Practice for Improving Quality method is a multidimensional approach aimed at changing organizational culture and sustaining behavioural change. Consequently, it may work best when it is adopted in its entirety rather than in parts. Although we cannot explain the difference in results reported by Walsh and colleagues⁴ and our study without more detailed information about the study by Walsh and colleagues, it is possible that the multidimensional processes for organizational and behavioural change were more rigorously implemented in our study.

In this study, not all aspects of the Evidence-based Practice for Improving Quality process were consistently implemented across all units. Thus, it is difficult to identify exactly which interventions were most effective. Because our model is a method rather than a prescription of interventions, it is potentially generalizable to other neonatal ICUs, but it will need to be tested by others. Finally, we caution that this method is not a substitute for randomized controlled trials, but it may be more appropriate when investigating the effects of change in complex adaptive systems.³⁰

Limitations

The limitations of our study included the subjective definition of bronchopulmonary dysplasia; concerns about anonymity that may have precluded feedback of site-specific qualitative results; logistical delays in reporting that sometimes impaired our ability to provide control charts in a timely manner; the lack of uniform criteria for routine screening, which may have decreased the incidence of retinopathy of prematurity in the infection group; and the withdrawal of a large neonatal ICU from the infection group. The study was stopped early even though we were about 500 patients short of the estimated sample size because we felt that there was sufficient power to analyze the results.

Conclusions

Our study suggests that the Evidence-based Practice for Improving Quality method is effective in reducing the incidence of nosocomial infection and bronchopulmonary dysplasia in neonatal ICUs. The method is potentially generalizable to other areas of health care and may improve the efficiency and reduce the costs of quality improvement efforts.

Presented in part at the Pediatric Academic Societies Annual Meeting, San Francisco, California, 2006.

Supplementary Material

[Résumé]

Click here to view.

[Online Appendices]

Click here to view.

Acknowledgements

We thank Ross Baker, Robert Liston, Jochen Moehr, Abraham Peliowski, Paul Thiessen and Elizabeth Whynot for their advice and insights, Claudio Martin and Robert Platt, who were members of the Data Monitoring and Safety Committee, and the staff of the Canadian Neonatal Network EPIQ Study Coordinating Center (Aireen Wingert, Kate Zhang, Qiaohao Zhu and Carmen Reider) for their tireless and diligent work.

Footnotes

This article has been peer reviewed.

Previously published at www.cmaj.ca

Competing interests: None declared.

Contributors: Shoo Lee was the principal investigator. He conceived and designed the study, obtained peer-reviewed grant funding, conducted the study, analyzed and interpreted the results and wrote the manuscript. Khalid Aziz, Nalini Singhal, Catherine Cronin, Andrew James, David Lee, Derek Matthew, Arne Ohlsson, Koravangattu Sankaran, Mary Seshis, Anne Synnes, Robin Walker and Robin Whyte were site investigators. They contributed to the study design and grant application, supervised research assistants and conduct of the study at their respective hospitals, participated in the development and implementation of practice change strategies, interpreted the data and reviewed and edited the manuscript. Joanne Langley, Ying MacNab, Arne Ohlsson, Bonnie Stevens and Peter von Dadelszen are members of the research methods committee. They designed and conducted a training program for participants in study methodology, evidence review, quality improvements techniques, team building and practice change strategies, and were a resource for the site investigators. They contributed to the study design and grant application, interpreted the data and reviewed the edited manuscript. Joanne Langley also provided expert advice on infection control strategies, and Bonnie Stevens conducted a qualitative study to examine barriers to change. All of the authors approved the final version submitted for publication.

Funding: This study was supported by grant MOP-53115 from the Canadian Institutes of Health Research. Additional funding was provided by BC’s Children’s Hospital Foundation; Calgary Regional Health Authority; Dalhousie University Neonatal-Perinatal Research Fund; Division of Neonatology, Children’s Hospital of Eastern Ontario; Child Health Program, Health Care Corporation of St John’s; Janeway Hospital Foundation; Newfoundland and Labrador Centre for Applied Health Research; The Neonatology Program, Hospital for Sick Children; Lawson Research Institute; Mount Sinai Hospital; Ontario Ministry of Health and Long-Term Care; Saint Boniface Hospital, Saint Joseph’s Health Centre; University of Saskatchewan Neonatal Research Fund; University of Western Ontario; Victoria General Hospital; and the Winnipeg Health Sciences Centre.

REFERENCES

1. O’Connor GT, Plume SK, Olmstead EM, et al. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. The Northern New England Cardiovascular Disease Study Group. JAMA. 1996;275:841–6. [PubMed]

2. Horbar JD, Rogowski J, Plsek PE, et al. Collaborative quality improvement for neonatal intensive care. NIC/Q Project Investigators of the Vermont Oxford Network. Pediatrics. 2001;107:14–22. [PubMed]

3. Horbar JD, Carpenter JH, Buzas J, et al. Collaborative quality improvement to promote evidence based surfactant for preterm infants: a cluster randomized trial. BMJ. 2004;329:1004–7. [PMC free article] [PubMed]

4. Walsh M, Laptook A, Kazzi SN, et al. A cluster randomized trial of benchmarking and multimodal quality improvement to improve survival free of bronchopulmonary dysplasia in infants <1250 grams birthweight. Pediatrics. 2007;119:876–90. [PubMed]

5. Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff. 2005;24:138–50. [PubMed]

6. Tarnow-Mordi WO, Parry G. The CRIB Score. Lancet. 1993;342:193–8. [PubMed]

7. Sankaran K, Chien LY, Walker R, et al. Variations in mortality rates among Canadian neonatal intensive care units. CMAJ. 2002;166:173–8. [PMC free article] [PubMed]

8. Chien LY, Whyte R, Thiessen P, et al. Canadian Neonatal Network. SNAP-II predicts severe intraventricular hemorrhage and chronic lung disease. J Perinatol. 2002;22:26–30. [PubMed]

9. Evans N, Hutchinson J, Simpson JM, et al. Prenatal predictors of mortality in very preterm infants cared for in the Australian and New Zealand Neonatal Network. Arch Dis Child Fetal Neonatal Ed. 2007;92:F34–40. [PMC free article] [PubMed]

10. Marshall G, Tapia JL, D’Apremont I, et al. A new score for predicting neonatal very low birth weight mortality risk in the NEOCOSUR South American Network. J Perinatol. 2005;25:577–82. [PubMed]

11. Richardson DK, Corcoran JD, Escobar G, et al. Canadian NICU Network. SNAP-II and SNAPPE-II – newborn illness severity and mortality risk scores. J Pediatr. 2001;138:92–100. [PubMed]

12. Synnes AR, Chien LY, Peliowski A, et al. Canadian Neonatal Network. Variations in intraventricular hemorrhage incidence rates among Canadian neonatal intensive care units. J Pediatr. 2001;138:525–31. [PubMed]

13. MacNab YC, Qiu Z, Gustafson P, et al. Hierarchical Bayes analysis of multilevel health services data: a Canadian neonatal mortality study. Health Serv Outcomes Res Methodol. 2004;5:5–26.

14. Lee SK, McMillan DD, Ohlsson A, et al. Variations in outcomes and practices in the Canadian NICU Network: 1996–1997. Pediatrics. 2000;106:1070–9. [PubMed]

15. Alemi F, Moore S, Headrick L, et al. Rapid improvement teams. Jt Comm J Qual Improv. 1998;24:119–29. [PubMed]

16. Plsek PE. Quality improvement methods in clinical medicine. Pediatrics. 1999;103:203–14. [PubMed]

17. Stevens B, Lee SK, Law M, et al. Canadian Neonatal Network EPIQ Study Group. A qualitative examination of changing practice in Canadian neonatal intensive care units. J Eval Clin Pract. 2007;13:287–94. [PubMed]

18. Plsek PE. An introduction to control charts. Qual Manag Health Care. 1992;1:65–7. [PubMed]

19. Garner JS, Jarvis WR, Emori TG, et al. CDC definitions for nosocomial infections, 1998. Am J Infect Control. 1988;16:128–40. [PubMed]

20. Shennan AT, Dunn MS, Ohlsson A, et al. Abnormal pulmonary outcomes in preterm infants: prediction from oxygen requirement in the neonatal period. Pediatrics. 1988;82:527–32. [PubMed]

21. Fetus and Newborn Committee, Canadian Pediatric Society. Routine screening cranial ultrasound examinations for the prediction of long term neurodevelopmental outcomes in preterm infants. Paediatr Child Health. 2001;6:39–43. [PMC free article] [PubMed]

22. Committee members. An international classification of retinopathy of prematurity. Pediatrics. 1984;74:127–33. [PubMed]

23. Reese AB, King MJ, Owens WC. A classification of retrolental fibroplasia. Am J Ophthalmol. 1953;36:1333–5. [PubMed]

24. Bell MJ, Ternberg JL, Feigin RD, et al. Neonatal necrotizing enterocolitis. Therapeutic decisions based upon clinical staging. Ann Surg. 1978;187:1–7. [PubMed]

25. Stroup WW. Power analysis based on spatial effects mixed models. A tool for comparing design and analysis strategies in the presence of spatial variability. J Agric Biol Environ Stat. 2002;4:491–511.

26. Chien LY, MacNab YC, Aziz K, et al. Variations in central venous catheter related infection risks among Canadian neonatal intensive care units. Pediatr Infect Dis J. 2002;21:505–11. [PubMed]

27. Batalden PB, Davidoff F. What is quality improvement and how can it transform health care? Qual Saf Health Care. 2007;16:2–3. [PMC free article] [PubMed]

28. Hart CWM. The Hawthorne experiments. Can J Econ Polit Sci. 1943;9:150–63.

29. Pawson R, Tilley N. Realistic evaluation. London (UK): Sage Publications; 1997.

30. Berwick DM. The science of improvement. JAMA. 2008;299:1182–4. [PubMed]

Articles from CMAJ : Canadian Medical Association Journal are provided here courtesy of
Canadian Medical Association