|Home | About | Journals | Submit | Contact Us | Français|
Variation in health care delivery and outcomes in NICUs may be partly explained by differences in safety culture.
To describe NICU caregiver assessments of safety culture, explore the variability within and between NICUs on safety culture domains, and test for association with caregiver characteristics.
We surveyed NICU caregivers in a convenience sample of 12 hospitals from a single health care system, using the Safety Attitudes Questionnaire (SAQ). The six scales of the SAQ include teamwork climate, safety climate, job satisfaction, stress recognition, perception of management, and working conditions. For each NICU we calculated scale means, standard deviations and percent positives (percent agreement).
We found substantial variation in safety culture domains among participating NICUs. A composite mean score across the six safety culture domains ranged from 56.3 to 77.8 on a 100-point scale and NICUs in the top four NICUs were significantly different from the bottom four (p < .001). Across the six domains, respondent assessments varied widely, but were least positive on perceptions of management (3–80% positive; mean 33.3%) and stress recognition (18–61% positive; mean 41.3%). Comparisons of SAQ scale scores between NICUs and a previously published cohort of adult ICUs generally revealed higher scores for NICUs. Physicians composite scores were 8.2 (p = .04) and 9.5 (p =.02) points higher than nurses and ancillary personnel.
Significant variation and scope for improvement in safety culture exists among this sample of NICUs. The NICU variation was similar to variation in adult ICUs, but NICU scores were generally higher than adult ICU scores. Future studies should validate whether safety culture as measured with the SAQ correlates with clinical and operational outcomes in the NICU setting.
In neonatology there is persistent variation in health care delivery and outcomes.1–7 Differences among NICUs with regard to safety culture may in part explain this phenomenon. Preterm infants are fragile, often very ill, and exposed to complex and prolonged intensive health care interventions. This makes them vulnerable to lapses in teamwork and patient safety.
The Joint Commission requires that institutions measure their safety culture on an ongoing basis.8 The Safety Attitudes Questionnaire (SAQ) is one of the most widely used instruments to measure safety culture9, has good psychometric properties10, and is responsive to interventions.11,12 In the adult care setting, a growing literature links improvements in safety culture to improvements in care, such as reduced medication errors, length of stay, nursing turnover rates13, and central line associated blood stream infections.14
Unfortunately, investigations of safety culture and teamwork in the NICU are uncommon.15–18 Safety culture instruments, such as the SAQ, should be applied to specific settings to examine and ensure that their results are meaningful, interpretable, actionable, and reliable. This study adds to the empiric body of research by describing variation of safety culture domains in a sample of twelve NICUs, highlights opportunities for improvement, and investigates the influence of respondent characteristics safety culture scores. Additionally, we investigate the use of an SAQ composite measure, scale scores, and comparisons within and outside of NICUs.
Please refer to our accompanying paper in this issue for a detailed description of the patient sample, study procedures and instrument measures (data for both papers were handled in the same manner).19 In brief, the SAQ (ICU Version) was administered to all caregivers in twelve NICUs in a faith-based non-profit health system in 2004 using procedures, which had previously yielded high response rates.20 Institutional Review Board approval was obtained for this research.19
This study addresses the following research questions (RQ) through a secondary analysis of prospectively collected data:
We investigated associations between the composite score and respondent characteristics in bivariate analyses and input characteristics associated at p ≤ .1 in a hierarchical model. Respondents were nested within NICUs. The multivariate model was developed to adjust for potential confounding of scale scores due to respondent characteristics. A two-tailed p-value < .05 was considered statistically significant.
Mean scale scores were computed for each NICU. We tested for statistical differences in scale scores between NICUs ranking among the top and bottom four NICUs. Unadjusted analyses compared the combined means of relevant NICUs via Student’s t-tests. Adjusted analyses used fixed effect linear models informed by the previous multivariate analysis to compare adjusted mean scale scores between the top and bottom four NICUs.
For comparability with previously published literature using the SAQ we present scale and item level percent positives (percent respondents answering agree slightly or agree strongly). SAQ scale scores 60% positive or higher are considered favorable, with a goal of at least 80% positive. In addition to the 12 NICUs reported here, we also evaluated the percent positive scale scores between the NICUs and a previously published cohort of 71 adult ICUs22 using independent samples t-tests.
We received completed surveys from 547 of 639 respondents; overall response rate was 86% (range, 69–100%). Table 1 presents response rates and respondent characteristics. Appendix 1 displays respondent characteristics by site. After adjusting for respondent characteristics only NICU site was independently associated with the composite. There was a trend that job position was associated with the composite (p = .06). Comparisons between job positions and the composite revealed that on average, nurses and ancillary personnel rated safety culture 8.2 (p = .04) and 9.5 (p = .02) points lower than doctors. There was little difference between nurses and ancillary personnel. Figure 1 exhibits the differences in composite safety culture scores between nurses and doctors by NICU.
We found wide variation in mean scores across each of the domains, and even more so between percent positive scores (see Table 2). Safety culture composite scores varied by over 20 points across NICUs and individual scale scores ranged even greater. After adjusting for job position, patient safety attitudes varied widely between the top and bottom four NICUs across all domains. For illustration, we exhibit mean item scores from the teamwork and perception of management scales (see Table 3).
Respondent and NICU level variability by scale and item are exhibited in Table 4. These items are used for calculation of scale scores and constitute the SAQ Short Form, the version most widely used at this time. On average, about half of the respondents in a given NICU reported good teamwork climate (54%) and good safety climate (55%), one out of three reported positive perceptions of management (33%), and almost two out of three reported good job satisfaction (63%).
The lowest percent positive scale score across the 12 NICUs was perceptions of management: 37% agreed that administration supported their daily efforts, and 43% agreed that staffing levels were sufficient to handle the number of patients.
Compared with the adult ICU sample, NICUs exhibited significantly higher percent positive (percent agreement) domain scores for safety climate (t = −2.588, p = .011), job satisfaction (t = −2.043, p = .044), and working conditions (t = −2.298, p = .024).
Key findings include: (1) significant variation in safety culture domains among this sample of NICUs; (2) opportunities for improvement within all domains measured by the SAQ; (3) patterns of more positive safety culture domains in NICUs relative to a large sample of adult ICUs from the same time period and (4) a trend for an association between job position and safety culture where physicians assess teamwork and safety related norms more positively than nurses and other ancillary personnel.
Our finding of significant variation in patient safety culture among NICUs is consistent with the existing literature. We previously demonstrated significant between and within hospital variation in safety culture domains among 203 clinical sites from health care facilities in the United Kingdom, the United States, and New Zealand,10 consistent with Singer and colleagues’ work.23,24
Based on the mean scale scores, the safety profile of the participating NICUs was quite variable in that some NICUs excelled in one subscale but lagged in another. For example NICU E may find opportunities for improvement in interventions that address teamwork climate, whereas NICU F may focus on stress recognition and perceptions of management. This implies that one safety culture intervention bundle is unlikely to be useful for all; rather interventions may need to be tailored to individual NICU profiles. Moreover, a composite index of the 6 safety culture domains, while statistically justifiable, but may significantly dilute the domain-level variability within a NICU.
The variation demonstrated across the 12 NICUs suggests opportunities for improvement in safety culture and quality of care. A growing literature demonstrates such a link. For example, the SAQ has demonstrated sensitivity to quality improvement interventions in the operating room25 and obstetric setting26, and has been associated with reduced blood stream infections14, medication errors and lengths of stay.13 In addition, safety culture has been shown to predict success in implementing complex quality improvement projects22, implying that attention to safety culture may be a necessary co-intervention for many quality improvement projects.
Responses to the SAQ exhibited two general themes: perceived lack of support from hospital management and concerns about aspects of teamwork and collaborative communication regarding errors. These issues are amenable to intervention that could be studied in future work.
With regard to perceptions of management, many providers expressed a lack of support in their daily efforts by management and that staffing may be insufficient to handle the work load. Managers may not easily be able to remedy such grievances because of financial constraints. However, administrators could employ several managerial tools to motivate staff and create a supportive work environment in which clinical providers feel that their concerns are being considered.27 Management is essential to promoting an environment of collegiality, teamwork and common purpose. Good teamwork climate scores have been associated with lower levels of clinical caregiver burnout from their work which may have important implications for nurse retention and innovation readiness.28
Additionally, item level scores from the teamwork and perception of management domains have been shown to be sensitive to interventions. For example, collaborative rounds29, aviation based crew resource management training30, or improved communication in hierarchy31,32 can improve teamwork whereas Leadership WalkRounds33,34 can improve perceptions of management.
Compared to adult ICUs, NICU percent positive scale scores were higher in safety climate, job satisfaction, and working conditions. NICU teamwork climate and perceptions of management trended toward higher scores, though they did not reach statistical significance. Stress recognition scores were relatively low in NICUs and ICUs, and did not significantly differ between them. Overall, NICU and adult ICU samples exhibited similar amounts of variability, and the lowest percent positive scores for NICUs (perceptions of management and working conditions) were also the lowest percent positive scores for adult ICUs.
Consistent with most reports across a variety of health care settings, NICU physicians rated patient safety culture significantly higher than nurses and ancillary personnel.10,16,24,35,36 In previous work, we found that compared to physicians, nurses were less comfortable to speak up, thought their input was not well received, felt conflict resolution was often inappropriate, and desired more input to decision-making.36 Singer and colleagues suggested that nurses may experience safety deficiencies more acutely because of their tighter employment relationship with the hospital.24
Since hospitals are required to assess their safety culture on an ongoing basis, safety culture surveys might be used for comparing hospitals or units within hospitals against their peers. Systematic differences in responses by job position suggest that NICU comparisons should use consistent survey inclusion criteria such that representative and meaningful results can be garnered through questionnaire assessments. This would reduce the ability of a NICU to improve its ratings artificially by preferentially asking physicians to respond.
The results of this study must be seen within the context of its observational design. Associations between safety attitudes and other variables do not necessarily indicate causality. Our findings may be confounded by unobserved variables, such as respondent income or experience in other health care settings. Within the confines of the available dataset, we have tried to maximize internal validity by adjusting for possible confounding using multivariate modeling. Our accompanying paper further supports the use of SAQ scale scores as statistically valid consensus views for a specific NICU 19.
Administration of the SAQ during departmental meetings may potentially introduce unwanted selection and response bias due to the social desirability of positive responses and the sample of providers who attend the meetings. However, if such bias existed, it would have likely resulted in higher SAQ scores, therefore strengthening our overall conclusions.
Finally, we studied a relatively small and non-random sample of care providers in twelve NICUs. It is unclear whether our results are generalizable to other NICUs, but the significant variability found here within a small sample of NICUs improves generalizability.
We found significant variation and scope for improvement in safety culture domains among this sample of NICUs. The NICU variation was similar to variation in adult ICUs, but NICU scores were generally higher than adult ICU scores. Future studies should validate whether safety culture correlates with clinical and operational outcomes in the NICU setting.37
Grant support: Jochen Profit’s contribution is supported in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development #1 K23 HD056298-01 (PI: Profit). Dr. Petersen was a recipient of the American Heart Association Established Investigator Award (#0540043N) at the time this work was conducted. Drs. Petersen, Hysong, and Mr. Mei also receive support from a Veterans Administration Center Grant (VA HSR&D CoE HFP90-20). Dr. Hysong’s contribution is supported in part by the Department of Veterans Affairs Health Services Research and Development Program (#CD 2-07-0818). Dr. Thomas’ effort is supported in part by grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development #1 K24 HD053771-01 (PI: Thomas) and #1 PO1 HS1154401 (PI: Thomas). Dr. Sexton received support from an Agency for Healthcare Research and Quality (AHRQ) grant # 1UC1HS014246. Dr. Etchegaray’s effort is supported by a K02 award from AHRQ #1 K02 HS017145-02 and the University of Texas at Houston-Memorial Hermann Center for Quality and Safety.
In addition to thanking the NICU personnel who participated by sharing their assessments, we would like to acknowledge the contribution of the study staff, Christen Fullwood, Chris Holzmueller, Angelina Barbosa and Linda Marcellino.
The authors have no financial relationships relevant to this article to disclose.
AUTHOR CONTRIBUTIONSJP - Designed the study questions, analyses, and drafted the manuscript. JE - Contributed to the design and framing of the study questions and the analyses. He also edited the manuscript. LP - Helped to frame the study question in a variation and performance measurement context. Helped interpret study findings, reviewed and edited the manuscript. BS - Conducted the original data collection, transmitted a de-identified data set to Dr. Profit, helped frame and design study questions, interpreted study findings, and reviewed and edited the manuscript. SH - Contributed to interpretation of analysis, framing of results in an industrial/organizational context, and edited the manuscript. MM - Developed an analysis plan, conducted analyses, reviewed results with study members, and edited the manuscript. ET - Acted as senior advisor to the project. Helped with framing of study questions, interpretation of results, contextual background and review/editing of manuscript.