|Home | About | Journals | Submit | Contact Us | Français|
Safety culture may influence patient outcomes, but evidence is limited. We sought to determine if intensive care unit (ICU) safety culture is independently associated with outcomes.
Cohort study combining safety culture survey data with the Project IMPACT Critical Care Medicine (PICCM) clinical database.
Thirty ICUs participating in the PICCM database.
A total of 65 978 patients admitted January 2001–March 2005.
Hospital mortality and length of stay (LOS).
From December 2003 to April 2004, we surveyed study ICUs using the Safety Attitudes Questionnaire-ICU version, a validated instrument that assesses safety culture across six factors. We calculated factor mean and percent-positive scores (% respondents with mean score ≥75 on a 0–100 scale) for each ICU, and generated case-mix adjusted, patient-level, ICU-clustered regression analyses to determine the independent association of safety culture and outcome.
We achieved a 47.9% response (2103 of 4373 ICU personnel). Culture scores were mostly low to moderate and varied across ICUs (range: 13–88, percent-positive scores). After adjustment for patient, hospital and ICU characteristics, for every 10% decrease in ICU perceptions of management percent-positive score, the odds ratio for hospital mortality was 1.24 (95% CI: 1.07–1.44; P = 0.005). For every 10% decrease in ICU safety climate percent-positive score, LOS increased 15% (95% CI: 1−30%; P = 0.03). Sensitivity analyses for non-response bias consistently associated safety climate with outcome, but also yielded some counterintuitive results.
In a multicenter study conducted in the USA, perceptions of management and safety climate were moderately associated with outcomes. Future work should further develop methods of assessing safety culture and association with outcomes.
Safety culture has been defined as ‘the product of individual and group values, attitudes, perceptions, competencies and patterns of behavior that determine the commitment to, and the style and proficiency of, an organization's health and safety management’ . Despite high profile calls for enhancing safety culture in medicine [2, 3], the actual impact on patient outcomes is unclear. Early investigations in critical care showed conflicting results. A 1986 study examined the role of organizational factors in 13 hospitals and found that intensive care unit (ICU) personnel coordination and interaction appeared to relate to severity-adjusted mortality . However, two follow-up studies only found an association between organizational culture and length of stay (LOS), but not mortality [5, 6]. A possible reason for these disparate results is that the first study relied on site visits and ICU directors' assessments, although the subsequent studies directly surveyed personnel. Studies by our group and others have shown that ICU directors may not accurately estimate their units' culture [7, 8]. Other studies also suggested a potential relationship between ICU culture and patient outcomes [9–11], but a recent review concluded that, whereas some evidence existed, articulating the nature of that relationship was difficult. In particular, stronger methodologic definitions and operationalizations of both culture and outcomes were recommended .
We therefore conducted a multicenter cohort study of ICU safety culture and outcomes: ICUTEAMS—Intensive Care Unit Teamwork, Error, and Attitudes towards Management Survey. We directly surveyed personnel in a large network of ICUs located in the USA, using a validated, aviation safety research based survey instrument designed to measure ICU safety culture. We then linked safety culture data with contemporaneous ICU outcome and administrative data, and adjusted for differences in severity of illness using a well-established clinical and physiologic risk adjustment tool. Our primary objective was to determine if ICU safety culture is independently associated with patient hospital mortality and LOS.
We conducted a cohort study combining ICU safety culture survey data with the Project IMPACT Critical Care Medicine (PICCM) clinical database and generated patient-level, ICU-clustered regression models to determine the independent association of ICU safety culture with patient hospital mortality and LOS. The study cohort consisted of 30 of the 115 PICCM ICUs, whose medical and nursing directors agreed to participate in our study. PICCM is a US ICU network that collects and aggregates outcome data for benchmarking, quality improvement and research, and was founded by the Society of Critical Care Medicine. Member ICUs submit data quarterly or more frequently for all admitted patients or a random sample of at least 50% of all admitted patients. Data collectors are given detailed operational definitions, clinical support and training. Data on organizational and staffing characteristics are provided by the ICUs to PICCM. An independent study found high agreement rates between the PICCM database and patient charts . The University of Pittsburgh Institutional Review Board approved the study protocol.
We measured each study ICU's safety culture by surveying their personnel and aggregating their responses. Between December 2003 and April 2004, we conducted a mailed, self-administered survey of all personnel in participating PICCM ICUs, using a safety culture survey instrument.
The nursing directors or their designates served as the study coordinators for their ICUs. ICU personnel were defined as those personnel whose clinical work was based in the study coordinators' ICU (e.g. ICU nurses, intensivists) or personnel the study coordinators deemed to have a significant work commitment to their ICU (e.g. consultants, pharmacists), including part-time employees, residents and ward clerks, and who had worked in their ICU at least 1 month prior to survey administration. We worked closely with each nursing director to ensure accurate capture of all ICU personnel.
We shipped to study coordinators all materials needed for local survey administration, including detailed instructions, a copy of their ICU's personnel list, extra questionnaires, posters and sealed, addressed envelopes for each ICU personnel member. These envelopes contained a questionnaire, cover letter, pencil with eraser and a postage-paid return envelope (direct to our research office). Response rates by job category were monitored biweekly and communicated to the study coordinators to aid follow-up of low-responder groups. Study coordinators were encouraged to distribute surveys at routine staff meetings, given extra surveys in case of loss and asked to hang posters advertising the survey in prominent areas (e.g. break rooms). We offered a coffee and breakfast reward to the ICU with the highest response rate to incentivize survey response.
We assigned unique site codes to each ICU and pre-printed these codes on the questionnaires. This protected ICU and personnel confidentiality and allowed monitoring of response rates. Questionnaires were only identified by site code and job category and contained no personal identifiers. Return envelopes were only labeled with our research office's mailing address.
We used the Safety Attitudes Questionnaire (SAQ)—ICU version , a validated, healthcare adaptation of the Cockpit Management Attitudes Questionnaire . The Cockpit Management Attitudes Questionnaire is a survey instrument developed over 30 years ago by safety researchers to measure attitudinal determinants amongst aviation personnel, has been widely used in multiple airlines and countries, and is reliable, sensitive to change and predictive of flight crew performance [15–17]. The SAQ was specifically designed to measure safety culture at both the individual and group level (e.g. at the ICU or job category level). Both the healthcare and aviation versions of this survey instrument were shown to identify variability within and between hospitals  and airlines , and the SAQ's full derivation, validation and psychometric properties were recently published . Briefly, the SAQ is a one-page, 60 item survey instrument that assesses safety culture across six factors—perceptions of management, job satisfaction, working conditions, stress recognition, teamwork climate and safety climate. The SAQ defines safety climate as perceptions of a strong and proactive organizational commitment to safety, as one aspect of overall safety culture. Each item is measured on a 5-point Likert scale (Disagree Strongly to Agree Strongly), which is then converted to a 0–100 scale. Each factor score equals the mean score of its component survey items. A positive score is defined as ≥75 out of 100. To count as a positive score for a given factor, the survey respondent must answer, on average, Agree Slightly or higher to all related items. Group factor scores are quantified in two ways: (i) the mean scores of all group members and (ii) the percentage of group members with positive scores. An ICU's factor scores are therefore composed of its personnel's factor scores. Mean score provides a rough point estimate, whereas percent-positive more precisely assesses the homogeneity of factors within a group. For example, in an ICU where half the personnel report teamwork climate scores of 100, although the other half report scores of 50, the mean ICU score would be a positive score of 75. Percent-positive would identify that only half of this ICU's personnel felt positively about teamwork and therefore more accurately captures the degree of variability in perceptions of teamwork within this ICU.
Our primary outcomes were patient hospital mortality and LOS. To avoid counting multiple outcomes for a single patient, only the first ICU admission for each patient was considered. PICCM provided outcome data for each study ICU, linked to our safety culture data. We addressed potential confounding by controlling for severity of illness and additional variables related to outcome in critical care.
After survey completion, we sent the survey data to PICCM for linkage to contemporaneous (January 2001–March 2005) outcome data for each participating ICU. PICCM also provided patient characteristics, ICU and hospital administrative data. PICCM then served as honest broker and de-identified the ICUs. The final dataset therefore consisted of patient characteristics and outcomes, and for each patient, the safety culture of their ICU, as well as the administrative characteristics of their ICU and hospital. We received this final dataset from PICCM in June 2005.
We assessed severity of illness using SAPS II (Simplified Acute Physiology Score), a validated severity score that assigns weighted points for age, type of ICU admission, coexisting illness and physiologic abnormalities . We also adjusted for hospital and ICU characteristics (intensivist staffing model, number of ICU beds, ICU type). For our analysis, we dichotomized intensivist staffing into ‘high-intensity’ [mandatory intensivist consultation or closed ICU [all care directed by intensivist)] and ‘low-intensity’ (no intensivist or elective intensivist consultation) . As almost all study ICUs were non-academic and had 2:1 patient–nurse ratios, we did not include academic status and patient–nurse ratios in the final models.
We applied the χ2 test, Student's t-test or non-parametric counterpart for the univariate comparisons. We constructed a multivariable logistic regression model for patient hospital mortality and linear regression model for LOS, using the generalized estimating equations approach to account for possible clustering effect due to ICU membership and natural log transformation of LOS. We first included the six safety culture factors in the models, quantified for each ICU using percentage of positive scores. We began by individually modeling each culture factor alone, and then included all six factors in the multivariable models to determine whether individual associations remained significant. We found no indication of significant collinearity between the six factors as determined by variance inflation factors and eigenvalues. We then sequentially added patient severity of illness to the models using SAPS II, followed by hospital and ICU-level variables, to better understand the relationship between safety culture and the primary outcomes. We calculated regression coefficients and associated P-values for each safety culture factor and estimated the odds ratio of hospital mortality and the percent change in LOS for every 10% decrease in an ICU's percentage of positive scores.
In post hoc sensitivity analysis, we added ICU survey response rate to the final models as tertiles and as a continuous variable and assessed the impact on results. We also reran the final models using only those ICUs with a ≥50% survey response rate and compared the characteristics of ICUs with <50% response vs. those with a ≥50% response. Lastly, we plotted ICU-level survey response rate vs. clinical performance index as per Rapoport et al. . This index is the difference between observed hospital survival rate and survival rate predicted by severity of illness at ICU admission. We also examined survey response rate vs. hospital and ICU characteristics. All statistical analyses were performed in SAS 9.1 (SAS Institute, Cary, NC, USA), assuming statistical significance at P < 0.05.
The 30 study ICUs were mostly community, non-profit, mixed medical/surgical units with low-intensity intensivist coverage and 2:1 patient–nurse ratios. Patients (n = 65 978) were predominantly white and elderly, with a mean SAPS II score of 31.6, hospital mortality of 12.7% and median hospital LOS of 7 days. The most frequent admission diagnostic categories were coronary artery disease (7.5%), medical (5.9%) and post-operative (5.9%) cardiovascular dysfunction and sepsis (5.6%) (Table 1).
Overall survey response rate was 47.9% (2103/4394), with variation between ICUs (17.2–98.4%) and personnel categories (8.6–77.8%; Table 2). Nurses had a higher response rate than physicians (57.3 vs. 24.6%, P < 0.0001) and critical care physicians had a higher response rate than non-critical care physicians (39.7 vs. 21.5%, P < 0.0001). Nurses comprised 41.6% of all personnel and returned 49.8% of all surveys.
Safety culture factor scores were mostly low to moderate, and varied widely across ICUs (range: 13–88, percentage of positive scores; Fig. 1). Of the six factors, job satisfaction varied the most across ICUs (range: 27–88 percentage of positive scores) and stress recognition the least (range: 26–62 percentage of positive scores). Nurses had lower safety culture scores than physicians, for all factors and significant within-ICU safety culture variation existed between personnel (Appendix 1).
Lower perceptions of management were significantly associated with higher hospital mortality (Fig. 2, top panel). This relationship was seen in individual modeling of perceptions of management alone, and in multivariate modeling of all culture scores without adjustment (P = 0.02), with adjustment for patient severity of illness (P = 0.001) and with additional adjustment for hospital and ICU characteristics (P = 0.005; Table 3, top panel). In the final model, for every 10% decrease in an ICU's percentage of positive scores for perceptions of management, the increased odds of death were 1.24 (95% CI: 1.07–1.44). An equivalent odds ratio was observed for every 2.4 point increase in SAPS II score (1.24, 95% CI: 1.22–1.25; P < 0.0001). No other safety culture factor had a significant relationship with hospital mortality.
In individual culture score modeling, lower safety climate, perceptions of management and job satisfaction were significantly associated with increased hospital LOS. In multivariate modeling of all culture scores, only lower safety climate was significantly associated with increased hospital LOS in the final model accounting for patient, hospital and ICU characteristics (P = 0.03; Fig. 2, bottom panel). For every 10% decrease in an ICU's percentage of positive scores for safety climate, LOS increased 15% (95% CI: 1–30%; Table 3, bottom panel). In comparison, for every 10 point increase in SAPS II score, LOS increased 7% (95% CI: 5–9%; P < 0.0001).
Adding ICU-level survey response rate to the final models had no significant effect on results. ICU-level and patient characteristics were overall similar between ICUs with <50% (n = 13) and ≥50% (n = 17) response rates, although mortality was slightly lower in the higher response rate ICUs (11.9 vs. 14.0%) (Appendix 2). Running the models using only ICUs with ≥50% response rates altered results. For the mortality model, perceptions of management lost significance (P = 0.94) and lower safety climate became significantly associated with higher mortality (P = 0.002). Higher response ICUs had somewhat higher (mean 33.3 vs. 30.2%) and more uniform (SD 9.4 vs. 13.1%) perceptions of management percent-positive scores vs. lower response ICUs. Counterintuitively, lower working conditions became associated with lower mortality (P = 0.02). For the LOS model, lower safety climate remained significantly associated with higher LOS (P < 0.0001). Lower job satisfaction also became associated with higher LOS, and counterintuitively, lower working conditions, stress recognition and teamwork climate became associated with lower LOS (all P < 0.0001). We found no significant relationship between ICU-level survey response rate and clinical performance index, ICU type and number of ICU beds (Appendix 3). ICUs with high-intensity intensivist staffing had lower survey response rates than low-intensity ICUS (44 vs. 54%).
Our study represents one of the largest, multicenter attempts to examine the relationship between safety culture and outcomes. We found that lower perceptions of management were independently associated with increased hospital mortality and that the magnitude of this association was comparable to a moderate increase in SAPS II score, such as from severe hypertension or advanced age. We also found that lower safety climate, expressed as perceptions of organizational commitment to safety, was independently associated with increased hospital LOS. However, our study's moderate survey response and lack of robustness to sensitivity analysis limit our ability to draw definitive conclusions.
Perceptions of management refers to approval of hospital managerial actions, and is derived from survey items such as ‘Hospital administration supports my daily efforts’ and ‘Hospital management does not knowingly compromise the safety of patients’ . Although West et al.  surveyed human resource directors, and not the personnel, of acute care hospitals, they similarly found association between human resource practices and patient mortality. Several reasons might explain our finding. First, poor perceptions of management may represent poor management per se, or poor perceptions of in fact effective management. However, as part of effective management is communication and attention to perception, poor personnel perceptions of management likely do reflect poor management. Second, ICU personnel that disapprove of hospital management may feel less vested in their work, with attendant decrement to bedside patient care. Third, high patient mortality itself may lead to personnel burnout and stress, which may worsen perceptions of hospital management. Lastly, and most concerning, poor perceptions of management may reflect poor hospital management practices that negatively impact patient outcome.
Safety climate refers to perceptions of a strong and proactive organizational commitment to safety, and is derived from survey items such as ‘Medical errors are handled appropriately in this ICU’ and ‘I would feel safe being treated here as a patient’ . Poor attention to safety may lead to poor care, medical errors and subsequent increased LOS. A small study recently reported that implementation of a comprehensive unit-based safety program in two ICUs resulted in improved safety climate, as well as reduction in LOS and medication errors . This pilot interventional study supports the notion that safety climate and LOS are related and provides preliminary evidence that interventions targeted at improving safety can improve both culture and outcome.
Bedside caregivers have direct knowledge of how hospital policy affects patient care and can provide unvarnished insight into the safety culture of their hospital. Despite the inherent ‘chicken or egg’ caveat of observational studies, our results suggest that increased attention to ICU personnel's perceptions of their hospital's management practices and commitment to patient safety may be warranted, and that ICUs that score poorly in these areas merit evaluation. Of note, our study's lack of finding of association between the other safety culture factors and outcome does not mean that, for example, teamwork should be ignored. Our null finding may represent the relatively greater importance of perceptions of management and safety climate for outcome, or that our study design or survey instrument lacked the power or sensitivity to find smaller associations. A recent study that failed to find a significant association between Leapfrog safety survey scores and hospital mortality similarly concluded that their null result may have been due to lack of power, but that their findings should not be interpreted as indicating that safe practices are not important . Non-response bias may also have limited our ability to detect a signal. Although difficult to determine bias direction, it is possible that ICU personnel dissatisfied with their work environment had lower response rates. This would result in falsely higher safety culture scores and obscure any safety culture–outcome relationship. Our results illustrate the challenge of measuring safety culture amongst healthcare personnel, who often have poor response rates . Future work should examine the determinants of survey non-response and safety culture in healthcare, and whether interventions and management changes that improve safety culture also improve outcome.
Our study's main limitation is its moderate response rate. We used multiple strategies to minimize non-response, including endorsement from PICCM, use of a 1-page questionnaire and incentives . However, achieving high response rates among healthcare personnel is a well-known challenge in survey research, and our overall response rate of almost half is comparable to recent safety culture [28, 29] and provider workforce studies [30, 31]. We attempted to describe potential non-response bias by performing multiple post hoc sensitivity analyses. Adding ICU-level survey response rate to our regression models had minimal impact and we found little evidence for a significant relationship between ICU-level survey response rate, and severity-adjusted mortality and hospital and ICU characteristics. However, restricting analyses to those ICUs with higher response rates yielded some counterintuitive results and perceptions of management no longer appeared significant. The lack of robustness to post hoc sensitivity analysis for perceptions of management may be due to a true lack of association, or lower power from the smaller number of analyzed ICUs. The lower mortality of higher response ICUs, combined with their higher and more uniform perceptions of management scores, may have also made detection of a culture–outcome relationship difficult. Notably, worse safety climate was consistently associated with longer LOS, even when restricting analysis to the higher response ICUs. This suggests that safety climate may be the culture factor most robustly associated with patient outcome. A recent study similarly found that hospitals with better safety climate had lower relative incidence of patient safety related adverse events .
Our study cohort was a self-selected group of predominantly community, non-profit, mixed medical/surgical ICUs that devote financial and staffing resources to participate in PICCM, and thus generalizability to other ICUs may be limited. We also were only able to recruit a subset of all PICCM ICUs. A previous safety culture study similarly found it necessary to contact more than 90 hospitals to obtain a 30 hospital sample and reported that safety performance was not consistently related to study participation . However, analyzing ICUs with a high degree of uniformity and commitment to self-appraisal would likely bias against finding a signal between safety culture and patient outcomes. Although determining direction of bias is difficult, a broader and larger sample would have provided greater power to answer study objectives. We analyzed patient outcome data from a wider time period that encompassed our survey administration time period as organizational culture changes slowly and to provide a larger analysis sample. Significant culture variation during this time period could have potentially distorted study findings. Lastly, although most survey respondents were nurses, nurses comprise the bulk of an ICU's personnel, are physically in the ICU most, and thus nurses may contribute the most to an ICU's overall culture.
In a multicenter study of US ICUs, perceptions of management and safety climate were moderately associated with patient outcomes. Our study illustrates the significant challenges of safety culture and outcomes research. Future work should continue to develop methods for assessing safety culture and association with patient outcomes.
We gratefully acknowledge and thank the Unit Coordinators and all personnel of the participating PICCM ICUs, as well as the PICCM database managers.
|Teamwork climate||Job satisfaction||Perceptions of management||Safety climate||Working conditions||Stress recognition|
|Mean||% Positive||Mean||% Positive||Mean||% Positive||Mean||% Positive||Mean||% Positive||Mean||% Positive|
|Bedside nurse, LVN/LPN||75.9||50.0||73.5||60.0||60.6||50.0||77.5||60.0||65.6||40.0||56.7||20.0|
|Bedside nurse, RN||71.5||48.8||70.2||48.4||58.7||29.4||72.0||47.6||66.3||40.0||68.9||45.7|
|Charge nurses/nurse managers||78.4||71.3||77.8||71.3||67.5||50.0||76.9||68.7||69.5||48.0||68.9||44.0|
|Critical care attending physicians||81.6||74.7||77.8||69.9||68.7||54.2||75.5||56.6||72.2||57.8||69.1||55.4|
|Critical care fellows/residents||88.4||88.9||86.1||88.9||70.1||33.3||82.9||66.7||78.5||66.7||73.6||44.4|
|Medical attending physicians||81.6||77.0||81.9||76.4||73.3||58.8||74.8||58.1||74.4||61.5||72.2||58.8|
|Surgical attending physicians||78.2||64.2||79.2||71.6||68.5||46.3||71.1||41.8||70.3||41.8||60.5||32.8|
Within-ICU variation also existed between personnel; standard deviations were generally in the 15–25 range, for mean safety scores that were predominantly in the 50–75 range (data not shown).
LVN, licensed vocational nurse; LPN, licensed practical nurse; RN, registered nurse.
|ICUs (n= 30)||<50% response (n = 13)||≥50% response (n = 17)||Patients (n= 65 978)||<50% response (n = 13)||≥50% response (n = 17)|
|Teaching status (n)||Age, mean (SD)||62.0 (18)||61.7 (18.0)|
|Community||12||16||Female sex (%)||46.1||44.7|
|# of ICU beds (mean, SD)||19.6 (15.4)||17.3 (7.4)||White||83.2||84.2|
|ICU type (n)||Black||11.9||11.5|
|Medical||2||3||SAPS II scorea (mean, SD)||32.1 (17.1)||31.3 (16.9)|
|Patient: nurse ratio (n)||Hospital LOS, # days (median, IQR)||7 (4–13)||7 (4–12)|
|2:1||12||16||APACHE II diagnostic category (%)|
|1:1||0||1||Coronary artery disease||8.9%||6.7%|
|Other||1||0||Cardiovascular dysfunction, other||6.7%||5.5%|
|Intensivist staffing modelb (n)||Post-operative cardiovascular dysfunction, other||3.3%||7.3%|
|High-intensity||3||5||Gastrointestinal bleeding (not including shock)||4.8%||5.4%|
|# of personnel (mean, SD)||164.7 (77.3)||131.3 (61.1)||Respiratory dysfunction, other||5.3%||4.2%|
|Neurologic dysfunction, other||3.2%||4.7%|
|Post-operative, peripheral vascular surgery||4.0%||3.8%|
aData available on 54 110 patients.
bWe defined ‘high-intensity’ as mandatory intensivist consultation or closed ICU (all care directed by intensivist) and ‘low-intensity’ as no intensivist involvement or elective intensivist consultation .
We found no significant relationship between ICU-level survey response rate and clinical performance index, ICU type and number of ICU beds. ICUs with high-intensity intensivist staffing had lower survey response rates than low-intensity ICUS (44 vs. 54%).
Clinical performance index as per Rapoport et al. . This index is the difference between observed hospital survival rate and survival rate predicted by severity of illness at ICU admission.
Intensivist staffing model: High-intensity ICU % survey response (mean, SD): (44%, 14%); Low-intensity ICU % survey response (mean, SD): (54%, 21%).
ICU type: Medical ICU % survey response (mean, SD): (52%, 18%); Surgical ICU % survey response (mean, SD): (56%, 20%); Mixed ICU % survey response (mean, SD): (51%, 21%).
This work was supported by a National Institutes of Health training grant (02-T32HL07820-06) to D.T.H. during the study's execution and by an internal seed grant from the Department of Critical Care Medicine, University of Pittsburgh.