Overall, the results from the psychometric analyses--intraclass correlations (ICCs), design effects, MCFA results, model fit indices, item factor loadings, internal consistency reliability analyses, and dimension intercorrelations--all provide solid evidence supporting the 12 dimensions and 42 items included in the AHRQ Hospital Survey on Patient Safety Culture as having acceptable psychometric properties at the individual, unit and hospital levels of analysis, with a few exceptions. Our multilevel psychometric results indicate that both unit and hospital membership influence how individuals respond on the survey. The findings support our conclusion that the survey measures what it is supposed to: group culture at these higher levels, not just individual attitudes.
The Staffing composite fell slightly below cutoffs in a number of areas. Individual level factor analyses found that the percent of variance accounted for by Staffing fell slightly below the 50% rule of thumb, at 47%. In addition, one item in the Staffing composite had low within-unit and within-hospital factor loadings (.36--just below the .40 cutoff); the unit-level model fit was just below the .90 cutoff (CFI = .88); and the overall composite had low reliability (.62--below the .70 cutoff). Despite these findings, we recommend that the Staffing composite and items be retained due to the importance of staffing as emphasized in the 2003 Institute of Medicine report [30
]. In addition, problems with staffing are often identified as a major theme of written comments on the survey. The factor analysis and reliability results did not point to any item in the Staffing composite that if dropped would improve the psychometric properties of the composite, which also indicates that the composite cannot be improved by dropping any of its three items.
The only other composite with a problematic psychometric finding was Supervisor/Manager Expectations & Actions Promoting Patient Safety in which the hospital-level model fit was lower than the cutoff of .90 (CFI = .82). Given that all other psychometrics for this scale were good, and its conceptual importance to patient safety, we also recommend retaining this composite.
The strongest relationships among the patient safety culture dimensions were between Overall Perceptions of Patient Safety and Patient Safety Grade and Management Support for Patient Safety. These strong correlations attest to the construct validity of the Overall Perceptions of Patient Safety composite. The findings also point to the important role hospital management plays in achieving patient safety [31
] since staff rated their units higher on Patient Safety Grade when they perceived that hospital management supported patient safety.
Surprisingly, the weakest relationship was between Nonpunitive Response to Error and Frequency of Event Reporting. The existence of a nonpunitive culture appears to be only moderately associated with perceptions of event reporting. The strongest relationship with event reporting was with Feedback and Communication About Error, which highlights the importance of open communication about error and giving feedback about changes put into place based on event reports as potential means for increasing event reporting.
The one-item measure of the number of events staff reported in the past 12 months was disappointingly not related to any of the patient safety culture dimensions, perhaps due to the fact that our descriptive analysis discovered that 46% of staff had reported no events in the past year. For now, rather than using this as an outcome variable, perhaps it is best used as a descriptive measure to assess changes in staff event reporting over time until event reporting becomes more of a norm for staff in hospitals.
A strength of the survey is that it assesses a number of key cultural dimensions related to patient safety, focused at both the unit/department level, as well as hospital-wide. This multi-dimensional approach provides a level of specificity that makes it useful as a tool to guide patient safety improvement interventions. The results from the survey can be used to diagnose the current status of patient safety culture; raise staff awareness about patient safety; evaluate the impact of patient safety interventions and programs; trend culture change over time; conduct benchmarking with other hospitals; and fulfill regulatory directives and requirements [14
It is also important to keep in mind that a quantitative survey is only one method that can be used to assess patient safety culture. Qualitative approaches involving observation, focus groups and interviews can provide more in-depth analysis and understanding of underlying cultural values and deeper cultural assumptions to complement data obtained from quantitative culture surveys. Additional methodological approaches can also be used to identify patient safety vulnerabilities, such as medical record review; patient safety indicators [32
]; use of trigger tools to identify and quantify patient harm [33
]; use of data from event reporting systems; root cause analysis; failure mode and effects analysis (FMEA); and probabilistic risk assessment [34
Given widespread international interest in patient safety, the World Health Organization (WHO) is undertaking a multi-year High 5 s Project http://www.who.int/patientsafety/solutions/high5s/en/index.html
to achieve reductions in high risk patient safety problems. Hospitals in participating countries have implemented the AHRQ Hospital SOPS to assess baseline patient safety culture and will track culture change over time as the initiative progresses. In addition, the European Network for Patient Safety (EUNetPaS-- http://90plan.ovh.net/~extranetn/
) aims to establish an umbrella network of European Union Member States and stakeholders to encourage and enhance collaboration in the field of patient safety. One of the EUNetPaS key goals is to promote a culture of patient safety.
With the AHRQ Hospital SOPS translated into 18 languages and administered in over 30 countries, it is clear that there is a need for patient safety culture assessment tools around the world. A number of researchers that have administered the AHRQ Hospital SOPS in different countries have published psychometric results [35
]. Analyses conducted by Smits et al (2009) in the Netherlands found strong psychometric support for 11 dimensions, with considerable unit-level variation. It is hoped that this proliferation of the survey's use and testing will result in a greater understanding of patient safety culture internationally as well as shed light on how to conduct cross-cultural comparisons on the survey results.
The ultimate goal of patient safety efforts is to reduce the risk of health care associated injury or harm to patients. A limitation of this study is that we were unable to examine the relationship between patient safety culture survey scores and indicators of actual patient harm either at the unit or hospital levels. Evidence about the criterion-related validity of patient safety culture instruments is much needed to examine the nature of the relationship between patient safety culture and patient outcomes. While there is abundant theory, case studies, and descriptive research on culture and culture change, there is still very little criterion-related research that links culture to "hard," non-perceptual outcomes like patient harm or cost savings. These are the data that move boards-of-directors and administrators to allocate resources and take action and are critical to telling the story of how patient safety culture impacts the bottom line.
More research is also needed about how to change culture. Hospitals that plan to implement patient safety culture interventions should work together with health services researchers to design rigorous studies of their interventions. Such collaborative research can produce evidence of the efficacy of cultural interventions that can be shared among hospitals interested in applying proven methods to guide how to change their patient safety culture in areas that need improvement.