|Home | About | Journals | Submit | Contact Us | Français|
Many thousands of patients die every year in the United States as a result of serious and largely preventable safety events or medical errors. Safety events are common in hospitalized children. We conducted a quality improvement initiative to implement cultural and system changes with the goal of reducing serious safety events (SSEs) by 80% within 4 years at our large, urban pediatric hospital.
A multidisciplinary SSE reduction team reviewed the safety literature, examined recent SSEs, interviewed internal leaders, and visited other leading organizations. Senior hospital leaders provided oversight, monitored progress, and helped to overcome barriers. Interventions focused on: (1) error prevention; (2) restructuring patient safety governance; (3) a new root cause analysis process and a common cause database; (4) a highly visible lessons learned program; and (5) specific tactical interventions for high-risk areas. Our outcome measures were the rate of SSEs and the change in patient safety culture.
SSEs per 10000 adjusted patient-days decreased from a mean of 0.9 at baseline to 0.3 (P < .0001). The days between SSEs increased from a mean of 19.4 at baseline to 55.2 (P < .0001). After a worsening of patient safety culture outcomes in the first year of intervention, significant improvements were observed between 2007 and 2009.
Our multifaceted approach was associated with a significant and sustained reduction of SSEs and improvements in patient safety culture. Multisite studies are needed to better understand contextual factors and the significance of specific interventions.
A decade ago, 2 groundbreaking reports from the Institute of Medicine revealed that defective systems, rather than calloused or careless providers, were the cause of patient safety errors.1,2 Since then, many hospitals have implemented or revised systems to improve patient safety.3–5 In addition, many specialty societies and nongovernmental organizations have focused on improving patient safety.3,6–12 Although there have been some impressive improvements,4 patients continue to suffer from serious and preventable safety events.3,13 Recent multicenter data illustrate that patient harm remains common, and little, if at all, changed over a period from 2002 to 2007.14 Safety events in hospitalized children are common, and they are associated with increased length of stay, in-hospital mortality, and total charges.15–19 Up to 60% of such events may be preventable.17
Cincinnati Children’s Hospital Medical Center (CCHMC) has maintained a significant focus on improving patient safety for many years.20 Three strategic plans since 2001 confirmed quality as our core business strategy and committed us to transforming health care delivery and established organizational readiness via leadership focus, transparency, data infrastructure to support improvement, and continuously increasing improvement capability. Specific improvement initiatives led to measurable reductions in adverse events.22–23 However, during this same time, the number of serious safety events (SSEs) at CCHMC, defined as deviation from generally accepted performance standards resulting in severe or permanent harm,24 remained stable. Therefore, in 2006, leadership committed to an improvement effort focused on cultural and system changes, with the aim of reducing SSEs to 0.2 per 10000 adjusted patient-days by June 30, 2010. We report here on the details and results of that initiative.
CCHMC is a large, urban pediatric academic medical center. In fiscal year 2010, CCHMC had >32000 inpatient admissions and 125000 emergency department (ED) visits and performed 31000 surgical procedures. The hospital serves a diverse local, national, and international population; ~43% of patients have Medicaid insurance.
In 2006, a SSE reduction team was formed, consisting of quality and safety leaders, risk management representatives, project managers, a senior decision analyst, and a consultant human factors expert. The faculty leader devoted 50% of his time. The other team members amounted to 4.75 full-time–equivalent employees. Senior hospital leaders provided oversight, monitored progress toward goals, and helped to overcome barriers.
The team reviewed the safety literature (with particular attention focused on reliability in health care),4,25,26 examined the 35 most-recent SSEs at CCHMC (applying methods from Healthcare Performance Improvement24 to analyze causes of the events), and created a common cause database. The team then reviewed CCHMC results of the Hospital Survey on Patient Safety Culture (HSPSC) developed by the Agency for Healthcare Research and Quality (AHRQ).27,28 Finally, the team hired a consultant from Healthcare Performance Improvement who conducted interviews with >100 CCHMC leaders, physicians, and clinical staff. Team members also participated with other health care organizations in AHRQ’s High Reliability Organization Learning Network,7 which included visits to organizations focused on reducing SSEs.29,30 Evidence from the available literature and expert opinion were then used to develop a key driver diagram (Fig 1).
The primary outcome measure was the rate of SSEs per 10000 adjusted patient-days from January 2003 through June 2011. The secondary outcome was patient safety culture over 5 years (2005–2009).
Serious or permanent harm was defined by using a framework developed by Healthcare Performance Improvement24 (Table 2). Harm could result from errors of commission or omission. A review panel consisting of the chief of staff, patient safety officer, and 2 attorneys from risk management reviewed all potential SSEs identified by an anonymous, web-based patient safety reporting system, supplemented by a trigger tool system,19 telephone call alerts, and direct communications to the safety officer. The panel met biweekly and used a standardized evaluation to determine performance deviation and assessment of harm. Emphasis was on consistency and precedent in determination of whether an SSE had occurred. Clinicians with subspecialty knowledge expertise were consulted in most cases.
Each year since 2005, all patient care providers at CCHMC have been asked to anonymously complete the web-based AHRQ HSPSC.27,28 The survey asks respondents to share their experiences and perceptions over the previous 12 months regarding 7 unit-level and 3 hospital-level dimensions of safety culture and 4 outcome variables. Demographic data were not collected to protect respondent anonymity.
The institutional review board determined that this was a local systems improvement project and did not require their review and approval.
Statistical process control charts were used to examine the impact of interventions on the number of SSEs.39,40 The number of events per 10000 adjusted patient-days (including inpatient admissions, ED visits, and short stays) was tracked on a U (unit) chart based on count data with a Poisson distribution. The days between SSEs were tracked on a T (time between) chart, used to track rare events and with the expectation that the number of days between events would grow after intervention.41 The upper and lower control limits on the charts establish the margins within which the measurement will be found ~99% of the time. Wilcoxon rank-sum test analyses of the change in the rate of events were also conducted.
For survey data, as recommended by AHRQ,28 the percentage of positive responses was calculated (strongly agree/agree or always/most of the time) across items in each HSPSC dimension. Responses to negatively worded questions were reversed before calculating the percentage of positive responses. For patient safety grade, a positive response was denoted by a response of excellent or very good. For the number of events reported, a positive response was assigned if a respondent submitted >1 report in the previous year. Finally, the mean percentages for each dimension and overall respondents were calculated. Scores for each dimension could range from 0% to 100%, with higher scores indicating a more positive response. The mean percentage over each year of the survey was adjusted to account for individual differences in how long the respondent worked at CCHMC, in the current hospital area or unit, and in the current specialty or profession; the number of weeks per year usually worked; and whether the respondent had direct or indirect patient contact.
All data analyses were performed by using SAS version 9.2 (SAS Institute, Inc, Cary, NC). Adjusted mean percentages and 95% confidence intervals were calculated by using the LSMEANS option in PROC GLM. Statistical differences across years were obtained by using pair-wise multiple comparison tests with Bonferroni corrections.
After intervention implementation, the number of SSEs per 10000 adjusted patient-days (Fig 2) significantly decreased from a mean of 0.9 to 0.3 (P < .0001). Days between SSEs (Fig 3) increased from a mean of 19.4 to 55.2 (P < .0001). During this same time, patient volume increased; monthly average adjusted patient-days were 13686 during the baseline period and 17521 during the study period. The reduction in SSEs occurred gradually but then seemed to stabilize at 0.3 SSE per 10000 adjusted patient-days. We believe this reflects a combined effect from system improvements and cultural change.
This combination of interventions, by nature, required time and relentless clarity of vision by the organization, with dedicated resources and focus on key clinical areas. As more employees were trained, we discussed methods to observe and measure performance of expected safety behaviors. We decided to focus, instead, on using the perceptions and qualitative feedback of safety coaches to reinforce behaviors and estimate the penetration of behaviors at the microsystem level. Senior leaders made transparent sharing of information regarding SSEs the expected norm, modeled the new culture through their actions, and directed attention to key clinical areas and processes by continuously focusing on results. The overall measure reflects performance across the organization, but the interventions required changes in dozens of clinical microsystems. Individual clinical areas adopted changes at varying paces, so persistence over time was vital to achieving the overall organizational goal. Specific tactical interventions were developed and implemented where they were needed, and results were monitored continuously at the microsystem level.31 Together, these interventions helped to focus leaders, physicians, and staff attention on patient safety and potential risks each day.
Response rates generally increased over time, especially for nurses, who received encouragement and incentives (eg, pizza lunches) to complete the survey. Between 2005 and 2007, there was a general decreasing trend in the mean percentage of respondents giving a positive response across several patient safety culture outcomes and dimensions (Table 3). However, significant improvements in positive responses in several outcomes and dimensions were seen between 2005 and 2009 and between 2007 and 2009.
After implementation of cultural and system changes to improve patient safety, we observed significant reductions in the number of SSEs and increases in the days between SSEs. Although we did not reach our goal of 0.2 events, both statistical process control and Wilcoxon rank-sum analyses confirm these improvements to be statistically significant. More importantly, they are unquestionably clinically significant, equating to 62 fewer SSEs over the last 5 years. During the initial phase of the interventions, results from the safety culture survey worsened. However, as the initiative progressed, there was improvement.
Our experience of significant reduction in SSEs is the first that we are aware of in the published literature. Although we do not have data on which drivers or interventions most affected this change, our bundle of an error prevention system, restructured safety governance, a cause analysis structure, lessons learned program, and specific tactical interventions was strongly associated with fewer SSEs. We believe this set of interventions, designed to reach every employee, is likely to achieve similar patient safety advances in other health systems if adjusted to the local context.
A more positive safety culture has been associated with fewer adverse events in hospitals.42 Many hospitals have measured and reported their patient safety culture; however, there are few reports of multiple surveys to determine the change in patient safety culture over time. Our leadership was acutely aware of the initial decrease in some patient safety culture outcomes in the second and third surveys. Such results have been reported previously and may be due to the increased focus on patient safety and error prevention and the perception that change is not happening fast enough.43–46 In addition, change takes time. Our leadership was confident that our interventions would eventually result in an improved safety culture.
Our overall response rates were calculated by using total employees as the denominator, rather than number of clinicians. Nonetheless, the improvement in the overall patient safety grade from 2007 (1 year after the interventions began) to 2009 was statistically significant. Although our study design did not allow us to attribute causation, our experience with this and other quality improvement initiatives leads us to believe that our interventions drove the culture change that was a prerequisite to safety outcomes improvement. In addition, our experience seems to affirm that process improvement initiatives alone are not sufficient to drive necessary culture change.
The major theoretical model for our approach was based on 5 key concepts guiding high reliability organizations: sensitivity to operations, reluctance to simplify, preoccupation with failure, deference to expertise, and resilience.4,5 These organizations strive to create a culture and processes that drastically reduce system failures and effectively respond when failures do occur.7 Our systematic, whole-system approach to reducing SSEs included a transparent, consistent method for identifying safety events; a standardized taxonomy for classifying failures; and a strict method to determine root causes, drive development of specific interventions, and focus organizational attention.
Some limitations of our study are that the interventions occurred at a single site, we do not have data from a control hospital, and multiple interventions were made simultaneously, resulting in what, in describing the etiology of disease, has been called a web of causation.31,47 We believe the simultaneous, multifaceted approach was crucial in changing behavior and culture and that no single change was responsible for the results obtained. In addition, this work was conducted over several years at a large pediatric medical center with a long history of improving care20 and a robust infrastructure for tracking outcomes and harm. However, we believe these methods can be generalized to other health care organizations if adapted to address contextual factors.48,49 Multisite studies and a detailed analysis of the percent penetration of the interventions will allow further improvement. For example, we are currently in a collaborative with all 8 children’s hospitals in Ohio to build improvement capability, reduce SSEs, and improve patient safety culture.
Improving patient safety is an ongoing challenge. We continue to focus on changing behavior and understand that this will require years of persistence. We believe additional factors and experience from outside health care will be important to further improvement. We have begun a new, organization-wide focus on situation awareness (ie, knowing what is going on around you and what is likely to happen next) to eliminate unrecognized clinical deterioration and delayed diagnosis as a cause of serious harm.7 We are also increasing emphasis on human factors engineering; that is, using knowledge about human behavior in system design and redesign.7,50
Our multifaceted approach, focusing on culture and system changes, was associated with a significant and sustained reduction of SSEs. This approach may be applicable to other sites. Multisite studies are warranted to better understand contextual factors and the significance of individual interventions.
All of the listed authors are responsible for the reported research. Dr Muething, Ms Goodfriend, Mr Anderson, and Dr Kotagal contributed to the conception and design of the project; Dr Muething, Dr Goudie, Ms Goodfriend, and Ms Bracke contributed to the acquisition of the data; Dr Muething, Dr Goudie, Ms Schoettker, and Ms Bracke contributed to the analysis and interpretation of the data; Dr Muething, Dr Goudie, and Ms Schoettker drafted the manuscript; and Dr Muething, Dr Goudie, Ms Schoettker, Dr Donnelly, Ms Goodfriend, Ms Bracke, Dr Brady, Dr Wheeler, Mr Anderson, and Dr Kotagal contributed to the critical revision of the manuscript for important intellectual content. All authors approved the manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Dr Goudie received funding through a grant from the Agency for Healthcare Research and Quality (grant U18HS016957) for his work on this project. The funding source had no involvement in the study design; collection, analysis, or interpretation of the data; writing of the report; or decision to submit this article for publication. Funded by the National Institutes of Health (NIH).