|Home | About | Journals | Submit | Contact Us | Français|
A patient safety learning system (sometimes called a critical incident reporting system) refers to structured reporting, collation, and analysis of critical incidents. To inform a provincial working group's recommendations for an Ontario Patient Safety Event Learning System, a systematic review was undertaken to determine design features that would optimize its adoption into the health care system and would inform implementation strategies.
The objective of this review was to address two research questions: (a) what are the barriers to and facilitators of successful adoption of a patient safety learning system reported by health professionals and (b) what design components maximize successful adoption and implementation? To answer the first question, we used a published systematic review. To answer the second question, we used scoping study methodology.
Common barriers reported in the literature by health care professionals included fear of blame, legal penalties, the perception that incident reporting does not improve patient safety, lack of organizational support, inadequate feedback, lack of knowledge about incident reporting systems, and lack of understanding about what constitutes an error. Common facilitators included a non-accusatory environment, the perception that incident reporting improves safety, clarification of the route of reporting and of how the system uses reports, enhanced feedback, role models (such as managers) using and promoting reporting, legislated protection of those who report, ability to report anonymously, education and training opportunities, and clear guidelines on what to report.
Components of a patient safety learning system that increased successful adoption and implementation were emphasis on a blame-free culture that encourages reporting and learning, clear guidelines on how and what to report, making sure the system is user-friendly, organizational development support for data analysis to generate meaningful learning outcomes, and multiple mechanisms to provide feedback through routes to reporters and the wider community (local meetings, email alerts, bulletins, paper contributions, etc.).
The design of a patient safety learning system can be optimized by an awareness of the barriers to and facilitators of successful adoption and implementation identified by health care professionals. Evaluation of the effectiveness of a patient safety learning system is needed to refine its design.
A patient safety learning system is defined in Ontario legislation as “any unintended event that occurs when a patient receives treatment in the hospital that (a) results in death or serious disability, injury, or harm to the patient and (b) does not result primarily from the patient's underlying medical condition or from a known risk inherent in providing treatment.”1
A patient safety learning system (also called a critical incident reporting system) refers to structured reporting, collation, and analysis of such incidents.2 Potential benefits of using this system are as follows:
As well as incidents that result in death, serious disability, injury, or harm, many experts believe that events that result in no harm or near misses should also be documented in a patient safety system. A “no harm” incident is a patient safety incident affecting a patient but causing no discernible harm, and a “near miss” incident does not reach the patient (replaces “close call”).3
However, many barriers to the adoption of patient safety learning systems are reported by health care professionals. Identifying these barriers and facilitators is important to guide development and implementation of a successful patient safety learning system. A systematic review on the effectiveness of patient safety learning systems did not find strong evidence that these systems performed better than other methods (such as medical chart review).4 However, the review concluded that these systems could be more effective if the criteria of incidents were explicit, they were led by clinical teams rather than centralized hospital departments, and they were embedded within organizations as part of a larger safety program.4
In Canada, Saskatchewan was the first province to require reporting of adverse events to the provincial Department of Health. Legislation required reports, investigation, and shared learning arising from critical incidents as of September 15, 2004.1 Since then, Manitoba and Quebec have passed legislation mandating reports of critical patient safety incidents. In 2008, Ontario amended Regulation 965 under the Public Hospitals Act.5 This amendment mandated disclosure of a critical incident to patients. Previous legislation did not require this transparency. While some provinces have enacted legislation for mandatory reporting of adverse health events, reporting of adverse events remains at the institutional level in many other provinces.
The objective of this review is to describe barriers, facilitators, characteristics and design of a patient safety learning system and the implementation considerations that will optimize adoption of the system into health care.
We completed a systematic review to identify relevant studies for each research question. For the first question, we will identify the barriers to and facilitators of adoption of a patient safety learning system. For the second research question, our goal was to capture a general overview of the components involved in maximizing a successful patient safety learning system but not examine any one component in depth.
We performed a literature search on November 27, 2015, using All Ovid MEDLINE, Embase, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Centre for Reviews and Dissemination (CRD) Health Technology Assessment Database, Cochrane Central Register of Controlled Trials, and National Health Service (NHS) Economic Evaluation Database for studies published from January 1, 2004, to November 27, 2015. We chose the literature search start date (2004) on the basis of the emerging popularity of patient safety learning systems at that time. Medical librarians developed search strategies using medical subject headings (MeSH) and key words. The final search strategy was peer-reviewed using the PRESS Checklist.6 See Appendix 1 for details, including all search terms. We hand-searched the reference lists of the included studies, along with health technology assessment websites and other sources, to identify additional relevant studies. No additional sources were found.
A single reviewer screened the abstracts and, for those studies meeting the eligibility criteria, we obtained full-text articles. We also examined reference lists for any relevant studies not identified through the search.
We extracted barriers and facilitators that influence the successful uptake of a patient safety learning system. Identification of a barrier, facilitator, and the components was done by one reviewer. A qualitative synthesis of the evidence was undertaken. A barrier was defined as anything that restrains or obstructs an individual from reporting and a facilitator was defined as anything that helps reporting by providing indirect or unobtrusive assistance, guidance, or supervision. Components were considered any factor identified in the research that was thought to be the reason for the success of the system. Components were categorized into 4 groups: organizational culture, data and data input, analysis, and feedback. Results are reported narratively.
The Assessment of Multiple Systematic Reviews (AMSTAR) measurement tool was used to assess the quality of any systematic reviews found from the literature search. Where available, we used the quality assessment of the primary studies done by the authors of the systematic review. The methodologic quality of the individual studies was assessed by various tools dependent on the study design. For descriptive studies the appropriateness of the research design, recruitment strategy and data collection, potential researcher bias, ethical considerations, data analysis, and reporting of study findings was reviewed. The Scottish Intercollegiate Guidelines Network (SIGN) 50 checklist was used for cohort and case-control studies. No quality assessment was done for scoping studies included to address the research question, as this step was not part of the methods.
In November 2015, a working group on patient safety learning systems was struck to recommend a design for a learning system concerning patient safety events in Ontario. Members of the panel included physicians with expertise in critical incidents, hospital administrators, and patient representatives. The panel members were consulted to help contextualize the information from this systematic review. However, the statements, conclusions, and views expressed in this report do not necessarily represent the views of working group members.
The database search yielded 1,839 citations published between January 1, 2005, and November 27, 2015. After removing duplicates, we reviewed titles and abstracts to identify potentially relevant articles. We obtained the full texts of these articles for further assessment. Fourteen studies (one systematic review, six literature reviews, three mixed-method studies, two pilot/implementation studies, one qualitative study, and one cross-sectional study) met the inclusion criteria for both research questions.
Figure 1 presents the flow diagram for the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).
To address the research question, “what are the barriers and facilitators of successful adoption of patient safety learning systems?” we used data from a 2015 systematic review published by Polisena and colleagues.8 This systematic review was chosen because it addressed the research question, was recent, and scored highly on AMSTAR (Appendix 2, Table A1). The objectives of the systematic review conducted by Polisena and colleagues were a) to explore factors that influence device-related incident recognition, reporting, and resolution and b) to investigate interventions or strategies that improve the recognition, reporting, and resolution of medical device–related incidents. However, the authors expanded the search to “incidents in a hospital” because most studies were concentrated on other health technology. Therefore, the systematic review conducted by Polisena and colleagues is generalizable to reports of critical incidents within hospitals.
Polisena and colleagues included 30 studies: nine studies published in the United States; five in the United Kingdom; four in Australia; three in Canada; two each in Italy and Korea; and one each in Turkey, China, Pakistan, France, and the Netherlands. Two studies examined incidents associated with medical equipment or devices, and one Canadian study investigated barriers to and facilitators of medication error reporting in four community hospitals. The remaining studies did not focus on incidents related to use of any specific health care technology. Of the selected studies the overall quality of evidence was moderate. Numerous studies described how participants were randomly selected; no studies reported any potential biases as a result of interaction between researchers and participants; and, although many studies described the statistical analyses where appropriate, many did not go into great detail about how study findings contributed to current practice or policy.
To address the research question “what components of a patient safety learning system maximize successful implementation?” we used 13 studies identified through the literature search we conducted. This research question focused on components of a patient safety learning system necessary to provide evidence for the working group to recommend a design for the Ontario patient safety learning system. Of the 13 studies, six studies were literature reviews, three studies were mixed-method studies, two were pilot/implementation studies, one was a qualitative study, and one was a cross-sectional study. No quality assessment was done, as this research question used scoping study methodology. Below are the characteristics of the 13 included studies (Table 1).
Identifying barriers to and facilitators of reporting is important to help design and implement patient safety learning systems. The most commonly cited barriers of reporting into a patient safety learning system included fear of blame, rejection of bureaucracy and managerial scrutiny, administrative sanctions, legal penalties, the perception that incident reporting does not improve patient safety, lack of organizational support, inadequate feedback, lack of knowledge about incident reporting systems, and lack of understanding about what constitutes an error.8 Other reported barriers include reporting forms that are too time-consuming to complete or incidents that were too trivial to report.8 Inaccessibility of the reporting system was also identified as a barrier. One study found that the level of harm, incident type, and profession (e.g., a US survey showed nurses were three times more likely to report no-harm events than physicians) influenced the rate of reporting.8
Common strategies to improve reporting rates that health care professionals consider effective included non-accusatory environment and enhancement of safety culture, clarification of the route of reporting and of how the system is used, enhanced feedback (e.g., acknowledgement of the report and description of the steps taken to analyze and address event), role models (such as managers) promoting and using reports, legislated protection of those who report, ability to report anonymously, introduction of a reward system, recruiting more staff for patient safety incident management, education and training opportunities, and clear guidelines on what to report.8 Other incentives identified by health care professionals were to obtain immediate help for the patient, to learn from mistakes, and to develop a system to minimize repetition of incidents.
Components of a patient safety learning system should address the common barriers and build on the facilitators that influence reporting. World Health Organization (WHO) guidelines state that learning systems are designed to foster continuous improvement in care delivery by identifying themes, reducing variation, facilitating the sharing of best practices, and stimulating system-wide improvements.21 For a patient safety learning system to be successful, learning the lessons depends upon four basic activities (organizational culture, data and data input, analysis, and feedback).16 For the purpose of this review, “data input” and “the data” were combined into one domain and “organizational culture” was added because of the emphasis on this component in the literature. These four domains will be used as a guide to classify the factors that maximize the successful implementation of patient safety learning systems.
A successful reporting system should be non-punitive, where health care professionals are comfortable reporting incidents and feel supported.12,17,18 To make staff aware of a new patient safety learning system, hospital management should define the purpose of the system before implementation, should define what must be reported, and should communicate these definitions. Explicitly stating the goals, mechanics, limitations, and protections of the system puts users more at ease.10 The hospital should provide training that builds knowledge about the system and how to use it and should provide continuing education. Training should be tailored to different types of health care professionals.12 The organization should prepare a “starter kit” for potential users and, if overlapping systems already exist in the hospital, decide how they will interact in the new system.
The hospital should address existing legal barriers to reporting and provide reporters with protection from disciplinary action.12 In terms of implementation, it is suggested to start small and gain some experience with pilot tests to ensure accuracy before rolling the system out for wider use. Focus group discussions also proposed different means to assure clinician buy-in, such as having chances to communicate with current system users.10 It is also recommended that integration with existing reporting structures, fitting the system in the workflow of practice, and promoting incident reporting by staff be targeted to maximize the success of a patient safety learning system.9,12
Last, mandatory reporting is generally used for accountability systems and voluntary reporting for learning systems.21 However, the literature suggests that a mandatory system would provide motivation to participate whereas voluntary reports would be of lower priority than other schedule demands.10
Studies reported several ways to make the reporting system easier to use. Interface specialists need to be included when designing the system to develop an intuitive and usable reporting form,10 as well as to reduce the length and difficulty of the reporting process. Reporting time should not exceed 5 minutes; 2 minutes is preferred. Health professionals consider reporting immediately following the event to be most beneficial, while others feel more comfortable reporting at their leisure.10 Hospitals need a trigger list so staff know what to report.2,13,17,19 The system should be user friendly and have a standardized process and classifications.13 The reporting form should be concise, clear, and focused on the story.17,19 Forms that have too many closed questions do not allow free expression of “what actually happened.” It is crucial that staff are given opportunities through open-ended questions to narrate their own version of events.11 Such data would reflect the true nature of the incident, would convey a better chronology of events, and would give a better feel for the many factors that link in the evolution of an incident.16
Data in the reporting form should include the following: who is involved in the incident (reporter, patient, attending physician, other staff involved), what happened (brief title, description of event, action taken, category, if anyone was harmed, delay in care, severity level, treatment), when the event took place (date/time occurred, date/time reported), where the event occurred (location where occurred/identified), how the event happened (method of identification, probability of reoccurrence, probability of detection), and why the event happened (apparent causes, human factors).20
Patient safety needs denominator data, meaning the database should ideally take into account the period over which reports were collected, the patient case-load and case-mix over that period, and (where possible) an indicator of staffing and other resources measured over time.2
Analysis should involve experts and be thorough yet timely.18 It is important that a standard method is adopted for analysis of the data to help staff perceive reporting as useful. A transparent analysis process is important for credibility of the system. A standard process should be developed for selecting and analyzing useful reports, and this process should be transparent.16 It is important not to overload the system with reports to the extent that effective analysis cannot be done. Two studies suggest establishing a task force to analyze reported data and generate strategies for improvement.12,19 This step requires someone able to analyze human factors and organizational issues to generate meaningful learning outcomes.17 In another study, health care professionals being interviewed stated that an intermediary in the system was important to carry out the analysis; this might be “… somebody [who would] evaluate the situation besides yourself” (said clinical assistants) or an individual trained to conduct initial reviews of the data for comprehensiveness (said physicians).10
Whoever conducts the analysis, it is essential that a dedicated trained individual or task force has protected time for timely investigation of serious events. The analysis phase is probably the lengthiest and will require experts to link various components of the system to the front-end failures that lead to an incident. Outside of analyzing specific incidents, it is useful to identify clusters of similar events or important cases where front-line staff can learn.17 Analysis of event reports is important to maximize the success of a patient safety learning system.9
The use of incident reports to improve patient safety is important.9 Feedback about the incident should be provided to those reporting and to those who are disseminating and implementing recommendations.18 The hospital should provide regular feedback on recent errors, associated hazards, and hazard control strategies. Ideally, a learning and dissemination strategy should be developed at the same time as a reporting system to ensure the objectives above are addressed. Users of the information at the local level should be involved in designing the feedback products, and feedback should also always encourage continued reporting.12,22
Dissemination of lessons learned is crucial in a patient safety learning system. Feedback should be through multiple sources from high-level managerial staff to front-line workers.16 The organization should build local or regional networks to expand learning systems and provide feedback and continuous education across hospitals. If the hospital wants to write recommendations for preventing events, involve important experts on the relevant topic. If the event involves a medical device, contact the manufacturer and try to publish recommendations jointly.19 Feedback can come in four general modes: information to the reporter of the incident, information to all front-line personnel, information to the reporter and wider reporting community, and action within local work systems.14 Feedback to reporters or the wider community can be provided through all possible means (local meetings, email alerts, bulletins, paper contributions, etc.)15,17,19
If a task force judges that the submitted incident warranted an analysis, invite the task force to edit any published recommendations that come from the incident.19 The goal of feedback must be to learn from mistakes and to ensure that systems are improved for better patient safety in the future. For the patient safety learning system to be effective, lessons learned from review of incidents need to be fed back to the individuals and department concerned in a timely balanced manner.2 Visible action should be taken to mitigate important risks identified in reports.17
Various barriers and facilitators reported in the literature will affect implementation and successful adoption of a patient incident reporting system. Knowledge of these barriers and facilitators can influence design of system components to optimize uptake and diffusion of information across the health care system. Not one component of a successful patient safety learning system was more pivotal than another. All components have to work cohesively to optimize use of the system and ultimately increase patient safety. Effectiveness of a reporting system should be evaluated to further refine design components.
Common barriers reported include fear of blame, the perception that incident reporting does not improve patient safety, lack of organizational support, inadequate feedback, lack of knowledge about incident reporting systems, and lack of knowledge about what constitutes an error. In other cases the reporting form was too time-consuming to complete or the incident was too trivial to report.
Facilitators of reporting included a non-accusatory environment and enhancement of safety culture, clarification of the route of reporting and of how the system is used, enhanced feedback (e.g., acknowledgement of the report and description of the steps taken to analyze and address event), role models (such as managers) promoting and using reports, legislated protection of those who report, ability to report anonymously, education and training opportunities, and clear guidelines on what to report. These factors can improve reporting rates. Findings from other studies report similar barriers and facilitators.10,12,23–32
Successful patient safety learning systems should target four areas: organizational culture, data input and the data, analysis, and feedback. Common components were to emphasize a blame-free culture that encourages reporting and learning. Second, reporting forms should be user-friendly, and those reporting into the learning system should have clear guidelines on how and what to report. Third, the analysis requires someone able to analyze human factors and organizational issues to generate meaningful learning outcomes. The analysis should be standardized, and the individual or task force analyzing the incident should have designated time to do so. Lastly, dissemination of lessons learned is crucial in a patient safety learning system. Feedback to reporters can be provided through all possible means (local meetings, email alerts, bulletins, paper contributions, etc.).
In order to have a successful patient safety learning system, the barriers and facilitators identified by health care professionals should be addressed and used to guide development and maximize effectiveness.
The medical editor was Elizabeth Betsch. Others involved in the development and production of this report were Claude Soulodre, Kellee Kaulback, Andrée Mitchell, Nancy Sikich, Michelle Rossi, and Irfan Dhalla.
Search date: Nov 27, 2015
Librarian: Corinne Holubowich
Databases searched: All Ovid MEDLINE, Embase, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, CRD Health Technology Assessment Database, Cochrane Central Register of Controlled Trials, and NHS Economic Evaluation Database
Database: EBM Reviews - Cochrane Central Register of Controlled Trials <October 2015>, EBM Reviews - Cochrane Database of Systematic Reviews <2005 to November 2015>, EBM Reviews - Database of Abstracts of Reviews of Effects <2nd Quarter 2015>, EBM Reviews - Health Technology Assessment <4th Quarter 2015>, EBM Reviews - NHS Economic Evaluation Database <2nd Quarter 2015>, Embase <1980 to 2015 Week 47>, All Ovid MEDLINE(R) <1946 to Present>
|1||((near miss or unexpected harm* or ((critical or patient safety or harmful or clinical) adj incident*)) adj4 (monitor* or report* or investigat* or system*)).tw.||1490|
|2||(incident reporting system* or error reporting system* or ((safety or incident or error) adj2 learning system*)).tw.||1142|
|4||exp *Risk Management/||55392|
|5||exp *Medical Errors/||49876|
|9||7 and 8||7490|
|14||9 and 13||820|
|15||3 or 14||2950|
|16||exp Health Plan Implementation/||87565|
|17||exp Program Evaluation/||73242|
|18||Health Services Administration/||134586|
|19||exp “Delivery of Health Care”/||3116936|
|20||exp Health Services/||5701042|
|22||exp Preventive Health Services/||517960|
|23||exp Decision Making/||376919|
|24||exp “diffusion of innovation”/||426709|
|26||exp Quality Assurance, Health Care/||2516993|
|29||(implementation* or quality improvement*).tw.||381697|
|31||15 and 30||2372|
|32||limit 31 to (english language and yr=“2004 -Current”) [Limit not valid in CDSR,DARE; records were retained]||1826|
|33||32 use pmoz,cctr,coch,dare,clhta,cleed||674|
|34||((near miss or unexpected harm* or ((critical or patient safety or harmful or clinical) adj incident*)) adj4 (monitor* or report* or investigat* or system*)).tw.||1490|
|35||(incident reporting system* or error reporting system* or ((safety or incident or error) adj2 learning system*)).tw.||1142|
|37||exp *risk management/||55392|
|38||exp *medical error/||49725|
|42||40 and 41||7482|
|49||44 and 48||1246|
|50||36 or 49||3292|
|51||health care planning/||83114|
|52||exp program evaluation/||73242|
|54||health care delivery/||212657|
|56||preventive health service/||35439|
|57||exp decision making/||376919|
|59||exp health care quality/||8038805|
|61||(implementation* or quality improvement*).tw.||381697|
|63||50 and 62||2395|
|64||limit 63 to (english language and yr=“2004 -Current”) [Limit not valid in CDSR,DARE; records were retained]||1869|
|65||64 use emez||1165|
|66||33 or 65||1839|
|67||66 use pmoz||664|
|68||66 use emez||1165|
|69||66 use cctr||5|
|70||66 use coch||3|
|71||66 use dare||0|
|72||66 use clhta||2|
|73||66 use cleed||0|
|74||remove duplicates from 66||1281|
|Author, Year||AMSTAR Scorea||(1) Provided Study Design||(2) Duplicate Study Selection||(3) Broad Literature Search||(4) Considered Status of Publication||(5) Listed Excluded Studies||(6) Provided Characteristics of Studies||(7) Assessed Scientific Quality||(8) Considered Quality in Report||(9) Methods to Combine Appropriate||(10) Assessed Publication Bias||(11) Stated Conflict of Interest|
|Polisena et al, 20158||8||b|
When a patient is harmed by receiving health care, health care professionals should report and learn from the event to avoid repeating it. We aimed to find what factors stop health care professionals from reporting problems and from making changes that will make patients safer in the future. We also looked at what factors make reporting and changes easier for health care professionals.
Several barriers blocked people from reporting on problems or from making changes to prevent problems: fear that they would be blamed for the problem, fear that they would be punished for breaking laws, fear that reporting problems doesn't improve patients’ safety, lack of support in the organization, lack of feedback, lack of knowledge about reporting systems, and no clear guidelines on what errors should be reported. People were more willing to report and learn from systems that increased patient safety if others did not try to lay blame for the problems, if reporting and learning were encouraged, if they were told clearly what and how to report, if the organization supported data analysis to come up with useful lessons, and if feedback was offered in several ways (like local meetings, email messages, and bulletins).
A learning system can best improve patient safety if we reduce barriers and make it easy to use.
Health Quality Ontario is the provincial advisor on the quality of health care. We are motivated by a single-minded purpose: Better health for all Ontarians.
We are a scientifically rigorous group with diverse areas of expertise. We strive for complete objectivity, and look at things from a vantage point that allows us to see the forest and the trees. We work in partnership with health care providers and organizations across the system, and engage with patients themselves, to help initiate substantial and sustainable change to the province's complex health system.
We define the meaning of quality as it pertains to health care, and provide strategic advice so all the parts of the system can improve. We also analyze virtually all aspects of Ontario's health care. This includes looking at the overall health of Ontarians, how well different areas of the system are working together, and most importantly, patient experience. We then produce comprehensive, objective reports based on data, facts and the voice of patients, caregivers and those who work each day in the health system. As well, we make recommendations on how to improve care using the best evidence. Finally, we support large scale quality improvements by working with our partners to facilitate ways for health care providers to learn from each other and share innovative approaches.
We recognize that, as a system, we have much to be proud of, but also that it often falls short of being the best it can be. Plus certain vulnerable segments of the population are not receiving acceptable levels of attention. Our intent at Health Quality Ontario is to continuously improve the quality of health care in this province regardless of who you are or where you live. We are driven by the desire to make the system better, and by the inarguable fact that better has no limit.