PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
 
BMJ Qual Saf. May 2012; 21(5): 369–380.
Published online Mar 15, 2012. doi:  10.1136/bmjqs-2011-000443
PMCID: PMC3332004

Development of an evidence-based framework of factors contributing to patient safety incidents in hospital settings: a systematic review

Abstract

Objective

The aim of this systematic review was to develop a ‘contributory factors framework’ from a synthesis of empirical work which summarises factors contributing to patient safety incidents in hospital settings.

Design

A mixed-methods systematic review of the literature was conducted.

Data sources

Electronic databases (Medline, PsycInfo, ISI Web of knowledge, CINAHL and EMBASE), article reference lists, patient safety websites, registered study databases and author contacts.

Eligibility criteria

Studies were included that reported data from primary research in secondary care aiming to identify the contributory factors to error or threats to patient safety.

Results

1502 potential articles were identified. 95 papers (representing 83 studies) which met the inclusion criteria were included, and 1676 contributory factors extracted. Initial coding of contributory factors by two independent reviewers resulted in 20 domains (eg, team factors, supervision and leadership). Each contributory factor was then coded by two reviewers to one of these 20 domains. The majority of studies identified active failures (errors and violations) as factors contributing to patient safety incidents. Individual factors, communication, and equipment and supplies were the other most frequently reported factors within the existing evidence base.

Conclusions

This review has culminated in an empirically based framework of the factors contributing to patient safety incidents. This framework has the potential to be applied across hospital settings to improve the identification and prevention of factors that cause harm to patients.

Keywords: Patient safety, medication error, medical error, compliance, human factors, attitudes, implementation science, safety culture, qualitative research, risk management, incident reporting, mortality

Introduction

Since the early 1990s high-risk organisations have adopted a systems approach to safety management.1 2 This approach recognises that the immediate causes of patient safety incidents are errors made by people at the frontline of operations (eg, in the case of medication administration, this is most likely to be a nurse). However, the importance of a systems approach is that it recognises that the organisations within which people work have inherent weaknesses (latent failures) that can arise from decisions made at more senior levels (eg, plans agreed, buildings designed, staffing levels approved, equipment procured) and those external to the organisation (eg, policies imposed, targets set, funding decisions, education provision) and that these failures manifest themselves in local working conditions that promote errors. Thus, a focus on individual responsibility for errors is likely to be ineffective as an incident reduction strategy. Based on this approach it can be argued that there are two main strategies to reduce medical error: reactive and proactive. The first relies on learning from (reacting to) previous incidents to minimise error in the future while the second is concerned with prospectively identifying the latent failures within organisations that represent the preconditions for errors, and addressing these before a serious event occurs. Incident reporting systems, root cause analysis of serious incidents, and case note review are all tools that have the potential to provide data about the prevalence and/or causes of medical errors. However, there is growing frustration with incident reporting systems with low rates of reporting, poorly designed reporting tools and inadequate feedback all being blamed for providing data that have little value in improving safety.3 4 Moreover, learning across all of these tools is predicated on the collection of data about the factors contributing to error.5 6 To date there is no evidence-based and standardised list of contributory factors that can be used as a basis for understanding causation. Without this, reactive systems are unlikely to provide the answers we are looking for.

In other industries such as nuclear power and transport, measurement tools have been developed to assess the extent to which organisational factors (eg, supervision, planning, communication, training, maintenance) represent a failure in the system.1 7 These tools do not rely on the retrospective analysis of adverse incidents but instead they allow the proactive monitoring of an organisation's safety. However, before such tools can be developed it is necessary to know what represents a latent failure within that particular industry. This systems approach to patient safety is well established in healthcare since the publication of ‘to err is human’ by the American Institute of Medicine8 and subsequent policy documents in the UK,9 10 and a number of frameworks for studying latent failures have been proposed (eg, Eindhoven classification,11 WHO patient safety classification,12 the London Protocol,13 the Veterans Affairs Root Cause Analysis System,14 the Australian Incident Monitoring System (AIMS)5). However, these frameworks are limited by the lack of an empirical basis and a reliance on classifications from non-healthcare settings15 16 that are very different to the structure and nature of healthcare.

The growing emphasis on systems thinking over the past 20 years in healthcare17 has meant that there is now a significant body of evidence in the scientific literature (eg, retrospective interview studies, real-time observational studies and aggregated data from incident reporting studies) that can be used as an empirical basis for generating a classification of the contributory factors that impact on healthcare in hospitals. Such a classification could serve to promote more effective organisational learning through the redesign of incident reporting systems and more effective root cause analysis of healthcare incidents. Such a classification system could also inform the development of intervention strategies to improve safety defences or directly address systems failures,18–22 and to guide the measurement tools used to evaluate policy and service level interventions.23

Thus, the aim of this literature review was to produce a framework of contributory factors that contribute to patient safety incidents within hospital settings. As such, it represents the first attempt to summarise the empirical evidence in this area and to use this evidence to develop a clearly defined and hierarchically ordered framework which describes contributory factors from proximal (sharp end) to distal (latent).

A secondary aim was to identify contributory factors that feature most strongly in the literature and which might therefore be appropriate targets for interventions designed to improve patient safety. Finally, we sought to assess the extent to which the contributory factors that were identified most frequently varied as a function of method of elicitation, hospital setting, and whether or not a human factors expert was involved in their identification.

Methods

Data sources and searches

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed in conducting this systematic review. A variety of strategies were used to search the literature to 20 November 2010. Clear identification of studies that identified the contributory factors of active failures was hampered by the lack of consistent terminology used across studies.

First, search terms were developed and electronic database searching was performed across the following databases: Medline, PsycInfo, ISI Web of knowledge, CINAHL and EMBASE. Second, the reference lists of all downloaded articles were manually searched to identify possible relevant papers. Third, a number of patient safety organisation websites were searched to identify other possible published or unpublished reports. Fourth, registered study databases were searched using the terms ‘patient safety’ to identify any ongoing or finished projects relevant to the current review that may have provided relevant material. A summary of these search strategies can be found in online appendix 1. Finally, key patient safety authors were contacted and asked to provide details of any relevant published or unpublished reports. This search strategy identified a total of 1502 potential articles. Of these, 95 papers (representing 83 studies) met the inclusion criteria and were included in the review (see figure 1). All article titles and abstracts were reviewed for inclusion (by RM). A random sample of 10% of the titles and abstracts were double coded with respect to inclusion or exclusion (by RS). The κ value of 0.73 indicated an acceptable level of agreement. If there was disagreement about inclusion or exclusion, the full text article was obtained and reviewed (by RM and RS) and agreement reached.

Figure 1
Flow chart of search strategy and included studies.

Study selection

Studies were included in this review if they reported data from:

  • (a) Secondary care or hospital environments.
  • (b) Primary research which either specifically aimed to identify the contributory factors (often referred to as ‘causes’ within studies) of active failures or threats to patient safety, or reported a clear framework for the categorisation of contributory factors of errors or threats to patient safety in the results section.

Studies were excluded if they reported data from:

  • (c) Active failures as causes of errors or threats to patient safety rather than underlying latent domains (eg, only specific human ‘errors’ causing failure of a barcode checking system24).
  • (d) Contributory factors of behaviours or processes that were not active failures (eg, factors affecting the likelihood of staff to report serious medication errors in hospitals 25).
  • (e) Case studies reporting contributory factors of a specific adverse event (eg, Chassin and Becher26).
  • (f) Studies that applied proactive risk assessment methods to identify potential failures (eg, failure mode and effects analysis) as these papers focused on exploring potential problems of specific elements of a healthcare system or process.

Data extraction and quality assessment

The study characteristics of 83 datasets were coded. All included articles were blind double coded (by RM and RS) and data were extracted and uploaded onto a Microsoft Access database. Kappas are reported only for dichotomous variables. Articles were coded according to the following characteristics: country of origin; description of setting; study method; study sample; theoretical frameworks informing the research (following the quality coding framework of Sirriyeh et al,27 studies were coded as: explicit use of theory—ie, explicit statement of theoretical framework applied to research; specific use of theory—ie, reference to specific theoretical basis; broad use of theory—ie, reference to broad theoretical basis; or none at all—ie, no theory mentioned); whether identification of contributory factors was a primary or secondary aim of the study (κ=0.66); whether contributory factors were identified by a human factors expert (κ=0.79); and finally, whether patients or staff reported the raw data used to identify contributory factors (κ=1—perfect agreement). Studies varied in the extent to which they used primary data collection methods to elicit contributory factors or whether they used a predefined set of contributory factors, therefore the following additional information was gathered to glean more details about the elicitation of contributory factors: whether the contributory factor list was fully developed before empirical data were collected (yes or no, κ=0.74); the method for eliciting contributory factors (if different from the overall study method); and any further details about the sample used to elicit contributory factors if different or a subset of the overall study sample. Disagreements were discussed and resolved. As we were interested in how contributory factors were identified, regardless of whether this was the primary aim of the study, we did not engage in any further ‘quality assessment’ coding, as often, very little detail about how contributory factors were identified was reported. All included papers and extracted data can be found in online appendix tables 1 and 2.

All contributory factors reported within the papers were transcribed verbatim onto a Microsoft Excel spreadsheet.

Data synthesis analysis

To develop the contributory factor framework, two of the authors (RL, a human factors expert and RM, a behavioural scientist) first independently grouped all the transcribed verbatim contributory factor items into categories according to their underlying semantic meaning (eg, equipment not working, equipment failure, equipment malfunction would all be grouped as equipment failure). Items could be categorised into more than one category. Second, each author further grouped these categories into their higher-order domains (eg, equipment failure was grouped with equipment unavailability and insufficient supplies to become ‘equipment and supplies)’. At this stage the authors did not explicitly distinguish between latent conditions and local working conditions. Next the two authors met to discuss and agree the number of each of the higher-order domains, and to label and define them (eg, equipment and supplies was defined as ‘the availability and functioning of equipment and supplies’). A decision was made to include all factors contributing to patient safety incidents in this framework—both the proximal factors (eg, active failures) and those more distal or external to the organisation (eg, design of equipment and supplies and external policy context). This process resulted in a framework of 20 domains and a definition for each (see figure 2). Finally, the same two authors applied the framework, again independently, to the raw data so as to classify each of the contributory factors based on the framework and to assess agreement. At first, 10% of the factors were coded and at this stage agreement was 55%. Following clarification and modification of definitions (eg, ‘human factors design of equipment and supplies’ became ‘design of equipment and supplies’), the remaining 90% of the contributory factors were coded. Agreement at this second stage was 90%. Disagreements were discussed and resolved through consensus.

Figure 2
The Yorkshire contributory factors framework.

To ensure that the framework had relevance and meaning beyond the two authors who developed the initial framework of domains, 10% of the datasets (n=9) and their respective contributory factors were extracted from the database and sent to two academic health professionals (IW, a general practitioner and JW, a hospital physician). Both were provided with instructions, definitions of each of the domains and were asked to code each of the contributory factors using the framework. They were asked to include ‘can't code’ when they were uncertain of the correct response. Initial agreement between the first two authors and each academic health professional was 62.5% (RL and RM with IW) and 85% (RL and RM with JW). After discussion with the first independent reviewer (IW) and further minor modification of the definitions of domains, agreement rose to 80.1%. Given that agreement with the second reviewer was initially high (85%) further discussion with this reviewer was not deemed necessary.

As noted in the introduction, contributory factors can vary according to their level of proximity to the ‘active failure’ being accorded to the individual, local working conditions (eg, management of staff and staffing levels) or more latent conditions (eg, design of equipment and supplies). The contributory factors elicited in this review also reflected these distinctions. In a final step, an expert panel of clinicians (n=5), researchers (n=8), managers (n=2) and lay people (n=2) were provided with a list of all contributory factors and definitions and asked to identify the extent to which each factor was removed in time and space from patient safety incidents on a five-point scale from 1 (very close in time and space) to 5 (very distant in time and space). Contributory factors scoring 4 or 5 were deemed to be more ‘latent’ organisational factors, while those scoring 2 or 3 were deemed to be more related to local working conditions or situational factors. This allowed us to ground the taxonomy in a hierarchical framework, which we have described in figure 2.

Results

Ninety-five studies fulfilled the inclusion criteria, reporting data from 83 independent datasets.28–122 A total of 1676 contributory factors were extracted. Studies reported a median of 15 contributory factors each (IQR 8–27). The lowest number of contributory factors extracted from a study was 3111 and the maximum was 100.60 All coded information about studies can be found in online appendix tables 1 and 2. For clarity of exposition, individual references are not included next to summaries of study characteristics except to highlight individual studies. Interested readers can find this information in the online appendix tables. A table containing all the extracted contributory factors and their categories is available from the first author on request.

Country of origin

The majority of the studies identified by this review were conducted in the USA (n=34), the UK (n=13), Australia (n=7) and Canada (n=5). One study reported multinational data from 27 countries,115 one reported data from three countries,100 and one reported data from the USA and Canada.121

Setting

Thirty studies reported data collected from general hospital settings. Other studies focused particularly on intensive care on its own (n=17), in combination with coronary care (n=1) or in combination with medicine and surgery (n=1); surgery settings (n=16, including one in combination with intensive care116); anaesthesia (n=7), maternity (n=2), pharmacy (n=2); or transfusion settings (n=2). Other settings included geriatric and cardiovascular wards,111 and the emergency department.107 Two studies reported incidents from US general reporting systems (the US Vaccine Adverse Event Reporting System,45 and the National Medication Error Database120). Finally, one study reported data from a cohort of student nurses.73

Aim of study (primary/secondary) and theoretical basis

The majority of studies explicitly aimed to identify contributory factors (or more commonly referred to as causes) of errors or active failures (n=55). Over half of the included studies made no reference to a theoretical basis driving the identification of contributory factors (n=48). When theory was explicitly mentioned and related to methodology (n=8), all studies referred to Reason's123 model of accident causation. Only six studies included explicit human factors expertise in the elicitation of contributory factors.

Description of empirical data collection methods

A third of studies (n=30) reported data collected as part of an incident reporting scheme based within the hospital; see online appendix table 1 for details. Typically these studies reported the frequency with which staff identified contributory factors of a reported incident from a predefined list (eg, Beckmann et al33) but they also included studies in which free text input from incident reports was analysed qualitatively (eg, Nast et al88). Other papers reported results from observational studies (n=14), interviews (n=9) and focus groups (n=1), surveys (n=8) or case note reviews (n=4). Seventeen studies reported using multiple methods; see online appendix table 2.

Use of a contributory factors framework

The coders assessed the extent to which studies had generated a deductive predefined list of contributory factors (eg, the London Protocol) which then informed data collection or whether studies used inductive methods to elicit contributory factors from participants. For example, within incident reporting studies, deductive use of lists would take the form of a tick box list given to participants, while within interview studies, a list of closed questions might be used to elicit responses about particular contributory factors. The use of a deductive list in these contexts means that no new contributory factors can be elicited from participants; rather only prevalence with which they are endorsed can be assessed.

In total, 46 studies used a predefined contributory factor list as a basis for data collection. Twenty-six of these were based solely on previous frameworks (eg, 7 studies used a variation of the Australian Incident Monitoring study framework,5 3 studies used the Eindhoven classification,11 2 studies used the London Protocol,13 and 14 reported frameworks from miscellaneous previous publications). Seven studies used a combination of literature reviewing and author or other expert opinion to identify the list of contributory factors; one study used previous literature (in addition to pilot work not reported in the paper93); and one study only used expert opinion.116 Twelve studies which used a predefined contributory factors list did not specify how that list was obtained. Two studies were unclear. Of the 34 remaining studies which elicited contributory factors from analysis of primary data, 25 used qualitative methods such as interviews, focus groups or free text coding of incident reports. Eight studies used observational methods, and two used both.

Identification of contributory factors

As described in the Methods section, through the coding of the 1676 contributory factors, a list of 20 contributory factor domains was independently identified by two reviewers (RM and RL) and this list was verified by two further coders (both clinicians, IW and JW). Based on this list we also sought to identify contributory factors that were identified most frequently within the literature. The number of times each of the 20 contributory factors was identified across all of the study settings is shown in table 1 (total column). Across study settings, the five contributory factors identified most frequently were active failures (slips, lapses, mistakes, deviations from policy) (18.2%), individual factors (11%), communication (7.9%), equipment and supplies (6.6%) and management of staff and staffing levels (5.8%). This pattern varied little according to the hospital setting in which the data were collected, with active failures and individual factors consistently being the most frequently identified contributory factors. However, there was some variation. For example, team factors (8.5%) were among the top five contributory factors for surgery, but for no other setting. For anaesthesia, equipment and supplies was the second most cited contributory factor, accounting for 15.2% of the codes. Physical environment was also among the top five factors for anaesthesia. For the general hospital setting, patient factors (7.4%) were among the highest ranked contributory factors but equipment and supplies were not.

Table 1
Frequency of identification for contributory factor domains by setting

Table 2 shows the contributory factors identified by each of the different study methodologies. Studies using incident reporting methodology more commonly identify active failures than interview or observational studies. This is intuitive as generally incident report forms are limiting in terms of the detail of the event which can be recounted and the options for contributory factors available to the reporter. Interview studies appear to more commonly identify individual factors and staff workload as contributory factors. Observation studies tend to identify equipment and supplies marginally more frequently than other methods.

Table 2
Frequency of identification for contributory factor domain by method

We also investigated variation in the identification of contributory factors as a function of whether or not a human factors expert was involved in the identification. Caution must be exercised due to the low number of studies explicitly utilising a human factors expert in the elicitation of contributory factors. However, there was some evidence that, compared with others, human factors experts tend to identify active failures less frequently (11% vs 19%) and identify more latent contributory factors such as team factors (10% vs 3%) and physical environment (7% vs 3%) more frequently. However, despite some evidence that human factors experts were more likely to identify distal than proximal causes, they were more likely to identify individual factors (eg, fatigue, inexperience, 16%) than others (11%). A similar pattern of findings was apparent when comparing studies that employed a theoretical framework in developing their contributory factors coding scheme with those that did not.

Figure 2 is a diagrammatic summary of the findings of the review which represents the speculated hierarchical nature of the identified domains. The diagram entitled ‘the Yorkshire contributory factors framework’ depicts the domains as a series of concentric circles, with active failures at the centre and external policy context as the outer circle. This diagram helps to illustrate the extent to which a domain is proximal to the active failure.

Discussion

As early as 1998, Vincent and colleagues produced a framework for analysing risk and safety in clinical medicine.124 In this influential article, Vincent refers to Reason's123 model of organisational safety, making a clear distinction between the active failures (slips, lapses, mistakes and violations) of healthcare professionals and the latent organisational failures that provide the conditions in which active failures occur. The past 20 years has seen a proliferation of research using this framework or similar models to understand the causes of patient safety incidents. However, to date, there has been no systematic review of this research and therefore existing frameworks for risk management have a theoretical, but not an empirical, basis.

In this review we identified 95 studies (83 independent datasets) that reported on primary research work with the aim of identifying the factors that contributed to patient safety incidents. A systematic review and analysis of these studies suggests that, despite the availability of frameworks and models that encourage the elicitation of latent and active failures (eg, the AIMS system5 asks people to record any physical environment, equipment or work practice or policy issues that contributed to the incident), the overwhelming majority of contributory factors that were identified in this review (irrespective of hospital setting or methodology) were active failures or individual factors. This tendency to focus on the proximal causes of the incident—although slightly less prevalent in our dataset where the reviewer was a human factors expert—was ubiquitous, with approximately 25% of the contributory factors identified as falling into one of these two domains (active failure or individual factor). In fact, despite claiming to investigate the causes of incidents, some studies did not go much beyond the immediate behaviour, performance or skills of the individual who was ‘responsible’ for the incident.73 96 97 Moreover, even when frameworks include systems factors, it is revealing that more attention may be given to the human factors than the systems factors. For example, within AIMS, 33 codes refer to human factors while 21 refer to systems factors. Within the Eindhoven classification (MEDICAL11) there are nine codes that refer to human failure but only four referring to technical and five referring to organisational failure. This emphasis on human failure, rather than latent failure, is much less profound in the London Protocol and WHO classifications. However, our review found that, to date, these frameworks have been used less frequently in published empirical work that identifies contributory factors.

Our review has informed the construction of a framework of contributory factors which includes 20 key domains and suggests the extent to which these are proximal or distal (active or latent failures). This pictorial representation is based on previously described accident causation models,1 17 together with the ratings of our expert group. Thus, it should be noted that while the evidence for the domains reflected within the framework is strong, future research is needed to clarify the exact positioning of the domains within the outer rings and the weighting of each domain (perhaps by varying the size of each segment). Although this framework has a greater number of domains than others (eg, the London Protocol includes just seven domains and the WHO classification specifies five main contributing factors) and therefore might be criticised for being more complex, it captures the full range of contributory factors (across different hospital settings) and gives a greater weighting to systems, rather than human failures. Moreover, some interesting findings have arisen from the work reported here, not least the slight differences in the identification of contributory factors for different settings. The fact that this framework differentiates between surgery, where teamwork was frequently identified, and anaesthesia, where equipment and supply issues were more pronounced, highlights its potential to be generalisable across specialties and error types and yet sufficiently detailed to pick up subtle differences between areas of the hospital to allow the targeting of appropriate interventions. Indeed, this framework has the potential to be used in a number of ways to support improvements to patient safety in practice. It can be used to improve the root cause analysis of serious patient safety incidents. For example, it could be used to analyse patient safety incidents to identify the prevalence of contributory factors and to provide feedback on the quality of existing incident analysis processes. The framework could also be used as a basis for the systematic collection of data about the factors contributing to patient safety incidents through the redesign of local and national reporting systems. The quality of the data elicited through existing reporting systems is often poor3 4 6 because healthcare professionals who are responsible for reporting errors focus predominantly on the individual and situational factors that are proximal to the error. Without guidance on other factors we may learn little about the organisational interventions that might better support safer practice. The framework may also help clinicians or managers to identify proactively poor safety performance at an organisational level and therefore guide risk management strategies. For example, the framework could be used as the basis for developing a measurement tool for patients to report on the local and organisational factors that impact on their care.

The findings reported here are important but should be treated with caution for two reasons. First, although we identified that active failures, individual factors such as knowledge and experience of the healthcare professionals, communication, and equipment and supplies were the contributory factors most frequently recorded in the literature, this should not be interpreted as reflecting the reality of accident causation. Almost half of the studies included in this review (n=48) did not refer to the use of a theoretical framework to support the identification of contributory factors and only eight made explicit links between theory and the identification of contributory factors. A third of the studies were based on analysing the data from incident reports, data that are often reported to be of poor quality.125 For example, some studies simply referred to active failures (eg, doctor prescribed the wrong drug dose) to explain another active failure or incident, rather than make any attempt to understand the reasons for this behaviour. Typically, incident reporting frameworks rely on those doing the reporting to select probable causes from a given list. This is problematic because the person completing the report may have very little understanding of the factors, active and latent, that contribute to incidents. In addition, when a tick box of contributory factors is available, this might not represent a complete list of possible contributory factors. Second, most staff are not trained in identification of systems failures and may neglect to look further than the proximal cause of the error (eg, a slip or lapse) when attributing causes to the incident. Together, the lack of a theoretical framework, the paucity of data available in many of the articles about the underlying causes of the incidents, and the lack of detail about contributory factors also meant that it was impossible to code approximately 15% of the contributory factors. It is also pertinent that only two of the studies reported here involved patients in defining the nature of a patient safety incident or in identifying causes.77 117 Therefore, it must be acknowledged that this framework does not encompass a patient perspective on the causes of safety incidents. This is certainly a worthy future endeavour.

While the findings about the prevalence of the contributory factors identified within the studies should be treated with caution, the variety of methods and the reach of the research across a range of hospital specialities provide strong grounds for arguing that this work captures the full range of contributory factors. Moreover, the rigorous process employed for coding the contributory factors and developing the classification of these factors means that the resulting framework has a strong evidence base. This is supported by the extent to which our own framework coincides with existing frameworks in this field.11–14 The framework (see figure 2) explicitly presents contributory factors at a number of different levels (active failures, situational factors, local working conditions, and organisational and external latent factors), which is a welcome addition to the literature. The majority of studies in this review focused on understanding the contributory factors through interviews with frontline staff and their observations and analyses of accidents. These staff may not have a sufficient grasp of the higher-level organisational factors or external policy context that impact on their performance and behaviour. Thus, future research should attempt to further verify the factors in the two outer circles of the framework. Finally, the clear definitions presented within the framework should aid its practical application, and the reliable attribution of contributory factors. In fact, without these definitions the coding task here (see above) was made much more difficult and distinguishing between some domains was problematic (eg, communication and teamwork). Initial pilot work using the framework to categorise contributory factors from 44 serious untoward incident reports within three UK hospital sites has been encouraging, with agreement between two independent assessors at 80%. This compares favourably to published inter-rater reliability of the Eindhoven classification (68%, κ=0.63).126

Conclusions and policy implications

The poor quality of the current evidence base and the lack of a consistently adopted framework limits the accurate reporting of factors that contribute to error and hence the opportunity to learn from error. We conducted a systematic review of contributory factors identified from a wide range of settings using multiple data collection methods. We then developed an empirically based framework of contributory factors. This framework has the potential to be applied across hospital settings to improve the identification and prevention of factors that cause harm to patients.

Supplementary Material

Supporting Statement:

Acknowledgments

We would like to thank the patient panel members and, in particular, Ted Clarke and Nigat Parveen for their input into this work. We would also like to acknowledge guidance from the improving safety through the involvement of patients Steering Group.

Footnotes

Contributors: RL and RM devised the review protocol. RM and RS performed searches according to protocol, coded studies and extracted data. RM, RL, SG, JW and IW developed the framework. RL, RM and SG contributed to the first draft of the manuscript. All authors commented on and revised subsequent drafts. All authors read and approved the final manuscript. RL is the guarantor.

Funding: This article presents independent research commissioned by the National Institute for Health Research (NIHR) under the Programme Grants for Applied Research (Improving safety through the involvement of patients). The views expressed in this article are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. This work was undertaken as part of the NIHR funded ‘Patient involvement in patient safety’ programme grant for applied research (Grant number: RP-PG-0108-10049).

Competing interests: All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: We would be happy to share the database of contributory factors extracted for each of the papers in this review.

References

1. Reason J. Understanding adverse events: human factors. Qual Health Care 1995;4:80–9. [PMC free article] [PubMed]
2. Carroll JS, Edmondson AC. Leading organisational learning in healthcare. Qual Saf Health Care 2002;11:51–6. [PMC free article] [PubMed]
3. Vincent C. Incident reporting and patient safety. BMJ 2007;334:51. [PMC free article] [PubMed]
4. Shojania KG. The frustrating case of incident reporting systems. Qual Saf Health Care 2008;17:400–2. [PubMed]
5. Runciman WB, Sellen A, Webb RK, et al. The Australian incident monitoring study. Errors, incidents and accidents in anaesthetic practice. Anaesth Intensive Care 1993;21:506–9. [PubMed]
6. Wu AW, Pronovost P, Morlock L. ICU incident reporting systems. J Crit Care 2002;17:86–94. [PubMed]
7. Reason J. Managing the risks of organisational accidents. Aldershot, UK: Ashgate, 1997.
8. Kohn LT, Corrigan JM, Donaldson MS. Committee on Quality of Health Care in America. To Err is Human: Building A Safer Health Care System. Washington DC: Institute of Medicine National Academy Press, 1999.
9. Department of Health An Organisation With A Memory. London: Crown Copyright: The Stationary Office, 2000.
10. Department of Health Doing Less Harm: Improving the Safety and Quality of Care Through Reporting, Analysing and Learning From Adverse Events Involving NHS Patients. London: Crown Copyright: The Stationary Office, 2001.
11. Van Vuuren W, Shea C, van der Schaaf TW. The Development of an Incident Analysis Tool for the Medical Field. Eindhoven: Report from Faculty of Technology Management, University of Eindhoven, 1997.
12. World Health Organisation Conceptual Framework for the International Classification for Patient Safety: Final Technical Report Version 1.1. Geneva: World Health Organisation, 2009.
13. Taylor-Adams S, Vincent C. Systems analysis of clinical incidents: the London Protocol. Clin Risk 2004;10:211–20.
14. Bagian JP, Gosbee J, Lee CZ, et al. The Veterans Affairs root cause analysis system in action. Jt Comm J Qual Improv 2002;28:531–45. [PubMed]
15. Gaba DM. Structural and organizational issues in patient safety: a comparison of health care to other high-hazard industries. Calif Manage Rev 2000;43:83–102.
16. Vincent C. Patient Safety. Oxford: BMJ Books, 2010.
17. Reason J. Human error: models and management. BMJ 2000;320:768–70. [PMC free article] [PubMed]
18. Toft B. External Inquiry into the Adverse Incident that Occurred at Queen's Medical Centre, Nottingham, 4th January 2001. London: Department of Health, 2001.
19. Lawton R, Gardner P, Green B, et al. An engineered solution to the maladministration of spinal injections. Qual Saf Health Care 2009;18:492–5. [PubMed]
20. Musson DM, Helmreich RL. Team training and resource management in health care: current issues and future directions. Harv Health Policy Rev 2004;5:25–35.
21. Leape LL. Why should we report adverse incidents? J Eval Clin Pract 1999;5:1–4. [PubMed]
22. Carthey J, de Leval MR, Reason J. The human factor in cardiac surgery: errors and near misses in a high technology medical domain. Ann Thorac Surg 2001;72:300–5. [PubMed]
23. Runciman WB, Baker GR, Michel P, et al. The epistemology of patient safety research. Int J Evid Based Healthc 2008;6:476–86. [PubMed]
24. Ohsaka A, Kobayashi M, Abe K. Causes of failure of a barcode-based pretransfusion check at the bedside: experience in a university hospital. Transfus Med 2008;18:216–22. [PubMed]
25. Crawford SY, Cohen MR, Tafesse E. Systems factors in the reporting of serious medication errors in hospitals. J Med Syst 2003;27:543–51. [PubMed]
26. Chassin MR, Becher EC. The wrong patient. Ann Intern Med 2002;136:826–33. [PubMed]
27. Sirriyeh R, Lawton R, Gardner P, et al. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. Published Online First: 16 March 2011. doi:10.1111/j.1365-2753.2011.01662.x. [PubMed]
28. Abramson NS, Silvasy Wald K, Grenvik AN, et al. Adverse occurrences in intensive care units. JAMA 1980;244:1582–4. [PubMed]
29. Ahmad I, Thompson A, Frawley M, et al. Five-year experience of critical incidents associated with patient-controlled analgesia in an Irish University Hospital. Ir J Med Sci 2010;179:393–7. [PubMed]
30. Alfredsdottir H, Bjornsdottir K. Nursing and patient safety in the operating room. J Adv Nurs 2008;61:29–37. [PubMed]
31. Anoosheh M, Ahmadi F, Faghihzadeh S, et al. Causes and management of nursing practice errors: a questionnaire survey of hospital nurses in Iran. Int Nurs Rev 2008;55:288–95. [PubMed]
32. Barach P, Johnson JK, Ahmad A, et al. A prospective observational study of human factors, adverse events, and patient outcomes in surgery for pediatric cardiac disease. J Thorac Cardiovasc Surg 2008;136:1422–8. [PubMed]
33. Beckmann U, Bohringer C, Carless R, et al. Evaluation of two methods for quality improvement in intensive care: facilitated incident monitoring and retrospective medical chart review. Crit Care Med 2003;31:1006–11. [PubMed]
34. Beckmann U, Gillies DM. Factors associated with reintubation in intensive care: an analysis of causes and outcomes. Chest 2001;120:538–42. [PubMed]
35. Beckmann U, Baldwin I, Hart GK, et al. The Australian Incident Monitoring Study in Intensive Care: aIMS-ICU. An analysis of the first year of reporting. Anaesth Intensive Care 1996;24:329. [PubMed]
36. Beckmann U, West LF, Groombridge GJ, et al. The Australian Incident Monitoring Study in Intensive Care: aIMS-ICU. The development and evaluation of an incident reporting system in intensive care. Anaesth Intensive Care 1996;24:314–19. [PubMed]
37. Beso A, Franklin BD, Barber N. The frequency and potential causes of dispensing errors in a hospital pharmacy. Pharm World Sci 2005;27:182–90. [PubMed]
38. Blike GT, Christoffersen K, Cravero JP, et al. A method for measuring system safety and latent errors associated with pediatric procedural sedation. Anesth Analg 2005;101:48–58. [PubMed]
39. Buckley TA, Short TG, Rowbottom YM, et al. Critical incident reporting in the intensive care unit. Anaesthesia 1997;52:403–9. [PubMed]
40. Busse DK, Wright DJ. Classification and analysis of incidents in complex medical environments. Top Health Inf Manage 2000;20:1–11. [PubMed]
41. Callum JL, Kaplan HS, Merkley LL, et al. Reporting of near-miss events for transfusion medicine: improving transfusion safety. Transfusion 2001;41:1204–11. [PubMed]
42. Catchpole KR, Giddings AE, de Leval MR, et al. Identification of systems failures in successful paediatric cardiac surgery. Ergonomics 2006;49:567–88. [PubMed]
43. Catchpole KR, Godden PJ, Giddings AE, et al. Identifying and Reducing Errors in the Operating Theatre: Patient Safety Research Programme PS 012, Final report. London: HMSO, 2005.
44. Catchpole KR, Giddings AEB, Wilkinson M, et al. Improving patient safety by identifying latent failures in successful operations. Surgery 2007;142:102–10. [PubMed]
45. Chang S, Pool V, O'Connell K, et al. Preventable mix-ups of tuberculin and vaccines: reports to the US vaccine and drug safety reporting systems. Drug Saf 2008;31:1027–33. [PubMed]
46. Charuluxananan S, Suraseranivongse S, Jantorn P, et al. Multicentered study of model of anesthesia related adverse events in Thailand by incident report (the Thai Anesthesia Incidents Monitoring Study): results. J Med Assoc Thai 2008;91:1011–19. [PubMed]
47. Chianca TC. Nursing faults in the recovery period of surgical patients. Rev Lat Am Enfermagem 2006;14:879–86. [PubMed]
48. Christian CK, Gustafson ML, Roth EM, et al. A prospective study of patient safety in the operating room. Surgery 2006;139:159–73. [PubMed]
49. Coombes ID, Stowasser DA, Coombes JA, et al. Why do interns make prescribing errors? A qualitative study. Med J Aust 2008;188:89–94. [PubMed]
50. Cooper JB, Newbower RS, Kitz RJ. An analysis of major errors and equipment failures in anesthesia management: considerations for prevention and detection. Anesthesiology 1984;60:34–42. [PubMed]
51. Cote CJ, Notterman DA, Karl HW, et al. Adverse sedation events in pediatrics: a critical incident analysis of contributing factors. Pediatrics 2000;105:805–14. [PubMed]
52. Cronin CM. Five years of learning from analysis of clinical occurrences in pediatric care using the London Protocol. Healthc Q 2006;9:16–21. [PubMed]
53. Cullen DJ, Sweitzer BJ, Bates DW, et al. Preventable adverse drug events in hospitalized patients: a comparative study in intensive care and general care units. Crit Care Med 1997;25:1289–97. [PubMed]
54. Currie M, Pybus DA, Torda TA. A prospective survey of anaesthetic critical events. A report on a pilot study of 88 cases. Anaesth Intensive Care 1988;16:103–7. [PubMed]
55. Currie M. A prospective survey of anaesthetic critical events in a teaching hospital. Anaesth Intensive Care 1989;17:403–11. [PubMed]
56. Davis P, Lay-Yee R, Briant R, et al. Adverse Events in New Zealand Public Hospitals: Principal Findings from a National Survey. Occasional Paper Number 3. Wellington, New Zealand: Ministry of Health, 2001.
57. Davis P, Lay-Yee R, Briant R, et al. Preventable in-hospital medical injury under the ‘no fault’ system in New Zealand. Qual Saf Health Care 2003;12:251–6. [PMC free article] [PubMed]
58. de Leval MR, Carthey J, Wright DJ, et al. Human factors and cardiac surgery: a multicenter study. J Thorac Cardiovasc Surg 2000;119:661–70. [PubMed]
59. Dean B, Schachter M, Vincent C, et al. Causes of prescribing errors in hospital inpatients: a prospective study. Lancet 2002;359:1373–8. [PubMed]
60. Dornan T, Ashcroft D, Healthfield H, et al. An in Depth Investigation into Causes of Prescribing Errors by Foundation Trainees in Relation to Their Medical Education. EQUIP Study. Manchester: Hope Hospital (University of Manchester Medical School Teaching Hospital), 2009.
61. ElBardissi AW, Wiegmann DA, Dearani JA, et al. Application of the human factors analysis and classification system methodology to the cardiovascular surgery operating room. Ann Thorac Surg 2007;83:1412–19. [PubMed]
62. Elnicki RA, Schmitt JP. Contribution of patient and hospital characteristics to adverse patient incidents. Health Serv Res 1980;15:397–414. [PMC free article] [PubMed]
63. Fabri PJ, Zayas-Castro JL. Human error, not communication and systems, underlies surgical complications. Surgery 2008;144:557–65. [PubMed]
64. Forster AJ, Fung I, Caughey S, et al. Adverse events detected by clinical surveillance on an obstetric service. Obstet Gynecol 2006;108:1073–83. [PubMed]
65. Frey B, Kehrer B, Losa M, et al. Comprehensive critical incident monitoring in a neonatal-pediatric intensive care unit: experience with the system approach. Intensive Care Med 2000;26:69–74. [PubMed]
66. Galletly DC, Mushet NN. Anaesthesia system errors. Anaesth Intensive Care 1991;19:66–73. [PubMed]
67. Galvan C, Bacha EA, Mohr J, et al. A human factors approach to understanding patient safety during pediatric cardiac surgery. Prog Pediatr Cardiol 2005;20:13–20.
68. Gawande AA, Zinner MJ, Studdert DM, et al. Analysis of errors reported by surgeons at three teaching hospitals. Surgery 2003;133:614–21. [PubMed]
69. Giraud T, Dhainaut JF, Vaxelaire JF, et al. Iatrogenic complications in adult intensive-care units—a prospective 2-center study. Crit Care Med 1993;21:40–51. [PubMed]
70. Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med 2005;165:1493–9. [PubMed]
71. Graf J, von den Driesch A, Koch KC, et al. Identification and characterization of errors and incidents in a medical intensive care unit. Acta Anaesthesiol Scand 2005;49:930–9. [PubMed]
72. Hamman WR, Beaudin-Seiler BM, Beaubien JM, et al. Using in situ simulation to identify and resolve latent environmental threats to patient safety: case study involving a labor and delivery ward. J Patient Saf 2009;5:184–7. [PubMed]
73. Harding L, Petrick T. Nursing student medication errors: a retrospective review. J Nurs Educ 2008;47:43–7. [PubMed]
74. Holzmueller CG, Pronovost PJ, Dickman F, et al. Creating the web-based intensive care unit safety reporting system. J Am Med Inform Assoc 2005;12:130–9. [PMC free article] [PubMed]
75. Horwitz LL, Meredith T, Schuur JD, et al. Dropping the baton: a qualitative analysis of failures during the transition from emergency department to inpatient care. Ann Emerg Med 2009;53:701–10. [PubMed]
76. Inoue K, Koizumi A. Application of human reliability analysis to nursing errors in hospitals. Risk Anal 2004;24:1459–73. [PubMed]
77. Itoh K, Andersen HB. Causes of medical errors as perceived by patients and healthcare staff. In: Aven T, Vinnem JE, editors. , eds. Risk, Reliability and Societal Safety. Oxford: Taylor & Francis, 2007:179–85.
78. Kaplan HS, Battles JB, van der Schaaf TW, et al. Identification and classification of the causes of events in transfusion medicine. Transfusion 1998;38:1071–81. [PubMed]
79. Khan FA, Hoda MQ. A prospective survey of intra-operative critical incidents in a teaching hospital in a developing country. Anaesthesia 2001;56:177–82. [PubMed]
80. Klanarong S, Chau-in W, Pulnitiporn A, et al. The Thai Anesthesia Incidents Study (THAI Study) of anesthetic equipment failure/malfunction: a qualitative analysis for risk factors. J Med Assoc Thai 2005;88(Suppl 7):S134–40. [PubMed]
81. Kopp BJ, Erstad BL, Allen ME, et al. Medication errors and adverse drug events in an intensive care unit: direct observation approach for detection. Crit Care Med 2006;34:415–25. [PubMed]
82. Kusumaphanyo C, Charuluxananan S, Sriramatr D, et al. The Thai Anesthesia Incident Monitoring Study (Thai AIMS) of anesthetic equipment failure/malfunction: an analysis of 1996 incident reports. J Med Assoc Thai 2009;92:1442–9. [PubMed]
83. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA 1995;274:35–43. [PubMed]
84. Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing. JAMA 1997;277:312–17. [PubMed]
85. Lundy D, Laspina S, Kaplan H, et al. Seven hundred and fifty-nine (759) chances to learn: a 3-year pilot project to analyse transfusion-related near-miss events in the Republic of Ireland. Vox Sang 2007;92:233–41. [PubMed]
86. Meurier CE, Vincent CA, Parmar DG. Learning from errors in nursing practice. J Adv Nurs 1997;26:111–19. [PubMed]
87. Morita K. Evaluation of improved safety management program for outpatient drug dispensing in terms of effect on potential adverse drug events. Kurume Med J 2004;51:151–7. [PubMed]
88. Nast PA, Avidan M, Harris CB, et al. Reporting and classification of patient safety events in a cardiothoracic intensive care unit and cardiothoracic postoperative care unit. J Thorac Cardiovasc Surg 2005;130:1137–43. [PubMed]
89. Neale G, Woloshynowych M, Vincent C. Exploring the causes of adverse events in NHS hospital practice. J R Soc Med 2001;94:322–30. [PMC free article] [PubMed]
90. Needham DM, Thompson DA, Holzmueller CG, et al. A system factors analysis of airway events from the Intensive Care Unit Safety Reporting System (ICUSRS). Crit Care Med 2004;32:2227–33. [PubMed]
91. Needham DM, Sinopoli DJ, Thompson DA, et al. A system factors analysis of ‘line, tube, and drain’ incidents in the intensive care unit. Crit Care Med 2005;33:1701–7. [PubMed]
92. Nuckols TK, Bell DS, Paddock SM, et al. Comparing process- and outcome-oriented approaches to voluntary incident reporting in two hospitals. Jt Comm J Qual Patient Saf 2009;35:139–45. [PubMed]
93. Nuckols TK, Bell DS, Paddock SM, et al. Contributing factors identified by hospital incident report narratives. Qual Saf Health Care 2008;17:368–72. [PubMed]
94. Parker SE, Laviana AA, Wadhera RK, et al. Development and evaluation of an observational tool for assessing surgical flow disruptions and their impact on surgical performance. World J Surg 2010;34:353–61. [PubMed]
95. Pearse RM, Dana EC, Lanigan CJ, et al. Organisational failures in urgent and emergency surgery. A potential peri-operative risk factor. Anaesthesia 2001;56:684–9. [PubMed]
96. Proctor ML, Pastore J, Gerstle JT, et al. Incidence of medical error and adverse outcomes on a pediatric general surgery service. J Pediatr Surg 2003;38:1361–5. [PubMed]
97. Rothschild JM, Landrigan CP, Cronin JW, et al. The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med 2005;33:1694–700. [PubMed]
98. Sanghera IS, Franklin BD, Dhillon S. The attitudes and beliefs of healthcare professionals on the causes and reporting of medication errors in a UK intensive care unit. Anaesthesia 2007;62:53–61. [PubMed]
99. Short TG, O'Regan A, Jayasuriya JP, et al. Improvements in anaesthetic care resulting from a critical incident report programme. Anaesthesia 1996;51:615–21. [PubMed]
100. Silen-Lipponen M, Tossavainen K, Turunen H, et al. Potential errors and their prevention in operating room teamwork as experienced by Finnish, British and American nurses. Int J Nurs Pract 2005;11:21–32. [PubMed]
101. Singh H, Thomas EJ, Wilson L, et al. Errors of diagnosis in pediatric practice: a multisite survey. Pediatrics 2010;126:70–9. [PMC free article] [PubMed]
102. Sinopoli DJ, Needham DM, Thompson DA, et al. Intensive care unit safety incidents for medical versus surgical patients: a prospective multicenter study. J Crit Care 2007;22:177–83. [PubMed]
103. Sintavanuruk K, Rodanant O, Kositanurit I, et al. The Thai Anesthesia Incident Monitoring Study (Thai AIMS) of endobronchial intubation: an analysis of 1996 incident reports. J Med Assoc Thai 2008;91:1854–61. [PubMed]
104. Skapik JL, Pronovost PJ, Miller MR, et al. Pediatric safety incidents from an intensive care reporting system. J Patient Saf 2009;5:95–101. [PubMed]
105. Skibinski KA, White BA, Lin LI, et al. Effects of technological interventions on the safety of a medication-use system. Am J Health Syst Pharm 2007;64:90–6. [PubMed]
106. Smits M, Zegers M, Groenewegen PP, et al. Exploring the causes of adverse events in hospitals and potential prevention strategies. Qual Saf Health Care 2010;19:e5. [PubMed]
107. Smits M, Groenewegen PP, Timmermans DR, et al. The nature and causes of unintended events reported at ten emergency departments. BMC Emerg Med 2009;9:16. [PMC free article] [PubMed]
108. Suresh G, Horbar JD, Plsek P, et al. Voluntary anonymous reporting of medical errors for neonatal intensive care. Pediatrics 2004;113:1609–18. [PubMed]
109. Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: an insidious contributor to medical mishaps. Acad Med 2004;79:186–94. [PubMed]
110. Tang FI, Sheu SJ, Yu S, et al. Nurses relate the contributing factors involved in medication errors. J Clin Nurs 2007;16:447–57. [PubMed]
111. Tissot E, Cornette C, Limat S, et al. Observational study of potential risk factors of medication administration errors. Pharm World Sci 2003;25:264–8. [PubMed]
112. Tucker AL, Singer SJ, Hayes JE, et al. Front-line staff perspectives on opportunities for improving the safety and efficiency of hospital work systems. Health Serv Res 2008;43:1807–29. [PMC free article] [PubMed]
113. Tucker AL, Spear SJ. Operational failures and interruptions in hospital nursing. Health Serv Res 2006;41:643–62. [PMC free article] [PubMed]
114. Tuttle D, Holloway R, Baird T, et al. Electronic reporting to improve patient safety. Qual Saf Health Care 2004;13:281–6. [PMC free article] [PubMed]
115. Valentin A, Capuzzo M, Guidet B, et al. Errors in administration of parenteral drugs in intensive care units: multinational prospective study. BMJ 2009;338:b814. [PMC free article] [PubMed]
116. van Beuzekom M, Akerboom SP, Boer F. Assessing system failures in operating rooms and intensive care units. Qual Saf Health Care 2007;16:45–50. [PMC free article] [PubMed]
117. Watt I, Birks Y, Entwistle V, et al. A Review of Strategies to Promote Patient Involvement, a Study to Explore Patient's Views and Attitudes and a Pilot Study to Evaluate the Acceptability of Selected Patient Involvement Strategies. York: University of York, 2009.
118. Wiegmann DA, ElBardissi AW, Dearani JA, et al. Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery 2007;142:658–65. [PubMed]
119. Williamson JA, Webb RK, Sellen A, et al. The Australian Incident Monitoring Study. Human failure: an analysis of 2000 incident reports. Anaesth Intensive Care 1993;21:678–83. [PubMed]
120. Wolf ZR, Hicks R, Serembus JF. Characteristics of medication errors made by students during the administration phase: a descriptive study. J Prof Nurs 2006;22:39–51. [PubMed]
121. Wong DR, Salm TJV, Ali IS, et al. Prospective assessment of intraoperative precursor events during cardiac surgery. Eur J Cardiothorac Surg 2006;29:447–55. [PubMed]
122. Wright D, Mackenzie SJ, Buchan I, et al. Critical incidents in the intensive therapy unit. Lancet 1991;338:676–8. [PubMed]
123. Reason J. Human Error. New York: Cambridge University Press, 1990.
124. Vincent C, Taylor-Adams S, Stanhope N. Framework for analysing risk and safety in clinical medicine. BMJ 1998;316:1154–7. [PMC free article] [PubMed]
125. Armitage G, Newell R, Wright J. Improving the quality of drug error reporting. J Eval Clin Pract 2010;16:1189–97. [PubMed]
126. Smits M, Janssen J, De Vet R, et al. Analysis of unintended events in hospitals: inter-rater reliability of constructing causal trees and classifying root causes. Int J Qual Health Care 2009;21:292–300. [PubMed]

Articles from BMJ Open Access are provided here courtesy of BMJ Group