PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of qualsafetyQuality and Safety in Health CareCurrent TOCInstructions for authors
 
Qual Saf Health Care. 2007 June; 16(3): 164–168.
PMCID: PMC2464990

Rates and types of events reported to established incident reporting systems in two US hospitals

Abstract

Background

US hospitals have had voluntary incident reporting systems for many years, but the effectiveness of these systems is unknown. To facilitate substantial improvements in patient safety, the systems should capture incidents reflecting the spectrum of adverse events that are known to occur in hospitals.

Objective

To characterise the incidents from established voluntary hospital reporting systems.

Design

Observational study examining about 1000 reports of hospitalised patients at each of two hospitals.

Patients and setting

16 575 randomly selected patients from an academic and a community hospital in the US in 2001.

Main outcome measures

Rates of incidents reported per hospitalised patient and characteristics of reported incidents.

Results

9% of patients had at least one reported incident; 17 incidents were reported per 1000 patient‐days in hospital. Nurses filed 89% of reports, physicians 1.9% and other providers 8.9%. The most common types were medication incidents (29%), falls (14%), operative incidents (15%) and miscellaneous incidents (16%); 59% seemed preventable and preventability was not clear for 32%. Among the potentially preventable incidents, 43% involved nurses, 16% physicians and 19% other types of providers. Qualitative examination of reports indicated that very few involved prescribing errors or high‐risk procedures.

Conclusions

Hospital reporting systems receive many reports, but capture a spectrum of incidents that differs from the adverse events known to occur in hospitals, thereby substantially underdetecting physician incidents, particularly those involving operations, high‐risk procedures and prescribing errors. Increasing the reporting of physician incidents will be essential to enhance the effectiveness of hospital reporting systems; therefore, barriers to reporting such incidents must be minimised.

Voluntary incident reporting in hospitals is a centrepiece of national patient safety policies in the US, the UK and Australia, because this practice has improved safety in other high‐risk industries.1,2,3,4,5 However, although voluntary incident reporting systems have long existed in the hospitals of the US,4,6,7 their effectiveness remains unclear.4,7,8

In US hospitals, reporting systems were developed to prevent and prepare for litigation, whereas in other industries reporting systems were designed to improve safety.6,9,10,11 In around 1965, risk managers adapted the critical incident technique to reduce events that could lead to malpractice claims against hospitals, particularly medication errors, falls, patient misidentification and retained foreign bodies after surgery.6,12 They also started using the reports as alerts to possible claims.10,11 Hospital reporting systems capture few of the errors and adverse events that are identifiable by other means13,14,15; nevertheless, some collect several thousand reports per year.14

Although published literature emphasises increasing reporting, this seems necessary but not sufficient to make these systems effective. To prevent serious harm to patients, the systems should capture incidents reflecting actual or potential risks of such harm.16 Large‐scale studies using medical‐record review have examined disabling and fatal adverse events that occurred during hospital care in the US, the UK and Australia.17,18,19,20 Assuming that the studies are accurate, reported incidents should be similar to the adverse events that occurred in the same country. In US hospitals, incidents should resemble adverse events from the Harvard and Utah/Colorado Medical Practice Studies, nearly all of which occurred during hospitalisation.19,20

In this study, our objectives were to determine how frequently incidents are reported in two US hospitals and to characterise the incidents in terms of type, preventability, location, harm, the types of providers reporting and the types of providers involved.

These two reporting systems are typical for US hospitals in terms of their purpose, functioning and definition of reportable incidents (Pham C, RAND Corporation, personal communication, March 21, 2006).6,8 Reporting practices and safety culture have changed only modestly in US hospitals in recent years,21 and most hospitals still use paper reports. In 2002, our study hospitals introduced electronic reporting. As this was not representative of how most systems operate, we studied the conditions of 2001.

Methods

We examined a representative sample of about 1000 incident reports of inpatients at each hospital.

Setting

The study hospitals were an academic tertiary referral centre (668 beds) and an affiliated community hospital (363 beds) in an urban area in Southern California.

In 2001, neither of the hospitals had changed reporting practices recently nor formally evaluated its patient safety culture. Risk/quality managers received 230 reports per month at the academic hospital and 100 at the community hospital. Reporting was voluntary but not anonymous. Reportable incidents included, “Any occurrence that is not consistent with the routine operation of the medical center and that potentially may, or actually did, result in injury, harm or loss to any patient, or visitor of the medical center.”

Incident reports were one‐page paper forms listing: patient identifiers, age, diagnosis, admission date; event date, time and location; and reporter name and profession. Structured questions addressed event type, harm/injury rating, falls and medication errors. Incident descriptions were entered in a 2.5‐inch× 4.25‐inch area plus additional pages when desired. Unit nurse managers reviewed reports, occasionally added comments and forwarded them to risk/quality managers, who summarised them in incident‐report databases. For events involving legal issues, providers sometimes called risk/quality mangers without filing reports. We excluded pharmacy systems documenting medication order changes, because they functioned separately and lacked detailed descriptions.

Subjects

To obtain a representative sample of reports, we (1) obtained discharge and incident‐report databases for 2001, (2) linked them to determine which patients had reports and (3) sampled enough number of patients to obtain about 1000 reports at each hospital. This required all 9850 patients at the community hospital. At the academic hospital, we used a SAS random‐number generator to select 6725 (30%) of 22 430 patients. Because discharge databases listed hospitalisations, we oversampled patients with multiple discharges and, therefore, weighted subsequent analyses to adjust for this.

The institutional review board for both hospitals approved the study and did not require informed consent.

Data collection

Reports were ineligible if they addressed only visitors or staff, or care not associated with hospitalisation. We abstracted study variables from information reporters provided in original reports; we did not corroborate reports with medical records. When multiple reports described an incident, we assessed them individually and in summary.

For reasons of confidentiality, one board‐certified internist (TKN) authorised by the risk/quality management departments abstracted all reports. We randomly selected 10% for re‐abstraction by this reviewer (intra‐rater reliability) and another 10% for abstraction by two secondary reviewers (inter‐rater reliability), a board‐certified internist (DSB) and a board‐certified pathologist (LHH). Secondary reviewers evaluated redacted copies lacking dates, times, and patient and provider identifiers. The principal reviewer developed data collection methods with the secondary reviewers, who had substantial patient‐safety research experience.

Study variables

We transcribed the type of provider who was reporting, location and harm/injury rating from reports. Physician reviewers judged risks of medical versus social harm, event types, preventability and, for potentially preventable incidents, provider types that seemed to be involved. We used Harvard Medical Practice Study categories for event type and location.22 We used a three‐point preventability scale (raters could not distinguish two intermediate categories from a common four‐point scale).23 Provider types included nurses, physicians, other providers and unknown providers.

Analytical methods

First, we computed reported incidents per 1000 patient‐days, percentages of subjects and hospitalisations associated with a report, and types of providers reporting.

Next, we examined harm/injury and risk‐of‐harm ratings, analysed event types across preventability categories, and qualitatively examined incidents in common categories. We examined locations and, for potentially preventable incidents, types of providers involved.

We conducted analyses using SAS V. 9.1.3. The MAGREE macro determined κ statistics, and Surveyfreq adjusted for the number of hospitalisations per subject (SAS/STAT software, Version 9.1.3 of the SAS System for Windows. Copyright © 2002 SAS Institute Inc. Cary, NC, USA.). We compared events across the two hospitals using the χ2 test and reported major differences.

Results

There were 2284 eligible reports addressing study subjects (fig 11).). We evaluated 2228 paper reports, as 56 that seemed eligible according to the incident‐report database were missing. Thirty‐nine incidents had multiple reports (40 additional reports in total). There were 2244 unique incidents (2188 paper reports plus 56 missing reports).

figure qc19901.f1
Figure 1 Sampling and data collection strategy.

To adjust for our sampling strategy, the subsequent results are weighted to represent all patients discharged from study hospitals in 2001. These 32 280 patients had an estimated 3911 incidents described in 3981 reports (3875, excluding missing reports). There were 17 incidents per 1000 patient‐days. In all, 9% of patients and 8% of hospitalisations had at least one report. Nurses filed 3407 (88%), physicians 73 (1.9%), other providers 346 (8.9%) and unknown providers 49 (1.3%).

Table 11 lists harm/injury and risk‐of‐harm ratings. Harm/injury ratings were qualitatively variable and often missing. Most reports described medical rather than social risks (intra‐rater, κ 0.74; inter‐rater, 0.48).

Table thumbnail
Table 1 Harms and risks identified in incident reports*

Table 22 lists event type by preventability and Table 33 provides examples. Medication events, operative events, miscellaneous events, falls and procedural events (intra‐rater, κ 0.92; inter‐rater, 0.86) were most common (fig 22).). Of them, 59% were preventable, 9% were not and 32% were of indeterminate preventability (intra‐rater, κ 0.86; inter‐rater, 0.68).

figure qc19901.f2
Figure 2 Reported incidents full adverse events from US studies.
Table thumbnail
Table 2 Event type versus preventability*
Table thumbnail
Table 3 Examples from common incident type and preventability categories

Half the incidents (1859) occurred in floor units, 21% (797) in intensive care units and 14% (544) in operating rooms; < 5% involved other locations.

Providers involved in potentially preventable incidents included: nurses, 43% (1507); physicians, 16% (556); and other providers, 19% (657); (intra‐rater, κ 0.72–0.89; inter‐rater, 0.36–0.61). Multiple types were involved in 239 (7%) and no type was identified for 993 (28%) .

Rates of reported incidents were similar between the hospitals; differences in other variables were modest. At the community hospital, physicians reported less often (1% vs 2%, p<0.001) but were involved more often (18% vs 15%; p<0.001). Incidents involved operations (10% vs 16%) and drugs (22% vs 32%) less often, and miscellaneous incidents were more common (24% vs 12%; (p<0.001).

Discussion

Summary

Our study documents the rates and types of incidents captured by two reporting systems that are typical among US hospitals. Of the hospitalisations, 8% involved incidents that providers, mainly nurses, found troubling enough to report. Consistent with historical uses,6 reports emphasised drug administration errors, falls, incorrect needle counts, identification issues and cardiac arrests. Two‐thirds occurred in patient rooms. Almost 60% were preventable and, for most of these, non‐physician providers seemed to be involved.

Context

To date, under‐reporting has been recognised as the salient limitation of the hospital reporting systems.8,13,14,15 We found, however, that reported incidents were more than twice as prevalent as adverse events in the Harvard and Utah/Colorado studies.20,24 Moreover, a recent study of electronic reporting documented twice the rates we observed.25 Perhaps a key issue is not whether hospital reporting systems can collect many incidents, but whether the incidents collected reflect the greatest threats to patient safety.

Hospital reporting systems seem to capture a different spectrum of events than the Harvard and Utah/Colorado studies established as priorities. Reviewing hospital medical records, those studies found that injuries involving operations, procedures and drugs were prevalent, but injuries due to falls were rare. Of the adverse events, 40% occurred in operating rooms, 25% in floor units, and 3% in intensive care.19,20 In comparison, the reporting systems that we studied identified more falls, drug errors and miscellaneous events occurring in patient rooms, but far fewer incidents involving surgery. Electronic systems capture similar incidents.25

Our qualitative observations also suggest that hospital reporting systems collect many reports of some subtypes while missing other important subtypes. For example, many procedural incidents were infiltrated peripheral intravenous lines; only a handful involved high‐risk procedures such as endoscopy, bronchoscopy or central line placement. Many drug incidents were omitted doses; few involved incorrect drugs or doses, which were common in the Utah/Colorado study.20

The reports probably emphasise issues such as peripheral line care and drug administration, because nurses file most reports and focus on issues they know well. However, in the Utah/Colorado study, physicians were responsible for 94% of all events (including non‐preventable ones), nurses for 2% and other providers for 3%.26 The Harvard study also focused on physician events.22 Thus, the fact that only 16% of reports in our study address physician care represents a major limitation of hospital reporting systems.

Reports probably underemphasise physician care for two reasons. First, reporting systems were developed to minimise litigation against hospitals and their employees.6 Hospitals have instead addressed physician care via peer review, credentialing, and morbidity and mortality conferences.27,28 Second, physicians are probably better than other providers in identifying physician errors and physician reporting is minimal, in both our studies and in one of electronic reporting.25

Increasing the reporting of physician incidents will not assure that hospital reporting systems are effective, but without it they cannot be. Hospitals, physicians and, when necessary, policymakers, should collaboratively resolve the structural and cultural barriers to reporting physician incidents. First, shifting the purpose of reporting from preventing litigation against hospitals to improving safety would make capturing physician incidents a higher priority. Second, barriers to reporting by physicians should be mitigated. Reports can be disclosed in litigation in some states,29,30,31 and hospitals sometimes limit employees to reporting. Cultural barriers perceived by physicians include: time and effort, lack of confidentiality, lack of feedback, poor understanding of what to report or how, fear of blame and reprisal, and doubts about the value of reporting.32,33,34

Limitations

Our study included two hospitals within one geographical area. Nevertheless, many patient safety studies address one hospital, and we included both academic and community settings.

Liability concerns limited report review methods; the results reflect the judgments of one physician with full access to the reports. Agreement with two secondary reviewers was generally moderate to substantial, although occasionally lower.35 However, the Utah/Colorado study also used one reviewer per case,20 and prior studies have documented similar to worse agreement among three reviewers.36,37,38

Conclusion

Our findings identify important limitations to incident reporting systems in US hospitals. The spectrum of incidents captured differs from the adverse events that are known to occur, and physician incidents are markedly under‐represented. Increasing the reporting of physician incidents will be essential to enhance the effectiveness of hospital‐reporting systems; therefore, barriers to reporting such incidents must be minimised.

Acknowledgements

We thank Martin F. Shapiro, Diana Hedges, Louise Underdahl, Barbara Browning, Sarah Foss and Kathleen Fragola who made this study possible. We also thank Lisa Judge who worked on report abstraction; Coralee Yale and Xuesheng Li who contributed their time to provide discharge data; and Mary Vaiana who edited the final draft.

Footnotes

Funding: This study was supported by a National Research Service Award fellowship, a Developmental Center grant (HS11512) from the Agency for Healthcare Research and Quality and the RAND Corporation.

Competing interests: None.

References

1. Agency for Healthcare Research and Quality Detail of performance analysis table, quality/safety of patient safety portfolio. Budget estimates for Appropriations Committees, fiscal year 2006. Available at: http://www.ahrq.gov/about/cj2006/cj06exbua.htm (accessed 31 Mar 2007)
2. S 544. The Patient Safety and Quality Improvement Act of2005
3. United Kingdom National Patient Safety Agency Seven steps to patient safety—your guide to safer patient care. Available at: http://www.npsa.nhs.uk/health/resources/7steps (accessed 31 Mar 2007)
4. Runciman W B. Iatrogenic injury in Australia. A report by the Australian Patient Safety Foundation for the National Health Priorities and Quality Branch. Canberra, 2000. Available at: http://www.apsf.net.au/dbfiles/Iatrogenic_Injury.pdf (accessed 31 Mar 2007)
5. Barach P, Small S D. Reporting and preventing medical mishaps: lessons from non‐medical near miss reporting systems. BMJ 2000. 320759–763.763 [PMC free article] [PubMed]
6. Seckler‐Walker J, Taylor‐Adams S. Clinical incident reporting. In: Vincent C, ed. Clinical risk management: enhancing patient safety. 2nd edn. London: BMJ Books, 2001. 419–438.438
7. Shaw R, Drever F, Hughes H. et al Adverse events and near miss reporting in the NHS. Qual Saf Health Care 2005. 14279–283.283 [PMC free article] [PubMed]
8. Wald H, Shojania K G. Incident reporting. In: Shojania KG, Duncan BW, McDonald KM, Wachter RM, eds. Making health care safer: a critical analysis of patient safety practices. Rockville, MD: Agency for Healthcare Research and Quality, 2001
9. Thompson D A, Lubomski L, Holzmueller C. et al Integrating the intensive care unit safety reporting system with existing incident reporting systems. Jt Comm J Qual Patient Saf 2005. 31585–593.593 [PubMed]
10. Dunn D. Incident reports—their purpose and scope. AORN J. 2003;78: 46, 49–61, 65–6,
11. Sullivan G H. Incident reports are a must. RN 2003. 6671–74.74 [PubMed]
12. Flanagan J C. The critical incident technique. Psychol Bull 1954. 51327–358.358 [PubMed]
13. Brennan T A, Localio A R, Leape L L. et al Identification of adverse events occurring during hospitalization. A cross‐sectional study of litigation, quality assurance, and medical records at two teaching hospitals. Ann Intern Med 1990. 112221–226.226 [PubMed]
14. Flynn E A, Barker K N, Pepper G A. et al Comparison of methods for detecting medication errors in 36 hospitals and skilled‐nursing facilities. Am J Health Syst Pharm 2002. 59436–446.446 [PubMed]
15. Murff H J, Patel V L, Hripcsak G. et al Detecting adverse events for patient safety research: a review of current methodologies. J Biomed Inform 2003. 36131–143.143 [PubMed]
16. Battles J B, Lilford R J. Organizing patient safety research to identify risks and hazards. Qual Saf Health Care 2003. 12(Suppl 2)ii2–ii7.ii7 [PMC free article] [PubMed]
17. Vincent C A, Neale G, Woloshynowych M. Adverse events in British hospitals: a preliminary retrospective review. BMJ 2001. 322517–519.519 [PMC free article] [PubMed]
18. Wilson R M, Runciman W B, Gibberd R W. et al The quality in Australian health care study. Med J Aust 1995. 163458–471.471 [PubMed]
19. Leape L L, Brennan T A, Laird N. et al The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991. 324377–384.384 [PubMed]
20. Thomas E J, Studdert D M, Burstin H R. et al Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000. 38261–271.271 [PubMed]
21. Longo D R, Hewett J E, Ge B. et al The long road to patient safety: a status report on patient safety systems. JAMA 2005. 2942858–2865.2865 [PubMed]
22. Harvard Medical Practice Study Patients, doctors and lawyers: medical injury, malpractice litigation and patient compensation in New York. The Report of the Harvard Medical Practice Study to the State of New York, New York, NY: Department of Health 1990
23. Dubois R W, Brook R H. Preventable deaths: who, how often, and why? Ann Intern Med 1988. 109582–589.589 [PubMed]
24. Brennan T A, Leape L L, Laird N M. et al Harvard Medical Practice Study I. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991. 324370–376.376 [PubMed]
25. Milch C E, Salem D N, Pauker S G. et al Voluntary electronic reporting of medical errors and adverse events. An analysis of 92,547 reports from 26 acute care hospitals. J Gen Intern Med 2006. 21165–170.170 [PMC free article] [PubMed]
26. Thomas E J, Studdert D M, Newhouse J P. et al Costs of medical injuries in Utah and Colorado. Inquiry 1999. 36255–264.264 [PubMed]
27. Pierluissi E, Fischer M A, Campbell A R. et al Discussion of medical errors in morbidity and mortality conferences. JAMA 2003. 2902838–2842.2842 [PubMed]
28. Spaeth R G, Pickering K C, Webb S M. Quality assurance and hospital structure: how the physician–hospital relationship affects quality measures. Ann Health Law 2003. 12235–247.247 [PubMed]
29. Liang B A, Coulson K M. Legal issues in performing patient safety work. Nurs Econ 2002. 20118–125.125 [PubMed]
30. Murphy E K. Incident reports may or may not be privileged information. AORN J 1990. 51851–2, 854.2, 854 [PubMed]
31. Gostin L. A public health approach to reducing error: medical malpractice as a barrier. JAMA 2000. 2831742–1743.1743 [PubMed]
32. Uribe C L, Schweikhart S B, Pathak D S. et al Perceived barriers to medical‐error reporting: an exploratory investigation. J Healthc Manag 2002. 47263–279.279 [PubMed]
33. Jeffe D B, Dunagan W C, Garbutt J. et al Using focus groups to understand physicians' and nurses' perspectives on error reporting in hospitals. Jt Comm J Qual Saf 2004. 30471–479.479 [PubMed]
34. Wild D, Bradley E H. The gap between nurses and residents in a community hospital's error‐reporting system. Jt Comm J Qual Patient Saf 2005. 3113–20.20 [PubMed]
35. Landis J R, Koch G G. The measurement of observer agreement for categorical data. Biometrics 1977. 33159–174.174 [PubMed]
36. Localio A R, Weaver S L, Landis J R. et al Identifying adverse events caused by medical care: degree of physician agreement in a retrospective chart review. Ann Intern Med 1996. 125457–464.464 [PubMed]
37. Brennan T A, Localio R J, Laird N L. Reliability and validity of judgments concerning adverse events suffered by hospitalized patients. Med Care 1989. 271148–1158.1158 [PubMed]
38. Thomas E J, Lipsitz S R, Studdert D M. et al The reliability of medical record review for estimating adverse event rates. Ann Intern Med 2002. 136 pp 812-6 Erratum in: Ann Intern Med 2002. 137147. [PubMed]

Articles from Quality & Safety in Health Care are provided here courtesy of BMJ Group