PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Patient Saf. Author manuscript; available in PMC 2013 June 9.
Published in final edited form as:
PMCID: PMC3677059
NIHMSID: NIHMS469734

Creating an Oversight Infrastructure for Electronic Health Record-Related Patient Safety Hazards

Hardeep Singh, MD, MPH,1 David C. Classen, MD, MS,2 and Dean F. Sittig, PhD3

Abstract

Electronic health records (EHRs) have potential quality and safety benefits. However, reports of EHR-related safety hazards are now emerging. The Office of the National Coordinator (ONC) for Health Information Technology (HIT) recently sponsored an Institute of Medicine committee to evaluate how HIT use affects patient safety. In this paper, we propose the creation of a national EHR oversight program to provide dedicated surveillance of EHR-related safety hazards and to promote learning from identified errors, close calls, and adverse events. The program calls for data gathering, investigation/analysis and regulatory components. The first two functions will depend on institution-level EHR safety committees that will investigate all known EHR-related adverse events and near-misses and report them nationally using standardized methods. These committees should also perform routine safety self-assessments to proactively identify new risks. Nationally, we propose the long-term creation of a centralized, non-partisan board with an appropriate legal and regulatory infrastructure to ensure the safety of EHRs. We discuss the rationale of the proposed oversight program and its potential organizational components and functions. These include mechanisms for robust data collection and analyses of all safety concerns using multiple methods that extend beyond reporting; multidisciplinary investigation of selected high-risk safety events; and enhanced coordination with other national agencies in order to facilitate broad dissemination of hazards information. Implementation of this proposed infrastructure can facilitate identification of EHR-related adverse events and errors and potentially create a safer and more effective EHR-based health care delivery system.

Introduction

Recent passage of the American Reinvestment and Recovery Act (ARRA) incentivizes health care providers and organizations to implement electronic health records (EHRs) at an unprecedented pace to meet reimbursement timelines. EHR implementation is difficult, costly, time-consuming, and might lead to unintended consequences.1 Post-implementation evaluation often reveals that EHR implementations do not meet minimum safety guidelines,2-4 a concern that is even more pressing now. The aggressive timeline proposed in the ARRA bill does not allow adequate customization of EHR systems to local workflows.5,6 Furthermore, clinicians increasingly share control of complex processes with computers; in some instances, they assume a higher-level oversight role and allow computers to make routine decisions and carry out appropriate actions (e.g., the computer automatically generates a lab order when certain medications are being ordered).7,8 As more advanced clinical decision support (CDS) is embedded into existing EHRs, clinicians and their patients are increasingly reliant upon decisions generated by these systems.9-11 The increasing scope and complexity of tasks that clinicians can perform using EHRs, combined with unprecedented pressure to rapidly adopt these systems, can create a potentially hazardous environment for patient safety.

Reports on EHR-related hazards are now emerging.12-22 Koppel et al. identified 22 types of errors in the computerized provider entry system within their EHR.12 Many EHR-related hazards occur at the “blunt end” of the healthcare system,23 with potential to affect large numbers of patients if not corrected. For instance, we identified a single software configuration error in the EHR that resulted in a lack of timely notification of abnormal test results to several providers, thus affecting a large number of patients.15 In light of these types of reports, the Office of the National Coordinator for Health Information Technology (ONC) recently sponsored an Institute of Medicine committee to “review the available evidence and the experience from the field on how the use of health information technology (HIT) affects the safety of patient care.”24 Two national databases have also been recently created to facilitate reporting of EHR-related incidents.25,26 These early EHR hazard reporting initiatives, though important, are insufficient by themselves to address the multitude of complex EHR- related safety concerns.21,27,27 Furthermore, EHR certification by the ONC Authorized Testing and Certification Bodies (ONC– ATCBs)28 does not guarantee that EHRs will actually be implemented and work as planned, therefore, ongoing system evaluations and modifications are necessary. At present, it is unclear which single agency is responsible for EHR oversight. Thus, we believe it necessary to establish an independent organized national infrastructure to actively monitor and improve the safety of EHR systems. In this paper, we propose the creation of a national EHR oversight program to provide dedicated surveillance of EHR-related safety events and to promote learning from identified hazards, close calls, and adverse events. We provide an overview of a proposed program and its rationale and discuss its potential organizational components and functions. Although our recommendations might not cover all aspects of EHR-related hazards, they are a starting point to stimulate the much needed discussion and debate in this area.

Organizational Infrastructure of a National EHR Safety Program

Because EHR-related safety issues are an emerging area of knowledge, an EHR safety oversight program should involve robust data gathering and data analysis components. Both mechanisms should be overseen by a new independent board specifically charged with ensuring safety of EHRs nationally. Many useful lessons have been learned from the success of the National Transportation Safety Board (NTSB),29 which could provide a model for a Congressionally-funded independent EHR safety board. Data gathering activities would necessitate an infrastructure for collecting adverse events and near-misses from various sources, whereas the analytic component would be charged with analyzing the collected information and developing and disseminating preventive strategies on a national scale. The national program should be supported by close collaboration with institutional-level safety initiatives, such as EHR safety committees.30 This organizational scheme is reminiscent of the unified and cohesive safety program used by the Department of Veterans Affairs National Center for Patient Safety (NCPS), which is charged to lead the VA’s patient safety improvement initiatives.31

At the institutional level, multidisciplinary EHR safety committees, including a designated EHR patient safety officer, could perform two essential functions: 1) investigate all known EHR-related adverse events and near-misses32 and report them to the national board using standardized methods;33 and 2) perform routine safety self-assessments, including tests of all EHR components and applications and “prospective risk assessments” to proactively identify new risks.34,35 Because providers in smaller practices might not have resources for these functions, their respective local health information exchanges,36 independent physicians’ associations,37 regional extension centers,38 quality improvement organizations,39 or accountable care organizations could house the needed technical resources. Many institutions and practices also have existing legal and risk management infrastructure that can be leveraged to perform these functions. These locally housed investigational and risk assessment initiatives will likely reveal many site-specific, contextual issues that affect EHR safety and these issues could then be addressed at the institutional level without the need for national interference.

At the national level, the proposed board would be charged with analyzing event reports from institutions and investigating major EHR-related incidents (such as those associated with harm to a large number of patients). Aggregate analysis of reports could be used to identify common unsafe conditions for specific EHRs and to inform widely released recommendations to mitigate risks. The board would have both investigative and regulatory components and functions. Potential other roles for the board may include the development of newer error surveillance methods,40 validation and oversight of EHR safety self-assessment procedures and on-site EHR safety inspections (perhaps in collaboration with hospital certification/accreditation organizations), and dissemination of safety guidelines41 and benchmarks. The board would work closely with EHR certifying organizations42 (and thus indirectly with EHR vendors) to improve EHR design and implementation and with other governmental agencies (such as National Institute of Standards and Technology [NIST] and ONC) to coordinate EHR-related rules and regulations. Some of the key functions of such an oversight program are described in the sections below.

Data Gathering Operations of the Program

Voluntary reporting could continue to be an important source of data about EHR patient safety risks.18 However, existing FDA databases for medical device errors43-46 appear to be seldom used for reporting EHR-related incidents47 and a multi-pronged approach to improve reporting is needed. First and foremost, vendor contracts need to be free of non-disclosure or gag clauses 48,49 in order to encourage users and institutions to report errors and adverse events. Second, at the institutional level, standardized models of data collection50 should be used (e.g., through systems such as the HIT Hazards Reporting Model [Hazard Tracker]).51 Third, error reporting initiatives should be integrated nationally. For instance, medical professional insurance carriers and the not-for-profit iHealth Alliance52 recently created PDR Secure™, a website that allows users to report EHR-related safety issues. Patient safety organizations are also working to develop a common format for reporting EHR problems.33 Additional safety events should be aggressively solicited from EHR-related sentinel events reported to Joint Commission, reports to the vendors, and reports from users, the media, and EHR certifying organizations. Lastly, “major EHR adverse events” need to be defined and their reporting should be mandated.

Although we emphasize reporting for initial discovery, we recognize that it is limited and often neglects minor or latent errors and near-misses; these minor incidents might not be directly implicated in an adverse event but are rich sources of information to prevent future events.29 Therefore, we propose development of additional methods to collect data.53 One possible example is automated reporting (such as that of software programs that prompt the user to automatically report an error to the developer when a software glitch is detected). Another area for development is non-voluntary surveillance approaches that rely on electronic triggers to initiate error reporting.54 For example, Koppel et al. found that orders entered by a physician and subsequently cancelled within 45 minutes were likely errors.54 We are actively investigating the creation of algorithms that could be run against a wide variety of EHR implementations to establish rates of errors and of other higher-risk scenarios (e.g., counting the number of duplicate patient records in the database or the percentage of orders entered via free text rather than via the structured data entry fields55). Methods to query large electronic repositories for safety events in near “real time” should also be developed. For example, through the newly created Sentinel System, the FDA can now query electronic health information of more than 60 million people to monitor the safety of approved medical products.56

Investigational and Analytic Roles of the Program

A major adverse event related to an EHR could affect thousands of patients if not corrected rapidly. For example, in 2006 the UK National Health Service was forced to notify over 900 clinicians that their patients who were prescribed Zyban may have mistakenly been given Viagra due to an error in the dispensing pharmacy’s medication mapping table.57 Although there was no reported harm, future similar events might not be harmless. We propose that the board charged with EHR oversight should create a special investigation team dedicated to “major EHR adverse events”. Whereas most adverse event investigations could be conducted at the institutional level and reported appropriately, events with particularly broad or catastrophic impact should be investigated by the team at the national board. This strategy will ensure rapid action and wide dissemination of any significant findings. Pre-defined criteria for triggering this level of investigation could specify, for instance, the number of patients at risk for harm (or harmed) in a single incident. For example, criteria for investigation of an unplanned EHR system downtime that adversely affects patient care could include combinations of parameters such as the following:58

  1. Lasts for more than 24 hours
  2. Harms or has potential to harm more than 10058 patients59
  3. Is not the direct result of a natural disaster60
  4. Occurs in organizations that have implemented the following components of an EHR: admission/discharge/transfer; clinical results review; provider order entry, communication, verification; barcode medication verification; picture archiving and communication; clinical documentation; alert notification; and participation in local health information exchange61,62
  5. Simultaneously affects at least two of the EHR components identified above

A dedicated multidisciplinary team of core investigators with expertise in safety, informatics, human factors, computer science, and clinical medicine should conduct these major EHR adverse event investigations. The team could also leverage the expertise of an external group of pre-screened, independent consultants composed of clinicians, statisticians, informaticians, lawyers, and human factors engineers who work on an as-needed basis. All board investigators should be given legal authority to examine all records confidentially (including system logs and patient records), to access all relevant computer systems, and to interview all personnel or patients whom they believe are essential to their investigation. If needed, error scenarios should be recreated with test patients in simulated settings using a full range of human factors engineering-based techniques for analysis.63 Rapid dissemination of identified issues and corrective actions may resemble the processes used by the Joint Commission and VA NCPS64 to issue advisories. The findings of these investigations should also be publically reported so that other vendors, healthcare organizations, and users can learn from them. To prioritize its efforts, the board could create an annual “Most Wanted List” of improvements, similar to that issued by NTSB65. Research and evaluation should also be a key component of this learning process. For instance, centralized, de-identified databases could be created to enable researchers to advance the science and evidence of EHR safety. Because many EHRs share common features and will be used nearly universally, this centralized and standardized approach to high-stakes investigations is likely be beneficial.

Regulatory Authority

In addition to providing a safety event reporting clearinghouse and a mechanism for major adverse event investigations, the new national EHR safety board must be charged with appropriate regulatory and legal authority to carry out effective oversight. No government agency is currently fully equipped to perform this function. Because of the increasing complexity and coupling of often unrelated HIT-enabled systems (e.g., from different vendors and/or organizations), the oversight board should coordinate with other agencies (such as ONC and NIST) and industrial trade associations (e.g., the Electronic Health Record Association62) in a cooperative fashion to investigate the causes and potential mitigating solutions to the problem(s) identified. This collaboration could be modeled after the aviation industry oversight provided by the Commercial Aviation Safety Team (CAST).66 Another important aspect of EHR safety oversight will be close collaboration between various state and federal agencies responsible for making and enforcing EHR-related rules, regulations, and certification standards. For example, the new “meaningful use” statute requires e-prescribing in the outpatient setting, but the federal government only recently modified laws restricting the electronic transfer of prescriptions for controlled substances.67

To ensure safety and compliance in certain high-risk areas, we recommend strategies to monitor uptake of board recommendations that are of critical importance. One way to do this will be to implement unannounced, randomly scheduled, on-site EHR safety inspections, much like the Line Operations Safety Audit (LOSA) used in aviation.68 These audits would involve interviews with key stakeholders (e.g., Chief Information Officer, Patient Safety Officer, etc.), observations of important clinical operations, and inspection of the human-computer interface as configured by the organization.69 Inspectors should be equipped with comprehensive EHR safety inspection guides consisting of a set of “red flags” (e.g., lack of evidence of computer system backups, outdated clinical decision support logic or no evidence of CDS testing) and “best practices” (e.g., pre-printed order and documentation forms for use when the EHR is unavailable, strict policies to address breaches of patient confidentiality and a robust clinician EHR training program) to help guide interviews and observations related to the respective high-risk area. All inspectors should be certified and pre-screened for financial and other conflicts of interest to ensure their impartiality. Inspectors might also build an audit approach modeled on the Leapfrog CPOE Flight Simulator.70 This simulator evaluates the decision support functionality of implemented EHR systems to measure hospital compliance with the National Quality Forum Safe Practice for CPOE.71

Although the approach should initially be non-punitive, eventually the oversight board should have authority to invoke penalties if needed to protect patients. Furthermore, penalties for responsible stakeholders for non-compliance with certain pre-set expectations (such as fines for non-reporting of major EHR adverse events, not fixing a major software bug) could only be ensured using this approach.

Next Steps

The proposal we present herein is ambitious, but a multifaceted and centralized approach to address EHR-related safety is perhaps the best strategy. However, this approach is challenging and any attempt to implement such an important oversight function should proceed using a carefully orchestrated, staged implementation pathway. Given the current state of affairs, some might argue that the scope and cost of the national oversight program might be beyond what is possible now. To jumpstart the creation of this program, we propose that local, institutional-level initiatives to collect and analyze data must be bolstered immediately. This would help characterize the various types and frequencies of EHR-related errors and adverse events. These initiatives must be synergistic with rigorous research to improve the “basic science” of EHR-related safety and simultaneously inform the creation of the national oversight program. Taking this approach would ensure that there is adequate strength of the evidence to justify the scope and cost of implementation of the independent national board, which clearly will take longer to get established. While it might be premature for us to lay out the precise implementation strategy of the entire proposed program, we propose two immediate next steps to advance this agenda:

  1. Establish a standardized reporting infrastructure to facilitate event reporting and investigation of EHR-related safety concerns. The infrastructure should specifically protect reporters and maintain incentives to report. One successful model for this is the Aviation Safety Reporting System (ASRS), a confidential reporting system in which all reports are de-identified before being entered into the incident database.72 In qualified cases, fines and penalties are waived for unintentional violations of federal aviation statutes and regulations which are reported to ASRS.
  2. Bring together a group of experts and federal stakeholders to explore and define investigational and analytic roles of a national oversight program. This group would need to tackle the following tasks in order to facilitate the creation of a national program:
    1. Define criteria for triggering an investigation and outline a methodology for ensuring timely action and wide dissemination of significant findings. They will also need to develop the investigation methodology and inspection guides in order to facilitate the process.
    2. Help establish the legal and regulatory infrastructure to create the new board and facilitate its work processes. Key issues they will need to address include how to interface with complex technical and organizational governance structures involved with EHR implementation and use at the institutional level, the technical feasibility of gaining access to and examining system logs ‘on demand’ during investigations and unannounced inspections, and maintaining the confidentiality of the discovery process.
    3. Explore how the board would establish rules and regulations that EHR vendors will need to follow in order to comply with the oversight program; none currently exist in the Office of the National Coordinator for Health Information Technology’s Authorized Testing and Certification Body (ONC-ATCB) process.73

Conclusion

Technological advances give rise to increasingly complex and multifaceted errors in healthcare. EHR-related errors must be conceptualized, analyzed, and mitigated using a robust oversight infrastructure. We propose the creation of a national oversight program which relies on local institution-level EHR safety data collection and analysis, and ultimately leads to the formation of a centralized, non-partisan, multi-disciplinary board specifically charged with ensuring EHR safety. If implemented, the proposed infrastructure could help to identify and reduce EHR-related adverse events and errors and create a safer and more effective EHR-based healthcare delivery system.

Acknowledgments

Dr. Singh is supported by an NIH K23 career development award (K23CA125585), the VA National Center of Patient Safety, Agency for Health Care Research and Quality, a SHARP contract from the Office of the National Coordinator for Health Information Technology (ONC #10510592), and in part by the Houston VA HSR&D Center of Excellence (HFP90-020). Dr. Sittig is supported in part by a grant from the National Library of Medicine R01-LM006942 and by a SHARP contract from the Office of the National Coordinator for Health Information Technology (ONC #10510592). These sources had no role in the preparation, review, or approval of the manuscript.

We thank Annie Bradford, PhD, for assistance with medical editing.

Footnotes

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or any of the other funding agencies. Dr. Classen is an employee of CSC, a technology services company.

Reference List

(1) Sittig D, Ash J. Clinical information Systems: Overcoming adverse consequences. Jones and Bartlett Publishers, LLC; Sudbury, MA: 2009.
(2) Sittig DF, Singh H. Eight Rights of Safe Electronic Health Record Use. JAMA. 2009;302:1111–1113. [PubMed]
(3) Kilbridge PM, Classen DC. The informatics opportunities at the intersection of patient safety and clinical informatics. J Am Med Inform Assoc. 2008;15:397–407. [PMC free article] [PubMed]
(4) Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood ) 2010;29:655–663. [PubMed]
(5) Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM. Lessons from “Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system” Pediatrics. 2006;118:797–801. [PubMed]
(6) Blumenthal D, Tavenner M. The “Meaningful Use” Regulation for Electronic Health Records. New England Journal of Medicine. 2010;363:501–504. [PubMed]
(7) Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997;4:364–375. [PMC free article] [PubMed]
(8) East TD, Wallace CJ, Morris AH, Gardner RM, Westenskow DR. Computers in critical care. Crit Care Nurs Clin North Am. 1995;7:203–217. [PubMed]
(9) Campbell EM, Sittig DF, Guappone KP, Dykstra RH, Ash JS. Overdependence on technology: an unintended adverse consequence of computerized provider order entry. AMIA Annu Symp Proc. 2007;94-98 [PMC free article] [PubMed]
(10) McKinley BA, Moore LJ, Sucher JF, et al. Computer protocol facilitates evidence-based care of sepsis in the surgical intensive care unit. J Trauma. 2011;70:1153–1166. [PubMed]
(11) Morris AH, Orme J, Jr., Truwit JD, et al. A replicable method for blood glucose control in critically Ill patients. Crit Care Med. 2008;36:1787–1795. [PubMed]
(12) Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197–1203. [PubMed]
(13) Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a medication dosing error related to CPOE. Journal Of The American Medical Informatics Association: JAMIA. 2005;12:377–382. [PMC free article] [PubMed]
(14) McDonald CJ. Computerization can create safety hazards: a bar-coding near miss. Ann Intern Med. 2006;144:510–516. [PubMed]
(15) Singh H, Wilson L, Petersen L, et al. Improving follow-up of abnormal cancer screens using electronic health records: trust but verify test result communication. BMC Medical Informatics and Decision Making. 2009;9 [PMC free article] [PubMed]
(16) Nerich V, Limat S, Demarchi M, et al. Computerized physician order entry of injectable antineoplastic drugs: an epidemiologic study of prescribing medication errors. Int J Med Inform. 2010;79:699–706. [PubMed]
(17) Schulte F, Schwartz E. As Doctors Shift to Electronic Health Systems, Signs of Harm Emerge. The Huffington Post. 2010
(18) Magrabi F, Ong MS, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc. 2010;17:663–670. [PMC free article] [PubMed]
(19) Magrabi F, Ong MS, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Inform Assoc. 2011 [PMC free article] [PubMed]
(20) Harrington L, Kennerly D, Johnson C. Safety issues related to the electronic medical record (EMR): synthesis of the literature from the last decade, 2000-2009. J Healthc Manag. 2011;56:31–43. [PubMed]
(21) Sittig DF, Singh H. Defining Health Information Technology-Related Errors: New Developments Since To Err Is Human. Arch Intern Med. 2011;171:1281–1284. [PMC free article] [PubMed]
(22) Leviss J. H.i.t. Or Miss: Lessons Learned from Health Information Technology Implementation. American Health Information Management Association; 2010.
(23) Reason J. Human error: models and management. BMJ. 2000;320:768–770. [PMC free article] [PubMed]
(24) Institute of Medicine [3-2-2011. 5-23-2011];Activity - Patient Safety and Health Information Technology. http://www.iom.edu/Activities/Quality/PatientSafetyHIT.aspx.
(25) PDR Secure [2010. 5-13-2011];EHR Safety Event Reporting Service. http://ehrevent.org/
(26) Mosquera Mary. [9-20-2011. 10-7-2011];AHRQ tests tool identify, report health IT hazards. http://govhealthit.com/news/ahrq-tests-tool-identify-and-report-health-it-hazards.
(27) Koppel R. Monitoring and evaluating the use of electronic health records. JAMA. 2010;303:1918–1919. [PubMed]
(29) Sittig DF, Classen DC. Safe electronic health record use requires a comprehensive monitoring and evaluation framework. JAMA. 2010;303:450–451. [PMC free article] [PubMed]
(30) Miller RA, Gardner RM. Recommendations for responsible monitoring and regulation of clinical software systems. American Medical Informatics Association, Computer-based Patient Record Institute, Medical Library Association, Association of Academic Health Science Libraries, American Health Information Management Association, American Nurses Association. J Am Med Inform Assoc. 1997;4:442–457. [PMC free article] [PubMed]
(31) VA National Center for Patient Safety [6-1-2011. 10-7-2011]; http://www.patientsafety.gov/
(32) Yackel TR, Embi PJ. Unintended errors with EHR-based result management: a case series. J Am Med Inform Assoc. 2010;17:104–107. [PMC free article] [PubMed]
(33) Agency for Healthcare Research and Quality Common Formats for Patient Safety Data Collection and Event Reporting. Federal Register. 2010;75:65359–65360.
(34) Bonnabry P, spont-Gros C, Grauser D, et al. A risk analysis method to evaluate the impact of a computerized provider order entry system on patient safety. J Am Med Inform Assoc. 2008;15:453–460. [PMC free article] [PubMed]
(35) DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using health care Failure Mode and Effect Analysis: the VA National Center for Patient Safety’s prospective risk analysis system. Jt Comm J Qual Improv. 2002;28:248–67. 209. [PubMed]
(36) Sittig DF, Joe JC. Toward a statewide health information technology center (abbreviated version) South Med J. 2010;103:1111–1114. [PubMed]
(37) Ash JS, Sittig DF, Wright A, et al. Clinical decision support in small community practice settings: a case study. J Am Med Inform Assoc. 2011 [PMC free article] [PubMed]
(38) Maxson E, Jain S, Kendall M, Mostashari F, Blumenthal D. The regional extension center program: helping physicians meaningfully use health information technology. Ann Intern Med. 2010;153:666–670. [PubMed]
(39) Baier R, Gardner R, Gravenstein S, Besdine R. Partnering to improve hospital-physician office communication through implementing care transitions best practices. Med Health R I. 2011;94:178–182. [PubMed]
(40) Dunn AG, Ong MS, Westbrook JI, Magrabi F, Coiera E, Wobcke W. A simulation framework for mapping risks in clinical processes: the case of in-patient transfers. J Am Med Inform Assoc. 2011;18:259–266. [PMC free article] [PubMed]
(41) Nelson NC. Downtime procedures for a clinical information system: a critical issue. J Crit Care. 2007;22:45–50. [PubMed]
(42) Office of the National Coordinator for Health Information Technology. Department of Health and Human Services Establishment of the permanent certification program for health information technology. Final rule. Fed Regist. 2011;76:1261–1331. [PubMed]
(43) U.S Food and Drug Administration MedSun: Medical Product Safety Network. 2011 http://www.fda.gov/MedicalDevices/Safety/MedSunMedicalProductSafetyNetwork/defa ult.htm.
(44) U.S Food and Drug Administration [2011. 5-23-2011];Medsun Reports. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/medsun/searchReport.cfm.
(45) U.S Food and Drug Administration MAUDE - Manufacturer and User Facility Device Experience. 2011 http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/search.CFM.
(46) U.S Food and Drug Administration [2011. 5-23-2011];MDR Database Search. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmdr/search.cfm?searchoptions=1.
(47) Myers RB, jones SL, Sittig DF. Review of reported clinical information system adverse events in US Food and Drug Administration databases. Appl Clin Inf. 2011;2:63–74. [PMC free article] [PubMed]
(48) Goodman KW, Berner ES, Dente MA, et al. Challenges in ethics, safety, best practices, and oversight regarding HIT vendors, their customers, and patients: a report of an AMIA special task force. J Am Med Inform Assoc. 2011;18:77–81. [PMC free article] [PubMed]
(49) Koppel R, Kreda D. Health Care Information Technology Vendors’ “Hold Harmless” Clause. JAMA. 2009;301:1276–1278. [PubMed]
(50) PSO Privacy Center [4-14-2011. 10-7-2011];AHRQ Common Formats Device or Medical/Surgical Supply, including HIT. https://www.psoppc.org/web/patientsafety/ahrq-common-formats-device-or-medical/surgical-supply-including-hit-device.
(51) Walker JM, Carayon P, Leveson N, et al. EHR safety: the way forward to safe and effective systems. J Am Med Inform Assoc. 2008;15:272–277. [PMC free article] [PubMed]
(52) [2011. 5-23-2011];EHR Safety Event Reporting System. http://www.ehrevent.org/
(53) Classen DC, Resar R, Griffin F, et al. Global Trigger Tool Shows That Adverse Events In Hospitals May Be Ten Times Greater Than Previously Measured. Health Aff. 2011;30:581–589. [PubMed]
(54) Koppel R, Leonard CE, Localio AR, Cohen A, Auten R, Strom BL. Identifying and quantifying medication errors: evaluation of rapidly discontinued medication orders submitted to a computerized physician order entry system. J Am Med Inform Assoc. 2008;15:461–465. [PMC free article] [PubMed]
(55) Sittig DF, Campbell E, Guappone K, Dykstra R, Ash JS. Recommendations for monitoring and evaluation of in-patient Computer-based Provider Order Entry systems: results of a Delphi survey. AMIA Annu Symp Proc. 2007;671-675 [PMC free article] [PubMed]
(56) Behrman RE, Benner JS, Brown JS, McClellan M, Woodcock J, Platt R. Developing the Sentinel System--a national resource for evidence development. N Engl J Med. 2011;364:498–499. [PubMed]
(57) Lister David. [12-14-2006. 5-23-2011];The Latest Smoking Cure: Viagra. http://www.timesonline.co.uk/tol/news/uk/health/article753765.ece.
(58) Sittig DF, Classen DC. Monitoring and Evaluating the Use of Electronic Health Records—Reply. JAMA. 2010;303:1918–1919. [PubMed]
(59) Kilbridge P. Computer Crash – Lessons from a System Failure. New England Journal of Medicine. 2003;348:881–882. [PubMed]
(60) Travis J. Scientist’s fears come true as hurricane floods New Orleans. Science. 2005;309:1656–1659. [PubMed]
(61) Fischetti L, Mon D, Ritter R, Rowlands D. HL7 HER System Functional Model, Release 1. Health Level Seven ®, Inc.; Ann Arbor, Mich: 2007. Direct Care Functions.
(62) HIMSS Analytics [2009. 5-20-2011];U.S. EMR Adoption ModelSM Trends. http://www.himssanalytics.org/docs/HA_EMRAM_Overview_ENG.pdf.
(63) Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform. 2004;37:56–76. [PubMed]
(64) VA National Center for Patient Safety [5-20-2011. 5-23-2011];VHA Patient Safety Alerts and Advisories. http://www.patientsafety.gov/alerts.html.
(65) National Transportation Safety Board [2011. 10-7-2011];NTSB Most Wanted List. http://www.ntsb.gov/safety/mwl.html.
(66) Pronovost PJ, Goeschel CA, Olsen KL, et al. Reducing health care hazards: lessons from the commercial aviation safety team. Health Aff (Millwood ) 2009;28:w479–w489. [PubMed]
(67) Department of Justice. Drug Enforcement Administration [3-31-2010];Electronic Prescriptions for Controlled Substances; Final Rule. http://www.deadiversion.usdoj.gov/fed_regs/rules/2010/fr0331.pdf.
(68) The University of Texas Human Factors Research Project [2002. 6-13-2011];Line Operations Safety Audit and Threat and Error Management. http://homepage.psy.utexas.edu/homepage/group/helmreichlab/aviation/LOSA/LOSA.ht ml.
(69) Schumacher RM, Patterson EM, North R, Zhang J, Lowry SZ, Quinn MT, Ramaiah M, U.S.Department of Commerce. National Institute of Standards and Technology [2011. 10-7-2011];Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records, NISTIR 7804 Draft. http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf.
(70) Kilbridge PM, Welebob EM, Classen DC. Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems. Qual Saf Health Care. 2006;15:81–84. [PMC free article] [PubMed]
(71) National Quality Forum (NQF) Safe Practice 16: Safe Adoption of Computerized Prescriber Order Entry: 209-215 in Safe Practices for Better Healthcare - 2010 Update: A Consensus Report. National Quality Forum; Washington, DC: 2010.
(72) NASA Aviation Safety Reporting System: Confidentiality and Incentives to Report. 2011 http://asrs.arc.nasa.gov/overview/confidentiality.html.
(73) Department of Health and Human Services Health Information Technology: Initial Set of Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology, 45 CRF Parts 170. Federal Register. 2010;75:44590–44654. [PubMed]