PMCCPMCCPMCC

Search tips
Search criteria 

Advanced


 
Formats
Article sections
Authors
Related links
Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
 
J Gen Intern Med. 2009 January; 24(1): 21–26.
Published online 2008 November 20. doi:  10.1007/s11606-008-0857-9
PMCID: PMC2607519
Copyright © Society of General Internal Medicine 2008
Computerized Provider Order Entry Adoption: Implications for Clinical Workflow
Emily M. Campbell, R.N, M.S.,1 Kenneth P. Guappone, M.D., Ph.D.,1,2 Dean F. Sittig, Ph.D.,3 Richard H. Dykstra, M.D., M.S.,1 and Joan S. Ash, M.B.A., Ph.D.corresponding author1
1Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Mail Code: BICC, 3181 SW Sam Jackson Park Road, Portland, OR 97239-3098 USA
2Providence Portland Medical Center, Portland, OR USA
3UT-Memorial Hermann Center for Healthcare Quality and Safety, University of Texas School of Health Information Sciences, 7000 Fannin, Suite 600, 77030 Houston, TX USA
Joan S. Ash, Phone: +1-503-4944540, ash/at/ohsu.edu.
corresponding authorCorresponding author.
Received July 28, 2008; Revised October 9, 2008; Accepted October 16, 2008.
Small right arrow pointing to: This article has been cited by other articles in PMC.
ABSTRACT
OBJECTIVE
To identify and describe unintended adverse consequences related to clinical workflow when implementing or using computerized provider order entry (CPOE) systems.
METHODS
We analyzed qualitative data from field observations and formal interviews gathered over a three-year period at five hospitals in three organizations. Five multidisciplinary researchers worked together to identify themes related to the impacts of CPOE systems on clinical workflow.
RESULTS
CPOE systems can affect clinical work by 1) introducing or exposing human/computer interaction problems, 2) altering the pace, sequencing, and dynamics of clinical activities, 3) providing only partial support for the work activities of all types of clinical personnel, 4) reducing clinical situation awareness, and 5) poorly reflecting organizational policy and procedure.
CONCLUSIONS
As CPOE systems evolve, those involved must take care to mitigate the many unintended adverse effects these systems have on clinical workflow. Workflow issues resulting from CPOE can be mitigated by iteratively altering both clinical workflow and the CPOE system until a satisfactory fit is achieved.
KEY WORDS: attitude to computers, hospital information systems, user–computer interface, physician order entry
Health care providers use computerized provider order entry (CPOE) systems to place orders for medications, laboratory tests and other ancillary services.1 CPOE has been shown to decrease medication ordering errors and redundant test ordering, promote practice standardization, and reduce overall healthcare costs.2–4 Despite these benefits, CPOE systems have yet to be widely adopted for several reasons, including the high cost of implementation, clinician resistance to technology, worry regarding practice disruption and loss of productivity, fear of technology failure, and the inability of some CPOE implementations to integrate with existing healthcare systems.5–7 Furthermore, there is evidence that unintended adverse consequences can surround the implementation and ongoing maintenance of these systems.8–10 Recent, conflicting reports about the role of CPOE in the reduction of medication errors and associated costs have cast some doubt on the actual scale of improvements to be gained as CPOE systems have generated new kinds of medical errors, negatively affected patient outcomes, and resulted in higher overall medical costs for those institutions implementing them.9,11–13 Thus, there remains a need for ongoing analysis of CPOE to understand the causes of these issues and help find solutions.
A growing body of research explores the impact of integrating clinical information systems, including CPOE, within healthcare.1,14–17 Regardless of the study focus, one theme consistently emerges: embedding CPOE in healthcare fundamentally changes the way clinicians coordinate their work activities and collaborate to deliver care.18–21 Indeed, in our prior work we identified nine broad categories of unintended adverse consequences related to CPOE, negative impact on workflow emerged as the most frequently occurring theme.18,22 The purpose of this current study was to explore these workflow issues in greater detail.
BACKGROUND
We broadly define workflow as the activities, tools, and processes needed to produce or modify work, products, or services.23 More specifically, clinical workflow encompasses all of the 1) activities, 2) technologies, 3) environments, 4) people, and 5) organizations engaged in providing and promoting health care.
A sociotechnical evaluation framework 18,23–25 views these five components of clinical work as a single work system; that is, the components cannot be effectively analyzed in isolation. To understand the effects of embedding CPOE into existing care delivery systems, we must focus on how the systems as a whole responds to the change. When using this approach, one should not separate the information technology system from its implementation. Even exquisitely designed and coded software can be implemented poorly. Conversely, poorly engineered software can promote process improvements if it is well implemented. Thus any evaluation of a CPOE system must study the system as configured, implemented and used 26.
Site Selection
Over the past three years, we visited five hospitals in three different healthcare delivery organizations using CPOE. The sites were selected based on their reputations for excellence in the use of CPOE; all sites have fully operational and functional systems that capture a minimum of 90% of all medical orders, and have been operating at this level for several years. In addition, we wanted to evaluate different types of CPOE systems (e.g., commercially built as well as “home-grown”) in different types of hospitals. The research sites included Wishard Memorial Hospital in Indianapolis, IN, using the locally developed Regenstrief system, Brigham and Women’s and Massachusetts’ General hospitals in Boston, both using in-house developed systems, The Faulkner Hospital in Boston, MA, using MediTech (Westwood, MA), and Alamance Regional Medical System in Burlington, NC, using Eclipsys (Boca Raton, FL). Complete site details are presented in Table 1.
Table 1
Table 1
Description of sites studied
Data Collection
The Institutional Review Boards of Oregon Health & Science University as well as each of our study sites approved this study. The research team consisted of two physicians, a nurse, a pharmacist, a librarian, a public health researcher, and a technically-oriented informaticist.
Our fieldwork used two approaches to data collection. The first consisted of hour-long, semi-structured oral history interviews with hospital administrators, physicians, nurses, pharmacists, and others suggested to us by local principal investigators. Our team leader (JA) conducted these interviews to elicit historical and current perspectives on the unintended consequences, whether positive or negative, related to CPOE implementation at each of the institutions. Table 2 includes a list of questions that served as a starting point for these interviews. Interviewee responses generated additional probing questions which differed during each interview. Interviews were recorded and later transcribed. All interviewees were formally consented.
Table 2
Table 2
General Interview Guide for Semi-Structured Interviews
Our second approach to data collection involved shadowing clinicians and other personnel interacting with CPOE systems during their work. The clinicians shadowed were selected by prior arrangement with study collaborators so that skeptics as well as accepting users were included. Subjects who agreed to be shadowed were formally consented. Researchers then unobtrusively observed the subjects for periods from 2–6 hours at various times during the day and night. When clinicians offered comments about the systems, we noted them. In addition, when we were unclear about specific activities, we asked questions for clarification. In addition to shadowing individuals, we also observed general work activities in ambulatory care centers, hospital wards, emergency rooms, surgical recovery areas, and other critical care units.
Data were collected from August 2004 through April 2005. During this time we conducted 32 semi-structured oral history interviews totaling 43 hours. We performed over 400 hours of observation that included shadowing 95 clinical providers (40% medical residents and staff physicians, 30% nurses, 10% pharmacists, and 20% other clinical personnel or IT staff) using CPOE systems in diverse settings. All transcripts and field notes were analyzed using N6 software (N6, QSR International Pty. Ltd., Melbourne, Australia, 2002).
Data Analysis
Each project team member independently reviewed an assigned selection of transcribed field notes or interviews to identify unintended consequences. The entire project team then met 36 times to collectively determine which data represented unintended consequences, and how the data could be meaningfully categorized. We specifically focused on unintended adverse consequences because these need to be carefully managed. We ultimately identified 324 instances of these related to CPOE. We used a card sort method 22 to develop provisional categories for those consequences that appeared to relate to the same content. Once initial groupings were assigned, the team iteratively reviewed each item in each category using a grounded theory approach, to confirm commonality among elements and to allow themes to emerge from the data.22 Nine categories emerged and have been described in detail elsewhere.18 Following this initial categorization we performed in-depth analysis of the nine categories using axial coding techniques25 to better understand the properties and dimensions of each. As noted, the largest unintended consequence category was workflow, which is our focus here. Axial coding resulted in five themes within that category.
We discuss each of the five themes below and provide representative examples of each. Although we note that general themes existed in all settings we studied, the representative examples we use to elaborate the themes are site-specific.
CPOE Introduction Exposes Human–Computer Interaction Problems
We found that ergonomic issues can disrupt workflow. For example, some disruptions arise when environments designed prior to the computer era cannot adequately accommodate new hardware. Mobile computers have little flat space to accommodate paper charts. One physician noted: “A computer that doesn’t have a place to put the chart down is no computer I am willing to use.” In addition, when workstations are in short supply, contention for computers can be high in busy work areas, especially after morning rounds.
We noticed many issues related to poor CPOE usability. These included overly cluttered screen design, poor use of available screen space, and inconsistencies in screen design. More specifically, we saw lists that could not be easily sorted, screens that were hard to read or annotate, minimum availability of system defaults, and lack of appropriate safeguards to prevent selecting the wrong patient or entering incorrect data, to name just a few. Not all of these problems occurred at all sites, though all sites reported software design issues that made some work processes awkward. For example, a researcher observed this example of suboptimal design: “I notice that the resident has to perform four mouse clicks to access an element on a list: 1) click on the pick list, 2) open the list with the down arrow, 3) select an item from the list, and 4) hit the return key to exit the pick list. Normally, this wouldn’t be much of a problem, but the list only contains one element!”
CPOE Changes Work Pace, Sequence, and Dynamics
With some CPOE systems, providers find it difficult to access patient information housed in clinical systems that are not integrated with CPOE, require separate system logins, or cannot be accessed simultaneously. A physician explained: “For me to get lab values I would have to exit out of the discharge summary, [look up the lab values] then bring [the discharge summary] up again. It is just easier for me to look up values on a separate computer.”
In addition, CPOE can force the provider to accommodate the system. For example, many systems provide minimal space for free-text entry or limit the use of timesaving shorthand (such as abbreviations or acronyms) and instead require data entry using nested menus, order sets, and pre-configured pick lists. A resident noted “…the order sets are organized in a linear fashion, for one problem at a time…most of [my patient’s] problems are multidimensional…I have to fill out several different order sets…one for each problem.”
The CPOE systems we studied often do not smoothly handle transitions in level or location of care. For example, it is quite common for an admitting clinician to begin to write orders for an emergency department patient prior to transfer to an inpatient bed. Because some CPOE implementations associate orders with a patient’s physical location, the system may prevent the admitting clinician from entering these orders. “There is a major problem with confusion over whether it is the floor accepting the patient or the ER transferring. The difference is who is responsible."
CPOE systems can force rigid scheduling of tests and medications. Some systems assign medication start times when the order becomes active (as opposed to when the medication is given) making it difficult for staff to alter the timing to match reality. This may cause delays in medication administration. “One problem was that the start time in our system doesn’t mean the time the medication is first administered—it means the time the order becomes active and then the administration times are automatically calculated based on that. [A physician ordered] a Q 12 medication. The first scheduled administration time was about 11 hours later so the patient’s post-transplant medication was delayed 11 hours.” In addition some CPOE systems make it difficult for clinical staff to alter the timing of doses when they cannot be given on schedule, such as when patients are absent from the nursing unit when medications are due. Even in systems where medication dosage times can be changed, often the system cannot automatically reschedule subsequent doses after this modification, requiring staff to alter each of the remaining dosage times manually to match the new, corrected administration schedule.
CPOE Provides Only Partial Support for the Work Activities of Involved Clinical Staff
These systems do not fully support the activities of all clinical staff who must process orders entered in the system. Nurses were the most vocal of the non-physician groups: “This is not a nursing system… the nurses are just saying ’Give me a template nurses can use. Give me standard order sets I can sign off with a single review. Get the standard nursing orders into the doctor’s order templates, so we don’t have to remind them to write an order for something like drawing arterial blood gases every 8 hours unless the patient condition changes.’”
Non-physician staff found it bothersome to receive alerts not applicable to them or their clinical setting. For example, some drug–drug interaction alerts may be highly desirable in one context and not another. One intensive care nurse observed: "… the [alerts] warning against prescribing heparin and aspirin—these are CCU patients, the system should know we are going to give these two meds together on this floor and quit warning us about them." In addition, because nurses do not prescribe medications, this alert is targeted at the wrong clinician.
CPOE Reduces Situation Awareness
Dourish and Bellotti define situation awareness as “the understanding of the activities of others which provides a context for your own activity.”27 Collaboration understandably improves when people develop and maintain awareness of what is going on around them.28 We found that CPOE systems, because they allow orders to be entered at any time by providers located outside of the hospital, can contribute to loss of situation awareness. For example: “It was not at all unusual in the paper world to have two or three people generate orders very close to each other but the common thing they had was a paper or a sheet. In the emergency department [there] was literally a different workstation about every two inches down there. We had a lot more instances of within thirty seconds of each other, two, sometimes three providers would enter the same order at approximately the same time and so it really forced us to go back and do more education on being careful to look and see what [orders are] active before you enter a new order.”
Finally, interesting situation awareness issues emerge when providers from different clinical services use CPOE to enter orders simultaneously on the same patient. The orders might appear to conflict, when in fact they do not. “I was sitting there in the ICU looking at my patient and …boom, an order for dopamine shows up. I didn’t write that…and I look at it…and turned out that it was written by the anesthesiologist getting ready for the case tomorrow. So I was seeing all of the pre-op medicines…a good thing, right? Except it surprised me. I’d never seen those orders before, and [the patient] looked like he didn’t need dopamine to me, so I just cancelled the order.”
CPOE Can Highlight Ineffective Implementation of Policy and Procedures
CPOE systems help to formalize organizational policies and procedures.29 In many cases, actual practice does not match this rigid “letter of the law,” so the CPOE system may introduce a significant amount of extra work (perceived or real): “We found that [obstetrics] was one of the most complex places in the hospital because patients were going from the screening room to the pod room to a labor room to the delivery room to postpartum and each of those are a different level of care and so orders need to be rewritten. Although nurses are very good about blending the orders as need be from one [level] to the other, the computer isn’t nearly as flexible."
Difficulties arise when standards are hard to interpret or implement, as when one clinician initiates patient care that must be monitored by other specialists: "Some orders [are] written by certain specialists like anesthesiologists [for] epidurals. No one wants to rewrite those orders. So how should those [orders] traverse the levels of care when the epidural catheter moves with the patient?” In such cases, CPOE can complicate already difficult issues.
By observing and interviewing clinicians, we found that embedding CPOE systems in clinical practice can disrupt work processes in several general ways, regardless of the site studied. Specifically: 1) CPOE systems can expose new human–computer interaction problems and exacerbate space constraints, 2) CPOE can alter the pacing, sequencing, and dynamics of work patterns,3) Despite the fact that these systems are ostensibly provider systems, they remain, at least in the sites we studied, predominantly physician systems, such that the workflow needs of non-physician personnel are not yet being addressed, 4) CPOE can impact situation awareness for providers, so that clinicians cannot guarantee that they are acting on complete information at all times, and 5) CPOE can be leveraged to poorly implement organizational policies and procedures, creating extra work or slowing down current work processes for providers. We observed these types of workflow disruptions at all institutions we visited, regardless of the CPOE system in use. As a result, these general themes provide the following focus points for improvements in CPOE system design, implementation, and evaluation.
CPOE Introduction Exposes Human–Computer Interaction Problems
End-users struggle with many human–computer interface issues when moving from a pen and paper environment that is flexible and highly portable to an electronic system that is much more rigid and fixed. Poor system interface design (e.g., overly complex screens, inconsistencies in the interface, poor grouping of like terms, etc.) can exacerbate this transition. It is imperative that system engineers use proven usability design standards to avoid implementing systems that violate basic principles. We look forward to the development of explicit interface design and usability criteria that must be met in order to certify CPOE systems through such organizations as the Certification Commission for Healthcare Information Technology (http://www.cchit.org).
CPOE Changes Work Pace, Sequence, and Dynamics
Alterations in work pace, sequence, and dynamics represent changes that emerge primarily from the difficulties inherent in attempting to customize the non-linear, iterative, ad hoc, interruption and exception driven activities of clinical care.18,23 Computerization of ordering can dramatically affect the care delivery process, as patterns of communication, cooperation, and collaborative work must shift to accommodate the technology. It is not surprising that the National Health Policy forum reported that clinician productivity can drop approximately 20% within the first three months of CPOE implementation,6 though other studies29,30 have indicated that productivity often improves over time as users gain proficiency with the system. CPOE can be improved through development of interoperability with and access to other clinical information systems and research about how users circumvent the system to get their work done. We acknowledge that these systems are relatively new, and that as they mature, they should be able to handle more complex care scenarios. We encourage careful design that incorporates non-standard scenarios into the workflow mix.
CPOE Design Affects all Clinical Personnel
Healthcare delivery is a complex activity system requiring the expertise of various professionals whose respective skills are interrelated and inseparable. Indeed, this distribution of work adds to the robustness of the health care system. However, current CPOE systems do not always accommodate the work needs of all levels of clinical personnel. In fact, many CPOE systems seem to provide support for only the physician’s work activities. Although physicians bear the legal responsibility for ordering and have the expertise needed for the decision-making required, the entire ordering-to-completion process includes many different levels of healthcare personnel. For these reasons, it is paramount that the roles of nursing, clerical, pharmacy and other ancillary staff are considered when CPOE systems are designed, implemented, and modified if clinical workflow is to proceed with minimal disruption. This does not imply that all care activities for all personnel must be incorporated into CPOE; instead, those activities pertaining to order management must be considered.
CPOE Can Impact Situation Awareness
CPOE systems can vastly improve situation awareness through functionality that integrates with other systems and subsequently displays information derived from these differing sources in a single location. In addition, clinical decision support tools can alert clinicians to potential problems that might otherwise go unnoticed (e.g., drug–drug or drug–allergy interactions). Because CPOE can standardize practice (e.g., through order sets, codification of procedures, etc.), it can provide a level of consistency in practice that can enhance situation awareness, as users can "expect" certain CPOE behavior, and adapt to CPOE processes. However, CPOE can also contribute to a general loss of situation awareness as it can change the pattern, style and timing of provider interactions. A certain degree of iterative and interactive communication among the various players is essential to promote and support situation awareness in medical work and decision making. Such awareness is vitally important for effective performance in any complex and dynamic environment.31 Without careful design to facilitate multiple provider communication the computerization of the health record can serve to isolate users from each other, depriving each of the benefit of coworkers’ understanding and insights regarding the clinical situation.
CPOE Can Highlight Ineffective or Incomplete Implementation of Policy and Procedures
While CPOE can be a highly effective and efficient tool for implementing organizational policy or procedure, using CPOE in this manner can bias workflow design towards an organizational perspective, one that emphasizes an explicit view of work: "those things that are documented, visible, and articulable [sic],”20 such as procedures and methods. This view fails to acknowledge the more tacit aspects of work processes-those activities carried out in everyday practice, which rely on human ingenuity and depend on rules of thumb or individual judgment for synthesis and completion.20 As a result, rote implementation of policy or procedure can highlight pronounced differences between organizational intention and provider practice, leading to the adoption of system workarounds by clinicians struggling to use a system that does not fully support their work.26 Any implementation of organizational rules or directives should be undertaken only after careful assessment of the impact of such changes on actual clinical work, to determine whether or not these rules can be practically integrated into workflow. In addition, care must be taken to assure that work practices mandated in CPOE systems are actually formally required, as opposed to representing "the way things have always been done." CPOE can make it easy to implement organizational changes. It is thus imperative that organizational mandates implemented through CPOE are rigorously tested using real-time scenarios to assure that these requirements not only make practical sense, but do not negatively impact workflow.18
LIMITATIONS
In this study, we only observed users interacting with CPOE systems. It is possible we might find different unintended adverse consequences had we evaluated user interactions with other systems. As a qualitative study, this investigation produced rich, in depth knowledge about five carefully selected sites. It is possible that these sites are not truly representative of all sites using CPOE.
The introduction of CPOE into the healthcare environment has a dramatic effect on clinical workflow. CPOE systems are tools intended to support and improve the delivery of care, and are not solutions for all problems related to clinical practice. We must take care to continually improve these systems if they are to fit seamlessly into clinical workflow. As we identify how, when, and where workflow problems arise, we gain insight for better system design and implementation. As CPOE systems evolve, ongoing care must be taken to reduce or resolve the many unintended adverse effects these systems have on clinical workflow. The five kinds of unintended and unanticipated consequences related to workflow with CPOE can be mitigated by iteratively altering both clinical workflow and the CPOE system until a comfortable and optimized fit is achieved.
Acknowledgements
We would like to thank all the individuals who allowed us to observe or interview them, and the experts who participated in the Menucha Conference. Special thanks go to the site principal investigators J. Marc Overhage, M.D., Ph.D., Eric G. Poon, M.D., M.P.H., and Carol Hudson, R.N. This work was funded by research grant LM06942 and training grant ASMM10031 from the U.S. National Library of Medicine, National Institutes of Health.
Conflict of Interest The authors state no conflicts of interest regarding the research or publication of this manuscript.
1. Poon E, Blumenthal D, Jaggi T, Honour M, Bates D, Kaushal R. Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. hospitals. Health Aff. 2004;23(4): 184–90. [PubMed]
2. Committee on Quality Health Care in America. To Err is Human: Building a Safer Health System. Washington, DC: Institute of Medicine; 1999.
3. Committee on Quality Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: Institute of Medicine; 2001.
4. The Leapfrog Group. Factsheet: Computer physician order entry. 2004 [accessed August 29, 2008]; Available from: http://www.leapfroggroup.org/media/file/Leapfrog-Computer_Physician_Order_Entry_Fact_Sheet.pdf.
5. Oren E, Shaffer E, Guglielmo B. Impact of emerging technologies on medication errors and adverse drug events. Am J Health Syst Pharm. 2003;6014:1447–58. [PubMed]
6. Sprague L. Electronic health records: How close? How far to go. NHPF Issue Brief. 2004;800:1–17. [PubMed]
7. Wears R, Berg M. Computer technology and clinical work: still waiting for Godot. JAMA. 2005;293(10):1261–1263. [PubMed]
8. Ash J, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11(2):104–112. [PMC free article] [PubMed]
9. Koppel R, Metlay J, Cohen A, Abaluck B, Localio A, SE K. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:10:1197–1203. [PubMed]
10. Kremsdorf R. CPOE: not the first step toward patient safety. Health Manag Technol. 2005;26(1):66. [PubMed]
11. Berger R, Kichak J. Computerized physician order entry: helpful or harmful. J Am Med Inform Assoc. 2004;11(2):100–3. [PMC free article] [PubMed]
12. Han Y, Carcillo J, Venkataraman S, Clark R, Watson R, Nguyen T, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116(6):1506–12. [PubMed]
13. Wang J, Lee H, Huang F, Chang P, Sheu J. Unexpected mortality in pediatric patients with postoperative Hirschsprung’s disease. Pediatr Surg Int. 2004;20(7):525–8. [PubMed]
14. Aarts J, Berg M. A tale of two hospitals: a sociotechnical appraisal of the introduction of computerized physician order entry in two Dutch hospitals. 2004;11(Pt 2):. Stud Health Technol Inform. 2004;107(Pt 2):999–1002. [PubMed]
15. Ash J, Anderson J, Gorman P, Zielstorff R, Norcross N, Pettit J, et al. Managing change: analysis of a hypothetical case. J Am Med Inform Assoc. 2000;7(2)125–34. [PMC free article] [PubMed]
16. Lorenzi N, Smith J, Conner S, Campion T. The success factor profile for clinical computer innovation. Stud Health Technol Inform. 2004;107(Pt 2)1077–80. [PubMed]
17. Poon E, Blumenthal D, Jaggi T, Honour M, Bates D, Kaushal R. Overcoming the barriers to the implementing computerized physician order entry systems in US hospitals: perspectives from senior management. AMIA Annu Symp Proc. 2003:975. [PMC free article] [PubMed]
18. Campbell E, Sittig D, Ash J, Guappone K, Dykstra R. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13(5):547–56. Sep–Oct. [PMC free article] [PubMed]
19. Rinkus S, Walji M, Johnson-Throop K, Malin J, Turley J, Smith J, et al. Human-centered design of a distributed knowledge management system. J Biomed Inform. 2005;38(1)4–17. [PubMed]
20. Sachs P. Transforming work: Collaboration, learning, and design. Commun ACM. 1995;38(9):36–44.
21. Pratt W, Reddy M, McDonald D, Tarczy-Hornoch P, Gennari J. Incorporating ideas from computer-supported cooperative work. J Biomed Inform. 2004;37(2):28–37. [PubMed]
22. Lincoln Y, Guba E. Natualistic Inquiry. Newbury Park, CA: Sage Publications, Inc.; 1985.
23. Berg M. Medical work and the computer-based patient record: a sociological perspective. Methods Inf Med. 1998;37(3)294–301. [PubMed]
24. Berg M. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inform. 1999;55(2):87–101. [PubMed]
25. Crabtree B, Miller WL. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications, Inc; 1999.
26. Harrison M, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—An Interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14(5):542–9. [PMC free article] [PubMed]
27. Dourish P, Bellotti V. Awareness and coordination in shared work spaces. ACM Conference on Computer-Supported Cooperative Work (CWSW ’92). 1992:107–14.
28. Endsley M. Toward a theory of situation awareness in dynamic systems. Hum Factors. 1995;37(1)32–64.
29. Kuperman G, Gibson R. Computer physician order entry: Benefits, costs, and issues. Ann Intern Med. 2003;139(1):31–9. [PubMed]
30. Overhage J, Perkins S, Tierney W, McDonald C. Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory primary care internal medicine practices 2001;8(4):361–71. [PMC free article] [PubMed]
31. Hazlehurst B, McMullen C, Gorman P. Distributed cognition in the heart room: How situation awareness arises from coordinated communications during cardiac surgery. J Biomed Inform. 2007;40(5):539–51. [PubMed]
Articles from Journal of General Internal Medicine are provided here courtesy of
Society of General Internal Medicine