Controlled Trial of Direct Physician Order Entry: Effects on Physicians' Time Utilization in Ambulatory Primary Care Internal Medicine Practices

J Am Med Inform Assoc. 2001 Jul-Aug; 8(4): 361–371.

PMCID: PMC130081

Controlled Trial of Direct Physician Order Entry

Effects on Physicians' Time Utilization in Ambulatory Primary Care Internal Medicine Practices

J. Marc Overhage, MD, PhD, Susan Perkins, PhD, William M. Tierney, MD, and Clement J. McDonald, MD

Affiliation of the authors: Regenstrief Institute for Health Care, Indianapolis, Indiana.

Correspondence and reprints: J. Marc Overhage, MD, PhD, Regenstrief Institute for Health Care, 5th Floor, Regenstrief Health Center, 1050 Wishard Boulevard, Indianapolis, IN 46202; e-mail: <joverhag/at/iupui.edu>.

Received February 6, 2001; Accepted March 16, 2001.

This article has been cited by other articles in PMC.

Abstract

Objective: Direct physician order entry (POE) offers many potential benefits, but evidence suggests that POE requires substantially more time than traditional paper-based ordering methods. The Medical Gopher is a well-accepted system for direct POE that has been in use for more than 15 years. The authors hypothesized that physicians using the Gopher would not spend any more time writing orders than physicians using paper-based methods.

Design: A randomized controlled trial of POE using the Medical Gopher system in 11 primary care internal medicine practices.

Measurements: The authors collected detailed time use data using time motion studies of the physicians and surveyed their opinions about the POE system.

Results: The authors found that physicians using the Gopher spent 2.2 min more per patient overall, but when duplicative and administrative tasks were taken into account, physicians were found to have spent only 0.43 min more per patient. With experience, the order entry time fell by 3.73 min per patient. The survey revealed that the physicians believed that the system improved their patient care and wanted the Gopher to continue to be available in their practices.

Conclusions: Little extra time, if any, was required for physicians to use the POE system. With experience in its use, physicians may even save time while enjoying the many benefits of POE.

Thirty years ago, Morris Collen argued, “Physicians should enter medical orders directly into the computer.”¹ More recently, the Institute of Medicine affirmed this view by stating, “Ideally, data in the [computer-based patient record] should be entered at its source by the record's primary user,”² and employers have identified direct physician order entry (POE) as a key quality improvement strategy.³ Technical limitations and process barriers have prevented most organizations from achieving Dr. Collen's goal,⁴ and efforts to require physicians to enter orders, create problem lists, and enter progress notes have met with varying degrees of success.⁴^–⁷

One of the major barriers to POE system use is the additional physician time it requires, compared with traditional paper-based methods.⁸^–¹⁰ Any change that requires more physician time will meet with significant resistance from physicians—time appears to be the primary determinant of physician satisfaction with order entry.¹¹

The Medical Gopher system, which has been used for 15 years in primary care internal medicine practices associated with Wishard Memorial Hospital, is a mature, well-accepted POE system. Throughout its development, we have tuned dozens of aspects of the Gopher to make it fast and easy to use. Even so, in 1993, we observed that, in the inpatient setting, entering daily orders using the Gopher required twice as much time as entering handwritten orders. With improvements we have made since that time, we hypothesized that the time required to use the current version of the Gopher would be comparable with that for traditional paper-based methods. To test this hypothesis, we carried out a randomized controlled trial of physician time use for POE using the Gopher compared with paper-based methods.

Methods

Setting

The Institutional Review Board of Indiana University–Purdue University at Indianapolis approved the study. We conducted the study in practices that are part of Indiana University Medical Group–Primary Care (IUMG-PC) from October 1996 to February 1998. The IUMG-PC is a physician-led primary care group that includes internists, pediatricians, family practitioners and obstetrician/gynecologists who deliver care to patients in 13 practices at 12 office locations in central Indiana. The group is an affiliate of the Indiana University School of Medicine (IUSM) and Wishard Memorial Hospital, and all physicians are faculty members of the IUSM. The IUMG-PC includes two types of practices—“community health centers,” which provide care primarily for uninsured and underinsured patients in comprehensive settings that include social work, nutrition, laboratory, pharmacy, radiology, and other services; and “commercial” practices that serve primarily a commercially insured population in a typical private practice setting.

Prior to the study, all providers handwrote their patient notes on encounter forms that were generated specifically for each patient encounter by the Regenstrief Medical Record System (RMRS).¹²^,¹³ They wrote prescriptions either on standard prescription forms or on paper medication forms generated by RMRS and wrote test orders longhand in a designated area of the encounter form. Data entry personnel manually abstracted selected data, such as vital signs, and recorded them in the RMRS. Office personnel generally completed test requisitions. Physicians typically completed all documentation for an encounter before the patient left the practice. Clinicians in two of the practices (at a single location) have used various versions of the Medical Gopher system for POE since 1984.¹⁴ The RMRS provided a nearly complete electronic medical record for the physician in these practices, with access to diagnostic test results, treatment information, dictated clinician's notes, reports, and other data. A traditional paper chart was maintained and was available to the clinician at each patient encounter.

Intervention

We conducted a randomized, controlled trial of the Medical Gopher¹⁰^,¹⁴ system. The Gopher provides order entry, clinical documentation, diagnostic result review, access to clinical knowledge bases, and links to billing functions. After leaving the examination room, the provider enters these data on workstations in a shared workroom or in the provider's office, using a series of forms, including the one shown in Figure 1

. There were no computers in the examination rooms.

Figure 1

Representative form for ordering a chest radiograph, from the Gopher clinical workstation.

For each encounter, the provider records billing diagnoses, medications prescribed, changes in allergies, test orders, referrals, nursing intervention orders and, optionally, notes. The system stores this information in a clinical data repository (the RMRS) from which providers can view the information as coded data or as a note with a more traditional appearance. The Gopher then uses the data to create billing documents (superbill), requisitions, consult requests, prescriptions, and patient education materials. When entering orders, including prescriptions, physicians can create instructions by typing them in free text, choosing fill-in-the-blank templates, or using their previous instructions for an order. They commonly use the last method when refilling medications.

Study Design

In the two practice types (community health center and commercial site), we used a random number generator to choose, as intervention practices, those in which physicians had never used the Gopher. (We excluded the two IUMG-PC practices that have been using POE for many years.) The remainder served as controls. In the intervention practices, we implemented the Gopher system for POE. In each practice we phased-in the implementation, which required approximately 2 to 3 months for hardware installation, configuration, and user training. Consequently, Gopher implementation in the intervention practices spanned approximately 1 year. Because of a dramatic turnover of physicians and practice manager at one site, we did not succeed in implementing the Gopher there. Physicians received approximately 30 minutes of “just-in-time” training by a clinical information specialist at their practice when the Gopher was initially implemented.

Because physicians' work is very fragmented and the duration of time spent on a specific task is very short,¹⁵ we chose a time-motion study rather than work-sampling methods.¹⁰^,¹⁶ Observation periods were one half day. We selected this interval because physicians' schedules are structured in half-day blocks, and some physicians had other responsibilities, such as administrative work or inpatient responsibilities, during some half days. In addition, physicians would typically have 8 to 10 patients scheduled for a half-day block, which ensured a somewhat representative sample of patient care.

We invited all full-time internists in the intervention and control practices to participate. Residents and fellows were not included in the study population. We chose to focus on internists because of the complexity of their task environment and to eliminate specialty as a source of variability. We attempted to observe each provider on at least three different days over a 2-year period, to allow for the day-to-day variability in physician patient loads and other non-physician factors. On some occasions, because of scheduling limitations, we observed the same physician for two consecutive half-day periods. One investigator (J.M.O.) selected observation dates on the basis of the observers' and providers' schedules. The observers attempted to begin their observations before physicians saw their first patients during the half-day block and to continue observations until the physicians had finished care for their last patients. Office personnel explained the observer's role to patients while directing them to an examination room, to limit disruptions to the physician's routine; however, most physicians introduced the observer to the patient.

Observers

We recruited observers among senior medical students at the IUSM. We selected senior medical students as observers because they are familiar with clinical practice, which enables them to identify and categorize clinical activities with less training; they are comfortable with the sights, sounds, and processes encountered in clinical settings; patients were comfortable with their presence; and they had 1-month blocks of vacation time to devote to observation. Observers were aware only that we were “attempting to understand how doctors use their time.”

Category Development

We developed the activity classification categories at our institution over several years. They have been used in time motion studies of house staff physicians' inpatient activities¹² and, in modified form, for physicians' outpatient activities.¹⁷ Initially, we developed a candidate list of categories during a series of “brainstorming” meetings of four internists/informatics specialists and review of published¹⁰^,¹⁷^–¹⁹ and unpublished^* provider time categorizations. Two physician researchers (W.M.T. and J.M.O.) tested the candidate categories and refined the list on the basis of their experience, by using them to categorize physician activities during observation. We also took into account recommendations made by others about the degree of granularity of information it was necessary to obtain.¹⁸ We developed operational definitions for the final list of categories to assist observers in classifying activities into the correct category. We clarified and, in one case, modified these definitions during observer training.

A major criterion in developing the categories was the ease with which the observers could classify the activity the physician was beginning. This resulted in categories that correspond to visible, physical activities, such as “using the telephone.” We then developed subcategories to capture the specifics of the activity, such as “talking to a patient” and “obtaining a test result” as subcategories of “on the phone.”

These major categories facilitated observation but were much less useful for analyzing the data. We therefore developed analysis categories that regrouped category–subcategory pairs into categories that were easier to interpret. The criteria for creating the analysis categories were to define clinically meaningful groupings that had face validity to a clinical manager; to be consistent with the literature to allow comparison; and to reflect aspects of the care process which we expected POE to change. The analysis categories divide tasks into Direct Patient Care, Indirect Patient Care, Administrative, and Miscellaneous categories. Direct patient care tasks included talking with and examining the patient. Indirect patient care tasks included reviewing data and recording data in support of care of an individual patient. We further subdivided the Indirect Patient Care category into Write, Read, and Other subcategories, since we expected the Gopher to specifically affect order-writing times, which we included in the Write subcategory along with writing notes. Administrative tasks included tasks such as reading mail and discussing practice management issues with the staff. Miscellaneous tasks included eating and personal conversations. The Appendix

contains a complete list of the category, subcategory, and analysis categories we used.

Hardware/Software

To perform the time motion study, we used a palm computer (Newton Message Pad 120, Newton Inc., Cupertino, California) running a program we developed specifically for recording this data. The medical student observers used a stylus on the palm computer's touch-sensitive screen, eliminating the need for a keyboard and facilitating data entry while standing. At the beginning of each half day, the observers recorded the name of the physician they observed, their own identity, and the practice in which they made the observations. During the observation period, the observers selected categories and subcategories of activities from the lists presented on the palm computer. As soon as they selected a category, the computer recorded the activity's start time, to millisecond precision. The observers could then identify the subcategory at their leisure (Figure 2

). We considered the start time of each activity to be the stop time of the immediately proceeding activity. When the observers finished the observation period, they entered a specific task (Start/Stop—Done for the day) to indicate the time the observation period ended. We transferred the data from the time motion observations from the palm computer to a personal computer for storage and analysis.

Figure 2

Screen from the palm computer used to record data during the time motion study, which shows the major category list (partially obscured) and a fully expanded subcategory list.

Training

One of the authors (J.M.O.) trained all the observers. In addition to giving a didactic presentation on use of the palm computer, the investigator familiarized the observers with the categories and their operational definitions. The observers then carried out 1 to 2 hours of practice observations. After the practice observations, the investigator debriefed the observers, reviewed their observation results, and answered questions.

Survey

To assess physicians' attitudes about the effects of using the Gopher, we gave them a self-administered six-question survey. The providers indicated their agreement or disagreement with statements about the effects of the Gopher, using a scale of 1 to 5, where 1 indicated strong agreement, 3 indicated neutrality, and 5 indicated strong disagreement. We sent the survey and a cover letter by fax to each provider who used the Gopher. If they did not return the surveys within 1 week, we sent a second copy and one of the investigators (J.M.O.) called them to encourage them to complete it.

Data Analysis

The data were read into a SAS (SAS Institute, Cary, North Carolina) data set for analysis. We computed task duration as the difference between the start time of one activity and the start time of the next activity. In addition, we extracted from our appointment system data on the number of patients seen by the physician during the observation period, and obtained detailed data about the physicians' interaction with the Medical Gopher from the Gopher log files. We generated descriptive statistics by intervention and control condition.

The primary outcomes were the physician's overall time spent per patient (defined as the total observation time divided by the number of patients seen) and the time spent on Indirect Patient Care–Write tasks per patient. To examine the potential effects of practice and provider, we used SAS to fit linear mixed models. The study design was provider nested within practice and practice nested within practice type (community health center or commercial). In the analysis, we treated practice and practice type as fixed effects and provider as a random effect.

In addition, we compared the physician's overall time spent per patient and the time spent on Indirect Patient Care–Write tasks per patient between the first and last intervention observations for each provider who had at least two intervention observations, using paired t-tests to determine whether there was a learning or experience effect.

We defined duplicative tasks as activities the physician continued to perform that were unnecessary, since the Gopher was performing the same function. For example, providers would complete a paper superbill, even though the Gopher generated a superbill automatically from the data they entered. Duplicative tasks persisted, because changing processes in clinical practices requires considerable time and effort and we simply had not yet changed the processes to eliminate these tasks. To obtain more refined estimates of the time effects of POE under ideal circumstances, we identified these tasks on the basis of category–subcategory pair, labeled them as duplicative tasks, and then repeated all analyses with these duplicative tasks removed. We also removed tasks classified as administrative or miscellaneous and repeated the above analyses for overall minutes spent per patient.

Results

Of the 35 physicians invited to participate in the study, only one declined, citing a concern that many of her patients had chosen her as a provider because they wanted a female physician and most of the observers were male. We observed 34 staff providers (14 under intervention conditions at least once and 20 under control conditions) at 11 practices (Table 1

) on 110 separate observation periods, for an average of 3.5 hours per observation period, or a total of 382 hours (almost 10 weeks) of observation. The physicians were observed on 2.8±1.9 occasions. The observations included 744 patient encounters, for an average of 6.8 patient encounters per observation period.

Table 1

Comparison of Intervention and Control Practices and Physicians Included in Study

During intervention observation periods, the providers used the Gopher to write all prescriptions and test orders for all their patients.

Figure 3

shows the time required for physicians to care for patients under intervention and control conditions. Table 2

, shows the total time spent per patient divided into the six main analysis categories. The distributions of time among these categories were similar for intervention and control conditions. The largest amount of time was devoted to Direct Patient Care tasks, such as examining the patient and talking to the patient. Indirect Patient Care–Write tasks, in which we expected POE to have the greatest effect, consumed the second largest amount of time. Miscellaneous tasks (such as using the rest room and social conversation, which are not patient care or administrative tasks) consumed approximately 20 percent of the overall time.

Figure 3

Time expended per patient by internists using traditional paper-based methods (control, gray) and the Gopher system (intervention, black). The differences were not statistically significant. The means were adjusted for the distribution of providers in (more ...)

Table 2

Time Spent on Task by Analysis Groups, by Control and Intervention

In the linear mixed model analysis, we found that the specific practice had no effect, so we removed this factor from the models. Practice type (commercial vs. community health center) was not always significant, but it was retained in all models because it was a stratification variable and because it was expected apriori to be significant. The random effect for provider was significant for overall time spent per patient (P<0.05). This implies that variation from physician to physician in overall time spent per patient is significant. The intraclass correlation ranged from 0.25 to 0.27. The intraclass correlation for Indirect Patient Care–Write tasks per patient was 0.16 but only marginally significant (P=0.15)

The models predicted that physicians using the Gopher would spend 2.12 minutes per patient (P=0.50) longer than when using traditional paper-based methods (an average of 36.3 minutes compared with 34.2 minutes per patient). When we removed the duplicative tasks, the intervention effect was 0.92 minutes (P=0.77). When we removed administrative tasks, the intervention effect was 1.63 minutes (P=0.47). When we removed both duplicative tasks and administrative tasks, the intervention effect was 0.43 minutes (P=0.85). For Indirect Patient Care–Write tasks, the intervention effect was 0.61 minutes (P=0.40) using all observations and -0.55 minutes (P=0.43) after removing duplicative tasks. Of the 14 physicians observed under intervention conditions, 12 had at least two intervention observations, allowing us to evaluate the effect of experience. When we compared times from first and last observations, we found that, on average, the physicians spent 3.73 fewer minutes per patient overall and 1.03 minutes fewer for Indirect Patient Care–Write per patient during the last observation than the first. However, these differences were not statistically significant.

Data extracted from the Medical Gopher for intervention providers during the observation periods showed that total time spent using the computer for orders and documentation was 2.28±2.18 minutes (range, 0.1 to 11.8 minutes) per encounter. Of this time, providers spent 54 to 98 percent on ordering and the remainder on reviewing clinical data, examining reference materials, and choosing billing diagnosis, evaluation, and management codes. Providers entered 1.14±1.28 problems and 0.02±0.16 allergies per encounter. They wrote 0.6±1.4 test orders (7.7±24.0 seconds) and 2.73±3.0 prescriptions (59±65 seconds) during each session. Providers chose one half of all orders from the various menus or sets of orders provided. They entered only 11 percent of instructions for tests and drugs by typing free text, whereas they created 41 percent of instructions by choosing from preconstructed “sigs” or instructions specific to the test or drug. The remaining 48 percent were “carried over” or copied from a previous “sig” for that order. Physicians reacted to the 20 drug interaction warnings provided by the system during the time they were observed by canceling the medication ordered 19 times (95 percent).

Survey Results

Intervention physicians completed 13 surveys (86 percent response rate) responding on a five-point Likert scale. They agreed that their work was done faster (1.92±0.90), easier (1.58±0.67), that the quality of documentation improved (1.42±0.67), workflow was improved (1.75±0.62) and quality of care improved (1.83±0.71). When asked if we should remove the Gopher from their practice, they strongly disagreed (3.82±0.40).

Discussion

Overall, physicians randomized to use the Gopher spent 2.12 minutes (6.2 percent) longer per patient writing orders than did control physicians who used paper-based methods. Unfortunately, physicians in the intervention group continued to perform certain tasks using paper-based methods even though the computer was automatically performing those tasks for them. For example, they would, in some cases, manually complete a paper superbill even though the Gopher created an acceptable superbill for them. The time wasted on these unnecessary or duplicative tasks was more than a minute per patient. If we could eliminate this wasted time through further training and process improvement, we estimate the net time cost of POE system use would be only 0.92 minutes per patient. Furthermore, the above figures include the time for users early in their experience with the computer as well as later. We found that the time costs declined with experience, so the time costs for using the POE system could be very small.

Only a few other studies report time data for order entry, and several of those report the blended results from implementing POE and a computerized patient record. Medscape, for example, cites 20 percent productivity improvements with their Logician product when implementing order entry along with a computerized patient record.¹⁹ In the Kaiser Permanente Northwest region, clinicians spent an additional 2.2 minutes per patient when the organization introduced POE and a computerized patient record into their outpatient practices.⁸ These physicians used the system to order tests and prescriptions only 70 to 80 percent of the time. We and others²⁰^,²¹ have previously reported that POE at least doubled the time to record orders for inpatient services.

Most of the time required to use POE is the time to enter individual orders. Developers have tailored the user interface for POE to minimize the time required to enter individual orders in outpatient prescription writing. Reported average times to generate prescriptions using computer-based POE systems or paper-based systems range from less than 30 seconds for the simplest to 252 seconds for complex prescriptions.²²^,²³

Most other data come from inpatient settings. In a study in Japan, investigators found that physicians and other personnel required 102±121 seconds to enter a new prescription, 76±109 seconds to enter a laboratory order, and 54±82 seconds for a radiological study.²⁴ Physicians using the HELP system at LDS Hospital required 132 seconds per order.²⁵ We previously reported average test-ordering times of 30 seconds per test on an earlier version of the Gopher used by medicine residents in an inpatient setting.¹⁴ The Gopher was much faster to use than any of these previously described systems, requiring only 20 seconds on average to complete a prescription. We attribute the speed with which physicians can write orders using the Gopher to parsimonious design, ease of locating desired items, and preconstructed or previous “sigs,” which the residents used almost 90 percent of the time.

Compared with their first observed use of POE, physicians' last observed use took 3.73 minutes less per patient. Others have reported similar results, stating that while POE initially requires more time to use than paper-based methods, this difference disappears over 4 to 8 weeks.²⁶ Schroeder and Pierpaoli²⁷ state that “after 1 or 2 weeks of use, most physicians find that they can enter orders in the computer terminals faster than they could write them by hand.” Finally, Allen et al.²⁸ report that order times were 82 seconds for novice users and 18 seconds for expert users in an outpatient prescription writing system. Not only are physicians learning to use the program, they are usually also learning a vocabulary, discovering what the things they want to order are called in the system.

The Medical Gopher provided a number of benefits while requiring slightly more time than paper-based methods. First, workflow was improved: Ancillary personal did not have to complete requisitions, records were legible and always available, physicians wrote consequent orders more often, and the practice saved money on forms. We have also shown, in previous studies,²⁹^–³¹ that cost efficiency can be improved by decreasing the number of tests ordered. Third, clinicians received feedback, such as drug– drug and drug–diagnosis interactions, passive and active order menus, and academic detailing as well as reminders about relevant test results that, as published evidence demonstrates, improves the quality of patient care.³²^–³⁴ Finally, clinicians had ready access—from home, from the hospital, and from secondary practice locations—to the data they recorded using the Gopher.

The study has some limitations. First, the results are based on use of a specific POE system and may not be generalizable to other systems. However, they do indicate that direct POE using a well-designed system may require only minimal additional time and may even, with continued use, represent a time savings. The Medical Gopher has been used in our outpatient practices for more than 15 years, and during that time has been carefully adapted to the patterns and processes of our practices. Approximately a quarter of the physicians in our study had some experience using the Gopher for inpatient order entry before the study.

Second, the providers in our study, who are salaried, may not be representative of physicians in private practice, who may be even more sensitive to time pressure. However, increasing numbers of clinicians are now salaried employees.³⁵ While the practices we studied are affiliated with a medical school and do host some resident teaching activities, they are managed to be viable independent practices, and all but one are remote from the school's campus. In addition, the volumes of patient visits we observed are consistent with Medical Group Management Association (MGMA) benchmarks of 14 to 18 patients per day for general internal medicine practices.³⁶ Third, the subjects of this study are general internists, and our results may not apply to physicians in other specialties, such as obstetrics or orthopedics.

Conclusion

In conclusion, little extra time, if any, was required for physicians to use the POE system in this study. With experience in its use, physicians may even save time while enjoying its many benefits. If the time required for POE use is comparable with the time required for use of paper-based systems, we have demonstrated that one of the major barriers to adoption can be overcome.

Appendix

Appendix: Task Categories

Table t

Appendix: Task Categories

Major Category	Minor Category	Analysis Grouping
Computer	Article	Miscellaneous
Computer	Drug Reference	Indirect Patient Care-Other
Computer	E-mail	Administration
Computer	Literature Search	Miscellaneous
Computer	Looking Up Data	Indirect Patient Care-Read
Computer	Other	Miscellaneous
Computer	Reviewing Dictations	Indirect Patient Care-Other
Computer	Writing Note	Indirect Patient Care-Write
Computer	Writing Orders/Scripts	Indirect Patient Care-Write
Examine/Read	Article	Miscellaneous
Examine/Read	Chart	Indirect Patient Care-Read
Examine/Read	Drug Reference	Indirect Patient Care-Other
Examine/Read	EKG	Indirect Patient Care-Read
Examine/Read	Mail	Administration
Examine/Read	Other	Miscellaneous
Examine/Read	Patient	Direct Patient Care
Examine/Read	Pocket Reference	Indirect Patient Care-Other
Examine/Read	Radiograph	Indirect Patient Care-Read
Examine/Read	Schedule	Administration
Examine/Read	Textbook	Miscellaneous
Forms	Consult	Indirect Patient Care-Write
Forms	H & P	Indirect Patient Care-Write
Forms	Insurance	Indirect Patient Care-Write
Forms	Other	Indirect Patient Care-Write
Forms	Requisition	Indirect Patient Care-Write
Forms	Superbill	Indirect Patient Care-Write
Forms	Work/School Release	Indirect Patient Care-Write
Looking for	Chart	Indirect Patient Care-Other
Looking for	Colleague	Indirect Patient Care-Other
Looking for	Consultant	Indirect Patient Care-Other
Looking for	Lab Result	Indirect Patient Care-Other
Looking for	Other	Indirect Patient Care-Other
Looking for	Patient	Indirect Patient Care-Other
Looking for	Radiograph	Indirect Patient Care-Other
Looking for	Supplies	Indirect Patient Care-Other
Miscellaneous	Eating	Miscellaneous
Miscellaneous	Idle	Miscellaneous
Miscellaneous	Other	Miscellaneous
Miscellaneous	Restroom	Miscellaneous
Oops	Can't Find Entry	Miscellaneous
Oops	Other	Miscellaneous
Phone	Getting Results	Indirect Patient Care-Read
Phone	Other	Miscellaneous
Phone	Paging	Indirect Patient Care-Other
Phone	Patient	Direct Patient Care
Phone	Personal	Miscellaneous
Phone	Scheduling Tests	Indirect Patient Care-Other
Procedure	EKG	Direct Patient Care
Procedure	IV	Direct Patient Care
Procedure	Joint Inj/Asp	Direct Patient Care
Procedure	Lab Test	Direct Patient Care
Procedure	Other	Direct Patient Care
Procedure	Pelvic Exam	Direct Patient Care
Procedure	Phlebotomy	Direct Patient Care
Talking	Advance Directives	Direct Patient Care
Talking	Colleague/Staff	Direct Patient Care
Talking	Colleague/Staff	Miscellaneous
Talking	Consultant	Direct Patient Care
Talking	Educating Patient	Direct Patient Care
Talking	Other	Miscellaneous
Talking	Patient History	Direct Patient Care
Talking	Patient Other	Direct Patient Care
Talking	Patient's Family	Direct Patient Care
Talking	Personal	Miscellaneous
Talking	Study Consent	Miscellaneous
Walking	Inside	Miscellaneous
Walking	Outside	Miscellaneous
Writing	Letter	Indirect Patient Care-Write
Writing	Notes	Indirect Patient Care-Write
Writing	Orders	Indirect Patient Care-Write
Writing	Other	Miscellaneous
Writing	Script	Indirect Patient Care-Write

Notes

This work was performed at the Regenstrief Institute for Health Care and was supported by contracts N01-LM-4-3510 and N01-LM-6-3546 from the National Library of Medicine.

Footnotes

^*Paul Tang, MD, written communication, Jul 12, 1995.

References

1. Collen MG. General requirements for a medical information system (MIS). Comput Biomed Res. 1970;3:393–406. [PubMed]

2. Dick RS, Steen EB, Detmer DE (eds). The Computer-based Patient Record: An Essential Technology for Health Care. Rev. ed. Committee on Improving the Patient Record, Division of Health Care Services, Institute of Medicine. Washington, DC: National Academy Press, 1997.

3. Bodenheimer T. The American health care system: the movement for improved quality in health care. N Engl J Med. 1999;340(6):488–92. [PubMed]

4. Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994;1:108–23. [PMC free article] [PubMed]

5. Kaplan B. Reducing barriers to physician data entry for computer-based patient records. Top Health Inf Manag. 1994;15(1):24–34. [PubMed]

6. Sullivan F, Mitchell E. Has general practitioner computing made a difference in patient care? A systematic review of published reports. BMJ. 1995;311:848–52. [PMC free article] [PubMed]

7. Massaro TA. Introducing physician order entry at a major academic medical center, part II: impact on organizational culture and behavior. Acad Med. 1993;68:20–5. [PubMed]

8. Chin HL, McClure P. Evaluating a comprehensive outpatient clinical information system: a case study and model for system evaluation. Proc 19th Annu Symp Comput Appl Med Care. 1995:717–21. [PMC free article] [PubMed]

9. Krall MA. Acceptance and performance by clinicians using an ambulatory electronic medical record in an HMO. Proc 19th Annu Symp Comput Appl Med Care. 1995:708–11. [PMC free article] [PubMed]

10. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations effects on resource utilization. JAMA. 1993;269:379–83. [PubMed]

11. Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc. 1996;3:42–55. [PMC free article] [PubMed]

12. McDonald CJ, Tierney WM, Overhage JM, et al. The Regenstrief Medical Record System: 20 years of experience in hospitals, clinics and neighborhood health centers. MD Comput. 1992;9(4):206–17. [PubMed]

13. McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inform. 1999;54(3):225–53. [PubMed]

14. McDonald CJ, Tierney WM. The Medical Gopher: a microcomputer system to help find, organize and decide about patient data. West J Med. 1986;145(6):829. [PMC free article] [PubMed]

15. Overhage JM, Tierney WM, McDonald CJ, Pickett KE. Computer-assisted order entry: impact on intern time use. Clin Res. 1991;39(3):729A.

16. Wirth P, Kahn L, Perkoff, GT. Comparability of two methods of time and motion study used in a clinical setting. Med Care. 1977;15:953–60. [PubMed]

17. Tang PC, Jaworski MA, Fallencer CA, et al. Methods for assessing information needs of clinicians in ambulatory care. Proc 19th Annu Symp Comput Appl Med Care. 1995:630–4. [PMC free article] [PubMed]

18. Smith DM, Martin DK, Langefeld CD, Miller ME, Freedman JA. Primary care physician productivity: the physician factor. J Gen Intern Med. 1995;10:495–503. [PubMed]

19. MedicaLogic. Ambulatory EMR: Establishing a Business Case. Hillsboro, Ore.: Medicalogic, 1999.

20. Bates DW. Boyle DL, Teich JM. Impact of computerized physician order entry on physician time. Proc 18th Annu Symp Comput Appl Med Care. 1994:996. [PMC free article] [PubMed]

21. Bates DW, Shu K, Narasimhan D, Horsky J. Comparing time spent writing orders on paper and physician computer order entry. Proc AMIA Annu Symp. 2000:965. Also available at http://medicine.ucsd.edu/f2000/D200364.htm; accessed Jan 21, 2001.

22. Donald JB. Online prescribing by computer. BMJ. 1986;292:937–9. [PMC free article] [PubMed]

23. Garrett LE, Hammond WE, Stead WW. The effects of computerized medical records on provider efficiency and quality of care. Methods Inform Med. 1986;25:151–7. [PubMed]

24. Ogura H, Sagara E, Yamamoto K, Furutani H, Kitazoe Y, Takeda Y. Analysis of the online order entry process in an integrated hospital information system. Comput Biol Med. 1985;15:381–95. [PubMed]

25. Lepage EF, Gardener RM, Laub RM, Golubjatnikov OK. Improving blood transfusion practice: role of a computerized hospital information system. Transfusion. 1992;32:253–9. [PubMed]

26. Reynolds RE, Heller EE. An academic medical center experience with a computerized hospital information system: the first four years. Proc 4th Annu Symp Comput Appl Med Care. 1980:3–16.

27. Schroeder CG, Pierpaoli PG. Direct order entry by physicians in a computerized hospital information system. Am J Hosp Pharm. 1986;43:355–9. [PubMed]

28. Allen SI, Johannes RS, Brown CS, Kafonek DM, Plexico PS. Prescription writing with a PC. Comput Methods Programs Biomed. 1986;22:127–35. [PubMed]

29. Tierney WM, Miller ME, McDonald CJ. Informing physicians of test charges reduces outpatient test ordering. N Engl J Med. 1990;322:1499–504. [PubMed]

30. Tierney WM, McDonald CJ, Martin DK, Hui SL, Rogers MP. Computerized display of past test results.: effect on outpatient testing. Ann Intern Med. 1987;107:569–74. [PubMed]

31. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results: effects on outpatient testing. JAMA. 1988;259:1194–8. [PubMed]

32. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311–6. [PubMed]

33. Larsen RA, Evans RS, Burke JP, Pestotnik SL, Gardner RM, Classen DC. Improved perioperative antibiotic use and reduced surgical wound infections through use of computer decision analysis. Infect Control Hosp Epidemiol. 1989;10:316–20. [PubMed]

34. Rind DM, Safran C, Phillips RS, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994;154:1511–7. [PubMed]

35. Robinson JC. Blended payment methods in physician organizations under managed care. JAMA. 1999;282(13):1258–63. [PubMed]

36. Medical Group Management Association. Physician Compensation and Production Survey 1998: Report Based on 1997 Data. Englewood, Colo.: MGMA, 1998:79.

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association