Search tips
Search criteria 


Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
J Gen Intern Med. 2008 April; 23(4): 442–446.
Published online 2008 March 29. doi:  10.1007/s11606-008-0505-4
PMCID: PMC2359504

A Mixed Method Study of the Merits of E-Prescribing Drug Alerts in Primary Care

Kate L. Lapane, PhD,corresponding author1 Molly E. Waring, MA PhD candidate,1 Karen L. Schneider, PhD,1 Catherine Dubé, EdD,1,2 and Brian J. Quilliam, PhD3



The objective of this paper was to describe primary care prescribers’ perspectives on electronic prescribing drug alerts at the point of prescribing.


We used a mixed-method study which included clinician surveys (web-based and paper) and focus groups with prescribers and staff.


Prescribers (n = 157) working in one of 64 practices using 1 of 6 e-prescribing technologies in 6 US states completed the quantitative survey and 276 prescribers and staff participated in focus groups.


The study measures self-reported frequency of overriding of drug alerts; open-ended responses to: “What do you think of the drug alerts your software generates for you?”


More than 40% of prescribers indicated they override drug–drug interactions most of the time or always (range by e-prescribing system, 25% to 50%). Participants indicated that the software and the interaction alerts were beneficial to patient safety and valued seeing drug–drug interactions for medications prescribed by others. However, they noted that alerts are too sensitive and often unnecessary. Participant suggestions included: (1) run drug alerts on an active medication list and (2) allow prescribers to set the threshold for severity of alerts.


Primary care prescribers recognize the patient safety value of drug prescribing alerts embedded within electronic prescribing software. Improvements to increase specificity and reduce alert overload are needed.

KEY WORDS: e-prescribing, electronic prescribing, drug alerts, primary care, medication use


Adverse drug events have been estimated to occur in 27.4% of community dwelling adults.1 Among Medicare beneficiaries in an outpatient setting, estimates of adverse drug events are higher.2 Nearly 60% of errors associated with preventable adverse drug events occur at the ordering stages of prescribing.2 Given the estimated $887 million dollars spent on preventable adverse drug events among Medicare recipients in the ambulatory setting,3 identifying viable interventions for reducing preventable adverse drug events is important.

E-prescribing technology solutions may provide opportunities to reduce preventable adverse drug events which occur at the ordering stage of the pharmacy care process. Electronic prescribing involves the direct computer to computer transmission of prescription medication information from prescriber office to community pharmacies. This relatively new innovation began in 2003, and in 2006, all but one state had regulations allowing e-prescribing (; accessed on April 10, 2007). Ninety-five percent of the software systems community retail pharmacies use have been certified on the SureScripts network, the largest pharmacy health information network (; accessed on April 10, 2007). While networks such as SureScripts provide the pipeline to transmit pharmacy health information from clinical practices to community pharmacies, physician e-prescribing software systems interface with the physician to e-prescribe. In 2006, more than 150,000 prescribers were using e-prescribing technology solutions in community settings (; accessed on April 10, 2007). One potential feature of the technology is the ability to alert prescribers to thousands of potential drug interactions (e.g., drug–disease, drug–drug, etc.) and drug allergies at the point of prescribing. While the efficiency gains of e-prescribing have been documented,4 the patient safety gains of this technology have not been studied extensively. Previous research has shown that drug alerts are frequently overridden because of poor specificity and high volume of alerts.5 However, electronic prescribing research is often system-specific6 or specific to geographic locales which may not be generalizable to the US health care system.7,8

As part of a larger study conducted to evaluate the proposed standards for new e-prescribing transactions, we had the opportunity to evaluate prescribers’ opinions about drug alerts embedded within e-prescribing applications. The analysis focuses on statements or comments about drug alerts at the point of prescribing.


The Brown Institutional Review Board approved the study protocol.

Study Sample

We identified states with the highest electronic prescribing activity on the SureScripts network in the fall of 2005. For reasons of logistics and feasibility, we selected a convenience sample of 6 states with the highest volume of e-prescribing transactions on the SureScripts network. Within these 6 states, SureScripts identified physician software systems willing to participate in the testing of the e-prescribing standards pilot project. The physician software systems participating in the study and their geographic representation (in brackets) included: OnCallData, InstantDX, LLC, Gaithersburg, MD (Rhode Island); PocketScript, Zix Corporation, Dallas, TX; (Massachusetts, New Jersey), Rcopia, DrFirst, Inc., Rockville, MD (Massachusetts); Care360, Medplus, Inc., Mason, OH (New Jersey, Florida); eMPOWERx, GoldStandard Multimedia, Inc., Tampa, FL (Florida); Touchworks, AllScripts, LLC, Chicago, IL(Nevada, Tennessee). Thus, 6 different e-prescribing systems were included in the study, but geographic location was highly correlated with software system. We required that each system vendor identify and enroll medical practices with a patient mix of at least 25% Medicare eligible patients. Physicians participating in the study received a $500 incentive. We cannot report participation rates with any level of certainty because our first contact with practices was when we received signed participation agreements.

The protocol of the larger e-prescribing standards study included an evaluation of medical practices, the physician software systems, and personnel in community pharmacies. The multi-component medical practice protocol consisted of surveys of prescribers, patients and staff; focus groups and semi-structured interviews with prescribers and staff conducted on-site; and at least one half day of site observation (including observation of patient–physician interactions related to medication use).

For the current study, we used a mixed methods approach that includes analysis of quantitative and qualitative data. Specifically, we analyzed the portion of the clinician survey relating to drug alerts, as well as the focus groups relating to drug alerting features of the software. The data for this study are derived from 64 practices, all experienced in electronic prescribing, which participated in on-site visits. All the data were collected before the new standards (and any changes to the electronic prescribing software to accommodate the electronic prescribing standards) were implemented.

Clinician Surveys

The survey was designed to capture relevant information regarding prescriber perceptions of e-prescribing on efficiency, workflow, and quality, as well as their perceptions about patient communication relevant to medication issues. Content of the survey was informed by input from a multidisciplinary advisory team including practicing prescribers, pharmacists, and researchers. Prescribers completed the survey in either paper or web-based format as part of the medical practice protocol.

We evaluated responses to a series of questions regarding the frequency with which overrides of drug alerts occurred. Separate questions evaluated overriding of drug–drug interactions, allergies, and drug alerts regarding dosing. Responses included “always”, “most of the time”, “sometimes”, and “never”. Given concerns regarding sparse data, we collapsed the “always” and “most of the time” responses into one category. Prescribers (n = 157) completed surveys available via the web (68%) or paper (32%) in advance of or during the site visit.

The analysis of the survey included descriptive statistics of the respondents (gender and job title) as well as descriptive statistics of the practices included in the study. Cross-tabulations of clinician responses to the drug alerting questions by physician software system were conducted. Fisher’s exact test was used to calculate more conservative p values owing to the small sample size.9

Focus Groups

We also analyzed information conducted during on-site focus groups with 276 prescribers and their staff. Two highly trained research assistants held focus groups (with a meal provided) before hours, at lunch, or after hours at the discretion of the practice between April and August 2006. Consent forms and demographic surveys were collected, and a sign listing the main topics for discussion was placed on the table for participants to view. An open-ended approach was used to elicit information about the benefits and drawbacks of e-prescribing. Focus group participants were asked to describe their experiences with e-prescribing software as well as their suggestions for improving e-prescribing. Participants spontaneously addressed patient safety issues and interaction alerts in the context of these discussions. Probes included questions about what aspects of e-prescribing are valuable, what participants found difficult, suggested improvements in office procedures and software functionality, and other resources that might be valuable. Other general probing was conducted using facilitative questions (Can you tell me more about that? “Any other opinions?) and clarification (summarizing and checking for accuracy, “When you say…, what do you mean by that?”).

Focus groups were recorded using 2 digital recorders with PZM microphones. Once all digital recordings were transcribed, research assistants double-checked every transcript for potential errors and corrected them as needed.

An extensive hierarchical coding structure was initially developed to handle the large volume of qualitative data. One of the authors (CD) designed the initial structure which was based on the focus group protocol and review of initial transcripts and revised and/or expanded during active coding. Codes were identified and defined so that diverse comments from participants could be collected in logical groupings for review and analysis. Using NVivo qualitative analysis software (version 7), 15 different parent nodes were defined. We focused our attention on 2 parent nodes, impact on clinical practice and software features, which were selected because our codebook instructed coders to place quotes related to alerting systems in these nodes. Within these parent nodes, we focused on nodes entitled patient safety, patient care, and drug alerts because the codebook explicitly directed coders to place quotes relating to alerts into these nodes. Another node, quality of care under impact on clinical practice was also examined to evaluate quotes related to how the alerts impacted practice and quality. Coders were trained in coding definitions and overall coding structure. A code book defined all codes and their relationships. All quotes for the current analysis were derived from the focus groups.

Consistency in the coding across team members was assured by extensive training, coding meetings, a coding handbook which provided the coding structure and definitions, group exercises, and by having 19% of the transcripts independently coded a second time by a different member of the coding team. Reports comparing the coding were generated and reviewed. These reports were used to identify any areas of coding that were not consistently applied by coders and for which additional training was required. Finally, a qualitative data review was conducted on the double coded transcripts. Passages coded by each coder commonly appeared twice, indicating effective coding among those transcripts by the research staff.


Table 1 shows the characteristics of the 64 practices participating in the study. Twenty (31%) of participating practices were solo practices, 20 (31%) were single specialty groups, and 12 (19%) were multi-specialty groups. Twenty-nine practices (45%) specialized in internal medicine and 24 (38%) practices in family medicine. Of participants in the prescriber survey, 57 of 157(36%) were female, 125 (80%) were physicians or residents, 13 (8%) were physician assistants, and 19 (12%) were nurse practitioners. The 64 focus groups included 276 participants. Focus groups were comprised mostly of prescribers (physicians, residents, nurse practitioners, and physician assistants; 64%). On average, prescribers were (mean ± SD) 46 ± 7 years old and in practice for 15.7 ± 6.2 years. Medical assistants, nurses, office managers, pharmacists, and other office staff also participated in focus groups.

Table 1
Characteristics of Practices

Figure Figure11 shows the frequency with which survey responders reported overriding drug alerts for allergies (panel a), dose checks (panel b), and drug–drug interactions (panel c). Overall, survey responders were least likely to override allergy-related drug alerts, but overrode drug–drug interaction alerts most of the time or always. Overall, 22 of 145 prescribers (15%) reported overriding drug-allergy alerts most of the time or always with variation in frequency of overriding drug alerts by e-prescribing software system ranging from 9% to 50% (p = 0.656 for overall comparison by e-prescribing software system). Nearly 1 in 4 respondents reported overriding drug–dose alerts most of the time or always (range, 13% to 33%; p = 0.006). More than 40% indicated they override drug–drug interactions most of the time or always (range, 25% to 50%; p = 0.374).

Figure 1
Frequency with which prescribers override drug alerts. Self-reported frequency of prescribers overriding drug alerts regarding allergies (a), dose checks (b), and drug–drug interactions (c) stratified by physician software system used by prescribers. ...

Qualitative data analysis revealed insights related to the use of drug alerts. First, few participants commented about allergy alerts. As reflected in the following statement, those who did comment found the allergy alert function useful and integrated the information while making their final medication decision.“…I like the fact that the allergies are in there so it’s a second check. First I would ask the patient if they have any allergies but also if I forget about that it would kind of bring it up and flag that.” While some prescribers noted that sometimes they overrode these alerts, they nevertheless agreed that these alerts were helpful. One clinician noted: “But we don’t always accept…this person has been taking Lasix for five years and never had a problem. I’m going to ignore that. It’s still a good thing to be reminded.”

With respect to drug–drug interactions, participants indicated that interaction alerts were beneficial to patient safety. Prescribers liked that drug–drug interaction alerts would appear for possible interactions with drugs prescribed by other providers; participants felt that this was a very positive feature of the software. “…e-prescribing…helped the quality because I was able to catch drug interactions…duplicate drugs….That’s stuff that can hurt people and also that wastes a lot of money.” However, many participants reported ignoring the drug–drug interaction alerts because of the number of trivial or unnecessary alerts. Comments such as “as a result of the unnecessary volume of warnings, the warnings themselves get ignored.” and “…it’s one of the things that should be fixed somehow because right now this is the boy who is crying wolf, and nobody pays attention to any warnings.”

Participants had suggestions for improving the drug–drug interaction alerts. Providers expressed annoyance at a drug–drug interaction alert for a drug the patient was no longer taking (for example, a short course of antibiotics). Running the drug alerts against a current drug regimen instead of the entire medication history was suggested to reduce the volume of warnings. In general, recommendations for improving the drug alerting software within e-prescribing included making the program less sensitive or more sensible or allowing providers to set their own level of severity. As one clinician suggested, “What they need to do and what some electronic medical record software systems have done is they prioritize the interaction alerts, maybe ten being the most serious and one being the least serious. And then each physician or each practice can kind of set their threshold.”


Drug alerting at the point of prescribing has the potential to improve patient safety in the outpatient setting.10 Prescribers in our study voiced the importance of drug alerting at the point of prescribing. Nevertheless, many override such alerts because of lack of specificity of the messages or irrelevance of the medication to the current drug regimen. Our findings are consistent with previous research demonstrating that such alerts are frequently overridden (49–96% of cases) because of poor specificity and high volume of alerts.5 Our study, however, demonstrated that opportunities for improving drug alerts in this setting are warranted and desired by prescribers. Prescribers recommended having the drug alerting system only run on active medications (rather than the entire medication history), as well as permitting the prescribers to set the desired severity threshold for the alerts.

Participants in our study desired the flexibility to tailor the use of automated warnings within their own practice. These findings were consistent with desires of primary care providers in the VA system.11 Allowing prescribers to control the threshold of severity of alerts to be shown in the practice is a reasonable suggestion for several reasons. First, drug alerting at the point of prescribing in ambulatory settings does not shift the work of evaluating the potential for medication harm upstream (from the pharmacist to the prescriber). Rather, this approach adds an additional layer of checking. The Omnibus Budget Reconciliation Act of 1990 requires pharmacists to perform a drug use review to evaluate prescribed drug therapy before dispensing to ensure that therapy is medically necessary, appropriate, and not likely to result in adverse events. Specifically, pharmacists evaluate: therapeutic duplication, therapeutic appropriateness, drug–allergy interactions, drug–disease contraindications, drug–drug interactions, correct dosage and duration of therapy, utilization, abuse, and appropriate use of generic products.12 Second, having a greater specificity of alerts—or a reduction in alert overload—may lead to less overriding.13 Further, drug alerting systems targeting specific issues and minimizing workflow disruptions have been shown to increase clinician acceptance of alerts in ambulatory settings.5

Several alternative suggestions to reduce alert overload were made by our study prescribers. Similar to previous reports,14 prescribers in our study suggested suppressing alerts for renewals of medication combinations that patients currently tolerate. Providers in our study often noted that short-term courses of therapy would continue to arise in the alerts, suggesting that the time frame for medication history on which the drug alerts are run should be evaluated. In e-prescribing technology solutions, medication history is available from payors, and a flow of information from retail pharmacies has recently been evaluated.15 While findings from the pilot testing of e-prescribing standards suggest that the comprehensiveness of medication history can be improved with retail pharmacy prescription data, standards on how far back medication history should be provided need to be considered. If the e-prescribing technology has the drug alerting component run on the entire medication history, it may actually exacerbate alert overload. The most straightforward solution may be for physician software products to incorporate the concept of an active medication list. The extent to which modifications to existing software are necessary to accommodate this notion is unknown.

Primary care physicians believe integrated electronic prescribing improves continuity of care.16 Prescribers in our study clearly noted that an advantage was the usefulness of the drug alerts when identifying prescriptions ordered by other prescribers. It is possible that prescribers are more likely to use these systems when caring for clinically challenging patients who receive more fragmented care.

The following limitations of the data should be considered. Although the data are from a geographically diverse group of physician practices, all practices in this study were currently using e-prescribing. The study sample is a convenience sample selected by software vendors, and the participants are likely to represent the most experienced e-prescribing users in primary care settings. Despite the sample, the findings did not appear to provide an overly optimistic view of drug alerts at the point of prescribing. Due to the sampling design, both geographic location and physician software system are confounded, and thus, results cannot be isolated.


A conservative estimate of 530,000 preventable adverse drug events occurs among outpatient Medicare patients yearly.2 Prescribers must have full knowledge of the current drug regimen to avoid preventable adverse drug events. E-prescribing adoption is likely to increase in the USA owing to the Medicare Modernization Act of 2003.17 Prescribers believe that refinements to the drug alerting systems are necessary to reduce common overriding of alerts. In addition to honing the specificity of the alerts and permitting prescribers to set the severity threshold for alerts, prescribers recommend having the drug alert algorithms run against current medication regimens.


We gratefully acknowledge the assistance of Ken Whittemore, RPh, MBA and Ajit Dhalve, PharmD, MBA, of SureScripts in facilitating access to the software vendors and physician practices. This research was funded by a cooperative agreement entitled “Maximizing effectiveness of e-prescribing between physicians and community pharmacies” from AHRQ (U18 HS016394-01) to SureScripts. Mr. Whittemore and Dr. Dhalve did not participate in the collection, analysis, or interpretation of these data.

Conflict of Interest A grant entitled “Maximizing effectiveness of e-prescribing between physicians and community pharmacies” funded by Agency for Healthcare Research and Quality Collaborative Agreement U18 HS016394-01 supported this research. Dr. Lapane [although not affiliated with SureScripts in any way (financial or otherwise)] was the principal investigator on the AHRQ-funded project. As a result, Brown University received a subcontract from SureScripts to conduct the evaluation of the AHRQ-funded project, funding for a graduate student to complete the final report to AHRQ, and funds to pay for the costs associated with performing a patient survey which were not allowable expenses on the AHRQ grant. None of the authors have any financial relationships with any of the vendors participating in this study. The authors have no other conflicts to declare.


1. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med. 2003;348:1556–64. [PubMed]
2. Gurwitz JH, Field TS, Harrold L, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003;289:1107–16. [PubMed]
3. Field TS, Gilman BH, Subramanian S, Fuller JC, Bates DW, Gurwitz JH. The costs associated with adverse drug events among older adults in the ambulatory setting. Med Care. 2005;43(12):1171–6. [PubMed]
4. Corley ST. Electronic prescribing: A review of costs and benefits. Top Health Inform Manage. 2003;24(1):29–38.
5. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138–47. [PMC free article] [PubMed]
6. Shah N, Seger AC, Seger DL, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc. 2006;13(1):5–11. [PMC free article] [PubMed]
7. Avery AJ, Savelyich BS, Sheikh A, Morris CJ, Bowler I, Teasdale S. Improving general practice computer systems for patient safety: qualitative study of key stakeholders. Qual Saf Health Care. 2007;16(1):28–33. [PMC free article] [PubMed]
8. Tamblyn R, Huang A, Perreault R, et al. The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ. 2003;169(6):549–56. [PMC free article] [PubMed]
9. Agresti A. Categorical data analysis: 2nd edition. New York: Wiley-Interscience; 2002.
10. Bell DS, Cretin S, Marken RS, Landman AB. A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities. J Am Med Inform Assoc. 2004;11:60–70. [PMC free article] [PubMed]
11. Spina J, Glassman PA, Belperio P, Cader R, Asch S. Primary Care Investigative Group of the VA Los Angeles Healthcare System. Clinical relevance of automated drug alerts from the perspective of medical providers. Am J Med Qual. 2005;20(1):7–14. [PubMed]
12. National Association of Boards of Pharmacy. Information packet: Omnibus Budget Reconciliation counseling and drug use review requirements. Park Ridge, IL, Publ. no. 101-508. 104 Stat 1388, 4401 (February, 1992).
13. Smith DH, Perrin N, Feldstein A, et al. The impact of prescribing safety alerts for elderly persons in an electronic medical record: an interrupted time series evaluation. Arch Intern Med. 2006;166(10):1098–104. [PubMed]
14. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003;163(21):2625–31. [PubMed]
15. Foxhall K. E-prescribing goes to Washington. With the federal government behind it, fselectronic prescribing is gaining speed, weight, and more standards. Healthc Inform. 2007;24(6):14–5. [PubMed]
16. Tamblyn R, Huang A, Kawasumi Y, et al. The development and evaluation of an integrated electronic prescribing and drug management system for primary care. J Am Med Inform Assoc. 2006;13(2):148–59. [PMC free article] [PubMed]
17. Bell DS, Friedman MA. E-prescribing and the Medicare Modernization Act of 2003. Health Aff. (Millwood). 2005;24(5):1159–69. [PubMed]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine