Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Pediatrics. Author manuscript; available in PMC 2011 May 17.
Published in final edited form as:
PMCID: PMC3095847

Improving Clinical Quality Indicators Through Electronic Health Records: It Takes More Than Just a Reminder

Dean F. Sittig, PhD,a Jonathan M. Teich, MD, PhD,b,c Jerome A. Osheroff, MD,d,e and Hardeep Singh, MD, MPHf

State-of-the-art electronic health record systems with advanced clinical decision support (CDS) capabilities can fundamentally improve quality and reduce costs of health care.1,2 However, these outcomes have not been universally achieved.3,4 As the study by Fiks et al5 in this issue of Pediatrics demonstrates, providing CDS in the form of “alerts” to encourage desired health care activities may not be sufficient to make a substantial impact.6 Maximizing the potential of CDS for improving quality and safety of care requires attention to several factors, not all of which are related to the computer system.7

The goal for the study by Fiks et al was to increase vaccination rates in asthmatic children, so in examining the results one must first consider what caused the low vaccination rate in their population. Several factors could account for the low initial vaccination rates and, hence, could explain the minimal improvements with alerting. Without knowledge about these factors, it may be too much to expect alerts alone to fix the problem. Alerts are helpful when an unusual occurrence must come to a physician’s attention or when a necessary process might be overlooked in a busy encounter. When other underlying problems lead to low vaccination rates, such as poor patient acceptance, difference of opinion about vaccinating patients late in the season, or low priority of vaccination when a patient has an acute problem, they must be addressed before the alert can be successful. Indeed, studies of influenza vaccination reminders in adults have had varying results, and in some cases these results were directly attributable to such noncomputable factors.8,9 It would have been enlightening if the decision support used in this study also captured the reasons for failure of the providers to act on the alert by having them select or enter a reason for nonvaccination.10

In addition, one should also consider whether presentation of the vaccination alert as soon as the patient encounter was opened within the electronic health record was the best CDS intervention to achieve the desired objectives, compared with other intervention types such as facesheet displays, order sets, patient education handouts, and end-of-visit forms. In a guide to CDS implementation that we published in 2005,11 we suggested that different types of CDS presentation, applied at different parts of the visit workflow, can have very different effects depending on what it is that one is trying to encourage the physician to do. Moreover, communication through group academic detailing(used in this study) may not be the best strategy to educate and change the behavior of clinicians regarding the concepts behind clinical alerts.12,13

To achieve a specific clinical objective by using a CDS intervention, one must consider whether the communication and acceptance groundwork has been laid to maximize the intervention’s impact, and also consider what type of CDS, applied when in the encounter, is likely to have the greatest impact.14 From the aforementioned CDS guidebooks11,14 and other published reviews of CDS effectiveness factors,15,16 we support the following list of questions to consider before the implementation of any real-time, point-of-care CDS intervention designed to interrupt clinicians during their work.

Communication and acceptance:

  • 1. Has the clinical rule or concept that will be promoted by the intervention been well communicated to the medical staff in advance?
  • 2. Does the intervention, if accepted, change the overall plan of care, or is it intended to cause a limited, corrective action (such as preventing an allergic reaction to a drug)?
  • 3. Are the data used to trigger the alert likely to be accurate and reliable, and are they a reliable indicator for the condition you are trying to change?
  • 4. What is the likelihood that the person receiving the alert will actually change his or her patient management as a result of the alert?
  • 5. Is the patient likely to agree that the recommended actions are beneficial?

Intervention technique:

  • 6. Is an alert the right type of intervention for the clinical objective, and is it presented at the right time?
  • 7. Is the intervention presented to the right person?
  • 8. Is the alert presented clearly, and with enough supporting information, so that the clinician feels confident in taking the recommended action immediately?
  • 9. Does the intervention slow down the workflow?
  • 10. Is the overall alert burden excessive (“alert fatigue”)? Were the study providers receiving other types of alerts at the same time?
  • 11. Is the clinical information system, including the use of CDS (eg, the alerts), well-liked and supported by clinicians in general?


  • 12. Is there a way to monitor the response to the alert on an ongoing basis?

Real-time, point-of-care CDS interventions can be highly effective if the right intervention for the desired clinical objective is used, if the recommendation has been accepted clinically by the physician and patient, if the alert is accurate and clearly understood, if it is presented at a point in the encounter at which the physician can confidently take action on it, and if it makes it easy for the physician to take such action without prolonging or confusing the workflow.

Improving clinical quality objectives through CDS, such as increasing influenza vaccination rates, can be substantially improved by using a systems perspective to address aspects of communication, medical acceptance, clinical workflow, choice of computerized display, and ongoing monitoring. Taken together such extensive interventions can lead to better health outcomes for our patients.


Opinions expressed in these commentaries are those of the authors and not necessarily those of the American Academy of Pediatrics or its Committees.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.


1. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR. Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med. 2009;169(2):108–114. [PubMed]
2. Kaushal R, Jha AK, Franz C, et al. Brigham and Women’s Hospital CPOE Working Group. Return on investment for a computerized physician order entry system. J Am Med Inform Assoc. 2006;13(3):261–266. [PMC free article] [PubMed]
3. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167(13):1400–1405. [PubMed]
4. Congressional Budget Office. Evidence on the costs and benefits of health information technology. [Accessed May 10, 2009]. Available at:
5. Fiks AG, Hunter KF, Localio AR, et al. Impact of electronic health record–based primary care clinical alerts on influenza vaccination for children and adolescents with asthma: a cluster-randomized trial. Pediatrics. 2009;124(3):159–169. [PubMed]
6. Singh H, Arora HS, Vij MS, Rao R, Khan MM, Petersen LA. Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc. 2007;14(4):459–466. [PMC free article] [PubMed]
7. Gerard MN, Trick WE, Das K, Charles-Damte M, Murphy GA, Benson IM. Use of clinical decision support to increase influenza vaccination: multi-year evolution of the system. J Am Med Inform Assoc. 2008;15(6):776–779. [PMC free article] [PubMed]
8. Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993;94(6):619–625. [PubMed]
9. Hak E, Hermens RP, Hoes AW, Verheij TJ, Kuyvenhoven MM, van Essen GA. Effectiveness of a co-ordinated nation-wide programme to improve influenza immunisation rates in the Netherlands. Scand J Prim Health Care. 2000;18(4):237–241. [PubMed]
10. Tang PC, LaRosa MP, Newcomb C, Gorden SM. Measuring the effects of reminders for outpatient influenza immunizations at the point of clinical opportunity. J Am Med Inform Assoc. 1999;6(2):115–121. [PMC free article] [PubMed]
11. Osheroff JA, Pifer EA, Teich JM, Sittig DF, Jenders RA. Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. Chicago, IL: Health Information and Management and Systems Society; 2005.
12. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976;295(24):1351–1355. [PubMed]
13. Simon SR, Smith DH, Feldstein AC, et al. Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. J Am Geriatr Soc. 2006;54(6):963–968. [PubMed]
14. Osheroff JA, editor. Improving Medication Use and Outcomes With Clinical Decision Support: A Step-by-Step Guide. Chicago, IL: Health Information and Management Systems Society; 2009.
15. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523–530. [PMC free article] [PubMed]
16. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW. Effects of computerized physician order entry on prescribing practices. Arch Intern Med. 2000;160(18):2741–2747. [PubMed]