We were able to successfully complete interviews with knowledgeable individuals for nine commercially available clinical information systems. The systems included are listed in . These systems represent a broad cross section of the inpatient and outpatient electronic medical record markets and include most of the major systems in both markets. All the systems included in the analysis were certified by the CCHIT. Based on data from HIMSS Analytics, these systems have a collective market share of 76% of the nonself-developed EMR market in the United States. 25
To protect the confidentiality of vendors, many of whom consider their product capabilities to be sensitive, our results are presented pseudonymously. We have identified the vendors that are included in our study, but the results are presented using code numbers (note that the results across sections and tables use consistent code numbers, so, for example, “System 3” in is the same system as “System 3” in . Note that there are two systems from GE and two systems from McKesson. These are actually distinct systems from acquisitions (GE acquired IDX in early 2006 and McKesson acquired Practice Partner in early 2007).
Table 3 Availability of Triggers
Table 4 Availability of Input Data Elements
Triggers are critical to providing event-driven, action-oriented, real-time clinical decision support and represent the initiating condition for a decision support intervention. shows the results of our analysis for triggers. All the triggers in the taxonomy were widely supported, with many of them being supported by all nine systems. One system was scored N/A for the “outpatient encounter opened” trigger because it was an inpatient-only system. Four systems were scored N/A for the hospital admission trigger because they are outpatient only systems. Two systems were unable to trigger decision support logic based on the entry of a new problem. In our earlier analysis, this trigger was mainly used to initiate care protocols and data entry forms (i.e., requesting information on severity or initiating a management plan when asthma is added to the problem list). Likewise, two systems were unable to trigger decision support based on the entry of weight, which is used for retrospective weight-based dosing checks (i.e., rechecking dose appropriateness when a new weight is entered for an infant). One system was unable to trigger based on storage of a new laboratory result—this was the second most commonly used trigger at Partners (responsible for triggering 998 rules) and is critical for panic laboratory value detection as well as detection of many adverse drug events. Likewise, one system was unable to trigger decision support based on the entry of an allergy (used for retrospective drug-allergy interaction and cross-sensitivity checks).
These omissions aside, six of the nine systems offered all the possible triggers (save for ones assessed as not applicable). System 2 missed a single trigger and System 3 missed two. System 8 offered only four of the nine triggers (with one not applicable and four missing).
Input Data Elements
Nearly all decision support rules require patient-specific data to make their inferences. shows the availability of the various data elements in the taxonomy in the nine systems. As with triggers, the four outpatient-only systems 2,4,5,9
were not rated on the “hospital unit” or “reason for admission” data elements. Seven of the fourteen data elements (laboratory result/observation, drug list, hospital unit, age, gender, allergy list, and weight) were available in all the information systems to which they were applicable. The other input data elements (diagnosis/problem, nondrug orders, family history, surgical history, reason for admission and prior visit types) were each missing from two systems. Most of these data elements were rarely used in the Partners knowledge base, but the problem/diagnosis input data element was used by 1,587 rules (particularly preventive care reminders which are often condition-specific, as in retinopathy screening for diabetic patients).
The system-by-system performance was quite variable. Five of the nine systems had no missing capabilities. Two systems missed only a single capability. However, System 3 missed five capabilities and System 8 missed six.
Triggers and input data elements represent the input arm of decision support. Interventions, by contrast, are efferent. The best decision support systems tailor their interventions based on the severity of clinical situation and the user's workflow, 1,17
so offering a broad palette of interventions is important. shows the availability of the various intervention types in the nine systems. As with triggers and input data elements, most systems supported most interventions. The most basic intervention type is notification (which might take the form of a pop-up, alert, telephonic page or e-mail among other possibilities) and, not surprisingly, all nine systems support notification. The ability to collect freetext in response to an alert or to show a data entry template was also universal. Only one system was unable to provide decision-support-informed defaults or pick lists. Likewise, only a single system was unable to perform logging in response to a decision support intervention. Two systems lacked the ability to show a guideline to a user and three were unable to seek approval in response to a decision (for, say, a high-cost therapy or restricted-use antibiotic).
Table 5 Availability of Interventions
Six of the nine systems offered all possible interventions, disregarding those assessed as not applicable. The same three systems that missed triggers also had missing interventions. Systems 2 and 8 missed two interventions, while System 3 missed 3 interventions.
The final axis of the taxonomy is the offered choice, shown in . Such choices are usually offered alongside a notification, as in the digoxin example given in the background section. Performance on the offered choice axis was much lower than for the other three dimensions of the taxonomy. Of the twelve offered choices, only three (override rule/keep order, cancel current order and enter weight, height, or age) were available in all nine systems. Three of the offered choices (defer warning, edit existing order, and set allergies) were available in less than half the systems.
Table 6 Availability of Offered Choices
Only a single system (System 5) supported all of the offered choices. Six of the nine systems had at least three gaps, while the worst-performing system (System 3) had a total of eight missing choice capabilities.
shows the number of capability deficiencies for each system by category. No system had all forty-two capabilities, although a single system (System 5) was missing only a single capability. Two systems had three gaps and two had four. The sixth best system had six gaps and the seventh had nine. The two worst-performing systems had eighteen gaps each: in other words, they were each missing 43% (18 of 42) of the decision support capabilities in the taxonomy.
Table 7 Deficiencies by System (Count of Capabilities Lacking in Each System Across the Four Axes)