The randomized, controlled trial ran for 30 weeks, beginning in October
1992. There were 6 different housestaff rotations during the 30 week period,
with 6 teams of faculty and housestaff per rotation.
Six physicians were excluded from the study because they received fewer
than five suggestions about corollary orders. This cutoff was chosen by
inspection of the distribution of number of suggested consequent orders. These
were mostly off-service physicians who covered night calls for one or two
nights but were not part of teams assigned to a service. A total of 86
housestaff physicians received more than 5 suggestions about corollary orders
during the study: 45 intervention physicians and 41 control physicians. Nine
physicians changed study status when they returned for a second rotation
during the study. For these physicians we only included data for the rotations
before they changed study status.
During the study, the intervention and control physicians cared for 2,181
different patients during 2,955 different admissions.
shows the demographic
and clinical characteristics of these patients. No significant differences
between intervention and control patients exist for any of these
Demographic Characteristics of Study and Control Patient Groups
Of these 2,181 patients, 1,686 (77.3%) had at least 1 order written (814
intervention patients and 872 control patients) that would trigger a
suggestion for a corollary order. In all, intervention and control physicians
entered 7,394 trigger orders which resulted in 11,404 suggestions for
corollary orders. On average, a trigger order generated suggestions for 1.5
corollary orders. Trigger orders made up 9.6% of all orders written for the
2,181 patients. Patients with at least 1 suggested corollary order per
admission had an average of 6.8 such suggestions per admission.
The effect of the computer suggestions was very strong, whether measured as
immediate, 24-hour, or hospital stay compliance. Intervention physicians
ordered the corollary orders required by our guidelines twice as often as
control physicians did, when measured by immediate compliance (46.3% versus
21.9%, p < 0.0001). Significant differences between study and control
physicians also appear in 24 hour compliance (50.4% vs 29.0%, p < 0.0001)
and hospital-stay compliance (55.9% vs 37.1%, p < 0.0001). Because
corollary orders for saline lock had such a large effect and are the least
significant clinically, we repeated the simple analyses excluding saline lock
orders and found immediate compliance was 46.4% vs. 27.6% (p < 0.0001),
24-hour compliance was 50.9% versus 35.3% (p < 0.0001) and hospital-stay
compliance was 56.0% vs. 43.5% (p < 0.0001). The effects were almost
identical whether measured on all of the data using a complicated GEE model or
measured as first occurrence compliance using a simple Student's t test. The
mean immediate compliance to the first occurrence of a suggestion was 48%
among intervention physicians and 23% among control physicians (p < 0.001).
The other first compliance scores and the significance levels were also very
close to their GEE counterparts.
is a histogram
comparing the 24 hour compliance of study and control physicians. There is
little overlap between the study and control populations. Several control
physicians had compliance rates below 20%, and no control physician reached a
compliance rate greater than 50%. On the other hand, study physicians all
maintained compliance rates of at least 30%, and some reached levels of
Histogram of individual physician 24-hour compliance.
There is very little difference between the immediate and 24-hour
compliance scores, indicating that corollary orders that are not written at
the same time as their trigger order are unlikely to be written later during
the same day.
The difference in the compliance scores of intervention and control
physicians shrinks by almost one fifth from immediate to hospital stay
compliance. This results from a greater increase in the control compliance.
Nonetheless, a large difference (18 percentage points) separates the
compliance scores of intervention and control physicians even when measured as
hospital stay compliance.
Breakdowns of compliance by trigger and corollary order illustrate the
kinds of items the intervention affected most extensively.
shows the 24-hour
compliance scores for intervention and control physicians broken down by the
25 most common trigger orders. shows comparable data broken down by the 25 most common
corollary orders. In both cases, the top 25 orders account for more than 80%
of the suggestions provided.
24-Hour Compliance Rate by Triggering Order for 25 Most Common Triggering
24-Hour Compliance by Triggering Order for 25 Most Common Corollary
The effect of the intervention varied by specific trigger-corollary order
pair. Computer reminders increased adherence to guidelines concerning many
important corollary orders. For example, they increased 24-hour compliance for
monitoring serum levels of gentamicin, vancomycin (though the value of
monitoring is debatable), and theophylline by 9, 26, and 24 percentage points
respectively. Differences persisted when hospital compliance was assessed. We
were surprised by these results because we had assumed that most physicians
were already complying fully with guidelines about antibiotic and theophylline
The reminders also caused large improvements in compliance with suggestions
to order prothrombin times after coumadin dosage changes, APTT after heparin
dose changes, baseline creatinines before vancomycin and aminoglycoside
antibiotics, and radiographs to check for line placement and lung status
during mechanical ventilation. The difference between intervention and control
compliance rates for these suggestions was as much as 25 percentage points. On
the other hand, computer suggestions to order baseline creatinine measurements
before starting administration of cimetidine or ranitidine had no effect. In
retrospect, we considered this a possible appropriate response to a guideline
with only a theoretic basis.
Pharmacists made 105 interventions with intervention physicians and 156
with control physicians (two-tailed p = 0.003) for errors considered to be
life threatening, severe, or significant.
There was no difference in maximum serum creatinine levels between the
groups (1.51 ± 1.25 for intervention patients versus 1.42 ± 0.88
for controls; p = 0.28).
Length of stay and total inpatient charges were not different for
intervention patients compared with control patients. The average length of
stay was 7.62 days for intervention patients and 8.12 days for control
patients, a difference of -0.5 days (95% confidence interval of the difference
is -0.17 to 1.19; p = 0.94). Average hospital charges were $8,073.52 for
intervention patients and $8,589.47 for control patients, a difference of
-$515.95 (95% confidence interval of the difference is -$828.41 to $1,316.85;
p = 0.68).
An increase in charges might have been expected, since the aim of all the
reminders was to increase the utilization of the suggested order items.
However, the variance and confidence intervals of charges and length of stay
are too large to conclude anything from these results.