We abstracted 352 charts from the 718 eligible subjects in the initial control period (Phase 1) and 104 charts from the 538 eligible subjects in the postintervention control period (Phase 3) (). By definition, all of these charts were handwritten. The EDECS system was used for 239 (64 percent) of the 374 patients cared for during the intervention period (Phase 2); the other 135 charts were handwritten. One hundred eighty-five different physicians each had primary responsibility for at least one of the 830 patients in the study group. One hundred forty-one physicians participated in one phase of the study, 35 physicians in the intervention phase and one of the two control phases, and 9 physicians in all three phases. Fifty percent of physicians saw one or two study patients; the average was 4.5 patients per physician, and the range was 1 to 32. Emergency medicine residents were the primary providers in 62 percent of all cases, pediatric residents in 30 percent, family medicine residents in 5 percent, and emergency department faculty in 3 percent. Pediatric interns and residents saw many more patients in Phase 1 (55 percent of all Phase 1 cases) than in the other phases, when emergency department residents saw 85 percent of patients. The average postgraduate year (PGY) increased throughout the study, because emergency medicine trainees are PGY 2-4, whereas pediatric trainees are PGY 1-3. Patient age and gender distributions were similar among phases; however, Phase 3 cases were of higher complexity than those of Phases 1 and 2.
Demographic and Clinical Features of Events, Patients, and Physicians, by Phase of Study
Seventy-six physicians cared for at least one of the 374 eligible patients during the intervention phase; the majority saw fewer than three patients (). Thirty-two percent of these 76 physicians never used EDECS, and 26 percent used it in every case (). The cases of the 25 percent of physicians who each saw more than five cases accounted for 64 percent of all cases seen with EDECS. This observation and the U-shaped distribution of the results suggest that EDECS use was not random; some physicians gravitated toward it while others avoided it. Emergency physicians used EDECS in 68 percent of eligible cases, whereas others used EDECS in 41 percent of eligible cases (difference, 27 percent; 95 percent CI, 14-40 percent). Seven of 44 (16 percent) emergency physicians and 17 of 32 (53 percent) other physicians (difference, 37 percent; 95 percent CI, 17-58 percent) never used EDECS.
Number of patients seen by each physician during the intervention phase (Phase 2). The total number of patients was 374.
Number of physicians using EDECS in each indicated percentage of cases.
Documentation was higher in Phase 2 than in Phase 1 for 20 of 21 essential history and physical examination items, and was higher in Phase 2 than in Phase 3 for all 21 items (). Overall documentation of the 21 items was 80 and 74 percent for control phases 1 and 3, respectively, and 92 percent for the intervention phase (Phase 2-3 difference, 13 percent; 95 percent CI, 10-15 percent). Documentation of handwritten charts was fairly constant across phases (80, 78, and 74 percent, respectively), and the combination of 78 percent documentation on the 135 handwritten Phase 2 charts and 100 percent documentation on the 239 EDECS charts produced the overall documentation percentage (92 percent). Thus, the use of EDECS in just over two thirds of eligible cases was sufficient to produce significant improvements in overall documentation for Phase 2.
Documentation of Medical History and Physical Examination Items
While EDECS stimulated the doctors to create more complete charts, it did not always change behavior in ways that reliably produced a more complete physical examination. For example, the percentage of ear examinations that included insufflation (blowing a puff or air toward the ear drum to see if it moves) was 15 percent in Phase 1, 22 percent in Phase 2 (10 percent on handwritten charts, 30 percent on EDECS charts), and 5 percent in Phase 3 (Phase 1-2 difference, 8 percent; 95 percent CI, 2-14 percent). Since EDECS encouraged (but did not mandate) the use of this technique, the three-fold improvement in compliance is notable; it demonstrates the power of real-time computer reminder systems. The failure to approach 100% compliance, however, highlights the difficulty of achieving universal, voluntary implementation of a practice guideline.
Percentage documentation of each desirable item in the after-care instructions was significantly higher in Phase 2 than in the other phases (). Overall documentation for the ten items was 48 percent in Phase 1, 81 percent in Phase 2, and 50 percent in Phase 3 (Phase 1-2 difference, 33 percent; 95 percent CI, 28-38 percent). The percentage documentation in handwritten after-care instructions remained constant across the three phases (48, 48, and 50 percent, respectively), and the 81 percent documentation in the intervention phase results from a blending of 100 percent documentation on 229 computer after-care instructions with 48 percent documentation on 129 handwritten charts from this phase. Covariates used in the logistic regressions included physician specialty, years of training, case complexity, and a dummy variable indicating whether the treating physician contributed six or more cases to the study. The inclusion of any combination of these did not change the magnitude and significance of findings regarding documentation of history, physical examination, and after-care instructions.
Documentation of After-care Instructions
Otitis media was the sole diagnosis in 24 percent of cases; viral syndrome was diagnosed in 52 percent of cases. Sepsis, fever without source, and all other diagnoses each accounted for less than 5 percent of cases, precluding any meaningful analysis of the effect of EDECS on these conditions. The percentage of cases diagnosed as otitis media declined from Phase 1 to Phase 2 and from Phase 2 to Phase 3. Patients seen with EDECS were less likely to be given this diagnosis (odds ratio, 0.38; 95 percent CI, 0.19-0.75), even after adjustments were made for phase of the study (secular trend), physician specialty, case complexity, and number of cases seen by physician (). The guideline and the EDECS program required physicians to document at least two abnormalities (of four separate ear examination variables) to justify the diagnosis of otitis media. The reduction in otitis media diagnoses when the EMR was used was predominantly due to a decrease in the number of cases of unsubstantiated otitis media diagnoses. In only two cases did physicians override the EDECS suggestion and make the diagnosis of otitis media in the absence of two findings. Use of EDECS increased the frequency of viral syndrome diagnoses; the magnitude of the increase mirrored the decrease in the frequency of otitis media diagnoses.
Percentage of cases diagnosed as otitis media by physical examination findings. The black portion of each column indicates diagnoses supported by physical findings; the white portion indicates diagnoses not supported by physical findings.
Across all patients, the use of diagnostic tests was deemed appropriate in the following percentage of cases: complete blood count, 84 percent; chest radiography, 70 percent; blood culture, 83 percent; urinalysis, 81 percent; urine culture, 84 percent; and lumbar puncture, 86 percent. There were no important differences in utilization or appropriateness of diagnostic tests among phases or between EDECS and handwritten charts in Phase 2. The decision to use or with-hold oral antibiotics was made appropriately in 96 percent of all cases, assuming the final diagnosis was correct. Fifty-eight percent of children were discharged home on oral antibiotics.
Intramuscular ceftriaxone, which at the time of the study was indicated only in fever without a source, was given to 10 percent of the patients discharged home. In only 35 (44 percent) of the 80 administrations was there evidence of this diagnosis. There was, however, no difference in rate or appropriateness of use among phases or between handwritten and EDECS charts.
Median total charges were similar in the three phases ($216, $216, and $222, respectively). Mean phase 3 charges ($635) were higher than charges in the other phases (Phase 1, $357; Phase 2, $387); however, this difference disappeared when charges were adjusted for patient complexity. The percentage of charges attributed to physician, facility, laboratory, and treatment remained consistent across phases, as did the appropriateness of these charges regardless of whether charges or log-charges were used.