During the 21-month period, 629 injured patients with physiologic compromise were identified, enrolled, and processed using manual data processing. Case ascertainment using electronic methods yielded 3,008 injured patients meeting the same inclusion criteria during the same time period. Four hundred eighteen patients matched between the two data processing groups and formed the primary cohort for comparison (). While electronic data processing yielded almost five times the number of subjects meeting inclusion criteria, there were a portion of patients (n = 35) who were missed by electronic processing, but captured by manual case ascertainment. An additional 211 patients in the manual processing group did not match to a record from the electronic group.
Schematic of patients included in manual versus electronic data processing.
Clinical, operational, procedural, and outcome variables are described for the various matched and unmatched groups in . Patients in the first three columns represent the manual data processing group (matched and unmatched to electronic cases), while those in the last column were only identified by electronic processing (the electronic-only group). In general, cases identified by manual methodology tended to have greater physiologic compromise (e.g., lower GCS, higher percentage of field intubations) and worse prognosis (e.g, higher mortality) than patients identified solely by electronic methods, although this was not universal in all groups. The median out-of-hospital time values fluctuated between groups, but overall were comparable. Mortality was lower in the electronic-only group (16%, 95% confidence interval [CI] = 14% to 18%) compared to the matched sample (22%, 95% CI = 18% to 27%) and the unmatched manual sample (36%, 95% CI = 29% to 43%), and was similar to cases identified solely by manual processing (13%, 95% CI = 4% to 31%). When the columns in are collapsed into complete electronic (n = 3,008) and manual (n = 629) cohorts, overall mortality in the electronic vs. manual cohorts was 18% (95% CI =16% to 20%) vs. 27% (95% CI = 23% to 31%).
Clinical, operational, procedural, and outcome information for the different manual and electronic data processing samples.*
Clinical, operational, procedural, and outcome variables (including the proportion of missing values) among matched patients who underwent both data processing strategies are compared in . Overall, there were very similar characteristics generated from both data processing approaches. However, there was a higher proportion of missing hospital outcomes with electronic data processing (87 out of 418, 21%, 95% CI = 17% to 25%) compared to the manual approach (11 out of 418, 3%, 95% CI = 1% to 5%). In addition, four patients identified as dying during their hospital stays with manual chart review were listed as survivors with electronic data processing methods.
Clinical, operational, procedural, and outcome information for the matched sample, separated by manual versus electronic data processing (n = 418).*
There was good agreement and validity between the two data processing approaches (). For categorical variables, kappa values ranged from 0.76 (intravenous line placement) to 0.97 (intubation attempt), with exact agreement from 67% to 99%. The intraclass correlation coefficient (ICC) for continuous terms ranged from 0.49 (response interval) to 0.97 (transport and total out-of-hospital intervals), and tended to be higher for variables measured throughout the out-of-hospital time period as opposed to single (i.e., initial) time points. The median difference was zero for all continuous variables, with all but two terms having an interquartile range (IQR) of zero for these differences. In-hospital mortality agreed exactly in 99% of cases (kappa 0.96), while hospital length of stay agreed exactly in 62% of cases (ICC 0.56).
Measures of agreement and validity for clinical, operational, procedural, and outcome information between manual versus electronic data processing (n = 418).*
There was some evidence of heteroscedasdicity among 5 of the 15 ordinal and continuous variables, as assessed by regressing differences against averaged values. The coefficients for these variables (initial respiratory rate 0.20, p = 0.01; initial heart rate −0.20, p = 0.008; lowest heart rate 0.24, p = 0.002; response interval −0.40, p < 0.0001; and length of stay 0.56, p < 0.0001) did not suggest a systematic over- or under-estimation of values for electronic data processing. The 10 remaining variables did not demonstrate statistical evidence of heteroscedasdicity (all p ≥ 0.20).
shows Bland-Altman plots for initial and lowest field sBP. There was less variability (as quantified by the 95% interval of differences) for the “lowest” values compared to initial values. Similar plots for additional clinical (GCS), operational (total out-of-hospital time), and outcome (hospital length of stay) measures are included in , and , respectively. Differences in GCS suggest that the most consistent agreement occurred at the ends of the GCS spectrum (particularly for initial GCS), and that there was improved agreement for the “lowest” GCS (as indicated by a narrower 95% interval of values). For total out-of-hospital time, most values clustered on the zero difference line, but those that differed tended to be under-estimated by electronically processed time values. Hospital length of stay had the lowest exact agreement (62%), with eight notable outlier values (including the single omitted 365 day outlier) that substantially increased the 95% interval of differences. Two-by-two tables for field procedures (intravenous line placement, intubation) and outcomes (mortality) and are included in .
Figure 2 Bland-Altman plots of field systolic blood pressure between electronic and manual data processing.*
Figure 3 Bland-Altman plots of field Glasgow Coma Scale (GCS) score between electronic and manual data processing.*
Figure 4 Bland-Altman plot of total out-of-hospital time interval (in minutes) between electronic and manual data processing (n = 354).*
Figure 5 Bland-Altman plot of hospital length of stay (in days) between electronic and manual data processing (n = 288).*
Figure 6 Two-by-two tables comparing electronic and manual data processing values for field interventions (intravenous line placement, intubation) and outcome (mortality).*