In this section, we present estimates of the experience premium in EMS performance, as measured by decreased out-of-hospital and on-scene durations, by estimating equation (1)
Our models are estimated by OLS cross-sectionally, with firm-by-municipality fixed effects, and with paramedic fixed effects. Standard errors are clustered at the paramedic level to allow for correlation in performance across incidents within paramedic.9
In the first row of , we document a relationship between recent (quarterly) EMT volume and performance that implies that a one standard deviation increase in the number of trauma runs accumulated by an individual paramedic over a given quarter is associated with a reduction of approximately 35 (10) seconds in total out-of-hospital (on-scene) time. Coefficient estimates are statistically significant at the 1 percent level.
There is a strong tie between retention and learning measured at the firm level, as high labor turnover rates may result in a loss of accumulated experience and hence lower productivity. The short timeframe and, for the most part, lack of individual identifiers led previous studies (e.g., Thiemann et al. 1999
; Peterson et al. 2004
; Marcin et al. 2007
) to focus on learning at the firm level, which confounds individual learning and spillover and turnover effects, with no ability to separate between these three vastly different mechanisms. Our data allows us to follow paramedics over a decade and a half and hence identify the contribution of these three channels to overall performance.
According to Rosen (1972)
, “learning by experience suggests that the production of knowledge is at least partially acquired through the productive process itself.” An additional EMS run adds to both the individual paramedic and the firm's stock of experience. Firm experience represents the extent to which individual experience spills over to other members of the firm. This calls for a specification in which recent firm volume is added to the one capturing recent paramedic volume.
The results, reported in the second and third rows of , reveal magnitudes for returns to paramedic volume similar to those in the first row. However, the effect of firm volume is of small magnitude as well as varying signs and degrees of significance for both on-scene and total out-of-hospital times. This is likely due to the limited scope for spillover in tasks performed in teams that rarely exceed two individuals.
To allow for heterogeneity in learning, we stratify the analysis by paramedic tenure under the hypothesis that paramedics of different tenures might exhibit different gains to recent volume. The third panel of reports estimates of models in which paramedic volume is interacted with four indicators of tenure quartiles. The quartiles correspond to <2, 2–4, 4–6.2, and over 6.2 years as a paramedic, respectively.
We find that the benefits of recent volume accrue differentially across tenure groups, with small gains in the first two quartiles of the tenure distribution. Paramedics with 6.2 years of tenure and above enjoy about 85 percent higher returns to volume than in the pooled regression for on-scene time. The analogous figure for total out-of-hospital time is 55 percent. This result is somewhat puzzling given the existing evidence that learning is stronger at the beginning of individuals' careers. Our results are consistent with the notion that recent volume affects performance more with greater exposure to trauma scenes and/or with the idea that more senior paramedics have greater influence over the various aspects of EMS delivery as well as the management of patients at the scene (e.g., less reliance on medical direction and better management of other paramedics). Alternatively, composition within tenure quartiles may account for observing learning only among long-tenured paramedics. While the upper tenure quartiles include only paramedics with relatively long job duration, paramedics with low tenure represent a mixture of individuals who are either in the beginning of a longer career in EMS or who do not stay in the profession very long. Therefore, our results are consistent with an explanation whereby short job duration reflects insufficient learning due to lack of either ability or effort.10
While existing volume–outcome studies are almost exclusively focused on providers of tertiary services (e.g., cardiac surgeons) where career durations are extensive, the data used typically span only 2–5 years, accommodating only recent volume effects. Our data covers 15 years of professionals whose average career duration is relatively short and therefore allows us to examine an alternative view of the volume–outcome relationship, in which the benefit accrues to cumulative rather than recent volume. The lower panel of reports parameter estimates from models in which the variable of interest is defined as the log of cumulative volume. The log transformation is used to account for the skewed nature of the cumulative volume distribution across paramedics, and it is commonly employed in the volume–outcome literature.11
Since we do not observe the full history of runs for paramedics who appear at the very beginning of the sample, we perform the analysis separately for all 2,010 paramedics (lower panel) and for 1,726 paramedics who where certified (for the first time) after 1991 (upper panel).12
The coefficient estimates for all paramedics are similar in magnitude to the ones for uncensored paramedics, suggesting that experience accumulated before 1991 has little or no relevance by 2001.13
These results allow us to compute the value of retention in performance units. More specifically, we compute the increase in time spent on scene that results from replacing the average paramedic in our data with a new one as follows:
is the coefficient estimate for log cumulative volume,
is the average cumulative volume across paramedics, and v1
is the average quarterly volume across paramedics. Equation (2)
computes the difference in average out-of-hospital (on-scene) times within a typical quarter between the average paramedic and a new one. The reductions in total out-of-hospital and on-scene times associated with the retention of the typical paramedic in our sample for an additional quarter are approximately 4 and 2 minutes, respectively.
plots the average monthly difference in total out-of-hospital and on-scene times between an experienced paramedic and a new one. The first period represents the hypothetical month of replacing the average paramedic (in volume terms) in our sample with a new one. We then use equation (2)
, where v1
is defined at the monthly level, for the subsequent 36 months. The figure describes the loss in total out-of-hospital and on-scene times from the replacement as we follow the new paramedic and compare her average performance to that of the experienced paramedic (had she not exited), therefore allowing for learning by doing by both paramedics. While the cost of turnover is higher in the first months following replacement, differences in performance are (slowly) exhausted over time. Here again, these results do not appear to be driven by attrition.
Cost of Replacing an Average Paramedic with a New One, by Month (Measured in Additional Minutes of Prehospital Time)
In the right most column of , we implement a falsification exercise by estimating all models described above, in which we replace the dependent variables, total out-of-hospital and on-scene times, with an alternative marker of system performance, dispatch time. Dispatch time is defined as the length of time between a 9-1-1 call and the moment paramedics are notified and dispatched to the scene. This measure provides the basis for a credible falsification test as, unlike time spent on scene or out of hospital, paramedics have no influence on it. If learning is the mechanism through which greater volume yields shorter on-scene and prehospital times, then we should not find a relationship between paramedic experience and a margin over which they have no influence, such as dispatch time. The results confirm this intuition, lending credibility to our results.
In , we explore the possibility that the performance benefits of experience operate in dimensions other than just mean prehospital times. Specifically, we posit that paramedic volume might shrink the conditional distributions of total out-of-hospital and on-scene duration. To this end, we employ quantile regression methods, which serve to describe how independent variables affect the entire distribution of the dependent variable, as opposed to just its mean. In this analysis, quantile regressions were estimated cross-sectionally and with firm-by-municipality fixed effects at seven different percentiles (0.05, 0.15, 0.25, 0.5, 0.75, 0.85, and 0.95).14
The results indicate that greater recent paramedic volume compresses the upper tail of the conditional on-scene time distribution.15
For instance, according to the firm-by-municipality fixed effects specification, a one standard deviation increase in the number of trauma runs accumulated by an individual paramedic over a given quarter reduces the 95th percentile of total out-of-hospital (on-scene) time by 38 (20) seconds, which is slightly more than double the median and mean effects.
Effect of Paramedic Experience on Quantiles of Performance
Interventions on scene, injury profile, trauma characteristics, and patient demographics are used, in part, to proxy for patient severity. While these may reflect severity only to a limited extent, concerns regarding omitted variables are not likely to be important given the current EMS system design. It is difficult to argue for a correlation between severity and experience due to the fact that dispatching for ALS incidents is determined by proximity to the scene and not by EMT reputation. However, even if EMS did match paramedic experience with indications of acuity, it is unlikely that more time-consuming, higher severity incidents would result in the least experienced crews being dispatched, which is the only mechanism that would account for our results. If there is matching between highly experienced EMTs and patient severity, our results underestimate
the true reduction in prehospital time due to an increase in experience.16
Finally, while our regressions control for indicators of prehospital interventions, one might worry about selection on the complexity of procedures performed by paramedics. For example, if inexperienced paramedics choose simpler procedures, which require relatively fewer minutes, we might infer that less experience results in shorter prehospital times conditional on procedures performed. This would lead us to underestimate the magnitude of the experience premium.