PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of hsresearchLink to Publisher's site
 
Health Serv Res. Jun 2009; 44(3): 902–925.
PMCID: PMC2699914
Retention, Learning by Doing, and Performance in Emergency Medical Services
Guy David and Tanguy Brachet
The Wharton School, University of Pennsylvania, 202 Colonial Penn Center, 3641 Locust Walk, Philadelphia, PA 19104
The Children's Hospital of Philadelphia (CHOP), the Department of Anesthesiology and Critical Care at CHOP, and the University of Pennsylvania School of Medicine, Philadelphia, PA
Address correspondence to Guy David, Ph.D., Assistant Professor of Health Care Management, The Wharton School, University of Pennsylvania, 202 Colonial Penn Center, 3641 Locust Walk, Philadelphia, PA 19104; e-mail gdavid/at/wharton.upenn.edu. Tanguy Brachet, Ph.D., Assistant Professor is with the Center for Outcomes Research at The Children's Hospital of Philadelphia (CHOP), the Department of Anesthesiology and Critical Care at CHOP, and the University of Pennsylvania School of Medicine, Philadelphia, PA.
Objectives
To examine the strength of the volume–outcome relationship among paramedics, a group of providers that has not been previously studied in this context. By identifying the effects of individual learning on performance, we also assess the value of paramedics' retention. The prehospital emergency medical services (EMS) setting allows us to interpret any volume–outcome relationship as learning by doing, uncontaminated by reputation-based referrals because ambulance units are dispatched based on proximity.
Data Sources
Incident-level EMS data spanning 1991 to 2005 from the Mississippi Emergency Medical Services Information System collected by the Mississippi Department of Health.
Research Design
Using linear and quantile methods with and without provider fixed effects, we estimate the relationship between experience accumulation and performance using the universe of trauma incidents involving injured patients (including motor vehicle crashes, falls, stabbings, and shootings).
Principal Findings
We find that greater individual volume is robustly related to improved performance. In addition, we find that the benefit of learning operates through both recent and past experiences, accrues differentially across tenure groups, and operates on both mean performance and the upper quantiles of the performance distribution.
Conclusions
Persistent past and current volume effects suggest that policy and managerial implications in EMS should be directed at retention efforts to take advantage of individual learning by paramedics.
Keywords: Volume–outcome, learning by doing, emergency medical services
While there is considerable evidence in the case of hospitals and surgeons that high volume is associated with better patient outcomes across a variety of medical conditions (Luft, Hunt, and Maerki 1987; Tu, Austin, and Chan 2001; Birkmeyer et al. 2002, 2003; Halm, Lee, and Chassin 2002; Elixhauser, Steiner, and Fraser 2003; Gandjour, Bannenberg, and Lauterbach 2003; Huckman and Pisano 2006; Marcin et al. 2007), to date no study has examined this relationship in a prehospital setting.
Emergency medical services (EMS) networks respond to, stabilize, and transport trauma patients involved in situations, such as automobile accidents, injuries from falls, and violence. EMS is a crucial part of the health care system and the public's emergency medical safety net. However, more than three decades after Congress passed the Emergency Medical Services Systems Act of 1973, trauma victims are treated by EMS with little or no evidence that the care they receive is optimal (Delbridge et al. 1998).
There is little research addressing the effectiveness of EMS practice (Pointer 2001) and there are even fewer studies examining the effect of experience accumulation, as proxied either by certification levels (Pollock, Brown, and Dunn 1997) or tenure (Soo et al. 1999) on performance. Our study is the first to use detailed data on experience accumulation in addition to certification levels and tenure to document the performance gains from higher volume.
Learning by doing is one channel through which skills are acquired and therefore provides a potential rationale for the observed positive correlation between volume and outcome. Under this mechanism, more acquired experience improves performance. Nevertheless, the causality may run in the opposite direction, through a selective referrals channel, whereby better performers command greater demand for their services (Luft, Hunt, and Maerki 1987).When using retrospective data to study volume–outcome relations for surgeons, the selective referrals problem is difficult to overcome credibly. Unlike settings which involve the choice of provider (e.g., elective surgery), the unpredictable nature of EMS does not lend itself to selective referrals. Ambulance units are dispatched based on proximity and not on reputation, and trauma victims are not offered the choice of emergency care providers on scene.
In this paper, we focus on trauma-related ambulance runs to study the effect of volume and experience accumulation on performance. The evidence regarding the appropriateness of prehospital interventions on trauma victims is mixed. However, the importance of getting the patient to definitive care as soon as possible, allowing only for the performance of essential procedures, is widely accepted in EMS and prehospital time markers are frequently used process measures. Shorter out-of-hospital time intervals represent a potentially important factor in survival, especially in the case of trauma incidents (Sampalis et al. 1993; Feero et al. 1995; Nichol et al. 1996; IOM 2000; Blackwell and Kaufman 2002; Carr et al. 2006). For trauma patients, the first hour of care (also referred to as the “golden hour”) is usually considered critical with some evidence that when the time from the incident to hospital treatment is within this critical first hour, the patient's likelihood of survival is increased (Rawlinson and Crews 2003). In trauma care, the goal is to get seriously injured patients into the operating room of a trauma center with an experienced team of appropriately specialized trauma surgeons as fast as possible. Therefore, it is not surprising that dispatching emphasizes ambulance proximity and EMS contracts often include response standards.
We hypothesize that paramedic learning will lead to shorter out-of-hospital times. In particular, as paramedics become more proficient in diagnosing patients' acuity, identifying the appropriate procedures, mastering protocols and techniques, gaining familiarity with local roads and traffic conditions, and identifying quicker routes to scene and to the hospital, the shorter is the time spent out of hospital, and, in particular, the time spent at the scene.
Using the universe of trauma-related ambulance runs in Mississippi between 1991 and 2005, we find that greater volume is robustly related to improved paramedic performance. In specifications that include both individual- and firm-level measures of recent volume, we find small productivity spillovers. In addition, we find that the benefits of recent volume accrue differentially across tenure groups, favoring paramedics with above median tenure.
Data limitations have led most researchers to focus on the effect of recent experience (e.g., volume in the past quarter or year) on performance.1 This is appropriate only if past experience rapidly loses its relevance. Alternatively, if past experience matters, high turnover rates may affect overall prehospital performance as paramedic human capital accumulation is subject to more frequent interruptions. The richness of our data allows us to follow 1,728 paramedics (85 percent of all paramedics in our data) for whom we observe their full histories in the profession. We construct measures of their tenure and cumulative experience as well as of recently acquired experience. Hence, by evaluating the contribution of cumulative experience, we assess the value of paramedic retention in performance units.
The benefit of greater experience operates both through recent and past experiences. This has important implications for retention: retaining the typical paramedic in our sample for an additional quarter is associated with a reduction in out-of-hospital time of approximately 4 minutes. Moreover, 3 years after replacement of an experienced paramedic by a new one, the loss in out-of-hospital time from turnover is between 1 minute and 90 seconds depending on functional form.
Finally, we find evidence that individual experience operates not only on mean performance but also on the upper quantiles of the paramedic performance distribution, which can be interpreted as greater experience leading to better standardization with the severity of trauma.
The EMS Workforce
The three national standard levels of training for Emergency Medical Technicians (EMT) are EMT-Basic (EMT-B), EMT-Intermediate (EMT-I), and EMT-Paramedic (EMT-P).2 In addition, Mississippi requires operators of ambulance vehicles to be EMS-Driver certified (EMS-D), by participating in a training program in operation of emergency vehicles. Licenses for EMTs must be renewed, typically every 2–3 years.
A combination of factors renders the hiring and retention of EMTs problematic, including concerns regarding personal safety, stressful working conditions, irregular hours, excessive training and requirements, limited mobility, and low wages (Institute of Medicine 2000).
Paramedics face a difficult, often hazardous, work environment. They are exposed to potentially infectious bodily fluids, for instance through contact with contaminated needlesticks, and to the hepatitis B virus (Delbridge et al. 1998). Moreover, they are frequently exposed to the threat of violence, incur injuries associated with lifting or falling, and face oncoming traffic at the scene of motor vehicle crashes (Franks, Kocher, and Chapman 2004). Occupational fatality rates for EMTs are comparable to those of police and fire personnel (Maguire et al. 2002).3
In 2001, the Journal of Emergency Medical Services conducted a salary survey consisting of 371 EMS organizations. The survey found salaries to be about 10 percent lower in the Southeastern Census region, which includes Mississippi, compared with the national average. Tenure for paramedics was 7.3 years in private ambulance companies compared with 10.8 years in fire-based EMS agencies. Finally the average yearly turnover rate for private ambulance companies was 18 percent, as compared with 6 percent for fire-based agencies.
EMS Performance
Some studies have found evidence that prehospital advanced life support (ALS) interventions reduce the severity of injuries, lower mortality rates in the first 24 hours, improve outcomes from multiple trauma, and improve survival among patients with blunt trauma (Isenberg and Bissell 2005). Nichol et al. (1996) perform a meta-analysis and find shorter out-of-hospital time to be related to lower mortality. However, overall, the medical literature provides mixed evidence regarding the effectiveness of out-of-hospital management of trauma victims (Carr et al. 2006; Delbridge and March 2007).
Rural trauma patients, on average, face considerably lengthier transport to definitive care (Isenberg and Bissell 2005). Studies focusing on rural areas have found increased survival as a result of on-scene interventions, suggesting that delaying the onset of definitive care may be justified despite the prolonged prehospital intervals (Branas et al. 2005).
To date, only one study has examined the effect of ambulance crew's tenure on outcomes of out-of-hospital cardiac arrests (Soo et al. 1999). The authors found improved survival when patients were attended by paramedics with >1 year of tenure or technicians with >4 years of tenure. The study focused on Nottingham, England, and found no effect of tenure on the time interval spent at the scene. However, it did not look at trauma victims, omitted incident-level characteristics, ignored team composition, and studied tenure (months since completion of training) as opposed to volume (number of runs).
The medical literature provides little guidance as to the right approach for managing out-of-hospital trauma victims. Yet conditional on the characteristics of patients, paramedics, injury, medical interventions performed, and of the scene, there is no dispute that shorter out-of-hospital intervals are preferred by patients and paramedics to longer ones. In this paper, we study the effects of changes in individual paramedic volume on total out-of-hospital time, which is considered a key marker of EMS performance (Carr et al. 2006). In addition, we repeat our analysis for the time spent at the scene of trauma incidents, conjecturing that better diagnostic and therapeutic expertise may reduce on-scene time and hence contribute to the reduction in total out-of-hospital time.
Our data were obtained from the Office of Emergency Planning and Response at the Mississippi Department of Health. Since 1991, this office has systematically collected incident-level EMS data through the Mississippi Emergency Medical Services Information System (MEMSIS). The raw data are recorded at the individual patient level by local EMS providers.
We study incidents between 1991 and 2005 for which the initial call was related to trauma, defined as motor vehicle crashes, pedestrian injuries, stabbings, assaults, gunshots, or falls. To focus on EMS runs where time to definitive care is most likely to be important, we exclude cases of death on arrival and limit the sample to calls involving at least one patient injury and ending in transfer to hospitals by ground transportation.4 All time stamps in the MEMSIS data are rounded to the nearest minute.
Given the highly skewed nature of reported interval times and the possibility of extreme values due to miscoding, we exclude calls for which either the reported time from dispatch to arrival on the scene (i.e., response time) or the reported time from leaving the scene to arrival to a hospital (i.e., transport time) exceeds 60 minutes. This criterion excluded <1 percent of trauma observations that satisfied the other criteria and does not affect our results.
Detailed data on procedures performed by paramedics are available only for the 2001 to 2005 period. While we restrict our volume–outcome analysis to this latter period, we use data for all years (1991–2005) to construct the history of paramedics' experiences, encompassing approximately 613,000 trauma runs. Fifteen percent of paramedics who appear in the 2001–2005 sample are left-censored (i.e., appear in the 1991 data and for whom the entry date is unknown) and the rest enter after 1991. The long time series and the relatively large number of uncensored paramedics enable us to account more precisely for the cumulative number of runs of each individual paramedic. The final sample includes approximately 175,000 observations.
Prehospital emergency medical care is provided by a network of ambulance services organized by county or city. We identify 86 contracting municipalities in Mississippi, where 56 different EMS providers operate. EMS dispatchers can be accessed either by 911 or enhanced-911 service throughout the region. Transport services are provided by fire department agencies, private ambulance services, or hospital-based agencies. In our final sample, 98 percent of all trauma transports are provided by ALS-trained providers. Approximately 10 percent of trauma patients in our sample were directly transported, via ground units, to a Level I trauma facility.5
ALS-certified paramedics can engage in advanced airway management, cardiac monitoring, drug therapy, and other advanced techniques that are prohibited by EMTs with lower certification levels. Team composition may therefore affect prehospital intervals through the quantity and complexity of procedures performed on trauma patients. We control for the experience and certification levels of both team members as well as for the procedures performed.
On-scene (and total out-of-hospital) time is also expected to increase with the number of victims at the scene. Approximately 75 percent of trauma incidents involved a single patient and 98 percent involved at most three individuals. To proxy for underlying patient severity, we control for the procedure indicators, the type of trauma, and patient demographic characteristics. In addition, we control for all possible combinations of the injured body part (i.e., arm, leg, chest, hip, back, neck, head, face, abdomen, and eye) and the type of injury (i.e., pain, burn, laceration, soft tissue, blunt, fracture, dislocation, penetrating trauma, and amputation). These injury combinations are likely to be correlated with the patient severity, which, in turn, is likely to be an important determinant of out-of-hospital time.
EMS in Mississippi is provided by several different types of agencies. In our data, about 19 percent of agencies are community based (mostly integrated with local fire departments), 27 percent are hospital based, and the remaining 54 percent are large private ambulance companies. EMS agencies integrated into and operated by hospitals may have different approaches to the role of paramedics due to closer medical supervision. Similarly, paramedics working for a large private multistate company may have access to better training, equipment, and operate under more stringent protocols compared with a small local fire-based agency.
While we are interested in the effect of paramedic volume on total out-of-hospital and on-scene times, there are many other factors that may affect these markers of performance. These confounders, most of which are summarized in Table 1, include the type of trauma, the incident location, patient characteristics, 42 indicators of procedures performed (listed in Table 2), the number of victims, year, month, day of week, hour of day, the certification levels of the EMTs in each unit, the type of firm that employs them, and the municipality they operate in.6 When studying on-scene time we also include the time it took the team to arrive at the scene.
Table 1
Table 1
Summary Statistics
Table 2
Table 2
Returns to Paramedic Experience
We estimate several models linking paramedic volume with out-of-hospital and on-scene times. The input of interest is the experience of paramedics, as measured by different functions of their accumulated volume as of the date of the incident.
Consider a trauma scene at date t in which injured patient k requires some prehospital care by paramedic i. Prehospital time may be written as
A mathematical equation, expression, or formula.
 Object name is hesr0044-0902-m1.jpg
(1)
where Tikt is either the time spent on scene or the total out-of-hospital time for paramedic i, patient k, and date t. VOLit is a measure of paramedic i's volume as of date t. Xkt captures the characteristics of the patient, such as her age, sex, and race as well as all interactions of injury type and injured body part. It also captures characteristics of the incident, such as type and location of trauma. Wit includes other paramedic characteristics, such as their certification levels, and the type of firm they work for. ϕt is a vector of indicators for hour of day, day of the week, month, and year. μi are individual EMT fixed effects and epsilonikt is a random disturbance. In some of our specifications, our measure of volume, VOLit, represents the aggregate experience level of paramedic i in period t, and in other specifications, the experience accrued solely in the last quarter. VOLit accumulates experience over running 3-month windows, recording paramedic volume at a given date as the number of trauma runs accumulated during the prior 91 days. This measure is more precise than fixed calendar quarters, used extensively in the literature due to data limitations, as it responds instantaneously to any changes in the recent experience profile. Experience accumulation with moving windows can be viewed as smoothing the calendar quarter step function and alleviating the imprecision that increases the further incidents are from the beginning of the quarter.
EMT fixed effects are introduced to mitigate concerns that the volume–outcome relationship we observe is driven by composition effects. For example, low quality/ability paramedics might be lower volume, whereas more able paramedics may accumulate more experience by working more intensely and/or staying in the profession longer. Hence, EMT fixed effects guarantee that the experience parameter in [1] is identified from improvements in performance within paramedic.
While EMT fixed effects provide the most flexible specification for studying individual learning-by-doing, we also estimate firm-by-municipality fixed effect models. This captures systematic time-invariant differences across municipality–firm pairs in the structure of EMS. The interaction between contracting municipality characteristics, such as urban–rural continuum or trauma infrastructure, and firm characteristics, such as EMS protocols, training and equipment, may be correlated with both paramedic volume and on-scene time.
While fixed effects capture time-invariant factors related to individual performance and to volume, such as ability, it may be important to consider bias resulting from evolutionary forces. In particular, a volume–outcome relation may simply result from attrition of poor performers. To address this concern, we perform commonly used variable addition tests (Verbeek and Nijman 1992; Wooldridge 2002) and find no evidence of attrition bias.7
Conditional on observables and on the fixed effects, epsilonikt is likely to be unrelated to any EMT characteristics, be they quality, ability, or, importantly, experience, since the selective referral channel is unlikely to drive the relationship between volume and out-of-hospital time intervals.
In this section, we present estimates of the experience premium in EMS performance, as measured by decreased out-of-hospital and on-scene durations, by estimating equation (1).8
Our models are estimated by OLS cross-sectionally, with firm-by-municipality fixed effects, and with paramedic fixed effects. Standard errors are clustered at the paramedic level to allow for correlation in performance across incidents within paramedic.9
In the first row of Table 2, we document a relationship between recent (quarterly) EMT volume and performance that implies that a one standard deviation increase in the number of trauma runs accumulated by an individual paramedic over a given quarter is associated with a reduction of approximately 35 (10) seconds in total out-of-hospital (on-scene) time. Coefficient estimates are statistically significant at the 1 percent level.
There is a strong tie between retention and learning measured at the firm level, as high labor turnover rates may result in a loss of accumulated experience and hence lower productivity. The short timeframe and, for the most part, lack of individual identifiers led previous studies (e.g., Thiemann et al. 1999; Peterson et al. 2004; Marcin et al. 2007) to focus on learning at the firm level, which confounds individual learning and spillover and turnover effects, with no ability to separate between these three vastly different mechanisms. Our data allows us to follow paramedics over a decade and a half and hence identify the contribution of these three channels to overall performance.
According to Rosen (1972), “learning by experience suggests that the production of knowledge is at least partially acquired through the productive process itself.” An additional EMS run adds to both the individual paramedic and the firm's stock of experience. Firm experience represents the extent to which individual experience spills over to other members of the firm. This calls for a specification in which recent firm volume is added to the one capturing recent paramedic volume.
The results, reported in the second and third rows of Table 2, reveal magnitudes for returns to paramedic volume similar to those in the first row. However, the effect of firm volume is of small magnitude as well as varying signs and degrees of significance for both on-scene and total out-of-hospital times. This is likely due to the limited scope for spillover in tasks performed in teams that rarely exceed two individuals.
To allow for heterogeneity in learning, we stratify the analysis by paramedic tenure under the hypothesis that paramedics of different tenures might exhibit different gains to recent volume. The third panel of Table 2 reports estimates of models in which paramedic volume is interacted with four indicators of tenure quartiles. The quartiles correspond to <2, 2–4, 4–6.2, and over 6.2 years as a paramedic, respectively.
We find that the benefits of recent volume accrue differentially across tenure groups, with small gains in the first two quartiles of the tenure distribution. Paramedics with 6.2 years of tenure and above enjoy about 85 percent higher returns to volume than in the pooled regression for on-scene time. The analogous figure for total out-of-hospital time is 55 percent. This result is somewhat puzzling given the existing evidence that learning is stronger at the beginning of individuals' careers. Our results are consistent with the notion that recent volume affects performance more with greater exposure to trauma scenes and/or with the idea that more senior paramedics have greater influence over the various aspects of EMS delivery as well as the management of patients at the scene (e.g., less reliance on medical direction and better management of other paramedics). Alternatively, composition within tenure quartiles may account for observing learning only among long-tenured paramedics. While the upper tenure quartiles include only paramedics with relatively long job duration, paramedics with low tenure represent a mixture of individuals who are either in the beginning of a longer career in EMS or who do not stay in the profession very long. Therefore, our results are consistent with an explanation whereby short job duration reflects insufficient learning due to lack of either ability or effort.10
While existing volume–outcome studies are almost exclusively focused on providers of tertiary services (e.g., cardiac surgeons) where career durations are extensive, the data used typically span only 2–5 years, accommodating only recent volume effects. Our data covers 15 years of professionals whose average career duration is relatively short and therefore allows us to examine an alternative view of the volume–outcome relationship, in which the benefit accrues to cumulative rather than recent volume. The lower panel of Table 2 reports parameter estimates from models in which the variable of interest is defined as the log of cumulative volume. The log transformation is used to account for the skewed nature of the cumulative volume distribution across paramedics, and it is commonly employed in the volume–outcome literature.11
Since we do not observe the full history of runs for paramedics who appear at the very beginning of the sample, we perform the analysis separately for all 2,010 paramedics (lower panel) and for 1,726 paramedics who where certified (for the first time) after 1991 (upper panel).12 The coefficient estimates for all paramedics are similar in magnitude to the ones for uncensored paramedics, suggesting that experience accumulated before 1991 has little or no relevance by 2001.13
These results allow us to compute the value of retention in performance units. More specifically, we compute the increase in time spent on scene that results from replacing the average paramedic in our data with a new one as follows:
A mathematical equation, expression, or formula.
 Object name is hesr0044-0902-m2.jpg
(2)
where An external file that holds a picture, illustration, etc.
Object name is hesr0044-0902-mu1.jpg is the coefficient estimate for log cumulative volume, An external file that holds a picture, illustration, etc.
Object name is hesr0044-0902-mu2.jpg is the average cumulative volume across paramedics, and v1 is the average quarterly volume across paramedics. Equation (2) computes the difference in average out-of-hospital (on-scene) times within a typical quarter between the average paramedic and a new one. The reductions in total out-of-hospital and on-scene times associated with the retention of the typical paramedic in our sample for an additional quarter are approximately 4 and 2 minutes, respectively.
Figure 1 plots the average monthly difference in total out-of-hospital and on-scene times between an experienced paramedic and a new one. The first period represents the hypothetical month of replacing the average paramedic (in volume terms) in our sample with a new one. We then use equation (2), where v1 is defined at the monthly level, for the subsequent 36 months. The figure describes the loss in total out-of-hospital and on-scene times from the replacement as we follow the new paramedic and compare her average performance to that of the experienced paramedic (had she not exited), therefore allowing for learning by doing by both paramedics. While the cost of turnover is higher in the first months following replacement, differences in performance are (slowly) exhausted over time. Here again, these results do not appear to be driven by attrition.
Figure 1
Figure 1
Cost of Replacing an Average Paramedic with a New One, by Month (Measured in Additional Minutes of Prehospital Time)
In the right most column of Table 2, we implement a falsification exercise by estimating all models described above, in which we replace the dependent variables, total out-of-hospital and on-scene times, with an alternative marker of system performance, dispatch time. Dispatch time is defined as the length of time between a 9-1-1 call and the moment paramedics are notified and dispatched to the scene. This measure provides the basis for a credible falsification test as, unlike time spent on scene or out of hospital, paramedics have no influence on it. If learning is the mechanism through which greater volume yields shorter on-scene and prehospital times, then we should not find a relationship between paramedic experience and a margin over which they have no influence, such as dispatch time. The results confirm this intuition, lending credibility to our results.
In Table 3, we explore the possibility that the performance benefits of experience operate in dimensions other than just mean prehospital times. Specifically, we posit that paramedic volume might shrink the conditional distributions of total out-of-hospital and on-scene duration. To this end, we employ quantile regression methods, which serve to describe how independent variables affect the entire distribution of the dependent variable, as opposed to just its mean. In this analysis, quantile regressions were estimated cross-sectionally and with firm-by-municipality fixed effects at seven different percentiles (0.05, 0.15, 0.25, 0.5, 0.75, 0.85, and 0.95).14 The results indicate that greater recent paramedic volume compresses the upper tail of the conditional on-scene time distribution.15 For instance, according to the firm-by-municipality fixed effects specification, a one standard deviation increase in the number of trauma runs accumulated by an individual paramedic over a given quarter reduces the 95th percentile of total out-of-hospital (on-scene) time by 38 (20) seconds, which is slightly more than double the median and mean effects.
Table 3
Table 3
Effect of Paramedic Experience on Quantiles of Performance
Interventions on scene, injury profile, trauma characteristics, and patient demographics are used, in part, to proxy for patient severity. While these may reflect severity only to a limited extent, concerns regarding omitted variables are not likely to be important given the current EMS system design. It is difficult to argue for a correlation between severity and experience due to the fact that dispatching for ALS incidents is determined by proximity to the scene and not by EMT reputation. However, even if EMS did match paramedic experience with indications of acuity, it is unlikely that more time-consuming, higher severity incidents would result in the least experienced crews being dispatched, which is the only mechanism that would account for our results. If there is matching between highly experienced EMTs and patient severity, our results underestimate the true reduction in prehospital time due to an increase in experience.16
Finally, while our regressions control for indicators of prehospital interventions, one might worry about selection on the complexity of procedures performed by paramedics. For example, if inexperienced paramedics choose simpler procedures, which require relatively fewer minutes, we might infer that less experience results in shorter prehospital times conditional on procedures performed. This would lead us to underestimate the magnitude of the experience premium.
Our results have policy implications for local governments, charged with EMS delivery, and for public and private insurers, who design reimbursement schedules for emergency transports.
In the context of hospitals, policy relevance and implications vary depending on the underlying mechanism behind the volume–outcome relation (Luft, Hunt, and Maerki 1987). When it is driven by selective referrals, no regulatory remedies are advised, as procedures are already concentrated in facilities that provide greater value for patients. Conversely, if causation runs from volume to outcome, the benefit of centralizing care in a small number of facilities may call for regulating entry as well as existing capacity using antitrust analysis of hospital mergers (Birkmeyer et al. 2002; Gaynor, Seider, and Vogt 2005) or policies such as regionalization (Luft, Hunt, and Maerki 1987).
We find that greater volume is robustly related to reduced total out-of-hospital time as well as time at the trauma scene. This benefit operates through both recent and past volume and accrues differentially across tenure groups. In light of our findings, and taking into account the important institutional details that govern the field of EMS, we examine three policies designed to increase volume: consolidation of EMS delivery organizations, redesigning the allocation of workload to insure that current volume is allocated across fewer paramedics, and improving retention of paramedics to prevent past volume from being lost to turnover.
Firm-level studies, which dominate the literature, focus on firm-level interventions by suggesting that large-scale effects raise the benefit from consolidation. While subtle, this assertion relies on the importance of facility-level volume beyond the individual provider's volume. In our data, we find little or no such incremental firm-scale effects. While the importance of distinguishing between learning at the individual level and at the firm level was raised (Luft, Hunt, and Maerki 1987) and tested (Huckman and Pisano 2006) in the case of surgeons, no study to date has examined the returns to volume in a prehospital setting.17 In EMS, absent any knowledge spillovers across paramedics, the individual volume effect we find cannot autonomously constitute grounds for supporting larger firms and therefore yields no direct antitrust implications for mergers. Nevertheless, the massive consolidation of EMS delivery organizations, mainly in the early to mid 1990s, can be justified on different grounds, such as eliminating duplication and lowering average costs through economies of scale (David and Chiang 2009).
As all benefits of greater volume accrue to individual paramedics with no evidence of its transferability across paramedics, policies designed to expose paramedics to greater volume should be considered. Individual volume can be increased in one of two ways, static or dynamic, with each way requiring its own distinct mechanisms. The first requires redesigning the allocation of workload across individuals such that within a given time frame, a given paramedic is handed more runs. The second calls for policies to reduce labor turnover rates among paramedics.
Allocation of workload across paramedics can raise volume by insuring the concentration of runs at the hands of fewer paramedics. But unlike surgery, where individual capacity can be efficiently reached through scheduling, the random nature and location of trauma events requires potentially long periods of inactivity as well as geographical spread of units. Reducing the number of paramedics, regardless of their volume–outcome quality benefits, would translate into longer wait times between initial emergency calls and arrival at the trauma scene. In addition, imposing longer shifts is likely to lead to fatigue, burnout, and dissatisfaction. Using this mechanism is more likely to weaken performance than to improve it through increased individual volume, especially since the effect of recent volume on performance is small. On the other hand, our finding that past volume is important in explaining current outcomes suggests concentrating volume in the hands of paramedics who have already accumulated experience (Gaynor, Seider, and Vogt 2005). Absent mechanisms to induce demand, the salient route for increasing lifetime volume is by increasing the career duration for paramedics. This calls for improving their retention.
In addition, we find that volume is related to improvements in both mean performance and in the upper quantiles of the performance distribution, which can be interpreted as greater experience leading to better standardization with the severity of trauma. This suggests that the reduction in average prehospital times is one of many potential benefits of retention, which may include mentorship, improved quality of care, greater stability of EMS teams, etc. These benefits should be weighted against all the costs associated with retention.
While more research is needed, our results suggest that policy and managerial implications regarding volume–outcome relations in EMS should be directed toward retention to take advantage of individual human capital accumulation.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: We are grateful to Charlie Branas, Sandy Schwartz, Jeff Silber, Mark Pauly, Melinda Pitts, and participants at the International Health Economics Association 6th World Congress, Copenhagen, Denmark, for helpful comments on an earlier version of the paper. Special thanks to Suzy Wheeler, Bill Thomas, Donna Smith, and Mark Allen from the Bureau of Emergency Medical Services at the Mississippi Department of Health and to Jim Pollard, Business Development Director at American Medical Response for their endless patience and support. Arthur Chiang, Ted Gomez, Ben Kowitt, and Robert Medina provided exceptional research assistance. Financial support from the Leonard Davis Institute of Health Economics at the University of Pennsylvania is gratefully acknowledged.
Disclaimer: No conflicts of interest.
Disclosure: None.
NOTES
1For in-depth reviews of the literature, see Halm, Lee, and Chassin (2002) and Gandjour, Bannenberg, and Lauterbach (2003).
2In Mississippi, the state's Department of Health uses the National Registry of Emergency Medical Technicians' (NREMT) examination system. All levels of EMTs in Mississippi must perform National Registry practical skills and pass a written examination to be certified as EMT-B, EMT-I, or EMT-P.
3There are 12.7 fatalities per 100,000 EMS workers annually, which compares with 14.2 for police and 16.5 for firefighters, and a national average of 5 fatalities across all professions.
4A number of companies in Mississippi provide air ambulance services. We exclude <400 such observations, in which helicopters and fixed wings were dispatched. Therefore, all runs in our data involve ground transportation.
5In 2004, the Mississippi trauma system included two Level I trauma centers, five Level II, eight Level III, and 51 Level-IV trauma centers, as defined by the American College of Surgeons. Level IV trauma centers provide initial evaluation and assessment of injured patients; Level III trauma centers must offer continuous general surgical coverage and can manage the initial care of many injured patients; Level II trauma centers must be able to provide initial care to the severely injured patient, and Level I trauma centers must have a full range of trauma capabilities, including an emergency department, a full-service surgical suite, intensive care unit, and diagnostic imaging.
6All results are insensitive to distinguishing between procedures performed on scene versus en route to the hospital.
7We use a modified Verbeek and Nijman (1992) variable addition test described in Wooldridge (2002) in which leads and lags of selection indicators are added as regressors. This approach is attractive in this context since it is implementable using a fixed effects specification.
8Our results are robust to successive addition of possible confounders. The full results are available upon request.
9The subsequent discussion refers to results from the EMT fixed effects specifications.
10As mentioned earlier, this result does not appear to be consistent with time varying paramedic characteristics, such as learning about match quality.
11Transformations other than log yielded very similar results but did not fit the data as well.
12Based on the full record of certification and recertification and interviews with local administrators, we conclude that out-of-state mobility is an extreme event.
13For further discussion of estimation with censored regressors, see Rigobon and Stoker (2007).
14Estimates from OLS describe how average prehospital times vary with paramedic volume, all else being held constant. In contrast, quantile regression provides a description of the entire conditional distribution of the outcome variable, and its dispersion in particular. Each distinct percentile is associated to a separate parameter estimate such that, as is the case here, negative estimates that are increasing in magnitude with the quantiles imply reduced dispersion in prehospital times as paramedics gain experience.
15The standard errors are estimated under the assumption of conditional homoskedasticity and no within-cluster correlation. Therefore, standard errors are likely to understate the true variability of the estimates. EMT fixed effect models, while desirable, are computationally impractical.
16To explore the possibility of nonrandom assignment of paramedics to incidents, we regress patient and scene characteristics on paramedic volume. We find that paramedic experience is not systematically related to patient and scene characteristics.
17Caution should be exercised when generalizing our results to areas where spillover effects can result from selective referrals as opposed to learning-by-doing, such as elective surgical procedures. For example, Birkmeyer et al. (2003) find hospital volume effects in addition to surgeon volume effects. This may suggest that hospital reputation is not merely the sum of the reputation of its physicians.
Supporting Information
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.
  • Birkmeyer JD, Siewers AE, Finlayson EVA, Stukel TA, Lucas FL, Batista I, Welch HG, Wennberg DE. Hospital Volume and Surgical Mortality in the United States. New England Journal of Medicine. 2002;346(15):1128–37. [PubMed]
  • Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. Surgeon Volume and Operative Mortality in the United States. New England Journal of Medicine. 2003;349(22):2117–27. [PubMed]
  • Blackwell TH, Kaufman JS. Response Time Effectiveness: Comparison of Response Time and Survival in an Urban Emergency Medical Services System. Academic Emergency Medicine. 2002;9(4):288–95. [PubMed]
  • Branas CC, Ellen J, Williams JC, Schwab CW, Teter HM, Flanigan MC, Blatt AJ, ReVelle CS. Access to Trauma Centers in the United States. The Journal of American Medical Association. 2005;293:2626–33. [PubMed]
  • Carr BG, Caplan JM, Pryor JP, Branas CC. A Meta-Analysis of Prehospital Care Times for Trauma. Prehospital Emergency Care. 2006;10(2):198–206. [PubMed]
  • David G, Chiang A. The Determinants of Public versus Private Provision of Emergency Medical Services. International Journal of Industrial Organization. 2009;27(2):312–9.
  • Delbridge TR, Bailey B, Chew JL, Conn AKT, Krakeel JJ, Manz D, Miller DR, O'Malley PJ, Ryan SD, Spaite DW, Stewart RD, Suter RE, Wilson EM. EMS Agenda for the Future: Where We Are…Where We Want to Be. Prehospital Emergency Care. 1998;2:1–12. [PubMed]
  • Delbridge TR, March JA. Variability in Out-of-Hospital Trauma Care: So What? Annals of Emergency Medicine. 2007;49(3):302–3. [PubMed]
  • Elixhauser A, Steiner C, Fraser I. Volume Thresholds and Hospital Characteristics in the United States. Health Affairs. 2003;22(2):167–77. [PubMed]
  • Feero S, Hedges JR, Simmons E, Irwin L. Does Out-of-Hospital EMS Time Affect Trauma Survival? American Journal of Emergency Medicine. 1995;13(2):133–35. [PubMed]
  • Franks PE, Kocher N, Chapman S. “Emergency Medical Technicians and Paramedics in California.” University of California, San Francisco Center for the Health Professions.
  • Gandjour A, Bannenberg A, Lauterbach KW. Threshold Volumes Associated with Higher Survival in Health Care: A Systematic Review. Medical Care. 2003;41(10):1129–41. [PubMed]
  • Gaynor M, Seider H, Vogt WB. The Volume–Outcome Effect, Scale Economies, and Learning-by-Doing. The American Economic Review. 2005;95(2):243–7.
  • Halm EA, Lee C, Chassin MR. Is Volume Related to Outcome in Health Care? A Systematic Review and Methodologic Critique of the Literature. Annals of Internal Medicine. 2002;137(6):511–20. [PubMed]
  • Huckman R, Pisano G. The Firm-Specificity of Individual Performance: Evidence from Cardiac Surgery. Management Science. 2006;52(4):473–88.
  • Institute of Medicine. Emergency Medical Services: At the Crossroads. Washington, DC: The National Academies Press; 2000. Committee on the Future of Emergency Care in the United States Health System.
  • Isenberg D, Bissell R. Does Advanced Life Support Provide Benefits to Patients? A Literature Review. Prehospital and Disaster Medicine. 2005;20(4):265–70. [PubMed]
  • Luft HS, Hunt SS, Maerki SC. The Volume-Outcome Relationship: Practice-Makes-Perfect or Selective-Referral Patterns? Health Services Research. 1987;22(2):157–82. [PMC free article] [PubMed]
  • Maguire BJ, Hunting KL, Smith GS, Levick NR. Occupational Fatalities in Emergency Medical Services: A Hidden Crisis. Annals of Emergency Medicine. 2002;40:625–32. [PubMed]
  • Marcin JP, Li Z, Kravitz RL, Dai JJ, Rocke DM, Romano PS. The CABG Surgery Volume-Outcome Relationship: Temporal Trends and Selection Effects in California, 1998–2004. Health Services Research. 2007;43(1):174–92. [PMC free article] [PubMed]
  • Nichol G, Detsky A, Stiell I, O'Rourke K, Wells G, Laupacis A. Effectiveness of Emergency Medical Services for Victims of Out-of-Hospital Cardiac Arrest: A Meta Analysis. Annals of Emergency Medicine. 1996;17(6):700–10. [PubMed]
  • Peterson ED, Coombs LP, DeLong ER, Haan CK, Ferguson TB. Procedural Volume as a Marker of Quality for CABG Surgery. The Journal of American Medical Association. 2004;291(2):195–201. [PubMed]
  • Pointer JE. Experience and Mentoring Requirements for Competence in New/Inexperienced Paramedics. Prehospital Emergency Care. 2001;5:379–83. [PubMed]
  • Pollock MJ, Brown LH, Dunn KA. The Perceived Importance of Paramedic Skills and the Emphasis They Receive during EMS Education Programs. Prehospital Emergency Care. 1997;1:263–8. [PubMed]
  • Rawlinson C, Crews P. Rural Healthy People 2010. Vol. 1. College Station, TX: Texas A&M University System Health Science Center; 2003. Access to Quality Health Services in Rural Areas —Emergency Medical Services: A Literature Review; pp. 77–82.
  • Rigobon R, Stoker T. Bias from Censored Regressors: Basic Issues. International Economic Review. 2007;40(4):1441–67.
  • Rosen S. Learning and Experience in the Labor Market. Journal of Human Resources. 1972;7(3):326–42.
  • Sampalis JS, Lavoie A, Williams JI, Mulder DS, Kalina M. Impact of On-site Care, Pre-Hospital time, and Level of Inhospital Care on Survival in Severely Injured Patients. Journal of Trauma-Injury Infection and Critical Care. 1993;34(2):252–61. [PubMed]
  • Soo LH, Gray D, Young T, Skene A, Hampton JR. Influence of Ambulance Crew's Length of Experience on the Outcome of Out-of Hospital Cardiac Arrest. European Heart Journal. 1999;20:535–40. [PubMed]
  • Thiemann D, Coresh J, Oetgen WJ, Powe NR. The Association between Hospital Volume and Survival after Acute Myocardial Infarction in Elderly Patients. New England Journal of Medicine. 1999;340:1640–8. [PubMed]
  • Tu JV, Austin PC, Chan BT. Relationship between Annual Volume of Patients Treated by Admitting Physician and Mortality after Acute Myocardial Infarction. The Journal of American Medical Association. 2001;285(24):3116–22. [PubMed]
  • Verbeek M, Nijman T. Testing for Selectivity Bias in Panel Data Models. International Economic Review. 1992;33(3):681–703.
  • Wooldridge J. Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: MIT Press; 2002.
Articles from Health Services Research are provided here courtesy of
Health Research & Educational Trust