|Home | About | Journals | Submit | Contact Us | Français|
Each July thousands begin medical residencies and acquire increased responsibility for patient care. Many have suggested that these new medical residents may produce errors and worsen patient outcomes—the so-called “July Effect;” however, we have found no U.S. evidence documenting this effect.
Determine whether fatal medication errors spike in July.
We examined all U.S. death certificates, 1979–2006 (n=62,338,584), focusing on medication errors (n=244,388). We compared the observed number of deaths in July with the number expected, determined by least-squares regression techniques. We compared the July Effect inside versus outside medical institutions. We also compared the July Effect in counties with versus without teaching hospitals.
JR = Observed number of July deaths / Expected number of July deaths.
Inside medical institutions, in counties containing teaching hospitals, fatal medication errors spiked by 10% in July and in no other month [JR=1.10 (1.06–1.14)]. In contrast, there was no July spike in counties without teaching hospitals. The greater the concentration of teaching hospitals in a region, the greater the July spike (r=.80; P=.005). These findings held only for medication errors, not for other causes of death.
We found a significant July spike in fatal medication errors inside medical institutions. After assessing competing explanations, we concluded that the July mortality spike results at least partly from changes associated with the arrival of new medical residents.
Inexperienced medical staff are often considered a possible source of medical errors.1–6 One way to examine the relationship between inexperience and medical error is to study changes in the number of medical errors in July, when thousands begin medical residencies and fellowships.1,7–11 This approach allows one to test the hypothesis that inexperienced residents are associated with increased medical errors1,8,9,11–15—the so-called “July Effect.”
Previous attempts to detect the July Effect have mostly failed,1,8–17 perhaps because these studies examined small,8,10–13,15–17 non-geographically representative samples,8–17 spanning a limited period,11–16 although a study of anaesthesia trainees at one Australian hospital over a 5-year period did demonstrate an increase in the rate of undesirable events in February—the first month of their academic year.1 In contrast, our study examines a large, nationwide mortality dataset spanning 28 years. Unlike many other studies,18 we focus on fatal medication errors—an indicator of important medical mistakes. We use these errors to test the “New Resident Hypothesis”—the arrival of new medical residents in July is associated with increased fatal medication errors.
We examined all official U.S. computerized death certificates (n=62,338,584).19 Our dataset begins with 1979, when hospital status (e.g., inpatient) was first recorded, and ends with 2006, the latest data year available.
We assumed that, inside medical settings, fatal medication errors are more likely to be influenced by inexperienced residents than by patients. In contrast, outside medical settings, we assumed that inexperienced residents play a relatively smaller role, while the patient plays a correspondingly larger role. For example, Phillips, Barker, and Eguchi 20 showed that a significantly larger fraction of medication errors outside medical institutions involved alcohol—indicating the reduced importance of medical residents and the increased importance of the patient.
Therefore, we focused on persons dying inside medical settings: inpatients, outpatients, and those dying in the emergency department. Outpatients are included in our analysis because, in our dataset, 'outpatient' officially refers to persons receiving medical care inside medical institutions, without being admitted to the hospital.21 Because outpatient and ED settings are not distinguished in our dataset, we cannot analyze these settings separately. We compare persons dying inside medical institutions (inpatients, outpatients/ED) with persons dying before reaching medical institutions (those dead on arrival “DOA”).
Geographic detail and exact date of death are unavailable after 2004; consequently, all analyses requiring this information omit later data. In some analyses, we examined both primary and secondary causes of death. For these analyses, our study period begins with 1983, when secondary causes of death were first coded on computerized certificates.
We define fatal medication errors as deaths in which medication errors are recorded as the primary cause of death. All other causes of death analyzed are also defined according to the primary cause. Officially acknowledged medication errors (n=244,388) are coded E850-E858 in the International Classification of Diseases, 9th Revision (ICD9) 22 and X40-X44 in the 10th revision (ICD10).23 Medication error involves “accidental overdose of drug, wrong drug given or taken in error, and drug taken inadvertently [and] accidents in the use of drugs and biologicals in medical and surgical procedures.”22,23 This category is equivalent to “fatal preventable adverse drug event” used elsewhere.24
The ICD category “medication errors” is distinct from the ICD category “adverse effects,” which we also examined. Adverse effects signify cases where “correct drug [was] properly administered in therapeutic or prophylactic dosage, as the cause of adverse effect” 22,23 (E930-E949 (ICD9); Y40-59 (ICD10)). This category includes unexpected allergic reactions resulting from proper drug administration and is equivalent to “fatal non-preventable adverse drug events,” used elsewhere.24 In addition to these categories, we examined surgical errors (E870-E876 (ICD9); Y60-Y69 (ICD10)), external causes (e.g., accidents, homicides, and suicides), and all deaths combined.
In contrast to many other studies2,25–27 we analyze: (1) only preventable adverse effects;25 (2) only medication errors (rather than combining several types of medical errors like medicinal and surgical);2,26 (3) only fatal medication errors;27 (4) only those medication errors coded as the primary cause of death (rather than medication errors coded as primary, secondary, and/or tertiary).2,26 In addition, we examine a nationwide dataset, whereas most other studies extrapolate to nationwide figures from small non-geographically representative samples.2,26 For these reasons, the number of medication errors in our study differs from the number in other studies.
Computerized death certificates do not record whether the patient died in a teaching hospital, but they do record the county of death (1979-2004). Starting in 1980, American Hospital Association (AHA) surveys 28–30 recorded hospital types in each county. We used these surveys to identify counties containing major teaching hospitals 9 near the beginning (1980), middle (1992), and end (2004) of our study period. For each county, we calculated the proportion of hospitals that are major teaching hospitals; we assumed that this proportion is a good indicator of the influence of teaching hospitals and of medical residents in a county. A related indicator, the proportion of patients treated in major teaching hospitals, cannot be accurately measured with AHA datasets.
In addition to computerized death certificates and AHA surveys, we examined monthly data from three other datasets:
The latter two datasets provide complete information from January through November but omit a varying number of days in December. Consequently, we did not analyze December data for these datasets.
We used two procedures to estimate significance levels, depending on the dataset investigated. For the death certificate data, we used standard procedures.34–45 These procedures cannot be easily employed for the other datasets examined because these datasets use very complex multi-stage cluster sampling techniques.46 For these datasets, we estimated significance levels with bootstrap procedures.46
For each of the 28 years under analysis, we determined a least-squares regression equation 34 for the monthly data; this procedure allowed us to estimate the expected number of events in a given month of a given year. In this regression procedure, we used two independent variables: (1) number of days in the month (28–31), and (2) number of the month (1–12). We then summed the 28 expected values for a given month to determine the total expected value for that month during the entire 28-year study period.34
We generated a regression equation for each year separately, rather than a regression equation for all years combined, because the first procedure corrects for possible changes in the monthly distribution from one year to another. The second procedure, using combined data, generates nearly identical expected values for each of the 12 months. For example, there is a correlation of 0.999 between the expected number of monthly deaths generated by the two procedures. (All correlations reported in this paper are the standard Pearson correlations.)
When analyzing mortality from each cause separately, we used linear regression because there is a linear pattern in monthly deaths from each cause under study—for the 28 years combined, the quadratic regression coefficients were insignificant (b2=-0.29; t=-0.14).34 Inspection of regression results for each year separately also reveals no significant departure from linearity.
Linear regressions were also appropriate for nearly all other analyses because the quadratic regression coefficients were insignificant: for inpatient mortality (b2=-0.29; t=-0.14), outpatient and ED mortality (b2=-4.74; t=-2.01), and DOA (b2=-2.29; t=-1.63). However, for ED admissions and for mortality from all causes combined, cubic regressions were appropriate.
D1 = July Effect inside teaching hospital counties—July Effect outside teaching hospital counties
We expect D1 to be both statistically significant and to have a positive value, thus requiring a one-tailed test.
D2 = July Effect inside medical institutions—July Effect outside medical institutions
Here too, we expect the difference to be both statistically significant and to have a positive value, thus requiring a one-tailed test.
Unless otherwise stated, all our significance tests are two-tailed.
Figure 1 displays for each month the ratio:
for inpatient deaths from medication errors. When R exceeds 1.00, observed mortality exceeds the number expected. In July, observed mortality significantly exceeded the expected level [1.062 (1.023-1.100)]. In all other months, mortality levels did not deviate significantly from expected. Henceforth, we use “JR” to indicate the value of R for July.
Figure 1 reveals a July Effect for data aggregated for 28 years. The July Effect was also evident when each year was examined separately. For inpatient deaths from medication errors, JR exceeded 1.00 for 21 of the 28 years (P=0.006; one-tailed binomial test). During the study period, JR displayed no trend (b=0.0003; t=0.104; P=N.S.). In particular, JR did not decline after July 1, 2003, when resident hours were reduced.27 In the three years before this reduction (2000-2002), the average JR was 1.03; in the three years after this reduction (2004-2006), the average JR was 1.05.
Figure 2 displays JR for medication errors occurring in three settings: inpatient, outpatient/ED, and DOA. As expected, JR was not elevated [0.998 (0.945-1.052)] for DOA but was elevated for inpatients [1.062 (1.023-1.100)] and for those dying in outpatient/ED settings [1.060 (1.025-1.095)]. Henceforth, we combine these “intra-institutional” deaths. As in Figure 1, mortality from intra-institutional medication errors spiked only in July. For medication errors, JR inside medical institutions [1.061 (1.035-1.087)] was significantly larger than JR for DOA [0.998 (0.945-1.052); P=0.02; one-tailed ratio of ratios Z-test].45
The July spike for intra-institutional medication errors does not appear to have resulted from a rise in admissions to medical institutions, because inpatient admissions decreased in July [-3% (-5% to -2%)]; and neither increased nor decreased significantly for outpatient admissions [0% (-2% to 2%)] or ED admissions [1% (0% to 2%)].
Figure 3 compares JR for medication errors with JR for other causes of death. Except for medication errors, no cause of death displayed a significant July Effect inside medical institutions. In particular, JR was not elevated for adverse effects, i.e., for medication deaths not considered to result from error. Similarly, there was no July Effect for deaths inside medical institutions from all causes combined.
Given the “New Resident Hypothesis,” JR should be largest in geographic regions with the largest concentrations of teaching hospitals. To test this prediction, we calculated the proportion:
for each of the nine officially defined U.S. regions.35 There was a strong regional correlation between this proportion and JR (r=0.80; t=3.54; n=9; P=0.005, one-tailed test). Thus, the greater the concentration of teaching hospitals in a region, the greater the July Effect for intra-institutional medication errors in that region.
In contrast, the comparison causes of death in Figure 3 did not display regional correlations of this sort (for intra-institutional mortality from adverse effects: r=-0.34; t=-0.95; P=N.S.; for surgical errors: r=-0.13; t=-0.36; P=N.S.; for all causes: r=0.46; t=1.36; P=N.S.).
Given the “New Resident Hypothesis,” the July Effect should be concentrated in counties with teaching hospitals. To test this prediction, we examined counties with teaching hospitals near the beginning, middle, and end of the study period (Fig. 4). Henceforth, we term these “teaching hospital counties” and compare them with all other counties. As expected, for teaching hospital counties, JR was elevated (by 10%) for intra-institutional medication error deaths [JR=1.10 (1.06-1.14)]. In contrast, JR was not elevated for all remaining counties [JR=1.03 (1.00-1.07)]. JR for teaching hospital counties was significantly larger than JR for all remaining counties (P=0.03; one-tailed ratio of ratios Z-test).45 The comparison causes of death displayed no July Effect either for teaching hospital counties or for other counties (Fig. 4).
In a further test of the “New Resident Hypothesis,” we compared the following proportion for two groups:
Group 1 consists of 102 counties for which the proportion of teaching hospitals increased over time. Group 2 consists of all remaining 2,324 counties. For Group 2 (the overwhelming majority of all counties) the annual JR (1979-2004) decreased over time (b=-0.009). In contrast, for Group 1, the annual JR (1979-2004) increased over time (b=0+.0008). Consistent with the “New Resident Hypothesis,” the slope for Group 1 significantly exceeds the slope for Group 2 (t=2.12; d.f.=48; P= 0.02, one-tailed test).34
The above analyses examined medication errors coded as primary cause of death. If, in July, death registrars are unusually likely to code medication errors as a primary rather than as a secondary cause, then there should be a compensatory drop in medication errors coded as a secondary cause. However, July medication errors coded as a secondary cause decreased by only 9 [JR=0.99 (0.95 to 1.04)]. In contrast, these errors increased by 379 when coded as a primary cause.
If, in July, death registrars are unusually likely to ascribe a death to medication error rather than to adverse effects or suicides by medication, then these latter causes should decrease in July. However, neither adverse effects nor medication suicides decreased significantly in July. In July, adverse effects decreased by only 33 [JR=0.91 (0.81 to 1.01)]; medication suicides decreased by only 51 [JR=0.97 (0.93 to 1.02)].
Inside medical institutions, fatal medication errors spiked in July and in no other month. This July spike appeared only in counties containing teaching hospitals; in these counties, July mortality from medication errors was 10% above the expected level. These findings were evident only for medication errors and not for other causes of death or for deaths outside medical institutions.
Although our findings are consistent with the “New Resident Hypothesis,” other hypotheses are conceivable; these are assessed below.
If the July Effect in fact resulted from these summertime behavioral changes, then there should be a general summertime increase in medication errors—not only in July but also in August. No such August spike is found.
The analyses above suggest that, at present, the New Resident Hypothesis is the best available explanation for our findings.
Our use of official, computerized death certificates offers significant advantages: this dataset enabled us to examine a large, nationwide, multi-decadal sample and thereby detect a statistically significant July spike not found in earlier studies. However, our dataset is limited to the most severe type of medication errors (those resulting in death) and provides little detail about each medication error.
In part, because of these limitations, several questions remain: Is there a July Effect for non-fatal medication errors? What are the detailed mechanisms contributing to the July Effect (e.g., miscommunication, inadequate oversight)? Why is there a July spike in fatal medication errors but not in fatal surgical errors? These important questions require further study, perhaps with different kinds of datasets that provide more detail per case.
Despite these gaps in research, our findings have several implications for medical policy—they provide fresh evidence for: 1) re-evaluating responsibilities assigned to new residents; 2) increasing supervision of new residents; 3) increasing education concerned with medication safety. Incorporating these changes might reduce both fatal and non-fatal medication errors and thereby reduce the substantial costs1 associated with medication errors.
Our nationwide, multi-decadal study enabled us to discover previously unknown evidence for a July spike in fatal medication errors. This spike seems to result at least partly from changes associated with new medical residents. The July Effect seems to be a significant public health problem and warrants further investigation.
Funding: Marian E. Smith Foundation. We thank Kimberly M. Brewer, Robert D. Kleinberg, and Miranda Phillips for valuable comments and suggestions.
Conflict of Interest None disclosed.
Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.