|Home | About | Journals | Submit | Contact Us | Français|
The United States instituted restrictions on resident work-hours in July 2003. The clinical impact of this reform on critically ill patients is unknown.
We sought to examine the association of the resident work-hours reform with mortality for patients in medical and surgical intensive care units (ICUs).
We conducted a retrospective cohort study, comparing mortality trends before and after July 1, 2003, in teaching and non-teaching hospitals.
The study included 230,151 adult patients admitted to 104 different ICUs at 40 hospitals participating in the APACHE IV clinical information system from July 1, 2001, to June 30, 2005.
The primary exposure was the date of admission, relative to the implementation of the work-hours regulations. The primary outcome was in-hospital mortality; a secondary outcome was ICU mortality. The analysis included 79,377 patients in 12 academic hospitals; 73,580 patients in 12 community hospitals with residents; and 77,194 patients in 16 non-teaching hospitals. Risk-adjusted mortality improved in hospitals of all teaching levels during the study period. There were no significant differences in the mortality trends between hospitals of different teaching intensities, as demonstrated by non-significant interaction between time and teaching status (global test of interaction p=0.56).
There was a decrease in in-hospital mortality in ICU patients during the years of observation. This decrease was not associated with hospital teaching status, suggesting no net positive or negative association of the resident work-hours regulations with a major patient-centered outcome.
In July, 2003, the Accreditation Council for Graduate Medical Education (ACGME) enforced a policy regarding the work-hours of all trainees of ACGME-accredited residency programs limiting the total number of weekly working hours, the number of consecutive working hours, and the number of working days without a break . These regulations were implemented in response to concerns about the effects of sleep deprivation on residents, including increased medical errors, impairment of learning, and physical risks such as motor vehicle accidents [2–6]. Prior to the changes there had been little research on patient safety related to resident work-hours, and the reform has been controversial. Many raised concerns about indirect negative consequences, including fragmentation of care and loss of educational opportunities [7–10].
Several recent studies have examined the effect of the work-hours regulations on patient outcomes to evaluate this issue. A study of a medical teaching service showed that three clinical parameters (intensive care unit utilization, rate of discharge to home, and pharmacist interventions to prevent error) improved after the reform . Additional studies have shown either no change or a small relative mortality improvement in medical and surgical patients in teaching hospitals, compared to non-teaching hospitals [12–14]. However, these studies utilized administrative and claims data, with limited ability to account for variation in severity of illness. Furthermore, these studies have evaluated broad patient populations; the effect of the work-hours reform may vary in different clinical settings. Critically ill patients, for example, may be more sensitive to errors made due to physician fatigue. In fact, one study of interns in the ICU suggested that extended shifts led to more medical errors . Alternatively, increased hand-offs may lead to loss of clinical details that could negatively impact care of complex patients.
The purpose of this study was to examine the temporal association between the mandatory resident work-hours reform and mortality trends for critically ill patients. To do so, we used the Acute Physiology and Chronic Health Evaluation (APACHE) IV database, a voluntary clinical registry of patients admitted to ICUs including a validated severity of illness scoring system [17, 18].
We performed a retrospective cohort study using the APACHE IV dataset, a voluntary clinical information system for ICUs in U.S. hospitals. This database includes detailed information on patient demographics, physiologic parameters at the time of admission, and clinical outcomes . The University of Pennsylvania institutional review board exempted this study from human subjects review as a secondary analysis of a de-identified database.
We included patients admitted during the two years before and the two years after the implementation of the work-hours reform on July 1, 2003. We excluded hospitals that did not participate in the APACHE clinical information system both before and after this date. We also excluded patients whose outcomes were unknown, patients in the hospital on the day of implementation, and patients less than 18 years old on admission. To allow for the assumption of independence among observations, we kept only a single, random ICU admission for patients with multiple admissions during the study period.
We defined the exposure as the occurrence of the admission on or after July 1, 2003. The date of each admission was categorized as pre-reform year 2 (July 1, 2001, to June 30, 2002), pre-reform year 1 (July 1, 2002, to June 30, 2003), post-reform year 1 (July 1, 2003, to June 30, 2004), or post-reform year 2 (July 1, 2004, to June 30, 2005). The primary outcome was in-hospital mortality. A secondary outcome was ICU mortality. Because changes in rates of transfers could mask true changes in mortality, we also evaluated the proportions of all patients discharged to other facilities [19, 20].
We categorized hospitals as either non-teaching, community with residents, or academic based on their reported level of housestaff involvement in patient care. Non-teaching hospitals reported the absence of any residency programs. Community hospitals with residents reported a residency program in at least one specialty. Academic hospitals reported membership in the Association of American Medical Colleges Council of Teaching Hospitals (COTH).
Severity of illness was measured by the APACHE III score . Details of the scoring system are publicly available . We used quadratic spline modeling to estimate the relationship between APACHE III and mortality. Other variables included in the analyses were the length of stay in the hospital prior to ICU admission, location prior to ICU admission (operating room, emergency room, general floors, direct admission, or transfer from another hospital), demographic information (age, gender, and race/ethnicity), and hospital-level information (size and region) .
Patient characteristics and unadjusted outcomes were calculated using descriptive statistics and compared across time periods using chi-square tests and the non-parametric test for trend, as appropriate. To examine the independent effect of work-hours on inhospital mortality we used a hierarchical multivariable logistic regression model with hospital-specific random-effects . This method mitigates confounding by hospital and accounts for patient-level clustering within centers. Potential confounders were chosen a priori based on previous studies [21, 24–26]; they were gender, pre-admission hospital length-of-stay, source of admission, and severity of illness. We used conditional standardization to calculate adjusted in-hospital mortality using mean, modal, and typical values for the model covariates .
To test the effect of the work-hours reform on mortality, the mortality trends of academic hospitals and community hospitals with residents were compared to the baseline trend of non-teaching hospitals. To do this we fit the regression model described above with a fixed effect for hospital teaching status and interaction terms between each category of teaching status and time period. These terms measure if the before-after effect of the work-hours regulations was different for teaching and non-teaching hospitals (i.e., a difference in differences approach). Each interaction term has an odds ratio that describes the magnitude of its effect. This approach is more robust than simply evaluating temporal trends in teaching hospitals, which could be confounded by temporal changes unrelated to the reform.
The collective significance of the interaction terms was first assessed using a likelihood ratio test. Each interaction term was then individually tested for significance using a Wald chi-square test, which assessed whether the odds ratio of each interaction term equaled 1. A statistically significant test would support the conclusion that the reform was associated with a change in mortality trends overall (likelihood ratio test) and in each year and by each hospital type separately (Wald test).
For our models we could treat the two pre-reform years either as independent or as a single time period to estimate the pre-exposure mortality risk. In order to make this determination, a test of controls was performed. The mortality levels were allowed to vary among hospitals of different teaching levels and were assumed to be unaffected by the impending reform prior to its actual implementation. Interaction terms were fit between each teaching level and each time period before the reform. If the test of controls was significant, we assumed that the divergence in mortality trends was unrelated to the reform and planned to use only pre-reform year 1 as the referent group. If not, we planned to group all observations before the reform into a single referent group.
We repeated the analyses using ICU mortality as the outcome variable. Additionally, we repeated the analyses for a subgroup of patients admitted with surgical diagnoses, since Previous literature suggests that there may be a different effect of work-hours on surgical services, compared to medical services . We also tested some of our assumptions through sensitivity analyses. We evaluated the effect of including only one random admission for each patient with multiple admissions by repeating the analyses both with all the admissions and with a sample including only the first admission for all patients in the database. We tested the effect of the most extremely imbalanced hospital participation before and after the reform by repeating analyses on the study population after dropping those hospitals with less than 100 patients either before or after the reform, in case these hospitals yielded unstable estimates in the model. We tested our assumption that patients admitted prior to July 1, 2003, were unaffected by the reform by repeating the analyses using pre-reform year 2 as the baseline.
All tests were two-tailed, and p≤0.05 was considered significant. All statistical analyses were performed using STATA 10 (StataCorp, College Station, Texas).
The dataset initially contained 259,490 admissions at 56 hospitals. The exclusion strategy is shown in Figure 1. The final analysis included 230,151 patients (88.7%) at 40 hospitals with a total of 104 ICUs.
Characteristics of the patients and the hospitals are summarized in Tables 1 and and2.2. The analysis included 12 academic hospitals, 12 community hospitals with residents, and 16 non-teaching hospitals. Academic hospitals were largest, had the highest proportion of specialty ICUs, and admitted the highest case-loads. Patients had similar severity of illness scores across all hospital types and time periods. A higher proportion of patients in academic hospitals were mechanically ventilated than in other hospitals. The proportions of patients transferred to other hospitals increased at non-teaching hospitals in post-reform year 2 but remained constant at other hospitals. Unadjusted in-hospital mortality overall showed a modest improvement over time. When stratified by teaching status, the mortality improved over time in academic hospitals (non-parametric test for trend p < 0.001) and community hospitals with residents (p=0.038), and it increased over time in non-teaching hospitals (p=0.025).
After adjustment for age, gender, severity of illness, pre-admission hospital length of stay, pre-admission location, and hospital teaching status, all hospitals had improved in-hospital mortality in the years after the reform (Figure 2). Improvement was statistically significant in academic hospitals in all years after pre-reform year 2 (p=0.03 in pre-reform year 1, p<0.001 in post-reform years 1 and 2); and in community hospitals with residents and non-teaching hospitals in post-reform year 2 (p<0.001).
In the test of controls for in-hospital mortality, the interaction term between academic hospitals and pre-reform year 1was significant (p<0.05), indicating that academic hospitals had a different mortality trend than non-teaching hospitals before July 1, 2003. Since this was assumed to be unrelated to the work-hours reform, we used pre-reform year 1 as the baseline to evaluate the overall effect of the reform. In the final model the likelihood ratio test comparing models with and without the interaction terms was not significant (p=0.56), indicating that the overall temporal improvements in mortality associated with the reform were not different between teaching and non-teaching hospitals. The individual interaction terms were also not significant (Table 3), indicating that there were also no differences in any one year or at any teaching level.
Results were similar in the analysis of ICU mortality trends. Adjusted mortality estimates are illustrated in figure 2. The likelihood ratio test for the collective significance of the interaction terms was not significant (p=0.23), indicating no divergence in trends in hospitals of different teaching levels. The individual interaction terms were also not significant (p ≥ 0.05 for all interaction terms).
In a sub-group analysis, in-hospital mortality in patients admitted with surgical and non-surgical diagnoses were studied separately. The likelihood ratio tests evaluating the overall interaction between teaching status and time period were not significant in either subgroup (p=0.08 in non-surgical patients and p=0.88 in surgical patients).
In a sensitivity analysis, results using only the first admission or all admissions instead of one random admission for patients with multiple admissions and including subjects present on the day of implementation were unaltered. Similarly, excluding hospitals with minimal participation (less than 100 patients) in the registry either before or after the reform date, and using admissions during pre-reform year 2 to estimate the baseline mortality risk also showed no evidence for an association between mortality trends and teaching status.
In hospitals participating in the APACHE IV clinical information system, implementation of the ACGME resident work-hours regulations was not discernibly associated with the mortality trends of critically ill patients. In-hospital mortality in all hospitals improved over time, and the degree of change was not significantly different between teaching and non-teaching hospitals. These findings are similar to studies of broad patient groups that used administrative rather than clinical risk-adjustment [13, 14]. We extend this work by demonstrating the absence of a major association of work-hours and patient outcomes in the ICU, where patients are perhaps at highest risk for adverse events and most sensitive to changes in physician staffing.
Proponents of the work-hours reform have argued that decreased physician fatigue might improve patient care, as errors in decision-making would be decreased . However, others have felt that the work-hours limits would negatively impact patient care, due to increased hand-offs and decreased housestaff responsibility . Our study does not provide evidence for either a dramatic improvement or decline in patient outcomes.
Several factors might explain this finding. Severely ill hospitalized patients may be less sensitive to resident staffing patterns than previously thought. Also, perhaps the hospitals in this sample compensated for fewer resident work-hours by means that have offset the negative effects of the regulations. Indeed, many institutions have increased the role of non-physician providers and/or have transferred more decision-making to more senior physicians [29, 30]. Finally, there may be both negative and positive effects of work-hours limitations that counterbalance each other.
There are several limitations to this study. First, this is an observational study using an existing database. As such, some details are unavailable, such as specific information about ICU staffing, whether programs achieved compliance with the regulations and when they did so, and whether there were changes in admission policies or resident case-loads. Our methods assume that all hospitals adhered to the regulations within two years of the implementation date; however, many programs struggle with compliance even now, more than five years later. Thus, the lack of association may simply reflect exposure misclassification for many hospitals. Also, we cannot account for other interventions that may have coincided with the reform. The fact that we used non-teaching hospitals to control for the effects of other changes in critical care organization and practices is an important strength. Second, participation in this clinical registry is voluntary. Participating hospitals are highly motivated toward quality improvement and thus may not be representative of all hospitals. Nonetheless, hospitals in APACHE vary widely in size, region and teaching status, making it likely that important differences in outcome trends would be observed. Third, there is some risk for misclassification of teaching status, as it was defined according to self-report. Distinguishing academic hospitals from community hospitals with residents may not accurately define the teaching intensity of the participating hospitals. However, the fact that their temporal trends were similar to the trend in hospitals with no housestaff whatsoever supports the validity of our findings. Another limitation of our teaching status definition is that it assumes that all patients at teaching hospitals are covered by housestaff. Patients at any teaching hospital may be admitted to non-teaching services. We believe that this is likely to be infrequent with respect to ICU care, since all medical and surgical residency programs require ICU rotations; however, we cannot validate this assumption in this dataset.
Another important limitation is that the reform was likely adopted at different times at different programs. Some anticipated it and made changes early while others struggled to comply until after July 1, 2003, contaminating the exposure classification. To address this problem, we studied a four-year period. That we were able to adequately adjust for severity of illness and used non-teaching hospitals as controls also mitigates the impact of varying adoption time. Still, a very gradual overall effect due to averaging over all the hospitals may have been undetectable.
A final limitation is that this database restricts our analysis to in-hospital mortality, rather than 30-day mortality. We cannot account for patients that died after being transferred to other facilities. The change in discharge practices in non-teaching hospitals could, in fact, reflect that the actual trend in mortality in non-teaching hospitals is different than observed; that fewer patients were discharged home in the later years of observation may represent a higher mortality than was estimated in those time periods. Teaching hospitals had little change in their discharge practices. It is possible, then, that the improved mortality of patients at teaching hospitals may be greater than at non-teaching hospitals. If anything, this potentially suggests a positive effect of the work-hour regulations.
Future studies could address some of these limitations. For example, data from the ACGME regarding date of actual compliance with the regulations could be linked to the APACHE database to minimize misclassification of exposure. Alternatively, a retrospective cohort study of a small sample of hospitals could be undertaken to more clearly define the exposure and outcome, minimize misclassification, and obtain more detailed data about potential confounders. These and the other future studies described above will shed further light on this very important debate.
Our findings imply that limiting residents’ work-hours does not significantly impact patient mortality, supporting the notion that more humane working conditions for residents can be accomplished without compromising patient care. However, this study does not measure all possible adverse effects of the regulations. Although mortality is a major patient-centered outcome, it may not be best measure of the effects of changing resident staffing patterns. We do not capture the rate of non-fatal medical errors or any changes in patient or family satisfaction with care, nor do we know the economic costs of the measures undertaken to compensate for the changes. Furthermore, we do not yet understand the short- and long-term educational implications. Having fewer work-hours may limit clinical and didactic educational opportunities, or it may improve the retention of those opportunities obtained. By incompletely understanding the effects of reducing the overall time of training, we risk training less competent future physicians. Further investigation in this area is imperative.
The introduction of ACGME duty-hour rules was not associated with a demonstrable change in the trend of decreasing mortality of critically ill patients in teaching hospitals compared to the trends observed in non-teaching hospitals. These regulations have changed the nature of medical education and the role of housestaff in the management of critical illness. Further investigation of the long term effects of the work-hours changes is needed to inform the ongoing evolution of physician training, to better understand the implications for resource utilization, and ultimately, to inform further efforts to improve patient safety and quality of care for critically ill patients while training the next generation of physicians.
Financial support: NIH T32 HL007891-10