|Home | About | Journals | Submit | Contact Us | Français|
To conduct a statewide analysis of the effect of New York's regulations, limiting internal medicine and family practice residents’ work hours, on patient mortality.
Retrospective study of inpatient discharge files for 1988 (before the regulations) and 1991 (after the regulations).
Adult patients discharged from New York teaching hospitals (170,214) and nonteaching hospitals (143,455) with a principal diagnosis of congestive heart failure, acute myocardial infarction, or pneumonia, for the years 1988 and 1991 (periods before and after Code 405 regulations went into law). Patients from nonteaching hospitals served as controls.
Combined unadjusted mortality for congestive heart failure, acute myocardial infarction, and pneumonia patients declined between 1988 and 1991 in both teaching (14.1% to 13.0%; P = .0001) and nonteaching hospitals (14.0% to 12.5%; P = .0001). Adjusted mortality also declined between 1988 and 1991 in both teaching (odds ratio [OR], death 1991/1988, 0.868; 95% confidence interval [CI], 0.843 to 0.894; P = .0001) and nonteaching hospitals (OR, death 1991/1988, 0.853; 95% CI, 0.826 to 0.881; P = .0001). This beneficial trend toward lower mortality over time was nearly identical between teaching and nonteaching hospitals (P = .4348).
New York's mandated limitations on residents’ work hours do not appear to have positively or negatively affected in-hospital mortality from congestive heart failure, acute myocardial infarction, or pneumonia in teaching hospitals.
The death of a young woman being treated for pneumonia in a New York teaching hospital in 1984 led to a grand jury investigation of the circumstances surrounding her care.1 A panel was formed to address the findings of the grand jury and to make recommendations to remedy deficiencies. The panel determined that supervision of residents by attending physicians was inadequate and that fatigue and sleep deprivation from lengthy continuous working hours contributed to poor quality of patient care. A new law, referred to as “Code 405,” became effective in 1989 and addressed supervision and limitations of work hours for residents as well as the reduction in the ancillary work they performed.2 The regulations were primarily intended to improve patient care by reducing work-related stress and fatigue of residents. The only evaluations of Code 405 were retrospective surveys of residents that demonstrated subjective improvement in their professional and personal life styles.3,4 The enactment of Code 405 at the state level and these observations have helped drive the current demand for investigation and reform at a national level in the United States. In the year 2000, the American Medical Association House of Delegates adopted a resolution which called for evaluation of fatigue and sleep deprivation on medical education and physician performance. In the Annual 2001 Reports of the Medical Association, the Resident and Fellow Section focused on work hours reform and the need to enforce the guidelines of the Accreditation Council on Graduate Medical Education (ACGME).5 On April 30, 2001, a petition was filed with the federal government's Occupational Safety and Health Administration. Sponsors of the petition called for a nationwide implementation of regulations paralleling those of Code 405 to limit the work hours of residents and clinical fellows.6 ACGME work hour regulations were made a requirement of accreditation beginning July 1, 2003 nationwide.
Evidence of changes in patient outcome related to physician performance from Code 405 has been notably absent in the reform debates. Critics claimed that loss of continuity of care due to shorter work hours and more resident transitions would result in worse patient care, while advocates postulated that less fatigued doctors would provide better care. These concerns were expressed both before Code 405 and afterward in the surveys of house staff and attending physicians. It was our objective to learn whether patient care was improved (or harmed) by assessing mortality rates in all the teaching and nonteaching hospitals of the state of New York. The evolution and increasing use of outcome studies using large database analysis stimulated us to use this methodology to examine the effect of Code 405. Our goal was to compare mortality before and after the regulations simultaneously in teaching and nonteaching hospitals, with the nonteaching hospitals (unaffected by Code 405) serving as a control group. We chose common medical conditions that would focus on internal medicine and family practice residency programs in the teaching hospitals.
Institutional review and approval of this study for the protection of human subjects was obtained.
We compared the adjusted odds of death in 1991 relative to 1988 for teaching hospitals, then did the same for nonteaching hospitals and subsequently analyzed whether there was any difference between these 2 odds ratios.
We identified the study population using the New York Statewide Planning and Research Cooperative System (SPARCS) database. We obtained administratively releasable (no patient-identifying data) inpatient discharge files for calendar years 1988 and 1991 for acute care hospitals only. The files indicated death or location to which living patients were discharged. We chose 1988 to represent the “before” condition and 1991 to represent the “after” condition. Code 405 was enacted in 2 sections: the first on January 1, 1989, affecting ancillary duties, and the second on July 1, 1989, which affected supervision and hours of work.
The total number of records for each of the 2 years was approximately 2.6 million. Our goal was to select large numbers of patients whose treatment would be influenced by residents most affected by Code 405 regulations. Three principal diagnoses, congestive heart failure (CHF), acute myocardial infarction (AMI), and pneumonia (PNU), were chosen, being frequently found in hospitalized patients and commonly studied in other claims-based analyses.7–15 These diagnoses were identified by International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes which, based on review of current literature, were deemed to accurately represent these 3 medical conditions (Table 1).16
Patients were categorized according to whether they were treated in teaching hospitals or in nonteaching hospitals. Hospitals were identified in the SPARCS database by operating certificate number. The national resident matching program matching list for 1991 was used to identify the teaching hospitals where internal medicine or family practice programs were offered. We identified 71 teaching and 181 nonteaching acute care hospitals. No veterans hospitals were included.
Since the aim of this study was to assess medical services treating all 3 diagnoses, pediatric patients (under 18 years of age) were excluded. All patients transferred to another acute care facility were also excluded. Finally, we excluded acute myocardial infarction patients with principal diagnosis code 410. × 2, which indicated a subsequent episode of care for a myocardial infarction that had already been treated within the preceding 8 weeks. The final sample size for each diagnosis was congestive heart failure (N = 116,919); acute myocardial infarction (N = 67,310); and pneumonia (N = 129,440).
Differences in patient characteristics between 1988 and 1991 were assessed using χ2 tests for categorical variables and t tests for continuous variables. Odds ratios (OR) for mortality were derived from logistic regression models and P values based on the associated χ2 test. All significance tests were two tailed.
We adjusted for confounders by including in every model a set of patient demographic variables, patient clinical variables, and hospital variables. Patient demographic variables were: age, age2, race, sex, source of admission (physician referral, transfer from an acute care facility, transfer from a nursing home or other health care facility, or other unknown source), type of admission (emergency vs nonemergency), and type of insurance (fee for service, including Medicare, Health Maintenance Organizations, and commercial; Medicaid; self-pay; and other). The variable indicating type of insurance was included in our model as a surrogate for socioeconomic status. Clinical diagnosis codes listed as additional diagnoses in the database were used to identify comorbidities. These comorbidities have been previously validated and are listed in Table 2.17 The hospital level variable was teaching status. An indicator variable denoting whether or not a hospital had invasive capabilities (coronary stent, coronary angioplasty, and coronary artery bypass surgery) was created for acute myocardial infarction and congestive heart failure. In our analysis of acute myocardial infarction, this variable was used to stratify the sample population with separate multivariate and univariate analyses carried out on each stratum.
The data were first analyzed with all 3 diagnoses considered together, and then separate analyses were carried out for each individual diagnosis. For each analysis, multivariable logistic regression models were constructed to estimate the odds of death in 1991 relative to 1988, for both teaching and nonteaching hospitals. Our combined models with all 3 diagnoses considered together could detect a 2.4% reduction in the odds of death between 1988 and 1991 in nonteaching hospitals, and a 2.0% reduction in teaching hospitals with 80% power.18
Models were developed on one half of the sample population and then validated for the other half. The validation was carried out by constructing a dummy variable interaction model identical to that mentioned above but with an additional dummy variable that signified whether a particular patient was in the development or validation set. For all 3 diagnoses there was no “group” by variable interaction term that was significant at the 5% level. Final models were thus constructed using both the development and validation data sets.
To assess the strength of our models, we calculated the C-statistic for each one. The C-statistics were 0.780 for acute myocardial infarction, 0.703 for congestive heart failure, 0.759 for pneumonia, and 0.734 for our combined model.
Each model included a term interacting teaching status with year. The interaction between teaching status and the year of hospitalization was a test of the null hypothesis that the odds of death associated with teaching status was equivalent across years with and without Code 405. A significant coefficient on the interaction between teaching status and year was taken as indication of a possible effect of Code 405.
Relative to patients in 1988, patients in both nonteaching and teaching hospitals in 1991 were more likely to have been transferred from a nonacute health care facility and more likely to be nonwhite. Patients in 1991 in both types of hospitals were more likely to have other neurological disorders, complicated diabetes, renal failure, chronic pulmonary disorder, AIDS, and congestive heart failure as comorbidities, while less likely to have diabetes, peripheral vascular disorder, and hypothyroidism (Table 2). Patients in 1991 in teaching hospitals were less likely to have anemia, less likely to be admitted as an emergency case, but more likely to be female, and more likely to be transferred from another acute care facility. Patients in nonteaching hospitals in 1991 were more likely to be admitted as an emergency case, more likely to have anemia, more likely to have hypertension, less likely to have cardiac arrhythmia, and less likely to be transferred from an acute care facility relative to patients in 1988.
Overall unadjusted mortality for teaching and nonteaching hospitals declined between 1988 and 1991 (Table 3). Only acute myocardial infarction patients in noninvasive teaching hospitals and pneumonia patients in teaching hospitals failed to experience a significant decline in unadjusted mortality between 1988 and 1991.
Note that adding adjustment variables (as listed in Table 3) had 2 effects on the results concerning changes in mortality between 1988 and 1991 (Table 4). Whereas unadjusted results showed that survival from pneumonia in teaching hospitals did not significantly change between 1988 and 1991 (Table 3), the adjusted results indicate a lower odds of death in 1991 relative to 1988. Furthermore, with respect to survival from AMI in invasive nonteaching hospitals, the unadjusted results showed a significant reduction in mortality between the study years, whereas the adjusted results (Table 4) indicated no significant change in the odds of death between 1988 and 1991.
Overall hospitalization in 1991 was associated with lower odds of death in both teaching and nonteaching hospitals. If there was a beneficial Code 405 effect, we would expect to see a greater decline in mortality among teaching hospitals than nonteaching hospitals. Teaching hospitals showed no increased improvement as compared to nonteaching hospitals for any of the 3 conditions when considered individually or when combined as a single group overall. For each of the 3 conditions examined, the ratio of the odds ratios (for mortality in 1991 vs 1988) comparing teaching to nonteaching hospitals could not be distinguished from unity and in all cases this ratio had a relatively narrow confidence interval.
We asked whether the size of the residency program may have influenced these results. We therefore adjusted our model for residency size by including a continuous variable for the combined number of internal medicine and family practice residents at each hospital. We found no difference in our results for AMI, CHF, and PNU. For AMI, the ratio (between teaching and nonteaching) of the changes in mortality (between 1988 and 1991) was 1.061; 95% confidence interval (CI), 0.969 to 1.163; for CHF 0.993; 95% CI, 0.918 to 1.074, and for PNU 0.997; 95% CI, 0.932 to 1.067. The coefficient on residency size was never significant in any model. Another approach to account for residency size was also taken. We grouped hospitals with internal medicine and family practice residents by the combined size of these 2 programs into quartiles. For each quartile we then fit a separate model to examine the relative changes in mortality (in 1988 vs 1991) between that quartile of teaching hospitals and all nonteaching hospitals. Again, there was no significant interaction between year of hospitalization and teaching status. The ratio of the change in the highest quartile (largest combined population of internal medicine and family practice residents) was 1.009; 0.866 to 1.175 for AMI, 1.037, 95% CI 0.974 to 1.190 for CHF, 1.002, 95% CI 0.896 to 1.121 for PNU. The other quartiles showed similar results.
The purpose of this study was to identify changes in outcomes for patients in teaching hospitals following the enactment of Code 405 using 3 common illnesses—congestive heart failure, acute myocardial infarction, and pneumonia. The teaching programs involved in the care of these patients would be internal medicine and family practice residencies. In order to control for time-related changes in treatment, the same assessment was made in the nonteaching hospitals. If Code 405 had a beneficial effect on teaching hospitals, then we would expect to see a greater impact on mortality over time in teaching hospitals than in nonteaching hospitals. This was not the case. We found that indeed survival for all 3 diagnoses increased similarly over time in both teaching and nonteaching hospitals. Code 405 therefore appears to have had no statistically detectable effect on in-hospital mortality. Had we only studied teaching hospitals, we would have wrongly concluded that the drop in mortality from 1988 to 1991 was possibly due to the effect of Code 405. Comparing the adjusted change in mortality between 1988 and 1991 in teaching hospitals to that of nonteaching hospitals allowed us to control for other temporal factors present in both teaching and nonteaching hospitals.
One interpretation of the study findings may be that the negative effect of reduced continuity of care due to shorter hours and more resident transitions was offset by improved patient care by less fatigued residents. The reports of resident and attending physician surveys support this possibility. The survey of internal medicine residents by Conigliaro et al. in March of 1991 found that while residents uniformly welcomed the reduced fatigue and increased spare time, they and the attending staff expressed concerns that the continuity of patient care was suffering under the new conditions.3 The same views resulted from a similar survey of obstetrics-gynecology residents.4 The conclusion, once again, was that Code 405 did not seem to be achieving its patient care-related goals but did document that residents viewed the changes as positively affecting their personal lives.
The only study of Code 405 on patient outcomes in the literature was reported by Laine et al., who carried out a retrospective cohort study on a single general internal medicine service of The New York Hospital.19 They found that complications were more frequent in 1989, immediately after Code 405, than in 1988, but there were no significant differences in length of stay or in-hospital mortality between these two periods. These results led Laine et al. to conclude that Code 405 was not achieving its goals of improving patient outcome but acknowledged that the small number of patients in a single institution were important limitations affecting this conclusion. Our large patient population derived from an all-inclusive statewide hospital database was designed to address these limitations. The conclusion nevertheless remains the same: there is no consistent evidence of additional improvement (or decline) in the quality of patient care (as measured by in-hospital mortality) by internal medicine and family practice residents in New York State teaching hospitals because of the enactment of Code 405.
The SPARCS database was in part developed to aid the state of New York in determining hospital reimbursement. Thus, coding choice may have been influenced by reimbursement schedules. We are unaware of any changes in reimbursement schedules occurring between 1988 and 1991 that would have uniquely affected teaching hospitals or nonteaching hospitals. We did not have unique patient identifiers and thus 30-day mortality, which probably would have been a more sensitive outcome measure than in-hospital mortality, could not be calculated. Sufficient information to fully determine socioeconomic status (SES) was also lacking. However, we did adjust for type of insurance (fee for service, Medicaid, and none) as a surrogate for socioeconomic status for the purpose of our analysis.
If compliance with the regulations was poor, a favorable impact might not be observed. A group of hospitals filed suit to delay the enactment of Code 405 claiming insufficient time to achieve compliance. Recognizing the possibilities of delay (even though the court dismissed the lawsuit), we chose 1991 rather than 1990, the year immediately following enactment. Universal annual inspections for compliance were not performed but some hospitals were inspected, particularly those in New York City. The New York State Department of Health's inspections from 1990 to 1994 were reported by the New York City Public Advocate's office in 1994.20 The earliest and most frequent violations were found in surgery programs while internal medicine departments were only cited after 1991. In addition, Laine et al. used 1988 and late (October) 1989 and Conigliaro et al. chose 1991 for their analyses. Thus, we believe the optimal years for comparison were 1988 and 1991 and optimal diagnoses were medical (CHF, AMI, PNU) and not surgical. Another strength of our study in choosing medical patients was that the regulations were more stringent for medicine than surgical trainees. It remains that we found no effect, favorable or unfavorable, from Code 405, and that lack of compliance is one possible explanation.
Based on our analysis and reports in the literature, there seems to be no clear evidence that reducing the work hours of internal medicine or family practice residents improves the outcomes of patients, though we did find reports of improvements in the personal lives of internal medicine and ob-gyn residents. Promoting and protecting the educational elements of residency by reducing work hours is sufficient justification for reform. Patient mortality (the sentinel case resulting in Code 405) was not affected in this instance of an attempt to control resident working conditions.
The authors wish to acknowledge Wei Chen, MS for data conversion and management, Richard Ittenbach, PhD for statistical consultation, and Carole McMonagle and Annette Ferraro for secretarial assistance.