The Impact of ACGME Work-Hour Reforms on the Operative Experience of Fellows in Surgical Subspecialty Programs

doi:10.4300/JGME-D-10-00174.1

J Grad Med Educ. 2011 March; 3(1): 111–117.

PMCID: PMC3186271

The Impact of ACGME Work-Hour Reforms on the Operative Experience of Fellows in Surgical Subspecialty Programs

Christopher Simien, PhD, Kathleen D Holt, PhD, and Thomas H Richter, MA

Corresponding author: Christopher Simien, PhD, Applications and Data Analysis Department, Accreditation Council for Graduate Medical Education, 515 N State Street, Suite 2000, Chicago, IL 60657, 312.755.7110, Email: csimien/at/acgme.org

Abstract

Background

In July 2003, the Accreditation Council for Graduate Medical Education (ACGME) introduced a set of regulations that mandated a reduction in the number of hours that medical residents can work. These requirements have generated controversy among medical educators, with some expressing concern that reducing resident hours may limit clinical exposure and competency, particularly in surgical specialties.

Objective

This study examines the impact of duty hour restrictions on resident operative experience in residents in 2 surgical subspecialties since the implementation of the ACGME duty hour limits.

Method

We examined operative log data for vascular surgery and pediatric surgery, using the academic year immediately preceding the duty hour restrictions, 2002 to 2003, as a baseline for comparison to subsequent academic years through 2006 to 2007 for vascular surgery and 2007 to 2008 for pediatric surgery.

Results

Graduating fellows in pediatric surgery showed no change in their total operative volume following duty hour restrictions. The pediatric-defined category of neonate procedures showed an increase following duty hour restrictions. Graduating fellows in vascular surgery showed an increase in total major procedures as surgeon. The vascular-defined categories of endovascular-diagnostic, endovascular-therapeutic, and endovascular-graft procedures also increased.

Conclusions

The reduction of duty hours has not resulted in a decrease in operative volume as some have predicted. Operative volume in pediatric surgery remained mainly unchanged, whereas operative volume in vascular surgery increased. We explore possible explanations for the observed findings.

Editor's Note: The ACGME News and Views section of JGME includes data reports, updates, and perspectives from the ACGME and its review committees. The decision to publish the article is made by the ACGME.

Introduction

In July 2003, the Accreditation Council for Graduate Medical Education (ACGME) introduced standards that limited the duty hours of medical residents in accredited programs. The purpose was to advance patient safety as well as to enhance resident education and well-being.¹

The requirements have generated controversy among medical educators with some expressing concern that reducing resident hours may limit clinical exposure and competency.² Although some studies have demonstrated a decrease in operative volume,^,3–9 much of the research into the effect of the duty hour restrictions has not shown a decrease in operative experience.¹⁰^–^,21 Most of these studies have focused on an evaluation of core specialties (those specialties that typically follow medical school and that lead to initial board certification). We examined the impact of duty hour restrictions in a previous study²² but limited our study to the examination of core specialties as well. Few studies have examined fellowship programs (which residents enter after completion of core-specialty programs and which allow residents to subspecialize). Thus, it is difficult to draw conclusions about the impact of duty hour restrictions on all levels of postgraduate education. We are aware of only 2 studies of the duty hour restrictions' impact on subspecialties,²⁰^,²¹ with both assessing the effect of the limits on pediatric surgery fellows. Although neither found a significant decrease in operative surgical experience, these studies may have been unable to detect differences because they were limited to relatively short periods and small sample sizes. To fully understand the impact of duty hour restrictions, research must include detailed data from both surgical residency and fellowship programs. Although core-specialty programs may show no effects of restrictions, it is possible that the restrictions could affect fellows' workload. Our current study uses detailed ACGME case volume data to examine changes in graduating fellows' surgical experience in 2 surgical subspecialties, pediatric surgery and vascular surgery.

Methods

The ACGME-accredited surgical-specialty programs must submit a final summary of their residents' comprehensive surgical experience gained during training, which is submitted after the resident has completed training. Throughout training, the residents submit data through a web-based mechanism called the ACGME Resident Case Log System. The ACGME uses the system only for program evaluation purposes, not to evaluate individual residents.

We limited our analyses to the 2 subspecialties of pediatric surgery and vascular surgery because these were the only surgical subspecialties in which case log data were collected before and after the duty hour limits went into effect in July 2003. The analyses were limited to those procedures in which the resident served in the role of surgeon. We excluded data in which the resident served in a role other than surgeon, such as teaching assistant or first assistant, because nonsurgeon data are not a primary focus of the Residency Review Committee case-log review process and the number of these procedures recorded in the case log system is low. In addition, although we believe the self-reporting of surgeon data is accurate, we were concerned that residents may not have been as diligent in fully logging nonsurgeon data.

Our analyses examined pediatric surgery case-log data from 2002 to 2003 through the 2007 to 2008 academic year. These data included the total number of procedures performed as surgeon, as well as the total numbers for the specific groups of procedures known as defined categories (tumors, neonate, and important pediatric surgical cases). The defined categories are an important subset of all the procedures tracked by the Residency Review Committee that make it easier to compare procedural groupings across years.

We also examined vascular surgery data from 2002 to 2003 through the 2006 to 2007 academic year. The vascular surgery data included total major procedures performed as surgeon as well as surgical volume for the vascular-defined categories (abdominal, cerebrovascular, peripheral, complex, endovascular-diagnostic, endovascular-therapeutic, and endovascular-graft procedures). During the course of data collection, vascular surgery programs began to move from an accredited length of training of 1 year to 2 years. To reduce the possibility of confounding because of change in program length (1 year versus 2 years), we limited our analyses to those residents having 2 years of training. In addition, the procedural classifications for vascular surgery changed appreciably in July of 2007. We did not use any vascular data after that point in our analyses.

Statistical Analyses

We performed all statistical comparisons with the SAS statistical software package version 9.2 (SAS Institute, Cary, NC). To determine whether academic year affected the total number of programs or graduating residents, we performed 1-way goodness-of-fit χ² tests. To determine whether academic year affected the average number of graduates per program, we used 1-way analysis of variance.

We used hierarchical linear modeling²³ to measure changes in operative volume by year. Hierarchical linear modeling is a regression-based procedure that can be applied to repeated-measures data and accommodates both continuous and categorical variables. Hierarchical linear modeling is particularly helpful for analysis of clustered data where the predictors are nonindependent. That is the case here in which we have data at 2 levels within an organizational hierarchy—residents within programs. In all analyses, we also specified the conservative Kenward-Roger approximation as the method for computing the denominator degrees of freedom for the fixed effects.

Results

Study Sample

We used the SAS PROC FREQ procedure to compute 1-way goodness-of-fit χ² tests on the number of programs and the number of graduating residents for each year since the implementation of duty hour restrictions. There were no significant changes in the total number of programs during this period for pediatric surgery (χ²

1.0732, df

5, P

.05), but the results were significant for vascular surgery reflecting the increase in programs with residents training for 2 years (χ²

15.1005, df

4, P

.01). There were also no changes in the total number of graduates in pediatric surgery (χ²

1.7143, df

5, P

.05), but vascular surgery results again showed a significant increase (χ²

25.1071, df

4, P

.0001). A 1-way analysis of variance revealed no differences in the number of graduates per program by academic year for pediatric surgery (F_5,158

1.52, P

.05) nor for vascular surgery (F_4,184

0.62, P

.05). The table provides descriptive information about the number of programs and graduates.

table

Population of Programs and Graduates

Operative Volume

Duty hour restrictions were in effect at the start of July 2003. Therefore, we selected the preceding academic year, July 1, 2002, through June 30, 2003, as the baseline for comparison to subsequent academic years in our analysis of the pediatric and vascular surgery data.

The total number of procedures per resident in pediatric surgery (see figure 1) was not significant (F_5,132

0.32, P

.05). No subsequent academic year had operative volume that was different from the baseline. The effects of academic year on operative volume were significant only in the neonate pediatric surgery-defined category (F_5,130

4.13, P

.01; see figure 2), and individual contrasts showed that each subsequent year following duty hour reform was significantly higher for this category than at baseline.

figure 1

Pediatric Surgery Average Total Procedures as Surgeon by Academic Year (Graduate Averages)

figure 2

Pediatric Surgery Average Total Procedures as Surgeon in Each Defined Category by Academic Year (Graduate Averages)

For vascular surgery, analyses showed that total procedures per resident were significant (F_4,171

9.90, P

.0001). Each academic year, except 2003 to 2004, which immediately followed the baseline, had an operative volume that was significantly higher compared with the baseline (see figure 3 ).). The effects of academic year on operative volume were significant in 3 of the 7 defined categories (see figure 4) in vascular surgery. These were the categories of endovascular diagnostic (F_4,172

4.05, P

.01), endovascular therapeutic (F_4,162

14.82, P

.0001), and endovascular graft (F_4,163

23.82, P

.0001). With the exception of 2003 to 2004 for endovascular therapeutic and endovascular graft, individual contrasts showed that each subsequent year following duty hour reform was significantly higher than the 2002 to 2003 baseline.

figure 3

Vascular Surgery Average Total Major Procedures as Surgeon by Academic Year (Graduate Averages)

figure 4

Vascular Surgery Average Total Procedures as Surgeon in Each Defined Category by Academic Year (Graduate Averages)

Discussion

The introduction of duty hour limits raised concerns that the restrictions would result in an unacceptable reduction in operative experience for residents training in the United States. Several studies have failed to consistently identify a decrease in operative volume since the passing of these restrictions. However, few of the research studies have examined the restrictions' impact on subspecialties. Although the responsibilities and educational objectives of fellows in subspecialties are different from residents in core specialty programs, the ACGME duty hour restrictions do not currently differentiate among the levels of training. Thus, we must assess the impact of duty hour restrictions on subspecialties to ensure that the restrictions do not more adversely affect these programs.

Our results do not show a decrease in operative experience following duty hour reform, suggesting that duty hour restrictions have not restricted the opportunities nor the experiences of these fellowship surgical programs. Interestingly, our data show, in some instances, an increase in operative volume. Although our analysis of the data in the case logs of pediatric surgery residents showed no change in total operative volume, there was an increase in 1 defined category of pediatric surgery, as well as an increase in the total number of procedures in vascular surgery and an increase in 3 of vascular surgery's 7 defined categories. It is unlikely that the duty hour restrictions directly caused these increases. Instead, other explanations are more likely. For example, the increase in vascular surgery is primarily within the endovascular-defined categories, which may reflect changing technology and a movement toward these less-invasive procedures.²⁴ In addition, the increases in both subspecialties may have occurred because these programs have instituted compensatory mechanisms, such as offsetting the workload of postgraduate fellows by transferring some patient care-related issues to physician extenders²¹ or by instituting modifications to program structure, such as adjustments to on-call systems.²⁵ An additional possibility for the increase in operative volume for pediatric and vascular fellows may be their greater participation in surgical cases to offset reduced participation by general surgery and integrated vascular surgery residents whose duty hours were also limited by the ACGME restrictions. Our major finding stands, however: duty hour restrictions did not decrease surgical operative volume in these surgical subspecialties.

It is important to note that the use of ACGME operative case volume data may have several limitations. First, although the issue of competence is outside the scope of the present investigation, we do not believe the number of procedures performed is the sole predictor of surgical competence. However, the ACGME Residency Review Committees use case volume extensively when programs are reviewed, and the committee members feel that case volume is a valid, though certainly not the only, tool for assessment of competence. Second, an underlying assumption of our article is that ACGME restrictions led to a decrease in duty hours. However, the case log data cannot directly address the question as to whether residents are complying with the duty hour standards. These data do not contain information on the number of duty hours actually worked by residents. The ACGME continues to monitor all programs closely to ensure compliance with the restrictions. Third, fellows or fellowship program staff members enter cases into the case log system, and data may at times contain errors, which may lead to underreporting or overreporting of case volume. For example, the primary purpose of case log data has been to document the satisfactory completion of a sufficient number of procedures. Residents may fail to record additional procedures after they have documented the required number,²⁶ resulting in an underreporting of actual cases. However, we would not expect this to account for the observed findings of increases over time in several of the procedures.

We should further note under limitations of this study that the numbers of procedures per graduate are cumulative, and the number of years served under duty hour restrictions influences these numbers. However, given that both pediatric surgery and vascular surgery are 2-year programs, all residents in the 2005 to 2006 academic years and after were serving under the new restrictions. In addition, although we examined all ACGME-accredited programs in pediatric and vascular surgery (2-year programs), there are small numbers of residents in these programs—1 or 2 residents per program. Finally, the ACGME program requirements stipulate that the resident provides preoperative and postoperative care as well as the operative procedure itself.²⁷^;²⁸ Currently, there is no way to quantitatively assess the performance of preoperative and postoperative care, and it is possible that duty hour restrictions have impacted these without reducing procedural experience. Future studies must also consider the complexity of operative cases to make sure that residents are not substituting a greater number of simple and repetitive procedures to the exclusion of more complex and intricate procedures.

Conclusions

The issue of regulating resident work hours continues to be the subject of active policy debate. Many have been concerned that duty hour reform would lead to a decline in operative volume and jeopardize the quality of resident training. Our data show that the patterns of operative volume for fellows have not declined. In fact, operative volume may have increased in vascular surgery and in limited areas within pediatric surgery despite duty hour reforms.

Footnotes

All authors are in the Applications and Data Analysis Department of the Accreditation Council for Graduate Medical Education. Christopher Simien, PhD, is Senior Data Analyst; Kathleen D. Holt, PhD, is Senior Data Analyst and Director of Special Projects and is Adjunct Professor in the Department of Family Medicine at the University of Rochester; and Thomas H. Richter, MA, is Director.

References

1. Philibert I, Friedmann P, Williams WT. New requirements for resident duty hours. JAMA. 2002;288(9):1112–1114. [PubMed]

2. Karamanoukian RL, Ku JK, DeLaRosa J, Karamanoukian HL, Evans GR. The effects of restricted work hours on clinical training. Am Surg. 2006;72(1):19–21. [PubMed]

3. Kairys JC, McGuire K, Crawford AG, Yeo CJ. Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees? J Am Coll Surg. 2008;206(5):804–811. [PubMed]

4. Damadi A, Davis AT, Saxe A, Apelgren K. ACGME duty hour restrictions decrease resident operative volume: a 5-year comparison at an ACGME-accredited university general surgery residency. J Surg Educ. 2007;64(5):256–259. [PubMed]

5. Carlin AM, Gasevic E, Shepard AD. Effect of the 80-hour work week on resident operative experience in general surgery. Am J Surg. 2007;193(3):326–329. [PubMed]

6. Feanny MA, Scott BG, Mattox KL, Hirshberg A. Impact of the 80-hour work week on resident emergency operative experience. Am J Surg. 2005;190(6):947–949. [PubMed]

7. Jarman BT, Miller MR, Brown RS, et al. The 80-hour work week: will we have less-experienced graduating surgeons? Curr Surg. 2004;61(6):612–615. [PubMed]

8. Blanchard MH, Amini SB, Frank TM. Impact of work hour restrictions on resident case experience in an obstetrics and gynecology residency program. Am J Obstet Gynecol. 2004;191(5):1746–1751. [PubMed]

9. Espey E, Ogburn T, Puscheck E. Impact of duty hour limitations on resident and student education in obstetrics and gynecology. J Reprod Med. 2007;52(5):345–348. [PubMed]

10. Mendoza KA, Britt LD. Resident operative experience during the transition to work-hour reform. Arch Surg. 2005;140(2):137–145. [PubMed]

11. Ferguson CM, Kellogg KC, Hutter MM, Warshaw AL. Effect of work-hour reforms on operative case volume of surgical residents. Curr Surg. 2005;62(5):535–538. [PubMed]

12. McElearney ST, Saalwachter AR, Hedrick TL, Pruett TL, Sanfey HA, Sawyer RG. Effect of the 80-hour work week on cases performed by general surgery residents. Am Surg. 2005;71(7):552–555. [PubMed]

13. de Virgilio C, Yaghoubian A, Lewis RJ, Stabile BE, Putnam BA. The 80-hour resident workweek does not adversely affect patient outcomes or resident education. Curr Surg. 2006;636:435–439. [PubMed]

14. Izu BS, Johnson RM, Termuhlen PM, Little AG. Effect of the 30-hour work limit on resident experience and education. J Surg Educ. 2007;64(6):361–364. [PubMed]

15. Shin S, Britt R, Britt LD. Effect of the 80-hour work week on resident case coverage. J Am Coll Surg. 2008;206(5):798–800. corrected article 2008;207(1):148–150. [PubMed]

16. Tran J, Lewis R, de Virgilio C. The effect of the 80-hour work week on general surgery resident operative case volume. Am Surg. 2006;72(10):924–928. [PubMed]

17. Barden C. B, Specht MC, McCarter MD, Daly JM, Fahey TJ., III Effects of limited work hours on surgical training. J Am Coll Surg. 2002;195(4):531–538. [PubMed]

18. Hassett JM, Nawotniak R, Cummiskey D, et al. Maintaining outcomes in a surgical residency while complying with resident working hour regulations. Surgery. 2002;132(4):635–639. [PubMed]

19. Schneider JR, Coyle JJ, Ryan ER, Bell RH, Jr, DaRosa DA. Implementation and evaluation of a new surgical residency model. J Am Coll Surg. 2007;205(3):393–404. [PubMed]

20. Spencer AU, Teitelbaum DH. Impact of work-hour restrictions on residents' operative volume on a subspecialty surgical service. J Am Coll Surg. 2005;200(5):670–676. [PubMed]

21. Ladd AP. Pediatric surgery fellowship compliance to the 80-hour work week. J Pediatr Surg. 2006;41(4):687–692. [PubMed]

22. Simien C, Holt KD, Richter TH, et al. Resident operative experience in general surgery, plastic surgery, and urology 5 years after implementation of the ACGME duty hour policy. Ann Surg. 2010;252(2):383–389. [PubMed]

23. Singer JD. Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat. 1998;24(4):323–355.

24. Grabo DJ, DiMuzio PJ, Kairys JC, McIlhenny SE, Crawford AG, Yeo CJ. Have endovascular procedures negatively impacted general surgery training? Ann Surg. 2007;246(3):472–477. [PubMed]

25. Nuthalapaty FS, Carver AR, Nuthalapaty ES, Ramsey PS. The scope of duty hour-associated residency structure modifications. Am J Obstet Gynecol. 2006;194(1):282–288. [PubMed]

26. Jagsi R, Shapiro J, Weissman JS, Dorer DJ, Weinstein DF. The educational impact of ACGME limits on resident and fellow duty hours: a pre-post survey study. Acad Med. 2006;81(12):1059–1068. [PubMed]

27. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in pediatric surgery. Available at: http://www.acgme.org/acWebsite/downloads/RRC_progReq/445_pediatric_surgery_07012007_u08102008.pdf. Accessed August 10, 2008.

28. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in vascular surgery. Available at http://www.acgme.org/acWebsite/downloads/RRC_progReq/450_vascular_surgery_07012007_u07012008.pdf. Accessed July 1, 2008.

Articles from Journal of Graduate Medical Education are provided here courtesy of
Accreditation Council for Graduate Medical Education