The ACGME-accredited surgical-specialty programs must submit a final summary of their residents' comprehensive surgical experience gained during training, which is submitted after the resident has completed training. Throughout training, the residents submit data through a web-based mechanism called the ACGME Resident Case Log System. The ACGME uses the system only for program evaluation purposes, not to evaluate individual residents.
We limited our analyses to the 2 subspecialties of pediatric surgery and vascular surgery because these were the only surgical subspecialties in which case log data were collected before and after the duty hour limits went into effect in July 2003. The analyses were limited to those procedures in which the resident served in the role of surgeon. We excluded data in which the resident served in a role other than surgeon, such as teaching assistant or first assistant, because nonsurgeon data are not a primary focus of the Residency Review Committee case-log review process and the number of these procedures recorded in the case log system is low. In addition, although we believe the self-reporting of surgeon data is accurate, we were concerned that residents may not have been as diligent in fully logging nonsurgeon data.
Our analyses examined pediatric surgery case-log data from 2002 to 2003 through the 2007 to 2008 academic year. These data included the total number of procedures performed as surgeon, as well as the total numbers for the specific groups of procedures known as defined categories (tumors, neonate, and important pediatric surgical cases). The defined categories are an important subset of all the procedures tracked by the Residency Review Committee that make it easier to compare procedural groupings across years.
We also examined vascular surgery data from 2002 to 2003 through the 2006 to 2007 academic year. The vascular surgery data included total major procedures performed as surgeon as well as surgical volume for the vascular-defined categories (abdominal, cerebrovascular, peripheral, complex, endovascular-diagnostic, endovascular-therapeutic, and endovascular-graft procedures). During the course of data collection, vascular surgery programs began to move from an accredited length of training of 1 year to 2 years. To reduce the possibility of confounding because of change in program length (1 year versus 2 years), we limited our analyses to those residents having 2 years of training. In addition, the procedural classifications for vascular surgery changed appreciably in July of 2007. We did not use any vascular data after that point in our analyses.
We performed all statistical comparisons with the SAS statistical software package version 9.2 (SAS Institute, Cary, NC). To determine whether academic year affected the total number of programs or graduating residents, we performed 1-way goodness-of-fit χ2 tests. To determine whether academic year affected the average number of graduates per program, we used 1-way analysis of variance.
We used hierarchical linear modeling23
to measure changes in operative volume by year. Hierarchical linear modeling is a regression-based procedure that can be applied to repeated-measures data and accommodates both continuous and categorical variables. Hierarchical linear modeling is particularly helpful for analysis of clustered data where the predictors are nonindependent. That is the case here in which we have data at 2 levels within an organizational hierarchy—residents within programs. In all analyses, we also specified the conservative Kenward-Roger approximation as the method for computing the denominator degrees of freedom for the fixed effects.