|Home | About | Journals | Submit | Contact Us | Français|
In July 2011, the Accreditation Council for Graduate Medical Education (ACGME) will implemented stricter duty-hour limits and related changes to the training environment. This may affect preventable adverse event (PAE) rates.
To estimate direct costs under various implementation approaches, and examine net costs to teaching hospitals and cost-effectiveness to society across a range of hypothetical changes in PAEs.
A decision-analytical model represented direct costs and PAE rates, mortality, and costs.
Published literature and publicly available data.
Patients admitted to hospitals with ACGME-accredited programs.
All teaching hospitals, major teaching hospitals, society.
ACGME’s 2011 Common Program Requirements.
Direct annual costs (all accredited hospitals), net cost (major teaching hospitals), cost per death averted (society).
Nationwide, duty-hour changes would cost $177 million annually if interns maintain current productivity, vs. up to $982 million if they transfer work to a mixture of substitutes; training-environment changes will cost $204 million. If PAEs decline by 7.2–25.8%, net costs to major teaching hospitals will be zero. If PAEs fall by 3%, the cost to society per death averted would be –$523,000 (95%-confidence interval: –$1.82 million to $685,000) to $2.44 million ($271,000 to $6.91 million). If PAEs rise, the policy will be cost-increasing for teaching hospitals and society.
The total direct annual cost nationwide would be up to $1.34 billion using nurse practitioners/physician assistants, $1.64 billion using attending physicians, $820 million hiring additional residents, vs. 1.42 billion using mixed substitutes.
The effect on PAEs is unknown. Data were limited for some model parameters.
Implementation decisions greatly affect the cost. Unless PAEs decline substantially, teaching hospitals will lose money. If PAEs decline modestly, the requirements might be cost-saving or cost-effective to society.
The online version of this article (doi:10.1007/s11606-011-1775-9) contains supplementary material, which is available to authorized users.
The optimal duty hours for resident physicians continue to be debated. The Accreditation Council for Graduate Medical Education (ACGME) limited duty hours in 2003. In 2008, an Institute of Medicine (IOM) report reviewed the growing body of literature and recommended stricter duty-hour limits, reduced workload, greater supervision, and more meticulous handoffs of patient care 1. Shortly thereafter, ACGME undertook its own reevaluation of duty-hour standards and implemented new Common Program Requirements in July 2011 2,3. Table 1 compares these duty-hour policies.
ACGME’s revised requirements address issues that the IOM raised and challenges with implementing the 2003 policies. Post-graduate-year-one residents (PGY1s, or interns) now work no more than 16 hours continuously and have on-site supervision at all times. For more senior (PGY2+) residents, the maximum extended (call) shift has declined from 30 to 28 hours in all but exceptional circumstances. Other duty-hour changes consist of eliminating continuity clinics on post-call days, relaxing the minimum time off between shifts from 10 to 8 hours, limiting night float to six consecutive nights, and restricting moonlighting. Related changes to the training environment include educating residents and faculty members about fatigue and patient safety, standardizing the process of granting autonomy, systematizing handovers of patient care, providing transportation or sleep facilities after extended shifts, and undergoing annual site visits by the ACGME 3,4.
Given the extent of these changes, their cost implications warrant evaluation. We previously estimated the cost of the IOM duty-hour policies 5, but the ACGME policies differ. In addition to informing the debate about duty-hour limits, examining the costs of different implementation approaches can facilitate planning and decision making by teaching institutions. Further, preventable adverse events (PAEs, injuries due to medical errors) and their costs should be considered. If residents commit fewer fatigue-related errors 6, PAEs could decline. Many factors influence patient safety, however, and reduced continuity of care might produce more PAEs.
The first objective of this analysis was, therefore, to estimate the direct costs associated with different strategies for implementing the 2011 ACGME policies. The second objective was to examine net costs to major teaching hospitals and cost-effectiveness to society (cost per PAE-related death averted), accounting for potential effects on PAEs. This analysis extends work commissioned by the ACGME 7.
The analysis considered cost implications from three perspectives: direct costs across all U.S. hospitals with ACGME-accredited residency programs, net costs to major teaching hospitals, and cost-effectiveness to society. Our first step was to calculate the direct annual costs associated with individual 2011 requirements. The second step was to develop a decision-analytical model representing these direct costs as well as PAE rates, mortality, and costs among patients admitted to teaching hospitals under the 2003 vs. 2011 policies. The model enabled us to perform probabilistic analyses that accounted for uncertainty in model parameters and produced means with 95% confidence intervals. An Appendix includes the model and parameters (available on line). When there was uncertainty about a particular parameter, we selected a most likely value and range of possible values. The time horizon was one year. All costs represent 2008 dollars.
To estimate direct costs for individual requirements, we ascertained baseline conditions (conditions between July 1, 2003 and July 1, 2011) and identified changes occurring after July 2011. We searched PubMed and examined recent systematic reviews to identify original research studies describing baseline conditions in U.S. residency programs 8,9. Individual requirements were included in the cost analysis if they: (1) differ from typical practices in residency programs at baseline, and (2) would involve quantifiable costs (e.g., we excluded abstract principles). Our on-line report to ACGME contains additional detail 7.
We focused on extended shifts among PGY1s (16-hour limit) and PGY2+ residents (28-hour limit) because most programs already adhered to the other 2011 requirements 8,10–13. Model parameters included the frequency and duration of extended shifts and current weeks on inpatient rotations (excluding night float and home call).
Baseline As of July 2010, 94% of internal medicine, surgery, and pediatrics programs did not adhere to the 16-hour limit for PGY1s 11; we assumed that no programs did in this analysis. As of 2007–2008, 7% of all residents did not adhere to the current 30-hour limit 12. We based other parameters on the 2007–2008 survey of all residents 12, a national survey of PGY1 residents from 2003–2004 14, 46 articles describing call in one or more programs 7,15, a survey of residents in multiple specialties at two hospitals 15, and other publications. (See on-line Appendix to this article and on-line report to ACGME).
2011 Policies Next, we quantified excess work (baseline work that would be beyond the new limit) for each duty-hour change.
For the 16-hour shift for PGY1s, ACGME representatives anticipate that most programs will reorganize schedules within rotations instead of transferring work among residents or to other providers; we call this Scenario X. PGY1s could work two 16-hour shifts during a 48-hour period, for example, rather than one 30-hour shift. Reorganizing schedules would preserve educational opportunities and continuity of care as well as limit costs. Small programs (≤3 PGY1s) will probably need to transfer excess work to other providers to ensure continuous coverage of patients 16.
However, at medium and large programs, PGY1s’ availability may be asynchronous with the work to be performed or the schedule changes may create inefficiencies in work flow, particularly for residents working at or above the 2003 limits. Consequently, we also estimated total direct costs assuming all PGY1s transfer all excess work to other providers; this is Scenario Y.
Next, we selected substitute providers. After 2003, programs shifted work among existing residents, hired midlevel providers (nurse practitioners and physician assistants), and gave attendings additional responsibilities; teaching hospitals hired more nurses 10,17–25. Few programs expanded, likely because the Balanced Budget Act caps Medicare-supported residency positions 26. On this basis, we considered four substitution strategies: a mixed-substitute, an all-midlevel, an all-attending, and an additional-residents strategy. We included only the portion of substitutes’ time that would be attributable to satisfying the new requirements, and assumed that work would distribute naturally among substitutes. We also considered the potential efficiency of substitutes relative to residents.
For the mixed-substitute strategy, we assumed that 0.5 hours of attending work plus 0.25 hours of nursing work would replace each hour of PGY1s’ excess work 15. For PGY2+ specialty residents and subspecialty residents, respectively, other residents and attending physicians would cover the two hours eliminated from the post-call day, creating opportunity costs (resources that can no longer be used for their original purposes). We selected the mixed-substitute strategy for analyses of net costs and cost-effectiveness because the mixed-substitute and midlevel strategies involved similar costs and appeared most plausible.
Four requirements met criteria for inclusion in the cost analysis: educating residents and faculty members about safety issues, providing transportation or sleep facilities for post-call residents, systematizing patient handovers, and undergoing annual site visits. Because few publications included relevant information, we made several assumptions in conjunction with ACGME representatives.
Baseline One publication describes baseline handover procedures 27. ACGME has yet to begin annual site visits. The other two requirements did not appear to have been widely implemented at baseline.
2011 Policies We presumed that residents and faculty would spend four hours in initial training and one hour in refresher training annually. Each extended shift, we surmised, would involve a round-trip taxi fare of $50 (range: $20–$80). The assumed cost of purchasing or developing computerized handover systems was $60,000 for large sponsoring institutions (range: $35,000–$80,000); $40,000 for medium institutions ($10,000–$50,000); and $10,000 for small institutions ($5,000–$35,000). We included the opportunity cost of handover activities, assuming one hour per PGY1–inpatient day (range: 1 to – 1 hour). Finally, we estimated direct and opportunity costs associated with annual site visits based on fees per visit, visit durations, and types of institutional representatives involved (residents, faculty members, hospital administrators, and managerial support staff) 16.
To estimate opportunity costs associated with changes to the training environment and to determine the cost of hiring additional residents, we valued residents’ labor based on the average hourly cost of stipends plus benefits (range: from this value up to a value based on average per-resident expenditures on graduate medical education from all sources). When transferring excess work from residents to other workers, we used wage and benefit data from the Department of Labor and salary data from the American Association of Medical Colleges, which were similar to other sources 5,28–30 (see online Appendix).
Baseline PAE incidence and attributable mortality were based on the one U.S. study that has measured these for a representative sample of hospitalizations, a study in Utah and Colorado from 1996 31,32. Event rates did not vary with hospital teaching status 33,34. Ranges for the probabilistic analyses accounted for limitations revealed by subsequent studies 35,36. We assumed that the relative risk of PAEs in teaching hospitals after 2003 was 0.99 (range: 0.9-1.0) because a systematic review identified several favorable studies but no clear pattern of improvement 8.
2011 Policies We assumed that the relative risk of PAEs under the 2011 vs. 2003 policies would range from 0.9 to 1.1.
We used the Utah/Colorado study to estimate average costs per PAE in 2008, adjusted for geographic and temporal cost trends. That study estimated disability, lifetime health care utilization, lost wages, and the cost of hiring other persons to perform household duties as of 1996 (discounting future costs at 2.75%). Drawing from a related study, we estimated the percentage of inpatient PAE costs absorbed by teaching hospitals 37,38.
We calculated the direct cost of individual requirements in Microsoft Excel, using the most likely values for each parameter. For duty-hour requirements, we multiplied hours of excess work per resident per week, numbers of affected weeks, the populations of affected residents, and substitutes’ hourly wages plus benefits. For training-environment requirements, multiplying the resources used by the cost per resource yielded one-time costs and annually recurring costs. To amortize one-time costs, we used a five-percent interest rate over five years.
We created the decision model in TreeAge Pro version 1.0.2. To produce each mean and 95%-confidence interval, we sampled from triangular distributions of probabilities for each model parameter in 10,000 Monte Carlo trials. Parameters were assumed to be uncorrelated.
Direct Costs We estimated total direct costs across all hospitals with ACGME-accredited programs as a function of the number of PGY1s transferring excess work to other providers, replicating the analysis for each of the four substitution strategies.
Net Costs and Cost-effectiveness Probabilistic analyses estimated net costs and cost-effectiveness as functions of the effect of the policy on PAEs. These analyses focused on major teaching hospitals (members of the Council of Teaching Hospitals, which includes large and medium sponsoring institutions). To address uncertainty about the number of PGY1s transferring excess work to other providers, we replicated the analysis for Scenarios X and Y. Different versions of the model represented the hospital and societal perspectives. Both versions included the direct annual cost of the 2011 policies. The major-teaching-hospital version included PAE costs absorbed by hospitals whereas the societal version included all PAE costs.
Table 2 lists direct annual costs of individual requirements relative to current policies. Nationwide, duty-hour changes would cost $177 million under Scenario X and $982 million under Scenario Y; training-environment changes will cost $204 million.
Figure 1 depicts the total direct annual cost nationwide of the four different substitution scenarios as a function of the number of PGY1s transferring excess work. Considering the range of mean modeled costs from Scenario X to Scenario Y, the mixed-substitute strategy would cost $398 million (95% Confidence Interval [CI] $225–$591 million) to $1.42 billion (95% CI $1.05–$1.91 billion). Results differ slightly from Table Table22 due to use of probability model rather than direct calculation. The midlevel-provider strategy would cost $499 million (95%-CI $294–716 million) to $1.34 billion (95% CI $1.07–$1.64 billion.) The attending strategy would cost $611 million (95%-CI $355–877 million) to $1.64 billion (95% CI $1.20–$2.18 billion.) The additional-residents strategy would cost $364 million (95%-CI $194–559 million) to $820 million (95% CI $519 million–$1.24 billion.)
Figures 2 through through44 examine the net cost and cost-effectiveness as functions of the policies’ potential effects on PAEs. From the major-teaching-hospital perspective (Figure 2), the 2011 policies could be cost-saving if PAEs decline >7.2% (relative risk [RR] <0.928) under Scenario X and >25.8% under Scenario Y (RR < 0.742).
From the societal perspective, a decline in PAEs of >1.9% (RR < 0.981) could make the policies cost-saving under Scenario X whereas a reduction of >6.8% (RR < 0.932) could under Scenario Y (Figure 3).
If PAEs fall by 1%, the incremental cost to society per PAE-related death averted (Figure 4) would be $1.76 million (95%-confidence interval [CI] -$286,000 to $5.71 million) per death averted under Scenario X vs. $10.5 million (CI $3.89 million to $24.6 million) under Scenario Y. If PAEs drop by 3%, Scenario X would cost -$523,000 (CI -$1.82 million to $685,000) per death averted and Scenario Y would cost $2.44 million (CI $271,000 to $6.91 million).
This analysis examines the cost implications of changes to resident physicians’ duty hours and the training environment that ACGME started requiring in July 2011. Relative to its 2003 policies, the 2011 policies could have a total annualized direct cost of about $398 million nationwide if implemented in the manner that ACGME anticipates. However, alternative implementation approaches could raise this to $1.3 billion or higher. Major teaching hospitals are likely to experience financial losses unless PAEs fall substantially. If direct costs are at the lower end of the range and PAEs decline by at least 2-3%, the policies could be cost-saving to society. They are less likely to be cost-effective if direct costs are higher and cannot be cost-effective if PAEs rise.
How do the 2011 requirements compare with alternative policies? Focusing on duty-hour limits, we previously estimated the cost of the IOM policies at $1.7 billion annually nationwide (inflated to 2008) 5. In comparison, the 2011 duty-hour requirements would cost $177 million to $982 million. The 2011 policies cost less mainly because extended shifts will change substantially for PGY1s rather than for all residents.
Although the ACGME requirements have now been implemented, duty-hour policies seem far from settled. In September 2010, the Occupational Health and Safety Administration (OSHA) received a petition arguing for even stricter limits. The agency announced, in surprisingly strong language, its intention to investigate 39,40. Then in October, the U.S. Supreme Court ruled, in an unrelated case, that residents are employees rather than students, eliminating long-standing uncertainty 41. This seemingly clarifies that OSHA has the authority to regulate resident duty hours. Should OSHA favor the limits advocated in the recent petition (Table 1), costs would be even higher than for the IOM duty-hour limits, possibly much higher, because all residents would have 16-hour shifts.
Although the 2011 duty-hour limits would cost less than alternative policies, the total burden to teaching hospitals could be substantial, depending on programs’ implementation decisions. While costs would be much lower if interns maintain their current productivity under modified schedules, educational opportunities may decline because optimal schedules could differ between learning and service objectives. After 2003, operative experience fell for some surgical residents, and elective rotations and teaching conference attendance declined at some Internal Medicine programs 8,20.
If educational or logistical concerns induce programs to hire additional providers, the next question is which providers to choose. Expanding the population of residents would be the least costly approach if the expenditures were limited to stipends and benefits. The Council for Graduate Medical Education has recommended increasing residency positions 42. However, residents need educational opportunities in diverse settings, not only on inpatient rotations, so adding positions can create sizeable indirect costs as well logistical concerns. Consequently, without changes to public financing policies, programs seem more likely to hire more nurse practitioners, physician assistants, and faculty. The Medicare Payment Advisory Commission (MedPAC) recently recommended exploring whether Medicare should vary for support residency positions across specialties, or base some payments to teaching institutions on the attainment of performance standards 43.
Implementation and financing issues will be heightened for small programs in community hospitals. Given class sizes, these programs will almost surely shift work to other providers. Minor teaching hospitals treat more Medicare patients and, therefore, may be disproportionately affected by financing policies 26. Finally, because residents care for only a fraction of the hospitals’ patients, reducing resident fatigue will not have a large effect on PAEs in these hospitals.
While reducing medical errors is one of the motivations for limiting duty hours, it is possible that errors could rise, even at major teaching hospitals. On the one hand, sleep deprivation impairs residents’ clinical performance. Shorter shifts have been associated with fewer medical errors in five out of five studies 8,44,45. On the other hand, patient handovers generally increase as duty-hours decline and the risk of PAEs is five-fold higher when interns are cross-covering 46. However, systematizing handover procedures can mitigate the risks associated with discontinuities of care 47. Also, the 2003 duty-hour limits did not appear to worsen patient outcomes 8. Thus, the requirement addressing patient handovers, together with the supervision and patient-safety education ones, increases the likelihood that the ACGME’s 2011 policies will have neutral or possibly favorable clinical effects.
If PAEs decline even modestly, this could make the ACGME policies cost-saving or cost-effective from the societal perspective. A 3% decline (5.7 fewer PAEs per 10,000 admissions) would be associated with cost-effectiveness ratios of –$520,000 to $2.4 million dollars per PAE-related death averted, for example. Regulatory agencies have generally considered $2.1 million to $7.9 million per statistical life to be cost-effective (inflated to 2008) 48. However, PAEs generally affect older and sicker individuals 36, whereas statistical lives are based on young healthy adults. Our 95%-confidence limits encompass this difference in life expectancy. For a 3% decline in PAEs, the upper 95%-confidence limit is $685,000 to $6.9 million per death averted, which still puts the 2011 policies within the potentially cost-effective range.
Our analysis has several limitations. The long-term cost-effectiveness of the 2011 policies could also be affected by physicians’ competence after completing training. Modeled costs may differ from future expenditures by residency programs. Transferring work on an hour-for-hour basis does not equate with hiring full-time-equivalent staff members at the program level. However, modeling can estimate costs when programs are still deciding how to implement the changes. We included opportunity costs, which do not represent actual expenditures but acknowledge that residents give up alternative uses of their time. Model parameters were based on disparate sources, each with its own limitations; however, probabilistic analyses addressed these uncertainties. We did not include some costs, such as recruiting and training substitutes. Training environment costs rested on assumptions, but single-variable sensitivity analysis demonstrated that these had small effects overall (see report to ACGME) 7. Finally, the data on PAEs and their costs are several years old.
In conclusion, how programs are implementing the new ACGME policies is not yet known but this will have a substantial effect on the total direct cost. The new policies will result in net financial losses for teaching hospitals unless PAEs decline substantially. The effect of the 2011 ACGME Common Program Requirements on PAEs is not yet known, but modest declines in PAEs might make these policies cost-saving or cost-effective to society.
An earlier version of this analysis was commissioned by the Accreditation Council for Graduate Medical Education (ACGME). ACGME representatives provided information related to selected model parameters, as specified in the paper. The authors were wholly responsible for conducting the analysis and preparing the manuscript; ACGME played no other roles in these activities. The authors performed a similar analysis for the Institute of Medicine in 2008. The authors have no other conflicts of interest. Teryl Nuckols, MD, MSHS is currently supported by a Mentored Clinical Scientist Career Development Award (K08) from the Agency for Healthcare Research and Quality (grant number HS17954).
The authors thank Jan Wilson, who provided research support, and Jodi Friedman, MD, who provided helpful feedback on the analysis.
An erratum to this article can be found at http://dx.doi.org/10.1007/s11606-011-1827-1.