Potential Unintended Consequences Due to Medicare’s “No Pay for Errors Rule”? A Randomized Controlled Trial of an Educational Intervention with Internal Medicine Residents

doi:10.1007/s11606-010-1395-9

J Gen Intern Med. 2010 October; 25(10): 1097–1101.

Published online 2010 June 8. doi: 10.1007/s11606-010-1395-9

PMCID: PMC2955462

Potential Unintended Consequences Due to Medicare’s “No Pay for Errors Rule”? A Randomized Controlled Trial of an Educational Intervention with Internal Medicine Residents

Somnath Mookherjee, MD,¹ Arpana R. Vidyarthi, MD,¹ Sumant R. Ranji, MD,¹ Judy Maselli, MSPH,² Robert M. Wachter, MD,¹ and Robert B. Baron, MD, MS¹

¹Department of Medicine, Division of Hospital Medicine, University of California, San Francisco, 533 Parnassus Avenue, U-101, Box 0131, San Francisco, CA 94143 USA

²Department of Medicine, Division of General Internal Medicine, University of California, San Francisco, 3333 California St, Box 1211, San Francisco, CA 94118 USA

Somnath Mookherjee, Fax: +1-415-5142094, Email: smookherjee/at/medicine.ucsf.edu.

Corresponding author.

Received November 15, 2009; Revised April 9, 2010; Accepted April 20, 2010.

Abstract

BACKGROUND

Medicare has selected 10 hospital-acquired conditions for which it will not reimburse hospitals unless the condition was documented as “present on admission.” This “no pay for errors” rule may have a profound effect on the clinical practice of physicians.

OBJECTIVE

To determine how physicians might change their behavior after learning about the Medicare rule.

DESIGN

We conducted a randomized trial of a brief educational intervention embedded in an online survey, using clinical vignettes to estimate behavioral changes.

PARTICIPANTS

At a university-based internal medicine residency program, 168 internal medicine residents were eligible to participate.

INTERVENTION

Residents were randomized to receive a one-page description of Medicare’s “no pay for errors” rule with pre-vignette reminders (intervention group) or no information (control group). Residents responded to five clinical vignettes in which “no pay for errors” conditions might be present on admission.

MAIN MEASURES

Primary outcome was selection of the single most clinically appropriate option from three clinical practice choices presented for each clinical vignette.

KEY RESULTS

Survey administered from December 2008 to March 2009. There were 119 responses (71%). In four of five vignettes, the intervention group was less likely to select the most clinically appropriate response. This was statistically significant in two of the cases. Most residents were aware of the rule but not its impact and specifics. Residents acknowledged responsibility to know Medicare documentation rules but felt poorly trained to do so. Residents educated about the Medicare’s “no pay for errors” were less likely to select the most clinically appropriate responses to clinical vignettes. Such choices, if implemented in practice, have the potential for causing patient harm through unnecessary tests, procedures, and other interventions.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-010-1395-9) contains supplementary material, which is available to authorized users.

KEY WORDS: errors, Medicare, unintended consequences, residents

INTRODUCTION

On 01 October 2008, Medicare’s “no pay for errors” rule went into effect for every acute care hospital in the United States¹. This policy is designed to improve the care of Medicare patients by incentivizing hospitals to prevent designated hospital-acquired adverse events. Medicare will not reimburse the hospital for increased costs due to any of these adverse events unless documented by the physician as having been present on admission.

While it is important to understand how physicians may change their practices as a result of these new reimbursement rules, it may be particularly important to understand the impact on residents. The Accreditation Council for Graduate Medical Education (ACGME) has mandated that trainees achieve competency in systems-based practice, which is described as “awareness of and responsiveness to the larger context and system of health care”². The implementation of this new Medicare rule provides the ideal conditions for a natural experiment to understand whether residents are achieving this competency. Residents care for a large number of Medicare inpatients³^,⁴ and potential changes in their practices as a consequence of this rule may have clinical ramifications for these patients. Moreover, characterizing the resident physician response to the rule provides a framework to understand the potential for “no pay for errors” rules to result in unintended consequences throughout our healthcare system.

We performed a randomized trial of a brief educational intervention, seeking to determine whether more knowledge of the changes in Medicare reimbursement might lead resident physicians to provide care that sought to maximize revenue while conflicting with evidence-based guidelines.

METHODS

Sites and Subjects

On October 1st, 2008, the Medicare rule on hospital-acquired conditions went into effect. We conducted our study at the University of California, San Francisco (UCSF) between December 2008 and March 2009. None of the three clinical sites at which residents rotate made a systematic effort to educate residents about the rule (except for one lecture attended by 7% of the residents) or encourage changes in documentation or practices because of the rule.

All UCSF internal medicine residents were eligible to participate (N

168). The study was approved by UCSF’s institutional review board.

Randomization

Eighty-four residents were assigned to each group by equal size block-randomization using the Microsoft Excel © computerized random number generator.

Intervention

We designed our brief educational intervention to resemble common ways that new billing, reimbursement, and documentation information is conveyed to clinicians, both in content and in appearance. Clinicians often receive emails and brief communications from various stakeholders seeking to improve reimbursement, providing information meant to influence documentation and billing practices. At our institution, physicians frequently receive emails from medical staff leaders on important patient safety, quality, and reimbursement issues comprised of a single paragraph summary of the issue followed by a bulleted list of salient data. Following this format, the initial part of our intervention was a one paragraph explanation of the rule, followed by a list of Medicare’s ten “no pay for errors” conditions. This statement was integrated into the survey for the intervention group and was visible on a standard sized computer monitor without scrolling.

Additionally, to simulate clinical reminders that aim to influence both the documentation and clinical practices of physicians⁵^,⁶, we inserted a single sentence prior to each clinical vignette in the intervention group, reminding the subject of the applicable “No pay for error” condition.

Measurements

To measure potential changes in behavior as a result of the intervention, we used an experimental vignette design⁷. In this study design, two groups of subjects are presented with identical vignettes and asked to make selections on multiple-choice clinical questions. One group is presented with additional information (intervention group) in order to determine how that information may influence their choices in comparison to the control group (Fig. 1).

Figure 1

Study flow of participants.

Experimental vignettes were initially developed by the principal investigator (SM), and then reviewed and modified by UCSF experts in the fields of medical education, outcomes research, and quality improvement. Questions measuring resident knowledge and attitudes underwent the same iterative process. The survey and vignettes were felt to represent strong content validity, and were pre-tested on non-internal medicine housestaff and recent internal medicine residency graduates.

Each vignette was designed to have a single most clinically appropriate answer. This correct clinical choice, however, might place reimbursement at risk. Two other answers were presented that were clinically inappropriate but were designed to protect hospital reimbursement in the face of the new policy. For example, for hospital-acquired “vascular catheter-associated infection,” a vignette described a patient without evidence of infection who needed a central line. Three options were presented: one option represented appropriate care (place the central line) and two reflected unnecessary or inappropriate care (place the line and send blood cultures or place the line and start prophylactic antibiotics) that might be seen as helping to defend hospital reimbursement in the context of the new Medicare policy. The final survey including the clinical vignettes is shown in the ^{Online Appendix}.

The primary outcome for this study was the selection of the pre-designated most clinically appropriate choice for the clinical vignettes.

Data Collection

The pre-randomized groups received identical email invitations, and were then directed to either the intervention or control group online survey, which used the proprietary Zoomerang © platform. A coffee card was offered as incentive to complete the survey. The survey was available from December 2008 through March 2009, and periodic reminders were sent to residents who had not completed it. Participation was voluntary, and consent was implied by completing the survey.

Statistical Analysis

In addition to direct comparisons between groups, we performed multivariate logistic regression to adjust for confounding baseline characteristics. Residents were dichotomized by baseline knowledge, by their sense of personal responsibility to “know new Medicare rules regarding hospital reimbursement,” and by the amount of teaching they had received about Medicare reimbursement prior to the survey. We used logistic regression to adjust for these baseline characteristics and determined odds ratios and 95% confidence intervals (95% CI). To compare the attitudes of the control and intervention groups, the means of responses on a five point Likert scale of the control group were compared to the means of the intervention groups’ responses using t-tests, and p-values were obtained. All analyses were carried out using SAS, version 9.2 (SAS Institute, Inc., Cary, NC).

RESULTS

Respondent Characteristics

Of 168 internal medicine residents surveyed, 119 responded (71%). There were no significant differences in the control and intervention groups in age breakdown, post-graduate year, or number of hours of teaching about Medicare reimbursement. The respondents were equally distributed across post-graduate year (Table 1).

Table 1

Demographic Information of Responding Internal Medicine Residents

Changes in Responses to Clinical Vignettes as a Result of Being Informed of the New Rule

Residents in the intervention group tended to be less likely than those in the control group to select the most appropriate clinical practice choice in the clinical vignettes. In four out of five clinical vignettes, a lower percentage of residents in the intervention group selected the most appropriate choice; two of these four differences were statistically significant (Table 2). These results were unchanged after a multivariate logistic regression adjusting for baseline knowledge, self-described responsibility to know about new Medicare reimbursement rules, self-described awareness of new Medicare rule, and hours of teaching about Medicare reimbursement (not shown).

Table 2

Responses to Clinical Vignettes-Percentage Selecting Most Appropriate Clinical Practice Choice

Baseline Knowledge Regarding Medicare Reimbursement Rules

There were no significant differences between the control and intervention groups in the baseline knowledge of residents regarding Medicare reimbursement rules. The large majority of residents were aware of the basics of Medicare, but most harbored misconceptions about the scope of the new “no pay for errors” policy, with 83% [n

99] believing incorrectly that hospitals would receive no reimbursement at all if a patient had a preventable adverse event, and 87% [n

103] believing that Medicare has forecasted over one billion dollars in cost savings from the policy, a vast overestimate⁸.

While residents tended to know that documenting that a “preventable adverse event” was present on admission allowed for full Medicare payment for that condition [n

108, 91%], they were unsure as to what the specific conditions were. For example, residents knew that conditions such as “Foreign Object Retained after Surgery,” [n

117, 98%] and “Falls and Trauma Occurring in the Hospital,” [n

114, 96%] were considered preventable complications by Medicare. However, only 52% [n

62] knew tham “Manifestations of Poor Control of Blood Sugar Levels” was a “no pay” condition and only 67% [n

80] knew that “Pulmonary Edema from Excessive Intravenous Fluids” was not.

Attitudes Around Medicare Reimbursement Rules and Preventable Complications

Residents strongly agreed that it was their responsibility to reduce hospital-acquired infections and preventable complications in their patients [using a five-point Likert scale with 5 = strongly agree, 1 = strongly disagree; mean score

4.8, SD 0.6]. They also agreed that it was their responsibility to know new Medicare rules affecting hospital reimbursement [mean 3.8, SD 0.9] and to change their documentation practices based on new Medicare reimbursement rule [mean 3.7, SD 0.9]. Residents did not feel, however, that they had been adequately trained in reimbursement, disagreeing with the statement “I have received sufficient training about Medicare reimbursement during residency to document appropriately in the medical record,” [mean 1.8, SD 0.8] (Table 3).

Table 3

Internal Medicine Residents’ Attitudes Around Medicare Reimbursement Rules and Preventable Complications

DISCUSSION

This randomized trial of a brief educational intervention of internal medicine residents showed that residents who were informed about the Medicare “no pay for errors” rule tended to be less likely to choose the most appropriate clinical practice choices in response to clinical vignettes. While actual behaviors were not measured in this study, if these clinical practice choices were to be implemented in practice, they could result in patient harm through unnecessary tests, procedures, and other interventions. Most residents were aware of the new rule, but most had important misconceptions regarding the scope and content of the rule, revealing an important gap in resident competence in systems-based practice.

There is a growing trend in the United States for insurers to use incentives and disincentives in an attempt to lower costs and increase quality of care⁹. One result of this movement is that clinicians now frequently receive brief communications seeking to modify their documentation and billing practices. It is critical for practicing physicians to be aware of these new regulations, as well as to appropriately respond to the steady stream of information, which is often limited and may well be misleading, meant to influence their clinical practices.

It has long been recognized that societal systemic “purposive” changes can result in unintended consequences¹⁰. This phenomenon has been described in other large health systems changes, such as the implementation of the “time to antibiotics for pneumonia” performance measure, which may result in the inappropriate early use of antibiotics in an attempt to excel on a performance measure¹¹^–¹³. Similarly, the physician response to the new “no pay for errors” rules could have major consequences, both intended and unintended¹⁴^,¹⁵. Our study supports concerns that giving clinicians information regarding “no pay for errors” and other reimbursement rules may lead to unintended consequences with the potential to harm patient care unless such education balances individual patient needs with a more systems and reimbursement-based emphasis.

These findings have implications for both graduate medical education and for the care of hospitalized patients in general. Most importantly, in the face of financial incentives and disincentives meant to influence their behavior, physicians must continue to make patient-centered, evidence-based clinical decisions. Our study demonstrates how physicians may infer that they are expected to perform certain actions simply as a result of being informed about a new rule. Although physician professionalism should act as a countervailing force when inevitable tensions between reimbursement and patients’ best interests arise, history tells us that reimbursement-centered decisions sometimes prevail¹⁶^,¹⁷. Therefore, when providing information to clinicians about changes in payment policies, the underlying goal should be emphasized: increased patient safety and quality of care, not necessarily increased reimbursement.

This study has several limitations. Most importantly, since clinical vignettes variably predict true clinical behavior¹⁸^–²¹, it is not possible to generalize these results to actual clinical practices. Based on this study, we are unable to determine whether residents changed their actual behaviors as a result of being informed of new reimbursement rules, and if so, whether this had a deleterious effect on patients. Secondly, the clinical vignettes were reviewed by experts in medical education, outcomes research, and quality improvement and were felt to have strong content validity, however, they did not undergo external validation. Finally, this study was performed with internal medicine residents at a single university-based training program. Therefore, these results may not be applicable to practicing clinicians or to training programs in community-based hospitals.

“No pay for errors” rules and similar incentives meant to lower costs and improve quality and safety are here to stay. Our data support concerns that these rules can result in unintended consequences. Therefore, education in systems-based practice should teach trainees about these and other reimbursement rules, while strongly emphasizing the need to follow evidence-based medicine and a patient-centered approach. Further clinical research will be needed to determine if physician practices actually change after the implementation of “No pay for errors” rules, and whether these changes positively or negatively impact the care of patients.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Online Appendix^{(84K, pdf)}

(PDF 84 kb)

Acknowledgements

Financial Support: UCSF Office of GME funded the survey and incentives to participate.

Prior presentation (earlier version of this work): Poster presentation at the Society For Hospital Medicine Annual Meeting, Chicago, May 16 2009.

Conflict of Interest Information None of the authors report a conflict of interest.

Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

References

1. CMS Improves Patient Safety For Medicare and Medicaid by Addressing Never Events. Centers for Medicare and Medicaid Services, Office of Public Affairs; August 04, 2008. Available at: http://www.cms.hhs.gov/apps/media/press/factsheet.asp?Counter=3224&intNumPerPage=10&checkDate=&checkKey=&srchType=1&numDays=3500&srchOpt=0&srchData=&keywordType=All&chkNewsType=6&intPage=&showAll=&pYear=&year=&desc=false&cboOrder=date. Accessed April 27, 2010.

2. ACGME Program Requirements for Resident Education In Internal Medicine. Accreditation Council for Graduate Medical Education 2009. Available at: http://www.acgme.org/acwebsite/rrc_140/140_prindex.asp. Accessed April 27, 2010.

3. Brotherton SE, Etzel SI. Graduate medical education, 2008–2009. JAMA. 2009;302(12):1357–72. doi: 10.1001/jama.2009.1221. [PubMed] [Cross Ref]

4. Teaching hospitals: What Roles Do Teaching Hospitals Fulfill? Vol. 2009. Washington, D.C.: Association of American Medical Colleges.; 2009. Available at: http://www.aamc.org/teachinghospitals.htm. Accessed April 27, 2010.

5. Galanter WL, Hier DB, Jao C, Sarne D. Computerized physician order entry of medications and clinical decision support can improve problem list documentation compliance. Int J Med Inform. 2008. [PubMed]

6. Richter E, Shelton A, Yu Y. Best practices for improving revenue capture through documentation. Healthc Financ Manage. 2007;61(6):44–7. [PubMed]

7. Ballard DW, Reed ME, Wang H, Arroyo L, Benedetti N, Hsu J. Influence of patient costs and requests on emergency physician decisionmaking. Ann Emerg Med. 2008;52(6):643–650. doi: 10.1016/j.annemergmed.2008.03.001. [PubMed] [Cross Ref]

8. Pear R. Medicare Says It Won't Cover Hospital Errors. The New York Times 2007 August 19, 2007.

9. Fuhrmans V. Insurers Stop Paying for Care Linked to Errors. Wallstreet Journal 2008 January 15, 2008.

10. Merton RK. The Unanticipated Consequences of Purposive Social Action. American Sociological Review. 1936;1(6):894–904. doi: 10.2307/2084615. [Cross Ref]

11. Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med. 2008;149(1):29–32. [PubMed]

12. Nicks BA, Manthey DE, Fitch MT. The Centers for Medicare and Medicaid Services (CMS) community-acquired pneumonia core measures lead to unnecessary antibiotic administration by emergency physicians. Acad Emerg Med. 2009;16(2):184–7. doi: 10.1111/j.1553-2712.2008.00320.x. [PubMed] [Cross Ref]

13. Welker JA, Huston M, McCue JD. Antibiotic timing and errors in diagnosing pneumonia. Arch Intern Med. 2008;168(4):351–6. doi: 10.1001/archinternmed.2007.84. [PubMed] [Cross Ref]

14. Wachter RM, Foster NE, Dudley RA. Medicare's decision to withhold payment for hospital errors: the devil is in the details. Jt Comm J Qual Patient Saf. 2008;34(2):116–23. [PubMed]

15. Saint S, Meddings JA, Calfee D, Kowalski CP, Krein SL. Catheter-associated urinary tract infection and the Medicare rule changes. Ann Intern Med. 2009;150(12):877–84. [PMC free article] [PubMed]

16. Gawande A. The Cost Conundrum: What a Texas town can teach us about health care. The New Yorker; July 1 2009. Available at: http://www.newyorker.com/reporting/2009/06/01/090601fa_fact_gawande. Accessed April 27, 2010.

17. Sox HC, ed. Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002;136(3):243–6. [PubMed]

18. Carey TS, Garrett J. Patterns of ordering diagnostic tests for patients with acute low back pain. The North Carolina Back Pain Project. Ann Intern Med. 1996;125(10):807–14. [PubMed]

19. Jones TV, Gerrity MS, Earp J. Written case simulations: do they predict physicians' behavior? J Clin Epidemiol. 1990;43(8):805–15. doi: 10.1016/0895-4356(90)90241-G. [PubMed] [Cross Ref]

20. Morrell DC, Roland MO. Analysis of referral behaviour: responses to simulated case histories may not reflect real clinical behaviour. Br J Gen Pract. 1990;40(334):182–5. [PMC free article] [PubMed]

21. Peabody JW, Luck J, Glassman P, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med. 2004;141(10):771–80. [PubMed]

Articles from Journal of General Internal Medicine are provided here courtesy of
Society of General Internal Medicine