|Home | About | Journals | Submit | Contact Us | Français|
To determine what factors contribute to successful appeals of nursing home deficiencies in the Informal Dispute Resolution (IDR) process.
We merged CMS data about IDRs with OSCAR data about nursing home characteristics. We performed multivariate statistical analyses to predict successful appeals as a function of characteristics of the deficiency being appealed, the survey that triggered the deficiency, characteristics of the nursing home, and the state.
All nursing homes nationally in the period 2005–2008.
Successful appeals were defined as those in which the deficiency was removed or its severity or scope reduced. Independent variables included the CMS measures of severity and scope of deficiency, abuse and neglect, substandard care, total number of deficiencies in the survey, whether the IDR was triggered by a survey or complaint, facility ownership and reputation, and state stringency of regulation.
26% of submitted IDRs were successful in 2005–2008. Success was more likely for less severe deficiencies, when deficiencies were triggered by a survey rather than a complaint, and when fewer deficiencies were included in the appeal. Facility ownership and state stringency of regulation were not significantly associated with the IDR success.
Overall, 2.6% of deficiencies issued were overturned through the IDR process. Further study is required to determine the appropriateness of these overturned cases and the opportunities they offer to improve the survey process.
The quality of care of nursing homes in the United States is regulated by standards adopted by both the federal and state governments. States monitor compliance with these standards through a system of annual inspections (called surveys) as well as ad-hoc inspections triggered by residents’ or families’ complaints. The surveys are executed by a team of professionals who issue deficiencies for those areas in which the facility is found to be in non-compliance. The federal government has set 175 standards that facilities have to meet, and some states add additional standards of their own.1,2 On average, facilities fail to meet about 7 federal standards in every inspection and only around 8% of facilities nationally meet all federal standards.3
Once cited for a federal deficiency, nursing homes can appeal. Until 1995 the only available option was the formal appeal, a lengthy and costly judicial process, starting with an administrative law judge and often culminating in the court system.4 Subsequently, the Centers for Medicare and Medicaid Services (CMS) introduced the Informal Dispute Resolution (IDR) process, which allows nursing homes to appeal deficiencies in a much faster, less costly, and less formal fashion.5 The IDR is an additional option, which nursing homes can choose without forfeiting their right to the formal appeal. A report by the Office of Inspector General showed that states were generally using the IDR process as specified in the state Operations Manual.6 Mukamel et al.5 have shown that about 10% of surveys and complaints are appealed through the IDR process, and that in general it is the more severe deficiencies for which nursing homes request an IDR. The propensity to use the IDR process varies substantially across states, ranging from no IDR requests in some states to over 30% of all deficiencies in others.
We examine the likelihood that an IDR request will be successful in reversing the deficiency or lowering its scope or severity. We use national data to provide descriptive statistics on the success rate that nursing homes experience and then apply multivariate regression techniques to understand the factors that contribute to their success.
We obtained CMS data about all IDR requests submitted between 2005 and 2008 by all nursing homes in the United States. The data included facility identifiers, the original deficiency (or tag) being disputed, its scope and severity, and the outcome of the IDR. The scope of the deficiency refers to the number of residents affected by it and the severity refers to the degree of harm inflicted. The outcome of the IDR could have been one of the following: complete removal of the deficiency, scope and/or severity reduction, wording change in the written report, or no change.
These data were matched to the 2004–2008 Online Survey, Certification, and Reporting (OSCAR) data and to a data file containing complaints filed against nursing homes from 2004 to 2008. Matching of IDRs to surveys or complaints was done by facility ID and date. The OSCAR data also provided information about the total number of beds in the facility and ownership status.
The unit of observation was the individual deficiency. During the study period there were a total of 20,930 deficiencies included in IDR requests. Of those, 5% could not be matched to the survey and complaint data and less than 1% had one other covariate missing. Thus the analytical sample included 19,738 deficiencies for which IDRs have been requested by 5,283 nursing homes. We further identified 168 deficiencies as outliers based on deviance residuals in the multivariate analysis described below. We excluded the whole survey or complaint associated with these outlier deficiencies. Thus the final sample included 18,766 deficiencies (89.7% of the initial sample) submitted by 5,251 nursing homes.
The dependent variable was a dichotomous variable defined to equal 1, if the appeal was successful, i.e. the deficiency was removed or the scope or severity were reduced. Otherwise it was set to zero. Thus the regression model predicts the odds of success.
The independent variables included the following:
We estimated a multivariate logistic model. The unit of analysis was the individual deficiency being appealed through the IDR process. The model predicted if the appeal succeeded, as a function of all the independent variables. Because observations were clustered within nursing homes and within states, the models included fixed state effects and random nursing home effects.
Table 1 shows descriptive statistics for the sample. The average nursing home requested an IDR for 3.81 deficiencies during the four years we studied. It succeeded, namely the deficiency was either completely removed or its severity and scope reduced, in 26.4% of its appeals.
Table 2 presents the results of the multivariate analysis. Odds ratio for dichotomous variables compare the odds for the indicated category to the odds of a reference category (e.g. non-profit facility to the reference of a for-profit facility). Odds ratios for continuous variables compare the odds between the variable at its mean value and the variable at its mean plus 1 standard deviation (SD). The following summarizes our findings:
The state stringency also is not a significant predictor of successful appeals.
The IDR is a relatively new policy initiative implemented by CMS to streamline the process that nursing homes can use if they choose to appeal deficiencies issued by state surveyors. Little is known about how it works. A previous study5 examined nursing homes’ decisions about whether to avail themselves of this mechanism. In this paper we examine what influences their success, once the IDR has been requested.
We observe an overall success rate of 26%. Absent a gold standard against which to determine both the false positive and false negative rates – namely the appeals that should not have been granted and were, and vice versa – we cannot determine if this rate is too low, too high, or appropriate. It is noteworthy, however, that when considered together with the overall rate of IDR submission rate of 10%, the overall rate of overturned deficiencies is low, at 2.6% of all issued deficiencies. Despite that, advocates often feel that the IDR process has weakened the survey process as a whole.10_ENREF_10_ENREF_10_ENREF_10_ENREF_10_ENREF_10 Addressing this issue is outside the scope of this study.
We find that the likelihood of success depends on characteristics of the specific deficiency being appealed, the IDR as a whole, as well as characteristics of the survey or complaint which triggered them. However, characteristics of the facility, such as ownership, do not matter. There does not seem to be a bias against a particular type of facility, at least to the degree that we can detect in this study.
Furthermore, we are not finding a relationship to the HRSI, a measure of the stringency of the regulatory process in each state. This is somewhat more surprising. Perhaps we are not detecting this effect as statistically significant because we are estimating models with fixed state effects and this variable is a state-level variable that is highly correlated with the state fixed effect, thus resulting in inflated standard errors. However, even if we ignore the statistical significance test, the OR estimated for a 1SD increase in the HRSI is only 1.05, the lowest OR of all the factors we estimated, suggesting that if indeed states’ propensity to regulate mattered, it is one of the least important factors in the success of appeals. It is also interesting that the OR is greater than 1, indicating that in more stringent states, everything else being equal, nursing homes are more likely to succeed in their IDRs. It is unclear what the reason for this might be, and further research is needed.
In terms of the most important factors influencing success, the two that stand out are whether the IDR was triggered by a survey or by a complaint, with ORs on the two extremes for the range of our estimates, at 1.74 and 0.66 respectively. This phenomenon should also be investigated further to determine if this large divergence can be attributed to the “accuracy hypothesis” – i.e. that complaints generate more accurate deficiencies, or the “political hypothesis” – i.e. that a complaint has a well defined constituency to which the regulator is accountable and, therefore, dismissing the deficiency is more difficult.
Whether a 26% success rate over all is reasonable is a matter of judgment. Some may view it as indicative of responsiveness to the concerns of nursing homes. Advocates_ENREF_1010 have raised concerns that the IDR process further weakens a regulatory process that has been criticized by the General Accounting Office as inadequate to ensure adequate levels of quality.11–15_ENREF_10_ENREF_10 Others, on the other hand, may argue that overturning a quarter of the deficiencies is a high rate, raising questions about the appropriateness of the survey process itself. However, if one combines the finding in this paper with the overall IDR submission rate reported in Mukamel et al.5 of 10%, then the real rate at which deficiencies are overturned is relatively low, at only 2.6%. Efforts to improve the survey process should focus on understanding these 2.6% of deficiencies, as these might be viewed as the “error rate” in the survey and complaint review process.
The authors gratefully acknowledge funding from the National Institutes of Aging, Grant# AG027420 and State Directors of Licensing and Certification Offices for insightful comments.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
None of the authors have any financial conflicts.
Dana B. Mukamel, Department of Medicine, University of California, Irvine, Health Policy Research Institute, 100 Theory Suite 110, Irvine, CA 92697-5800; Telephone: 949-824-8873; Fax: 949-824-3388; Email: dmukamel/at/uci.edu.
Yue Li, Community & Preventive Medicine; University of Rochester Medical Center, 601 Elmwood Ave. Room 1-9892, Rochester, NY 14642; Telephone: 585-275-3276; Email: yue_li/at/urmc.rochester.edu.
David L. Weimer, University of Wisconsin – Madison, LaFollette School of Public Affairs, 1225 Observatory Drive, Madison, WI 53706; Telephone: 608-262-5713, Fax: 608-265-3233, Email: weimer/at/lafollette.wisc.edu.
William D. Spector, Center for Delivery, Organization, and Markets, Agency for Healthcare Research & Quality; 540 Gaither Road, 5th Floor Rockville, MD 20852; Telephone: 301-427-1446; Fax: 301-427-1430; Email: WSpector/at/AHRQ.GOV.
Lauren Bailey, University of California, Irvine, Health Policy Research Institute, 100 Theory Suite 110, Irvine, CA 92697-5800; Telephone: 949-824-5867; Fax: 949-824-3388; Email: baileyl/at/hs.uci.edu.
Charlene Harrington, Department of Social and Behavioral Sciences, University of California, San Francisco, 3333 California Street Suite 455, San Francisco, CA 94118; Telephone: 415-476-4030; Fax: 415-476-6552; Email: charlene.harrington/at/ucsf.edu.