Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Am Med Dir Assoc. Author manuscript; available in PMC Feb 1, 2014.
Published in final edited form as:
PMCID: PMC3563759
What factors contribute to successful appeals of nursing homes’ deficiencies in the informal dispute resolution process?
Dana B. Mukamel, Ph.D., Professor and Senior Fellow,corresponding author Yue Li, Ph.D., Associate Professor, David L. Weimer, Ph.D., Professor, William D. Spector, Ph.D., Senior Social Scientist, Lauren Bailey, M.S., and Charlene Harrington, Ph.D., R.N.
Dana B. Mukamel, Department of Medicine, University of California, Irvine, Health Policy Research Institute, 100 Theory Suite 110, Irvine, CA 92697-5800; Telephone: 949-824-8873; Fax: 949-824-3388; dmukamel/at/;
corresponding authorCorresponding author.
To determine what factors contribute to successful appeals of nursing home deficiencies in the Informal Dispute Resolution (IDR) process.
We merged CMS data about IDRs with OSCAR data about nursing home characteristics. We performed multivariate statistical analyses to predict successful appeals as a function of characteristics of the deficiency being appealed, the survey that triggered the deficiency, characteristics of the nursing home, and the state.
All nursing homes nationally in the period 2005–2008.
Successful appeals were defined as those in which the deficiency was removed or its severity or scope reduced. Independent variables included the CMS measures of severity and scope of deficiency, abuse and neglect, substandard care, total number of deficiencies in the survey, whether the IDR was triggered by a survey or complaint, facility ownership and reputation, and state stringency of regulation.
26% of submitted IDRs were successful in 2005–2008. Success was more likely for less severe deficiencies, when deficiencies were triggered by a survey rather than a complaint, and when fewer deficiencies were included in the appeal. Facility ownership and state stringency of regulation were not significantly associated with the IDR success.
Overall, 2.6% of deficiencies issued were overturned through the IDR process. Further study is required to determine the appropriateness of these overturned cases and the opportunities they offer to improve the survey process.
Keywords: nursing homes, quality, deficiencies, regulation, appeal
The quality of care of nursing homes in the United States is regulated by standards adopted by both the federal and state governments. States monitor compliance with these standards through a system of annual inspections (called surveys) as well as ad-hoc inspections triggered by residents’ or families’ complaints. The surveys are executed by a team of professionals who issue deficiencies for those areas in which the facility is found to be in non-compliance. The federal government has set 175 standards that facilities have to meet, and some states add additional standards of their own.1,2 On average, facilities fail to meet about 7 federal standards in every inspection and only around 8% of facilities nationally meet all federal standards.3
Once cited for a federal deficiency, nursing homes can appeal. Until 1995 the only available option was the formal appeal, a lengthy and costly judicial process, starting with an administrative law judge and often culminating in the court system.4 Subsequently, the Centers for Medicare and Medicaid Services (CMS) introduced the Informal Dispute Resolution (IDR) process, which allows nursing homes to appeal deficiencies in a much faster, less costly, and less formal fashion.5 The IDR is an additional option, which nursing homes can choose without forfeiting their right to the formal appeal. A report by the Office of Inspector General showed that states were generally using the IDR process as specified in the state Operations Manual.6 Mukamel et al.5 have shown that about 10% of surveys and complaints are appealed through the IDR process, and that in general it is the more severe deficiencies for which nursing homes request an IDR. The propensity to use the IDR process varies substantially across states, ranging from no IDR requests in some states to over 30% of all deficiencies in others.
We examine the likelihood that an IDR request will be successful in reversing the deficiency or lowering its scope or severity. We use national data to provide descriptive statistics on the success rate that nursing homes experience and then apply multivariate regression techniques to understand the factors that contribute to their success.
We obtained CMS data about all IDR requests submitted between 2005 and 2008 by all nursing homes in the United States. The data included facility identifiers, the original deficiency (or tag) being disputed, its scope and severity, and the outcome of the IDR. The scope of the deficiency refers to the number of residents affected by it and the severity refers to the degree of harm inflicted. The outcome of the IDR could have been one of the following: complete removal of the deficiency, scope and/or severity reduction, wording change in the written report, or no change.
These data were matched to the 2004–2008 Online Survey, Certification, and Reporting (OSCAR) data and to a data file containing complaints filed against nursing homes from 2004 to 2008. Matching of IDRs to surveys or complaints was done by facility ID and date. The OSCAR data also provided information about the total number of beds in the facility and ownership status.
The unit of observation was the individual deficiency. During the study period there were a total of 20,930 deficiencies included in IDR requests. Of those, 5% could not be matched to the survey and complaint data and less than 1% had one other covariate missing. Thus the analytical sample included 19,738 deficiencies for which IDRs have been requested by 5,283 nursing homes. We further identified 168 deficiencies as outliers based on deviance residuals in the multivariate analysis described below. We excluded the whole survey or complaint associated with these outlier deficiencies. Thus the final sample included 18,766 deficiencies (89.7% of the initial sample) submitted by 5,251 nursing homes.
The dependent variable was a dichotomous variable defined to equal 1, if the appeal was successful, i.e. the deficiency was removed or the scope or severity were reduced. Otherwise it was set to zero. Thus the regression model predicts the odds of success.
The independent variables included the following:
  • Variables describing the deficiency triggering the IDR request: 1) Severity/scope score of the deficiency being disputed, based on the CMS Health Inspection Score.7 Higher values mean higher severity, higher scope, or both. 2) A dichotomous variable indicating if the deficiency was one of 42 considered by CMS as “substandard quality of care” deficiencies. 3) A dichotomous variable indicating if the deficiency was one of 6 considered by CMS as “abuse or neglect.” 4) The severity/scope score of the worst deficiency in the same IDR application.
  • Variables describing the survey or complaint triggering the IDR request: 1) The total number of deficiencies in the same survey or complaint that triggered the IDR request. 2) The number of deficiencies included in the IDR the survey or complaint triggered. 3) Two dichotomous variables indicating if the disputed deficiency was issued during an annual survey (yes=1), or a complaint (yes=1) with the reference being an inspection that serves both as an annual survey and a complaint survey (both variables taking the value zero).
  • Variables describing the nursing home: 1) Two dichotomous variables indicating nonprofit ownership (yes=1) and government ownership (yes=1) with the reference being for-profit ownership (both variables taking the value zero). 2) Nursing home size measured by number of beds (divided by 10). 3) A variable measuring the reputation of the nursing home based on its historical performance defined as its total severity/scope score based on the CMS Score for all the deficiencies it received in the prior year. Larger values mean worse reputation.
  • State regulatory stringency: This variable, measured by the Harrington Regulation Stringency Index (HRSI),2 measures the stringency of states’ regulation relative to each other. Higher values indicate states that tend to be more stringent in the way they interpret the standards and the way they monitor compliance. It is defined as the sum of five standardized components describing the regulatory process, as follows: 1) the state average number of deficiencies per facility, 2) the percent of facilities with at least one deficiency, 3) the percent of facilities with a deficiency at G level or higher (actual harm or serious jeopardy to residents), 4) the percent of facilities with substandard care, and 5) the average number of state and federal Civil Monetary Penalties (CMPs) issued per facility. This measure has been shown in previous studies to have face validity and was used in measuring the cost,8 quality, and cost effectiveness9 of state regulation.
  • Year variables: Because the data spanned the period 2005–2008 we included dichotomous variables for the years 2006–2008 (with 2005 being the reference year) to control for any secular trends.
We estimated a multivariate logistic model. The unit of analysis was the individual deficiency being appealed through the IDR process. The model predicted if the appeal succeeded, as a function of all the independent variables. Because observations were clustered within nursing homes and within states, the models included fixed state effects and random nursing home effects.
Table 1 shows descriptive statistics for the sample. The average nursing home requested an IDR for 3.81 deficiencies during the four years we studied. It succeeded, namely the deficiency was either completely removed or its severity and scope reduced, in 26.4% of its appeals.
Table 1
Table 1
Descriptive Statistics
Table 2 presents the results of the multivariate analysis. Odds ratio for dichotomous variables compare the odds for the indicated category to the odds of a reference category (e.g. non-profit facility to the reference of a for-profit facility). Odds ratios for continuous variables compare the odds between the variable at its mean value and the variable at its mean plus 1 standard deviation (SD). The following summarizes our findings:
  • Severity of the disputed deficiency: The severity of the disputed deficiency lowers the likelihood of a successful appeal. The odds ratio (OR) for a 1 SD increase in the overall CMS severity/scope score is 0.87 (p=0.001). Similarly, abuse and neglect and substandard quality deficiencies lower the likelihood of success with ORs of 0.86 and 0.84 respectively (p=0.033 & p<0.001). Furthermore, the higher the severity and scope of the worst deficiency in the IDR package also lowers the likelihood of success. An increase of 1 SD results in an OR of 0.79 (p<0.001).
  • Variables describing the survey or complaint triggering the IDR request: While the number of deficiencies disputed at the same time lowers the likelihood of success of each individual deficiency (OR 0.63 for a 1SD increase with a p<0.001) a higher number of deficiencies in the survey or complaint that triggered the IDR increases the likelihood of success (OR of 1.17 and p<0.001).
  • Survey or complaint: Relative to IDRs disputing deficiencies resulting from a joint survey/complaint, IDRs triggered by annual surveys are much more likely to succeed, with an OR of 1.74 (p<0.001), and those triggered by a complaint are much less likely to succeed, with an OR of 0.66 (p<0.001).
  • Variables describing the nursing home: Ownership does not significantly affect the likelihood of success, when we control for the characteristics of the IDR itself and the characteristics of the triggering survey or complaint. Bed size was also not significant. However, the facility’s reputation effect, as measured by its previous year deficiencies, is important, with an OR of 0.93 (p=0.009).
Table 2
Table 2
Logistic Model Predicting Successful Appeal of Deficiencies via an IDR Request
The state stringency also is not a significant predictor of successful appeals.
The IDR is a relatively new policy initiative implemented by CMS to streamline the process that nursing homes can use if they choose to appeal deficiencies issued by state surveyors. Little is known about how it works. A previous study5 examined nursing homes’ decisions about whether to avail themselves of this mechanism. In this paper we examine what influences their success, once the IDR has been requested.
We observe an overall success rate of 26%. Absent a gold standard against which to determine both the false positive and false negative rates – namely the appeals that should not have been granted and were, and vice versa – we cannot determine if this rate is too low, too high, or appropriate. It is noteworthy, however, that when considered together with the overall rate of IDR submission rate of 10%, the overall rate of overturned deficiencies is low, at 2.6% of all issued deficiencies. Despite that, advocates often feel that the IDR process has weakened the survey process as a whole.10_ENREF_10_ENREF_10_ENREF_10_ENREF_10_ENREF_10 Addressing this issue is outside the scope of this study.
We find that the likelihood of success depends on characteristics of the specific deficiency being appealed, the IDR as a whole, as well as characteristics of the survey or complaint which triggered them. However, characteristics of the facility, such as ownership, do not matter. There does not seem to be a bias against a particular type of facility, at least to the degree that we can detect in this study.
Furthermore, we are not finding a relationship to the HRSI, a measure of the stringency of the regulatory process in each state. This is somewhat more surprising. Perhaps we are not detecting this effect as statistically significant because we are estimating models with fixed state effects and this variable is a state-level variable that is highly correlated with the state fixed effect, thus resulting in inflated standard errors. However, even if we ignore the statistical significance test, the OR estimated for a 1SD increase in the HRSI is only 1.05, the lowest OR of all the factors we estimated, suggesting that if indeed states’ propensity to regulate mattered, it is one of the least important factors in the success of appeals. It is also interesting that the OR is greater than 1, indicating that in more stringent states, everything else being equal, nursing homes are more likely to succeed in their IDRs. It is unclear what the reason for this might be, and further research is needed.
In terms of the most important factors influencing success, the two that stand out are whether the IDR was triggered by a survey or by a complaint, with ORs on the two extremes for the range of our estimates, at 1.74 and 0.66 respectively. This phenomenon should also be investigated further to determine if this large divergence can be attributed to the “accuracy hypothesis” – i.e. that complaints generate more accurate deficiencies, or the “political hypothesis” – i.e. that a complaint has a well defined constituency to which the regulator is accountable and, therefore, dismissing the deficiency is more difficult.
Whether a 26% success rate over all is reasonable is a matter of judgment. Some may view it as indicative of responsiveness to the concerns of nursing homes. Advocates_ENREF_1010 have raised concerns that the IDR process further weakens a regulatory process that has been criticized by the General Accounting Office as inadequate to ensure adequate levels of quality.1115_ENREF_10_ENREF_10 Others, on the other hand, may argue that overturning a quarter of the deficiencies is a high rate, raising questions about the appropriateness of the survey process itself. However, if one combines the finding in this paper with the overall IDR submission rate reported in Mukamel et al.5 of 10%, then the real rate at which deficiencies are overturned is relatively low, at only 2.6%. Efforts to improve the survey process should focus on understanding these 2.6% of deficiencies, as these might be viewed as the “error rate” in the survey and complaint review process.
The authors gratefully acknowledge funding from the National Institutes of Aging, Grant# AG027420 and State Directors of Licensing and Certification Offices for insightful comments.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
None of the authors have any financial conflicts.
Contributor Information
Dana B. Mukamel, Department of Medicine, University of California, Irvine, Health Policy Research Institute, 100 Theory Suite 110, Irvine, CA 92697-5800; Telephone: 949-824-8873; Fax: 949-824-3388; dmukamel/at/
Yue Li, Community & Preventive Medicine; University of Rochester Medical Center, 601 Elmwood Ave. Room 1-9892, Rochester, NY 14642; Telephone: 585-275-3276; yue_li/at/
David L. Weimer, University of Wisconsin – Madison, LaFollette School of Public Affairs, 1225 Observatory Drive, Madison, WI 53706; Telephone: 608-262-5713, Fax: 608-265-3233, weimer/at/
William D. Spector, Center for Delivery, Organization, and Markets, Agency for Healthcare Research & Quality; 540 Gaither Road, 5th Floor Rockville, MD 20852; Telephone: 301-427-1446; Fax: 301-427-1430; WSpector/at/AHRQ.GOV.
Lauren Bailey, University of California, Irvine, Health Policy Research Institute, 100 Theory Suite 110, Irvine, CA 92697-5800; Telephone: 949-824-5867; Fax: 949-824-3388; baileyl/at/
Charlene Harrington, Department of Social and Behavioral Sciences, University of California, San Francisco, 3333 California Street Suite 455, San Francisco, CA 94118; Telephone: 415-476-4030; Fax: 415-476-6552; charlene.harrington/at/
1. Winzelberg GS. The quest for nursing home quality: learning history's lessons. Archives of Internal Medicine. 2003;163(21):2552–2556. [PubMed]
2. Harrington C, Mullan JT, Carrillo H. State nursing home enforcement systems. J Health Polit Policy Law. 2004 Feb;29(1):43–73. [PubMed]
3. Centers for Medicare & Medicaid Services. [Accessed 3/15/11];Nursing Home Data Compendium. 2009
4. Abrams Fensterman Law Firm. [Accessed June 16, 2011];The"Step-Up" in Enforcement of Nursing Homes. Recent Survey Trends.
5. Mukamel DB, Weimer DL, Li Y, Bailey L, Spector WD, Harrington C. Nursing homes appeals of deficiencies: the informal dispute resolution process. J Am Med Dir Assoc. 2012 Jul;13(6):512–516. [PMC free article] [PubMed]
6. Office of Inspector General, Department of Health. Informal Dispute Resolution for Nursing Facilities. 2005 Mar; OEI-06-02-00750.
7. Centers for Medicare & Medicaid Services. [Accessed July 24, 2012];Design for Nursing Home Compare Five-Star Quality Rating System: Technical Users' Guide. 2010 Jul; 2010;
8. Mukamel DB, Li Y, Harrington C, Spector WD, Weimer DL, Bailey L. Does state regulation of quality impose costs on nursing homes? Medical Care. 2011 Jun;49(6):529–534. [PubMed]
9. Mukamel DB, Weimer DL, Harrington C, Spector WD, Ladd H, Li Y. The Effect of State Regulatory Stringency on Nursing Home Quality. Health Services Research. 2012 in press. [PMC free article] [PubMed]
10. Edelman TS. Comment on CMS-2435-P Civil Money Penalties for Nursing Homes. Washington, DC: Center for Medicare Advocacy; 2010.
11. U.S. General Accounting Office. Nursing Homes Quality: Prevalence of Serious Problems, While Declining, Reinforces Importance of Enhanced Oversight--Report to Congressional Requesters. Washington, DC: GAO; 2003.
12. U.S. Government Accountability Office. Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not Deterred Some Homes from Repeatedly Harming Residents. Washington, D.C.: GAO; 2007.
13. U.S. Government Accountability Office. Nursing Homes: Federal Monitoring Surveys Demonstrate Continued Understatement of Serious Care Problems and CMS Oversight Weakness. Washington, DC: GAO; 2008.
14. U.S. Government Accountability Office. Nursing Homes: Addressing the Factors Underlying Understatement of Serious Care Problems Requires Sustained CMS and State Commitment. Washington, DC: GAO; 2009.
15. Office USGA. Nursing Homes: Some Improvement Seen in Understatement of Serious Deficiencies, but Implications for the Longer-Term Trend Are Unclear. Washington, DC: GAO; 2010.