As can be seen in Table match rates in all years (1999-2007) among residents who were 65 years old or older exceed 95% with slightly higher numbers pertaining to earlier years, precisely because we had more data to correct "errors".
Match Rate between MDS records and Medicare's Enrollment File from the CMS MDS Registry by calendar year
Table presents a sample description of the new admissions entering US nursing homes in 2000, 2002, 2004 and 2006 based upon the Medicare denominator file information including date of birth, gender and race along with several summary measures from the admission MDS records. As can be seen, the average age at first nursing home entry is 81 years and this has remained fairly stable over the course of the decade. On the other hand, we are seeing a substantial increase in the number of diagnoses reported on the Medicare hospital claim preceding patients' first admission to a nursing home. The steady increase in the ADL dependency of the admission population is consistent with the number of diagnoses, but we don't see that increase in impairment reflected in either the level of cognitive impairment or acuity as reflected by the CHESS score. Interestingly, while the age distribution didn't change much over the period (~ 20% for those 65-74 and over 30% for those over 85), the proportion of male admission did increase by over 2 percentage points.
General Characteristics of Population by Year
We examined the completeness of the MDS relative to those admitted under the SNF benefit and found that no MDS of any type was found for 9.4% of SNF episodes in 2000 but this decreased to only 5% by 2006. Nonetheless, only 82% of these assessments were of an admission type (admission, 5-days, or re-admission) which one would expect since this is required.
Next, we found that among deaths in the Medicare files, 84.4% had an MDS discharge record indicating death at discharge and an additional 12.4% had a discharge to some other location (generally hospital), meaning that fewer than 4% of deaths were missing a discharge record and by 2006 this was only 2.6%. Amongst the population of cases with an MDS discharge record, we found that by 2006 94.8% of individuals with a discharge had died according to the Medicare files, or just over 5% of MDS discharges had a death filed incorrectly.
Figure presents the proportion of Medicare paid hospitalizations from nursing home of fee-for-service beneficiaries which were recorded by an MDS discharge tracking form, indicating discharge to a hospital, within 3 days of the inpatient admission day (either before or after). As can be seen, the rate of MDS discharge records reporting a hospital discharge rose from about 81% in 1999 to almost 90% by 2006, with the largest improvement occurring around 2001.
Percent of Medicare Claims Hospitalizations Identified from MDS Discharge Records: 1999-2007. (N = 4,395,102).
We examined all the MDS discharge tracking forms indicating discharge to hospital. The number of discharge tracking forms to hospital rose from about 1.1 million in 1999 to almost 1.5 million in 2006. Throughout the period, we identified Medicare inpatient claims for only 78% of the discharge records. The timing of the Medicare claim was on the same day as the MDS discharge record for about 75-79% and within 1-7 days for about 15-18%. The remaining 6-10% had a discharge MDS that was filed during their hospital stay or even after the hospital stay ended.
Among the 22% of MDS discharge tracking forms without an associated Medicare hospitalization claim, between 81% and 87% were in a nursing home according to the Residential History File, and an increasing number were in the emergency department (5% in 1999 increasing to 8% in 2006) or under observation days in the acute hospital (7% in 1999 increasing to 9% in 2006). The proportion of MDS discharge assessments indicating hospitalization that could be matched to inpatient claims varied across states, for example, between 66.3% in Arizona to 85.6% in Kansas in 2005.
Table presents the results of comparing the MDS diagnoses on the admission assessment with those on the Medicare hospital claim discharge diagnoses for selected diagnoses. Presented are the PPV, the PPV inter-quartile range across the states and the sensitivity and specificity of the Claims based vs. the MDS based diagnoses. We conducted the analyses for all years between 2000 and 2006 but since the pattern of results was quite similar across all years, we only present the most recent year. Additional File 1
presents a summary of this information for all years of data. With a few exceptions, most of the diagnoses have PPV in excess of .6. Heart failure, diabetes and COPD/asthma/emphysema all had high PPV levels while Depression, stroke and any dementia had lower PPV levels. The PPV of Parkinson's Disease changed substantially over the study period, from .76 in 2000 to .60 in 2006, with a relatively high inter-quartile range but diabetes also declined over the period [see Additional File 1
]. The sensitivity levels of the MDS to identifying "true" positives in the Medicare claims are similar to the PPV with certain exceptions but, specificity levels of the MDS diagnoses was high, meaning there is substantial agreement with respect to a diagnosis not being present. By and large, the inter-state variation as measured by the inter-quartile range for most of the comparisons is relatively small, particularly for those diagnoses with high levels of PPV.
Positive Predictive Value of MDS based diagnosis relative to ICD-9 Diagnoses on the Medicare Hospital Claim for selected Diagnoses:
To address the issue of inter-facility variation in the correspondence between Medicare hospital discharge diagnoses and MDS based admission diagnoses, we calculated the PPV at the level of the individual facility for facilities with a minimum of 100 admissions during the course of 2006. Figure presents the distribution of facilities with various levels of PPV for the diagnosis of heart failure. As can be seen, most facilities with large numbers of admissions from hospital in the year have reasonably high PPV levels when comparing Medicare hospital claims diagnoses with diagnoses on the admission MDS. Indeed, nearly 70% of facilities have a PPV in excess of 70% for heart failure, reflecting the high average of .78 for heart failure in 2006. However, some facilities have quite low levels of PPV in spite of the fact that they have many admissions per year directly from hospital.
Facility Variation in the Positive Predictive Value of Medicare Hospital Claims for Heart Failure and MDS admission assessments indicating a diagnosis of Congestive Heart Failure. N = 13,520 Nursing Facilities.
Table presents the results of the internal consistency analyses comparing how well MDS items that should be logically related actually do correspond in terms of the positive predictive value, along with the inter-state variation in the positive predictive value of the association between the two variables. (See Additional File 2
for tables summarizing these statistics for all years of data.) All the ADL related items demonstrate very high levels of internal consistency that has been very consistent over time. Interestingly, the correspondence between having a Cognitive Performance Scale score of zero (no obvious signs of cognitive impairment or memory loss) and the presence of a check-box diagnosis of Alzheimer's disease or any dementia is quite high (>.95) and the inter-quartile range based upon state level average PPV levels is very small, suggesting comparably high rates of association across the country. Joint pain, vision, asthma/COPD/emphysema were only moderately associated with their respective criterion variables but cancer and chemotherapy and pressure ulcer care and pressure ulcers were reasonably highly associated.
Positive Predictive Value and Internal Consistency of selected MDS items on the Admission MDS for 2006 and inter-quartile range across states
Researchers have constructed various multi-item scales from the item set in the MDS. Using data from the new admission cohort in 2006, we calculated the standardized Chronbach's alpha reliability (inter-item consistency) coefficient for the ADL scale, social engagement scale, the mood (depression) scale, the behavior problem index and the pain scale[20
]. We first tested Chronbach's alpha for the entire population of new admissions in 2006. Next, separately for strata defined based upon the median ADL score and the mid-point of the CPS, we calculated Chronbach's alpha for the four sub-groups since we anticipated that such different patient groups might exhibit different patterns of correlation among the items in the scales. The standardized reliability coefficient for the long form ADL scale was .90 (data available from authors upon request), the Social Engagement Scale was .63, the mood scale was .65, the behavior scale was .53 (although without the manic depression diagnosis item it is .66) and pain was .5.
Table presents the results of the stratified analyses, revealing standardized alpha coefficients for the sub-populations defined on the basis of ADL and CPS groups. As can be seen, they are relatively close to those of the overall population and close to the levels reported by the scale developers. The difference between the response patterns among the cohorts defined by the intersection of high and low ADL and CPS is not large, but reveals interesting patterns. Consistent with the expectation that staff have more difficulty assessing cognitively impaired residents, the standardized alpha coefficient for all scales is consistently lower among those with low CPS. The pattern with respect to high vs. low ADL performance is more subtle; while the low ADL cohort reveals lower reliability for social engagement and depression, the difference is quite small for behavior and seems to move in the opposite direction for pain intensity.
Standardized Alpha Multi-Item Scale Reliability & Internal Consistency Stratified by Median Activities of Daily Living
Various "validity" studies of the MDS and its applications have been undertaken, often comparing "research" measurements done by clinicians or research assistants with information in the most recent MDS in the residents' files. Another approach to testing the construct validity of some aspect of the MDS is to examine the relationship between selected items and concepts which the literature and clinical practice tells us should be related to readily measured "objective" outcomes like death or hospitalization. We tested the "predictive validity" of the CHESS scale, which was designed as a composite measure of medical stability, frailty and clinical acuity, to predict mortality amongst frail elders in institutional settings[31
]. We identified new admissions to US nursing homes in 2006 and observed them for at least one year, regardless of whether they remained in the nursing home, to determine whether they'd died according to the Medicare enrollment file. Figure summarizes the relationship between quartiles of the CHESS scale and one year mortality, stratified by age upon admission. As can be seen, there is a doubling in the one year mortality rate among 85 year olds between the lowest and highest quartile of the CHESS scale, from around .30 to .60. Perhaps because this is a new admission cohort, many of whom do not remain nursing home residents but are re-hospitalized and die or return under hospice care, we see a strong monotonic effect of chronological age categories on one year mortality within quartiles of the CHESS score until the highest risk CHESS category is reached, at which point age doesn't appear to matter. We conducted similaer analyses of the predictive value of the long form ADL scale as well as the CPS and both were found to be strongly related to one year mortality, although not as strong as the CHESS since it was designed precisely to be predictive of survival.
One Year Survival by CHESS Score Level and Age. N = 718,555.