Although insurers and public health agencies use outcome measures of inpatient care because of their ease of use and ready availability, these measures need appropriate validation and examination. Risk-adjusted readmission rates demonstrate this problem. Although early work found higher readmission rates at facilities with inadequate processes of care, such as poor discharge readiness and medication changes close to discharge (Ashton et al. 1997
), there is no conclusive evidence to use risk-adjusted readmission rates as a measure of quality of inpatient care for all conditions. These prior studies did not account for the quality of care provided by outpatient facilities after discharge. Our study examined the effect of both inpatient and outpatient facilities on risk-adjusted readmission rates at various time periods after discharge with premature infants treated in NICUs as a test case. We initially found a large variation in readmission rates across NICUs up to 1 year after discharge. These results are more complex, as much of this variation may be related to the outpatient facility attended by a child. In fact, outpatient facilities with higher readmission rates were more likely to have decreased quality of care, whereas no characteristic of poor-quality NICUs was associated with higher readmission rates. Outpatient facilities, not NICUs, were statistically associated with changes in readmission rates when both sites of care were included in the analysis. Therefore, before using readmission rates to measure the care of any inpatient facility, we must examine the impact of the outpatient care received by the patients after discharge. Alternatively, for outcomes such as readmission rates that measure the care of more than one provider, variations may suggest something about the quality of the inpatient–outpatient dyad, not one or the other. Because most outpatient providers admit patients to a limited number of facilities, similar analyses should occur to determine whether variations in readmission rates for other conditions measure the care provided by hospitals, outpatient practices, or a combination of the two.
Our study is consistent with prior work that failed to detect an association between lower-quality inpatient care and higher readmission rates. There are several potential explanations. Severity of illness was consistently associated with readmissions, especially during the first month after discharge, as seen in other studies (Weissman et al. 1999
; Kossovsky et al. 2000
;). When the effect of inpatient care on readmission risk is greatest, a patient's illness severity may overwhelm the effect from NICUs. We saw this effect, as patient factors explained 10 times more variation than NICU site of care within 3 months of discharge. It is possible that readmission rates immediately after discharge may distinguish higher-performing hospitals from lower-quality facilities at hospitals that see thousands of very-low-birth weight infants each year. However, our data suggest that illness severity will continue to be a powerful predictor of readmissions and hamper our ability to detect clinically relevant differences between hospitals.
Further after discharge, when the association between illness severity and readmission rates begins to lessen, other factors such as the quality of outpatient care and sociodemographic factors become more important. The problem with analyzing the effect of NICUs on the outcomes of premature infants after discharge is that the course of care for these infants is not limited to the care provided by the NICU; much of this care is also carried out in an outpatient setting. The fact that hospitals tend to discharge patients to a group of outpatient facilities must be accounted for in any analysis of readmission rates. When outpatient facilities were added to the typical, naïve model (Model N), little additional explained variation was added to the model. However, when we reverse the inclusion sequence, the amount of variation associated with outpatient facilities was reversed. This feature of the data, and the fact that characteristics of poor outpatient facilities, not characteristics of NICUs, were associated with higher readmission rates, support the idea that both inpatient and outpatient sites of care must be accounted for in any analysis of readmission rates as a quality measure.
The explained variation in readmission rates found in this study is consistent with other studies of length of stay or costs, which found an R2
of 10–18 percent for various risk adjustment models that predict health care payments and an R2
of 9–14 percent for predicted length of stay for patients with pneumonia (Schwartz and Ash 2003
). In each case, there is random variation in these outcomes. For readmissions, potential causes of random variation include accidental trauma or an outbreak of influenza. As with length of stay or cost, though, the combination of inpatient and outpatient facilities is significantly associated with variations in readmissions even though many readmissions are unavoidable.
We found increasing amounts of variation explained by outpatient sites as the time after discharge increased. Besides the quality of these outpatient facilities, the outpatient site may add information about the patients who attend that facility (Huang et al. 2005
). The choice to visit a given caregiver may provide additional information about the patient, which is not otherwise measurable. For example, a recent study of medical report cards found that approximately 40 percent of patients would choose a provider based on their interpersonal skills rather than their measured technical quality (Fung et al. 2005
). Other studies have shown that factors such as gender and race may influence the choice of a provider (Montgomery and Fahey 2001
; Schnatz et al. 2007
;). Some of the variations in readmission rate explained by outpatient site may be related to patient preferences, as well as the quality of care at that facility.
This study has several limitations. Our data included all inpatient and outpatient encounters with the medical system, linked to clinical data from the NICU hospitalization. However, it could not pinpoint the specific processes of care experienced by the child after discharge. Also, physicians who practice within a managed care system may be different than other physicians, possibly by adhering more rigidly to practice guidelines (Schur, Mueller, and Berk 1999
). This self-selection could affect the quality of care provided by the staff. Even so, we found significant differences in the care provided by individual NICUs, who had different styles of discharging prematurely born infants, and outpatient facilities. Finally, some inpatient or outpatient facilities may have unstable results because of their small volume of patients. Since some patients will continue to receive treatment at low-volume facilities, additional work needs to develop fair methods of measuring the care provided by these facilities. Our results remained similar after excluding low-volume facilities, suggesting that attendance at very small facilities was not the primary explanation for the results in our study.
In summary, measures such as readmission rates that are influenced by the care received by multiple different providers may not be able to assess the care of one group independent of the other. Similar evaluations should occur for other conditions before using risk-adjusted readmission rates to measure inpatient or outpatient quality of care.