|Home | About | Journals | Submit | Contact Us | Français|
Validate risk-adjusted readmission rates as a measure of inpatient quality of care after accounting for outpatient facilities, using premature infants as a test case.
Surviving infants born between January 1, 1998 and December 12, 2001 at five Northern California Kaiser Permanente neonatal intensive care units (NICU) with 1-year follow-up at 32 outpatient facilities.
Using a retrospective cohort of premature infants (N=898), Poisson's regression models determined the risk-adjusted variation in unplanned readmissions between 0–1 month, 0–3 months, 3–6 months, and 3–12 months after discharge attributable to patient factors, NICUs, and outpatient facilities.
Prospectively collected maternal and infant hospital data were linked to inpatient, outpatient, and pharmacy databases.
Medical and sociodemographic factors explained the largest amount of variation in risk-adjusted readmission rates. NICU facilities were significantly associated with readmission rates up to 1 year after discharge, but the outpatient facility where patients received outpatient care can explain much of this variation. Characteristics of outpatient facilities, not the NICUs, were associated with variations in readmission rates.
Ignoring outpatient facilities leads to an overstatement of the effect of NICUs on readmissions and ignores a significant cause of variations in readmissions.
The assessment of health care quality has become more important with the increase in information available to the general public, which includes Medicare's “Hospital Compare” website (U.S. Department of Health & Human Services 2008) and the publication of hospital infection rates in Pennsylvania (Pennsylvania Health Care Cost Containment Council 2008). One proposed measure is risk-adjusted readmission rates. Early research (Ashton et al. 1997) suggested that lower-quality providers incompletely evaluate or manage a patient in 13 studies, leading to higher readmission rates. Similar results were noted more recently for coronary artery bypass graft surgery (Hannan et al. 2003). However, recently published work in congestive heart failure (Kossovsky et al. 2000; Luthi et al. 2004; Fonarow et al. 2007;) did not find such an association, leading to questions about the value of readmission rates as a quality measure (Clarke 2004). There are several potential reasons for these conflicting results. Hospitals may have different admission criteria for patients with a given disease, both for the initial hospitalization and for the readmission (Goodman et al. 1994). Available risk-adjustment models may not adequately control for these differences. The prevalence of readmissions may be too low to detect a difference between hospitals. Finally, most studies did not account for the outpatient facility where the patient receives care after discharge. Outpatient care may influence a patient's risk of readmission, as demonstrated by research on medication discrepancies (Coleman et al. 2005) or other interventions (Rich et al. 1995; Benbassat and Taragin 2000;). However, no study has examined the combined effect of inpatient and outpatient facilities on readmission rates.
To properly evaluate risk-adjusted readmission rates as an inpatient quality of care measure, we must examine a population of patients with high readmission rates, uniform admission criteria, and valid measures of inpatient care as a “test case.” Prematurely born infants are such a group. Besides their high overall readmission rates, (Cavalier et al. 1996; Furman et al. 1996; Joffe et al. 1999; Smith et al. 2004; Morris, Gard, and Kennedy 2005; Underwood, Danielsen, and Gilbert 2007;) all infants born under 34 weeks gestational age (GA) are admitted to a neonatal intensive care unit (NICU), which eliminates differences in admission criteria. Prior work has also suggested that higher quality NICUs have higher volumes and lower complication rates than their peers (Phibbs et al. 1996; Lorch et al. 2007; Phibbs et al. 2007;). Premature infants, who are interesting in their own right, also can serve as an analog for patients with other chronic illnesses, such as congestive heart failure. The goal of this study, then, was to assess risk-adjusted readmission rates as a measure of inpatient quality of care by studying their role in NICUs. The specific aims were to (1) determine the statistical significance and explained variation in risk-adjusted readmission rates attributable to the NICU when outpatient facilities are omitted—the most commonly used analysis; (2) determine the change in explained variation when outpatient facilities were added to the analysis; and (3) measure the association between readmission rates and characteristics of a high-quality NICU or outpatient facility (Phibbs et al. 1996; Lorch et al. 2007; Phibbs et al. 2007;). To answer these three aims, we will present the results of three separate analyses using data from a retrospective, population cohort of premature infants with complete follow-up data through 1 year after discharge. The first, “naïve” analysis determines the variation in readmission rates attributable to the NICU while ignoring outpatient facilities (aim 1). We then add outpatient facilities to the naïve model and determine the variation explained by NICUs and outpatient facilities (aim 2). The third aim of this study will clarify which previously validated facility-quality metrics are associated with lower readmission rates.
The Infant Functional Status Study (IFS) (Bakewell-Sachs et al. 2009) was a retrospective study of health resources used by premature infants at the Kaiser Permanente Medical Center Program (KPMCP), which is an integrated managed care organization whose perinatal outcomes have been previously described (Escobar et al. 1995; Escobar 1999; Joffe et al. 1999; Newman et al. 1999; Smith et al. 2004;). Infants eligible for the IFS study were born at five KPMCP hospitals between 1998 and 2001 with a GA of 34 weeks or less and received outpatient care at 1 of 32 outpatient practices between 1998 and 2002. The NICUs included in this study were labeled as level IIIA or IIIB units by the American Academy of Pediatrics and had an average daily census over 10. None of these units trained pediatric residents (Silber et al. 2009). Exclusion criteria included major congenital anomalies; need for mechanical ventilation at home after discharge; placement of a ventriculo-peritoneal shunt; and loss to follow-up within 1 year of discharge because of cancellation of their insurance. This project only examined higher-risk infants born at a GA of 32 weeks or less. Excluded infants in this GA range were similar to included infants in terms of demographic factors, length of NICU stay, and need for mechanical ventilation. The Institutional Review Boards of both The Children's Hospital of Philadelphia and KPMCP approved this project.
The initial source for our data was the Kaiser Permanente Neonatal Minimum Data Set, which prospectively collects information on demographic, maternal and birth history, and the postnatal hospital course for all NICU admissions in KPMCP. These data are validated monthly through random chart review (Escobar et al. 1997). We then linked these records to the KPMCP hospitalization database, the outpatient visit database, and the pharmacy database. Data on medical care occurring outside the KPMCP system, which made up <1 percent of all encounters, were also included using insurance claims received by KPMCP.
Each infant was assigned to the NICU that provided the majority of care over the last month of the child's NICU course. We assigned each infant to the outpatient facility used for a minimum of 50 percent of all well-child visits. These visits were identified through an ICD-9CM code of V20.0 or V20.1.
Our primary outcome was any hospital admission between 0 and 1 month of NICU discharge; 0–3 months; 3–6 months; and 3–12 months. The time frames were chosen to encompass the wide range of times (between 30 days after discharge and 1 year) in previous studies. Even though the validity of an association between inpatient care and a rehospitalization 90 or more days after discharge is suspect, we included these longer time frames to test whether ignoring outpatient facilities could result in an association between late rehospitalizations and inpatient care. As a secondary analysis, we limited our analysis to those conditions that may be reasonably associated with the care delivered by a NICU. These conditions included (1) all respiratory readmissions, as NICUs have different rates of bronchopulmonary dysplasia (BPD) (Zeitlin et al. 2008), (2) failure-to-thrive or placement of gastrostomy tube, as premature infants have high rates of feeding difficulties (Rommel et al. 2003), and (3) eye surgery from retinopathy of prematurity (Zeitlin et al. 2008).
We included factors previously associated with increased readmission rates of premature infants in the literature (Cavalier et al. 1996; Furman et al. 1996; Joffe et al. 1999; Smith et al. 2004; Morris, Gard, and Kennedy 2005; Underwood, Danielsen, and Gilbert 2007;). These factors included GA, measured from available clinical and ultrasound data; gender; mother's racial/ethnic status; and 12-hour SNAP score as a measure of initial illness severity. Four complications of prematurity were also included: BPD, defined as the need for supplemental oxygen support at 36 weeks postmenstrual age (Shennan et al. 1988); necrotizing enterocolitis, defined as Bell's stage II or greater (Walsh and Kliegman 1986); stage III or IV intraventricular hemorrhage (Papile et al. 1978); and stage II or higher retinopathy of prematurity (Anonymous 1984). The number of previous children from obstetric records was used as a proxy for the number of siblings currently in the household (Lorch et al. 2007). Individual information on income during the year of the study was not available, so we used the median household income of the home zip code of the family as a proxy measure (Krieger et al. 2003).
We developed a severity adjustment model using all confounding factors listed above except for medical complications because they may be associated with the quality of NICU care received by the child (Rosenbaum 1984). Non-significant variables were then dropped from the models one at a time until the reduced model was statistically different from the full model using the likelihood ratio test.
We defined several NICU characteristics that may be associated with poor quality: NICU volume and average postmenstrual age at discharge. Higher volume has been suggested as a measure of NICU quality (Phibbs et al. 1996, 2007; Lorch et al. 2007). Discharge at a younger postmenstrual age, when infants may be less physiologically mature, may result in higher readmission rates. We also defined several risk-adjusted characteristics of outpatient facilities that may represent nonrecommended care, based on the lack of treatment efficacy (Gadomski et al. 1994; Patel, Gouin, and Platt 2003;) or prior consensus conference recommendations (National Heart Lung and Blood Institute National Asthma Education and Prevention Program 1997, 2003).
We dichotomized each variable into higher- and lower-than-expected categories and added singly to each readmission model.
To determine the amount of variation attributable to the NICU or to the outpatient facility and address potential colinearity between these factors, we constructed several multivariable Poisson's models. We added NICU indicator variables or outpatient facility indicator variables to a model containing the patient factors listed above in the following order:
We used a measure of R2 (Cameron and Windmeijer 1996) for Poisson's models that maintains the interpretation of R2 as “explained variation.” We report the marginal contribution of the NICU and outpatient factors to the R2 calculation. The variation attributed to the NICU should be similar in the N+O and O+N models if the variation contributed by the NICU and outpatient facilities was independent of each other. However, the results would differ if some of the impact of NICUs on readmission rates is a result of the outpatient facilities they discharge patients to. Statistical significance was determined from the contribution of the NICU and outpatient facility indicators in full model. These p-values do not change whether we used the N+O model or the O+N model. For aim 3, we added facility and NICU characteristics to the final risk-adjustment model and report the incident rate ratio for each readmission outcome measure. Outpatient facility characteristics were added singly to the final model because of colinearity between these characteristics. Standard errors were calculated with bootstrap techniques because of potential overdispersion of the data.
Five NICUs and 32 outpatient sites were identified as sites of care for the 892 infants in this study. Each NICU sent children to a median of 12 outpatient centers (range 9–16) after discharge, whereas over 80 percent of the outpatient centers received infants from two or fewer NICUs. The infants had an average GA of 29.5±2.2 weeks and an average birth weight of 1.358±428 g. 16.6 percent had a diagnosis of BPD and 1.8 percent had NEC. 45.5 percent were white non-Hispanics, 20.5 percent were white Hispanics, and 11.2 percent were black. There were 330 readmissions within 1 year of discharge (Figure 1). The most common reasons for a readmission were respiratory viral illnesses, diarrhea, and dehydration. Over 52 percent of the readmissions occurred within 3 months of discharge, with a steady decrease until 12 months after discharge (p<.05 by ANOVA for trend). A similar trend was noted for NICU-sensitive readmissions.
Several medical and socioeconomic factors were associated with higher readmission rates (Table 1). Younger GA was strongly associated with higher readmission rates, regardless of the chosen time frame after discharge. NEC was the only complication associated with higher readmission rates 0–3 months after discharge. Among the socioeconomic factors, only additional children at home were associated with higher readmissions. SNAP score was not associated with any readmission measure and was dropped from the final models.
Figure 2 shows the amount of variation in each readmission measure that was explained by three groups of variables: patient-level factors, the NICU facility, and the outpatient facility. Patient-level factors predicted the largest amount of variation in readmission rates regardless of the time after discharge. In the naïve model, when NICUs were analyzed by themselves (Model N), the site of NICU care was significantly associated with readmissions 0–3, 3–6, and 3–12 months after discharge. When we added the outpatient facilities to this naïve model (Model N+O), very little additional variation was attributed to the outpatient facilities. However, when we included outpatient facilities in the analysis before the NICU (Model O+N), much of the variation in readmission rates is now attributed to the outpatient facility. For example, these three groups of factors explained 14.2 percent of the variation in readmissions between 3 and 12 months after discharge. Models N and N+O attribute 29.6 percent of this explainable variation to the NICU (4.2/14.2 percent, see Figure 2) and 6.3 percent of this explainable variation to the outpatient facility (0.9/14.2 percent). When we reverse the inclusion sequence, though, the amount of variation attributable to NICUs and outpatient facilities is reversed: now, 29.6 percent of the variation is attributed to the outpatient facilities and characteristics, while only 6.3 percent of the variation was attributed to the NICUs. The collinearity between NICUs and outpatient facilities makes it difficult to confidently assign variations in readmission rates to the NICUs alone using just this type of data.
In contrast to the naïve model (Model N), where NICUs were associated with readmissions at 0–3, 3–6, and 3–12 months after discharge, NICUs lost their significant association with readmission rates when both NICU and outpatient facilities were analyzed together. Outpatient facilities were significantly associated with readmissions between 3–6 and 3–12 months after discharge. These results did not change when we limited the study to patients who attended outpatient facilities that cared for 10 or more patients in our cohort. For the predefined list of NICU-sensitive conditions, neither the site of NICU care nor the site of outpatient care was associated with significant variation at any time point after discharge.
The results from our first two aims suggest that, when outpatient facilities are ignored from an analysis of NICUs, variations in readmission rates may be erroneously attributed to the NICU when they could be attributed to attendance at specific outpatient facilities or the combination of the two facility types. To further explore these results, we then examined the association between facility characteristics and readmission rates (Table 1). No characteristic of the NICU was associated with higher readmission rates except day of discharge and readmissions 3–6 months after discharge. Use of oral albuterol at higher-than-expected rates was associated with more readmissions within 3 months of NICU discharge [incident rate ratio (IRR) 1.74, 95 percent CI 1.18–2.56], and higher use of antibiotics for viral illness was associated with a higher rate of readmissions between 3 and 6 months after NICU discharge (IRR 2.09, 95 percent CI 1.08–4.07) and between 3 and 12 months after NICU discharge (IRR 1.92, 95 percent CI 1.11–3.31). Neither measure is recommended treatment (Figure 3). Our results did not change when we limited the analysis to patients who attended facilities that care for ten or more infants in our cohort. When we analyzed these factors as continuous variables, instead of dichotomized into “higher” or “lower-than-expected,” we found similar results. Use of oral albuterol at higher rates was associated with more readmissions within 3 months of NICU discharge, up to a 9 percent higher observed rate (IRR for 1 percent difference between observed and expected rates of use 1.06 [95 percent CI 1.02–1.10)]. Above this threshold, the risk of readmission decreased slightly, suggesting that the association is highest around 9 percent. When observed rates of antibiotic use increased from 0 to 5 percent above expected, each 1 percent increase was associated with a 41 percent (3–6 months after discharge) and 38 percent increase (3–12 months after discharge) in readmissions. Changes in antibiotic rates above the 5 percent level were not associated with any further increase in risk of readmission. No other facility variable when analyzed as a continuous measure was associated with a change in readmission rates at any time period.
Although insurers and public health agencies use outcome measures of inpatient care because of their ease of use and ready availability, these measures need appropriate validation and examination. Risk-adjusted readmission rates demonstrate this problem. Although early work found higher readmission rates at facilities with inadequate processes of care, such as poor discharge readiness and medication changes close to discharge (Ashton et al. 1997), there is no conclusive evidence to use risk-adjusted readmission rates as a measure of quality of inpatient care for all conditions. These prior studies did not account for the quality of care provided by outpatient facilities after discharge. Our study examined the effect of both inpatient and outpatient facilities on risk-adjusted readmission rates at various time periods after discharge with premature infants treated in NICUs as a test case. We initially found a large variation in readmission rates across NICUs up to 1 year after discharge. These results are more complex, as much of this variation may be related to the outpatient facility attended by a child. In fact, outpatient facilities with higher readmission rates were more likely to have decreased quality of care, whereas no characteristic of poor-quality NICUs was associated with higher readmission rates. Outpatient facilities, not NICUs, were statistically associated with changes in readmission rates when both sites of care were included in the analysis. Therefore, before using readmission rates to measure the care of any inpatient facility, we must examine the impact of the outpatient care received by the patients after discharge. Alternatively, for outcomes such as readmission rates that measure the care of more than one provider, variations may suggest something about the quality of the inpatient–outpatient dyad, not one or the other. Because most outpatient providers admit patients to a limited number of facilities, similar analyses should occur to determine whether variations in readmission rates for other conditions measure the care provided by hospitals, outpatient practices, or a combination of the two.
Our study is consistent with prior work that failed to detect an association between lower-quality inpatient care and higher readmission rates. There are several potential explanations. Severity of illness was consistently associated with readmissions, especially during the first month after discharge, as seen in other studies (Weissman et al. 1999; Kossovsky et al. 2000;). When the effect of inpatient care on readmission risk is greatest, a patient's illness severity may overwhelm the effect from NICUs. We saw this effect, as patient factors explained 10 times more variation than NICU site of care within 3 months of discharge. It is possible that readmission rates immediately after discharge may distinguish higher-performing hospitals from lower-quality facilities at hospitals that see thousands of very-low-birth weight infants each year. However, our data suggest that illness severity will continue to be a powerful predictor of readmissions and hamper our ability to detect clinically relevant differences between hospitals.
Further after discharge, when the association between illness severity and readmission rates begins to lessen, other factors such as the quality of outpatient care and sociodemographic factors become more important. The problem with analyzing the effect of NICUs on the outcomes of premature infants after discharge is that the course of care for these infants is not limited to the care provided by the NICU; much of this care is also carried out in an outpatient setting. The fact that hospitals tend to discharge patients to a group of outpatient facilities must be accounted for in any analysis of readmission rates. When outpatient facilities were added to the typical, naïve model (Model N), little additional explained variation was added to the model. However, when we reverse the inclusion sequence, the amount of variation associated with outpatient facilities was reversed. This feature of the data, and the fact that characteristics of poor outpatient facilities, not characteristics of NICUs, were associated with higher readmission rates, support the idea that both inpatient and outpatient sites of care must be accounted for in any analysis of readmission rates as a quality measure.
The explained variation in readmission rates found in this study is consistent with other studies of length of stay or costs, which found an R2 of 10–18 percent for various risk adjustment models that predict health care payments and an R2 of 9–14 percent for predicted length of stay for patients with pneumonia (Schwartz and Ash 2003). In each case, there is random variation in these outcomes. For readmissions, potential causes of random variation include accidental trauma or an outbreak of influenza. As with length of stay or cost, though, the combination of inpatient and outpatient facilities is significantly associated with variations in readmissions even though many readmissions are unavoidable.
We found increasing amounts of variation explained by outpatient sites as the time after discharge increased. Besides the quality of these outpatient facilities, the outpatient site may add information about the patients who attend that facility (Huang et al. 2005). The choice to visit a given caregiver may provide additional information about the patient, which is not otherwise measurable. For example, a recent study of medical report cards found that approximately 40 percent of patients would choose a provider based on their interpersonal skills rather than their measured technical quality (Fung et al. 2005). Other studies have shown that factors such as gender and race may influence the choice of a provider (Montgomery and Fahey 2001; Schnatz et al. 2007;). Some of the variations in readmission rate explained by outpatient site may be related to patient preferences, as well as the quality of care at that facility.
This study has several limitations. Our data included all inpatient and outpatient encounters with the medical system, linked to clinical data from the NICU hospitalization. However, it could not pinpoint the specific processes of care experienced by the child after discharge. Also, physicians who practice within a managed care system may be different than other physicians, possibly by adhering more rigidly to practice guidelines (Schur, Mueller, and Berk 1999). This self-selection could affect the quality of care provided by the staff. Even so, we found significant differences in the care provided by individual NICUs, who had different styles of discharging prematurely born infants, and outpatient facilities. Finally, some inpatient or outpatient facilities may have unstable results because of their small volume of patients. Since some patients will continue to receive treatment at low-volume facilities, additional work needs to develop fair methods of measuring the care provided by these facilities. Our results remained similar after excluding low-volume facilities, suggesting that attendance at very small facilities was not the primary explanation for the results in our study.
In summary, measures such as readmission rates that are influenced by the care received by multiple different providers may not be able to assess the care of one group independent of the other. Similar evaluations should occur for other conditions before using risk-adjusted readmission rates to measure inpatient or outpatient quality of care.
Joint Acknowledgment/Disclosure Statement: This project was funded by a grant from the Maternal and Child Health Bureau, grant number 1 R40 MC05474-01-00. The authors have no conflicts of interest to disclose. The authors would like to acknowledge Marla N. Gardner and John D. Greene for their assistance with data collection, and John D. Greene for his assistance with data management in this project.
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.