Healthcare providers strive to provide high quality care to their patients. However, in order to determine whether we are actually providing outstanding care, quality measures that are valid, reliable, and easy to collect are needed. We attempted to demonstrate that surfactant use in infants 30 to 34 weeks gestation with RDS may be a useful process measure of hospital quality.
Surfactant use in moderately preterm infants with RDS has the potential to be an ideal quality indicator if it can be accurately measured and can be shown to be a valid measure of quality. The best processes to use as quality indicators are those care practices with strong face validity and evidence linking them to improved patient outcomes. Although there are very few evidence-based therapies in neonatology, the efficacy of surfactant therapy for RDS has been proven in multiple clinical trials and a recent American Academy of Pediatrics report states that surfactant should be given to intubated infants with RDS regardless of exposure to antenatal steroids or gestational age [10
In an effort to establish the construct validity of surfactant use in infants 30 to 34 weeks' gestation with RDS, we have shown significant variation in rates of surfactant use across hospitals and that hospital surfactant use was associated with variation in two of the most commonly used measures of neonatal quality -- volume of admissions [16
] and in-hospital mortality [17
]. These findings are consistent with other examinations of variation in care practices published in the literature. For example, Horbar et al. documented variation in the use of early surfactant among VLBW infants similar to the variation we observed in larger preterm infants with RDS [13
]. Additionally, our finding that hospital rates of surfactant use are associated with NICU volume of admissions is consistent with previous reports demonstrating a relationship between NICU volume of admissions and improved mortality [16
]. While these results are encouraging, limitations in adequate measurement of this quality metric including defining RDS using administrative data, adequately adjusting for severity of illness, and accounting for care received prior to transfer preclude us from establishing the validity of this measure as a marker of neonatal quality of care. Additionally, this study did not attempt to demonstrate the stability of the proposed quality measure over time.
Process measures require accurate identification of the appropriate eligible patient population. Identifying patients with RDS who are eligible for surfactant treatment was particularly challenging using administrative data. The RDS definition used in this study included infants requiring either NCPAP or MV in the first 48 hours of life. We believe this is a reasonable, practical definition for a clinical condition whose definition has varied even across clinical trials of surfactant therapy [4
]. The definition included both NCPAP and MV in an effort to eliminate the effects of practice variation in ventilatory management across hospitals because the same infant, with the same severity of RDS, may be treated with NCPAP at one center and with MV at another center. However, in doing so, the denominator of eligible patients included some infants who could be successfully managed on NCPAP and some with more mild RDS who might not require surfactant treatment. The fact that there was still significant variation among hospitals in the use of surfactant when we used the ICD-9-CM diagnosis code to identify infants with RDS and when we restricted the population to only those infants requiring mechanical ventilation suggests that true variation exists. However, we acknowledge that there is a potential that systematic differences in RDS severity across hospitals or more liberal use of NCPAP in some centers could explain some of the variation in surfactant rates seen across hospitals. Future efforts to develop process measures in neonatology should focus on care practices with a clear and well-defined eligible patient population.
When the eligible patient population can be defined appropriately, process measures are typically insensitive to differences in case mix and severity of illness [1
]. However, outcome measures of quality such as rates of in-hospital mortality (as used in this study for testing construct validity) require adequate risk adjustment to allow for fair and accurate comparisons across hospitals [27
]. Administrative data is the most accessible comparative database for examining all patients admitted to a hospital and therefore is an important source of data for quality measurement [33
]. However, administrative data sets often lack the richness of clinical data sources and they do not typically include physiologic measures of illness severity that can be used for adjustment when making comparisons across hospitals. While it is clear that physiologic measures of illness severity can provide a more nuanced adjustment for differences in case-mix related to illness severity, it is unclear how much these measures add or subtract to adjustments made using demographic and clinical variables as done commonly in other studies of variation in care [16
]. The variables used to adjust for differences in case-mix when measuring surfactant use and in-hospital mortality in this study included an available subset of the demographic variables used in other studies. This specific combination of variables may not have fully accounted for differences in case-mix across hospitals and for differences in severity of illness; therefore, some of the variation we observed may be attributed to residual differences in case mix that were unaccounted for by our adjustment model. Future efforts to use administrative data to measure neonatal quality will need to first focus on developing valid risk adjustment models. More refined neonatal risk adjustment models are currently under development for use with the PHIS database (personal communication Matt Hall, CHCA).
Many neonatal quality measures are complicated by the impact of patient transfers between hospitals, which is a common occurrence in neonatal and perinatal care [15
]. For example, development of valid measures of antenatal steroid use in eligible pregnant women is complicated by the fact that steroids may be given in multiple settings including outpatient clinics and referring hospitals. Measuring surfactant use in infants with RDS is similarly complicated. We do not have information on whether infants received surfactant at an outside hospital prior to transfer. In both our initial analysis and in sensitivity analyses, results were not affected by the day of life when the infant was admitted, making it less likely that receipt of surfactant at an outside hospital influenced whether the infant was truly eligible for surfactant. However, it is possible that variation in surfactant use across hospitals actually reflects differences the care provided before transfer, instead of differences in the quality of care provided by the accepting facility. Future efforts to develop neonatal process measures of quality will likely need to focus on aspects of care that clearly occur at a single location or on developing accurate ways to assess and attribute care provided across multiple locations.
The hospitals that submit data to PHIS are mainly academic children's hospitals. Academic hospitals may vary in their quality of care compared to non-academic hospitals. In addition, we excluded centers with <40 eligible infants in the study period. These smaller NICUs may be less likely to provide high quality care compared to larger NICUs [16
]. Therefore, additional efforts are needed to generalize these results and to understand how potential quality measures perform in non-academic or smaller NICUs.