|Home | About | Journals | Submit | Contact Us | Français|
We examined the effect of hospital type and medical coverage on the risk of 1-year mortality of very low birth weight (VLBW) infants while adjusting for possible selection bias.
The study population was limited to singleton live birth infants having birth weight between 500 and 1,500 grams with no congenital anomalies who were born in Arkansas hospitals between 2001 and 2007. Propensity score (PS) matching and PS covariate adjustment were used to mitigate selection bias. Additionally, a conventional multivariable logistic regression model was used for comparison purposes.
Generally, all three analytical approaches provided consistent results in terms of the estimated relative risk, absolute risk reduction, and the number needed to treat (NNT). Using the PS matching method, VLBW infants delivered at a hospital with a neonatal intensive care unit (NICU) were associated with a 35% relative decrease (95% bootstrap CI: 18.5% – 48.9%) in the risk of 1-year mortality as compared to those infants delivered at non-NICU hospitals. Furthermore, our results showed that on average, 16 VLBW infants (95% bootstrap CI: 11 – 32), would need to be delivered at a hospital with an NICU to prevent one additional death at one year. However, there was not a difference in the risk of 1-year mortality between VLBW infants born to Medicaid-insured versus non-Medicaid-insured women.
Estimated relative risk of infant mortality was significantly lower for births that occurred in hospitals with an NICU; therefore, greater efforts should be made to deliver VLBW neonates in an NICU hospital.
Each year there are approximately 4 million births in the United States.1,2,3 Very low birth weight (VLBW) infants (< 1,500 g) constitute < 1.5% of the total annual births.3 Premature birth and intrauterine growth restriction are considered the primary causes of VLBW. Despite tremendous improvement in the care of VLBW infants, the mortality and morbidity rates remain high.1,2,4 In 2006, the mortality rate for VLBW infants was 24% (240/1000 live births).5 In addition to the increased risk of mortality within the first year of life, VLBW infants are also at risk of multiple clinical complications such as hypothermia, intracranial hemorrhage, cerebral palsy, and other neurological problems.6 Therefore, the management of these high risk neonates requires special attention in all facets of their care.
A major concern when dealing with this high-risk population is ensuring that they are delivered in hospitals with proper levels and adequate volumes of neonatal care. Several studies have shown an association between lower mortality rates in hospitals with higher level of care and higher volumes of patients versus those with lower levels and lower volumes.7–11 Phibbs and colleagues, using birth certificates, hospital discharge, and death certificates for VLBW infants born in California hospitals from 1991 to 2000, showed that mortality among VLBW infants was lowest for those delivered at hospitals with a neonatal intensive care unit (NICU) that had both a high level of care and a high volume of such patients.7 Cifuentes and colleagues evaluated the mortality rate in low birth weight (LBW) infants according to four levels of NICU care: no NICU, intermediate NICU, community NICU, and regional NICU. They concluded that greater efforts should be made to deliver infants with expected birth weight of < 2,000 g at hospitals with a regional NICU given that the odds ratios associated with mortality for LBW infants were higher when infants were delivered at a hospital with no NICU (OR = 2.38) or intermediate NICU (OR = 1.92).9 The American Academy of Pediatrics and American College of Obstetrics and Gynecology have defined levels of neonatal care (levels 1 – 3), based on availability of services. A recent meta-analysis observed increased odds of death for VLBW neonates delivered outside level 3 centers (OR =1.62).12 While several other investigators have assessed the relationship between level of NICU care and infant outcome, the number of population-based studies on survival outcome among VLBW infants is limited.1,7 More importantly, given the nature of observational studies, few investigators have thoroughly considered potential selection bias. While it is common in observational research to use an overinclusive regression model with potential confounders, this approach tends to result in a complex multivariable model.
In addition to the level of hospital care and other well-known factors related to infant mortality such as mother’s race and infant’s gender, another potential factor in determining the survival outcome for VLBW infants is whether or not the mother is Medicaid-insured. Medicaid-insured women have low incomes and numerous other characteristics which put them at high risk for a preterm birth. However, a recent literature review of studies of birth outcomes for women on Medicaid concluded that insurance by Medicaid itself was not associated with the likelihood of a preterm or low birth weight birth, once other risk factors are taken into account.13 It is possible, however, that within the sub-group of infants born prematurely or at low birth weights, Medicaid insurance is a marker for factors that may increase infant mortality.
In this study, we aimed to evaluate the differences in the risk of 1-year mortality for VLBW infants born at hospitals with NICUs compared to those infants born at hospitals without an NICU. In addition, we assessed whether there was a difference in 1-year mortality between infants born to women covered by Medicaid and infants born to women not covered by Medicaid while adjusting for possible selection bias using propensity score adjustment methods and the conventional multivariable regression approach.
Generally, NICU hospitals correspond to levels of care 2B and 3 A–C as defined in Guidelines for Perinatal Care.14 However, Arkansas lacks a formal state policy for the designation of hospitals at one of three levels of perinatal care.15,16 Therefore, for simplicity and ease of ascertainment, the authors chose to identify Arkansas hospitals as NICU if they had a board certified neonatologist providing intensive care services for infants born in that hospital. Otherwise, the hospitals were identified as non-NICU hospitals, which correspond to levels 1 and 2A.14 Although most other states have designated levels of care, most states have not adopted all 5 levels into their designations,15 and the direction of care provided to neonates, unfortunately, is often not enforced.5
In the medical literature, neonatal death is defined as death within 28 days after birth. However, this definition of a “neonatal death” may be biased, because it does not account for deaths of infants who survive longer than 28 days but still do not survive their initial hospitalization after delivery.7 For the purpose of this paper, the term “infant mortality” refers to all deaths that occurred within the first year after delivery.
Using the Arkansas birth vital records database in conjunction with a set of Medicaid claims for pregnancy linked to birth certificates for women covered by Medicaid in Arkansas who delivered infants between April 1, 2001 and December 31, 2007, all live births were identified. The matched claims data were used to flag the vital records if the mother had Medicaid coverage for prenatal care and delivery services. The matching process is described elsewhere.17 Of the 281,460 birth records, 2,652 singleton live birth infants were identified as having a birth weight between 500 g and 1,500 g with no congenital anomalies. (This group will be referred to as VLBW infants hereafter.) Of these, 49% were flagged as having Medicaid coverage.
Given that this is a population-based study, propensity score (PS) analysis was used to adjust for potential confounding factors across the two hospital types. The PS is a model-based conditional probability of an infant’s membership in the reference group, given his or her observed covariates.18 The main purpose of PS adjustment is to balance observed covariates among individuals from different groups in order to mimic randomized clinical trials, and on average, remove the bias in the background covariates.18–20 The current literature describes three of the most common applications of propensity scores which include matching, stratification, and regression adjustment.18,21,22 This paper focuses on PS matching and PS regression adjustment.
The PS matching approach involved forming matched sets of infants born at NICU hospitals and those born at non-NICU hospitals who have comparable PS values. In particular, we used a SAS user macro, which uses a greedy matching algorithm based on 5→1 digit matching.23,24 Initially, the algorithm attempted to match NICU and non-NICU infants based on the first five PS digits. If no matches were found, then matching was attempted on the first four digits. This pattern was continued down to the first PS digit.23 On the other hand, the PS regression adjustment approach directly used the propensity scores as a covariate in the model to control for confounding.25,26 The PS covariate may either be continuous or categorical.20 For this study, the continuous PS was incorporated into the model.
The objective of the intended analysis was to estimate relative risks (RRs) and 95% confidence intervals (CIs) associated with the risk of 1-year mortality comparing two hospital types and insurance types, while controlling for selection bias, using logistic regression.
Using the PS-based methods, several preliminary steps were taken to complete our stated objective. These steps are described as follows:
Prior to any PS correction, we examined how the NICU and non-NICU hospitals differed initially given the variables believed to be related to infant mortality. For the continuous variables, we used a two-sample t-test with hospital type as the fixed factor. For the dichotomous variables, we used either a chi-square test or Fisher’s exact test, as appropriate. The results are given in Table 1.
There exists little guidance or consensus for building the PS model.20 We used a somewhat overinclusive PS model which included all covariates listed in Table 1 believed to be potentially related to mortality and hospital type according to our panel of experts. Thus, our PSs were estimated by a logistic regression model with hospital type as the dependent variable and all potential confounding variables in Table 1 as independent variables (even if p > 0.05). Based on the estimated PSs, a graphical representation of the initial balance check is given in Figure 1.
We evaluated the success of the PS methods to achieve balance in the distributions of all the observed covariates between the two hospital types using standardized difference plots.22,27 For continuous variables, the standardized difference was defined as
where NICU and non-NICU denote the sample means, and are the sample variances of the covariate in the NICU and non-NICU hospital settings, respectively. For dichotomous covariates, the standardized difference was defined as
where pNICU and pnon-NICU denote the prevalence of the dichotomous covariate for the respective hospital settings. The results of this balance check based on PS matching are displayed in Figure 2a. As an analog to the conventional p-value of greater than 0.05 used to denote statistical non-significance, an absolute standardized difference of less than 10% suggests a negligible imbalance between the two hospital settings for a given covariate.22,27
For the PS regression adjustment approach, an evaluation of the success of the propensity adjustment was made based on a weighted standardized difference of each covariate between the two hospital types.20,28 By weighted standardized difference, it is meant that the average of the standardized differences computed for each PS associated with each infant in our study population is obtained for a given covariate.28 The results of this balance check are depicted in Figure 2b.
After evaluating the success of the PS balance checks, we used a semi-parametric modeling approach to model mortality rate as a function of hospital type, insurance type, mother’s race, infant’s gender, birth weight, and gestational age. The model was implemented twice, first based on the results of PS matching and secondly based on adding the PS to the semi-parametric model as a covariate. The relationship between an infant’s risk of 1-year mortality with both birth weight and gestational age is expected to be nonlinear.1,8 Thus, the semi-parametric approach provided a flexible model in which trend for numerical variables (birth weight and gestational age) could be initially assessed using smoothing splines.29 Both models exposed a higher-order effect for birth weight; thus, for the final logistic regression models, both linear and quadratic terms for birth weight were included.
For comparative purposes, a multivariable logistic regression model was used to estimate the risk in 1-year mortality associated with hospital delivery type and insurance type while adjusting for the vector of confounding factors in Table 1. This model will be referred to as a “conventional model”. In observational research, conventional regression modeling is typically implemented as opposed to using a PS adjustment. Although many researchers have shown that using the vector of confounders directly in the model produces similar results as PS approaches, we point out some important advantages of using propensity-based methods in the Discussion section.25,30
While the focus of our paper is to estimate relative risk, using PS-based methods and conventional regression to adjust for potential selection bias, other authors have argued the need to report other measures such as absolute risk reduction and the number needed to treat (NNT – the reciprocal of the absolute risk reduction) when the outcome is dichotomous.31–33 Therefore, we report both the relative risk and absolute risk reduction for the two hospital types, insurance types, maternal races, and genders of infants along with the respective 95% bootstrap CIs.34 However, we calculated the NNT for just hospital type as it was the only variable to which NNT applied. In addition, hospital type-specific mortality curves were derived using a logistic regression model at a fixed insurance type (Medicaid), gender (female), mother’s race (Caucasian), and designated gestational ages of 22, 24, 26, 28, 30, and 32 weeks (Figure 3).
Table 1 shows mean and standard deviation for the continuous variables and percentages for categorical variables along with the respective p-values. Out of 28 variables before any type of PS adjustment, 17 variables differ between the two hospital types. In particular, mothers who delivered at an NICU hospital were slightly older, married, and received more education. However, there were fewer Caucasian, Medicaid-insured women delivering at an NICU compared to a non-NICU hospital. With regard to risk variables, these mothers were less likely to be smokers and had a lower risk of having an abruption, placenta previa, seizures during labor, and uterine bleeding. However, the NICU mothers were more likely to have pregnancy associated hypertension and labor induction. These women tended to receive adequate prenatal care, report prenatal care, and have a higher rate in parity. Additionally, their offspring were more likely to be female and have lower birth weights. Given these results, there is evidence that hospital type was confounded with factors that are associated with infant mortality.
Figure 1 depicts the distributions of the predicted propensity scores for NICU and non-NICU deliveries. While there is some overlap between the distributions, it is clear that for propensity scores higher than 0.5 the number of VLBW infants delivered at NICU hospitals is far greater than that for non-NICU hospitals. Additionally, in Figure 2a, the standardized difference plot provides further evidence of an imbalance in measures of this list of covariates between the two hospital types. Based on a rule of thumb suggesting that absolute standardized differences of at least 10% indicate imbalance,22,27,35 14 variables were deemed to be imbalanced between the two hospital types.
While there exists imbalance for several covariates prior to any PS adjustment, the result of using either a PS greedy 5→1 digit matching algorithm (producing 647 matched pairs given our sample) or regression adjustment displays good balance (see Figures 2a and 2b). In both figures, one can observe that both the standardized differences under the matching and the weighted standardized differences using the PS regression adjustment are all within the 10% threshold. Thus, further analysis using either the matched data set or the PS as an additional continuous covariate was possible.
Table 2 provides results from the logistic regression models accounting for selection bias using the three analytical methods discussed: PS matching, PS regression adjustment, and conventional regression. In particular, the estimated adjusted relative risks and absolute risk reduction are presented for each of the three methods along with the corresponding 95% bootstrap CIs based on 2,000 bootstrap samples. For PS matching, the model included the following predictors: hospital type, insurance type, maternal race, infant’s gender, birth weight, birth weight squared, and gestational age. This set of predictors will be referred to as the “base set.” For the PS regression adjustment, the model included the base set along with the continuous PS as an additional covariate. For the conventional regression approach, the model included the base set along with the variables found in Table 1 that were not already in the base set.
Overall, the three analytical approaches yielded consistent results. For example, using the PS matching, the marginal probability of 1-year mortality in our study population was 0.184 if all VLBW infants were delivered at hospitals with no NICU, and 0.120 if they were all delivered at hospitals with NICU. Thus, in terms of relative risk, infants delivered at an NICU hospital were associated with a 35% relative decrease (bootstrap 95% CI: 18.5% – 48.9%) in the risk of 1-year mortality. The absolute reduction in 1-year mortality was 6.4%, with a bootstrap 95% CI of (3.2% – 9.7%); therefore, the NNT is 15.6 (bootstrap 95% CI: 10.3 – 31.6). Given our population, this suggests that on average 16 VLBW infants (95% CI: 11 – 32) would need to be delivered at a hospital with NICU to prevent one additional death at one year. For the other primary predictor of interest (insurance type), there was not a difference in the risk of 1-year mortality between infants born to Medicaid-insured versus non-Medicaid-insured women (RR=1.027, (bootstrap 95% CI: 0.819 – 1.296)). Furthermore, the absolute risk reduction was negligible at 0.4% (bootstrap 95% CI: 0.06% – 3.9%).
Using a logistic regression model to explore the effect of infant’s birth weight on the risk of 1-year mortality, gestational age-specific predicted mortality curves were estimated at designated gestational ages of 22, 24, 26, 28, 30, and 32 weeks, a fixed insurance type (Medicaid), gender (female), and mother’s race (Caucasian) (Figure 3). Due to the similarity in the results from the three analytical methods, we simply provide the mortality curves based on the PS matching analysis. As one might expect, the trend in the data indicates that the predicted mortality improves for both hospital types as birth weight and gestational age increase. Moreover, infants delivered at hospitals with NICUs consistently had lower risk of 1-year mortality across all combinations of gestational age and birth weight.
We investigated the impact of delivery hospital type on the risk of 1-year mortality for VLBW infants. Based on the adjusted relative risk of 1-year mortality, there is a statistically significant survival advantage for VLBW infants delivered at hospitals with an NICU compared to those born at a non-NICU. Moreover, this is also true for infants with birthweights between 500 and 999 grams and gestational age ranging from 22 to 27 weeks, supporting the recommendations of the American Academy of Pediatrics to deliver those babies in a level 3B nursery.14
In addition, we investigated if there was a discrepancy in the risk of 1-year mortality between neonates born to mothers with Medicaid coverage and those born to mothers without Medicaid coverage. Based on our study sample, there was not a statistical difference between the two types of medical coverage. In Arkansas, Medicaid coverage is liberal compared to many states and is provided at 200% above the federal poverty level. We speculate that the lack of any appreciable difference in outcome of VLBW neonates in the Medicaid population is due to the liberal eligibility of more pregnant women and the progressive attitude of Arkansas Medicaid in encouraging appropriate prenatal care.
For premature infants, there are several studies suggesting a better survival outcome for female infants.1,4,36–38 While many of the studies report the results in terms of odds ratios, the results from our study are consistent with these findings. For example, under PS matching, the estimated adjusted relative risk was 0.637 (95% CI: 0.497 – 0.813), thus suggesting there is a 36.3% decrease in the risk of 1-year mortality for female infants compared to males. On the other hand, there has not been consistent evidence showing a significant association between the risk of 1-year mortality and race. Some researchers have noted a distinct advantage among premature African-American infants.1,36,39 However, other researchers have demonstrated no significant differences among races.40,41 In our study, infants born to Caucasian mothers were not statistically different from infants born to non-Caucasian mothers in terms of 1-year mortality.
Potential deficiencies of observational studies are selection bias and confounding. If these shortcomings are not addressed, standard statistical methods applied to such studies may yield biased estimates of the true effects.20 While a common approach for handling these drawbacks is the use of conventional multivariable regression models, many authors recommend using a PS analytical method as an alternative. In particular, they note several advantages, which include 1) a more parsimonious model, 2) more robust parameter estimates in the absence of a correctly specified model, and 3) the ability to identify situations where potential confounders do not adequately overlap between treatment groups.20,21,25,42 While in our study the analyses using PS adjustment and conventional logistic regression provide similar results and either approach may be appropriate, nonetheless, propensity scoring is considered a more robust approach.20
Our study has some limitations. It has been shown that both level of care and volume of NICU infants are important predictors of neonatal outcome.7,11 We explored the effect of volume on 1-year infant mortality by comparing low volume NICUs to a high volume (≥100 VLBW deliveries per year) NICU. In Arkansas, six hospitals met our study criteria for NICU designation. Of which, only one hospital could be classified as high volume. This analysis indicated no statistical significance with respect to volume. The limited number of NICUs may have contributed to the inconsistent result of not showing an infant outcome advantage for the tertiary center. Furthermore, we did not assess morbidity in this population. While infant mortality may be objective, it may not be the most sensitive way to assess quality of care in this population. Also, given that infant mortality was defined as death within the first year after delivery, this could include some post-neonatal deaths not related to VLBW status, such as by accidents. Finally, gestational age in our data set could be inaccurate since early ultrasound dating was not assured and since there are discrepancies between menstrual dating and ultrasound in 18% of pregnancies.43
On the other hand, our study has several strengths. First, we assessed NICU care in a rural state with relatively few local hospitals with neonatology staffing. Second, we used PS methodology, which is considered a more robust approach to handling selection bias. Finally, we assessed an entire state rather than use a database that may have inherent bias for (or against) quality assessment.
Based on our usage of three analytical approaches for handling potential selection bias, Medicaid coverage was not associated with increased infant mortality in Arkansas, which has liberal reimbursement policies. However, the estimated risk of 1-year infant mortality was significantly lower for VLBW births that occurred in hospitals with an NICU. Thus, in summary, every effort should be made to deliver VLBW neonates in an NICU hospital.
Joint Acknowledgment/Disclosure Statement: The linked Medicaid claims-birth certificate data were created as a component of the evaluation of Antenatal Guidelines, Education and Learning System (ANGELS) initiative. The ANGELS initiative is supported by the Arkansas Division of Medical Services with the collaboration of the Arkansas Department of Health. In addition, the project described was partially supported, in part, by Award Number 1UL1RR029884 from the National Center for Research Resources. Finally, the authors are grateful to the referees and the managing editor for critically reading the manuscript and for their valuable suggestions and comments which have led to an improved presentation.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.