We present the results from the conditional logistic regression in . The top set of results is based on the specification that uses dummy variables to identify the effect. The bottom set of results is based on the specification that included the estimated hospital specific DRG out-of-pocket price. We include both a primary and a fixed effect specification for the medical and surgical samples. The primary specification controls for hospital type (public, teaching, for-profit, size, trauma); hospital type interacted with drive time; and the hospital–patient service matches. The fixed effect specification includes the patient–hospital service match variables but not the hospital type variables.
Conditional Logistic Regression of the Probability of Choosing a Hospital from Set of Available Hospitals
The conditional logit results indicate that the HSI had an effect for patients with a medical diagnosis who are affiliated with the engineers’ union. The HSI eligibility × union × engineers’ union × postinteraction illustrates this effect. Unconditional probabilities computed from the coefficients suggest that this group used an HSI eligible hospital in the postperiod 2.92 times more often than in the preperiod, whereas the SNU control group was only 0.64 times as likely. The increase in the likelihood that members of the engineers’ union pick an HSI eligible hospital in the postperiod relative to the preperiod was about 4.6 times greater than the equivalent relative likelihood for the nonunion controls. In contrast, the hospital choices of patients affiliated with the machinists’ union (i.e., the use of HSI eligible hospitals in the postperiod compared with the preperiod) were not significantly different than the SNU control group. This result is robust to the fixed effects specification. Patients in both unions were unresponsive to the incentive if they had a surgical diagnosis. One should use caution when interpreting the coefficient on the HSI eligibility variable because it must be interpreted in the context of its interactions.
Perhaps a more intuitive way to consider the impact of the HSI is to examine the pre–post difference (i.e., “the difference-in-difference”) in the predicted probabilities that the three groups of patients (engineers’ union, machinists’ union, and SNU) would select one of the hospitals eligible for the HSI. Unlike the odds presented above, these predicted probabilities are computed from the coefficient estimates and are based on the average patient characteristics across the sample in the pre- and postperiods. The predicted probabilities indicate a large and significant change in the probability that patients affiliated with the engineers’ union would select an HSI hospital, with the probability moving from 0.12 in the preperiod to 0.26 in the postperiod, a difference of 0.14. The comparable probabilities for the machinists’ and SNU beneficiaries were 0.16–0.10 and 0.19–0.14, respectively. This represents a “difference-in-difference” of about 0.20 for the engineers’ union patients relative to the SNU patients, a meaningful effect size due to the HSI. In contrast, the difference between the machinists’ and nonunion patients was negligible. Furthermore there was not a significant difference among any of the groups for surgical diagnoses.
Results from the price regressions in tell a similar story as the dummy variable specification. In the medical sample, members of the engineer's union were significantly less likely to be admitted to a hospital that had a positive price. This result was consistent across the primary and fixed effects specifications, though the union coefficient loses significance in the fixed effects specification. This is possibly because the out-of-pocket price is endogenous if attractive hospitals can negotiate higher prices. Hospital attributes like reputation and amenities are likely to be fixed in the short-run and thus controlled for in the hospital fixed effects, leading to a smaller and insignificant coefficient. Note that price varies within hospital across DRGs, and identification requires that reputation and amenities do not vary across DRGs, which is unlikely in practice.
We examined the robustness of our results to a number of variations, including the analysis sample, the model specification used in the conditional logit, and an alternative estimation approach to the conditional logit model. For the sample, we examined the sensitivity of the results to our decision to drop patients admitted (a) via the Emergency Department, (b) for nongeneral hospital services, and (c) in the first 2 months (July/August) of the postperiod. The results were highly robust to these sample exclusions.
Within each sample, we estimated various specifications of the model, with each differing from the specification reported in by dropping or adding sets of covariates. The most parsimonious specification included only the HSI eligibility variable and union variables and their interactions. The most complete specification included interactions between patient-level variables (e.g., age, gender) and drive time. Finally, we examined whether the conditional logit results were sensitive to the choice set definition. The results using a more restrictive definition (e.g., only hospitals within 100 minutes of patients’ residences) are comparable. However, if we further narrow the choice set, the number of admissions in the sample becomes smaller and an overly narrow definition, say only hospitals within 60 minutes, leads to insignificant but qualitatively similar results.
In summary, our main finding, that the HSI influenced hospital selection for medical patients affiliated with the engineering union (but not the machinists’ union) relative to SNU patients, was generally robust to differences in sample and model specification. The lack of a finding for surgical patients was highly robust to changes in sample and specification.
We also examined an alternative estimation approach to the conditional logit, estimating instead a simple logistic regression where the dependent variable in both the pre- and postperiods took on a value of one if the hospitalized patient chose a hospital that qualified for the HSI in the postperiod, and zero if the patient chose one of the hospitals that did not qualify for the HSI in the postperiod. In essence the simple logistic regression model is a different specification of the conditional logit model that uses less information, and also requires more assumptions in the specification of the model covariates. For example, to include distance as a covariate, we had to compute a differential distance variable, defined as the difference in distance from patients’ residences to the closest HSI and non-HSI hospitals. The results from this simple logit were not robust in terms of statistical significance, though the direction of the coefficients was similar. As mentioned above, we believe the conditional logit model is superior because it uses the full set of information including the relative differences in attributes between every hospital in the choice set.
Conclusions and Limitations
The results from our analyses suggest the HSI influenced the particular hospital chosen for those patients affiliated with the engineers’ union and admitted for a medical diagnosis, with the “difference-in-difference” estimated to be 0.20 for the engineers’ union relative to the SNU group. Unlike those patients affiliated with the engineers’ union, those affiliated with the machinists’ union were no more likely than SNU beneficiaries to select an HSI compliant hospital in the postperiod. Members of the engineers’ union are more highly educated than members of the machinists’ union and may have been better able to learn about and take advantage of the incentive during the relatively short follow-up period.
There are a number of limitations, however. First, patients affiliated with the machinists’ union may have learned about the HSI eventually (e.g., the firm could have provided more education and outreach to the machinist population) and reacted similarly to patients affiliated with the engineers’ union; the relatively short follow-up period would not have picked up such a lag. Second, although we had a sizeable number of hospitalized patients, we may have had limited power to detect an effect in the surgical sample. We did, however, estimate a large and statistically significant effect for the medical diagnosis sample, which includes a smaller number of patients.
Third, only about 18 percent of hospitals in the market were eligible for the HSI for most admissions. Given the likely importance of the admitting physician in the hospital selection decision, and the fact that most physicians have limited hospital staff privileges, it seems likely that if more hospitals had met the safety criteria, the incentive may have had a larger effect. On the contrary, if most hospitals qualified, the choice set would differ little from the choices available before the HSI, and the HSI would not likely affect the distribution of patients across hospitals.
Fourth, while our results suggest that the HSI influenced some patients’ decisions, the design of HSI does not allow us to comment on the size of the incentive needed to change behavior, because there was no variation in the coinsurance rate in the treatment group.
Fifth, we did not estimate the financial return of the HSI to the plan sponsor (i.e., the firm), though when deciding to implement the tiered hospital network, the firm believed, as the Leapfrog Group does, that safer hospitals are ultimately less costly due to shorter lengths of stay and fewer complications. If true, this would suggest that coinsurance reductions could yield value for the plan sponsor if patients choose safer hospitals as a result. Interestingly, after we collected data, the engineers’ union contract dropped the HSI (an outcome of 2006 labor negotiations between the firm and the engineers’ union), eliminating the tiered hospital benefit design from the very group in which we estimated its significant impact. This loss underscores the challenge in assessing the effectiveness of benefit innovations that typically require a learning period on the part of employees, when these innovations are “on the table” during regular wage and benefit negotiations.
Sixth, we specifically focused on a tiered hospital network designed to provide consumers with an incentive to use safer hospitals. While there are alternative ways to improve hospital safety, including direct supply side incentives (e.g., P4P), accreditation, or regulation, our study did not consider the relative cost-benefit of these diverse approaches.
In summary, our results suggest that the HSI resulted in a shift among hospitals of patients with medical admissions in a specific union group and employer. Our findings could reflect, to a considerable degree, the unique characteristics of this employer and the community in which the patients were located, and the particular benefit design offered by the employer. It seems likely that innovations in benefit designs, payment strategies, and cost-sharing arrangements will continue to grow in number, as employers and their health plan agents seek greater value for their health care dollars. Many of these innovations will be nonstandardized and locally developed, providing additional opportunities for research. By building on and extending the analysis reported in this paper, researchers can create a stronger foundation for future decisions of public and private payers regarding the value of these approaches. In particular, it will be important for researchers to address the net financial benefits of different health benefit innovations.