|Home | About | Journals | Submit | Contact Us | Français|
To examine whether low-income parents of children enrolled in the New York State Children's Health Insurance Program (SCHIP) choose managed care plans with better quality of care.
2001 New York SCHIP evaluation data; 2001 New York State Managed Care Plan Performance Report; 2000 New York State Managed Care Enrollment Report.
Each market was defined as a county. A final sample of 2,325 new enrollees was analyzed after excluding those in markets with only one SCHIP plan. Plan quality was measured using seven Consumer Assessment of Health Plans Survey (CAHPS) and three Health Plan Employer Data and Information Set (HEDIS) scores. A conditional logit model was applied with plan and individual/family characteristics as covariates.
There were 30 plans in the 45 defined markets. The choice probability increased 2.5 percentage points for each unit increase in the average CAHPS score, and the association was significantly larger in children with special health care needs. However, HEDIS did not show any statistically significant association with plan choice.
Low-income parents do choose managed care plans with higher CAHPS scores for their newly enrolled children, suggesting that overall quality could improve over time because of the dynamics of enrollment.
During the last two decades, enrollment in managed care plans has increased from 27 percent of the employees with employer-based insurance coverage in 1988 to 97 percent in 2007 (Kaiser Family Foundation 2007). The rise in managed care has heightened the importance of measuring and improving quality of managed care plans. Quality measurement tools have been developed since the 1990s, such as the Consumer Assessment of Health Plans Survey (CAHPS) and the Health Plan Employer Data and Information Set (HEDIS). In addition, the federal and state governments have invested in managed care quality improvement, including developing measurement tools. In New York State, all managed care plans have been required to report quality data to the Department of Health since 1994 (NYSDOH 2002). Plans with quality measures lower than the state average have to submit a report including identification of the reasons and actions to be taken (NYSDOH 2000). A quality incentive program has been implemented in Medicaid managed care plans since 2001, where plans can get a premium bonus up to 3 percent depending on whether their quality measures exceed the 75th percentile of the benchmarks (Zuckerman 2007).
In addition to government regulation and other quality improvement methods, market competition based on consumer choice could be one approach to improving managed care quality, which hinges on the consumer's ability to choose managed care plans with higher quality. Quite a few studies have been conducted on the effect of quality on health plan choice. Early studies demonstrated that Medicare beneficiaries with higher incomes and employer-sponsored insurance coverage were more likely to purchase high-quality supplemental insurance using proxies of quality (Rice, McCall, and Boismier 1991), and that noisy quality measures of quality were better than nothing (Harris 1996). More recent studies can be classified into three groups: laboratory experiments; empirical studies on the effect of quality reports; and empirical studies on the effect of plan quality, regardless of the sources of quality information. In experiments, the information on plan quality and other plan features of hypothetical health plans is the only source of information, and they directly test whether quality would affect plan choice. Laboratory experiments consistently showed that better quality led to a higher demand after controlling for other factors (Spranca et al. 2000; Schoenbaum et al. 2001; Harris 2002; Uhrig and Short 2002).
The first study systematically investigating the effect of quality reports (HEDIS 2.0) on plan choice was done by Chernew and Scanlon (1998), where they analyzed the choices of 5,795 active, nonunion employees from a Fortune 100 company. Though some quality measures were positively correlated with the probability of choice, there existed a weak or counterintuitive relationship between quality measures and choice for other measures such as surgical care, physician quality, and preventive care. A similar study by the same authors also demonstrated that employees did not respond strongly to health plan quality ratings (Scanlon and Chernew 1999). The CAHPS demonstration in Washington showed that those state employees who used the CAHPS report were more likely to switch health plans (Guadagnoli et al. 2000). Two randomized-controlled trials conducted among new Medicaid beneficiaries in New Jersey and Iowa did not yield significant overall effect of CAHPS reports (Farley et al. 2002a, b). Several studies made use of natural experiments occurring at Harvard University, General Motors, and in the federal government system and illustrated that quality information did affect the consumer's plan choice (Beaulieu 2002; Scanlon et al. 2002; Wedig and Tai-Seale 2002). Jin and Sorensen (2006) measured the effect of publicized health plan rating information of the National Committee for Quality Assurance (NCQA) after controlling for the effect of nonpublic health plan ratings, and they concluded that publicized information had an effect on plan choice, especially among individuals who were choosing a plan for the first time. Health plan switching behavior, however, was not associated with quality information, according to a study by Abraham et al. (2006). One empirical study by Chernew et al. (2004) directly investigated the effect of plan quality (as measured by CAHPS and HEDIS) on employer's plan choice, finding that large employers were more likely to select health plans with higher quality measures.
More than four million children across the United States are enrolled in the State Children's Health Insurance Program (SCHIP), mostly in managed care plans (Shone and Szilagyi 2005). The New York SCHIP program1 relies on managed care plans to provide health services for enrollees, and plans receive monthly prepaid premiums from families or the state government or both. In 2005, 28 managed care plans provided SCHIP; this accounted for 74 percent of the managed care organizations operating in the state (NYSDOH 2006). The state government closely monitors managed care quality of care and releases annual plan quality reports.
Few studies have examined the relationship between health plan quality and enrollment in SCHIP plans. This is important because of the implications it has for market-driven improvements in health care quality, either by simply directing more enrollment to high-quality plans or by providing a market advantage to high-quality firms. Of note, price does not play a role in SCHIP plan choice because the premium is determined by family income and is the same across SCHIP plans, suggesting that differences in quality would be central to enrollment choice. Though it is critical to understand the causal effect of quality on plan choice, a positive association of quality with plan choice among new enrollees implies that overall quality could improve over time simply because of the dynamics of enrollment. Despite the policy debates and research focusing on quality of care in the last two decades, it remains unclear whether quality improvement can be better achieved by regulatory approaches or through market competition, and our study could inform policymakers and the SCHIP administration.
We based our analysis on the assumption that consumers are rational agents that maximize utilities reflecting preferences across alternatives varying in benefits and costs. There are several plan attributes that could affect consumer's demand, including price, coverage, quality, geographic location, and the information about these attributes, while individual or family characteristics modify the association between plan attributes and choice (e.g., Scanlon et al. 2002; Scanlon, Chernew, and Lave 1997; Tay 2003). In the New York SCHIP market, both price and coverage are the same for each individual across available SCHIP plans, because monthly premium is determined by family income and the service package is standardized. Consequently, SCHIP plans are competing on geographic location and quality of care. Similar to the location differentiation in hospital competition (Tay 2003), the geographic location of the provider network is important since it involves traveling cost. However, competition on location in the health insurance industry is less obvious than that in the hospital industry because each health plan has many providers scattered across its service region. As to quality of care, the key question is whether consumers have sufficient information to support their decision. Given that this is a low-income population, these families are not expected to read the annual managed care plan quality reports, but they can get information from commercial advertisements, friends, relatives, coworkers, employers, primary care physicians, pediatricians, or outreach workers (Guadagnoli et al. 2000).
There are several possible pathways for eligible children to enroll in the program: directly through SCHIP plans or community-based facilitated enrollers; local departments of social services can also direct those non-Medicaid eligible children to facilitated enrollers or SCHIP plans (NYSDOH 2005). One question is what role facilitated enrollers and plans play in the plan choice process. As to managed care plans, when parents walk into the door of a plan, they have already made their choices. Though it is possible that enrollment choices are made by facilitated enrollers, it does not affect our conclusions as long as the choices reflect parents' preferences.
Mechanisms by which health plans are reimbursed could affect the market of quality, and thereby, the relationship we observed between quality and consumer plan choice. For example, plans may emphasize some aspects of quality that purchasers use for contracting. However, the New York SCHIP plans are reimbursed based on their cost experiences, with no specific quality metrics incorporated into the contracting.2 SCHIP contracting has no pay-for-performance or risk-adjustment mechanisms. In addition, research has shown that plans might provide some high-quality services to attract low-cost enrollees (e.g., Glazer and McGuire 2002; Cao and McGuire 2003). This is possible in the SCHIP market, but due to the fact that payment is based on cost experience, financial incentives for risk selection could be reduced greatly if not entirely eliminated. Therefore, because of the limited incentives, supply-side factors likely play a very small role in the observed relationship between quality and plan choice in SCHIP markets.
Under the rationality assumption, other things being equal, consumers would choose plans with higher quality of care (Spranca et al. 2000; Schoenbaum et al. 2001; Harris 2002; Uhrig and Short 2002). In reality, however, questions remain on how parents obtain quality information or whether low-income parents are able to process this type of information. Since the ability to process information is important in evaluating plan quality and is affected by the family's experience with health plan services, families with more experience with health plan service would value plan quality differently. Individuals with chronic diseases, prior insurance experience, and higher family incomes would have more opportunities to experience health plan services. Furthermore, parents with more education can process the information better. Therefore, we hypothesized the following: plans with better quality of care were more likely to be chosen; parents with better education, higher family income, and parents of children with prior insurance before SCHIP or special health care needs were more likely to choose plans with better quality of care.
We used the New York SCHIP evaluation data for this study from a stratified random sample of 2,644 new enrollees who enrolled at the end of 2001 or in early 2002 (see details in Szilagyi et al. 2004). The parents/guardians were interviewed via telephone twice during a 12-month period. As noted previously, because of the nature of existing contracting models, the supply-side payment mechanisms play a very small role in plan choice. As a result, the analysis was based primarily on the demand-side factors laid out in the conceptual model. In addition, only one child in each family was sampled and the analysis was conducted at the individual level.
Quality measures came from the 2001 New York State Managed Care Plan Performance Report, including seven CAHPS measures and three HEDIS measures. CAHPS measures included five composites and two overall ratings: “provider communication,”“getting care quickly,”“problems with getting care needed,”“problems with services,”“called or wrote health plan with complaints,”“overall rating of personal doctor or nurse,” and “overall rating of health plan” (NYSDOH 2002). All measures were in percentage. Some correlation coefficients among the measures were high, for instance, between “provider communication” and “getting care quickly” (0.93), while others were small such as between “getting care quickly” and “overall rating of health plan” (<0.01). In order to avoid the collinearity problem in the regression, we aggregated these measures according to the NCQA's standards and guidelines for MCO accreditations, with equal weights for all the measures except the overall rating of health plan, whose weight was twice of those of other measures (NCQA 2004).
Three HEDIS measures were used: “preventive care visits,”“use of appropriate medications for children with asthma,” and “childhood immunization,” since these are pediatric care specific measures and with fewer missing values. These HEDIS measures were not highly correlated, with correlation coefficients ranging from <0.01 to 0.14. The final HEDIS measure was the weighted average of three measures (NCQA 2004). When CAHPS or HEDIS measures were not available for SCHIP, those of Medicaid or commercial lines were used, since the plan quality measures were moderately correlated among different product lines within a health plan or between calendar years.3
The 2000 New York State Managed Care Annual Enrollment Report, which contains annual enrollment of all product lines, was used to define competitors. Each market was defined as a county in the state because it is mandated that each eligible child must join a SCHIP plan in the county where he/she resides.
Since some plans had a very small number of SCHIP enrollees, for instance, <50 enrollees, they might not have participated in the market competition in SCHIP. Therefore, two approaches were used to address this issue. First, only plans with at least 1 percent of the SCHIP market share were considered as competitors. Second, if a plan had <1 percent of the SCHIP market share but was flagged as a SCHIP plan or had a SCHIP business line in a county according to the performance report, then it was counted as a competitor in that market. These two methods assure that both actual and potential competitors are in the defined markets.
Xim is each of m individual (or family) characteristics, CAHPSh is the summarized CAHPS score of plan h, HEDISh is the summarized HEDIS score of plan h, and Zh is a vector of plan h's characteristics.
The dependent variable was the outcome of choice for each child-plan combination in a market, taking value 1 (chosen) or 0 (not chosen). The independent variables included individual/family characteristics, plan quality measures, other plan characteristics, and interaction terms between individual and plan variables.
The individual/family characteristics included child race, prior insurance status, family income, presence of a special health care need (using the screener by National Children with Special Needs Survey, CDC 2004), parent highest education, and presence of usual source of care before joining SCHIP.4 All individual/family characteristics were entered as interaction terms either with health plan quality measures or other health plan characteristics. Plan quality measures included the weighted averages of seven CAHPS scores and three HEDIS scores. Since the correlations between the interaction terms of CAHPS and the interaction terms of HEDIS were extremely high, with most of them being over 0.9, dummy variables of CAHPS and HEDIS were used for the interaction terms, using the 75th percentiles as the cut off points.5 Plan characteristics consisted of profit status, the total number of enrollees (in 1,000s), having a commercial product line, and outreach/marketing activities measured by the number of full-time equivalents (FTE) of outreach/marketing staff for SCHIP or Medicaid eligible children in 2001.6 The number of SCHIP and Medicaid eligible children was approximated by the population under 19 years old with family incomes below the federal poverty level (FPL) (FedStats 2005). Interaction terms between outreach/marketing activities and child race, parent education, and presence of special health care needs were also included in the model.7
Geographic distance measures were not available, but this is unlikely to create any omitted variable bias in the coefficients of interest. Unlike the hospital market, it is difficult to measure the geographic distance between an enrollee's residence and a plan's provider network because there are many providers within a plan, and enrollees can be referred to different providers. Additionally, plan quality is measured at the plan level, while distance between individuals and providers is quantified at the individual level.
A Hausman test was used to test the Independence from Irrelevant Alternatives (IIA) Assumption. We also applied a mixed logit model that is not subject to the IIA assumption (Train 2003) as a sensitivity analysis.8 In order to avoid the incorrect inferences for interaction effects, marginal effects were calculated as derivatives for continuous variables or differences for discrete variables. For continuous variables, marginal effects were evaluated at the current values. Standard errors were calculated using bootstrap with 200 replications. Sampling weights were used in the estimation so that the results can be generalized to the population.
Among the 2,644 new enrollees, 317 individuals were in markets with only one SCHIP plan and two individuals did not have correct home addresses. The final sample consisted of 2,325 individuals. Nearly 40 percent were below 6 years of age, and half were male (Table 1). About one fifth of the sample was non-Hispanic white, one third non-Hispanic black, and about 50 percent Hispanic. Parents or guardians receiving some college or higher education accounted for one third of the sample. More than 80 percent of the families were below 160 percent the FPL, while only about 4 percent of the families had income above 250 percent FPL and were required to pay the full monthly premium. One of every six children had at least one special health care need, and only about one third of the children had any type of health insurance during the 12-month period before enrollment.
Seventeen one-plan markets were excluded, and there were 45 markets in the analysis. The average number of SCHIP plans in a market was about five, varying greatly across markets (Table 2). The average number of SCHIP enrollees across markets was 11,000, with the smallest market being 100 enrollees and the largest being about 110,000 enrollees.
There were 30 SCHIP plans in the final sample. The average CAHPS score was 74.1, while the average HEDIS score was lower (67.6) but with a larger standard deviation of 6.9 (Table 3). The correlation coefficient between the average CAHPS and HEDIS scores was 0.49. The average number of SCHIP enrollees was about 17,000, varying from 100 to about 87,000 enrollees. Outreach activity ranged from one FTE on SCHIP or Medicaid business to 300 FTEs. Seventy-three percent were nonprofit plans and 40 percent had a commercial product line.
The marginal effects based on the conditional logit model are shown in Table 4. For one unit increase in the average HEDIS score, a plan's choice probability increased 0.05 percentage point, but it was not statistically significant at the 10 percent level.9 The choice probability increased 2.5 percentage points for every unit increase in the average CAHPS score. The estimate of CAHPS on plan choice was significantly larger (0.35 percentage point) among children with special health care needs, but no other significant interaction effects were observed between CAHPS/HEDIS and individual/family characteristics as hypothesized. Parents of children with a usual source of care were less likely to choose a plan with higher CAHPS scores (−0.24 percentage point).
Nonprofit plans had a much higher probability of being chosen (11.9 percentage points), and the association existed primarily among Hispanic children. Surprisingly, larger plans as measured by the total number of enrollees were less likely to be chosen (−0.24 percentage point). Although no significant overall association of outreach activity with plan choice was detected, the association was tremendous among children with better-educated parents, Hispanic children, and black children, ranging 8.9–32.9 percentage points.
The conditional logit model failed the Hausman test on profit status (p<0.001). However, the estimates from a mixed logit model were very similar to those of the conditional logit model, including the magnitudes of coefficient and the significances (data not shown).
This study found that for the dimensions of quality measured by CAHPS, health plans with better quality were more likely to be chosen. The estimate of the CAHPS score was also practically important. If a plan improves its average CAHPS score by a standard deviation (3.57 percentage points), the associated increase in the choice probability would be 9.03 percentage points. However, for the dimensions of quality measured by the HEDIS scores, no relationship between plan quality and plan choice was noted. This is not surprising because consumers are more likely to be influenced by the quality aspects measured by CAHPS than those measured by HEDIS, which are about clinical or “technical” processes. This low-income population could get quality information measured by CAHPS more easily from their relatives, friends, and coworkers, etc. From the perspective of quality improvement, plans have less influence on the interpersonal quality dimensions as measured by CAHPS compared with those of HEDIS, because CAHPS measures are mostly based on provider-level interactions. Nonetheless, plans could improve CAHPS scores through educating providers regarding culture sensitivity, increasing reimbursement on counseling activities, or sending patients' feedback to providers.
Contrary to two previous randomized-controlled trials assessing the effect of CAHPS report cards on choice among Medicaid beneficiaries (Farley et al. 2002a, b), we found a positive association between health plan quality as measured by CAHPS and plan choice. One possible reason is that those two trials evaluated the effect of additional quality information, while this study examined whether consumer plan choice was associated with quality, regardless of the sources of quality information. Overall, Farley et al. (2002a, b) did not find any significant effect of CAHPS report cards, but among some subpopulations, there was an effect of report cards on plan choice if these Medicaid beneficiaries did read the report cards and understand them. Furthermore, even in the absence of CAHPS reports in the control group, the authors (Farley et al. 2002b) showed that the switching rate from the lower-rated plans was higher than that from the higher-rated plans, suggesting that consumers can perceive some quality dimensions as measured by CAHPS in the absence of report cards.
Individuals with special health care needs do value CAHPS more than those without special health care needs. It is possible that they can perceive quality information better or they put more efforts in collecting quality information. Risk selection behavior could potentially affect this conclusion. In general, this is not a problem due to the pressure of competition because a plan should practice at least an average level of risk selection in order to survive. Further, the SCHIP pricing policy is based on cost experience, which reduces financial incentives for risk selection.
There are several limitations in this study. First, the conditional logit model is subject to the IIA assumption. Because calculating marginal effects from the mixed logit model is computationally infeasible, we used the conditional logit model to interpret the results. Using the conditional logit model would lead to similar marginal effects when the distributions of independent variables of the individuals whose coefficients are close to the means are similar to those of other individuals in the sample or when the distributions of individual characteristics are independent of those of coefficients. In other words, marginal effects from the conditional logit model could be biased if the above conditions are not satisfied.
Second, it is possible that quality and plan choice are endogenous. Plans might selectively enroll or disenroll certain enrollees, which in turn could affect plan quality, because quality measures came from members who stayed in a plan for at least 12 months and they were not risk adjusted. Prior studies have suggested that individual characteristics are associated with plan quality measures (e.g., Zaslavsky et al. 2001; Carlson et al. 2002). Several factors, however, might mitigate this issue: (1) SCHIP enrollees only accounted for a small portion of total plan enrollees (on average 7.71 percent) based on the 2000 New York State Managed Care Enrollment Report; (2) compared with the commercial markets, SCHIP has a relatively homogeneous population; (3) plans were reimbursed on a cost basis and their incentives for risk selection were greatly reduced though not necessarily eliminated.
Third, given that the study was based on cross-sectional data, we should be cautious and not interpret the positive relationship between quality of care and plan choice as causal. It is critical to investigate whether there is a causal effect of quality on plan choice among this population because plans have incentives to improve quality only when the relationship is causal, on which policies should be based.
Our results indicate that low-income parents in New York SCHIP do choose managed care plans with better quality (as measured by CAHPS) for their children. Unless the disenrollment rates are higher among high-quality plans, a positive association of quality with plan choice among new enrollees would suggest that average quality in the market could improve overtime. Since more new enrollees would enroll into high-quality plans, the market share of high-quality plans increases over time. However, further study is greatly needed to investigate the structural relationship between quality and plan choice.
Joint Acknowledgment/Disclosure Statement: Although the work presented here was not funded, we would like to acknowledge support for the original collection of data to evaluate the New York SCHIP from the Agency for Health Care Research and Quality (HS10450), the David and Lucile Packard Foundation, the Health Resources and Services Administration, and the New York State Department of Health (T016804).
Disclosures: There are no conflicts of interests. The research was not funded by any organization and none of the listed authors received any form of financial and material support for the project. The roles of all key individuals are recognized.
Disclaimers: The views expressed here are those of the authors and do not necessarily represent those of the RAND Corporation or the University of Rochester.
1As of 2001, the New York SCHIP program included an expansion of Medicaid (Child Health Plus A) and a separate program (Child Health Plus B). In this study, SCHIP means Child Health Plus B only.
2The SCHIP pricing policy was obtained through personal communications with Satya Pabuwal and Gary Teitel from New York State Insurance Department in January 2006 and June 2007. The 2006 plan rate application package is available at http://www.ins.state.ny.us/ihealth.htm (accessed on July 2, 2007).
3If the SCHIP line score for a plan does not exist in the 2001 report, the SCHIP line or other product line scores of the same plan in the 2001 or 2002 reports would be used in the following sequence: 2001 SCHIP scores, 2002 SCHIP scores, 2001 Medicaid scores, 2001 commercial scores, 2002 Medicaid scores, and 2002 commercial scores. Plan quality measures of different product lines within a plan were moderately correlated, with correlation coefficients between 0.20 and 0.80. For the quality measures of the same plan in 2001 and 2002, CAHPS measures were highly correlated with five correlation coefficients greater than 0.80 (0.81–0.95) and two about 0.70, while those for HEDIS measures ranged from 0.57 to 0.73.
4Child age, child gender, parent employment status, household composition (single parent or not), self-rated child health status, whether an individual first heard about SCHIP through physicians, and whether an individual got the application form from physician offices were tested during the model specification, but they did not enter the final model based on Akaike's information criteria.
5The 65th and 85th percentiles were used as cutoff points in the sensitivity analyses. When using the 65th percentile as the cutoff point, the effect of HEDIS became significant (0.43 percentage point), while that of CAHPS disappeared. However, the effect of HEDIS was not practically important. The interaction effect of having special health care needs with HEDIS was 0.05 percentage point and statistically significant at the 10 percent level. Using the 85th percentiles as the cutoff points failed to change the conclusions.
6Twenty-five out of 30 plans had Medicaid product lines, and they did not report outreach/marketing staff for SCHIP and Medicaid separately. Because no outreach/marketing information was available for year 2000, 2001 plan outreach activities were used for this purpose.
7Other interactions between outreach activity and individual/family characteristics were also tested but they did not enter the final model based on Akaike's information criteria.
8We used a GAUSS program by Kenneth Train, available at http://elsa.berkeley.edu/Software/abstracts/train0196.html. Accessed on June 12, 2006.
9Since the economic theory predicts that better quality of care should attract consumers and there is a strong prior in this study context, we used the 10 percent level to confirm what the theory predicts.
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.