Search tips
Search criteria 


Logo of hsresearchLink to Publisher's site
Health Serv Res. 2002 June; 37(3): 551–571.
PMCID: PMC1434651

Can High Quality Overcome Consumer Resistance to Restricted Provider Access? Evidence from a Health Plan Choice Experiment



To investigate the impact of quality information on the willingness of consumers to enroll in health plans that restrict provider access.

Data Sources and Setting

A survey administered to respondents between the ages of 25 and 64 in the West Los Angeles area with private health insurance.

Study Design

An experimental approach is used to measure the effect of variation in provider network features and information about the quality of network physicians on hypothetical plan choices. Conditional logit models are used to analyze the experimental choice data. Next, choice model parameter estimates are used to simulate the impact of changes in plan features on the market shares of competing health plans and to calculate the quality level required to make consumers indifferent to changes in provider access.

Principal Findings

The presence of quality information reduced the importance of provider network features in plan choices as hypothesized. However, there were not statistically meaningful differences by type of quality measure (i.e., consumer assessed versus expert assessed). The results imply that large quality differences are required to make consumers indifferent to changes in provider access. The impact of quality on plan choices depended more on the particular measure and less on the type of measure. Quality ratings based on the proportion of survey respondents “extremely satisfied with results of care” had the greatest impact on plan choice while the proportion of network doctors “affiliated with university medical centers” had the least. Other consumer and expert assessed measures had more comparable effects.


Overall the results provide empirical evidence that consumers are willing to trade high quality for restrictions on provider access. This willingness to trade implies that relatively small plans that place restrictions on provider access can successfully compete against less restrictive plans when they can demonstrate high quality. However, the results of this study suggest that in many cases, the level of quality required for consumers to accept access restrictions may be so high as to be unattainable. The results provide empirical support for the current focus of decision support efforts on consumer assessed quality measures. At the same time, however, the results suggest that consumers would also value quality measures based on expert assessments. This finding is relevant given the lack of comparative quality information based on expert judgment and research suggesting that consumers have apprehensions about their ability to meaningfully interpret performance-based quality measures.

Keywords: Health plan choice, consumer, quality, quality information

Consumer preferences for health plan provider network features and the impact of information on these preferences have important implications for the ability of managed care to generate cost savings and quality improvements. Research and anecdotal evidence suggest that consumers place great importance on their ability to choose physicians and to maintain existing relationships with them (Davis and Schoen 1997; Scanlon et al. 1997). Managed-care plans have responded by expanding the size of physician networks, loosening restrictions on referrals to network specialists, and offering coinsurance for out-of-network care. However, there is concern that plans with large networks of unaffiliated physicians will not be able to match the cost savings and quality of care achieved by smaller health plans with more highly integrated delivery systems (Miller and Luft 1997; CBO 1995; Staines 1993).

The impact of quality information on consumers' willingness to enroll in plans with restrictive provider networks is not well understood. If high quality reduces the value of unrestricted provider access, then the availability of quality information may improve the competitive position of plans that demonstrate high quality regardless of their organizational structure. To date, however, there is little empirical information about this relationship to guide health plan managers' network design decisions.

The idea that quality information can improve quality of care motivates substantial public and private sector investment in the collection and dissemination of plan performance measures designed to support consumers' plan choices. These efforts have focused heavily on enrollee assessments of nontechnical aspects of care processes and outcomes. There have also been efforts to provide consumers with HEDIS (Health Plan Employer Data and Information System)-based information about plan compliance with technical care processes known to be associated with better health outcomes. The prominence of consumer-assessed quality information is based in part on research suggesting that consumers understand and, thus, value this form of information more than they do process-based performance measures (Scanlon and Chernew 2000; Hibbard and Jewett 1997; McCormack et al. 1996; Jewett and Hibbard 1996; Walker 1990). To date, however, quality measures based on medical experts' judgments have received little attention, despite their potential to expand consideration given to technical quality in plan choices without requiring consumers to obtain clinical expertise.

This study addresses two research questions: (1) How does information about the quality of network physicians affect consumer willingness to enroll in health plans with restrictive provider networks? (2) What is the relative impact of consumer- and expert-assessed quality information on willingness to enroll in restrictive plans?

This study employs an experimental approach to measure the effect of network features and quality information on hypothetical plan choices. The experiment is administered in the form of a survey that asks respondents to make choices among hypothetical sets of plan alternatives. The results are used to predict how the market share of a plan that restricts access to network physicians is affected by the availability of quality information and changes in the quality of its network physicians or other network features, such as out-of-network coinsurance and referral to specialists.


Plan Choice in the Absence of Quality Information

Managed-care organizations compete for market share on the basis of cost and thus have a strong incentive to control use of services. The potential negative impacts of cost-control strategies on the quality of managed care have received widespread attention over the past decade (Miller and Luft 1997; Greenfield et al. 1995; Luft 1994, 1988). These concerns are particularly salient to consumers, because the quality of network physicians is largely unknown at the time that enrollment choices are made. Managed-care plans have responded to consumers' desire for provider choice by expanding the size of provider networks, loosening restrictions on self-referrals to specialists, and providing coinsurance for outside-network services (Gabel et al. 1997).

In consumers' eyes, the quality of managed-care plans stems in large part from the quality of the individual physicians participating in plan networks (AHCRQ/Kaiser 2000; AHCPR/Kaiser/PSRA 1996; Gibbs et al. 1996; Lublin et al. 1995). In turn, consumers view physician quality as a function of interpersonal skills of providers, technical aspects of care processes, and ultimate health outcomes (AHCPR/Kaiser/PSRA 1996; Lublin et al. 1995; Walker 1990). Technical quality is highly dependent on the circumstances surrounding receipt where patients may not know immediately, or may never know, whether experienced outcomes resulted from received care. At the same time, physicians' interpersonal skills can be known through experience over multiple visits. In the absence of reliable external sources of information, consumers learn about quality through personal experience and the experiences of trusted friends and family members (AHCRQ/Kaiser 2000; AHCPR/Kaiser/PSRA 1996). The number of doctors participating in plan networks and rules governing physician choice and access to specialty care determine the cost and ease with which patients can search for physicians once enrolled.

The growth of plans with large, unrestrictive provider networks has important implications for cost and quality. Though these plans may be popular with consumers, the lack of communication and coordination among loosely affiliated physicians may impede the delivery of high-quality, cost-efficient care, thereby reducing the ability of managed-care organizations to generate continued cost savings (Spragins 1997; Goldman et al. 1995). Moreover, this type of health plan has little incentive to invest in improving the quality of its physician network. This is because individual physicians often have affiliations with multiple health plans, making it difficult for a single plan to internalize the benefits of encouraging affiliated physicians to engage in the quality improvement activities it sponsors. Whether smaller plans with highly integrated delivery systems can compete without sacrificing quality of care and efficiency depends on the nature of consumer demand for physician network characteristics and how increasing the availability of consumer information about quality will affect demand.

Impact of Information on Plan Choice

Wider availability of physician quality information may reduce consumer demand for large plans with unrestricted provider choice. High quality may simply compensate for the disutility consumers derive from access restrictions. Alternatively, the ability to directly choose plans that include high-quality physicians may reduce the value of large networks of physicians. If enrollees perceive network physicians to be of high quality and incur substantial costs searching for alternative physicians, economic theory suggests that consumers will perceive the likely benefits of searching for alternative physicians to be low (Hoerger and Howard 1995; Satterthwaite 1979); thereby reducing option value inherent in large, unrestrictive provider networks.

In either case, the increased availability of quality information may mean that more highly integrated plans with smaller, more restrictive networks can successfully compete with larger, less restrictive plans on the basis of quality. If not, these plans may find themselves at a competitive disadvantage, and the ability of market-based competition among health plans to produce continued cost savings and to maintain quality of care may be reduced.

Health plan quality measures in current use generally fall into two broad categories.1 The first are measures based on the rate at which health plans engage in care processes associated with improved health outcomes. The HEDIS is the most prominent example of performance-based health plan quality measures. The second are derived from surveys of health plan enrollees' experiences with access, interpersonal aspects of care, and outcomes, with the Consumer Assessment of Health Plans (CAHPS®) being the most prominent current example.

Studies suggest that consumers fail to use available quality information in making plan and provider choices (Scanlon and Chernew 2000; Hibbard and Jewett 1997). The problem appears to be rooted in consumers' lack of understanding of the organization and delivery of care in managed-care settings and that consumers tend to rely on measures they more fully understand, such as interpersonal skills of providers as well as respect and care shown by providers toward patients (Hibbard and Jewett 1997). Lubalin and others (1995) find that consumers do not appear willing to trust other consumers' judgments about technical aspects of care while they do appear willing to trust consumers' judgments on more subjective aspects of care processes and outcomes. Consumer preference for nontechnical, informal sources of quality information is reflected in a number of focus group studies and surveys and appears to be driven by a perceived lack of understanding of process-based measures (McCormack et al. 1996; Jewett and Hibbard 1996; Walker 1990).

Consumers often rely on experts to form judgments about technical aspects of quality and base purchase decisions on them in situations where product features are difficult to understand (Hibbard et al. 1997). However, the use of expert quality judgments has received relatively little attention in the drive to inform consumers about health-care quality. Consumer information based on expert judgments can take a variety of forms. Credentialing through exams is one way that experts convey to consumers that individual practitioners have attained a given level of competency in a particular field. Increasingly, health plans provide enrollees with information about the board certification and educational attainment of affiliated physicians. Affiliation with elite institutions, such as university medical centers or so-called “centers of excellence,” is another way of conveying competency in the eyes of experts. Finally, expert judgments about the competency of peers can be rendered more directly in the form of surveys or consensus panels. Ratings of hospitals published annually in U.S. News and World Report are a well-known example of such a measure.

Quality measures based on expert judgment have been largely, if not totally, missing from consumer information efforts to date. Nonetheless, two surveys of health-care consumers suggest that a general disinclination to use expert judgment to inform health-care decisions may be declining over time (AHCPR/Kaiser/PSRA 1996; AHCRQ/Kaiser 2000). Survey results from 1996 and 2000 suggest that consumers rely most heavily on informal sources of information, such as friends and family members when making health-care decisions. At the same time, respondents report increasing willingness over time to choose physicians, hospitals, and health plans that have been recommended by medical experts.

A number of studies examine the impact of quality information on plan choice using observational research designs and find little or no effect (Chernew and Scanlon 1997; Scanlon and Chernew 1999; Farley et al. 2000). By contrast, two experimental studies of hypothetical plan choices show a stronger link between plan choice and quality information (Sainfort and Booske 1996; Spranca et al. 2000).

Although there is little direct evidence on the topic, two studies suggest an interaction between quality and the importance of provider network characteristics in health plan choices. Several studies offer indirect evidence. Harris and Keane (1999) found that consumers do not necessarily prefer plans that offer unrestricted access to providers. Their study used data from a 1988 survey of Medicare beneficiaries living in Minneapolis–St. Paul, Minnesota. They found that the plurality of elderly consumers were enrolled in a well-established staff-model HMO (Health Maintenance Organization), the quality of which was perceived to be higher than Independent Practice Association (IPA) plans in the market and comparable to that offered through fee-for-service supplemental insurance plans. Findings from the laboratory component of CAHPS® suggest that subjects' preferences for HMO plans were more sensitive to the presence and form of the quality information than were preferences for PPO (Point of Service) plans (Spranca et al. 2000).

Experimental Approaches to Measuring Consumer Preferences

This study uses an experimental design to examine the impact of different types of quality information on consumers' hypothetical willingness to enroll in health plans with restrictive provider networks. Experiments are particularly valuable in situations where the real world does not provide sufficient variation in the treatment of research interest (Ryan 1999; Cook and Campbell 1979; Newhouse and McCellan 1998). This study is a case in point. Without an experimental design it would be difficult, if not impossible, to assess the potential impact of expert-assessed quality information on trade-offs among health plan attributes, because these types of measures are not currently distributed to well-defined populations of health plan choosers. Moreover, readily available observational data often lack information needed to characterize the choice set from which the sample member chose his or her current plan. Even when it is possible to observe the decision maker's choice set (for example, data from a single employer group), there is often too little independent variation among attributes across the set of plan alternatives to permit powerful statistical tests of the impact of features on plan choice.

Experimental Design and Administration

The experiment was administered in the form of a survey where respondents were assigned to a series of choice sets comprised of hypothetical health plan profiles and asked to indicate for each set the plan in which he or she would be most likely to enroll. In order to minimize the potential for confounding distractions while maintaining realism, the survey instrument was extensively pilot-tested before fielding. The final version of the survey contained nine choice sets each comprised of three plan profiles. Each plan is comprised of four to seven attributes, depending on whether the choice set contained quality information. Choice sets varied in terms of (1) the presence or absence of quality information and (2) the form of the quality information. This was done by randomly assigning subjects to one of three types of choice sets or experimental arms at each choice opportunity. Plans in Arm 1 contained only provider network features and no quality information. Plans in Arm 2 contained network features and assessments of the quality of network doctors drawing from the range of various types of expert-assessed quality measures discussed in the Background section. Plans in Arm 3 contained provider network features and consumer assessments of the quality of network doctors similar to CAHPS® measures.

The levels that each attribute can take on are intended to reflect a realistic range of values. The six quality measures are presented in terms of proportions of plan doctors or plan enrollees in the upper portions of the corresponding distributions (i.e., proportion of plan doctors rated “excellent” or enrollees “extremely satisfied”). This method of presentation parallels the approach adopted by the Health Care Financing Administration to present CAHPS® data to Medicare beneficiaries (Kosiak 1998).2

Provider network features included the proportion of local doctors affiliated with the network (25 percent, 50 percent, 75 percent), whether the plan allows enrollees to self-refer to network specialists (Yes, No), out-of-network copayments (10 percent, 30 percent, 50 percent), and whether the subject's personal doctor or nurse is included in the network (Yes, No). The restrictiveness of each plan's provider network varies along a continuum and depends on the level of the attributes describing the network. Compared to less-restrictive plans, more-restrictive plans require greater out-of-network copayments, have a smaller proportion of local doctors in the network, do not allow self-referrals to network specialists, and do not include the subject's current provider.

Consumer assessments take the form of the reported proportion of enrollees using care within the last six months who report being “extremely satisfied” (10 percent, 30 percent, 50 percent) with involvement in care decisions, explanations of care, and results of care. Expert judgments take the form of the proportion of network doctors rated “excellent” by a panel of medical experts (25 percent, 50 percent, 75 percent), the proportion of network doctors scoring “above average” on a test of clinical knowledge administered by a professional organization (25 percent, 50 percent, 75 percent), and the proportion of network doctors affiliated with local university medical centers (25 percent, 50 percent, 75 percent).

The set of attribute levels can be used to generate thousands of unique plan profiles—too many to be considered by an individual subject. For this reason, a smaller subset of 162 plan profiles in each arm was drawn from the set of all possible plan profiles using an algorithm that seeks to optimize the precision of the estimated main effects of each attribute on enrollment probabilities (Zwerina et al. 1996). The selected plan profiles from 54 unique choice sets in each arm and each respondent was randomly assigned to consider three sets from each of the three arms.

The instructions asked respondents to choose for themselves—not for their dependents and spouses—and to disregard information about the affiliation of one's own doctor or nurse if the respondent did not have a current provider relationship. Respondents were asked to assume that each of the plans in the choice set required a $25 per month contribution3; prescription drugs were covered with a $7 copayment, outpatient physician visits were covered with a $10 copayment, stays at network hospitals were covered in full, and all plans required that enrollees select a primary care physician.

The experiment was administered in spring 2000 to 206 adults between the ages of 25 to 64 in the Los Angeles metropolitan area who had private insurance obtained through an employer or purchased individually. It was administered in two ways in order to balance reliability and cost considerations. First, the survey was administered to 105 respondents in a group setting at facilities located on site at RAND in Santa Monica, California. Group administration enables researchers to closely monitor subjects and motivate subjects to attend to the experimental task with generous financial compensation for time and travel costs. Second, the survey was administered to 101 shoppers in a West Los Angeles mall. Lower labor and recruiting costs, as well as smaller incentive payments (because of lower travel costs and smaller time commitment), made mall intercept administration substantially less expensive than group administration.

The resulting two samples are comparable in a number of respects. In both samples, roughly 85 percent of respondents reported obtaining health insurance through an employer and 70 percent of those respondents have a choice of two or more health plans. Also members of both samples provided similar responses to health status questions. At the same time, group administration respondents were older, more affluent, more educated, less likely to be enrolled in a plan that requires choice of a primary care physician, less likely to be female, and more likely to be white compared to mall intercept respondents. Overall, study participants are more affluent, highly educated, and ethnically diverse compared with the overall population with private health insurance in the United States (Cogswell et al. 1997; NCHS 2000). Despite their relative affluence, participants are less likely than other Americans to have employer-sponsored insurance reflecting the lower rate of employer-sponsored coverage in Los Angeles and California as a whole (Brown et al. 2000).

Analysis of Experimental Choices

Theoretical Model

The conditional logit model (McFadden 1973) underlies the analysis of the experimental data. This model and variations of it form the basis of many other studies of consumer choice among competing alternatives, both experimental and nonexperimental, in the marketing and consumer economics as well as health services research literature (Zwerina et al. 1996; Bunch et al. 1996; Kuhfeld et al. 1994; Feldman et al. 1989; Revelt and Train 1998; Harris and Keane 1999; Chernew and Scanlon 1998). The model assumes that consumers view health plans as bundles of attributes and choose plans by comparing attributes across alternatives and choosing the bundle that offers the greatest utility, or in other words, the most desirable combination of attributes. In formal terms, the utility of a particular plan to decision maker i in health plan j in choice set t is written

equation image
equation image

where Xijt is a vector of health plan features and β is a vector of weights placed on each feature which does not vary across decision makers and is unique to a particular arm of the experiment.4 The error term epsilonijt represents individual deviations in utility that are assumed to be randomly distributed across decision makers, health-plan alternatives, and choice sets. The indicator variable dijt takes on the value one signifying that the decision maker chooses plan j when the utility derived from plan j is greater than the utility derived from any other plan in the choice set t.

Under the assumption that epsilonijt is distributed standard extreme value, the probability that a decision maker chooses plan j can be written

equation image

Standard statistical software is used to obtain a maximum likelihood estimate of the model parameters

An external file that holds a picture, illustration, etc.
Object name is hesr_037_mu1.jpg

. Separate sets of coefficients are estimated for each of the three arms of the experiment.

Preliminary Analyses

Method Effects

Several preliminary analyses were conducted to assess whether respondents' choices were influenced by the design of the experiment. The results suggested that both mode of administration (i.e., group versus mall administration) and position of the hypothetical plan on the page (i.e., left, middle, right) influenced plan choices. Because neither of these two method effects influenced the statistical significance or substantive interpretation of the findings, no controls for either effect are included in the final models.

Independence of Irrelevant Alternatives

Conditional logit models rely on the restrictive assumption of the independence of irrelevant alternatives (IIA) implied by the independent errors assumption in equation 1. Models that require IIA can result in unrealistic predictions about the nature of substitution across plans to the extent that there are theoretically unobserved attributes and unobserved differences in tastes across decision makers that are not directly captured in the model (Hausman and Wise 1978). In order to assess the impact of the IIA assumption, preliminary analysis was conducted using random parameters (or heterogeneous) logit models that control directly for taste differences across consumers. These results provided statistically significant evidence of preference heterogeneity, suggesting that the IIA assumption is in fact violated in the conditional logit model.5 However, the decision was made to report the simpler conditional logit coefficients, because the implied choice behavior of an average decision maker in the simulations reported in the next section was virtually unchanged across the two types of logit models. Nonetheless, the random parameters model would be more appropriate for modeling the entrance and exit of plans from a market. This is not done in this application.


Coefficient Estimates

Overall, experimental design was successful in yielding precise estimates of the impact of plan features on plan choices (see Table 1). The coefficient estimates measuring plan features in each of the three models are all significantly different from zero at the .01 level with the exception of the university affiliation of network doctors in Model 2 which is significant at the .05 level. All of the coefficients also have the expected sign. The sign and precision of these estimates suggest that the experimental choices were not idiosyncratic and responded to variation in plan features in the manner that the choice theory predicts.

Table 1
Estimated Conditional Logit Coefficients and Standard Errors

The availability of quality information reduces the magnitude of the coefficients on provider network features by roughly one-third to one-half. Likelihood ratio tests conducted separately for each of the two types of information (Model 1 versus Model 2 and Model 1 versus Model 3) rejected the hypotheses that the presence of quality information left the coefficient estimates on provider network features unchanged. The coefficients on provider network features for the expert-assessment arm are 15 to 40 percent smaller than those for the consumer-assessment arm. However, a likelihood ratio test was used to test the hypothesis that the coefficients on provider network features differ depending on the type of quality information provided (Model 2 versus Model 3) and the test failed to reject the hypothesis of no difference. Although the results provide statistical evidence that quality information reduces the importance of provider network features, it is difficult to interpret the magnitude of this effect from the raw logit coefficients because of unknown scale differences across models. The simulation results provided in the next section help to illustrate the magnitude of these effects.

Marginal Probabilities

In this section, the coefficients reported in Table 1 are used in simulations to predict the impact of access expansions and improved quality ratings on plan choice probabilities. The simulation starts with a hypothetical market where two plans with identical provider network features and quality ratings each have a 50 percent share of the market. Next, one of the two plans, the “Expanding Plan,” seeks to gain market share by changing provider network features to reduce restrictions on physician access or by engaging in quality improvement activities that result in improved quality ratings. The remaining plan is referred to as the “Restrictive Plan.” Table 2 reports the average change in the probability of choosing the Expanded Plan over the Restrictive Plan resulting from changes in the network features of the Expanding Plan, holding all other features constant. The simulated access expansions and quality improvements are meant to be possible, without necessarily being typical. For example, a 25 percentage point increase in the number of doctors with university affiliations could arise, if a plan granted affiliation to a university-based medical group or through an expansion of a university-based group already in the plan, perhaps through the purchase of community-based medical groups.

Table 2
Marginal Effects of Expanded Access to Network Providers and Improved Quality Ratings on the Average Predicted Probability of Enrolling in the “Expanding Plan”

Table 2 shows the availability of quality information reduces the importance of network features in plan choices, even when the information suggests the competing plans are of equivalent quality. The presence of quality information reduces the impact of changes in network features on the probability of choosing the Expanding Plan by roughly one-half to one-third. For example, allowing enrollees to self-refer to network specialists increases the probability of choosing the Expanding Plan by 28.8 percentage points when no quality information is provided. This increase falls to 16.5 percentage points in the case when expert assessments are available and to 19.0 percentage points in the case of consumer assessments. Table 2 also illustrates the relative importance of specialist referral and maintaining relationships with one's own doctor. In each of the three models, the impact on choice probabilities of adding self-referral to specialists and having one's own doctor in the network are roughly double that of a fairly substantial percentage point increase in the proportion of local doctors in the network and coinsurance rate.

Turning to quality rating improvements, the simulation results reveal that the impact of quality depends more on the actual measure than the type of measure. Increasing the proportion of doctors affiliated with university medical centers by 25 percentage points increases the probability of enrolling in the Expanding Plan by 4 percentage points. By contrast, increasing the proportion of enrollees satisfied with the results of care provided by network doctors by a similar amount increases the enrollment probability by 19.6 percentage points. The estimated impacts of each of the four remaining quality measures are much more similar in magnitude. Twenty-five percentage point improvements in each of these four measures increases the probability of enrolling in the Expanding Plan between 7.8 and 9.3 percentage points, each well within a standard deviation of the other.

Finally, the results suggest that while respondents used both types of quality information when available, the quality ratings were less important than access to specialists and having one's own doctor in the network. The only quality measure as influential as either of these two network features was satisfaction with results of care. At the same time, the quality ratings were more influential than the proportion of local doctors in the network.

Access–Quality Trade-Offs

This section presents estimates of the percentage point improvements in the quality ratings of competing plans required to keep consumers indifferent on average to increased provider access in the Expanding Plan. Table 3 reveals wide variation in the quality improvements in the Restrictive Plan required to compensate for expanded provider access across type of quality measure and network feature. Consumers required the smallest changes in the results of care measure, reflecting the high relative importance placed on the measure. By contrast, consumers required the largest changes in the measure of medical center affiliation, reflecting the low relative importance placed on this measure. In the case of specialist referral and having one's own doctor in the network, even a maximum possible, and not unrealistic, difference of 100 percent in the proportion of doctors with university affiliations is not sufficient to make respondents indifferent. Overall, modest quality differentials were required to compensate for expansions in the proportion of local doctors in the network and reduced out-of-network coinsurance, reflecting their low relative importance. By contrast, substantial differentials were required to compensate for expanded specialist referral and having one's own doctor in the network.

Table 3
Percentage Point Changes in Quality Ratings Required to Make Consumers Indifferent to Expansion in Provider Access


The apparent willingness to trade access restrictions for quality implies that relatively small plans that restrict access to a limited set of physicians can successfully compete against less restrictive plans when they can demonstrate high quality. However, the results of this study suggest that in many cases, the level of quality required for consumers to accept access restrictions may be so high as to be unattainable. Because the experimental setting provides a high degree of assurance that decision makers actually used the provided quality information, the results represent an upper bound on the effect of information on plan choice. Thus, real-world quality differentials required to drive market share are likely to be substantially larger than those suggested here.

The results add to a substantial body of empirical literature indicating that consumers highly value other consumers assessments of health-plan quality, in particular, their assessments of outcomes of care. At the same time, the results also suggest that consumers value various forms of expert assessment measures as least as much as they value consumer assessments of care processes. Although the practical and political barriers to their widespread use are considerable, the results of this study suggest that the benefit of expert assessments to consumers, and likewise to provider organizations held in high esteem by experts, may be substantial.

Although the experimental design used here overcomes practical barriers to studying the impact of information on plan choice in real-world settings, the choices that respondents make are hypothetical and may not reflect real-world choices. Experiments are valuable because they reveal, with a relatively high degree of certainty, whether an intervention—regardless of whether or not it exists in the real world—can have an effect, albeit under special circumstances. Thus, experimental findings are best viewed in the context of a larger body of empirical research, rather than as definitive. Because of the greater certainty that subjects are exposed to quality information (as opposed to throwing it away), experimental designs can help determine whether the weak effect of information in observational settings is due to inattention or irrelevance.

Generalizability is limited because the results reflect individual, rather than joint, preferences. Respondents were asked to choose for themselves and not other family members in order to reduce potential confounding effects from joint decisions and unobserved plan options available to family members other than respondents. An interesting and important topic for further research is to explore how the process of balancing needs and preferences of various family members affects trade-offs between access and quality.

Future work in this area should focus on understanding the relationship among health status, willingness to accept access restrictions, and the level at which provider quality is measured. Those in poor health have more specific medical needs and more established relationships with physicians and find physician quality measured at the plan or network level uninformative. By contrast, plan level quality may be of greater interest to healthier individuals who do not yet know which type of care will be needed, if needed.


1These measurement systems are discussed in detail by Scanlon and Chernew (2000), Epstein (1988), and McGee et al. (1996).

2The quality ratings used in the experiment are meant to reflect reporting methods in current use. An anonymous reviewer correctly points out that in some contexts, measures of dispersion may be as or more valuable than measures of mean quality. However, the impact of reporting method on choice behavior is beyond the scope of this study.

3Level out-of-pocket premium contributions eliminate the possibility of decision makers using premium variation as a proxy for unstated quality differences across plan alternatives. Level out-of-pocket premiums may also make the choice task realistic as Long and Marquis (1999) report that only a quarter of employers provide level premium contributions resulting in out-of-pocket premium differences.

4The subscript indexing the three arms of the experiment is suppressed to simplify notation.

5Estimating variance parameters does not change the statistical significance of the other parameters in the model.

This project was supported by Agency for Healthcare Research and Quality Grants RO3HS10367 and T32- HS00046.0


  • Agency for Health Care Policy and Research Kaiser Family Foundation; Princeton Survey Research Associates. Rockville MD: AHCRQ/Kaiser; 1996. Month. “Americans as Health Care Consumers: The Role of Quality Information.”
  • Agency for Health Care Research and Quality and Kaiser Family Foundation. Rockville MD: AHCRQ/Kaiser; 2000. Month. “National Survey on Americans as Health Care Consumers: An Update on the Role of Quality Information.”
  • Brown RE, Wyn R, Teleki S. 2000. “Disparities in Health Insurance and Access to Care Across U.S. Cities.” UCLA Center for Health Policy Research.
  • Bunch DS, Louviere JJ, Anderson D. 1996. “A Comparison of Experimental Design Strategies for Multinomial Logit Models: The Case of Generic Attributes.” Working paper, Graduate School of Management, University of California at Davis.
  • Carson RT, Louviere JJ, Anderson D, Aradie P, Bunch D. “Experimental Analysis of Choice.” Marketing Letters. 1994;5(4):351–68.
  • Chernew M, Scanlon DP. “Health Plan Report Cards and Insurance Choice.” Inquiry. 1998;35(1):9–22. [PubMed]
  • Cogswell ME, Nelson D, Koplan SP. “Surveying Managed Care Members on Chronic Disease.” Health Affairs. 1997;16(6):219–27. [PubMed]
  • Congressional Budget Office. 1995. “The Effects of Managed Care and Managed Competition.” Memorandum February.
  • Cook TD, Campbell DT. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally College Publishing; 1979.
  • Davis K, Schoen C. Managed Care, Choice, and Patient Satisfaction. New York: 1997.
  • Epstein Arnold M. “Rolling Down the Runway: The Challenges Ahead for Quality Report Cards.” Journal of the American Medical Association. 1988;279(21):1691–6. [PubMed]
  • Farley D, Short PF, Elliot MN, Kanouse DE, Brown JA, Hays RD. RAND, Santa Monica, CA: 2000. “Effects of CAHPS® Health Plan Performance Information on Plan Choices by New Jersey Medicaid Beneficiaries.” Unpublished manuscript. [PMC free article] [PubMed]
  • Feldman RM, Finch B, Dowd, Cassou S. “The Demand for Employer-Based Health Insurance.” Journal of Human Resources. 1989;23(1):115–42.
  • Gabel JR, Ginsberg PB, Hunt KA. “Small Employers and their Health Benefits, 1988–1996: An Awkward Adolescence.” Health Affairs. 1997;16(5):103–10. [PubMed]
  • Gibbs DA, Sangl JA, Burrus B. “Consumer Perspectives on Infor-mation Needs for Health Plan Choice.” Health Care Financing Review. 1996;18(1):55–73. [PubMed]
  • Goldman DP, Hosek SD, Dixon LS, Sloss ES. “The Effects of Benefits of Design and Managed Care on Health Care Costs.” Journal of Health Economics. 1995;14:401–18. [PubMed]
  • Greenfield S, Rogers W, Mangotich M, Carney M, Tarlov AR. “Outcomes of Patients with Hypertension and Non–Insulin-Dependent Diabetes Mellitus Treated by Different Systems and Specialties.” Journal of the American Medical Association. 1995;274(18):1436–44. [PubMed]
  • Harris KM, Keane MP. “A Model of Health Plan Choice: Inferring Preferences and Perceptions from a Combination of Revealed Preference and Attitudinal Data.” Journal of Econometrics. 1999;(89):131–57.
  • Hausman JA, Wise D. “A Conditional Probit Model for Qualitative Choice: Discrete Choice Recognizing Interdependence and Heterogeneous Preferences.” Econometrica. 1978;42(2):403–26.
  • “Health Care in America: Your Money or Your Life.” Economist. 1998 Mar 7; [PubMed]
  • Hibbard JH, Jewett JJ. “Will Quality Report Cards Help?” Health Affairs. 1997;16(3):218–28. [PubMed]
  • Hibbard JH, Slovic P, Jewett JJ. “Informing Consumer Decisions in Health Care: Implications from Decision-Making Research.” Milbank Quarterly. 1997;75(3):395–414. [PubMed]
  • Hoerger TJ, Howard LZ. “Search Behavior and Choice of Physician in the Market for Prenatal Care.” Medical Care. 1995;33(4):332–49. [PubMed]
  • Jewett JJ, Hibbard JH. “Comprehension of Quality of Care Indicators: Differences among Privately Insured, Publicly Insured, and Uninsured.” Health Care Financing Review. 1996;18(1):750–94. [PubMed]
  • King G, Tomz M, Wittenberg J. “Making the Most of Statistical Analysis: Improving Interpretation and Presentation.” American Journal of Political Science. 2000;44(2):347–61.
  • Kosiak B. Arlington Va December: 1998. “Meet HCFA's Medicare Health Plan Reporting System.” pp. 11–12. Paper presented at Making Quality Count: Helping Consumers Make Better Health Plan Choices sponsored by Health Care Financing Administration.
  • Kuhfeld WF, Tobias RD, Garratt M. “Efficient Experimental Design with Marketing Research Applications.” Journal of Marketing Research. 1994;31:545–57.
  • Larson E. “The Soul of an HMO.” Time. 1996;147(4):44–52. [PubMed]
  • Louviere JJ, Bagozzi R. Advanced Methods of Marketing Research. Oxford: Blackwell; 1994. “Conjoint Analysis.” pp. 223–59.
  • Lubalin J, Schnaier J, Forsyth B, Gibbs D. Final Report. Rockville MD: 1995. “Design of a Survey to Monitor Consumers' Access to Care, Use of Health Services, Health Outcomes, Patient Satisfaction.” (282-92-0045) submitted to Agency for Health Care Policy and Research.
  • Luft HS, Luft HS. HMOs and the Elderly. Ann Arbor MI: Health Administration Press; 1994. “Introduction.” pp. 1–12.
  • McCormack LA, Garfinkel SA, Schaier JA, Lee AJ, Sangl JA. “Consumer Information Development and Use.” Health Care Financing Review. 1996;18(1):15–29. [PubMed]
  • McFadden D, Zarembka P. New York: Academic Press; 1973. “Conditional Logit Analysis of Qualitative Choice Behavior Frontiers in Econometrics.” pp. 105–142.
  • McGee J, Kanouse DE, Sofaer S, Hargraves JL, Hoy E, Kleimann S. “Making Survey Results Easy to Report to Consumers: How Reporting Needs Guided Survey Design in CAHPS.” Medical Care. 1999;37(3):M532–40. [PubMed]
  • Miller LH, Luft HS. “Does Managed Care Lead to Better or Worse Quality of Care?” Health Affairs. 1997;16(5):7–25. [PubMed]
  • National Center for Health Statistics Health: United States. Hyattsville MD: U.S. Department of Health and Human Services; 2000. “With Adolescent Health Chartbook.” Table 1 p. 123 and Table 128 p. 339.
  • Newhouse JP. the RAND Health Insurance Group Free for All: Lessons from the RAND Health Insurance Experiment. Cambridge MA: Harvard University Press; 1993.
  • Newhouse JP, McClellan M. “Econometrics in Outcomes Research: The Use of Instrumental Variables.” Annual Review of Public Health. 1998;19:17–34. [PubMed]
  • Orme B. Sawtooth Software, Inc; 1998. “Sample Issues for Conjoint Analysis Studies.”
  • Revelt D, Train K. “Incentives for Appliance Efficiency: Random-Parameter Logit Models of Household's Choices.” Review of Economics and Statistics. 1998;80(4):647–57.
  • Ryan M. “Role for Conjoint Analysis in Technology Assessment in Health Care.” Journal of Technology Assessment. 1999;15(3):443–57. [PubMed]
  • Sainfort F, Booske BC. “Role of Information in Consumer Selection of Health Plans.” Health Care Financing Review. 1996;18(1):31–54. [PubMed]
  • Satterthwaite MA. “Consumer Information, Equilibrium Industry Price, and the Number of Sellers.” Bell Journal of Economics. 1979;10:483.
  • Scanlon DP, Chernew M. “HEDIS Measures and Managed Care Enrollment.” Medical Care Research and Review. 1999;56(2):60–84. [PubMed]
  • Scanlon DP, Chernew M. “Managed Care and Performance Measurement: Implications for Insurance Markets.” North American Actuarial Journal. 2000;4(2):128–34.
  • Scanlon DP, Chernew M, Lave JR. “Consumer Health Plan Choice: Current Knowledge and Future Directions.” Annual Review of Public Health. 1997;18:507–28. [PubMed]
  • Smith MA. Paper presented at Making Quality Count: Helping Consumers Make Better Health Plan Choices, sponsored by Health Care Financing Administration. Arlington Va: 1998. “Environmental Scan.” pp. 11–12.
  • Spragins Ellyn. “Take My Freedom Please.” Newsweek. 1997;7:81–2. [PubMed]
  • Spranca M, Kanouse DE, Elliot M, Short PF, Farley DO, Hays RD. “Do Consumer Reports of Health Plan Quality Affect Health Plan Selection?” Health Services Research. 2000;35(5):933–47. [PMC free article] [PubMed]
  • Staines VS. “Potential Impact of Managed Care on National Health Spending.” Health Affairs. 1993;12:48–57. [PubMed]
  • Walker AJ, Lohr K. Medicare: A Strategy for Quality Assurance. Vol. 2. Washington DC: National Academy Press; 1990. “Results of the Medicare Beneficiary and Physician Focus Groups.” pp. 35–90.
  • Zwerina K, Huber J, Kuhfeld WF. Duke University; 1996. “A General Method for Constructing Efficient Choice Experiments.” Working paper.

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust