|Home | About | Journals | Submit | Contact Us | Français|
The recently enacted Patient Protection and Affordable Care Act includes a major expansion of Medicaid to low-income adults in 2014. This paper describes the Oregon Health Study, a randomized controlled trial that will be able to shed some light on the likely effects of such expansions. In 2008, Oregon randomly drew names from a waiting list for its previously closed public insurance program. Our analysis of enrollment into this program found that people who signed up for the waiting list and enrolled in the Oregon Medicaid program were likely to have worse health than those who did not. However, actual enrollment was fairly low, partly because many applicants did not meet eligibility standards.
One of the primary components of the recently enacted health reform law, the Patient Protection and Affordable Care Act of 2010, is a major expansion of Medicaid, particularly to low-income adults. The probable impact of such an expansion on the newly covered population is of obvious interest. This article describes an ongoing Medicaid expansion experiment in Oregon that provides a unique opportunity to investigate its impact through randomized evaluation.
For a limited time in early 2008, Oregon had a reservation list (a waiting list) for enrollment in its previously closed program that expanded Medicaid coverage to low-income adults. More than 85,000 people put their names on the list, but the state did not have enough funding to cover them all. Between March and October 2008, about 30,000 names were randomly drawn from the list, and those selected were permitted to apply for coverage.
Oregon’s coverage expansion essentially amounts to a randomized controlled trial of expanding public health insurance coverage for low-income adults. Those not selected from the list provide a control group for those who were. Working closely with Oregon’s Division of Medical Assistance Programs and the Office for Oregon Health Policy and Research, we launched the Oregon Health Study—a multiprong research effort designed to measure the effects of this expansion on myriad outcomes. To our knowledge, the study is the first randomized controlled trial on the effects of covering the formerly uninsured. In this paper we describe the Oregon experiment—and some of our planned analyses—in more detail.
The parallel between the Oregon expansion and the new federally mandated expansion makes the results of our study particularly timely. Of course, the populations affected in Oregon and nationally, as well as the enrollment process, differ in important ways. Caution is thus warranted in generalizing results from our study to other settings.
Two issues discussed below are worth highlighting at the outset. First, Oregon is different from the rest of the country. For instance, its population is whiter, and it has demonstrated a particular interest in Medicaid policy innovation. Second, enrollment in the Oregon Medicaid expansion was voluntary. There was no mandate like the one for individuals to purchase insurance in the federal health reform legislation. Take-up of coverage was low among the people selected from the list.
Low take-up, or a relatively small portion of selected individuals who acquire insurance, does not pose a problem for causal inference within the confines of our study. However, it does suggest caution in applying our results to a mandatory expansion. In that case, even with imperfect enforcement, we might expect substantially higher take-up. Furthermore, different types of people would gain coverage if there were a mandate for everyone to do so.
A large amount of scholarly research has examined the impact of insurance on health outcomes and the use of health care. Broadly speaking, this research fits into three main categories: observational studies, quasi-experimental studies, and randomized experiments.
Hundreds of observational studies over the past several decades have compared the health of and health care use by insured and uninsured populations. The Institute of Medicine (IOM) reviewed this literature and reported that the lack of health insurance leads to thousands of preventable deaths each year in the United States.1 The studies collectively suggest that expanding insurance coverage is critically important to improving public health.2 Yet there remains much debate about the relationship between health spending and health outcomes.3–5
Some reviewers have pointed out limitations in the literature from the observational studies.6,7 In particular, few of the studies effectively controlled for underlying differences in health status and other characteristics between uninsured and insured patients. If the two groups are different in unobserved ways, whether or not they have insurance might not be the cause of their different outcomes.
Quasi-experimental studies rely on differences in the availability of public insurance, or other policy changes, to assess the causal effects of insurance. For example, variation across time and space in state Medicaid expansions allows researchers to try to assess the effects of such expansions.
The effects have been mixed. Some studies have found evidence that public health insurance reduces mortality among infants and children8–10 and improves some outcomes—although not mortality—among the elderly.11–14 Although they are much more persuasive than observational studies, quasi-experimental studies are not truly randomized. Thus, investigators must rely on the assumption that the people whose health insurance was affected by environmental or policy changes are otherwise identical to the people in the comparison group.
A randomized experiment provides a particularly compelling way of assessing the causal impact of health insurance. The random assignment of people into either the treatment or the control group means that the groups’ characteristics do not differ systematically, except for the variable being studied: in this case, the presence or absence of health insurance. Researchers can thus be sure that the variable is causing any differences. Of course, randomized experiments have their limitations, some of which we detail below in the context of our study.
Before ours, there was only one randomized study of health insurance in the United States: the RAND Health Insurance Experiment of the mid-1970s.15 This federally funded experiment randomly assigned participants to plans with varying amounts for copayments and maximum out-of-pocket spending limits. Researchers followed the participants for three to five years, collecting biometric health measures such as blood pressure.
The strength of the experimental design is partly responsible for the enduring impact of the RAND study, which remains a standard reference in the health insurance literature. However, the study also has two key limitations for current policy analysis. First, it ended more than twenty-five years ago, and health care was very different then. Second, it was designed to measure the impact of insurance generosity, not the impact of having any insurance at all. No one in the RAND study was assigned to a “no coverage” group. The maximum out-of-pocket exposure for participants was $1,000.
Whether insurance with lower cost sharing leads to improved health is an important question. But it is very different from the one that our study asks: What are the effects of having health insurance?
Oregon’s Medicaid program, the Oregon Health Plan, has two separate parts. Oregon Health Plan Plus is for the categorically eligible Medicaid population, which includes groups such as low-income children, pregnant women, welfare recipients, and poor elderly and disabled populations. Oregon Health Plan Standard is for poor adults who are financially but not categorically eligible for the Plus program. The Standard program, which is the subject of our study, provides relatively comprehensive benefits with no consumer cost sharing and minimal premiums.16,17
Eligibility for the Standard plan is limited to adults ages 19–64 who are Oregon residents and U.S. citizens or legal immigrants, and who have been without health insurance for at least six months, have incomes below the federal poverty level, and have less than $2,000 in assets.
Budgetary shortfalls forced the closure of the standard plan to new enrollment in 2004. In early 2008 the state determined that it had the resources to enroll an additional 10,000 adults. Because Oregon’s Department of Human Services correctly anticipated that the demand for the program would far exceed the new slots, the agency received permission from the Centers for Medicare and Medicaid Services (CMS) to open a waiting list and randomly draw names from it to determine who would be offered the opportunity to enroll. This process was chosen because it gave everyone on the list an equal opportunity to be selected.18
Thanks to an aggressive outreach campaign, by the February 29, 2008, deadline more than 85,000 people were on the list.19
To sign up, people had to provide their name, date of birth, sex, address, telephone number, and preferred language of communication. Enrollment forms were available in English and Spanish. People were also asked to list the name, sex, and date of birth of anyone age nineteen or older in the household whom they wished to add to the reservation list.20
We used this information to exclude from our study population anyone who gave an address outside of Oregon, was born before 1944 or after 1989, had enrolled in the plan as of January 2008, gave an institutional address, or was signed up by an unrelated third party. Those signed up by a third party were not likely to be eligible or to enroll. These exclusions left us with a sample of more than 70,000.
Drawings of 3,000–6,000 names from the waiting list were conducted monthly from March to October 2008. People whose names were drawn were eligible to apply for Oregon Health Plan Standard coverage and were sent application materials. Everyone received a two-page application form; it could be accompanied by up to eight supplemental forms.
The main form asked for the names of all household members applying for coverage and inquired about their Oregon residence, U.S. citizenship, insurance coverage over the past six months, household income over the past two months, and assets. Documentation of identity and citizenship and proof of income had to be returned with the completed form.21
Those who submitted the required paperwork and met the eligibility criteria were enrolled in the plan. All enrolled individuals had to recertify their eligibility every six months.
For the Oregon Health Study, which is ongoing, we are collecting data from several sources to compare the “treatment group”—in other words, the 29,411 people whose names were randomly drawn from the waiting list—and the control group, drawn from those who were not selected from the list using the same random selection procedure. We will assess the causal effects of coverage expansions on a wide range of outcomes, including access to and use of health care, household finances and medical debt, health behaviors, physical and mental health outcomes, labor-market outcomes, and other measures of well-being.
The analysis will help us shed light not just on the impacts of insurance on health and well-being, but on the channels through which health insurance affects outcomes. We hope to examine variations in subpopulations of particular interest, such as the elderly and those in poor health.
The Oregon Health Study will rely on three primary data sources: mail and telephone surveys, in-person interviews and health screenings, and administrative records. Data collection began in the summer of 2008 and will continue at least through late 2010.
We surveyed the people on the waiting list by mail and phone, contacting those who were selected from the list and those who were not. The survey included questions on current insurance status and use of health care, health care costs and financial strain, health status, and demographic factors.
An initial survey in the summer and fall of 2008 of 29,172 of the people selected and another 28,381 who were not selected yielded a 45 percent response rate. We conducted a follow-up survey of the same individuals a year later, including additional efforts to boost response rates, and we achieved an effective response rate of slightly above 50 percent.
We began an intensive effort to collect data in person in September 2009; the effort will continue into late 2010. We targeted a randomly selected subset of the larger study panel from the greater Portland area. Preliminary results suggest that we will achieve a response rate of about 70 percent. We expect to complete more than 10,000 interviews.
The in-person data collection complements the mail and phone surveys in three important ways. First, the comprehensive set of interview questions will give us a much more detailed picture of individuals’ health and health care experiences. Second, we will collect objective physical health measurements such as blood pressure, height, and weight, as well as blood samples so that we can evaluate cholesterol, blood sugar, and measures of immune response. Such objective physiological measures are particularly important because insurance may affect not only health but also the likelihood of diagnosis and thus individuals’ knowledge of their health conditions. In fact, the RAND study found effects of health insurance generosity on objectively measured blood pressure when respondents did not report any changes themselves. People may be less likely to acquire insurance if they do not know that they are sick. Third, the response rate of in-person contacts should be much higher than that of mail and phone surveys.
We have worked with the State of Oregon to match data from the waiting list with data from various administrative sources. Information on the applications submitted by people who were selected from the list allows us to investigate who chose to enroll.
Data on Oregon Health Plan coverage for the entire list allow us to examine the impact of the selection off of the reservation list on public insurance coverage, and how this has varied over time.
In addition, we have matched information from the list with statewide data on hospital inpatient services, and we intend to include emergency department and mortality records in the future. We can use these data to investigate the impact of the random selection on health care use and mortality for the entire list, as well as to detect any differences in participation in the other types of data collection among those selected versus those not selected from the reservation list.
The primary analysis will compare all of those selected in the lottery (whether they were eventually insured or not) with those not selected in the lottery (again, whether they were eventually insured or not). This is the “intent to treat” analysis common to randomized controlled trials, and it provides an estimate of the impact of eligibility to apply for Oregon Health Plan Standard that is not biased by imperfect rates of enrollment.
We can also estimate the effect of enrolling in insurance itself that is similarly unbiased, analogous to the effect of “treatment on the treated.” Of course, enrollment rates among those who are eligible still affect the interpretation of these results: Our estimate of the impact of insurance coverage applies specifically to the type of people who gain coverage in response to being selected.
The randomized allocation of health insurance allows us to overcome many of the difficulties that observational studies face in drawing causal inferences about the impact of health insurance eligibility or coverage. However, two important caveats must be borne in mind in interpreting our analyses.
First, attrition within the study population may contaminate the random design, if people drop out of the treatment and control groups at different rates. This is a particular concern for those surveyed by mail, whose response rates are lowest.
From this perspective, a key advantage of analyzing outcomes through administrative data such as hospital discharge information is that these data cover all of the individuals participating in the lottery. The disadvantage of the administrative data is that they do not span all of the outcomes of interest.
However, we can use the administrative data to estimate the extent of nonresponse bias in the survey data. For example, for outcomes in the administrative data that are similar to those in the survey data, such as hospital utilization, we can see if the effect of gaining access to insurance is the same for those who responded to the survey as for those who did not.
Second, considerable caution must be exercised in extrapolating from our estimates of the causal impact of insurance eligibility and coverage to other settings, including the recent federally mandated expansion of Medicaid. Our results are naturally most relevant for populations, insurance plans, health care environments, and enrollment mechanisms closest to those in our study.
In the remainder of this paper, we therefore provide more details on who signed up for the Oregon waiting list and who enrolled in the Standard plan if they were allowed to do so. We also discuss the health care environment. These facts shed light on the nature of the low-income population that sought insurance coverage and was eligible for it.
To find out what predisposed people to sign up for the waiting list and to see how representative our study population is, we examined data from the Current Population Survey (CPS), a monthly nationally representative household survey of the U.S. civilian, noninstitutionalized population, for the period 2004–6; the Centers for Disease Control and Prevention’s (CDC’s) Behavioral Risk Factor Surveillance System (BRFSS) for the period 2004–7; and our data from the list.
We compared characteristics of the low-income uninsured adult population of Oregon; the low-income uninsured population of the United States; those who signed up for the waiting list; and those from the list who responded to our initial mail and phone survey (Exhibit 1). It should be noted that some people in each of these groups may have been eligible for but not enrolled in public insurance.
Overall, the low-income uninsured population of Oregon—the target population for the waiting list—is roughly comparable to that of the United States as a whole. The most striking difference is that the Oregon population has more whites and substantially fewer African Americans.
Individuals in Oregon who joined the waiting list are somewhat older and appear to be sicker than the overall target population. For example, of those who signed up, about 30 percent report having been diagnosed with high blood pressure, 11 percent with diabetes, and 17 percent with asthma. Prior studies of people enrolled in the standard plan had very similar results.22 Thus, our survey respondents seem to be representative of the plan’s members in terms of their health status.
It is not surprising that the older and sicker members of the eligible population would be the ones who most wanted health insurance, and that they would thus be more likely to put their names on the waiting list. But that finding does suggest caution in extrapolating our findings on the impact of insurance to cases involving a healthier or richer population. Evidence from the RAND experiment and other studies suggests that the benefits of health insurance are concentrated among low-income individuals in poor health, such as those who signed up for the waiting list.15,23,24
Although still low-income, people on the list reported incomes that were much higher than those of the low-income uninsured populations of Oregon or the United States. In fact, only 61 percent of the people in our initial survey reported incomes below the federal poverty level.
This suggests that some of those who joined the waiting list may have been confused about the program. Only households with incomes below the federal poverty level are eligible for the Standard plan. Another possibility is that the incomes of this population are volatile enough that people were uncertain about their future eligibility.
Another relevant factor in assessing the generalizability of results from the Oregon Health Study is whether the environment in which this population receives health care is similar to that in other states.
Oregon is similar to the national average in how many of its hospital admissions are in public hospitals (just under 15 percent for both), and in uncompensated care as a fraction of gross hospital charges (6.8 percent for Oregon and 5.6 percent nationally).25,26 Oregon has 335 physicians per 100,000 residents, which is similar to the U.S. ratio of 330 per 100,000.27,28
The Behavioral Risk Factor Surveillance System data also show that low-income uninsured Oregonians have an access gap relative to their insured counterparts that is similar to that in the rest of the country. Together, this suggests that the consequences of gaining insurance coverage for low-income adults may be similar in Oregon and in the nation overall.
The phenomenon of low take-up rates—that is, low rates of participation—for social safety-net programs in the United States is widespread. The abundant research on possible causes has focused on the potential roles of stigma, transaction costs, and lack of information.29
Although there are limited data about the take-up of Medicaid among adults, some evidence suggests that take-up among newly eligible pregnant women and children is under 35 percent and that only 50 percent of eligible adults without private insurance are enrolled.8,30–32
There are several reasons why we might have expected higher-than-usual take-up in our setting. First, members of this population expressed some interest in coverage by joining the waiting list. Second, unlike standard Medicaid programs, Oregon Health Plan Standard did not give people the option to enroll only when they required medical care. People selected from the list had only forty-five days to apply for insurance; after this period, they were no longer eligible to apply.
Take-up is particularly interesting in this setting because one of the standard explanations for low Medicaid take-up is that individuals who are not enrolled are still “conditionally covered.” In other words, they do not sign up for coverage until they become ill or pregnant.30
Less than one-third of the 29,411 individuals selected from the list ended up enrolled in Oregon Health Plan Standard. Of those selected, 17,962 (61 percent) submitted applications, and 8,704 (30 percent) were approved for coverage.
Thus, there are two sources of slippage in take-up. Almost 40 percent of those selected did not apply, and about half of those who applied were ineligible.
We explored some of the reasons for these slippages using two data sources. First, our mail and phone survey asked people who had not returned their applications to indicate why. Second, when individuals did apply but were denied coverage, administrative data provided the reason for denial. Combining these sources, we can compare those who applied to those who did not, and those who were approved to those who were denied (Exhibit 2).
Of course, an important caveat here is that those who responded to our survey also appear—based on administrative data—to be more likely to apply and enroll than those who did not respond. Therefore, our survey respondents are not a random sample of those who did not enroll in insurance. In contrast, the administrative data contains all of the people on the waiting list, whether or not they applied or enrolled.
Not surprisingly, individuals who applied look like they would be greater consumers of health care than those who did not apply. They were three years older than those who did not apply, and, among survey respondents, they were more likely to report being in fair or poor health and having days impaired by poor health. Those who enrolled were older and in worse health than the general waiting list population, who in turn were older and sicker than the low-income uninsured in general.
Having income or assets above the limits of eligibility was a major factor in both sources of slippage. Applicants reported lower household income than nonapplicants, on average—$11,790 versus $15,064 (Exhibit 2). The same is true for those whose applications were approved compared to those who were denied coverage—$ 8,513 versus $15,255.
Of the survey respondents who could have applied but did not, 23 percent said that they believed their income or assets to be too high (Exhibit 3). And 55 percent of submitted applications were denied on the basis of excessive income or assets (Exhibit 4).
These results suggest that take-up among those who are truly eligible for the program is much higher than the raw 30 percent enrollment among those who were selected from the list. Calculations suggest that 19 percent of those selected had income or assets above the limits, while an additional 4 percent already had alternative insurance. This implies a corrected take-up rate among the eligible applicants of 39 percent, but this is probably an underestimate.
Those selected who did not return either surveys or applications probably had rates of ineligibility that were at least as high. Furthermore, almost 10 percent of the surveys that were mailed were returned because of bad addresses. A similar share of applications may have been undeliverable. These additional adjustments suggest a true take-up rate of about 50 percent among selected people who were eligible and received an application.
There are clearly still barriers to enrollment, even among those who are eligible. Despite efforts by the state to facilitate enrollment by those selected, paperwork requirements seem to pose a major hurdle. A third of those who submitted applications failed to return all of the necessary documentation in time (Exhibit 4). Likewise, a third of those who did not apply attributed their failure to not having completed the application, finding the paperwork a hassle, or not having the appropriate documentation (Exhibit 3).
Some of this shortfall may reflect actual ineligibility, but that is unlikely to explain all of it. There are indications that people who are more comfortable with paperwork or more motivated to get insurance coverage—for example, those who signed up for the list as soon as they could and who provided more complete contact information—were both more likely to apply and more likely to be approved (results not shown).
The imperfect take-up of Oregon Health Plan Standard coverage by those selected from the waiting list thus appears to be attributable both to ineligibility and to difficulty obtaining coverage for the truly eligible. This is consistent with findings from other studies that document the barriers posed by complicated eligibility rules. It also suggests that better information and streamlined processes can increase take-up.33
Understanding the determinants of insurance take-up helps put future results from the Oregon Health Study in context. Our analysis should be interpreted as estimating the effect of public health insurance on those who would obtain insurance when offered the opportunity: people who are likely to be older and sicker than those who would not. Caution must be exercised in extrapolating from our results to very different populations, or to very different types of insurance governed by different enrollment or eligibility rules.
The selection of names from the Oregon health insurance waiting list provides an exciting opportunity for research. It allows us to bring the strengths of randomized experiments to the study of the causal effects of expanding public health insurance in a low-income population with a wide variety of outcomes.
Both participation in the waiting list and take-up of the program were higher for people in worse health, who may be the most motivated to obtain insurance. In addition, as the evidence from the RAND experiment suggests,15 they may be the most likely to benefit from insurance.
Take-up of the coverage was fairly low, partly because not everyone on the waiting list was truly eligible. That so many people with income above the eligibility limit participated—and, if selected, applied for coverage—suggests that there is substantial unmet demand for health insurance among those just above the federal poverty level. These and future results from the Oregon Health Study are likely to continue to shed light on the large Medicaid expansion now scheduled to take effect nationwide in 2014.
This research was conducted in conjunction with an evaluation of Oregon’s Medicaid program, funded in part by the Centers for Medicare and Medicaid Services. The authors gratefully acknowledge funding from the Office of the Assistant Secretary for Planning and Evaluation in the Department of Health and Human Services, the California HealthCare Foundation, the John D. and Catherine T. MacArthur Foundation, the National Institute on Aging, the Robert Wood Johnson Foundation, the Sloan Foundation, and the Social Security Administration. In addition to the authors named above, the Oregon Health Study Group includes Mira Bernstein, Matthew Carlson, Tina Edlund, Charles Gallia, Jonathan Gruber, Joseph Newhouse, Eric Schneider, and Jeanene Smith. The authors thank Tatyana Deryugina, Ahmed Jaber, and Zirui Song for expert research assistance.
Heidi Allen, Center for Outcomes Research and Education, Providence Health System, in Portland, Oregon.
Katherine Baicker, Department of Health Policy and Management, School of Public Health, Harvard University, in Boston, Massachusetts.
Amy Finkelstein, Department of Economics at the Massachusetts Institute of Technology, in Cambridge.
Sarah Taubman, National Bureau of Economic Research, in Cambridge.
Bill J. Wright, Center for Outcomes Research and Education, Providence Health System.