|Home | About | Journals | Submit | Contact Us | Français|
Medicaid claims and eligibility data, particularly when linked to other sources of patient-level and contextual information, represent a powerful and under-used resource for health services research on the use and outcomes of prescription drugs. However, their effective use poses many methodological and inferential challenges. This article reviews strengths, limitations, challenges, and recommended strategies in using Medicaid data for research on the initiation, continuation, and outcomes of prescription drug therapies. Drawing from published research using Medicaid data by the investigators and other groups, we review several key validity and methodological issues. We discuss strategies for claims-based identification of diagnostic subgroups and procedures, measuring and modeling initiation and persistence of regimens, analysis of treatment disparities, and examination of comorbidity patterns. Based on this review, we discuss “best practices” for appropriate data use and validity checking, approaches to statistical modeling of longitudinal patterns in the presence of typical challenges, and strategies for strengthening the power and potential of Medicaid datasets. Finally, we discuss policy implications, including the potential for the research use of Medicare Part D data and the need for further initiatives to systematically develop and optimally use research datasets that link Medicaid and other sources of clinical and outcome information.
Medicaid data have great potential for examining patterns of medication use and outcomes, but pose many methodological difficulties. We discuss their strengths, challenges, and strategies to address these challenges, with illustrations from work by our group and others.
Medicaid files provide a unique and powerful perspective on key components of health care in the United States. Their strengths and comparative advantages are summarized in Table 1. A key strength is the very large number of individuals represented and corresponding statistical power for fine-grained analyses of important subgroups, rare conditions, complex patterns of comorbidity, and adverse events. The data are indispensable for studying health care for vulnerable subgroups such as minority, low-income and/or disabled individuals. They offer a detailed, longitudinal record of utilization, diagnoses, procedures, and prescriptions across the full range of health care settings. Whereas participants in randomized clinical trials (RCTs) of drugs sometimes fail to represent the full medical and social diversity of the populations who ultimately use them, Medicaid data include a broader population, not limited by clinical-trial exclusions and capturing off-label use. And although commercial-insurance databases typically include mainly individuals well enough to work and their dependents, Medicaid beneficiaries reflect a broad range of health status, including many with disabilities and severe chronic conditions.1–3
Medicaid is the predominant payer for low-income Americans and is crucial for people with disabilities. In 2006, excluding State Child Health Insurance Program (SCHIP) participants, an estimated 56.3 million individuals were enrolled,4 or almost 1 in 5 Americans, including 31.1 million children, 16.2 million nonelderly adults, 6.1 million elderly, and 9.7 million blind and disabled persons. Racial and ethnic minorities are strongly represented; racial/ethnic breakdowns for 2003 (the most recent available) include 11.7 million African Americans (almost 1 in 3 nationally) and 9.8 million Latinos (almost 1 in 4 nationally).5,6 Unlike typical commercial insurance databases, race and ethnicity are recorded for most enrollees. Thus, Medicaid files are particularly important for examining outcomes in diverse populations and identifying treatment disparities.
Because beneficiaries have access to a generally similar package of benefits, Medicaid data provide important insight into nonfinancial sources of disparities. Such studies are particularly valuable when they go beyond simple cross-sectional analyses to longitudinal modeling. Within-person analyses of duration and consistency of treatment spells can shed important light on processes and potentially modifiable factors leading to disparate health outcomes. For example, longitudinal analyses of antiretroviral medication use among Medicaid beneficiaries with HIV/AIDS found that racial/ethnic disparities in treatment included both later initiation of regimens and less consistent use after treatment initiation by African American and Hispanic beneficiaries, suggesting the need to examine nonfinancial barriers to adherence in long-term antiretroviral therapy, and implement better strategies for supporting patients in remaining on these regimens consistently.7
Data on treatments and diagnoses come from providers, avoiding self-report and nonresponse biases that are issues in interview-based studies.8 Difficult-to-interview subgroups (eg, those with neurologic, psychiatric, or cognitive incapacities) are fully included. Medicaid is a central payer for persons with mental illness or substance abuse and an essential source for studying their care. Very large datasets can be obtained and updated for a fraction of the cost of comparably sized studies involving primary data collection, providing continuous, longitudinal information on healthcare encounters, their accompanying recorded diagnoses and procedures, and filled prescriptions. Detailed information on the dates of events can be used to construct episodes of care, support close examination of the timing of events in relation to treatment initiation, and facilitate use of event history methods to model hazards of outcomes such as hospitalizations.9 The data support studies of off-label use and outcomes for types of patients excluded from clinical trials.
Power for studying low-prevalence conditions and outcomes is a key comparative advantage. Examples include studies of multiple-diagnosed subgroups (eg, HIV/AIDS patients with mental retardation)10 and uncommon, severe medication outcomes that premarketing trials have insufficient power to capture.11 Use of data from multiple states increases power for such analyses. For example, Olfson and colleagues12 used 50-state Medicaid data in a case–control study of risks of suicide attempts and suicide deaths in severely depressed children and adults; in children and adolescents, antidepressant drug treatment was associated with both outcomes. The statistical power of the 50-state dataset was key to detecting associations between treatment and these infrequent events.
Selecting optimal strategies for research using Medicaid claims involves assessing data quality, understanding limitations, identifying potential sources of error, and adopting effective methods for restricting their impact. Several key challenges are summarized in Table 2. Ideal solutions remain elusive for some problems, and no set of rules can be applied mechanically, but an increasing body of research has demonstrated strategies for addressing these challenges. The International Society for Pharmacoeconomics and Outcomes Research’s Task Force on Retrospective Databases provides a user-friendly checklist for decision makers considering this sort of research.13 Key first steps are thoughtful selection of research questions based on extensive familiarity with both national and state Medicaid program characteristics, and understanding the institutional processes that determine patient eligibility and produce and process claims.
Table 2 presents, for each major challenge, relevant strategies and best practices. A central limitation with many implications is that Medicaid data are collected for administrative rather than research purposes. Thus, understanding nonclinical influences in coding processes, such as coverage and reimbursement considerations, is essential. Claims represent raw data that must be organized into meaningful diagnostic and treatment variables, whose validity and sensitivity to alternative methodological choices must be carefully considered before analysis can proceed. Care and thoroughness in this process, informed by a clear understanding of relevant clinical, administrative, and coding issues, is critical to effective inference.
Moving from these broad issues to more specific analytic challenges, several that occur in many if not most Medicaid studies are highlighted in Table 2. Especially important, as discussed below, are assessing completeness of information on health care encounters; identification of relevant clinical subgroups based on diagnostic information in claims; choice and validity of measures of medication use based on filled-prescription data; and selection of statistical modeling techniques appropriate to the structure of the data.
As with other claims datasets, information on health care encounters, prescription drug utilization and diagnoses in Medicaid files is based on provider report. Although recall and social desirability biases characteristic of self-report data are avoided,14 conditions that have not come to medical attention are not identified and provider-recorded diagnoses may reflect nonmedical considerations such as reimbursement policies and rates. It is also important to assess the possibility of receipt of services that do not appear in Medicaid claims files and select inclusion criteria that minimize the potential for incomplete data. For example, state psychiatric hospitals do not bill Medicaid for most recipients utilizing their care. Beneficiaries with dual Medicare-Medicaid eligibility and participants in capitated managed care programs may receive services not reflected in Medicaid claims files. Although claims files in some states include “pseudo-claims” for services to capitated beneficiaries, such data cannot be assumed to be complete without validation studies supporting their usability. For dual-eligibles, although many services eligible for Medicare funding will generate a Medicaid copayment, some may not.15 Although matching with Medicare files to retrieve corresponding Medicare claims is a complex procedure, such matching is the best way to assure completeness of care histories in studies that include dual-eligibles.
Diagnostic classification of beneficiaries is a critical foundation for subsequent analyses. Although diagnostic codes in claims serve as a useful proxy for underlying medical conditions, it is helpful to think of them as representing a provider behavior that is also influenced by other factors. Thus, trends in the reported rates of conditions may reflect changes in provider-level patterns of recognition and labeling of conditions, as opposed to true changes in underlying prevalence. For example, analyzing Medicare claims data, Crystal and colleagues16 found that the proportion of elderly beneficiaries diagnosed with depression doubled between 1992 and 1998, likely representing a shift in diagnosing behavior rather than in underlying depression.
Validity of detection algorithms in claims data for diagnostic groups17 and procedures18 can be examined with a 2-by-2 framework familiar from the medical screening literature. Comparisons between algorithm-based decision rules and some specified “gold standard” produce true positives, false positives, true negatives, and false negatives. Sensitivity represents the algorithm’s effectiveness in classifying diagnosis-positive clients (based on the external standard) as diagnosis positive (based on the claims). Specificity is the detection algorithm’s capacity to classify diagnosis-negative clients as diagnosis negative.
As an illustration of application of this framework to validation of claims-based diagnostic measures, sensitivity of HIV/AIDS case identification was examined using New Jersey Medicaid data, with the state’s HIV/AIDS Registry as the “gold standard.”19 Building on work by Keyes and others,20–22 claims-based algorithms for identifying AIDS cases were applied to Medicaid data and compared with those constructed by matching to the Registry. An algorithm using nonpharmacy claims produced 88% sensitivity; adding pharmacy claims produced 95% sensitivity.
Similarly, validation studies have been conducted for claims-based measures of psychiatric conditions. Agreement with medical records has been found to be high for more serious disorders, such as schizophrenia,23 but lower for conditions such as minor depression.24 Lurie and colleagues25 tested an algorithm using 2 years of Medicaid claims, classifying a patient as having schizophrenia who had 1 inpatient or 2 outpatient encounters with this diagnosis. Results were compared with an external standard using psychiatrist ratings of symptom profiles, producing an estimated specificity of 87% for the claims-based algorithm.
Prescription fill data are widely used to examine duration and consistency of medication use spells. Such claims-based measures can be used in conjunction with, or in place of, other adherence measures such as self-reports, pill counts, electronic monitoring, and laboratory results. Measures of refill persistence have been shown to correlate with these other more costly and intrusive measures. Unlike self-reports, refill persistence measures are not affected by inaccurate or biased recall, and have been shown to predict clinical outcomes.7 Grymonpre et al26 reported strong concordance between claims-derived measures of medication availability and pill counts. Nachega et al27 found a significant relationship between a claims-based measure of antiretroviral adherence and survival among South African adults with HIV. In interpreting results, however, it is important to understand persistence measures as reflecting drug availability to the patient, not actual drug-taking behavior. Filled prescriptions, representing a necessary, though not sufficient, condition for drug ingestion, indicate a lower bound for adherence. Also, pharmacy claims will not reflect drug samples obtained from physicians’ offices, drugs received while hospitalized, over-the-counter drugs, and nonformulary drugs purchased out-of-pocket.28
Many measures of refill persistence have been proposed, most based on fill dates and days supplied. Measures can be distinguished on several dimensions including continuous versus dichotomous measures, single or multiple observation intervals, and definition of the “denominator” time period for calculation of medication availability (eg, time from initiation to last filled prescription vs. time from initiation to end of observation period).29 Results using alternative measures seem to vary mainly by choice of denominator rather than other characteristics. In one recent study, 5 measures using total study evaluation period in the denominator produced almost identical results.30
Among available measures, there seems to be no single accepted standard; choices are best made in light of the subpopulations and clinical issues at hand. Clinical and treatment standards considerations will help rule out certain measures and recommend others. In some instances, adequate care would involve continuous treatment of an indefinite period after therapy initiation. In studies of HIV treatment, for example, one would usually expect to observe consistent, indefinite receipt of antiretroviral therapy once started, although specific drugs in the regimen might change. In such a situation, therefore, it would be appropriate to treat both temporary gaps and discontinuation as suboptimal, and examine medication possession from treatment initiation to the last date of observation, as applied in antiretroviral persistence studies among New Jersey Medicaid beneficiaries.31
Clinical circumstances also inform choice of continuous versus binary persistence measures. In HIV disease, strict adherence is necessary to achieve viral suppression and prevent viral resistance, so a binary measure of “strict adherence” would be appropriate. In other circumstances (eg, antipsychotic therapy among beneficiaries with schizophrenia), there may be no clear “threshold” of adherence for effectiveness, and the challenge for quality may be a matter of moving the entire distribution upward. In such cases, continuous measures may be best.32
Clinical guidelines may provide benchmarks for adequate treatment duration. For example, Health and Employer Data and Information Set (HEDIS)33 standards specify minimum duration of antidepressant treatment of acute and continuation phases of care. Although usually used by managed care organizations, such measures can also provide an episode-of-care benchmark for fee-for-service Medicaid data.
Construction of a refill persistence measure must also take into account the complexity of many treatment regimens, including medication switching, polypharmacy, and introduction and diffusion of new drugs. For example, claims histories for persons with bipolar disorder may include changes from lithium to anticonvulsants (ie, gabapentin) and may be further complicated by diagnosis-related concerns such as early mis-diagnosis of major depressive disorder and concurrent diagnosis of a seizure disorder. Clinically informed analytic choices are important to distinguish periods of appropriate treatment from nontreatment, and are a necessary condition for meaningful interpretation of findings. Thus, researchers should be wary of simply selecting a single measure and mechanically applying the required numbers. Use of sensitivity analyses to examine robustness of findings across alternative measures will strengthen confidence in results.
A key challenge in analyzing outcomes is to address possible sources of confounding due to unobserved clinical differences. It is important to understand patient-level and other sources of variation in treatment and to make optimal use of clinical information that does exist in diagnostic histories in the claims files. As in smaller-scale studies, indices such as the Charlson Comorbidity Index are often used to control for co-occurring conditions.34,35 However, such indices may under-adjust, as they force weights on the various coexisting conditions that may not be suited to the research question at hand. With the extensive statistical power of most claims-based studies, conservation of degrees of freedom may not be an important consideration and it may be appropriate to adjust separately for each relevant co-occurring condition. Propensity score matching (PSM) is gaining popularity to partially address the problem of selection bias into treatment groups, but researchers should understand that biases may remain due to unobserved differences between groups being compared. Another adjustment strategy well-suited to Medicaid claims data involves reducing heterogeneity by focusing on restricted subgroups. For example, in their study of the association between suicidality and antidepressant use, Olfson and colleagues12 limited their analyses to individuals who had previously experienced a hospitalization for depression.
Improvements in statistical and econometric techniques also have promise for more effective and appropriate use of Medicaid claims data. There has been much interest in instrumental variables approaches to address bias due to unobserved clinical factors motivating prescription of treatments (“confounding by indication”).36 Although it is often difficult to identify suitable instruments for this type of analysis (ie, high correlation with treatment but not with the outcome), there is often enormous variability in prescription drug use by Medicaid populations across time, geographical location, and provider that stems from causes that are exogenous with respect to individuals’ clinical characteristics, and may offer the potential for such analyses. For example, provider-level measures of treatment patterns that can be calculated from Medicaid data are probably less subject to confounding by indication than are patient-level measures of treatment. Brookhart and colleagues37 used physicians’ treatment preferences, operationalized as the treatment received by the previous patient with the same condition seen by the physician, as an instrumental variable for exposure to particular prescription drug treatments. Similarly, treatment variations across space (geographical area) and time can be exploited in analyses as relatively exogenous sources of treatment variation.
Changes in formularies, cost-sharing, and other policies also offer opportunities to examine the consequences of natural experiments in Medicaid. For example, Soumerai and colleagues38 examined effects of a 3 prescription monthly cap on noninstitutionalized patients with schizophrenia. Although medication use was reduced, clinic visits and ER use increased. Medicaid data on dual-eligibles also provide a baseline for future studies of the effects of Medicare Part D implementation.
Finally, problems of data censoring are a common challenge in longitudinal analyses of Medicaid files. For example, in analyzing duration of treatment spells, discontinuation often does not take place by the end of the observation period. Survival analysis methods are an appropriate tool in such situations. Kaplan–Meier estimation can be used to examine distribution of treatment duration and bivariate differences, whereas proportional hazards (Cox) regression can be used to examine effects of multiple covariates on time-to-discontinuation. Akincigil et al39 used these models to examine time-to-discontinuation of beta-blocker and angiotensin converting enzyme (ACE) inhibitor therapy after acute myocardial infarction, finding substantial socioeconomic disparities.
An essential foundation for policy development and quality improvement is improved understanding of the ways in which prescription drugs are used in routine clinical care, which often differs widely from populations and diagnoses in premarketing trials. Despite practical challenges such as time lags in data availability, Medicaid data are particularly well-suited to monitoring rates, predictors, and other aspects of use, including disparities, dosages, durations and associated diagnoses. For example, across a range of chronic health conditions, claims-based studies have frequently documented patterns of inconsistent use or early discontinuation of therapies that represent major missed public health opportunities.7,16,31,32,39,40 Similarly, such studies have documented racial/ethnic disparities in care, even when financial coverage is comparable, providing insight into actionable intervening variables that could reduce disparities,41,42 and have received substantial attention in policy documents such as the National Healthcare Disparities Report.43
With respect to treatment outcomes and drug safety, many methodological challenges present themselves. Nevertheless, Medicaid data offer an extremely important resource for such studies. Although RCTs remain the gold standard, such data are often unavailable for a host of questions that are critically important to better-informed therapeutic decision making and improved safety and treatment outcomes. Even if there is a sharp increase in national investment in needed RCTs, such studies will never be able to answer every important question about outcomes for every beneficiary subgroup, condition, and drug or drug combination, given the many feasibility, economic, ethical, duration, and other constraints on RCT implementation. Absence of needed RCT data is particularly marked for low-income, minority, elderly and disabled populations, often under-represented in RCTs. Medicaid data provide the power to examine use and outcomes in these and other important subpopulations.
To make more effective use of the considerable potential for research on pharmaceutical use and outcomes among Medicaid beneficiaries, including dual-eligibles, several public policy initiatives merit consideration. Many studies to date have been conducted on an ad hoc basis, with datasets assembled to address a single research topic. However, effective use of these powerful but large and complex datasets is labor-intensive, requiring long-term cumulative experience, considerable investment in database development and management, linkage across multiple data sources, documentation, sensitivity analyses, and procedures for data security and rigorous protection of confidentiality, among other steps.
Optimally, costs of such investments would be amortized over multiple studies rather than on a single-study basis. These analyses require multiple types of expertise including knowledge of the conditions, populations and drugs at issue, detailed understanding of the programs and data systems, and advanced statistical and modeling methods; thus, infrastructure is needed to support and sustain multidisciplinary teams. Similarly, there is need for an array of validation studies, such as comparisons of the performance of claims-based algorithms for identification of populations with particular conditions to external criteria based on expert clinical assessment. Such studies have typically been conducted as byproducts of substantive investigations, but they deserve encouragement and funding in their own right. Thus, a more systematic program of support is needed for development and linking of administrative databases, validation studies, and utilization of these data resources to address use and outcomes of prescription drug therapy, particularly for low-income, disabled, and elderly individuals reliant on publicly-funded coverage. Such a program could usefully incorporate systematic initiatives for linkages of claims data to other sources of data on beneficiaries’ health status and outcomes, such as disease registries, surveys, electronic medical record data, and birth and death records. The National Cancer Institute has linked its Surveillance, Epidemiology and End Results (SEER) cancer registry data with Medicare claims, creating a merged database that has been a productive research tool. This represents a promising model that could be further strengthened by linkage to data on prescription drugs beyond those administered in physicians’ offices, such as Part D data. More systematic development of research datasets that link Medicaid data to other data files providing clinical and outcome data for beneficiaries will expand the potential of research with these datasets and the ability to address problems of confounding; this is a key strategy for reducing missing-variable bias and improving measurement of outcomes. For example, the potential of data linkages between Medicaid claims and vital-statistics files (eg, birth certificates) has been illustrated in important research by Cooper et al11 on medication effects on birth-defect rates, which prompted a policy response in the form of an FDA safety alert.44
A related concern for the future is the need for systematic data merging, warehousing, and access initiatives to address fragmentation of data for beneficiaries of public coverage that has been a by-product of initiatives aimed at reform. These changes include expansion of capitated Medicaid managed care programs, managed care carve-outs for specific services, such as behavioral health, and the advent of Medicare Part D. Under the Medicare Modernization Act (MMA; Pub. L. 108–173), on January 1, 2006, responsibility for prescription drug benefits for individuals dually eligible for Medicare and Medicaid was shifted from state Medicaid programs to privately-administered Medicare Prescription Drug Plans (PDPs). These individuals include elderly beneficiaries receiving Medicaid, as well as many persons with disabilities. Thus, future studies of prescription drug use and outcomes for elderly and disabled individuals reliant on publicly funded health coverage will need to access prescription drug claims data from the PDPs and merge them with other sources of data, such as Medicare Part A and B claims, and Medicaid claims for non-Medicare-covered services. Such studies will need to reintegrate now-fragmented data from multiple databases on the same individuals, while carefully protecting beneficiary confidentiality. Under MMA, PDPs provide claims-level data to the Centers for Medicare and Medicaid Services (CMS) that are used for tracking out-of-pocket costs and other purposes. These data have the potential to serve as the foundation for a powerful national research database on prescription drug use and outcomes by the elderly and disabled. However, by mid-2007, procedures for research access to Part D data were still a work in progress, and it may be a long and complex road to achieve their great research potential.
Although much more needs to be done, the existing body of work using Medicaid data to examine prescription drug use, although modest in volume, has already done much to demonstrate their power to address critical questions of prescription drug use and safety. For example, the Presidential New Freedom Commission on Mental Health made extensive use of Medicaid-based studies.45 Given its growing financial costs ($315 billion in state and federal expenditures in 2005) and public health importance as the payer for almost 1 in 5 Americans, Medicaid is a chronic subject of intense policy debate,46 and it is important that issues of utilization, quality and outcomes of care in Medicaid be adequately addressed with available data as a basis for rational discussion of policy choices. Medicaid data resources represent an important opportunity for needed studies in coming years. Further development of merged research datasets that link claims with other sources of clinical and outcome information; systematic efforts to support and disseminate validation studies; thoughtful exploitation of natural experiments; careful selection of research questions to draw on the data’s strengths while avoiding threats to validity; and further development of statistical and analytic methods to address potential sources of bias are among the tools that will help to realize this potential.
Supported by the Agency for Healthcare Research and Quality through a cooperative agreement for the Center for Research and Education on Mental Health Therapeutics at Rutgers (HS016097), as part of AHRQ’s Centers for Education and Research on Therapeutics (CERTs) program; with additional support from NIMH grants MH058984, MH60831, and MH076206; and AHRQ grant HS011825.
An earlier version was presented at the AHRQ-sponsored conference on Comparative Effectiveness and Safety: Emerging Methods Symposium, June 19–20, 2006, Rockville, MD.