|Home | About | Journals | Submit | Contact Us | Français|
Under Medicare Part D, patient characteristics influence plan choice, which in turn influences Part D coverage gap entry. We compared pre-defined propensity score (PS) and high-dimensional propensity score (hdPS) approaches to address such ‘confounding by health system use’ in assessing whether coverage gap entry is associated with cardiovascular events or death.
We followed 243,079 Medicare patients aged 65+ with linked prescription, medical, and plan-specific data in 2005–2007. Patients reached the coverage gap and were followed until an event or year’s end. Exposed patients were responsible for drug costs in the gap; unexposed patients (patients with non-Part D drug insurance and Part D patients receiving a low-income subsidy (LIS)) received financial assistance. Exposed patients were 1:1 PS- or hdPS-matched to unexposed patients. The PS model included 52 predefined covariates; the hdPS model added 400 empirically identified covariates. Hazard ratios for death and any of five cardiovascular outcomes were compared. In sensitivity analyses, we explored residual confounding using only LIS patients in the unexposed group.
In unadjusted analyses, exposed patients had no greater hazard of death (HR=1.00; 95% CI, 0.84–1.20) or other outcomes. PS- (HR=1.29;0.99–1.66) and hdPS- (HR=1.11;0.86–1.42) matched analyses showed elevated but non-significant hazards of death. In sensitivity analyses, the PS analysis showed a protective effect (HR=0.78;0.61–0.98), while the hdPS analysis (HR=1.06;0.82–1.37) confirmed the main hdPS findings.
Although the PS-matched analysis suggested elevated though non-significant hazards of death among patients with no financial assistance during the gap, the hdPS analysis produced lower estimates that were stable across sensitivity analyses.
Assessing whether changes in insurance coverage affect patients’ health outcomes is difficult due to complex relations between patients and their benefits, a phenomenon similar to confounding by indication in drug safety or effectiveness studies.1 One example of such “confounding by health system use” is apparent in the context of patients’ experiences after reaching the Medicare Part D coverage gap, a period during which some patients are responsible for 100% of drug costs, while other patients with low socioeconomic status and/or poor health receive financial assistance (known as the low-income subsidy), thus avoiding an interruption in benefits. Because a patient’s demographic and health characteristics influence both the Part D plan in which he enrolls, and in turn, both patient and plan characteristics influence post-enrollment drug use and whether the patient reaches the coverage gap spending threshold and is exposed,2 confounding by health system use is present.
Propensity score (PS) techniques are a commonly used approach to mitigate confounding,3, 4 particularly in the setting of studies that use large claims databases. In an predefined PS (PS) approach, patient- and plan-specific covariates are defined a priori and included in the PS model based on subject matter knowledge and the strength of their association with exposure and outcome.5 However, residual confounding by covariates not included in the PS model may remain,6–8 and identification of additional confounders can be difficult in the context of confounding by health systems use.9 An alternative approach is the use of a high-dimensional PS (hdPS). Unlike the PS, a hdPS allows for the empirical identification of potential confounders based on their prevalence, frequency, and potential to cause bias.9 This technique takes advantage of the availability of rich data from multiple data dimensions (e.g., inpatient diagnoses, demographics, and prescription drug use) to identify additional, previously unconsidered covariates to minimize residual confounding.
In a prior study,10 we described an approximately twofold increased rate of drug discontinuation after Part D coverage gap entry among patients responsible for all drug costs versus those who have no gap in coverage, prompting concerns that the likelihood of adverse health outcomes might also increase. Evaluating whether the risk of adverse health outcomes differs between these two patient groups requires careful consideration of confounding by health system use. In the present study, we compare PS and hdPS approaches to assess the likelihood of adverse outcomes as a result of Part D coverage gap entry and subsequent discontinuation of medications. These analyses are conducted in the same population, with linked demographic, medical, prescription drug, vital status, and insurance plan data, which were used to study the likelihood of drug discontinuation after Part D coverage gap entry. This study addresses not only questions regarding the coverage gap’s health effects but also which methods may be most useful in adjusting for confounding by health system use in health policy studies.
We conducted two prospective open cohort studies using Medicare patients eligible in 2005–2006 (Early Part D Cohort) and 2006–2007 (Established Part D Cohort). Patients entered the cohort on the date they reached the coverage gap spending threshold, defined as combined plan and beneficiary out-of-pocket spending of $2,250 in 2006 and $2,400 in 2007. All patients, regardless of whether they received financial assistance during the gap, were censored on the earliest date of an outcome of interest, reaching the catastrophic coverage spending threshold (at which point drug insurance coverage resumes for patients without financial assistance in the gap) or study year’s end.
Community-dwelling, fee-for-service Medicare patients enrolled in a Part D or retiree (i.e., non-Part D, employer-sponsored) drug plan administered by a pharmacy benefits management company (PBM) were eligible for the study. Patients’ PBM prescription drug claims were linked to Medicare Parts A, B, skilled nursing facility, home health, hospice, durable medical equipment, demographic, and vital status data. Plan-specific data regarding drug cost-sharing and financial assistance during the coverage gap were linked for each Part D enrolled patient. Part D plans were characterized as providing no or generic-only drug coverage during the gap. None of the retiree plans had a coverage gap feature.
We established two cohorts of patients age ≥65. Both cohorts had Medicare eligibility and ≥1 inpatient or outpatient healthcare claim in both the baseline and study years. We further limited both cohorts to patients with ≥1 inpatient or outpatient cardiovascular diagnosis (hyperlipidemia, hypertension, atrial fibrillation, congestive heart failure, or cardiovascular disease; codes in the Appendix) during the baseline year and/or 2 months prior to reaching the coverage gap spending threshold and/or receipt of a cardiovascular drug within the 2 months prior to reaching the threshold and to those with no nursing home or hospice admissions from January 1 of the baseline year until reaching the coverage gap spending threshold.
We hypothesized that a patient’s plan enrollment and subsequent drug utilization would be good predictors of whether he would reach the coverage gap spending threshold; however, baseline year drug use was not available for the Early Part D cohort. To ensure comparable drug data from both cohorts, we limited our cohorts to patients who reached the threshold ≥ 60 days after plan enrollment.
Plan enrollment and patients’ out-of-pocket spending were used to define patients’ exposure, described previously.10 Briefly, Part D patients who did not receive a low-income subsidy were responsible for all drug costs in the coverage gap and were considered exposed. Part D plan enrollees with financial assistance for generic but not branded drugs during the coverage gap were classified as exposed. Two patient groups received financial assistance during the coverage gap and were combined and considered unexposed: Part D enrolled patients who received a low-income subsidy (LIS), and retirees enrolled in employer-sponsored (non-Part D) plans (retirees), none of which had a coverage gap.
Because retirees have employer-based drug insurance, they may also be more likely to carry supplemental non-Medicare health insurance. Claims paid by supplemental insurance would not appear in our linked Medicare Parts A and B data, yet our confounding adjustment for health service use depends on these claims. Retirees would appear falsely healthier than the exposed. Therefore, in sensitivity analyses, we use an unexposed comparator group composed entirely of LIS patients, who are similar to the exposed in their need for Part D drug insurance and the likelihood of having no supplemental insurance available.
Our primary outcome was death from any cause. Additionally, we assessed rates of first hospitalization with a primary or secondary diagnosis code for acute coronary syndrome with revascularization; congestive heart failure; and atrial fibrillation as well as rates for two composite outcomes, death or hospitalization for MI or stroke; and hospitalization for MI or stroke. Codes are in the Appendix.
A patient’s demographic and health characteristics influence both the Part D plan in which he enrolls (or in the case of patients dually eligible for Medicare and Medicaid, is assigned), and in turn, both patient and plan characteristics influence post-enrollment drug use and whether the patient reaches the coverage gap spending threshold and is exposed.2 Thus, there are two time periods during which confounding by health systems use needs to be addressed for covariates whose values may change over time. We used the 12 months of the baseline year to assess covariate information prior to Part D enrollment. To measure confounding in the post-enrollment period (post-January 1 of the study year), we again assessed covariates in the 2 months prior to reaching the coverage gap spending threshold. Information assessed during this 2-month “trigger period” is most likely to be associated with reaching the coverage gap spending threshold and entering the study as well as with outcomes. These two time periods combined constitute our primary covariate assessment strategy.
We used two approaches to address confounding by health systems use: a PS and an hdPS approach. Both the PS and hdPS models predicted each patient’s propensity to receive financial assistance to pay for drug costs upon reaching the coverage gap spending threshold. PS model covariates included age, gender, race, region of the U.S., rural/urban residence, median household income from census block data,11 time (in days) from plan enrollment to reaching the gap spending threshold, and total Medicare Parts A and B spending in the baseline year. The PS also incorporated diagnostic/health services covariates, assessed in the baseline year and again in the 2 months prior to cohort entry, including diagnosis of dementia, cancer, COPD/emphysema, renal failure, end-stage renal disease, depression, HIV/AIDS, diabetes, atrial fibrillation, hypercholesterolemia, hypertension, coronary artery disease, congestive heart failure, stroke, venous thromboembolism, myocardial infarction, and/or ACS + revascularization, Charlson comorbidity score,14 and number of office-based drug infusions, physician visits, and hospitalizations. Plan- and drug-based covariates, assessed in the 2 months prior to cohort entry, included number of unique drugs used12 and total [plan (i.e., all non-beneficiary sources) + beneficiary out-of-pocket] drug spending. Codes are in the Appendix.
To implement the hdPS approach, we first identified 10 domains from which covariates might be selected: drug use; plan cost-sharing details; and inpatient, outpatient, ambulatory, and skilled nursing facility diagnoses and procedures. Candidate covariates from these domains were ranked by prevalence, recurrence, and potential to cause bias, as described by Schneeweiss et al.9 Due to the likelihood of rare outcomes, we assessed each variable’s potential for confounding based solely on its association with exposure and screened candidate covariates so as to exclude covariates that were potential instruments.9, 13 We included the first 400 of these empirically identified covariates, a conservative approach based on prior studies that showed maximal confounding adjustment with 300, but selected covariates from fewer domains.13 These 400 empirically identified covariates, together with all PS model covariates, were included in the final hdPS model, run using the High Dimensional Pharmacoepidemiology SAS macro.14
Matching techniques using the PS or hdPS are likely to balance measured covariate distributions between the exposed and unexposed groups.15, 16 In our analyses, for each cohort, exposed patients were each matched to an unexposed patient using greedy matching with a matching caliper of ±0.05 using the untransformed PS.17 Exposed patients who could not be matched were removed from analyses.
After testing for effect measure modification by cohort using a Wald test and finding none, all analyses were conducted in the pooled cohort. Beneficiaries’ baseline characteristics were cross-tabulated by exposure status. We assessed the performance of PS and hdPS matching by comparing the Mahalanobis distance18 between the mean PS covariates in the matched groups. We used Cox proportional hazards models to estimate hazard ratios of death and each of the cardiovascular outcomes with no additional covariate adjustment or stratification for matched pairs 19, 20 Robust standard errors were used to adjust for the correlation among beneficiaries present in both cohorts.21 The Human Subjects Committee at Brigham and Women’s Hospital approved this study. Data use agreements were in place with all data providers.
Among patients who reached the gap spending threshold, 3% received no financial assistance for drugs during the coverage gap and were considered exposed (Table 1). At least 85% of patients had hypertension, ≥38% had coronary artery disease, and ≥24% had congestive heart failure. Patients in unexposed groups had fewer physician visits but used more drugs than exposed patients. These differences highlight imbalances in covariate distributions between groups. After matching, characteristics were largely balanced across the exposed and unexposed groups (Table 2). Of 8,103 exposed, 8,054 (99%) could be PS-matched and 7,984 (99%) could be hdPS-matched to an unexposed patient. In the sensitivity analyses, 7,479 (92%) could be PS-matched and 6,982 (86%) could be hdPS-matched (data not shown).
Patients experienced few events. In the PS-matched pooled cohort, the exposed death rate was 1.25-fold that of the unexposed, 5 versus 4 deaths/100 person-years (Table 3), and the rate of myocardial infarction, stroke, or death was 1.20-fold that of the unexposed.
In the crude analysis, exposed patients had an equal hazard of death during the coverage gap period compared with unexposed patients. In the main analysis, exposed patients had an elevated but non-significant hazard of death, PS-matched HR= 1.29 (95% CI, 0.99–1.66) and hdPS-matched HR=1.11 (0.86–1.42) (Table 4). In contrast, the PS-matched sensitivity analysis found that exposed patients were less likely to die, HR=0.78 (0.61–0.98). The hdPS-matched sensitivity analysis was consistent with the main analysis, HR=1.06 (0.82–1.37). Across all analyses, exposed patients had no greater hazards of any other cardiovascular outcome. In comparison to the PS-matched approach, the hdPS-matched approach had a slightly smaller Mahalanobis distance in the main analysis, suggesting less residual bias.22 The Mahalanobis distances in the PS- and hdPS-matched sensitivity analyses were nearly identical.
In this study of death and cardiovascular health outcomes among patients who reached the Part D coverage gap spending threshold, having no financial assistance to pay for drugs in the coverage gap was associated with no greater likelihood of death or other cardiovascular outcomes during the coverage gap period. A comparison of confounding adjustment approaches suggested that the hdPS approach produced clinically plausible, stable, and smaller effect estimates.23–28
Although the PS included 52 pre-defined covariates, the PS-matched results showed a marginally non-significant 29% increased risk of death, results that lack clinical plausibility (few drugs are associated with an even 25% reduction in 2-year mortality among patients who already had a cardiovascular event, so discontinuing drugs was unlikely to increase mortality risk by 29% in the short-term).29 In contrast, the hdPS approach empirically identified 400 additional potential confounders, all drawn from an existing database of rich patient and plan data, but not previously considered based on subject matter knowledge. The hdPS-matched results attenuated effects, producing an elevated but non-significant 11% increased risk of death. This effect estimate is consistent with effects observed in previous studies of the impact of drug discontinuation on rates of short-term cardiovascular events.23–28
In sensitivity analyses, the PS-matched approach found a counter-intuitive 22% reduced hazard of death, suggesting residual confounding by factors not considered a priori in the PS. Alternatively, the difference in PS-matched effect estimates for death in the main versus sensitivity analysis may reflect the potentially increased likelihood of missing claims among retirees as compared to the exposed. However, one would expect that the hdPS-matched sensitivity analysis would similarly highlight the effect of these missing claims. This is not the case: hdPS-matched estimates are stable in magnitude and direction across the main and sensitivity analyses. Residual confounding is a more likely, though not certain, explanation.
Our comparison of matching approaches reveals the tradeoff between bias and precision across results.30, 31 As compared to a 92% matching rate in the PS sensitivity analysis, the hdPS approach matched only 86%. The finer confounding control achieved by the hdPS resulted in fewer patients that were highly comparable based on their measured characteristics, reducing the precision of the estimates.30, 32 However, using the hdPS instead of the PS resulted in a loss of only 8% of exposed patients, which we consider to be a small price for improved validity.30, 31 Another potential limitation is in the definition of our exposure date, reaching the coverage gap spending threshold. While some Part D plans set higher spending thresholds, we chose to use the standard Part D benefit threshold to define a consistent exposure definition for Part D and non-Part D patients alike. To the extent that exposed patients entered the study early, even though they still had financial assistance to pay for drugs, our results are conservative. Based on Part D plan characteristics data,33 we believe this misclassification to be minimal.
As linked databases of patient- and plan-specific data become more ubiquitous, the use of approaches like hdPS can offer advantages to improve validity, particularly when there are complex relations between patients and plans that lead to confounding by health systems use and investigators are uncertain about the relevant confounders. Although both the PS- and hdPS-adjusted analyses suggested elevated, though non-significant, risks of death among patients who have no financial assistance to pay for drugs during the coverage gap period, the hdPS analyses produced results that were more clinically plausible and stable across sensitivity analyses. Thus, the use of hdPS may improve the quality of information available to policymakers when confounding by health system use is present.
Funding support: National Institute on Aging T32 AG000158 (Dr. Polinski); National Institute of Mental Health R01 5U01MH079175-02 (Dr. Schneeweiss); Agency for Healthcare Research and Quality K01 HS018088 (Dr. Rassen); and a research grant from CVS Caremark (Dr. Schneeweiss). The sponsors had no role in the design or conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.
Conflict of interest statement: Dr. Polinski is a consultant to Buccaneer Computer Systems and Service, Inc., a contractor for the Centers for Medicare and Medicaid Services. Within the past 5 years, Dr. Polinski’s spouse was employed as an engineer by DePuy Orthopaedics, a subsidiary of Johnson & Johnson, and had Johnson & Johnson stock totaling < $3100 in value. Dr. Schneeweiss was a paid member of the Scientific Advisory Board of HealthCore and is a consultant to World Health Information Science Consultants, LLC. Dr. Schneeweiss is Principal Investigator of the Brigham and Women’s Hospital DEcIDE Center on Comparative Effectiveness Research funded by AHRQ and the DEcIDE Methods Center. Dr. Schneeweiss received funding through investigator-initiated grants awarded to his employer, Brigham and Women’s Hospital, from Pfizer, Novartis, and Boehringer-Ingelheim. Dr. Glynn has worked on grants to the Brigham & Women’s Hospital, his employer, from Astra Zeneca and Novartis related to the design, statistical monitoring, and analysis of clinical trials in the setting of cardiovascular drugs. Dr. Glynn has worked on grants to the Brigham & Women’s Hospital, his employer, from Astra Zeneca and Novartis related to the design, statistical monitoring, and analysis of clinical trials in the setting of cardiovascular drugs. Dr. Glynn also signed a consulting agreement to give a one-time Grand Rounds talk on comparative effectiveness research methods at Merck. Dr. Rassen is a consultant to Phase Forward. Opinions expressed here are only those of the authors and not necessarily those of the agencies.