|Home | About | Journals | Submit | Contact Us | Français|
Selective prescribing of conventional antipsychotic medication (APM) to frailer patients is thought to have led to overestimation of the association with mortality in pharmacoepidemiologic studies relying on claims data. The authors assessed the validity of different analytic techniques to address such confounding. The cohort included 82,012 persons initiating APM use after admission to a nursing home in 45 states with 2001–2005 Medicaid/Medicare data, linked to clinical data (Minimum Data Set) and institutional characteristics. The authors compared the association between APM class and 180-day mortality with multivariate outcome modeling, propensity score (PS) adjustment, and instrumental variables. The unadjusted risk difference (per 100 patients) of 10.6 (95% confidence interval (CI): 9.4, 11.7) comparing use of conventional medication with atypical APM was reduced to 7.8 (95% CI: 6.6, 9.0) and 7.0 (95% CI: 5.8, 8.2) after PS adjustment and high-dimensional PS (hdPS) adjustment, respectively. Results were similar in analyses limited to claims-based Medicaid/Medicare variables (risk difference = 8.2 for PS, 7.1 for hdPS). Instrumental-variable estimates were imprecise (risk difference = 8.8, 95% CI: −1.3, 19.0) because of the weak instrument. These results suggest that residual confounding has a relatively small impact on the effect estimate and that hdPS methods based on claims alone provide estimates at least as good as those from conventional analyses using claims enriched with clinical information.
Dementia is highly prevalent in older populations, and the number of people with dementia is expected to nearly double worldwide by the year 2020 (1, 2), making clinical management of these patients an important public health concern. Behavioral disturbances—such as psychosis, agitation, aggression, irritability, disinhibition, and wandering—are often among the most distressing aspects of dementia and are the primary reason older adults with dementia enter nursing homes (3). Between one-fifth and one-third of all nursing home patients in North America are currently being given antipsychotic agents (4–8), mostly for the treatment of dementia-related behavioral disturbances, despite the fact that these medications are not approved by the Food and Drug Administration (FDA) for this indication. Over time, use has shifted from older, conventional agents to the newer atypical agents (9). However, neither class of antipsychotic medication has well-established efficacy in this older, vulnerable population, and recent data have raised serious safety concerns regarding both types of medications (10–14).
In 2005, the FDA issued an advisory warning that atypical antipsychotic medications were associated with a 60%–70% increased risk of death in comparison with placebo in 17 short-term randomized controlled trials carried out among older dementia patients (15), and “black-box” warnings were added to the labels of all atypical agents. Most of the deaths appeared to be cardiovascular or infectious in nature (10). In June 2008, the FDA issued a similar black-box warning for conventional antipsychotic agents after judging the overall weight of the evidence, which included nonrandomized studies based on health-care utilization databases (11–14).
Although these nonrandomized studies (12–14) estimated a greater risk of death with use of conventional antipsychotic agents than with use of atypical antipsychotic agents, the FDA judged that “the methodological limitations in these studies preclude any conclusion that conventional antipsychotics have a greater risk of death with use than atypical antipsychotics” (11). Indeed, despite their unique strengths for comparative safety research (16, 17), pharmacoepidemiologic studies based on administrative claims data have a number of inherent limitations, the most important of which is incomplete information on potential predictors of study outcomes that might also lead to selective prescribing, resulting in biased estimates of effect. It has been argued that such factors (e.g., limitations in cognitive and physical functioning) may have led to selective prescribing of conventional antipsychotics to patients who were closer to death (18), which would have resulted in overestimation of the association between use of conventional antipsychotic medication and short-term mortality (19). Because the FDA advisories do not acknowledge the increased risk associated with conventional antipsychotics as compared with atypical antipsychotics because of concerns about residual confounding, clinicians and policy-makers are left with inadequate guidance on the treatment decisions they face on a daily basis for a growing segment of the population.
In the current study, we sought to address whether the mortality risks are indeed equal for both classes of antipsychotic medication in older patients with dementia-related behavioral problems, so that we could understand whether one class has a safety advantage that merits preferential prescribing. We used a range of analytic approaches and data sources to mitigate residual confounding bias. Aside from addressing the specific clinical question at hand, we aimed to provide researchers with some guidance on the merits of these different approaches—specifically the merits of high-dimensional propensity score (hdPS) adjustment versus standard propensity score (PS) adjustment with enriched data sources—to enhance causal interpretation of the safety of other medications.
The study cohort was drawn from a merged data set consisting of Medicaid and Medicare claims, the Minimum Data Set (MDS), and the Online Survey, Certification and Reporting (OSCAR) data set in 45 US states (all states except Arizona, Delaware, Nevada, Oregon, and Rhode Island) for the years 2001–2005. The MDS is a federally mandated health assessment tool used in US nursing homes that captures information on physical, psychological, and psychosocial functioning, active clinical diagnoses, health conditions, treatments, and services. Assessments are required upon admission, when a significant change in clinical status occurs, and then periodically within specific guidelines (typically quarterly). OSCAR is a compilation of data collected during the inspection surveys conducted at nursing facilities to certify them for participation in the Medicare and Medicaid programs. Inspections occur at least once during a 15-month period. The OSCAR database includes information on nursing home operational characteristics and aggregate resident characteristics.
Our cohort consisted of all patients aged ≥65 years who were dually eligible for Medicaid and Medicare and initiated treatment with antipsychotic medication after admission to a nursing home. Patients who had filled a prescription for antipsychotic medication during the 6 months before the first antipsychotic medication prescription filled in the nursing home (prevalent users) were excluded, as were patients with a preexisting diagnosis of cancer, schizophrenia, or bipolar disorder, since these patients were unlikely to receive antipsychotic medication for age-related behavioral disturbances. The cohort was restricted to patients who had had an MDS assessment in the 3 months before treatment initiation, to ensure that recent information on health status was used, and to patients residing in nursing homes that contributed more than 5 patients during the study period, permitting more accurate estimation of the nursing homes’ true prescribing preference for use in the instrumental-variable (IV) approach.
The atypical antipsychotic agents studied included risperidone, olanzapine, quetiapine, aripiprazole, ziprasidone, and clozapine. Other antipsychotic medications studied were considered to be conventional agents, including haloperidol, thioridazine, chlorpromazine, perphenazine, and fluphenazine. The first new prescription filled for an antipsychotic agent following nursing home admission determined the treatment group into which a patient was classified: conventional or atypical. Patients who filled a prescription for both types of agents on the same day were excluded.
The study outcome was death from any cause within 180 days of treatment initiation, as recorded in the Social Security Death Master File (20). A 180-day follow-up period was selected for comparability with existing studies (12–14). Sociodemographic characteristics included age, sex, race, education, and geographic region (state). Clinical characteristics were determined on the basis of medication use, the most recent MDS assessment before treatment initiation, and International Classification of Diseases, Ninth Revision, diagnostic and procedure codes recorded in claims for hospitalizations and physician visits in the 6 months before treatment initiation. Variables considered included psychiatric morbidity, cardiovascular morbidity, cerebrovascular disease, Parkinson’s disease, epilepsy, diabetes, obesity, functional impairment, and Charlson comorbidity index (21). The use of health-care services that were potentially predictive of risk of death in the short term was also assessed (i.e., number of days hospitalized and number of distinct prescription drugs used, excluding antipsychotic medications) (22).
Data on nursing home characteristics that may be correlated with processes of care and thus risk of death were obtained from OSCAR. These included variables such as number of beds in the facility, occupancy rate, availability of Alzheimer’s and other special care units, staffing levels, type of ownership (e.g., for-profit), proportion of residents whose care was paid for by Medicare/Medicaid, and variables related to resident characteristics and quality indicators (e.g., proportion of residents bed-bound, proportion with pressure sores, proportion on psychoactive medication, number of deficiencies). A complete list of all patient- and facility-level covariates considered is included in Web Table 1 (http://aje.oxfordjournals.org/).
We performed 4 types of analyses representing increasing levels of confounding adjustment: traditional multivariable outcome modeling, PS analysis, hdPS analysis, and IV estimation. For consistency with the IV approach, primary analyses implemented a first-exposure-carried-forward approach without censoring (“initially treated,” similar to intention-to-treat analyses) and focused on comparisons of absolute risk.
Since none of the covariates individually met the 5% change-in-effect-estimate criterion we had postulated for selection of potentially confounding variables for adjustment in the multivariate outcome model (23), we forced selected case-mix variables into the model. The selection of variables was informed by prior work in this field, including the studies that formed the basis for the FDA black-box warning (12, 14, 24).
PSs at treatment initiation were derived from predicted probabilities estimated in logistic regression models of the use of conventional antipsychotic medication versus atypical antipsychotic medication. To evaluate the added value of clinical and facility-level information for confounding control, we fitted 4 different PS models based on 1) Medicare and Medicaid claims only, 2) claims and clinical (MDS) information, 3) claims and facility-level (OSCAR) information, and 4) claims, clinical, and facility-level information. The PS models included all variables specified as factors that might confound the causal drug-outcome association based on our contextual knowledge of the study question, as well as variables related to the outcome but not related (or only weakly related) to the exposure (the specific variables are listed in Web Table 1) (25). To assess the value of the PS models for confounding control, the balance of important risk factors for death between users of conventional and atypical antipsychotic medication was checked within deciles of the estimated PS (26, 27). We truncated 2.5% of the patients on either extreme of the PS distribution and adjusted the outcome model for PS decile (28). Alternatively, we matched 1:1 on the PS using a greedy match algorithm (29).
Similar analyses were conducted using the hdPS (30). A limitation of standard approaches to confounding adjustment which hdPSs attempt to overcome is their reliance on the investigator’s being able to specify all factors that may confound a causal drug-outcome association. The hdPS algorithm evaluates thousands of diagnoses, procedures, and pharmacy claim codes, as well as clinical (MDS) and facility (OSCAR) characteristics (referred to as data dimensions), to identify and prioritize those covariates that serve as proxies for unmeasured confounders. Specifically, the 200 most prevalent codes in each data dimension are prioritized by calculating for each covariate the possible amount of confounding it could adjust in a multiplicative model given a binary exposure and outcome, after adjustment for demographic covariates. These covariates are then sorted in descending order of confounding potential, and the top 500 empirical covariates are selected. These empirically identified confounders are combined with investigator-identified covariates (i.e., sociodemographic variables and general indicators of comorbidity) to improve confounding adjustment. We also conducted secondary PS- and hdPS-adjusted analyses with follow-up censored at the time of treatment discontinuation or treatment switch, allowing for a 30-day grace period (“as-treated”).
When different nursing homes use the two types of antipsychotic medication at sharply disparate rates, the choice is evidently influenced by factors that are independent of patient characteristics, and it is possible to use the nursing home’s preferred treatment as a substitute for the actual exposure (i.e., as an IV) in analyses. We classified nursing homes that prescribed conventional antipsychotic agents to at least 1 in every 3 patients as homes that preferred conventional antipsychotic agents and nursing homes that prescribed conventional antipsychotic agents to none of their patients as those that did not prefer this type of medication. Patients admitted to all other nursing homes (i.e., 0% < cumulative proportion < 33%) were excluded from the analysis. To avoid introducing an artificial correlation between this cumulative proportion and a given patient’s prescription, we calculated a separate proportion for each patient using all other patients admitted to that nursing home.
Using this preference or lack thereof as an IV (IV1), we computed differences in the risk of 180-day mortality between the conventional antipsychotic medication group and the atypical antipsychotic medication group, using the classic approach to IV estimation motivated from a causal theory framework—that is, a 2-stage linear regression with and without adjustment for measured covariates (31). We also conducted an analysis in which nursing home prescribing preference was determined by the treatment decision made for the patient most recently admitted (IV2) (32).
To assess instrument strength, we computed the square of the partial correlation between the instrument and treatment, conditional on other covariates in the model. This partial R2 can be interpreted as the proportion of the variance explained by the addition of the IV to the model. We measured change in imbalance for measured covariates, comparing the population stratified by the treatment with the population stratified by the IV. We used the percentage change in Mahalanobis distance (33) as a summary measure of change in covariate balance between exposure groups.
Finally, we conducted a sensitivity analysis to identify the strength of the residual confounding that would be necessary to fully explain the drug-outcome association estimated using these different methods of adjustment (34).
Because patient-level observations were clustered within nursing homes, robust calculations of standard errors of the regression parameters were performed using generalized estimating equations.
A total of 82,012 older nursing home patients were included in our cohort; of these, 7,252 (8.8%) were prescribed a conventional antipsychotic medication and 74,760 (91.2%) were prescribed an atypical formulation. The proportion of patients receiving a conventional agent decreased from 12.0% in 2001 to 8.0% in 2004 but increased again to 9.7% in 2005, the year in which the FDA issued warnings of excess mortality associated with atypical antipsychotic agents. Subjects in the conventional drug group were more likely to be male and nonwhite than those in the atypical drug group (Table 1). Compared with the atypical drug users, the conventional drug users were more likely to have cardiovascular disease (especially congestive heart failure and other cardiovascular disease) and less likely to have psychiatric comorbidity (especially dementia and depression). Both groups were relatively comparable in terms of the severity of cognitive and functional impairment and the presence of diagnosed behavioral problems based on MDS data. Users of conventional antipsychotic medication had lower rates of antidepressant and dementia medication use but higher rates of hypnotic medication use. They also were more likely to have a Charlson comorbidity index greater than 5. Conventional drug users were more likely than atypical drug users to reside in a nursing home without mental health staff and without team-based care (Table 1).
In the first 180 days of use, 30% of patients who began using conventional antipsychotic agents died, as compared with 20% of those who began using atypical agents (Table 2). The mean survival times for initiators of conventional and atypical antipsychotics were 30 days and 63 days, respectively. The excess of 10.56 deaths per 100 persons (95% confidence interval (CI): 9.42, 11.70) in conventional antipsychotic medication users as compared with atypical antipsychotic medication users was reduced to 8.29 per 100 persons (95% CI: 7.34, 9.24) after adjustment for the claims-based potential confounders listed in Table 3.
Adjustment for a wide range of sociodemographic and claims-based covariates through the use of PS deciles resulted in a risk difference of 8.17 per 100 persons (95% CI: 7.00, 9.34) (Table 4). Addition of clinical information (MDS) and nursing home characteristics (OSCAR) to the PS did not meaningfully change the effect estimate; the risk difference after adjustment for all 3 categories of covariates was 7.79 per 100 persons (95% CI: 6.61, 8.96). Use of hdPS further reduced the risk difference to 7.07 per 100 persons (95% CI: 5.89, 8.24) based on claims data only and 7.00 per 100 persons (95% CI: 5.83, 8.17) based on all covariate categories (Figure 1). A similar pattern was observed with PS matching (Table 4).
Secondary analyses using the “as-treated” approach yielded no substantive changes relative to these “initially treated” analyses. For example, the unadjusted incidence rate ratio of 2.36 (95% CI: 2.20, 2.54) was reduced to 1.99 (95% CI: 1.85, 2.14) after hdPS adjustment using claims, MDS, and OSCAR information. The effect was strongest during the first 40 days of treatment (hdPS-adjusted incidence rate ratio = 1.98, 95% CI: 1.82, 2.16); the incidence rate ratios decreased to 1.59 (95% CI: 1.35, 1.87) and 1.38 (95% CI: 1.13, 1.68) after 40–79 days and 80–180 days of medication use, respectively.
In the IV analyses, use of conventional antipsychotic agents continued to be associated with an increased risk of death within 180 days in comparison with use of atypical formulations. The adjusted risk differences were estimated at 8.84 per 100 persons (95% CI: −1.28, 18.95) for IV1 (a low rate of conventional medication prescription vs. a high rate) and 8.59 per 100 persons (95% CI: −8.97, 26.14) for IV2 (prior patient), implying that for every 100 patients prescribed a conventional drug instead of an atypical drug (i.e., the marginal patients whose treatment status can be affected by the instrument), there would be 9 additional deaths (Table 5, stage 2). However, both instruments were very weak predictors of the exposure, explaining only 2.6% and 0.4% of the variance, respectively (Table 5, stage 1). In addition, the IV approach was inefficient because of the relatively small sample size (i.e., only 7,252 subjects were exposed to conventional antipsychotic medications), resulting in wide confidence intervals. This problem was further compounded by the use of weak instruments that yielded highly variable estimates of exposure effects.
Figure 2 displays the strengths of the association (odds ratio (OR)) between exposure and a potential unmeasured confounder (OREC) and the association (relative risk (RR)) between confounder and outcome (RRCO) that would be required to fully explain the observed increased mortality associated with conventional antipsychotic medication use (hdPS-adjusted relative risk = 1.34) if in truth no such increase existed. For an unmeasured confounder present in 25% of the population, relative risks of 3 or more linking the hypothetical confounder to both conventional antipsychotic medication use and mortality would be needed to fully explain the observed association. For confounders present in 5% or 1% of the population, relative risks greater than 4.5 and 9, respectively, would be needed. For an apparent association of 1.28 (lower bound of the 95% confidence interval for the hdPS-adjusted analysis), the required strength would be greater than 2.5 for an unmeasured confounder present in, at most, 25% of the population.
In this study of 82,012 older patients initiating use of antipsychotic medication after admission to a nursing home, those for whom conventional agents were prescribed had a 34% higher risk of death in the short term than those for whom atypical agents were prescribed, which corresponds to an additional 7.0 deaths per 100 patients treated with conventional agents. This finding of an increased risk with conventional antipsychotic agents versus atypical agents is consistent with findings from earlier studies in both predominantly community-dwelling cohorts (12–14) and long-term-care cohorts (13).
Nonrandomized studies using health-care utilization data are particularly scrutinized for their limited control of confounders and their potential for misclassifying diagnoses (16). In this study context, concern has been raised that patients who are frail and at increased risk of death might be more likely to be prescribed a conventional antipsychotic agent than an atypical agent, resulting in overestimation of the association (18). However, an earlier study that assessed residual confounding using survey data found that correction for 5 factors that are unmeasured in claims data (i.e., body mass index, smoking, Activities of Daily Living score, cognitive impairment, and physical impairment) resulted in a stronger association rather than a weaker association (35).
Our study population consisted of patients eligible for Medicaid. While this might have affected the rate of usage of the cheaper conventional antipsychotic medications, this restriction should not have affected the validity of our findings, since the central issue determining internal validity is comparability between the subcohorts included. As long as socioeconomic status and its correlates do not modify the association between antipsychotic medication and short-term mortality, the findings should also be generalizable (i.e., externally valid) (36).
Confounder information derived from health-care utilization or claims data was supplemented with clinical assessment data as recorded in the MDS. The MDS, which is part of the US federally mandated process for clinical assessment of all residents in Medicare- or Medicaid-certified nursing homes, provides a comprehensive assessment of each resident’s functional capabilities and helps nursing home staff identify health problems. The data are reported by the nursing homes themselves and are reviewed by nursing home inspectors but are not formally checked to ensure accuracy, yet most elements of the MDS have been shown to demonstrate good reliability (37, 38). As potential confounders of the association between antipsychotic medication and all-cause mortality, MDS markers of ill health, such as severity of physical and cognitive impairment, were of particular interest in this study. In addition, it has been suggested that nursing home patients may be at increased risk of death simply by being admitted to a facility with poor quality indicators, such as a higher intensity of antipsychotic medication use (39). We accounted for potential nursing home quality indicators through use of the OSCAR database, which reflects the findings from state inspections and complaint investigations. Adjustment for these measured clinical and nursing home characteristics, however, did not meaningfully affect the effect estimates.
We used hdPS techniques in an effort to mitigate residual confounding by unobserved factors. The assumption underlying this approach is that health-care utilization or claims data can be viewed as a set of proxies that indirectly describe the health status of patients through the lenses of health-care providers and coders operating under the constraints of a specific health-care system. By measuring a large battery of proxy variables, this approach aims to increase the likelihood that in combination they serve as a good proxy for relevant unobserved confounding factors (30).
In contrast to “kitchen sink” models, which indiscriminately include covariates in a PS model, the hdPS algorithm implements current knowledge regarding appropriate variable selection for PS models (e.g., variables are selected on the basis of their potential for confounding, and potential instruments are removed after review of the univariate associations between the selected variables and the exposure and outcome) (25, 30). In addition, by considering a multitude of variables addressing the same construct of a confounding factor, the hdPS approach also reduces the potential impact on confounding adjustment of “missing” claims data, that is, those that were not observed by the physician or not recorded by the system. Since it has been found that inclusion of interactions leaves the hdPS-adjusted effect estimates largely unchanged (30), we did not include them in our models. By design, all empirical covariates are categorical, and therefore nonlinearity is not a concern when implementing the algorithm.
In some typical pharmacoepidemiologic studies of treatment effects, proxy adjustment via hdPS generated effect estimates closer to randomized trial findings than standard covariate adjustment of investigator-predefined variables (30). In our study, use of hdPS changed the absolute effect from 7.79 fewer deaths per 100 persons over 180 days to 7.00 per 100 persons and the relative effect from 1.36 to 1.34. Although we have no gold standard against which to evaluate our results, the monotonic trend of a reduction in effect size with increasing levels of adjustment (see Figure 1) hints at improved confounding control with each level of adjustment. These findings suggest that there may be some confounding caused by clinical characteristics other than the measures reported in the MDS. Alternatively, the MDS variables may have been reported with sufficient nondifferential misclassification to limit the ability to control fully for the confounding.
IV estimation, which by design can control for unmeasured patient characteristics, appeared to confirm the findings of an increased mortality risk with conventional antipsychotic agents, but the instruments were too weak to permit interpretation of the estimated risk differences (40).
The sensitivity analyses demonstrated that very strong mortality risk factors that are fairly imbalanced among exposure groups must be unmeasured and uncontrolled in our study to explain the observed association. The results have already been adjusted for most known, strong, independent mortality risk factors, and any unmeasured confounder of the required strength would also have to be independent of the confounders we adjusted for; that is, correlated confounders such as patient vulnerability are to some extent adjusted by factors like cognitive and functional impairment and awake time. Although it is unlikely that we missed such a strong single confounder, it is conceivable that several weaker confounders may have acted together and explain the apparent effect.
In summary, findings from nonrandomized studies assessing the safety of conventional antipsychotic agents versus atypical antipsychotic agents have been criticized for being caused by residual confounding by unmeasured factors channeling the prescribing of conventional agents to patients at higher risk of death. Use of different analytic techniques to mitigate the effects of confounding by measured factors and, to a lesser degree, unmeasured factors consistently decreased but did not eliminate the observed association in a population of nursing home patients. In this study, the addition of clinical characteristics indicative of frailty and typically not available in administrative claims data did not meaningfully affect the effect estimates.
While confirmation of these findings in other empirical examples is in order, our results suggest that hdPS adjustment based on claims data—which, in contrast to conventional confounder adjustment methods, does not rely on a limited number of investigator-specified covariates but taps into layers of information that investigators have until recently not exploited—may provide more valid estimates than conventional approaches using claims data enriched with clinical information, often believed to include important confounding variables. Given that plausible IVs have been difficult to find in epidemiology and medicine (including in this study) (31), the hdPS approach may offer a promising and practical alternative to enhance the causal interpretation of effect estimates when residual confounding by indication is a concern.
Author affiliations: Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts (Krista F. Huybrechts, Sebastian Schneeweiss); Department of Epidemiology, School of Public Health, Boston University, Boston, Massachusetts (Krista F. Huybrechts, Kenneth J. Rothman, Rebecca A. Silliman); Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina (M. Alan Brookhart); RTI Health Solutions, Research Triangle Park, North Carolina (Kenneth J. Rothman); Department of Medicine, School of Medicine, Boston University, Boston, Massachusetts (Rebecca A. Silliman); and Institute for Health, Health Care Policy and Aging Research, Rutgers University, New Brunswick, New Jersey (Tobias Gerhard, Stephen Crystal).
This research was supported in part by National Institute of Mental Health grant R01-MH078708 and National Library of Medicine grant R01-LM010213 (Drs. Huybrechts and Schneeweiss), as well as Agency for Healthcare Research and Quality/Food and Drug Administration Award HS017918 and Agency for Healthcare Research and Quality Award HS016097.
Conflict of interest: none declared.