Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
CNS Drugs. Author manuscript; available in PMC 2011 March 30.
Published in final edited form as:
CNS Drugs. 2009; 23(2): 171–180.
PMCID: PMC3067056

Assessing residual confounding of the association between antipsychotic medications and risk of death using survey data

Sebastian Schneeweiss, M.D., Sc.D., Soko Setoguchi, M.D., Dr.P.H., M. Alan Brookhart, PhD, Liljana Kaci, B.A., and Philip S. Wang, M.D., Dr.P.H.*



Studies using linked claims databases found that conventional antipsychotic medications (APMs) were associated with short-term mortality compared with atypical APMs. It has been suggested that such results may be due to residual confounding by factors that cannot be measured in claims databases. Using detailed survey data we identified the direction and magnitude of such residual confounding.


Cross-sectional survey data


Medicare Current Beneficiary Survey (MCBS)


17,776 survey participants ≥ 65 years


To determine the association between conventional APM use and potential confounding factors we assessed 5 factors not measured in Medicare claims data but in the MCBS: body mass index, smoking, activities of daily living (ADL) score, cognitive impairment, and Rosow-Breslau physical impairment scale. We estimated adjusted associations between these factors and APM use. Combined with literature estimates of the independent effect of confounders on death we computed the extent of residual confounding caused by a failure to adjust for these factors.


Comparing conventional APM users with atypical APM users, we found that not adjusting for impairments in ADL score led to an underestimation of the association with death (-13%) as did a failure to adjust for cognitive impairment (-7%). All 5 unmeasured confounders combined resulted in net confounding of -5% (range: -19% to +2%). After correction, the reported association between conventional APM use and death compared with atypical APM use was slightly increased from RR= 1.37 to RR = 1.44 (95% confidence interval: 1.33 to 1.56). Comparing any APM use with non-users would result in overestimations of > 50% if cognitive impairment remained unadjusted.


Claims data studies tend to underestimate the association of conventional APMs with death compared with atypical APMs because of residual confounding by measures of frailty. Studies comparing APM use with non-users may substantially overestimate harmful effects of APMs.

Keywords: Confounding (Epidemiology), Antipsychiotic Medications, Death, Elderly, External Adjustment


Antipsychotic medications (APMs) are disproportionately used in the elderly and are prescribed to over a quarter of Medicare beneficiaries in nursing homes.1,2,3 Reasons for this APM use include dementia, delirium, psychosis, agitation, and affective disorders, with much use outside FDA-approved indications.4,5 In addition to rising use, there have been rapid shifts from first-generation conventional agents (e.g., haloperidol, phenothiazines, and butyrophenones) to heavily marketed second-generation atypical agents (e.g., aripiprazole, clozapine, olanzapine, quetiapine, risperidone, and ziprasidone).6

In a Public Health Advisory in April 2005, the US Food and Drug Administration (FDA) warned that atypical APMs nearly doubled the risk of death vs. placebo in 17 short-term randomized controlled trials among an elderly population with dementia.7 “Black box” warnings were added to labels of all atypical APMs describing these risks, but the advisory did not extend to conventional APMs, although the FDA noted that this is an important issue to study in the future.7,8 Subsequent pharmacoepidemiologic studies showed a consistent 30% increase in short-term mortality (risk of death within 180 days) in new users of conventional APMs compared with atypical APMs.9,10

Pharmacoepidemiologic studies that relied on large health care utilization data to be adequately powered for identifying a difference in short-term mortality and to reflect routine clinical prescribing of APMs have come under criticism for their lack of information on important potential confounders such as limitations in cognitive and physical functioning, and other patient factors that may be related to death.11 It was argued that such factors may have led to selective prescribing of conventional APMs to patients closer to death,11 which would result in an overestimation of the association between APM use and short-term mortality.12 If one could assess the amount of residual confounding by unobserved factors in epidemiologic studies and correct the observed association for such bias, it would be possible to obtain better adjusted estimates of the association between conventional APM use and death. We have applied these methods to assess residual confounding caused by selective prescribing in other studies on drug effects.13,14,15

We sought to assess the magnitude of confounding bias caused by factors not observed in Medicare utilization data, including information on body mass index (BMI), smoking, and functional and cognitive impairment that was available in a representative survey of Medicare beneficiaries. This information was used to correct estimates of an association between atypical and conventional APMs and short-term mortality based on existing claims data studies.9,10



Linked Medicare claims data contain information on physician services provided and diagnoses recorded, hospital discharge diagnoses and procedures, and detailed pharmacy dispensing information. We identified five patient characteristics not measured in Medicare utilization data but available in a representative survey of Medicare beneficiaries that could act as confounders in a study of conventional vs. atypical APM use and short-term mortality: BMI, current smoking status, activities of daily living (ADL) score, cognitive impairment, and physical impairment. We then assessed their association with the use of conventional APM, atypical APM, or neither, based on data from a detailed in-home survey of Medicare beneficiaries.

The Medicare Current Beneficiary Survey (MCBS) was conducted in a sample of beneficiaries selected each year to be representative of the current Medicare population, including both aged and disabled beneficiaries living in the community or in institutions.16 The MCBS slightly over-sampled disabled patients (under 65 years of age) and the oldest-old (85 years of age or over).16 Data were obtained from face-to-face interviews by trained interviewers in the beneficiaries’ homes or facilities. In the community interview, an effort was made to interview all subjects directly. If a person was unable to answer all questions, a proxy respondent, usually a family member or close acquaintance, was asked to answer the questions. All MCBS participants were interviewed every 4 months and followed for up to 4 years. Each year, a supplemental sample was drawn and persons were added to the MCBS sample to account for growth in the Medicare population and to replenish the sample for surveyed seniors who died, left the survey population after 4 years, or were lost to follow-up.16 The survey was reported to have a high response rate (between 85% and 95%) and very high data completeness.17,18

Study Subjects

The MCBS sample was drawn from an enrollment list of all persons entitled to Medicare at the beginning of both study years, 2001 and 2002, and represented persons who were continuously enrolled throughout the full calendar year in the United States and Puerto Rico. The total sample size of the 2001 and 2002 MCBS was 25,587 subjects. For the present analysis, the study population was restricted to persons living in communities (23,120), which had 98.6% complete information of all critical items. Among these we further restricted the study population to MCBS respondents 65 years or older (17,776). The study was approved by the Center for Medicare and Medicaid Services and the Institutional Review Board of the Brigham and Women's Hospital.

Assessment of Medication Use

At every interview, drug names were recorded by the interviewer from medication bottles of participants in a free-text field of the questionnaire and subsequently checked and classified. 19 MCBS respondents were divided into three categories according to their drug use: (1) respondents who used conventional APMs in 2001 or 2002; (2) respondents who used atypical APMs in 2001 or 2002; and (3) respondents who used neither in 2001 and 2002 were classified as non-users. Users of conventional APMs who also used atypical APMs were excluded.

Assessment of Potential Confounders Measured in the MCBS

Potential confounders of interest assessed in the MCBS but unobserved in Medicare claims data included: BMI (weight in kg / [height in m]2), current smoking status, activities of daily living (ADL) score,20 cognitive impairment,21 and Rosow-Breslau physical impairment scale.22 Body mass index was dichotomized at a cutpoint of 30, according to WHO's definition for obesity. Smoking status was categorized into current versus former and never. A modified and validated ADL score was obtained from the MCBS data. 23 The Rosow-Breslau scale was computed according to the original instructions except for substituting “difficulty climbing stairs” with “difficulty stooping or kneeling” since the former but not the latter is a data item available in the MCBS. Cognitive impairment was ascertained from questions about Alzheimer's disease and memory loss.21

Statistical Analyses

Using the MCBS study population, we estimated the prevalence of exposure, p(E), and the prevalence of potential confounders, p(C), and the association between exposure and confounder, OREC. We used logistic regression to calculate the corresponding age-sex adjusted OREC, which was used for all subsequent analyses. We initially assumed the null hypothesis that there is no association between conventional APM use and death (RRED = 1). This assumption was only made to develop the formulas necessary to assess the magnitude of bias, but does not require that the true association really is 1.24 We derived estimates of the confounder-disease associations, RRCD, from the current medical literature. Literature estimates were derived from large cohort studies after an intensive literature search and expert consultations. If several valid literature estimates were identified, the average value was chosen for the base-case analysis. Based on these estimates and assumptions, a quadratic equation was derived to assess the direction and extent of residual confounding bias that would result from a failure to control for the list of these 5 possible confounders.15

We graphically explored the sensitivity to variations of our base-case literature estimate of RRCD. The joint distribution of unmeasured confounders that were observed only in the MCBS was not assessed because literature estimates were not available for many confounder combinations. Instead, we summed bias estimates over all confounders weighted by the prevalence of each confounder in the MCBS population. Finally, we calculated the maximum range of bias by summing all negative biases to yield a lower-bound estimate and all positive confounders to yield an upper bound. A spreadsheet including all calculations is available at


Among the community sample respondents, 17,776 were 65 and older and 42% were men (3,655). Table 1 shows characteristics of atypical APM users compared with conventional APM users as well as non-users. Users of atypical APMs more frequently had an ADL score ≥1 (58%) than those using conventional (45%) or non-users (29%), indicating their greater limitations in performing activities of daily living. Atypical APM users were more likely to have cognitive impairments (61%) than conventional APM users (52%) or non-users (12%). The distribution of physical impairments according to the Rosow-Breslau score as well as high BMI status were fairly similar among users of APMs.

Table 1
Body mass index, smoking, activities of daily living (ADL) score, cognitive impairments, and Rosow-Breslau work/walking impairment scale among community-dwelling Medicare beneficiaries 65 years or older using antipsychotic medications (APM) in 2001 and ...

Literature estimates of the associations between the individual possible confounders and the risk of death are shown in Table 2. Residual confounding bias in claims data risk estimates of an association between conventional APM use and the short-term risk of death were calculated separately for each of the five potential confounders (Table 3). Confounding bias due to failure to observe and adjust for each of five potential confounders was expressed as percent bias of the observed estimate from claims data. Uncontrolled ADL score was the strongest confounder, causing an underestimation by -13.0% when comparing conventional APM users with atypical APM users, followed by cognitive impairment (-7%, see Table 3). The net confounding bias, expressed as the sum of all component biases weighted by the population prevalence of each confounder, was -5%. The range from the most extreme negative bias to the most extreme positive bias was from -19% to +2%.

Table 2
Relative Risk Estimates* of the Associations Between Selected Potential Confounders and Death in Elderly Populations from the Medical Literature.
Table 3
Quantitative Assessment of Confounding Bias in Risk Estimates of Conventional APM Users vs. Atypical APM Users and Death.

When comparing conventional APM users with non-users (Table 4), we computed a large positive bias caused mostly by an unobserved ADL score (+23%) and cognitive impairment (+57%), which resulted in a net confounding bias of +7% (-5% to +83%). When comparing atypical APM users with non-users (Table 4), there was a slightly stronger bias caused by unobserved ADL scores (+42%) and cognitive impairment (+68%), yielding a net confounding bias of +11% (-5% to 113%).

Table 4
Quantitative assessment of confounding bias in risk estimates of APM use and death using different referent groups.

Bias estimates of the potential confounder ADL impairment and cognitive impairment were somewhat sensitive to changes in the estimate of the confounder-death associations derived from the literature (Figure 1a, and b).

Figure 1
Sensitivity of bias estimates towards misspecification of selected confounder-disease association (RRCD).


Two recent cohort studies using large linked health care utilization databases have consistently found that conventional APM use increases the risk of death in elderly patients compared with atypical APMs.9,10 One of these studies has been questioned because of the lack of information in claims data on important potential confounders such as cognitive, physical, and functional impairment.11 Our analysis demonstrates that confounding bias by patient factors that are not measured in claims data will not lead to an overestimation of the increased risk of death by conventional APMs but rather suggests a moderate underestimation. This result is mainly based on the observation that users of atypical APMs have more cognitive and functional impairment than those treated with conventional APMs.

A tendency that frail elderly patients with some behavioral disturbances are more likely to use APMs and are more likely to use atypical APMs has been observed before.4 In the present study we used three patient factors, cognitive, physical, and functional impairment, as proxies for frailty in elderly patients. Applying the resulting bias estimates to a recent cohort study based on Medicare claims data on the use of conventional versus atypical APMs5 resulted in an increase in the relative risk measure from 1.37 to 1.44 (Table 5). This result demonstrates the relatively small change of the association after adjusting for 5 important additional patient factors not observed in Medicare claims data. More importantly, it underlines that the observed increased risk of death in patients initiating atypical APMs is unlikely to be explained by confounding.

Table 5
Reported Associations between Conventional APM Users vs. Atypical APM Users and Death in Elderly Patients.

Our approach to assess direction and magnitude of unobserved confounding in claims data makes several simplifying assumptions. Exposure, confounder, and outcome were all coded as dichotomous variables. While this may not be of concern for the outcome of interest (mortality) and the drug use categories, for some confounders it may be an oversimplification, e.g., ADL score. Choosing alternative cut-points in confounder variables like ADL score may change the strength of an association; this is not likely if the underlying dose-response relationship is monotonic.26 Because of the “U”-shaped relationship between body mass index and mortality that would violate this assumption of monotonicity of a dose-response relationship, we computed the weighted bias average without the binary obesity indicator variable, which substantially increased the weighted bias estimate for the comparison between any APM and non-users. For our estimation of bias we assumed that the unobserved “true” drug-death association was 1. If the unknown true association is different from 1, our estimation of bias may be slightly inconsistent. However, the closer an association is to the null, the less our bias estimate will diverge from the true bias. Finally, we did not consider the joint distribution of unmeasured confounders. Instead, we computed a weighted sum of each potential confounder observed in the survey data as an approximation of the net bias. Although the extremes of this range assuming independence of individual biases are unlikely, their use will lead to a conservative interpretation of the data.25

Our bias estimates regarding cognitive and functional impairment when comparing any APM versus non-users were somewhat sensitive to the choice of the independent effect of cognitive and functional impairment on mortality as derived from the medical literature. This may not be surprising since in older patients APMs are frequently prescribed to patients with dementia and behavioral disturbances who have a higher risk of death so that any small change in the independent effect of the confounders must have influenced the estimate of bias meaningfully. The confounding bias arising in the comparison between conventional versus atypical APMs was much less sensitive to variation in the literature estimate.

Valid bias assessment depends on the survey being performed in a representative sample of the main claims data study. Given that the MCBS was designed to be representative for Medicare beneficiaries as well as the high response rate and data completeness, it is a valid and readily available source for bias assessment. Generalizability is slightly compromised by the fact that the MCBS time periods were not entirely overlapping with the studies the results were applied to, and the fact that the oldest-old were oversampled in MCBS. This analysis is limited to 5 unobserved confounders and does not address other sources of bias that may affect each of the observational studies in different ways in addition to covariates that should be routinely adjusted in claims data studies, including prior medication use, co-medications, and health services use.27 Each of the 5 unobserved confounders may have been reported with some degree of random misclassification, which would limit the ability to capture fully the confounding factor in the survey and would lead to an underestimation of bias. The survey used for this analysis was of limited size, occasionally resulting in wide confidence intervals. For the assessment of bias, the width of the confidence intervals as a measure of estimation precision has no implications as long as associations are estimated validly. Another limitation of MCBS is its cross-sectional nature that does not fully rule out the possibility that drug exposure resulted in changes of the measured characteristics and not vice versa.

Claims data studies tend to underestimate the association of conventional APMs with death compared with atypical APMs because of residual confounding by frailty measures. Studies comparing APM use with non-users may substantially overestimate harmful effects of APMs.


Dr. Schneeweiss received support from the National Institute on Aging (RO1-AG021950, RO1- AG023178), the National Institute of Mental Health (RO1-MH078708), and the Agency for Healthcare Research and Quality (2-RO1-HS10881), Department of Health and Human Services, Rockville, MD, and is principal investigator of the Brigham and Women's Hospital DEcIDE Research Center funded by the Agency for Healthcare Research and Quality.


1. Colenda CC, Mickus MA, Marcus SC, et al. Comparison of adult and geriatric psychiatric practice patterns: findings from the American Psychiatric Association's Practice Research Network. Am J Geriatr Psychiatry. 2002;10:609–617. [PubMed]
2. Giron MS, Forsell Y, Bernsten C, Thorslund M, Winblad B, Fastbom J. Psychotropic drug use in elderly people with and without dementia. Internal J Geriatr Psychiatry. 2001;16:900–6. [PubMed]
3. Breisacher BA, Limcangco R, Simoni-Wastila L, et al. The quality of antipsychotic drug prescribing in nursing homes. Arch Intern Med. 2005;165:1280–1285. [PubMed]
4. Alexopoulos GS, Streim J, Carpenter D, Docherty JP. Using antipsychotic agents in older patients: Expert Consensus Panel for Using Antipsychotic Drugs in Older Patients. J Clin Psychiatry. 2004;65(Suppl 2):5–99. [PubMed]
5. Wang PS, Brookhart MA, Setoguchi S, Patrick AR, Schneeweiss S. Psychotropic Medication Use for Behavioral Symptoms of Dementia. Curr Neurol Neurosci Rep. 2006;6:490–5. [PubMed]
6. Dewa CS, Remington G, Herrmann N, Fearnley J, Goering P. How much are atypical antipsychotic agents being used, and do they reach the populations who need them?: A Canadian experience. Clin Therapeutics. 2002;24(9):1466–76. [PubMed]
7. U.S. Food and Drug Administration FDA Public Health Advisory: Deaths with antipsychotics in elderly patients with behavioral disturbances. [12/15/06]. Available at:
8. Kuehn BM. FDA warns antipsychotic drugs may be risky for elderly. JAMA. 2005;293:2462. [PubMed]
9. Wang PS, Schneeweiss S, Avorn J, Fischer M, Mogun H, Solomon DH, Brookhart AM. Risk of death in elderly users of conventional vs. atypical antipsychotic medications. N Engl J Med. 2005;353:2335–41. [PubMed]
10. Schneeweiss S, Setoguchi S, Brookhart MA, Dormuth C, Wang PS. Mortality in users of Conventional and Atypical Antipsychotic Medications in British Columbia Senior. Can Med Assoc J. 2007;176:627–32. [PMC free article] [PubMed]
11. Stone M, Racoosin JA, Laughren T. Letter to the Editor: Conventional vs. atypical antipsychiotic medications. N Engl J Med. 2006;354:973.
12. Walker AM. Confounding by indication. Epidemiology. 1996;7:335–336. [PubMed]
13. Schneeweiss S, Wang P. Association between SSRI use and hip fractures and the effect of residual confounding bias in claims database studies. J Clin Psychopharm. 2004;24:632–8. [PubMed]
14. Schneeweiss S, Wang P. Claims data studies of sedative-hypnotics and hip fractures in the elderly: Exploring residual confounding using survey information. J Am Geriat Soc. 2005;53:948–54. [PubMed]
15. Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information: The example of COX2 inhibitors and myocardial infarction. Epidemiology. 2005;16:17–24. [PubMed]
17. Adler GS. A profile of the Medicare Current Beneficiary Survey. Health Care Financing Review. 1994;15:153–163. [PubMed]
18. Adler GS. Medicare beneficiaries rate their medical care. Health Care Financing Review. 1995;16:175–187. [PubMed]
19. Davis M, Poisal J, Chulis G, et al. Prescription drug coverage, utilization and spending among Medicare beneficiaries. Health Affairs. 1999;18:231–243. [PubMed]
20. Katz S, Ford AB, Moskowitz RW, et al. Studies of illness in the aged: the index of ADL, a standardized measure of biological and psychological function. JAMA. 1963;185:14–919. [PubMed]
21. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23:433–441. [PubMed]
22. Rosow I, Breslau N. A Guttman health scale for the aged. J Gerontol. 1966;21:556–559. [PubMed]
23. Saliba D, Orlando M, Wenger NS, et al. Identifying a short functional disability screen for older persons. J Gerontol. 2000;55:750–756. [PubMed]
24. Walker AM. Observation an Inference. Epidemiology Resources Inc.; Newton: 1991. pp. 120–124.
25. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Safety. 2006;15:291–303. [PubMed]
26. Rothman K, Greenland S. Modern Epidemiology. 2nd ed. Lippincott; Philadelphia: 1998. p. 205.
27. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58:323–37. [PubMed]
28. Guralnik JM, LaCroix AZ, Branch LG, Kasl SV, Wallace RB. Morbidity and disability in older persons in the years prior to death. Am J Public Health. 1991;81:443–7. [PubMed]
29. Branch LG, Katz S, Kniepmann K, Papsidero JA. A prospective study of functional status among community elders. Am J Public Health. 1984;74:266–8. [PubMed]
30. Ramos LR, Simoes EJ, Albert MS. Dependence in activities of daily living and cognitive impairment strongly predicted mortality in older urban residents in Brazil: A 2-year follow-up. J Am Geriatr Soc. 2001;49:1168–75. [PubMed]
31. Corrada MM, Kawas CH, Mozaffar F, Paganini-Hill A. Association of Body Mass Index and Weight Change with All-Cause Mortatliy in the elderly. Am J Epidemiol. 2006;163:938–49. [PMC free article] [PubMed]
32. Freedman DM, Ron E, Ballard-Barbash R, Doody MM, Linet MS. Body mass index and all-cause mortality in a nationwide US cohort. Int J Obest. 2006;30:822–9. [PubMed]
33. Hu FB, Willet WC, Li T, Stampfer MJ, Colditz GA, Manson JE. Adiposity as compared with physical activity in predicting mortality among women. N Engl J Med. 2004;351:2694–703. [PubMed]
34. Doll R, Peto R, Boreham J, Sutherland I. Mortality in relation to smoking: 50 years’ observations on male British doctors. BMJ. 2004;328:1519–28. [PMC free article] [PubMed]
35. Dewey ME, Saz P. Dementia, cognitive impairment and mortality in persons aged 65 and over living in the community: a systematic review of the literature. Int J Geriatr Psychiatry. 2001;16:751–61. [PubMed]
36. Velentgas P, Cali C, Diedrick G, et al. A survey of aspirin use, non-prescription NSAID use, and cigarette smoking among users and non-users of prescription NSAIDs: estimates of the effect of unmeasured confounding by these factors on studies of NSAID use and risk of myocardial infarction. Pharmacoepidemiol Drug Safety. 2001;10:S103.