PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jpmMary Ann Liebert, Inc.Mary Ann Liebert, Inc.JournalsSearchAlerts
Journal of Palliative Medicine
 
J Palliat Med. 2009 May; 12(5): 471–474.
PMCID: PMC2991179

When and How to Use Instrumental Variables in Palliative Care Research

Joan D. Penrod, Ph.D.,corresponding author1,,2 Nathan E. Goldstein, M.D.,1,,2 and Partha Deb, Ph.D.3

Introduction

Most palliative care researchers want to demonstrate the effect of a treatment or intervention X on outcome Y. For example, we want to say that patients treated by the hospital-based palliative care team will have fewer intensive care unit (ICU) admissions than those not treated by the team (usual care patients). Randomized controlled trials (RCT) are the strongest design for making causal inferences about the effect of treatment X on outcome Y. Random assignment of patients to the treatment or control groups ensures there are no systematic differences between the groups in characteristics, observed and unobserved, that may affect outcome. Consequently differences between the groups on the outcome of interest can be attributed to the treatment.

RCTs of palliative care treatments and interventions are difficult but not impossible to do. The difficulties arise from several sources, including enrollment bias, protocol adherence, and problems with external validity.13 Some physicians believe that palliative care is superior to usual care. Consequently, they believe it's unethical to encourage their patients to be in randomized trials of palliative care interventions.4 At the same, there are physicians who do not believe palliative care will benefit their patients and so, do not want them enrolled in a palliative care study.

In addition, it may be unethical for researchers to take part in an RCT of palliative care. If palliative care was an additional service and researchers did not know if it provided benefit or harm then randomization might be an option. However, for palliative care researchers who believe the specialized care is beneficial, an RCT becomes an ethical issue.

Adherence to the protocol can be challenging in an RCT of palliative care. Once patients are enrolled in an RCT, the crossover between treatment and usual care can be considerable.4 Physicians who gain direct or indirect positive experience with palliative care at the study hospital are likely to request palliative care consultation for their patients randomized to usual care. Similarly, family members and patients themselves may also request palliative care consultation. If these patients received palliative care as crossovers, “intent to treat,” or “as treated” analysis would produce biased estimates of the treatment effect.3,5

Finally, most RCTs sacrifice wide external validity for strong internal validity.6 That is to say, the trials are methodologically rigorous, but the results may not generalize to other more clinically relevant populations. For some program and policy research in palliative care, this trade-off may be inappropriate. For these reasons, many questions about the effects of palliative care on such outcomes as pain and other symptom management, family satisfaction with care, use of ICU care and costs will have to be answered using observational data. The analyses will have to use sophisticated methods that many of us in clinical research and practice may be less familiar with compared to the RCT.

In this article we first explain what instrument variable estimation is and we show its use in a classic study of treatment of acute myocardial infarction (AMI).7 We then discuss how instrumental variable estimation is used in a recent study from geriatrics literature. Finally, we suggest possible instrumental variables for a recent palliative care study. Readers will find a thorough and modern introduction to instrumental variables models and their applications in Stock and Watson.8

What is Instrumental Variable Estimation?

For all the reasons discussed previously, palliative care researchers frequently want to know the effect of a treatment (e.g., antipsychotic medication for delirium at end of life) or intervention (e.g., palliative care consultation) on outcomes (anxiety, family satisfaction, hospital costs) using observational data. The key difference between experimental and observational data is that the treatment is not allocated randomly in the latter. Consequently, the characteristics of those receiving the treatment or intervention will generally differ from the characteristics of those who do not (the comparison group). The researcher usually can observe and measure some of the characteristics of the sample, such as age and diagnosis, that may differ between the treated and untreated groups, and control for their effect on the outcome with multiple variable regression analysis. These variables are often called confounders because they distort the estimate of the effect of the treatment on the outcomes unless they are included in the regression. Researchers use multiple regression analysis to produce estimates of a treatment effect adjusted for observed covariates or confounders.

In observational data, because there is no random assignment of patients to treatment there is an important threat from unobserved factors that influence who receives the treatment or intervention and the outcome of interest. For example, if we are interested in estimating the effect of palliative care consultation on the costs of hospitalization and we can not randomly assign patients with advanced disease to receive consultation during hospitalization, the two groups may differ in observed and unobserved ways we can not measure but that affect the outcome. For example, patients who do not receive palliative care consultation may be those whose doctors do not think palliative care is helpful or they may be patients who do not trust doctors. As long as the researcher suspects that there is at least one important unobserved confounder, methods that just control for observed differences (e.g., least squares regression) will produce biased estimates of the treatment effect. The goal of instrumental variable estimation is to reduce or remove the bias resulting from unobserved differences in the groups.9,10

In instrumental variable estimation the researcher has to identify one or more variables, known as the instrumental variables or instruments that meet two requirements. First, the instrumental variables should be relevant, i.e., they should induce significant variation in the treatment. Second, the instrumental variables should be exogenous, i.e., they cannot influence the outcome directly—the only mechanism by which the instruments can affect the outcome is through their effect on the treatment.8,10,11 For most palliative care research, that means we are looking for factors that predict selection of patients into palliative care but do not affect subsequent outcomes, except through the choice of palliative versus usual care. Of the two requirements for a valid instrument, relevance is typically simpler to deal with because relevance can be checked using simple statistical tests.8 Exogeneity is a much more difficult issue because, in general, the exogeneity of an instrument cannot be checked statistically. Instead, one must argue from a conceptual perspective that there are no plausible mechanisms by which the instrument may affect the outcome directly. Thus, it is often conceptually useful to think of an instrumental variable as a device that acts as a randomizer.10

In their classic article on the effect of cardiac catheterization and revascularization for treatment of acute myocardial infarction (AMI) on in-hospital mortality of older adults, McClellan and colleagues7 argue that the treatment effect estimated from a regression of treatment choice on mortality might produce a biased estimate because of unmeasured differences in patient severity of illness that affects both treatment choice and outcomes. In addition, not all hospitals have catheterization capability. Cardiac catheterization is a diagnostic procedure, not a treatment for AMI, and hospitals with catheterization labs may be higher quality than those without the technology. Mortality may be higher (and quality lower) in those hospitals that treat fewer cardiac patients.

McClellan and colleagues7 used “differential distance” as the instrumental variable. They defined differential distance as the additional distance, if any, beyond the distance to the closest hospital a patient would have to go to reach a hospital with catheterization. Differential distance was strongly associated with treatment choice because patients with AMI typically go to the nearest hospital. The greater the distance, the less likely the patient is to be admitted to a hospital with catheterization. However, there is no reason to believe that differential distance from the hospital is associated with severity of the AMI and would have a direct effect on mortality. If households chose their residential location based on proximity to high-quality hospitals, then distance would be associated with hospital quality and thus directly associated with mortality; but this is implausible. Distance, in their framework, partially randomizes patients across hospitals of different quality and thus is an appropriate instrumental variable.

Instrumental variables estimation is typically done using two-stage least squares (2SLS). As the name implies, the logic of this method involves two stages. In the first stage, a regression that predicts choice of treatment is estimated with all covariates and the instrumental variables. In the second stage, the predicted value of treatment replaces actual treatment in a regression of the outcome on treatment and covariates. This two-stage approach eliminates the bias inherent in a regression of outcome on actual treatment. Note however, that although the logic of the method is in two stages, the actual computation is conducted in “one step” in most statistical packages.

A Recent Example of Instrumental Variable Estimation

To the best of our knowledge, instrumental variable estimation has not been used in a published study in the palliative care research literature. There are examples, however, where instrumental variable estimation has been used in populations relevant to the care of patients with serious illness.1216 We provide several examples of both appropriate and inappropriate use of instrumental variable estimation.

A recent study reported in The New England Journal of Medicine addressed the use of second generation atypical antipsychotic medications for treatment of older adults.17 Over the past several years, concerns have been raised about increased risk of death and the use of these medications in this population. At the time the study was performed, there were no strong data on the risk of death posed by the conventional antipsychotic medications. In the absence of data from a randomized controlled trial, Wang and colleagues used instrumental variable estimation to examine the risk of death within 180 days among elderly patients being started on conventional antipsychotic medication compared to the risk among those being started on atypical antipsychotic medications.17

The investigators used Medicare data for the analysis and recognized that there would be unmeasured confounding variables despite the numerous measured demographic and clinical characteristics of the study population available. The instrument used was the prescribing physician's preference for conventional or atypical antipsychotic medication, measured as the physician's choice of antipsychotic for his or her most recent patient newly started on medication before the index patient.17 These data were all available in the Medicare claims for the sample. They argued that the antipsychotic prescribing choice the physician made for his or her most recent patient before the index patient reflected his or her preference but was unrelated to risk of death in the index patient except through choice of medication.

Using 2SLS for the instrumental variable estimation and additional adjustment for measured patient characteristics, the risk difference of 180-day mortality between initiators of conventional and atypical antipsychotic medication was calculated. The investigators found in the instrumental variables analysis that conventional antipsychotics were associated with a higher risk of death within 180 days than were the atypical antipsychotics. In this case the adjusted estimates in the instrumental variable analyses did not differ from the traditional multivariable estimates.17 The instrumental variable model served to strengthen the findings from the traditional adjusted mortality model.

In an example in which instrumental variable estimation would help but is not employed is a recent study by Wright and colleagues18 examining the associations between end-of-life discussions with physicians and the medical care that terminally ill patients with cancer receive near death. The investigators acknowledge the existence of unmeasured selection factors, such as patient preference for less aggressive medical care that also make the patient more likely to initiate these discussions with their physicians.

Would it be possible to use instrumental variable estimation in this study and studies like it that are common in palliative care research? The investigator's challenge is to find an instrument that induces variation in the treatment and is related to the outcome only through its effect on choice of treatment. In some cases, institutional rules may provide the basis for an instrument. Suppose, for example, that patients who are admitted on weekends are, for an administrative reason, less likely to receive palliative care consultation. If we are interested in estimating the difference in family satisfaction with terminal hospital care for palliative versus usual care terminally ill patients, a “day of admission” indicator might be a valid instrument because it will be associated with the likelihood of receiving treatment but not directly associated with the outcome. Researchers may wish to consider information about physician practice style, such as quality of medical school attended as instruments. In situations where there are sufficiently many patients per physician, researchers can also consider using an indicator for each physician as a proxy for practice style. On the other hand, day of admission, as an instrument, would not be exogenous if other dimensions of care available to patients admitted on the weekend are different from care available to patients admitted on weekdays.

Finally, consider the use of patient religiosity as an instrument. There is a good chance religiosity would affect the likelihood of receiving palliative care, thus it is relevant. But religiosity of the patient may also affect other aspects of usual or palliative care, thus having a direct effect on the outcome. This makes patient religiosity a weak or inappropriate instrument.

These examples demonstrate the feasibility of using instrumental variables methods, but also the challenges in finding valid instruments in observational data. One way to strengthen the analysis is to find two different instruments and compare the results using each. But in practice, most researchers are appropriately satisfied if they find one good instrument.

Investing the effort into finding plausibly valid instruments is worthwhile because the biases in treatment effects from standard models may be large enough so as to render those estimates worthless.19 However, instrumental variable methods also should be used with caution. Instrumental variable estimates can be more biased than standard estimates if the instruments are not sufficiently strong predictors of treatment. There is a growing literature on “weak instruments” that addresses these issues (see, for example, Murray20). Exogeneity of the instrument is an important requirement. Even small associations between the instrument and the outcome can cause instrumental variables estimators to be biased.

Conclusion

In summary, we have tried to familiarize palliative care researchers with the problem of selection bias in observational data and the use of instrumental variable estimation to handle it. Instrumental variable estimation has been used in economic research for several decades to reduce or remove the bias resulting from unobserved differences between nonrandomized groups.9,10 Its use has been increasing over the past 15 years in health services and medical care research.16,2125 Because quasi-experimental and observational research designs are likely to dominate research on the effects of palliative care on patient, family and health system outcomes, palliative care researchers using these nonexperimental designs can strengthen the scientific rigor of their findings by using instrumental variable methods when possible. Palliative care patients and their families are challenging to study but their care is in desperate need of an improved evidence base to assure that the highest quality of care can be provided.

Researchers interested in applying instrumental variables estimation should proceed carefully. We have focused on the issue of choosing instrumental variables here, but there are other issues. First, in certain circumstances, applications of methods other than 2SLS may be well advised, among them being maximum likelihood methods and control function approaches.26 Second, although standard regression methods will produce biased estimates in the situations described above, instrumental variables estimates may also be biased if the underlying assumptions are not satisfied. Given the complexities, researchers would be well advised to consult with other researchers who have expertise in instrumental variables methods.

Acknowledgments

This project was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (project no. REA 08-260) and the Geriatric Research, Education, and Clinical Center, James J. Peters VA Medical Center, New York, NY. Dr. Goldstein was also supported by a Mentored Patient-Oriented Research Career Development Award from the National Institute of Aging (K23 AG025933).

References

1. Penrod JD. Morrison RS. Challenges for palliative care research. J Palliat Med. 2004;7:398–402. [PubMed]
2. Rinck GC. van den Bos GA. Kleijnen J. de Haes HJ. Schade E. Veenhof CH. Methodologic issues in effectiveness research on palliative cancer care: A systematic review. J Clin Oncol. 1997;15:1697–707. [PubMed]
3. Ewing G. Rogers M. Barclay S. McCabe J. Martin A. Todd C. Recruiting patients into a primary care based study of palliative care: Why is it so difficult? Palliat Med. 2004;18:452–459. [PubMed]
4. Grande GE. Todd CJ. Why are trials in palliative care so difficult? Palliat Med. 2000;14:69–74. [PubMed]
5. Marcus SM. Gibbons RD. Estimating the efficacy of receiving treatment in randomized clinical trials with noncompliance. Health Serv Outcomes Res Methodol. 2001;2:247–258.
6. Concato J. Shah N. Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:1887–1892. [PMC free article] [PubMed]
7. McClellan M. McNeil BJ. Newhouse JP. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994;272:859–866. [PubMed]
8. Stock J. Watson M. Introduction to Econometrics. Reading MA: Addison-Wesley; 2007.
9. D'Agostino RBJ. D'Agostino RBS. Estimating treatment effects using observational data. JAMA. 2007;297:314–316. [PubMed]
10. Newhouse JP. McClellan M. Econometrics in outcomes research: The use of instrumental variables. Annu Rev Public Health. 1998;19:17–34. [PubMed]
11. Angrist JD. Imbens GW. Rubin D. Identification of causal effects using instrumental variables. J Am Stat Assoc. 1996;91:444–455.
12. Brooks JM. Chrischilles EA. Scott SD. Chen-Hardee SS. Was breast conserving surgery underutilized for early stage breast cancer? Instrumental variables evidence for stage II patients from Iowa. Health Serv Res. 2003;38(6 Pt 1):1385–1402. [PMC free article] [PubMed]
13. Malkin JD. Broder MS. Keeler E. Do longer postpartum stays reduce newborn readmissions? Analysis using instrumental variables. Health Serv Res. 2000;35(5 Pt 2):1071–1091. [PMC free article] [PubMed]
14. Brookhart MA. Wang PS. Solomon DH. Schneeweiss S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology. 2006;17:268–275. [PMC free article] [PubMed]
15. Brookhart M. Wang P. Solomon D. Schneeweiss S. Instrumental variable analysis of secondary pharmacoepidemiologic data. Epidemiology. 2006;17:373–374. [PubMed]
16. Stukel TA. Fisher ES. Wennberg DE. Alter DA. Gottlieb DJ. Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias: Effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA. 2007;297:278–285. [PMC free article] [PubMed]
17. Wang PS. Schneeweiss S. Avorn J. Fischer MA. Mogun H. Solomon DH. Brookhart MA. Risk of death in elderly users of conventional vs. Atypical antipsychotic medications. N Engl J Med. 2005;353:2335–2341. [PubMed]
18. Wright AA. Zhang B. Ray A. Mack JW. Trice E. Balboni T. Mitchell SL. Jackson VA. Block SD. Maciejewski PK. Prigerson HG. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300:1665–1673. [PMC free article] [PubMed]
19. Linden A. Adams JL. Evaluating disease management programme effectiveness: An introduction to instrumental variables. J Eval Clin Pract. 2006;12:148–154. [PubMed]
20. Murray MP. Avoiding invalid instruments and coping with weak instruments. J Econ Perspect. 2006;20:111–132.
21. McClellan M. Newhouse JP. Overview of a special supplement issue of health services research on instrumental variables analysis applications in health services research. Health Serv Res. 2000;35(5 Part II):1061–1070. [PMC free article] [PubMed]
22. Setoguchi S. Wang P. Brookhart A. Canning C. Kaci L. Schneeweiss S. Potential causes of higher mortality in elderly users of conventional and atypical antipsychotic medications. J Am Geriatr Soc. 2008;56:1644–1650. [PubMed]
23. Vogel WB. Rittman M. Bradshaw P. Nissen D. Anderson L. Bates B. Marshall C. Outcomes from stroke rehabilitation in veterans affairs rehabilitation units: Detecting and correcting for selection bias. J Rehabil Res Dev. 2002;39:367–383. [PubMed]
24. Hadley J. Polsky D. Mandelblatt J. Mitchell JM. Weeks JC. Wang Q. Hwang YT. OPTIONS Research Team: An exploratory instrumental variable analysis of the outcomes of localized breast cancer treatments in a medicare population. Health Econ. 2003;12:171–186. [PubMed]
25. Kane R. Chen Q. Finch M. Blewett L. Burns R. Moskowitz M. Functional outcomes of posthospital care for stroke and hip fracture patients under medicare. J Am Geriatr Assoc. 1998;46:1525–1533. [PubMed]
26. Blundell R. Powell JL. Endogeneity in semiparametric binary regression models. Rev Econ Stud. 2004;71:655–679.

Articles from Journal of Palliative Medicine are provided here courtesy of Mary Ann Liebert, Inc.