|Home | About | Journals | Submit | Contact Us | Français|
Post-marketing observational studies of the safety and effectiveness of prescription medications are critically important, but fraught with methodological problems. The data sources available for such research often lack information on indications and other important confounders for the drug exposure under study. Instrumental variable methods have been proposed as a potential approach to control confounding by indication in non-experimental studies of treatment effects; however, good instruments are hard to find. We propose an instrument for use in pharmacoepidemiology that is based on a time-varying estimate of the prescribing physician’s preference for one drug relative to a competing therapy. The use of this instrument is illustrated in a study comparing the effect of exposure to COX-2 inhibitors with non-selective, non-steroidal anti-inflammatory medications on gastrointestinal complications.
Using conventional multivariable regression adjusting for 17 potential confounders, we found no protective effect due to COX-2 use within 120 days from the initial exposure (RD = -0.06 per 100 patients; 95% CI: -0.26 to 0.14). However, the proposed instrumental variable method attributed a protective effect to COX-2 exposure (RD = -1.31 per 100 patients; 95% CI: -2.42 to -0.20) compatible with randomized trial results (RD = -0.77 per 100 patients; 95% CI: -1.28 to -0.26).
The instrumental variable method that we have proposed appears to have substantially reduced the bias due to unobserved confounding. However, more work needs to be done to understand how sensitive these estimates are to modest violations of the instrumental variable assumptions.
Post-marketing observational studies are often conducted to evaluate the safety and effectiveness of drugs as they are used in routine practice. These studies are necessary since clinical trials often exclude populations that ultimately use the drug, such as the elderly, children, pregnant women, and patients with renal or liver diseases or other chronic illnesses.1 Since clinical trials are underpowered to detect uncommon adverse events, non-experimental post-marketing studies are also needed to evaluate the safety of approved drugs.2,3 This function of post-marketing research has come under increased scrutiny with the recent safety concerns pertaining to several widely-used therapeutic agents, including selective COX-2 inhibitors, hormone replacement therapy, and selective serotonin re-uptake inhibitors.
Studies of outcomes associated with exposure to pharmaceutical products as they are used in routine practice are inherently non-experimental. Often such studies are based on health care claims data containing longitudinal information on pharmacy dispensing, health care encounters, procedures, and ICD-coded diagnoses.4 While these files contain data on large populations over extended periods of time, they often lack detailed information on clinical indications for specific therapies. This problem is thought to be particularly acute in studies of intended drug effects because of the difficulty in adjusting for confounding by indication;5 i.e. patients who are thought to benefit most from a drug are more likely to receive therapy.6,7,8 Although epidemiologists have a variety of design options and analytic tools to adjust for measured confounders, pharmacoepidemiologic studies have consistently been criticized for having incomplete information on many potential predictors of study outcomes that might lead to selective prescribing.9, 10,11,12,13 Some authors argue that it is impossible to fully adjust confounding by indication in studies of intended drug effects using current epidemiologic methods.14
Ecologic, grouped treatment, and instrumental variable methods have been proposed as potential approaches to reduce confounding in clinical epidemiology.15,16,17,18,19,20,21 Although an individual patient’s treatment assignment might be confounded by unmeasured variables, it is thought that by using a variable that is related to the patient’s treatment, but unrelated (or only weakly related) to the unobserved patient risk factors, it might be possible to estimate a treatment effect for which the confounding bias is eliminated or strongly attenuated. Unfortunately, it is difficult to find high quality instruments in drug epidemiology. Hospital or region-level instruments have been used when substantial variations in practice patterns are thought to exist. The use of individual physicians as instruments has been proposed for situations in which physicians are thought to vary in their preference for different treatments under study.22
In this paper, we explore the use of a physician-level instrumental variable for studies that compare the short-term effects of two or more competing drug therapies. The instrument that we consider is a time-varying estimate of a physician’s relative preference for a given drug, where at least two therapeutic alternatives exist. We develop this idea in a study of the short-term effects of COX-2 inhibitor use on GI toxicity, compared with non-selective non-steroidal anti-inflammatory drugs (NSAIDs). This example was chosen for three reasons: 1) conventional claims data studies of the intended effects of COX-2 use are known to be problematic because of residual confounding by unmeasured factors, including aspirin use, body-mass index, physical activity, smoking status, alcohol consumption, and subtle GI symptoms not recorded in claims data; 2) recent research showed that physicians appear to differ substantially in their preference for prescribing COX-2 inhibitors compared with non-selective NSAIDs;23,24,25,26 and 3) randomized controlled trial (RCT) results are available, indicating a causal relation between COX-2 use and reduced GI toxicity and providing an accepted reference standard.
Our study population comprised a cohort of new oral NSAID users among Medicare beneficiaries 65 and older concurrently enrolled in the Pharmaceutical Assistance Contract for the Elderly (PACE) provided by the state of Pennsylvania. PACE has more generous income eligibility criteria than Medicaid and includes patients above the poverty level. To be eligible for PACE the annual income must be less than $13,000 if single and less than $16,200 if married. It reimburses the cost of all prescription medications including selective COX-2 inhibitors and non-selective NSAIDs, with a $6 co-payment. Enrollees of PACE were eligible for inclusion in the study if they filled a prescription for an oral preparation of a non-selective NSAID or selective COX-2 inhibitor (celecoxib, rofecoxib, valdecoxib) between January 1, 1999, and July 31, 2002, and demonstrated continuous health care system use. Continuous use was defined as filling at least one prescription drug and utilizing health care services during each of the 3 six-month periods before the index date defined below.
First-time NSAID use was defined as being an eligible beneficiary who filled at least one prescription for any oral preparation for an NSAID (either a COX-2 inhibitor or a non-selective NSAID) between January 1, 1999, and July 31, 2002, but did not use any NSAID during 18 months before the index date. The index date was the first date an NSAID prescription was filled. The analysis was limited to first-time users because a physician’s evaluation of patient risk factors would likely precede the first COX-2 or non-selective NSAID prescription. Any prescriptions following the index prescription were not considered.
Prescription drug information was assessed based on pharmacy claims from PACE. Computerized records include prescription drug name, dosage, quantity dispensed, days supplied, date dispensed, and the Universal Physician Identification Number (UPIN) for the prescribing physician. Outpatient and inpatient diagnoses, procedure codes, and dates of all inpatient and outpatient services obtained from Medicare claims data were linked to pharmacy dispensing data. Miscoding of drug information and misclassification of relevant ICD-9 diagnoses are described to be small or moderate.27,28,29,30 No information was available on over-the-counter NSAID or aspirin use, but beneficiary surveys have shown that over-the-counter aspirin use is similar in COX-2 and non-selective NSAID users in this population. 31 We omitted patients for whom the initial NSAID prescription did not contain a valid UPIN code. We obtained physician specialty information by linking the UPIN number to the American Medical Association’s (AMA) Masterfile of Physicians. Of the available sources for physician information, the AMA Masterfile has been found to be the most reliable source of information about physician specialty. 32
All personal identifiers were removed from the analytical data files to protect the privacy of subjects and their physicians. The study was approved by the institutional review board of Brigham and Women’s Hospital, Boston. We obtained Data Use Agreements from the Pharmaceutical Assistance Contract for the Elderly and the Center for Medicare and Medicaid Services.
GI complications were defined as either a hospitalization for GI hemorrhage and PUD complications including perforation (primary ICD-9 discharge diagnosis code 531.x, 532.x, 533.x, 534.x, 535.x in the first or second position) or an outpatient visit for a GI hemorrhage. These definitions were validated in 1762 patients in a hospital discharge database in Saskatchewan, with a composite positive predictive value (PPV) of 90%;33 similar PPVs were found in a regional Spanish discharge database.34
We estimated the effect of short-term COX-2 inhibitor use on GI complications compared with non-selective NSAIDs using conventional multivariable regression methods and an instrumental variable approach.
Instrumental variable analysis is a statistical approach that can be used to estimate a treatment effect in the presence of unmeasured confounding factors. This approach depends on the existence of a variable termed an instrument that is 1) associated with treatment, 2) unrelated to the confounders, and 3) unrelated to the outcome, other than through its association with actual treatment.35 36
The central issue in an instrumental variable analysis is the quality of the instrument. If assumptions 2 and 3 do not hold, an instrumental variable analysis can yield a biased estimate of the exposure effect. These assumptions are not completely testable with data, but investigators frequently examine the relationship between the instrument and the measured patient characteristics. Although the measured variables can be included in instrumental variable analysis, if an instrument is related to the measured attributes then it reasonable to expect that it is also related to the unmeasured attributes. A second critical issue is the strength of the relationship between the instrument and treatment. The more strongly an instrument is related to treatment, the more efficient the estimator will be, i.e. the smaller the standard error. The magnitude of the bias also depends on the strength of the association between the instrument and treatment, with weaker instruments yielding more biased estimates. More detailed discussions of instrumental variable methods and their assumptions can be found elsewhere.35 37
Our conceptual instrument is a physician’s current preference for a COX-2 inhibitor relative to a non-selective NSAID. The assumed causal relations motivating this choice of instrument are shown in Figure 1. The assumption that physicians vary in their preference for NSAIDs suggests that certain patients would be treated with a non-selective NSAID by some physicians and with a COX-2 inhibitor by others. In order for a physician’s preference to be a valid instrument, there can be no direct arrow from preference to the outcome. This requires that physicians can only influence short-term NSAID-related GI toxicity through the type of NSAID prescription that they assign. It is also necessary that there is no arrow between physician preference and the patient-level confounders. This assumption will be satisfied if physicians who prefer COX-2 inhibitors are not treating systematically different patients than physicians who prefer non-selective NSAIDs.
Unfortunately, physician preference is a variable that is not easily measured and one that can change in response to marketing activities by pharmaceutical companies, new information about safety and efficacy of drugs, and the physician’s own evolving clinical experience. The dynamic aspect of preference is particularly relevant with COX-2 inhibitors that were aggressively marketed, rapidly adopted by some physicians, not so rapidly by other physicians, and then subject to a variety of safety concerns that could have discouraged their use.24
We propose to estimate a physician’s current preference for a COX-2 inhibitor relative to a non-selective NSAID by using the type of the most recent new NSAID prescription written by that physician. According to this approach, if the last new NSAID prescription written by a physician was for a COX-2 inhibitor, then for the next patient the physician is classified as a “COX-2 prescriber.” Otherwise, he is classified as a “non-selective NSAID prescriber.” We also consider an alternative estimate of preference based on the historical proportion of a physician’s new NSAID prescriptions that were for a COX-2 inhibitor. This is an arguably more biased, but less variable estimate of preference.
To describe the statistical models used to estimate the effect of COX-2 inhibitors on GI toxicity, we introduce the following notation: let Z be the instrument indicating the last NSAID prescription written by the patient’s physician (if the last prescription was a COX-2 inhibitor, then Z=1; otherwise Z= 0); let Y be the disease outcome (GI complication within 60, 120, or 180 days); let X be a variable indicating whether a patient was actually exposed to a COX-2 inhibitor (X=1 if the patient was treated with a COX-2 inhibitor, otherwise X=0); and let all confounders be contained in the p-dimensional vector C. The parameter of interest is the risk difference (RD), i.e., the risk of a GI complication due to COX-2 use minus the risk of a GI complication due to non-selective NSAID use. We report a rescaled RD that is derived by multiplying the risk difference by 100 (i.e., the change in risk per 100 patients). We estimate this parameter in four ways: a crude RD, a multivariable adjusted RD, an unadjusted instrumental variable estimate of the RD, and an adjusted instrumental variable estimate of the RD.
The statistical approaches that we employ are based on standard linear regression models. Although not typically used for dichotomous outcomes, linear models are often preferable to non-linear models in the context of an instrumental variable analysis.38 To estimate the crude risk difference, we fit a simple linear regression model of the disease outcome on the exposure:
The adjusted risk difference is derived from least-squares estimation of a multivariable linear regression model that contains the exposure and the confounders:
Our instrumental variable estimators are derived via two-stage least-squares. For the unadjusted estimator, the first-stage involves estimating the expected value of the exposure given the instrument, Ê[X | Z]. This is done by taking the relative frequency of COX-2 exposure for Z=1 and Z=0. In the second stage, we fit the simple linear regression:
For this model, least-squares estimation of δ leads to the following estimator of the RD:
where Ê[Y | Z] is the relative frequency of the outcome within strata of the instrument. For the adjusted instrumental variable estimator, in the first-stage we estimate the expected value of the exposure given the instrument and confounders through least-squares estimation of the linear model
In the second stage, we use the predicted values from the first-stage and least-squares to fit the linear model:
These models were fit to the full study population using three different definitions of the follow-up period: 60, 120, and 180 days from the index date. We conducted a restricted analysis looking at only patients of primary care physicians (PCPs). By eliminating patients whose prescriptions were started by a specialist physician, we hoped to create a more homogeneous patient mix with respect to unmeasured variables. We also conducted an analysis restricted to patients with a diagnosis of osteoarthritis (OA) or rheumatoid arthritis (RA) since these were the patient populations included in the two major RCTs that examined the gastrointestinal effects of COX-2 inhibitors. 39,40
All analyses were done using the GENMOD procedure in SAS version 9.1 and the instrumental variable estimation procedure (ivreg) in Stata 7.0. Standard errors were estimated robustly to account for the clustering of patient-level observations within physicians.
We identified 50,548 new starters of either a COX-2 inhibitor or non-selective NSAID between January 1, 1999, and July 31, 2002. Of these 629 were excluded because of missing or invalid physician information associated with the pharmacy claim. Of the remaining 49,919, 17,646 (35.3%) were started on a non-selective NSAID while 32,273 (64.7%) were placed on a COX-2 inhibitor. The characteristics of the sample are given in Table 1.
There were strong associations between all measured patient-level characteristics and the actual use of COX-2 inhibitors (Table 2, column 1). Patients placed on COX-2 inhibitors were older and had more co-morbidities and were much more likely to have risk factors for NSAID associated GI toxicity, such as the use of warfarin, glucocorticoids, and a history of peptic ulcer disease and upper GI bleeds. While some of these characteristics were still associated with the instrumental variable in the full population (Table 2, column 2), the associations were strongly attenuated. The presence of some residual associations suggests that the instrumental variable assumptions are not completely satisfied. When we further restrict this analysis to patients of PCPs only, the associations between the instrument and the risk factors were changed in some cases from the association seen in the full population, but not in a substantial or systematic way (Table 2, column 3).
As required, the instrument was also related to treatment. Across the entire population, if the last prescription written by a physician was for a COX-2 then the probability that the next prescription would be for a COX-2 was 77.3%. On the other hand, if the last prescription written by a physician was for a non-selective NSAID then the probability that the next prescription would be for a COX-2 was only 54.5%. Among patients of PCPs only, these probabilities were 77.4% and 57.1%, respectively.
In Table 3, we present unadjusted associations between the actual treatment and the outcome and also between the instrument and the outcome. For all follow-up periods and restriction criteria considered, there was a positive association between actual COX-2 exposure and the occurrence of a GI bleed. In the unadjusted instrumental variable analysis, however, we observed a reduced risk of GI bleed among patients of physicians who had most recently prescribed a COX-2 inhibitor.
The estimated risk differences according to the various statistical approaches are presented in Table 4. None of the conventional analyses suggested a risk reduction in GI toxicity due to COX-2 inhibitor use relative to non-selective NSAIDs. For all estimates derived from conventional statistical models, the RD estimates were very close to 0 with narrow confidence intervals that included zero. Results from both the adjusted and unadjusted instrumental variable estimators provide evidence suggesting a protective effect of COX-2 exposure compared with non-selective NSAIDs. For the full population, the point estimate ranged from a risk reduction of 0.9 to 1.3 events per 100 patients. When the analysis was restricted to PCPs and when the time interval was extended to 180 days, the point estimates were only slightly attenuated, but the confidence intervals were wider and all contained zero. For the analysis restricted to patients with OA and RA, the estimated protective effect of COX-2 exposure was more pronounced, with point estimates of the risk reduction ranging from 1.5 to 2.1 events per 100 patients.
The conventional multivariable analysis was able to use more patients than the instrumental variable approach because the instrument is undefined for the first patient prescribed an NSAID by each physician during the study period. To explore whether or not sample differences could be influencing our results, we restricted the conventional analysis to the same set of patients used by the instrumental variable method. This approach yielded similar parameter estimates as the conventional method applied to the full sample. Our alternative instrumental variable analysis, which estimated preference using the historical proportion of a physician’s new NSAID prescriptions that were for a COX-2 inhibitor, yielded estimates that were attenuated to the null but still suggestive of a protective effect.
Contrary to RCT results showing that COX-2 inhibitors lead to a reduced risk of GI toxicity relative to non-selective NSAIDs, 39 40 our conventional multivariable analysis found no evidence of a gastro-protective effect attributable to COX-2 inhibitor use. Although we do not know the true degree of protection afforded by COX-2 inhibitors in our population, the absence of any apparent protective effect is more plausibly attributable to residual confounding by unmeasured patient characteristics rather than a total absence of such an effect.
In contrast to the conventional analysis, the instrumental variable approach proposed in this paper yielded evidence of a clinically significant protective effect due to COX-2 exposure, particularly for shorter term drug exposures. In Table 5, we compare the results obtained from the adjusted instrumental variable approach to results from the VIGOR and CLASS trials. 39 40 Since both of these trials studied populations with RA and/or OA, the instrumental variable estimates restricted to those patients are the most relevant. For all follow-up periods considered, the instrumental variable estimates are statistically similar to the trial results. However, for 60 and 120 day periods, the instrumental variable estimates are substantially larger in absolute magnitude. This suggests the possibility that in our population that is older and frailer than the trial population, COX-2 inhibitors have a greater protective effect. The attenuation of the instrumental variable estimate at 180 days is compatible with non-adherence and treatment crossover that is to be expected in an uncontrolled routine care setting.
Since instrumental variable methods are not commonly used in epidemiology, many readers will find the approach proposed in this paper to be counter-intuitive. One might question how the treatment assignment of one patient can be coupled with the outcome of another patient to estimate an exposure effect. For the case of NSAIDs, we have argued that treatment assignment depends on both physician preference and patient risk factors. The observation that NSAID prescribing depends partly on physician preference suggested the possibility that individual physicians could be used as the basis of a natural experiment. Since physician preference was unmeasured and dynamic, we used the last new NSAID prescription written as an estimate of the physician’s current preference. This led to a conceptual natural experiment in which the last prescription written becomes the “treatment arm assignment” for the next patient. An intention-to-treat (ITT) analysis of this natural experiment would be reasonable; however, ITT effect estimates are known to be biased towards the null. Instrumental variable estimators are an alternative to ITT methods and under certain assumptions can provide asymptotically unbiased estimates of treatment effects for both natural and randomized experiments in which treatment received is confounded. The unadjusted instrumental variable estimate (1) is an inflated ITT estimate of the RD where the inflation factor depends on the marginal probability of treatment in each arm.
Even though the estimates from the instrumental variable approach proposed in this paper are consistent with RCT results, the method and results should be interpreted with caution. In our example, physicians who are frequent users of COX-2 inhibitors are seeing higher risk patients, as evidenced by the association between some of the observed risk factors and the instrument. Although the instrumental variable estimate can be adjusted for these observed risk factors, it is reasonable to expect that the instrument is also related to the unobserved risk factors that confound the conventional analysis. In such a situation, some of what appears to be physician preference for a COX-2 inhibitor will actually be a clustering of high-risk patients within a particular practice. This clustering phenomenon leads to a violation of the instrumental variable assumptions and subsequent bias in the estimator. However, if there are important unmeasured confounders, then standard statistical estimates of the exposure effect will also be biased.
By using a variable that is strongly related to treatment, but only weakly related to unobserved risk factors, we hope to derive an estimate for which the confounding is strongly attenuated. We noted that the adjusted instrumental variable estimate was very close to the unadjusted estimate, indicating that the residual confounding in the unadjusted estimate due the association between the instrument and the measured risk factors was small. This leads us to speculate that the residual confounding due to the association between the instrument and the unmeasured risk factors may be similarly small. However, this cannot be verified from the data. Instruments that are weakly related to both treatment and the unobserved confounders can result in estimates that are more biased than estimates from conventional regression. Analytical work, simulation studies, and sensitivity analyses need to be done to understand how the bias in this instrumental variable approach compares with the bias of regression estimates under realistic assumptions about unobserved confounding and the clustering of those confounders within individual physicians.
Another potential source of bias in the instrumental variable method results from the possibility that a physician can influence the outcome in ways other than through the prescribing of an NSAID. For example, physicians who frequently prescribe COX-2 inhibitors may also be more likely to co-prescribe proton pump inhibitors (PPIs) for additional gastro-protection. In such a situation, the protective effect due to COX-2 exposure is partly attributable to use of a PPI. In principle it would be possible to remove this bias by creating additional treatment categories related to these combination therapies. Instrumental variable methods could then be used to simultaneously estimate the effect of each competing therapy.
For different drug exposure studies, alternative methods of estimating preference can be considered. If physician preference is not thought to change in time, preference can be estimated from the proportion of prescriptions written for a particular drug across the entire study period. When preference changes in time and there are sufficient numbers of patients per physician, more complex time-varying estimates of preference are possible. For example, the relative frequency of recently written COX-2 prescriptions could be used as an estimate of current preference. Given a set of alternative estimates of physician preference, the analyst should select the estimate that appears to be the most strongly related to the observed treatment among those estimates that are unrelated to observed patient characteristics.
Despite the limitations of the proposed instrumental variable analysis, the ability of this approach to attribute a clinically significant protective effect to COX-2 exposure similar to what was observed in RCTs is intriguing. When all the important confounders are measured, conventional statistical methods are the most appropriate way to analyze observational data in pharmacoepidemiology. For analyses of intended drug effects, however, it is often the case that many important confounders are unmeasured. Instrumental variable methods have been understudied in drug epidemiology and merit further consideration as a potential approach to deal with the vexing problem of confounding by unmeasured indication.
We acknowledge the thoughtful comments of Kenneth Rothman, Barry Kitch, Amanda Patrick, Til Sturmer, and two anonymous referees.