In this pilot study, we evaluated the feasibility of applying Bayesian data mining methods to longitudinal survey and administrative claims data. These methods have been most commonly applied to spontaneous adverse event reports such as those from the AERS database. As the main endpoint of this project, we successfully adapted these methods for use with administrative claims data and demonstrated concurrent validity with results from a separate analysis conducted using traditional epidemiologic and statistical methods. In contrast to the usual methods of analysis for observational data, however, the data mining approach did not require us to prespecify the outcomes of interest and identified several important associations between coxib use and cardiovascular and cerebrovascular events out of a total of 259 potential outcomes.
As anticipated from prior research, the data mining analysis identified several outcomes associated with celecoxib and rofecoxib. For celecoxib, the only association that exceeded an EB05/95 ratio of 1.0 was events associated with an AMI diagnosis. Other outcomes for which associations were of borderline significance included acute cerebrovascular disease and coronary atherosclerosis diagnoses, as well as osteoarthritis and rehabilitation care. Similar patterns were observed for rofecoxib, although the strong association with AMI was not observed. As is known from analysis of adverse event reports, significant associations resulting from data mining methods will not only reflect potential safety concerns but also disease indications for which the drug is prescribed. For that reason, content knowledge must be applied in order to differentiate these two possibilities. As we observed in our study, all significant and near-significant results either were for indications for the drug (e.g. osteoarthritis, rehabilitation diagnoses) or for ischemic vascular events.
Recognizing that the actual findings from this feasibility study are of somewhat lesser interest than its methodologic focus, they nevertheless deserve mention. Our observation that celecoxib was associated with an approximately two-fold increased risk of AMI has been observed in some (11
) but not all (14
) studies. Although we observed a significant association between rofecoxib and stroke events, as has been found previously (16
), we did not observe a significantly increased risk of AMI. This study was likely underpowered to establish a significant relationship.
The principal strength of our study lies in its uniqueness, as we are not aware of prior reports that have demonstrated success in adaptation of Bayesian data mining methods to longitudinal claims data. The promise that these methods could be applied in an automated way to perform routine signal detection to identify unrecognized adverse drug events soon after product launch using administrative data would be a substantial advance and would fill an important gap in postmarketing drug safety surveillance. We do not view these methods as ever replacing welldesigned postmarketing RCTs or observational studies. Rather, we believe them to be complementary to traditional methods by providing a tool adapted to longitudinal data sources such as claims data by which to identify safety signals that need to be pursued.
Despite our initial results from this pilot project, we are cautious with respect to the broad applicability of these methods without further research regarding their validity, precision, and power. In evaluating such efforts it is always desirable to have a gold standard set of results to which data mining analyses can be compared. We intentionally chose to evaluate coxib safety given the recent furor and numerous studies published on this topic. However, given the lack of consensus on even this subject, well designed simulation studies, or a pooled analysis of randomized controlled trials data where randomization controls for both measured and unmeasured confounders, may be the optimal next steps to ensure the robustness of our adaptation of data mining methods.
We used the AHRQ’s CCS classification system to group similar ICD9 codes together into 259 unique groups. The data mining analysis evaluated all of these outcomes simultaneously and did not require foreknowledge of which of the 259 groups were of particular interest. However, not all important events have specific enough ICD-9 codes to be useful for a claims-based analysis, much less fit into a well-defined CCS category. Moreover, the events of greatest interest in our analysis, and those most likely to represent true safety problems, were AMI and acute cerebrovascular disease. These conditions are relatively homogeneous with respect to the ICD-9 codes included in them and appeared as the primary diagnosis from a hospitalization. Different outcomes that are included in a more heterogeneous event group may be masked if they are rare compared to somewhat dissimilar events also included in that group. Nevertheless, it is possible to use a different event classification system that includes more or different groups; up to 750-1000 event groups is likely to be an upper limit, depending on the size of the data source.
We acknowledge a number of limitations of this study. First, neither pharmacy data from an administrative claims database, nor medication information collected during in-home interviews from the MCBS, accurately reflect actual medication taking behavior or precisely identify the start and end dates of drug use, and we lacked information on drug dose. Additionally, the sample size of the MCBS is relatively small, and this may have limited our ability to detect some important associations. Although we evaluated a number of potential confounders, and the MCBS collects data on covariates not routinely found in claims databases (e.g. race, BMI, smoking status, education), we recognize the possibility for residual confounding. However, of greater interest than our ability to answer content-related questions were our concordant findings between the Bayesian compared to traditional epidemiologic methods. We would expect the same sources of confounding to be operant using both methods, and our finding of concordant results between the two parallel methods is reassuring. Finally, we do not expect that this or any method will be adequate to detect significant increases in very rare “sentinel” events (e.g. Stevens-Johnson syndrome) when they occur, but these should nevertheless be pursued based on clinical relevance.
In summary, results from this pilot project demonstrated the feasibility of using Bayesian data mining methods to analyze administrative claims data. We also showed concurrent validity between the data mining results and traditional methods in the analysis of one particular outcome, AMI. These techniques appear to hold substantial promise to fill a large niche in the evaluation of drug safety for which the available tools for pharmacovigilance are few in number. However, despite these encouraging results, these approaches will require further validation before they can be recommended for widespread use.