The main goal of the study was to evaluate an approach that combines ADE evidence from two different sources: the FDA's spontaneous reporting system (AERS) and MFBM, and to demonstrate that the two sources combined together improve the precision of ADE detection. In the first study, the performance of the MFBM is assessed in an independent dataset (DrugBank). In the second study the GPS algorithm is applied to AERS. We showed that the GPS algorithm is an excellent method to generate sets of drugs highly associated with rhabdomyolysis. However, within the sets of drugs generated by GPS, the score provided by the algorithm includes many false positives, because there are confounding drugs with good scores that are not causally related to the ADE, and therefore that set could be improved by filtering out some of the confounders. The application of the fingerprint model rationalizes the AERS signals selected with the GPS score and provides a smaller set of candidates with better enrichment factors. Although the fingerprint model performs similarly in an independent dataset, the fingerprint model applied to the GPS candidates offers a better true positive/false positive ratio due to the higher concentration of rhabdomyolysis drugs provided by the GPS algorithm. The combined method cannot increase the number of true positives but considerably reduces the number of false positives detected by AERS analysis. When we ignored drugs already in our rhabdomyolysis training dataset, we found a twofold precision improvement. The simplicity of the model assists in highlighting the etiology of the ADE by identifying structurally similar drugs, for which information is available and can be used to help understand possible causes, such as the mechanism of action. Although we used the AERS data in this study to test the method for the ADE rhabdomyolysis, the method could also apply to the use of electronic health records as an initial source of ADE detection as well as to ADEs other than rhabdomyolysis.
This system is not designed to replace the existing pharmacovigilance methods used to evaluate the importance of the signals, ie, study of the potential relevance of the signal or the biological plausibility, but to enhance the existing methods, providing additional information to make decisions. The simplicity of the model allows the researcher to detect the drug in the training dataset for a given ADE that is most similar to the AERS ADE drug candidate, which is useful for examining the reports and the available information to decide the importance of the signal.
When the MFBM was applied to a large set of drugs in the DrugBank database19
(see supplementary tables S2–S4, available online only), the results were similar to those previously reported.11
The predictability of fingerprint-based models by themselves is limited due to the complexity associated with the modeling of complex human clinical adverse events. The performance of our fingerprint-based model was probably affected by high molecular variability, and by the large number of potential targets and biological mechanisms associated with the clinical adverse event rhabdomyolysis. Nevertheless, the model is very useful for generating sets of drugs with good enrichment factors.
The performance of the model is highly dependent on the suitable construction of a training dataset for a given ADE, requiring a heterogeneity representation of the different structural classes of drugs highly related to the adverse event. In addition, training datasets can be updated continuously in order to enhance performance whenever a new compound is identified as causing an event. Although BIT_MACCS fingerprints have been shown to be successful in representing molecular structure,55
alternative methods could be explored that use different structural representations for molecules. Different types of models that use complex data analysis methods, such as neural networks or support vector machines, could be developed, but these methodologies would increase the complexity of the process.
When evaluating performance, we considered drugs that were not yet known to cause rhabdomyolysis as false positives. However, it is possible that some of these drugs actually do cause rhabdomyolysis but that the association has not yet been discovered. Therefore, it is possible that the true false positive rate of this method is lower than we determined.