Recently, there has been an increased interest in “individualized medicine” in the research and treatment of cancer. This in turn has increased awareness of the study of pharmacogenetics and pharmacogenomics in cancer research. Pharmacogenetics is the study of the role of inheritance in individual variation in response to drugs, nutrients and other xenobiotics [1
]. In this post-genomic era, pharmacogenetics has evolved into pharmacogenomics, a discipline that has been heralded as one of the first major clinical applications of the striking advances that have occurred and continue to occur in human genomic science (http://www.fda.gov/cder/genomics/genomic_biomarkers_table.htm
]. Pharmacogenomics is the study of the influence of genetic variation across the entire genome on drug response (e.g., efficacy, toxicity) in patients.
In the late 1980s, the National Cancer Institute (NCI) developed a collection of human tumor cell lines (NCI60) from a variety of common solid tumors, such as lung, colon and breast, for anti-cancer drug screening [8
]. Recently, pharmacogenomic research has incorporated non-tumor cell-based model systems [9
]. The use of these non-tumor cell lines are gaining in popularly due to greater availability of samples. Hypotheses generated with the cell-based model system can then be tested in individuals treated with the drug, or followed-up in functional studies.
Currently, investigation of the genomic relationship with drug concentration endpoints from cell lines is often completed by either analyzing a drug concentration endpoint measured at a single drug dosage or a summary measure of the dose-response curve of the concentration endpoints (e.g., dose that inhibits 50% of cell growth, GI50). Drug concentration endpoints are any measurable cellular phenotypes that are related to drug concentration, one example being cytotoxicity (measured as the percent of surviving cells after exposure to the drug). An analysis of the impact of genetic variation on all aspects of the dose-response curve may lead to more insight into the pharmacogenomics of a particular drug. For example, by investigating the impact of genetic variation on the slope of the dose-response curve, one may be able to determine genetic variation responsible for differences in therapeutic index, the comparison of the amount of a therapeutic drug that causes the therapeutic effect to the amount that causes toxic effects, between subjects treated with the drug.
Although numerous methods exist to evaluate subject-specific effects on nonlinear dose response curves, application to pharmacogenomic studies has been lacking. One possible method, used in the past for the analysis of population pharmacokinetic studies, is a Bayesian hierarchical nonlinear model fit using Markov chain Monte Carlo (MCMC) [13
]. Bennett, Racine-Poon and Wakefield (1996) give a nice overview of MCMC methods for hierarchical nonlinear models in Markov Chain Monte Carlo in Practice
]. For a review of non-Bayesian estimation of nonlinear mixed effects model, the reader is refer to Davidian and Giltinan [16
Over the last few decades, applications of Bayesian methods by Markov chain Monte Carlo (MCMC) [18
], and in particular the Gibbs Sampler [20
], have increased with the advancement of computers and computational methods, particularly with their application to genetic data [21
]. Use of a Bayesian hierarchical nonlinear model, allows determination of the impact of genetic variation on all aspects of the dose-response curve, along with possible incorporation of prior knowledge into the model. In addition, by analyzing the data within a hierarchical nonlinear model, researchers are able to partition the variation in response to the drug into: genetic variation, unexplained “between-subject” variation, and within-subject variation (inter-observation-time variation, and residual random error). Understanding the magnitude of these various sources of variation has important clinical implications. For example, a large amount of unexplained between-subject variation in cytotoxicity can imply that a drug will be difficult to use in a heterogeneous population because of uncontrolled toxicity. Hence, the Bayesian hierarchical nonlinear model can offer insights into the understanding of the pharmacogenomics of a particular drug.
This paper outlines a Bayesian hierarchical nonlinear model for analysis of pharmacogenomic-cytotoxicity studies involving the use of a cell based “model system”[22
]. We begin by describing the motivating pharmacogenomic study involving the anti-cancer drug gemcitabine, used to treat a variety of solid cancer tumors such as pancreatic and breast cancer. This pharmacogenomic study will be used in the application of a Bayesian hierarchical nonlinear model to determine if mRNA expression for genes within the gemcitabine pathway is related to cytotoxicity. In addition, three simulation studies are presented comparing the findings from the hierarchical nonlinear model to the results from the analysis using the GI50,
a commonly used summary measure of the curve.