|Home | About | Journals | Submit | Contact Us | Français|
Genomewide Association Studies (GWAS) have identified thousands of consistently replicated associations between genetic markers and complex disease risk, including cancers. Alone, these markers have limited utility in risk prediction; however, when several of these markers are used in combination, the predictive performance appears to be similar to currently many available clinical predictors. Despite this, there are divergent views regarding the clinical validity and utility of these genetic markers in risk prediction. There are valid concerns, thus providing a direction for new lines of research. Herein, we outline the debate, and use the example of prostate cancer to highlight emerging evidence from studies that aim to address potential concerns. We also describe a translational framework which could be used to guide the development of a new generation of comprehensive research studies aimed at capitalizing on these exciting new discoveries.
Genome wide association studies (GWAS) provide a comprehensive and unbiased assessment of Single Nucleotide Polymorphisms (SNPs) across the genome, and test for their association with disease phenotypes. This study design has resulted in the identification of novel associations of SNPs with a variety of complex diseases such as cancer. The online reference, “A Catalog of Published Genome-Wide Association Studies” now lists approximately 375 SNPs associated with 27 cancer types . Associations listed in the catalog meet stringent criteria for genome-wide statistical significance and have been validated in independent study populations, thus reducing the likelihood of chance associations.
Although the risk alleles of these SNPs are common (>5%) in the general population, each has a small individual effect on disease risk with Odds Ratios (ORs) of about 1.2 [2,3]. However, larger ORs are observed when multiple SNPs are combined, as we have shown in prostate cancer (PCa) and others have shown in breast cancer [4-7], thus supporting their potential use in risk prediction.
Associations identified by GWAS add another class of risk factor that may be particularly useful for diseases in which the majority of cases are not explained by existing risk factors, such as PCa. Furthermore, these germ-line genetic markers are unique in that they can be objectively and accurately measured, do not change with age, and always precede phenotypes and diseases. The discovery of these risk-associated SNPs has spurred efforts to develop clinical applications, although there is controversy.
Although the highly significant and confirmed association findings of GWAS are not in question from a statistical perspective, there are divergent views on the clinical validity and clinical utility of the associated SNPs [7, 8]. There are ethical and technical arguments, both for and against clinical applications of these SNPs.
Opponents argue that ethically, the balance of benefit versus risk (beneficence/non-maleficence) is unclear (Box 1). For example, if high-risk results create excessive anxiety, or if low-risk results create a false sense of security, either of these may lead to inaction or over-reaction in subsequent medical decisions. Additionally, genetic discrimination is a concern. Overall, such testing information may be detrimental to health and well being, if not at the population level, then perhaps at the individual level.
Underscoring these ethical concerns, a variety of technical concerns have also been raised (Box 1). First, because many of these associated SNPs are in non-coding and intronic areas of the genome, the molecular mechanisms by which most of these SNPs act is poorly understood, thus leaving their causal role in question. Second, their predictive performance is generally modest as estimated by the area under the curve (AUC) statistic of the receiver operating characteristic (ROC) [9, 10]. Third, it is believed that only a small fraction of all common risk-associated SNPs have been identified to-date, and as a result, current risk estimates are likely to change as the risk prediction models are improved. This could contribute to future risk reclassification and result in varying interpretations at different times. Fourth, the health benefits are not clear; for example most of these SNPs cannot distinguish between clinically indolent and aggressive cancer, leading to concerns for overdiagnosis and overtreatment of indolent disease [12-15].
Proponents generally focus on the consistent associations between SNPs and disease risk, and independent predictive performance when combining multiple risk-associated SNPs, as evidence in support of SNP-based predictive testing. In terms of ethics, proponents emphasize autonomy, meaning the right of individuals to make informed decisions to access their own personal genetic information, free of paternalism. As for genetic discrimination, proponents counter that safeguards are already in place, including the Genetic Information Non-discrimination Act (GINA). Covering both ethical and technical issues, it can be argued that patients have the ability to seek third-party interpretation of results and to discuss appropriate medical actions based on their results, because genetic testing and genetic counseling are already well established aspects of medical care.
Despite concerns, the commercial testing market has moved forward with a rapid expansion of offerings of multi-SNP tests based on GWAS results, including tests on a direct-to-consumer (DTC) basis . The apparent commercial demand for these tests suggests a subset of patients has a need for the information. At any rate, these commercial offerings represent the most positive view of SNP-based testing; that they are ready for use by individuals.
However, change may be coming. Although the newly available tests have experienced very little oversight within the current “patchwork” of federal and state legislation pertaining to genetic testing and DTC testing [17, 18], the US FDA has recently asserted their authority to regulate DTC genetic testing kits on the basis they are medical devices. From May 2010 to August 2011, the USFDA has sent letters to at least 24 companies offering such tests; all were asked to provide evidence of either regulatory clearance or justification for exemption .
We remain at an early stage of judging the clinical validity and utility of GWAS SNPs and the subsequent multi-SNP tests. Current methods to assess the performance of genetic markers may be misleading, resulting in premature, sometimes incorrect, interpretations regarding potential utility or lack thereof. In addition, too little attention has been paid to evaluating the potential health benefits of the new risk prediction tests. As we describe in the following, empiric evidence from prostate cancer studies is beginning to address some of the primary concerns.
While GWAS findings have provided many novel biological insights that serve as leads for additional studies, it is well known that most GWAS associations cannot be explained by known causal mechanisms. Traditional molecular genetics approaches, as well as new methods and technology, are expected to illuminate causal links between diseases and the regions that harbor SNPs associated via GWAS. For example, we recently published a study demonstrating the potential interaction between the androgen receptor binding sites and many of the prostate cancer associated SNPs, suggesting an androgen dependent pathway by which many of these SNPs act . Future novel study designs such as these together with a more comprehensive assessment of the genome and a better understanding of the role of non-coding regions will result in an appreciation of the functional significance of these SNPs. In particular, next-generation sequencing and proteomics could reveal the functional impact of these sets of variants, which will be important to understand etiology and to eventually develop targeted therapies.
However, the process of functional characterization and therapeutic development may require many decades to complete, and should not be an impediment to risk assessment research. Given the massive public health impact of common diseases such as cancer, it could be argued that we should move forward with utilizing the best currently available information. Although all of the biological mechanisms are not yet understood, we already know that GWAS findings represent true associations in populations, based on consistent observations across independent study populations; this supports research to evaluate the validity and utility of these SNPs for risk prediction. Risk-assessment testing does not preclude additional mechanistic research into the causal role of current SNPs or the discovery of additional variants; rather results from GWAS should continue to stimulate additional research in closely related fields.
A major criticism of GWAS SNPs is the modest level of risk prediction, as assessed by AUC . In the case of PCa, an AUC of 62% can be obtained when using the very best baseline clinical parameters in combination (age, family history, free/total PSA ratio, number of cores at pre-study entry biopsy, and prostate volume) to predict PCa among repeat biopsies in the REDUCE study, which is 12% higher than chance (50%) (unpublished data). When 33 PCa risk-associated SNPs (Table 1.) are added to these clinical parameters, we observed a 66% AUC that is statistically significant. Although this AUC only represents a 4% absolute increase, it represents a 33% (4%/12%) relative improvement over the best clinical risk prediction model.
When assessing predictive performance, a more fundamental question is whether the findings have clinical meaning, such as the detection rate. Unfortunately, AUC is an abstract value that has no inherent clinical meaning. AUC assesses the ability to distinguish risk across all risk strata. However, if the goal is to identify men at considerably elevated risk, then methods based on a risk cutoff, such as positive predictive value (PPV), offer more clinical meaning. When we evaluated 28 PCa risk-associated SNPs within a Swedish population-based PCa case-control study (CAPS), the PPV of this test was 36% when 3-fold increased risk over population median risk was used to define high risk; this is comparable to PSA screening based on a 4 ng/mL cutoff . This result has clinical meaning, because PPV is the disease detection rate among subjects predicted to be at risk based on disease biomarkers. This also reinforces our belief that AUC should not be viewed in isolation, but rather, in context.
The discovery of additional risk-associated SNPs, and their later inclusion in risk prediction models is another source of concern because of the potential impact on individual test results. Risk-associated SNPs discovered to date most likely represent common genetic variants with a relatively larger effect that were relatively easy to identify. However, additional risk-associated SNPs with a smaller effect and/or rare risk-associated SNPs will almost certainly be discovered in the future from new GWAS, meta-analyses of GWAS, and re-sequencing studies. The question is whether the addition of smaller-effect and/or rare risk-associated SNPs would significantly improve the predictive performance of multi-SNP risk prediction models. Statistical modeling has suggested additional genetic markers may significantly improve predictive performance . However, an empirical analysis comparing the first 5, 14, and 28 PCa risk-associated SNPs discovered from GWAS suggests that PPV reaches a plateau after the most important SNPs have been included in the risk prediction model . While this result is promising, the impact on risk prediction and reclassification due to the addition of new genetic markers warrants further study, particularly as new high risk variants are identified by whole-genome sequencing.
Even if GWAS SNPs allow accurate prediction of overall risk, questions of health benefit remain. This is particularly important in a disease such as PCa, where most PCa tumors are not aggressive or life threatening, and thus treatment can cause more harm than good. Unfortunately, most GWAS SNPs identified to date are not associated with aggressiveness or survival, and are unable to predict these clinical features. This is not surprising, given the original studies primarily used early stage cases for association discovery and validation. Recent reports identified SNPs (rs4054823 at 17p12, rs6497287 at 15q13, rs2735839 at 19q13, and rs7679673 at 4q24) specifically associated with PCa progression or survival [15, 23-25]. Confirmation of these initial results is inconsistent and likely limited by different definitions of progression, differing study designs, and small numbers of advanced PCa cases. However, the initial findings offer important leads and design guidance for future studies in this important area.
When assessing the potential health benefit of SNP-based tests, a thorough understanding of the complexity of disease is critical. For example, SNP association with PCa is consistent with an alternative hypothesis of association with higher PSA levels and not PCa risk per se (i.e. PSA detection bias) [26-28]. This is because most PCa patients have higher PSA levels than controls in case-control studies conducted in developed countries where PSA screening is commonly used. Only well-designed prospective studies or clinical trials such as Prostate Cancer Prevention Trial (PCPT) and Reduction by Dutasteride of Prostate Cancer Events (REDUCE), where all study subjects undergo prostate biopsy regardless of PSA levels and other clinical parameters, can be used to dissect whether these SNPs are associated with PSA level, or PCa risk, or both. [29, 30]. As described later, we are actively pursuing this line of research.
Health benefit may also be demonstrated by comparative analyses, which provide context to new findings on risk-associated SNPs. For example, both family history and GWAS SNPs reflect the genetic risk of individuals, and each modestly predicts an individual's genetic risk for most complex diseases. Family history is promoted by the U.S. Surgeon General and the CDC, and is widely used in clinical settings to assess individual cancer risk and to guide clinical management [31, 32]. For PCa, the strength of association with the disease is stronger for PCa risk-associated SNPs than family history surveys, in prospective studies  and in the REDUCE study (unpublished data). We recognize that family history has advantages, such as the ability to obtain basic information by questionnaire or interview, versus SNP-based markers that require a laboratory test. However, neither is a perfect assessment of risk, and thus the overall balance of risks and benefits is key in determining health benefit. If the balance of risks and benefits of family history are acceptable for clinical risk assessment, then genetic score should likewise be considered as a compliment to family history, for improved PCa risk prediction.
With valid arguments rebutting the raised technical concerns, it may be time to consider how these discoveries may be further evaluated and then brought from the bench to the patient. For this we again use prostate cancer to highlight one possible translational research (TR) framework that will allow us to capitalize upon the exciting results from multi-SNP tests.
Contemporary notions of TR have extended this definition to include translation into the community, while also defining a series of intermediate phases that comprise TR [34-38]. Genomic TR is essentially a progression through several stages; T1) confirms association and establishes clinical validity; T2) clinical utility; T3) practice-based implementation research; and finally T4) population/community wide outcomes assessment.
After initial discovery of candidate associations, T1 research verifies associations and assesses analytical and clinical validity. A fundamental goal of T1 is to minimize the possibility of spurious associations due to both statistical and clinical causes. Statistical causes are addressed by utilizing independent populations with large numbers of samples for confirmation analyses, reducing the possibility of false positives due to chance. Clinical sources of spurious association are difficult to address in case-control studies, as described above for PSA detection bias, and may be addressed by prospective studies in which potential bias is minimized. T1 research aims to answer questions such as, “are these SNPs truly associated with PCa, or rather with PSA levels that lead to the detection of most PCa cases”? By answering this type of question, we can establish the validity of associations.
T2 research addresses whether the valid associations from T1 have clinical utility. The necessary approaches in T2 include prospective studies, either observational or interventional (clinical trials), and comparative effectiveness research (CER). Unfortunately, very few of the initially promising associations are tested in prospective studies that can pave the way through the T2 phase [39-40], in part because prospective studies are costly and require many years. One efficient approach is to utilize previously completed prospective studies, by examining predictors at baseline (e.g. clinical parameters and genotypes) in relation to outcome data. This approach is particularly appropriate for genetic studies in which genetic markers are practically blinded to patients and observers, reducing potential bias. CER is defined by the Institute of Medicine as, “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care” . By comparing genomics tests to existing clinical markers, CER gives clinical context to statistics. If T2 research is successful, then we have answered questions such as “how does the PPV of a combined SNP test for PCa risk compare to the PPV of family history or PSA”? Clear answers to these types of questions can inform the public discourse and the subsequent development of professional guidelines and public policy.
T3 research aims to maximize the utility that is established by T2 research, by examining the practical issues impacting clinical usage. These studies could examine physician motivation to offer tests, patient uptake of tests, patient interpretation of results, physician recommendations based on tests, and the downstream decisions of those receiving (or not) test results. T3 research may also explore the differential impact when testing is applied in various clinical settings (e.g. private practice versus specialty or academic centers) or implementation scenarios (e.g. population screening versus targeting to high risk families). Because T3 research is so wide-ranging, it may be necessary to direct research toward the most pressing issues in clinical implementation. Again, CER can be used to weigh a genomic test against existing methods. T3 research can ask, “do genomic test results for PCa risk alter perception and accordingly patterns of PSA screening or willingness to opt for chemoprevention”? Answers to these types of questions are intended to help capitalize on the potential positive health impacts of tests, and may further guide the development of public policy.
T4 research focuses on health outcomes amongst a wide community or population, following the introduction of a new intervention. Going beyond the well defined groups of patients typically studied in T3 research, T4 examines real world impact. For example, when new genomic tests are introduced, it is possible to monitor disease incidence using population based registries; if a decrease is observed, then this may be attributable to the test, particularly if evidence from T1 through T3 would predict the observed effect. Formal cost effectiveness analysis is also an important component of T4, utilizing real world data on cost, test usage, and outcomes. Questions addressed in T4 could include “following the widespread introduction of a new genomic risk assessment test, how many cases of PCa are prevented in a population, and at what financial cost”? By answering these questions, we can monitor whether the test is having the expected effects.
Currently, NCI-funded projects are heavily skewed toward T1, the early discovery phase of TR . If we are to reap the full benefit of the heavy investment in discovery approaches such as GWAS, then it is imperative that scientists and clinicians commit to carrying out T2, T3, and T4 research. Fortunately the NIH intends to promote translational research that is aimed at areas in which the FDA likewise intends to step up the regulation of tests, and this will promote research across the TR continuum .
Here, we outline our approach to a new study that covers many stages of TR to evaluate a multi-SNP panel in PCa risk prediction (figure 1). First, we will assess the clinical validity of each SNP association identified by GWAS as well as the analytic validity of risk prediction for PCa based on combined SNPs (T1). These analyses will utilize the existing REDUCE trial, a large prospective cohort . This study population is well suited to assess validity because 1) all study subjects were felt not to have PCa at study entry based on a prior negative biopsy, 2) all subjects underwent protocol-required biopsies at years two and four, regardless of PSA levels or other clinical variables, and 3) associations of SNPs with PCa have not been previously evaluated in this study population. The design of this population allows us to independently confirm SNP associations, as well as the more critical issue of whether each SNP is associated with PSA rather than PCa per se. This also allows us to assess the analytic validity of a risk model that is based on the combined panel of SNPs.
After establishing analytic and clinical validity, the next goal is to understand the clinical utility of the SNP panel, i.e. whether it adds value to existing clinical markers in predicting positive prostate biopsy (T2). Using a CER approach, we will compare the performance of the combined SNP panel with existing clinical markers in predicting positive biopsy, using clinically meaningful measurements such as PPV. Again, this is possible because both clinical parameters and genetic markers at baseline are available and all study subjects were systematically biopsied in the REDUCE study. In addition, the REDUCE study randomized subjects to dutasteride versus placebo, providing an opportunity to explore whether men at higher estimated PCa risk based on SNPs and family history respond better to chemoprevention with dutasteride.
Our next step is to evaluate some of the practical issues impacting clinical usage (T3). This will be accomplished by a new prospective randomized clinical trial to assess the impact of the SNP panel on risk perception and behavioral outcomes. Subject recruitment in the trial consists of men age 40-49 years, Caucasian, and never had prior PSA screening or PCa diagnosis. Baseline surveys will collect data on their perception of PCa risk, numeracy, and health attitudes. Subjects will then be randomized, with half to receive a standard risk assessment (family history and age), and the other half to receive a risk assessment based on SNPs plus standard risk assessment. Immediately following the disclosure of the risk assessment based on these two methods, we will assess the perception of risk in each group. After three months we will evaluate behavior outcomes such as discussion of results with family members, engaging in medical appointments, discussion of PCa screening options with a medical provider, engaging in PCa screening such as PSA and DRE, and uptake of preventative measures such as chemoprevention. By comparing the two randomization groups, we can measure the impact of the SNP panel on risk perception and behavioral outcomes. While there are many additional aspects that will remain to be addressed in future T3 studies, this prospective randomized clinical trial represents an important first step.
Finally, we will begin to examine the potential health outcomes if the SNP panel is implemented in clinical practice (T4). While realizing that it is too early to assess the impact of SNP-based risk prediction on incidence and mortality of PCa, we will focus on a cost-effectiveness analysis of genomic-targeted chemoprevention. Utilizing the data from T2 (PCa reduction rate using genomic-targeted and non-targeted approaches) and T3 (willingness to opt for chemoprevention based on perceived risk), we can estimate the cost of obtaining one quality adjusted life-year. These results will be important in understanding the likely impact of SNP-based risk assessment on medical practice and the community level outcomes.
Based on sets of SNPs identified by GWAS, there exists an opportunity to estimate disease risk earlier and more accurately. Although DTC genomics companies are already up and running, the divergent views on ethical and technical issues highlight several key areas in which additional research is still needed. We believe this early-stage technology holds great promise, and needs to be fully evaluated and developed. Additional work remains if we are to responsibly bring GWAS discoveries to bear on health outcomes. For this to happen, all stages of the TR continuum need to be pursued, which means placing additional emphasis on the later stages of TR. To this end, there are promising early results from a few studies that suggest individuals have used genomic profile information to make positive changes in their behavior [43-46]. It is crucial to extend these initially promising results to outcomes that are further down the pathway to clinical outcomes. As scientists and clinicians, we should embrace the opportunity to pursue TR as a means to improve human health.
This work was partially supported by National Cancer Institute grants CA140262-02 and CA148463-02.
Conflict of interest statement: No conflict of interest was declared