|Home | About | Journals | Submit | Contact Us | Français|
Predictive and prognostic biomarkers offer a potential means to personalize cancer medicine, although many reach the marketplace before they have been validated, and their adoption is often hindered by variable clinical evidence. Because of this variability in supporting evidence, clinical practice guidelines formulated by panels of subspecialty experts may be particularly important in guiding stakeholders’ acceptance and use of new personalized medicine biomarker tests and other nascent technologies. This article provides a structured review of the clinical evidence supporting 4 contemporary biomarker tests in colorectal cancer: K-ras and B-raf mutation analyses, mismatch repair protein testing, and the Oncotype DX Colon Cancer Assay. All 4 tests have been evaluated for guideline inclusion by the NCCN Guidelines Panel for Colon Cancer. This case study shows significant variability in the level of clinical evidence associated with these tests. In the cases of B-raf and mismatch repair protein testing, the available evidence is also inconsistent as it pertains to the specific NCCN guideline recommendation. Based on this uncertainty in the evidence base, the authors conclude that expert clinical judgment, experience, and consensus may be more heavily weighted than published clinical trial data in the evaluation of new personalized medicine biomarker tests. Potential implications of this conclusion and future directions for research are discussed.
Predictive and prognostic biomarkers offer the potential for personalized therapy in oncology. Despite a multitude of publications in the oncology literature identifying potential biomarkers, however, only a select few are recommended for use in clinical practice.1–5 A challenge to the adoption of personalized medicine biomarkers in oncology is the lack of a standardized validation process because of the heterogeneity of tumor types, treatments, and tests themselves.1,3,6–8 Validation studies also may be limited by small data sets, long time intervals required to achieve end points, statistical complexity, cost, and the bias inherent in retrospective analysis.1,4,5,9–11 Randomized, controlled data are required to clearly define which markers are predictive and which are prognostic, because single-arm studies can be misleading.1,4 New biomarker tests may become available before the arduous validation process is complete, requiring practitioners to navigate the competing pressures of the existing data, the “blogosphere” recommendations, cost, and the patients’ best interests. This paradigm exists in stark contrast to the process of new drug development, in which the FDA requires rigorous demonstration of safety and efficacy before granting approval for commercialization.11–13
Perhaps more so than in other medical specialties, oncology-focused consensus guidelines developed by multidisciplinary expert panels influence the decisions of clinicians, payers, and policy makers.14–22 Guideline adherence has been associated with improved outcomes in some studies.14,15,18,23 Worldwide, multiple cancer organizations produce practice guidelines. In the United States, ASCO, NCI, and NCCN publish clinical practice guidelines widely used in clinical practice. Among these, NCCN offers the most frequently updated practice guidelines specific to most common tumor types, and routinely include comprehensive diagnostic, risk stratification, treatment, and surveillance algorithms.24–26 The recent decision of the Centers for Medicare & Medic-aid Services (CMS) to recognize the NCCN Drugs & Biologics Compendium (NCCN Compendium) underscores the integral role of these guidelines in the oncology community.27
The published methodologies of the ASCO, NCI, and NCCN guidelines do not specifically describe the methods for evaluating nascent technologies such as new biomarkers, but they do acknowledge that expert clinical experience and judgment may be required when data pertaining to a specific intervention are incomplete.28,29 Currently, the NCCN guidelines are the only United States guidelines with recent updates reflecting new biomarker data across multiple malignancies. It stands to reason that timely, interim synthesis of the available evidence by expert guidelines panels may play a particularly important role in guiding practitioners’ use of new personalized medicine biomarkers with varying levels of supporting data at commercial release.
This article presents a case study of biomarker integration into the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) using examples from colorectal cancer, a disease in which molecular markers of prognosis and treatment efficacy have been studied extensively.30,31 The objective for this case study is to describe and compare the available clinical evidence for 4 heterogeneous but contemporary biomarker tests in colorectal cancer: K-ras and B-raf mutation analyses, mismatch repair protein (MMR) testing, and the Oncotype DX Colon Cancer Assay (Genomic Health, Inc., Redwood City, California).24 Each of these biomarker tests has been considered for guideline inclusion by the NCCN Guidelines Panel for Colon Cancer. The authors hypothesize that varying levels of and inconsistency in the clinical evidence supporting these biomarkers could impact the process of biomarker adoption by consensus guidelines, which rely on a combination of both clinical evidence and expert opinion.
K-ras and B-raf mutation analyses and MMR testing were selected as examples of contemporary bio-marker tests that recently have been incorporated into the NCCN Guidelines for Colon Cancers.24 The NCCN Categories of Evidence and Consensus are provided in the guidelines, available online, at www.NCCN.org. The Oncotype DX Colon Cancer Assay was selected for this case study because of its recent entry into the marketplace for use in stage II colon cancer risk assessment. This test is currently not explicitly referenced in the NCCN Guidelines, but the data are in the public domain. The NCCN Guidelines Panel for Colon Cancer has evaluated the body of evidence for multi-gene assays in stage II colon cancer with the conclusion that the data are insufficient to support the use of assays such as the Oncotype DX Colon Cancer Assay for treatment decision-making.32,33 This case study is not intended to be a comprehensive review of all biomarkers with potential relevance to colorectal cancer; the 4 selected examples represent a purposive convenience sample.
This case study uses a structured review of the 4 selected biomarkers. Searches were targeted to capture data pertaining to the specific NCCN guideline recommendation for each biomarker test.
A descriptive approach to characterizing the evidence was used because the body of evidence supporting the selected biomarkers is relatively small and heterogeneous and because both the NCCN and ASCO guidelines methodologies use similar qualitative analyses.28,29 For the published clinical studies identified, the study design, end points, and numbers of patients or specimens are summarized in table format. A review of the assay methods, analytic validity, and technology for each biomarker test is beyond the scope of this case study.
The NCCN Guidelines recommendation, date of inclusion if applicable, and category of evidence and consensus for each of the selected biomarker tests in colorectal cancer are presented in Table 1. For each of these biomarkers, the clinical evidence is summarized descriptively later. Literature search results are also presented for each biomarker (Tables 2–5) to provide specific references according to study type, along with end points and numbers of patients. Table 6 compares the evidence across the 4 biomarkers and summarizes the conclusions regarding the consistency of the cumulative evidence for prognostic value, predictive value, or both for each test.
The K-ras gene is a downstream target of epidermal growth factor receptor (EGFR) signaling. Activating mutations in codon 12 or 13 of this gene are present in approximately 30% to 40% of colorectal cancers.34 Retrospective subset analyses of tumor tissue samples from small clinical trials initially showed that tumor K-ras gene mutations are associated with lack of response to cetuximab and panitumumab, the 2 EGFR-targeted monoclonal antibodies approved for use in colorectal cancer.35–61 The strength of this association was later substantiated in retrospective analyses of patients treated in 6 large randomized studies.62–67 Search results are summarized in Table 2.
The evidence shows with great consistency that patients whose tumors harbor a mutation in the K-ras gene do not benefit from cetuximab or panitumumab, whether in monotherapy or in combination with chemotherapy, whereas those whose tumors are wild-type have significantly higher response rates and longer survival. These findings clearly establish tumor K-ras mutation as a predictive factor for non-response to EGFR-targeted therapy. Data are mixed regarding whether K-ras mutation is a negative prognostic factor independent of treatment with EGFR-targeted therapy.61,65–70
K-ras mutation analysis was included in the NCCN Guidelines as a Category 2A recommendation in 2008 before the package label for either antibody was changed and before FDA acknowledgment of K-ras testing as a standard. The guideline inclusion was based on the publication of a retrospective subset analysis of the randomized, phase III study of panitumumab versus best supportive care, and after national and international presentation of results from similar unplanned retrospective analyses of subsets from the randomized, phase III CRYSTAL and CAIRO2 trials, the randomized phase II OPUS trial, and a multicenter, multinational randomized phase III trial of cetuximab versus best supportive care.62,71–74
The B-raf gene encodes a protein kinase downstream in the K-ras pathway. Activating mutations in B-raf at the V600E site are present in approximately 10% of patients with metastatic colorectal cancer and seem to be mutually exclusive with activating K-ras mutations.40,68,75,76 Soon after publication and presentation of the data establishing K-ras as a predictive factor, a small, retrospective series suggested that patients with K-ras wild-type with mutations in B-raf at the V600E site who were treated with cetuximab or panitumumab had significantly poorer outcomes than those without the V600E mutation.40 These data were reinforced by 3 other small, retrospective, uncontrolled studies with similar findings.35,46,50
However, subset analysis of the randomized, phase III CAIRO-2 study was not consistent with these findings.75 In both treatment arms of CAI-RO-2, a B-raf V600E mutation was associated with shorter progression-free survival without any difference in response rate compared with wild-type tumors, suggesting prognostic as opposed to predictive value. This result was consistent with a retrospective analysis of samples from the randomized PET-ACC-3 adjuvant study of patients with locoregional disease and with subset data from the CRYSTAL trial presented at the ASCO 2010 Gastrointestinal Cancers Symposium.68,76 Among the 625 evaluable patients in the CRYSTAL trial whose tumors were nonmutated for the K-ras gene, B-raf mutation (present in 59 patients) was shown to be associated with significantly worse survival outcomes regardless of treatment arm. These preliminary results are cited in the NCCN Guidelines and therefore are included in Table 3 along with published search results.
Therefore, the B-raf V600E mutation seems to be a negative prognostic factor in patients with metastatic colorectal cancer, independent of treatment with EGFR-targeted agents. The evidence is not consistent regarding its predictive value for non-response to cetuximab and panitumumab.
Based on the data as of 2009, the NCCN guidelines added the statement that patients with nonmutated K-ras tumors known to harbor a B-raf V600E mutation are unlikely to benefit from cetuximab or panitumumab.24 This was listed as a Category 2A recommendation, but no specific recommendation was made regarding the performance of B-raf mutation analysis. After presentation of conflicting results from B-raf subset data of the CRYSTAL trial, this recommendation was amended to include the statement, “although the data are somewhat inconsistent.”
MMR deficiency is present in approximately 15% to 20% of colorectal cancers and may be from sporadic or inherited inactivation of a mismatch repair protein: MLH1, MSH2, MSH6, or PMS2.77–80 Tumor MMR testing historically has been reserved for patients meeting the Revised Bethesda Guidelines clinical criteria for genetic testing for hereditary nonpolyposis colorectal cancer (HNPCC).78,80 This case study focuses on the possible role of MMR deficiency as a predictive factor for lack of benefit from 5-fluorouracil chemotherapy in patients with stage II colon cancer. Among the 16 nonrandomized studies identified by this search, several suggest the possibility of improved outcomes with 5-fluorouracil–based treatment in patients with MMR-deficient locoregional colorectal cancers compared with patients with proficient MMR.82–85
However, some of these studies suggest that patients with MMR deficiency treated with 5-fluorouracil–based therapy experience no benefit or have a potentially worse outcome.85–93 Interpretation of all of these studies is confounded by the known strong positive prognostic value of MMR deficiency, and whether other prognostic factors such as B-raf mutation, which seems to be enriched in sporadic cases of MMR deficiency, were balanced across arms is unknown. Heterogeneity of stage, tumor location, and treatment (particularly whether combination therapies were used) further confounds interpretation of these data. In the 8 published, randomized studies identified, the results are also inconsistent.94–101 Among these, the most compelling published data for MMR deficiency as a predictive marker for lack of benefit from adjuvant 5-fluourouracil in locoregional colon cancer were shown by a retrospective, pooled analysis of MMR status in 570 tumor specimens of patients enrolled in randomized studies of adjuvant 5-fluorouracil with levamisole or leucovorin compared with no adjuvant therapy.98 This study showed no improvement in overall survival in patients with stage II and III colon cancer with MMR-deficient tumors treated with adjuvant 5-fluorouracil compared with those with MMR-proficient tumors, who did experience benefit. A recently published large meta-analysis, which was first presented at the 2008 ASCO Annual Meeting, corroborated this impression and further suggested the possibility of worse outcomes in patients with MMR-deficient stage II colon cancers treated with 5-fluourouracil.102,103
In contrast to these 2 studies, several other randomized studies have suggested that patients with MMR-deficient tumors may derive benefit from treatment with 5-fluourouracil–based therapies, although again interpretation is confounded by inclusion of patients with stage III disease and heterogeneous treatment, including 5-fluorouracil–based combination therapy arms and single-agent arms.94,97,101 Search results are summarized in Table 4.
Cumulatively, the data mentioned earlier provide somewhat equivocal evidence for MMR deficiency as a predictive marker for lack of benefit from 5-fluorouracil therapy in patients with locoregional colorectal cancer in general, and specifically in those with stage II colon cancer. The strong positive prognostic value of MMR deficiency is consistent across many large, randomized studies.30,31,68,77,79,104
The latest version of the NCCN Guidelines added MMR testing to the risk stratification algorithm for stage II colon cancer as a Category 2A recommendation based on the data for this test as a predictive marker for lack of benefit from adjuvant 5-fluorouracil therapy, citing results of the recent meta-analysis presented at the 2008 ASCO Annual Meeting.24,102
The Oncotype DX Colon Cancer Assay became available on the market in January of 2010.24,32 This assay characterizes gene expression in fixed, paraffin-embedded tumor specimens using reverse transcriptase-polymerase chain reaction (RT-PCR) to generate a 7-gene recurrence score for patients with stage II colon cancers.104 The assay showed quantitative precision and reproducibility in an initial development set from the C-01/C-02 randomized adjuvant studies of the National Surgical Adjuvant Breast and Bowel Project (NSABP), and subsequent results from its 4 development sets and validation set have been presented at national oncology conferences but remain unpublished at the time of writing.104–108 Validation of the Oncotype DX Colon Cancer Assay has been performed by retrospective, subset analysis of 1436 tissue blocks from patients with stage II colon cancer enrolled in the randomized QUASAR study comparing postsurgical adjuvant 5-flurouracil therapy with observation alone.109 In the validation set, the test showed discrimination of recurrence risk as a continuum between low, intermediate, and high recurrence score groups with estimated recurrence risk at 3 years of 12%, 18%, and 22%, respectively; the hazard raio for recurrence between the low- and high-risk groups was 1.47 (P = .046) using the Cox model. Search results are summarized in Table 5.
Based on these preliminary data, the Oncotype DX Colon Cancer Assay recurrence score seems to provide prognostic information independent of conventional risk factors, discriminating the absolute increase in recurrence risk at 3 years between low- and high-risk patients by approximately 10%.104 The Oncotype DX Colon Cancer Assay is not predictive of 5-fluorouracil benefit, however, because the recurrence risk reduction from chemotherapy seemed proportional across all risk groups in the QUASAR dataset.104
The Oncotype DX Colon Cancer Assay has not been included in NCCN Guidelines, which currently state that data are insufficient to recommend the use of multigene assay panels to determine adjuvant therapy.24
This article presents a case study of 4 contemporary examples of personalized medicine biomarkers in colorectal cancer, describing the available clinical evidence for each example. All 4 of these biomarkers have been evaluated for inclusion in the wide-reaching NCCN Guidelines. Based on this structured review, the authors conclude that the level of published clinical evidence for these biomarkers is variable, and in some cases, discordant in content. This finding suggests that, by necessity, the domains of expert experience, clinical judgment, and consensus may play a greater role than published clinical trial data in guideline development for new personalized medicine biomarkers.
Reliance on expert opinion may be both a strength and a potential limitation when evaluating new technologies with rapidly evolving data, such as biomarkers. On the one hand, reliance on expert opinion enables timely review and incorporation of new information, resulting in the most up-to-date, accessible, and useful guidelines for stakeholders who must assimilate new technologies. The above-discussed qualification in B-raf recommendations soon after presentation of new data reflects the dynamic and adaptive nature of the NCCN Guidelines. Other guidelines that rely on formal systematic review of the evidence are more laborious, require mature data sets and studies, and are therefore slow to respond to new data, rendering them less useful to practitioners.26 In a recent survey of 459 breast cancer surgeons, expert opinion followed by guidelines and consensus statements have been shown to have the strongest influence on decision-making in areas of scientific uncertainty. 22
Conversely, however, the strong reliance on expert opinion introduces potential for bias. The NCCN Guidelines development process uses rigorous safeguards, including strict conflict of interest disclosure requirements, an iterative process with review by and input from practitioners at member institutions, and inclusion of panel members representing diverse specialties and viewpoints.29,110 Among the 4 examples studied, the 3 tests that have been recommended by the NCCN Panel for Colon Cancer (K-ras and B-raf mutation analyses and MMR testing) are not proprietary to any single commercial entity, unlike the Oncotype DX Colon Cancer Assay, which has not been adopted by the guidelines.
These measures, however, are not protective of subtle factors that might influence the uptake of new personalized medicine technologies into oncology guidelines. For example, in the case of the new B-raf mutation analysis recommendation, it is possible that the momentum and enthusiasm generated by the K-ras biomarker discovery influenced panel members’ impression of level of evidence and likelihood of improvement in patient outcomes from the related downstream biomarker, B-raf V600E mutation. The threshold for biomarker recommendation may also vary by the type of malignancy because of differences in research funding, patient advocacy, the risk inherent to the specific tumor type, treatment efficacy and toxicity, and subspecialty bias, factors that are not addressed by this case study.
This case study highlights several other intriguing aspects of biomarker evaluation and uptake by these influential guidelines. First, the examples of K-ras and B-raf mutation analysis, MMR testing, and the Oncotype DX Colon Cancer Assay highlight the challenge of applying a uniform methodology to categorize the variable level of data generally associated with biomarker studies.1,111 Across the 4 tests, the clinical evidence consists largely of retrospective subset analyses of patient subsets derived from case series, cohorts, and prospective, randomized trials. The current NCCN Categories of Evidence and Consensus ratings do not clearly discriminate differences in level of available evidence for K-ras, B-raf, and MMR testing. All 3 of these recommendations are assessed as Category 2A level of evidence and consensus, whereas this structured review suggests a significantly lower level of evidence for B-raf testing compared with the other 2 tests at the time of its inclusion in the guidelines.
These examples also show that it is very difficult to show improvements in patient outcome, one of the domains included in the NCCN Guideline evaluation methodology, for biomarkers. Biomarker studies may require a different set of standardized end points than studies of therapeutic modalities.1 Inherent to the challenge of selecting appropriate end points for biomarker studies is the importance of determining whether a biomarker’s association with a specific outcome is because it is a predictive marker for response to the treatment being studied, or because it is a strong prognostic marker independent of treatment. Many studies claiming predictive value do not uniformly include a control arm to exclude the contamination of a strong prognostic marker, as exemplified by studies of both B-raf mutation analysis and MMR testing. In the case of the B-raf V600E mutation, the lack of controlled studies impedes clear designation of this marker as prognostic, predictive, or both; because of this, the initial NCCN Guideline recommendation for B-raf interpretation has been amended to reflect the inconsistency suggested by new, controlled data sets.24,76
Another observation from this case study is that NCCN Guideline inclusion often follows presentation of preliminary data at a national conference such as the ASCO Annual Meeting or a subspecialty symposium. This coincident timing likely follows naturally from the maturation of available evidence and consensus expert opinion from both organizations. However, given the previously discussed importance of guideline inclusion in the decisions of stakeholders, including policymakers, payors, and practitioners, this association in timing may merit further study.
Finally, although this case study focuses specifically on personalized medicine biomarker tests, the findings may apply to other types of nascent technologies and treatments in oncology that may be available to practitioners before the supporting evidence base is complete.
As long as medical technologies can reach the market before their optimal use has been comprehensively defined by validation studies, oncology practitioners are likely to continue to rely on the recommendations of recognized bodies of experts, such as NCCN panelists, for guidance in their use. The methods of evaluation and decision by these guideline bodies, therefore, are likely to play a significant, ongoing role in the adoption of personalized medicine bio-markers in oncology. Further study is warranted to understand the complex balance between emerging clinical evidence and expert opinion in the integration of personalized medicine technology into oncology practice guidelines.
This study was supported by a grant from the National Cancer Institute (P01CA130818) through the Center for Translational and Policy Research on Personalized Medicine (TRANSPERS) at the University of California, San Francisco. This study was also supported in part by the Maisin Foundation.
Robin K. Kelley, MD is a member of the NCCN Hepatobiliary Cancers Panel and has received an honorarium from Genomic Health, Inc. for participating in a speaker training program. Alan P. Venook, MD is a member of the NCCN Panels for Colon/Rectal/Anal Cancers and Hepatobiliary Cancers and the NCCN Board of Directors. Dr. Venook has held an advisory role to and received honoraria from Genomic Health, Inc.