|Home | About | Journals | Submit | Contact Us | Français|
As technologies evolve, and diagnostics move from detection of single biomarkers toward complex signatures, an increase in the clinical use and regulatory submission of complex signatures is anticipated. However, to date, no complex signatures have been approved as companion diagnostics. In this article, we will describe the potential benefit of complex signatures and their unique regulatory challenges including analytical performance validation, complex signature simulation, and clinical performance evaluation. We also will review the potential regulatory pathways for clearance, approval, or acceptance of complex signatures by the U.S. Food and Drug Administration (FDA). These regulatory pathways include regulations applicable to in vitro diagnostic devices, including companion diagnostic devices, the potential for labeling as a complementary diagnostic, and the biomarker qualification program.
Despite the genetic understanding of cancer as heterogeneous and complex, our current oncology paradigm has not yet incorporated complex signatures into the prediction of responses to targeted therapies. As we move away from single biomarkers to diagnose and treat cancer, it is critical to understand the regulatory pathways and implications for complex signatures. A complex signature (similar in concept to an in vitro diagnostic multivariate index assay) can be thought of as a combination of multiple variables using an interpretation function to yield a single, patient-specific result (e.g., a classification, score, or index) (1). Complex signature considerations have been relevant to multiple oncology in vitro diagnostic devices, including one measuring two analytes (see supplementary Table S1). For example, a complex signature could be based on measurements from a next generation sequencing (NGS) based test, a gene expression profile, a protein based profile, or an immune signature and has the potential to be tissue or liquid biopsy based. Taken together, the overall signature would be considered a biomarker, or a “defined characteristic that is measured as an indicator of normal biological processes, pathogenic processes, or responses to an exposure or intervention, including therapeutic interventions,” and would not be a direct assessment of how a patient feels, functions, or survives (2). From a drug development perspective, a complex signature could be used as a susceptibility/ risk, diagnostic, monitoring, prognostic, predictive, pharmacodynamic/ response, or safety biomarker.
There are many reasons why a complex signature could be advantageous for drug development. Historically, oncology drug development has been based on large Phase 3 clinical trials with unselected populations and prolonged follow-up, and infrequent genomic alterations could be overlooked or not examined. This paradigm is both protracted and costly. More recently, there have been numerous approvals of targeted therapies based on smaller, targeted subsets of patients. This adaptation of drug development to incorporate information generated in the genomic era allows for the introduction of innovative approaches to characterize patients most likely to benefit from a therapy. This changing paradigm presents its own challenges, including how to handle genomic heterogeneity, how to identify rare subsets of patients who might benefit from a particular therapy, and how to develop a biomarker test with the ability to generate actionable results. A complex signature has the potential to address many of these concerns.
Since a complex signature takes into account multiple genes and pathways, compared to a single gene approach, a complex signature may provide a more rational insight into the disease process and therefore the therapeutic targets. Complex signatures could allow for improved efficiency, particularly with respect to targeted therapy development. However, there are unique challenges to the development of a complex signature, as the assay development and validation involve multiple steps, and the interpretation of the test in a clinical trial with the incorporation of multiple genes/pathways may be difficult. While to date no complex signatures have been approved as companion diagnostics, some have been approved or cleared for other non-companion in vitro diagnostic device claims, and many are in development.
From a regulatory standpoint, complex signatures can become accepted and aid in the development of a device, a therapeutic product (drug or biologic) and a device as a companion diagnostic (or complementary diagnostic), and can also support the drug development process by establishing the signature for use in multiple development programs through biomarker qualification (see Figure 1). These pathways are related and potentially overlapping but are separated below for clarity.
Devices, including in vitro diagnostic devices, are classified based upon an evaluation of the amount of regulation that provides reasonable assurance of the device’s safety and effectiveness. Devices are classified into one of three classes – Class I, Class II, and Class III. Generally speaking, Class I devices are subject to the least regulations, and Class III devices are subject to the most regulation. The premarket submission pathway for medical devices depends on the class in which the device is regulated.
To date, some complex signature oncology in vitro diagnostic devices have been classified into Class III and some have been classified into Class II (see supplementary Table S1). The associated types of premarket submissions have included premarket approval application (PMA) submissions, evaluation of automatic class III devices (de novo submissions), and premarket notification (510(k)) submissions (3–6).
Cologuard™ is an example of a complex signature evaluated through PMA. The test is a colorectal cancer screening test which analyzes stool for the presence of 11 molecular markers, including hemoglobin and DNA markers, which may indicate the presence of colorectal cancer or advanced adenomas. Based on combined results of all the selected DNA markers and hemoglobin, a single qualitative Cologuard™ result, positive or negative, is determined (7). OVA1™, a complex signature, was classified through the de novo process as Class II. OVA1™ uses a blood sample to test for levels of five proteins and combines them into a single numerical score between 0 and 10 to indicate the likelihood that the pelvic mass is benign or malignant (8). OVA1™ is intended only for women, 18 years and older, who are already selected for surgery because of their pelvic mass. It is not intended for ovarian cancer screening or for a definitive diagnosis of ovarian cancer and interpreting the test result requires knowledge of whether the woman is pre- or post-menopausal. The Prosigna™ Breast Cancer Prognostic Gene Signature Assay on formalin-fixed paraffin-embedded breast tumor tissue is another example of a complex signature. This signature utilizes gene expression data, weighted together with clinical variables to generate a risk category and numerical score, to assess a patient’s risk of distant recurrence of disease and was found to be substantially equivalent through 510(k) (9). In summary, to date, a complex signature device for cancer screening has been considered as class III and evaluated by PMA, whereas complex signature devices for patients already selected for treatment or for cancer prognosis have been classified as class II.
Although no complex signatures have yet been approved as companion diagnostics, a complex signature could be used in a drug development program through the investigational new drug (IND), new drug application (NDA), or biologic licensing application (BLA) process. During the investigational use of the therapeutic product, the Center for Drug Evaluation and Research (CDER) will provide guidance on the signature development plan in the context of the drug and also on the appropriateness of the clinical trial design to support the overall development program. A complex signature companion diagnostic device would be reviewed (e.g., through a 510(k) or PMA) by the relevant diagnostic product review center (the Center for Devices and Radiological Health (CDRH) or the Center for Biologics Evaluation and Research (CBER)). Investigational devices are subject to FDA’s investigational device exemption (IDE) regulation, and the regulatory requirements for an investigational device are determined by the risk posed to subjects by use of that device. The nature of risks in the investigational device context is distinct from FDA’s risk evaluation in other contexts, including FDA’s assessment of potential risk during FDA’s classification of devices. The IDE regulation describes three categories of device studies: significant risk (SR) studies, non-significant risk (NSR) studies, and exempt studies (10). Sponsors should consult with the therapeutic product center and the relevant device center as to the regulatory responsibilities appropriate for a particular study.
Once data has been collected in clinical trials, at the time of the NDA or BLA submission, CDER would perform the NDA or BLA review with attention to the signatures value in appropriately selecting therapy for the studied population. The companion diagnostic device submission (e.g., 510(k) submission, de novo submission, or PMA submission) would be reviewed by the applicable device center with attention to whether the device is properly validated and meets the applicable standard for marketing authorization for the intended use. Generally, when FDA determines that a device is essential to the safe and effective use of a novel therapeutic product or new therapeutic product indication, FDA will not approve the therapeutic product or new indication if the companion diagnostic device is not approved or cleared for that indication. However, FDA may decide that it is appropriate to approve a therapeutic product even though a companion diagnostic device is not approved or cleared contemporaneously (11). For example, crizotinib for the treatment of patients with metastatic non-small cell lung cancer whose tumors are ROS1-positive was approved without a corresponding companion diagnostic device. In this case, the clinical benefit of the drug in a rare patient population with a fatal disease outweighed the risks and uncertainties associated with lack of a companion diagnostic, and the development of the companion diagnostic was agreed upon with the drug sponsor as a postmarketing commitment (12).
During the review of the device and drug, FDA may determine that the test is not essential for the safe and effective use of the drug but the test identifies a biomarker-defined subset of patients that responds differentially to a drug and aids in the risk/benefit assessment for individual patients. To date, there have been four such “complementary diagnostic” determinations, all of which were related to immunotherapies, whereby the PMA submissions for the tests have been approved (13–17).
The objective of this pathway is to establish the use of a biomarker, which could include a complex signature as a composite biomarker for a particular context of use, through consultation with FDA for use in multiple drug development programs (18). A pharmaceutical developer, health research organization, or patient foundation, consortium or government entity may request regulatory “qualification” of a biomarker beginning with an initiation phase and acceptance into the program. In the consultation and advice stage a briefing document is received, advice is given and there are face to face meetings. The review phase allows for the full submission package to be received and reviewed and qualification recommendation ultimately given. If qualified, a biomarker can be relied upon to have a specific interpretation and application in drug development and regulatory review within the stated context of use for clinical trials. One would not have to reconsider nor reconfirm the acceptance of the biomarker in subsequent studies when the biomarker is used in the same context, and the biomarker would not be limited to association with a particular drug. Thus qualification could be particularly advantageous for biomarkers with broader drug development and clinical trial applications. So far no complex signatures, or any biomarkers at all, have been qualified in oncology although some are under consideration (19).
Stakeholders may also wish to request a Critical Path Innovation Meeting (CPIM) to further discuss complex signature development plans with relevant FDA staff (20). The CPIM allows for a regulatory discussion of how the proposed methodology or technology may enhance drug development in a collaborative discussion about emerging science, tools and methods.
Typical analytical validation studies for in vitro diagnostics may include evaluation of pre-analytical factors, precision/reproducibility, performance at low levels, performance around the cutoff, accuracy, interference, cross-reactivity, carry-over, effect of excess/limiting sample, and stability. Precision/reproducibility and measuring/reporting range has been of particular concern in past FDA reviews, where it was important to determine the acceptable variability in the individual analytes (complex signature input) based on the acceptable impact of variability on the overall result (complex signature output).
FDA’s review of the complex signature devices described above illustrates the unique issues associated with the analytical performance validation of complex signature devices. For example, details of how issues associated with the analytical performance validation of ovarian adnexal mass assessment score test systems are provided in FDA’s Class II Special Controls Guidance Document entitled “Ovarian Adnexal Mass Assessment Score Test System” (21). Such details include that the samples in the precision/ reproducibility study should span the range of values (including those close to the cutoff(s) due to different combinations of the analytes), potential sources of test variability to be evaluated (including run, day, operator, site, instruments, and lot), impact of variability in the individual analytes on the overall complex signature result, and stability across the range of specimen parameters for use.
The usual precision study provides information for particular combinations of the amounts of individual analytes that were present in the samples. However, many possible combinations of the amounts of individual analyte (different inputs) may give the same value of the overall result (same output). As an example, consider results generated from analytes X plus Y each measured from 0 to 5 for a total score of 0 to 10. If analyte X has more variability than Y, an overall result of 5 would have more variability in some conditions (e.g., X = 5, Y = 0) than others (e.g., X = 0, Y = 5) (See Figure 2). Thus, a given complex signature result may have different precision characteristics for different input combinations. Since it would be impractical to directly evaluate the precision of all possible combinations of analytes, additional simulation can provide information about possible precision profile of the complex signature for different combinations of individual analyte values. One example simulation approach is to use profiles of individual analytes and Monte Carlo simulation of error propagation (21).
FDA’s review of the complex signature devices described above also illustrates the unique issues associated with the clinical performance evaluation of complex signature devices. For example, details of how issues associated with the clinical performance evaluation of breast cancer prognosis gene expression profiling test systems are provided in FDA’s Class II Special Controls Guidance Document entitled “Gene Expression Profiling Test System for Breast Cancer Prognosis” (22). The following are examples of recommendations from this guidance document. Clinical studies supporting the claims for a complex signature device should use patient samples that are representative of the intended use population and that are different from the training sets (specimens used to develop the complex signature). Key in vitro diagnostic clinical study design issues should include avoiding potential biases due to selection methods or analytical artifacts as well as appropriate use of clinical reference standard (“gold standard”). In order to demonstrate clinical performance for an original companion diagnostic claim, the requisite samples would be from the clinical trial(s) supporting the NDA or BLA.
Factors associated with heterogeneity in clinical performance have also arisen during FDA’s review of the complex signature devices described above. Examples include age and stool DNA methylation influencing clinical specificity of Cologuard™, ovarian adnexal mass assessment clinical cutoffs depending on menopausal status for OVA1™, and breast cancer prognosis categorization dependence on nodal status in Prosigna™ (7–9). In addition, considerations for clinically meaningful performance may include the “value added” compared to standard clinicopathologic covariates (e.g., stage, routine tests), and the rationale for reporting multiple interpretive categories and/or score, if applicable (22).
In addition, the interpretation of clinical trial results for demonstration of clinical utility employing a complex signature, which could include multiple different genes in a pathway, each with multiple variants, each with different frequencies in the population, and all resulting in inclusion on a clinical trial, may be challenging. FDA has not yet determined how such an indication would be granted, or if adequate representation of all variants studied on a clinical trial would be required. Internal FDA working groups have been formed to discuss these clinical trial issues and drug labeling implications.
In conclusion, there is a great need for novel biomarkers which incorporate molecular pathways and oncologic heterogeneity. Complex signatures from blood and tissue have the potential to positively impact drug development and subsequent targeted therapy clinical practice. FDA encourages dialogue during the development of complex signatures and targeted therapies in order to expedite drug development. FDA’s approach to this changing paradigm requires innovation and flexibility in order to meet the demands and challenges of complex signature development.
This is a U.S. Government work. There are no restrictions on its use.
Disclosure of Potential Conflicts of Interest
No potential conflicts of interest were disclosed by the authors.