|Home | About | Journals | Submit | Contact Us | Français|
The FDA is now exerting its regulatory authority over molecular diagnostics and their assays used for medical-decision making in clinical trials by performing pre-Investigational Device Exemption (IDE) reviews in all phases of clinical trials. This review assesses the analytical performance of the assay for the diagnostic and considers how that performance affects the diagnostic and the patient and their risks and benefits from treatment. This manuscript reviews the process of the first review that was performed on a new Children's Oncology Group (COG) Phase III trial in Acute Myelogenous Leukemia. The lessons learned and recommendations for how to prepare for and incorporate this new level of regulatory review into the protocol development process are presented.
On December 14, 2010 Dr. Elizabeth Mansfield from the Office of In Vitro Devices (OIVD) in the Center for Devices and Radiologic Health (CDRH) at the Food and Drug Administration (FDA) announced at a workshop introducing the Clinical Assay Development Program of the National Cancer Institute (NCI; Please see Williams et al, (1)) that the FDA would now perform pre-Investigational Device Exemption (IDE) and IDE reviews for all integral markers in all NCI-supported clinical trials. An integral marker is a marker or diagnostic that is essential for the performance of the trial and is used to assign treatment, perform risk stratification or to determine the dose of a therapeutic agent and is further described by Schilsky et al in the Focus section (2). Concerns about the reliability and reproducibility of genetic markers that are used for medical decision-making have been publicly expressed (3). Thus, the FDA is concerned that patients may be harmed by novel diagnostics rapidly entering clinical practice and used for medical decision making independent of corroborating clinical or pathologic evidence. Genetic tests in which an assay identifies a specific mutation in a gene for which there is a possible targeted therapy are an especial concern because the test is the sole arbiter of whether the patient may be a candidate for the therapy and there may be no ancillary clinical information that may assist the physician in medical decision-making.
The Children's Oncology Group (COG) randomized phase III acute myelogenous leukemia (AML) trial AAML1031 was affected by this announcement because the trial had been approved by the NCI Cancer Therapy Evaluation Program (CTEP), was in the last stages of Pediatric Central Institutional Review Board review but now needed a pre-IDE review by the FDA before it could be activated for patient enrollment. The design of AAML1031 included three genetic markers described below that were used, in part, to stratify patients into different risk groups that are used to modulate treatment intensity and identify patients for potential hematopoietic stem cell transplantation (HSCT). Patients are randomized to different induction therapies, some of which involve investigational agents provided by the NCI under an Investigational New Drug (IND) application, based on these risk categories. In addition, after an initial cycle of induction therapy, patients would be restaged to high or low risk cohorts based on the presence of Minimal Residual Disease (MRD) as determined by multidimensional flow cytometry at a 0.1% blast threshold. Those with MRD would receive more intensive chemotherapy and HSCT from the most suitable donor, and those without MRD would continue with standard chemotherapy. The identification of these three genetic markers and the MRD assay as markers that were essential or integral for the conduct of the trial prompted the COG, NCI and FDA to work together to expedite a pre-IDE review. The purpose of this manuscript is to characterize the lessons learned during this first FDA pre-IDE review so that clinicians, investigational scientists and clinical laboratory scientists may better understand the new processes and procedures related to the use of integral markers in clinical trials.
The FDA has a website (http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/default.htm) that is dedicated to defining the process and regulations surrounding approval and marketing of devices with all of the information needed to go through a pre-IDE review and then a formal IDE application if a device poses a “significant risk” to patients. In vitro diagnostics are “devices” as defined by the FDA (4) to be “an instrument, apparatus, implement, machine, … in vitro reagent, or other similar or related article … which is (1) recognized in the official National Formulary, or the United States Pharmacopeia, or any supplement to them, (2) intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease, in man or other animals, or (3) … does not achieve its primary intended purposes through chemical action within or on the body of man or other animals and which is not dependent upon being metabolized for the achievement of its primary intended purposes”. Thus, the scope of the pre-IDE and IDE is to assure that assays for use in medical decision-making are reliable and reproducible and that the risk-benefit of the use of the diagnostic is sufficiently safe to benefit the patient within its clinical context of use. The three genetic assays and 1 MRD assay in COG AAML1031 are in vitro diagnostics that, while not in the National Formulary or US Pharmacopeia, are intended for use in diagnosis of disease and do not act by chemical or biological means within the body.
The pre-IDE review identifies the concerns that the FDA may have for a formal IDE application. The pre-IDE review is an informal, non-binding review that may take 60 days but can be shorter or longer. In contrast, an IDE application has a firm 30 day time limit (Figure 1). The pre-IDE review has two components: an assessment of the analytical performance of the assay for the diagnostic and then an evaluation of the risk to the patient imposed by use of the diagnostic. The assessment of the risk imposed by the diagnostic has to be considered within the context of its use and whether the assay and diagnostic are “fit for purpose” (5,6). Often the risk of false negative or positive assay results may seem small if a patient has an advanced cancer with only a few months to live. However, that risk becomes much greater if the patient is being treated with curative intent or has a long expected survival as may occur in an adjuvant setting. No assay is absolutely free of errors so the risk of false positive or negative results needs to be presented within the protocol and informed consent. The analytical performance of the assay determines the probability of a false positive or negative result and its risk.
After determining that one or more devices are present in a trial, the FDA then evaluates whether the devices pose a “significant risk” to the patient. A “significant risk” is defined in 21 CFR 812.3(m)(3) as “a use of substantial importance in diagnosing, curing, mitigating, or treating disease, or otherwise preventing impairment of human health and presents a potential for serious risk to the health, safety, or welfare of a subject.“ In addition, the FDA has recently indicated (7) that “If [a diagnostic device is] used to make critical treatment decisions, such as patient selection, treatment assignment, or treatment arm, a diagnostic device is likely to be a significant risk device under 21 CFR 812.3(m)(3) …. In such cases, FDA will expect the sponsor to conduct the trial under full IDE regulations.” Thus, since integral markers are those that are used for medical decision-making for individual patients, they generally will be significant risk devices and need an IDE for performance of the assay in a clinical trial. In addition, the assay for such an integral marker will also need to be performed in a Clinical Laboratory Improvement Amendments (CLIA)-certified laboratory (See Schilsky et al (2 – this FOCUS section).
If a molecular diagnostic is an integral marker, the investigator and the clinical laboratory scientist need to assure that the molecular diagnostic performs adequately in samples similar to those of its intended clinical use. Often a retrospective analysis is performed with samples that have been collected as part of a previous clinical trial and the assay's analytical performance is validated including establishment of any cut-points. Critical considerations in this process include the reproducibility of the test, and, if assays are to be performed in more than one laboratory, the inter-laboratory concordance between test results. This process of analytical validation of an assay is also required for any laboratory-developed test (LDT) performed in a CLIA-accredited facility so that the performance characteristics of the LDT should be well known to the clinical laboratory scientist who develops and performs the assay. The FDA's Device Advice website (http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/default.htm) provides investigators and clinical laboratory scientists with contact information so that they may contact the appropriate FDA staff to begin discussion about the pre-IDE review. Diagnostics whose assays are integrated (performed in all subjects in a trial or a statistically defined subset but not used for decision-making) or for research (also not for decision-making in individual patients) are not subject to the FDA's pre-IDE review.
When data are sent to the FDA for the pre-IDE review, the information should be in the format that is also used for the IDE application (Table 1, Figure 1). The FDA reviews both the Letter (LOI), Concept or Protocol for the following information:
Since FDA assesses the analytical performance of the assay for the diagnostic, information on reliability and reproducibility need to be provided. The Cancer Diagnosis Program (CDP), CTEP, the Investigational Drug Steering Committee and national experts have created templates for immunohistochemistry (IHC) and fluorescence in situ hybridization (FISH) assays (with a somatic mutation template under development) that are posted on the CDP website (http://cdp.cancer.gov) to document an assay's analytical performance. Part of the FDA's review also evaluates whether the diagnostic depends on the “art of medicine”. This term refers in part to whether a qualified and certified diagnostician analyzes the test. Thus, tests like IHC and FISH assessed and interpreted by pathologists and cytogeneticists are part of the “art of medicine”. The other aspect of the art of medicine is whether other clinical or pathologic information supports the diagnostic's use in medical decision-making. Finally, the assay may generate risk if it is measured without an expert's interpretation of the test (e.g., scored or analyzed solely by a machine such as occurs with a sequencer) and is used for medical decision-making without any supporting clinicopathologic data. Such a test may be a somatic mutation that either classifies a disease or whose result assigns patients to a specific therapy without any supporting clinical or pathologic information. At the end of the pre-IDE review the investigators receive a report with any concerns of the FDA and whether an IDE is necessary. At this point the investigators may have a consultation with the FDA to get suggestions on how best to proceed. While the pre-IDE review is nonbinding, the FDA will expect investigators to respond to any concerns when they submit their IDE.
The FDA may recommend that a formal IDE application is not necessary for a particular assay. However, if the pre-IDE review indicates that a formal IDE application is required, then the formal application needs to be submitted before the trial can begin and the FDA review will be completed within 30 days (Figure 1). At the end of the formal review, an IDE is either granted that then allows the trial to proceed after other reviews and approvals have been obtained or there is further interaction with the FDA. The trial may also proceed if no comments are received within 30 days of receipt at FDA. An alternative path is is described below that permits the IDE application to be included
Once clinicians and laboratorians understand the nuances of the IDE review, it is important that they also anticipate how to fit the FDA IDE review into the workflow of creating the protocol. A major problem for clinicians and scientists involved with NCI-sponsored clinical trials concerns the need for faster protocol activation after approval of a Letter of Intent (LOI) for early phase trials or concepts for later phase trials. The NCI established the Operational Efficiency Working Group (OEWG) to “identify the institutional barriers that prolong the time from concept approval to accrual of the first patient, and develop solutions for overcoming these barriers.” As a result, CTEP has set in place a number of timelines mandating that protocols need to be developed and activated within 210 days for Phase I, I/II, and II LOI's, 240 days for Phase II concepts or 300 days for Phase III concepts after initial approval (Figure 2A). The FDA IDE process takes at least 90 days for the pre-IDE and IDE reviews and clearly must go forward in parallel to meet these timelines (Figure 2B). If the FDA needs to review a diagnostic, ideally the pre-IDE review should begin when the LOI or concept is approved and this requires that the assay for the diagnostic be analytically validated and locked down so that the analytical validity can be reported to the FDA. In addition, the NCI may facilitate the interaction between the FDA and the clinical investigators and laboratorians if the protocol chair and clinical laboratory scientists want that help. It is important to remember that the pre-IDE and IDE reviews are totally independent of the reviews of the LOI, Concept or Protocol but must be concluded before the trial may start. As a result, it is important to start the FDA pre-IDE review as early as possible in the protocol development process and to be sure that an integral assay is already developed and performance validated in a CLIA-certified laboratory.
The first trial to undergo this pre-IDE and IDE review is the Phase III trial in acute myelogenous leukemia from the Childrens Oncology Group (COG). In the COG AAML1031 trial three genetic markers are used to assign therapy. One is FLT3/ITD, a mutation in the receptor gene that is caused by duplication of a fragment of the juxtamembrane domain coding sequence of FLT3. The FLT3/ITD assay used in COG AAML1031 is a semiquantitative variation of the standard assay, where in addition to detection of the presence of FLT3/ITD, the ratio of the amount of mutant to wild type product is determined (FLT3/ITD allelic ratio; ITD-AR). Three separate cohorts of children with AML have demonstrated that in addition to the presence of FLT3/ITD, ITD-AR impacts outcome because those with ITD-AR ≥ 0.4 had a significantly worse survival than those with lower ITD-AR or those with wild type FLT3 (FLT3/WT) (8–13). The other two genetic markers used to stratify therapy in COG AAML1031 are the presence of mutations in Nucleophosmin 1 (NPM1) and CEPBα genes that are associated with a better outcome and are qualitative assays in which detection of any amount of somatic mutation qualifies as a positive assay (13). In the trial, the presence of FLT3-ITD with high ITD-AR leads to more intensive therapy treatment while the presence of mutations in NPM1 / CEPBα although not entirely mutually exclusive suggest that patients should receive less intense therapy. The COG investigators first performed these two assays as individual PCR based assays but then suggested that they might bundle the two qualitative assays into a duplex PCR assay with minimal analytical validation. The fourth integral marker is an MRD assay where result can cause a patient to be assigned to HSCT therapy.
The first important issue for the FDA was whether there were data to support the use of these diagnostics. As the Background and Rationale in the protocol carefully described, all markers had data to support their use in the form of multiple publications (8–13) as well as use in prior Phase III trials. The MRD assay had been in use for over a decade and its cut-off had been established in prior trials (14–16). Although this assay is also used to determine whether patients may be HSCT candidates - a procedure that carries a risk of death and serious side effects (17–22) - the assay is supported by other tests as well as is determined by the personal evaluation of a skilled, certified pathologist rather than solely measured by a machine. Thus, this assay falls within the category of the art of medicine and was not considered further by the FDA.
After an interchange between the investigators and members of the FDA's Office of In Vitro Diagnostic Devices at CDRH, a contact in the FDA was identified and sent the COG investigators a list of information that needed to be supplied for the pre-IDE review (Table 1). At this point the FDA suggested that investigators should note that the risk of the assays in a trial should be mentioned explicitly in the protocol and the informed consent in order to satisfy the requirements in section 3 of the list of questions (Table 1).
The FDA evaluated the analytical performance of each genetic assay using assay reliability and reproducibility characteristics provided by COG from their CLIA-accredited laboratories. The FDA wants trials to use assays and cut-points that are “locked down” and based on prior retro- and/or prospective experience. As an example, the COG indicated in their cover letter that it wanted to evaluate whether a lower percentage of blasts in the bone marrow might be used in the MRD assay to decide transplantation or removal from the study compared to the cut-off used in prior trials but did not yet have data in hand to justify the new cut-point. The FDA indicated that the previous cut-point should be still be used for this study, although data could be collected that might inform a more stringent cut-point in future trials. In addition, the FDA indicated that it was concerned that converting the NPM1 and CEBPα mutation assays into a duplex PCR assay when the only data provided and analyzed to date were with two separate independent assays was also problematic. In both cases the COG decided to follow the recommendations for the MRD cut-point and use the separate assays to determine NPM1 and CEBPα mutations for decision making in this trial (Table 2).
Once the pre-IDE review was completed and the FDA made its recommendations, the COG decided to use the alternative pathway for the IDE application. The IDE application may be submitted directly to the CDRH or it may be bundled with an Investigational New Drug (IND) application which is evaluated by either the Center for Drug Evaluation and Research (CDER) or the Center for Biologics Evaluation and Research (CBER) of the FDA. COG bundled the IDE application with an IND held by CTEP and provided a cover letter that requested CDER to consult CDRH's OIVD to evaluate their responses to the pre-IDE review. A benefit of this approach is that CTEP has a precedent that enables trials registered under INDs held by CTEP to begin accrual if all IRB and other approvals have been obtained. Bundling an IDE with an IND that is not held by CTEP to either CDER or the Center for Biologics Evaluation and Research (CBER) is appropriate but it would be wise to 1) also send information to OIVD directly as well as 2) wait the 30 days from receipt by the FDA when the FDA has to notify sponsors if they have concerns about either the IDE or IND. If comments are not received by investigators within 30 days of receipt by FDA, then the integral marker may be used in the trial.
With the emerging technologies for molecular genotyping in malignancies, large numbers of disease-associated mutations have been identified in recent years. Further, whole genome sequencing has provided a tool for unbiased evaluation of cancer genome and identification of molecular alterations in genes not previously suspected, increasing the number of genes implicated in cancer. In some cases, function-altering mutations have been shown to be associated with disease outcome or are potential targets for directed therapies. As a result of such associations, commercial interest in such biomarkers has increased. In the last decade there has been an upsurge in the number of patents for cancer genes and the patent holder can limit or completely block molecular evaluation of the genes in question. Patent holders may require laboratories to purchase a sub-license prior to testing for the gene in question, or to completely prevent individual laboratories from performing the assay, mandating all assays be performed in the company's central clinical laboratories. It is essential that investigators and their technology transfer offices determine whether a potential patent holder for a molecular diagnostic has granted licenses that may block the use of that diagnostic in clinical trials. A way that investigators may now do this is through Gene Cards (http://www.genecards.org) and GeneIP (http://www.xennexinc.com:8080/GeneIP/actions/Index.action) that identifies intellectual property related to specific genes that may be the basis for a molecular diagnostic.
The FDA recently has become directly involved in regulating the use of integral diagnostic assays in clinical trials based upon concerns that certain assays, particularly newer assays to detect specific mutations or molecular signatures, may not be fully validated for diagnostic use or that the variability and reliability of specific assays may not be fully defined. While few would disagree that clinical investigators should have the highest degree of confidence in the results of any diagnostic or investigational assay, it is not yet clear how the FDA's increased involvement in protocol development will improve the use of integral markers in clinical research. A key concern with implementation of this new oversight is not only the monetary costs associated with it, but the costs in terms of time to both need the requirements and potentially delay onset of clinical trials. As currently implemented, the process may take at least 3 months to complete, a particularly concerning delay in light of the historically long timelines of protocol development that resulted in an overhaul of protocol development processes for NCI-CTEP supported cancer clinical trials. Potential benefits of this process include the early provision of data on the analytical performance of assays for molecular diagnostics, increasing attention to development within the investigator and clinical laboratory scientist communities, and to potentially promoting earlier creation of high quality assays that can support clinically useful diagnostics.
One key question that emerged from this experience is whether it is more efficient to undergo a pre-IDE review or to proceed directly to an IDE application. While the pre-IDE review may take longer than 60 days, such a dialogue between investigators and FDA staff may facilitate a timely a successful IDE application. However, there are no mandatory timelines on completion of a pre-IDE review, and thus determining the most efficient route forward is not a straightforward process. FDA guidance in this area is needed and might consider metrics described by Poste et al. (23) in their overview for this FOCUS Section. Finally, based on our experience, we provide a list of recommendations that investigators who propose to use integral markers in their clinical trials ought to consider as early as possible during the development of their trial (Table 3).
The opinions expressed in this manuscript are those of the authors and do not necessarily represent those of the National Cancer Institute, the National Institutes of Health or the Department of Health and Human Services.