The Duke Cancer Center oversight mechanism was circumvented, according to Robert M. Califf, MD, director of the Duke Translational Medicine Institute and vice chancellor for Clinical and Translational Research at Duke University School of Medicine. “Potti and Nevins made the case to the institution that their work was different enough from what we do that they shouldn’t fall under our aegis and that they should have their own enterprise, focused on genomic research,” says Califf. “They argued that they were in a new area so they should have the freedom to operate on their own. In this case it was the so-called Institute for Genome Sciences and Policy.”
A committee at Duke University, cochaired by Robert M. Califf, MD, developed a plan for improving the quality and integrity of research.
This and other contributing factors — discontinuity in the Potti/ Nevins statistical team; reluctance to challenge a well-regarded tenured professor; confusion about what constitutes individual and institutional conflicts of interest; the IRB’s failure to inform external reviewers of questions raised by Baggerly, Coombes, and the NCI; and ambiguity about the need for an FDA Investigational Device Exemption (IDE) — are cited in the IOM report.
One of the most commercially successful omics-based tests currently in use, Oncotype DX, was developed by Genomic Health as a laboratory-developed test (LDT) without FDA review. However, in IOM’s report, Genomic Health’s chief scientific officer Steven Shak, MD, acknowledged that “the company benefited from prior interaction with FDA and the extensive background material FDA provides on its website about assay validation.” The report recommends that FDA clarify the regulation of omics-based tests by issuing guidance or regulation that specifies which omics-based tests require FDA review and at which point a review should occur. The FDA is encouraged to continue using and publicizing the pre-IDE process and to issue guidance for LDTs not currently reviewed by the FDA.
“I think it’s clear from the case studies that you can do the FDA process well or poorly and that you can do a laboratory-developed test process well or poorly,” says Debra Leonard, MD, PhD, professor and vice chair for laboratory medicine and director of clinical laboratories at Weill Medical College and director of clinical laboratories at New York-Presbyterian Hospital–Weill Cornell Medical Center. “Both are valuable pathways. The CLIA pathway allows you to move technologies into clinical practice in a cost-effective way that allows rapid access to those technologies. The FDA process requires documentation of safety and efficacy.”
Leonard, who also served on the IOM committee, points out that CLIA does not have much to say about the validation of molecular or genomic tests per se. To compensate, the College of American Pathologists (CAP) developed a checklist related to the development and validation of a test prior to clinical use, and because CAP has “deemed” status under CLIA, CAP accreditation is equivalent to CLIA accreditation. As the IOM report points out, if an omics-based test result is used for patient care, the test must be performed in a CLIA-certified lab.
Where the rules of evidence get tricky
Many medical schools now offer courses on basic principles of clinical research design and interpretation, but it can be easy to overlook those principles when laboratory-based researchers start doing nonexperimental or observational clinical research.
In experimental research, randomization is used to allocate groups of cell lines or people to an intervention vs. control. In an experiment, because you can “hold everything else equal” (except for the intervention), it is easy to learn about cause-and-effect or other associations.
In observational research, randomization cannot be done, and the rules of evidence — how to design and interpret a study so that you can trust its results — are much trickier. A culture clash can occur when laboratory-based researchers, used to the experimental method, start doing observational research but are unfamiliar with important rules of evidence.
— David F. Ransohoff, MD,
Professor of Medicine and Clinical Professor of Epidemiology
University of North Carolina School of Medicine
“FDA rules for evaluating diagnostics have gone through a fascinating evolution in the last four decades, and the current process doesn’t make clinical sense to a lot of people — including many at the FDA,” says Ransohoff, who is also a consultant to the FDA’s Immunology Devices Panel, Center for Devices and Radiological Health. “It’s not the FDA’s fault. They’ve never gotten the guidance they need from Congress or the administration.”