|Home | About | Journals | Submit | Contact Us | Français|
Analyzed the data: PD TJ CDM. Wrote the first draft of the manuscript: PD TJ CDM. Contributed to the writing of the manuscript: PD TJ CDM. ICMJE criteria for authorship read and met: PD TJ CDM. Agree with manuscript results and conclusions: PD TJ CDM.
Regulatory approval of new drugs is assumed to reflect a judgment that a medication's benefits outweigh its harms. Despite this, controversy over approved drugs is common. If sales can be considered a proxy for utility, the controversies surrounding even the most successful drugs (such as blockbuster drugs) seem all the more paradoxical, and have revealed the extent to which the success of many drugs has been driven by sophisticated marketing rather than verifiable evidence ,. But even among institutions that aim to provide the least biased, objective assessments of a drug's effects, determining “the truth” can be extremely difficult.
Consider the case of the influenza antiviral Tamiflu (oseltamivir). Prior to the global outbreak of H1N1 influenza in 2009, the United States alone had stockpiled nearly US$1.5 billion dollars worth of the antiviral . As the only drug in its class (neuraminidase inhibitors) available in oral form, Tamiflu was heralded as the key pharmacologic intervention for use during the early days of an influenza pandemic when a vaccine was yet to be produced. It would cut hospitalizations and save lives, said the US Department of Health and Human Services (HHS) . The Advisory Committee on Immunization Practices (ACIP, the group the US Centers for Disease Control and Prevention [CDC] uses to form national influenza control policy) said it would reduce the chances of developing complications from influenza . So, too, did the Australian Therapeutic Goods Administration  and the European Medicines Agency (EMA) .
Most (perhaps all) of these claims can be traced back to a single source: a meta-analysis published in 2003 that combined ten randomized clinical trials conducted during the late 1990s by the manufacturer prior to US registration of the drug . This analysis, conducted by Kaiser and colleagues, proposed that oseltamivir treatment of influenza reduced both secondary complications and hospital admission. In contrast, the Food and Drug Administration (FDA), which approved Tamiflu in 1999 and was aware of these same clinical trials, concluded that Tamiflu had not been shown to reduce complications, and required an explicit statement in the drug's label to that effect . FDA even cited Roche, Tamiflu's manufacturer, for violation of the law for claims made to the contrary .
Nor did the FDA approve an indication for Tamiflu in the prevention of transmission of influenza ,. This assumption was at the heart of the World Health Organization's (WHO) proposed plan to suppress an emergent pandemic through mass prophylaxis . While the WHO recently added Tamiflu to its Essential Medicines list, if FDA is right, the drug's effectiveness may be no better than aspirin or acetaminophen (paracetemol). The FDA has never clarified the many discrepancies in claims made over the effects of Tamiflu. Although it may have limited approval indications accordingly, the FDA has never challenged the US HHS or the US CDC for making far more ambitious claims. This means that critical analysis by an independent group such as a Cochrane review group is essential.
But which data should be used? In updating our Cochrane review of neuraminidase inhibitors, we have become convinced that the answer lies in analyzing clinical study reports rather than the traditional published trials appearing in biomedical journals . Clinical study reports contain the same information as journal papers (usually in standardized sections, including an introduction, methods, results, and conclusion ), but have far more detail: the study protocol, analysis plan, numerous tables, listings, and figures, among others. They are far larger (hundreds or thousands of pages), and represent the most complete synthesis of the planning, execution, and results of a clinical trial. Journal publications of clinical trials may generate media attention , propel researchers' careers, and generate some journals a revenue stream . However, when regulators decide whether to register a new drug in a manufacturer's application, they review the trial's clinical study report.
In 2010, we began our Cochrane review update using clinical study reports rather than published papers . We obtained some sections of these clinical study reports for the ten trials appearing in the Kaiser 2003 meta-analysis from Tamiflu's manufacturer, Roche—around 3,200 pages in total. In 2011, we obtained additional sections of clinical study reports for Tamiflu through a Freedom of Information request to the EMA, amounting to tens of thousands of pages. While extensive and detailed, it is important to note that what we have obtained is just a subset of the full clinical study reports in Roche's possession. Nonetheless, Box 1 provides a list of details we have already discovered—and would have never discovered without access to these documents. This information has turned our understanding of the drug's effects on its head. Other drugs for which previously unpublished, detailed clinical trial data have radically changed public knowledge of safety and efficacy include Avandia, Neurontin, and Vioxx (Table 1).
Taken together, these experiences suggest that any attempt at reliable evidence synthesis must begin with clinical study reports. Yet we are aware of only three other groups of independent researchers conducting a systematic review based entirely (or mostly) on these and other regulatory documents –. We can think of two major reasons this might be. First, outside of regulatory agencies, few researchers have ever heard of clinical study reports. Second, clinical study reports are massive in size and difficult to obtain, traditionally shared only with regulators. The first problem seems tractable, but gaining better access to manufacturers' clinical study reports requires shifting the status quo from a default position of confidentiality to one of disclosure.
In the mid-20th century, regulatory agencies became increasingly responsible to the public for ensuring the safety and effectiveness of approved medicines. This rise paralleled an increase in the number and complexity of clinical trials. Both in the United States and Europe, manufacturers would come to submit trial data to regulators with the assurance that authorities would treat all trade secret data as confidential. Even following passage of the 1966 Freedom of Information Act (FOIA), the FDA maintained that safety and effectiveness test data were not subject to the FOIA ,. One sign things may be changing came in November 2010, when the EMA announced its intention to make all industry clinical study reports about a drug publicly available as a matter of standard practice after reaching a regulatory decision ,. The EMA has also already improved its handling of data under Freedom of Information requests. But as we learned was true for Tamiflu, regulators may not be in possession of full trial reports of all studies on a given intervention, implying the necessity of obtaining reports from manufacturers . But at present, industry seems extremely reluctant to make its clinical study reports freely available.
In addition to clinical study reports, we also need access to regulatory information. By the very nature of their professional mandate, regulators may conduct some of the most thorough evaluations of a trial program, and efforts like ours aimed at up-to-date evidence synthesis are seriously deprived without access to their reports. While the FDA has increasingly published its reviews, memos, and other correspondence on its Drugs@FDA website (http://www.accessdata.fda.gov/scripts/cder/drugsatfda/), and additional documents are accessible through FOIA requests, the process can be very time-consuming, taking months or even years. Moreover, regulatory agencies' lack of public inventories of their documentary holdings complicates the retrieval of information. Ideally, we would also have details of the regulators' deliberations, which can serve as signposts to important issues that need investigating.
In December 2009, after we voiced serious concerns in the BMJ about Tamiflu's alleged ability to reduce complications , Roche wrote that it was “very happy to have its data reviewed by appropriate authorities or individuals,” and publicly pledged to release “full study reports” for ten trials “within the coming days” . But despite extensive correspondence over the next year and a half, Roche refused to provide any more than portions of the clinical study reports for the ten Kaiser studies (Table 2) and no reports for any of the additional Tamiflu trials we had subsequently identified and requested (Table 3). Reasons for refusing to share the full reports on Tamiflu kept changing, and none seemed credible (Tables 2 and and33).
There are strong ethical arguments for ensuring that all clinical study reports are publicly accessible. It is the public who take and pay for approved drugs, and therefore the public should have access to complete information about those drugs. We should also not lose sight of the fact that clinical trials are experiments conducted on humans that carry an assumption of contributing to medical knowledge. Non-disclosure of complete trial results undermines the philanthropy of human participants and sets back the pursuit of knowledge.
Potentially valid reasons for restricting the full public release of clinical study reports include: ensuring patient confidentiality (although this could be remedied with redaction); commercial secrets (although these should be clear after drugs have been registered, and we found no commercially sensitive information in the clinical study reports received from EMA with minimal redactions); and finally, industry concerns over adversaries' malicious “cherry picking” over large datasets (which could be reduced by requiring the prospective registration of research protocols). None appear insurmountable.
With the EMA's stated intentions on far wider data disclosure, we hope the debate may soon shift from one of whether to release regulatory data to the specifics of doing so. But until these policies go into effect—and perhaps even after they do—most drugs on the market will remain those approved in an era in which regulators protected industry's data . It is therefore vital to know where industry stands. If drug companies have legitimate reasons for maintaining the status quo of treating all of their data as trade secret, we have yet to hear them. We are all ears.
Translation of the Summary Points into Russian by Vasiliy Vlassov.
Translation of the Summary Points into Italian by Tom Jefferson.
Translation of the Summary Points into Japanese by Yuko Hara and Yasuyuki Kato.
Translation of the Summary Points into Croatian by Mirjana Huić.
Translation of the Summary Points into Danish by Andreas Lundh.
Translation of the Summary Points into German by Gerd Antes.
Translation of the Summary Points into Spanish by Juan Andres Leon.
Translation of the Summary Points into Chinese by Han-Pu Tung.
We thank our co-authors of the Cochrane review of neuraminidase inhibitors. We also thank Yuko Hara for helpful comments on the draft manuscript and Ted Postol for many valuable insights. We are also grateful to those who helped with translations.
All review authors have applied for and received competitive research grants. All review authors are co-recipients of a UK National Institute for Health Research grant to carry out a Cochrane review of neuraminidase inhibitors (http://www.hta.ac.uk/2352). In addition: TJ was an ad hoc consultant for F. Hoffman-La Roche Ltd in 1998–1999. TJ receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, none of which are on neuraminidase inhibitors. TJ is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 products unrelated to neuraminidase inhibitors. Since submission of this article, TJ has been retained as an expert consultant in a legal case involving Tamiflu. CDM provided expert advice to GlaxoSmithKline about vaccination against acute otitis media in 2008–2009. CDM receives royalties from books published through Blackwells BMJ Books and Elsevier. PD declares no further conflicts of interest.
Peter Doshi is funded by an institutional training grant from the Agency for Healthcare Research and Quality #T32HS019488. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Provenance: Not commissioned; externally peer reviewed.