Moves to improve the conduct and reporting of clinical trials have gathered pace this year. One issue that the BMJ has long grappled with is what to do about protocols of trials, and we announce a new policy that begins in January 2005.
Medical research should begin with a protocol that details the hypothesis, objectives, and methodology to be used in the study. Ideally the study should not deviate from the protocol, but in reality it often does. Sometimes deviating from the protocol is unavoidable—for example, when new evidence is published independently during the trial which invalidates the specified outcome. In other instances, protocol deviation may represent suppression of unfavourable findings in favour of reporting favourable ones.
Evidence of bias has been limited to case reports,1,2 but now we have research that begins to quantify the problem.3-5 Chan et al compared the completeness of outcomes reported in protocols versus outcomes reported in published research.4 Their cohort study showed that about 50% of efficacy outcomes and 65% of harms per trial were incompletely reported, and 62% of trials had at least one primary outcome that was added, changed, or removed compared with the protocol. That 86% of the authors surveyed denied the existence of unreported outcomes implies that the extent of reporting bias may be even greater. These findings are supported by two further studies, which show that selective and incomplete reporting of outcomes in published randomised controlled trials is common.3,5
The desire to eliminate bias in reporting research has prompted important initiatives over the last decade. These have culminated in the use of the CONSORT statement (the Consolidated Standards of Reporting Trials) and other guidelines such as QUOROM (Quality of Reporting of Meta-analyses).6,7 The CONSORT checklist stipulates that deviations in the protocol should be described including reasons for doing so. Randomised controlled trials submitted to the BMJ must be reported in accordance with the CONSORT statement and checklist. We require a declaration that the decision to publish was controlled by the authors and not the trial sponsor.8 From July 2005 all trials will also need to be registered with a suitable trial registry.9 Now authors must submit the trial protocol with their manuscript submission, and we will not send the manuscript to an external reviewer unless we have the protocol.
Our experience of chasing authors for trial protocols, when we have suspected deviation in the protocol or found it hard to fathom what the authors set out to do, has been miserable. Authors are reluctant to send us the protocol, and the review process can be considerably delayed. Our electronic manuscript submission system allows additional documents to be uploaded as supplementary data files, and we ask authors to add the protocol to their manuscript submission in this way. BMJ editors and reviewers of that particular manuscript will be able to see the protocol on our system and refer to it as necessary. We do not intend to publish the protocols at this stage.
We are not offering peer review of protocols as the Lancet does,10 although we may offer this in future. We are currently conducting research into what kind of protocol review trialists favour. In the meantime, we believe that identifying deviation from the protocol is another important step in ensuring that the findings of a study are reported with honesty and transparency.