PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of cmajCMAJ Information for AuthorsCMAJ Home Page
 
CMAJ. 2004 August 17; 171(4): 349–350.
PMCID: PMC509049

Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond

Reports of randomized controlled trials (RCTs) are the only tangible evidence of their conduct. Yet historians may well view the first 50 years of reporting of RCTs with some surprise. They will encounter what might be described as a cognitive dissonance: a disconnection between the increasing sophistication of the design (and the swelling cost) of these studies and the apparent lack of care — disastrous in some cases — with which they have been reported.

Improvements in the quality of reporting of RCTs must be based on strong evidence. A series of studies beginning in 1995 found empirical evidence that results may be biased when trials use inferior methods or are reported without adequate description of the methods; notably, failure to conceal the allocation process is associated with an exaggeration of the effectiveness of an intervention of 30% or more.1,2 The cause for concern is obvious: if the conduct or reporting of RCTs is poor, treatments may be introduced that are less effective than was thought or that might even be ineffective. This concern led the Consolidated Standards of Reporting Trials (CONSORT) group to formulate a series of recommendations in 19963 and 2001.4,5 This commentary summarizes developments within CONSORT since 2001.

In the 18 months since their publication, the revised CONSORT Statement4 and an explanatory document5were downloaded approximately 25 000 times from the Web sites of the 3 journals initially publishing them (the Annals of Internal Medicine, the Journal of the American Medical Association and The Lancet) and were cited nearly 500 times. There are about 20 000 hits weekly to the CONSORT Web site (www.consort-statement.org). International editorial groups, such as the World Association of Medical Editors and the Council of Science Editors, have endorsed the statement, as have several hundred biomedical journals, including this one.

Three studies have suggested that the quality of RCT reporting in journals that have endorsed the CONSORT statement is superior to that in journals that have not endorsed the statement.6,7,8 Even so, all journals continue to publish suboptimal reports of RCTs, and the global effort to improve the quality of reporting clinical research is far from over.

When formulating and revising the CONSORT recommendations for reporting the results of RCTs, the CONSORT group focused initially on trials that used a parallel group design with 2 treatment groups or “arms,” as this design is the most common in clinical research. However, a substantial minority of trials use alternative designs. Although most elements of the CONSORT statement apply equally to these other designs, certain elements need to be adapted, and in some cases additional elements need to be added for adequate reporting. Thus, our group is now developing CONSORT “extension papers” to fill in the gaps. A CONSORT extension paper for reporting randomized cluster (group) designs was published recently.9 Other extension papers in development consider equivalence and noninferiority, multiarmed parallel design, factorial (a special case of multiarmed) design, crossover design, and concurrent within individual trial designs. The 6 articles will have a standard structure, mirroring the features of previous publications.4,5

The poor quality of reporting of harm (in terms of safety and side effects) in RCTs has recently received considerable attention. Of 60 RCTs on antiretroviral treatment involving at least 100 patients, only a minority provided reasons for and numbers per arm of withdrawals resulting from toxic effects and numbers of participants with severe or life-threatening clinical adverse events.10These observations have been validated in a substantially larger study of 192 trials covering antiretroviral therapy and 6 other clinical content areas.11 To help address these problems the CONSORT Group developed a paper similar in format to the other extension papers. Ten recommendations that clarify harms-related issues are each accompanied by an explanation and examples to highlight specific aspects of proper reporting. For example, fever in vaccine trials may be defined with different cut-offs and may be measured at various body sites and at different times after vaccination.12 The results of such assessments are obviously problematic. The fourth recommendation asks authors to report whether the measurement instruments used to assess adverse events were standardized and validated. The CONSORT paper for reporting harms is forthcoming.

Several related initiatives have followed a similar model, aimed at improving the quality of reporting of other types of research: QUOROM for systematic reviews of randomized trials,13 MOOSE for systematic reviews of observational studies,14 STARD for diagnostic tests15,16 and (in the near future) guidelines for reports of controlled trials of botanical medicines and STROBE for observational studies. The groups developing these standards are seeking ways to work together under a unified “umbrella” so that they can capitalize on important synergies.

Continual review and updating of CONSORT are essential. Maintaining the activities of the CONSORT Group requires considerable effort, and a mechanism has been developed to monitor the evolving literature and help keep the CONSORT statement evidence-based. For some items on the CONSORT checklist, there is already solid evidence of an effect on the validity of the trial being reported. Methodological research validating other items is reported in a diverse set of journals, books and proceedings. To bring this body of evidence together, several CONSORT members have formed the ESCORT working party. They are starting to track down, appraise and annotate reports that provide Evidence Supporting (or refuting) the CONSORT Standards On Reporting Trials. The ESCORT group would appreciate receiving citations of reports readers consider relevant to any items on our checklist (via the CONSORT Web site).

These efforts would all benefit substantially if innovative funding mechanisms existed to provide practical support to build on and strengthen existing working relationships and extend this endeavour into new areas. However, with the exception of 5 years of a small amount of regular funding from the US National Library of Medicine, funding has been sporadic. Inconsistent funding does not provide the stability to enable long-term planning and execution of global efforts. Funding agencies spend millions of dollars to conduct RCTs, yet without sufficient attention to how those trials are conducted and reported, biases and misrepresentations will increasingly creep into their results and interpretations. Another paradox is that our requests for financial support to allow these groups to work together seem to fall on deaf ears, even though funding organizations are pushing for more international, interdisciplinary collaborations across multiple institutions, the exact makeup of CONSORT and the other groups.

More than 40 000 RCTs are now actively recruiting participants.17 If the results of these studies are to become the foundation of evidence-based health care, considerable energies and resources must be spent on improving their reporting.

Acknowledgments

Stephen Evans, Joel J. Gagnier, Peter C. Gøtzsche, David Grimes, Barbara Hawkins, Tom Lang, Yoon Kong Loke, David Sackett and Ida Sim provided valuable feedback on an earlier version of this article.

Footnotes

Editor's note: Up-to-date information about all CONSORT recommendations, including the extensions to specific trial designs and contexts and non-English-language versions, can be found at the CONSORT Web site (www.consort-statement.org), as can information about the other initiatives mentioned in this article.

This article has been peer reviewed.

Contributors: All 4 authors helped to conceive of the commentary and its contents and participated in its initial drafting and in the iterative revision process, including the final revision.

Competing interests: None declared.

Correspondence to: Dr. David Moher, Chalmers Research Group, Children's Hospital of Eastern Ontario Research Institute, Rm. 207, 401 Smyth Rd., Ottawa ON K1H 8L1; fax 613 738-4800; dmoher/at/uottawa.ca

References

1. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA1995;273:408-12. [PubMed]
2. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess2003;7:1-76. [PubMed]
3. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA1996;276:637-9. [PubMed]
4. Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet2001;357:1191-4. [PubMed]
5. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med2001;134:663-94. [PubMed]
6. Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. The CONSORT group. JAMA2001;285:1992-5. [PubMed]
7. Egger M, Jüni P, Bartlett C. Value of flow diagrams in reports of randomized controlled trials. The CONSORT group. JAMA2001;285:1996-9. [PubMed]
8. Devereaux PJ, Manns BJ, Ghali WA, Quan H, Guyatt GH. The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist. Control Clin Trials2002;23:380-8. [PubMed]
9. Campbell MK, Elbourne DR, Altman DG, for the CONSORT Group. CONSORT statement: extension to cluster randomised trials. BMJ 2004; 328: 702-8. Available: http://bmj.bmjjournals.com/cgi/content/full/328/7441/702%20(accessed 2004 May 21). [PMC free article] [PubMed]
10. Ioannidis JPA, Contopoulos-Ioannidis DG. Reporting of safety data from randomised trials. Lancet1998;352:1752-3. [PubMed]
11. Ioannidis JPA, Lau J. Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA2001;285:437-43. [PubMed]
12. Bonhoeffer J, Kohl K, Chen R, Duclos P, Heijbel H, Heininger U, et al. The Brighton Collaboration: addressing the need for standardized case definitions of adverse events following immunization (AEFI). Vaccine2002;21(3-4):298-302. [PubMed]
13. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup D, for the QUOROM group. Improving the quality of reporting of meta-analysis of randomized controlled trials: the QUOROM statement. Lancet1999;354: 1896-900. [PubMed]
14. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA2000;283:2008-12. [PubMed]
15. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al, for the STARD group. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD Initiative. Ann Intern Med2003; 138: 40-4. [PubMed]
16. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al, for the STARD group. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med2003; 138: W1-12. [PubMed]
17. CenterWatch Clinical trials listing service [online]. Boston: Thomson Centerwatch; 1995–2004. Available: www.centerwatch.com(accessed 2004 May 21).

Articles from CMAJ : Canadian Medical Association Journal are provided here courtesy of Canadian Medical Association