Our review identified numerous guidelines aiming to inform the content of clinical trial protocols. However, recommended concepts varied substantially across guidelines and the vast majority of guidelines did not describe their methods of development. When described, most included informal methods with limited stakeholder involvement and limited use of evidence to inform their recommendations. Similar findings have been reported elsewhere [20,
Very few concepts were recommended consistently across guidelines, including several whose importance is supported by empirical evidence. For example, only half of the more recent guidelines [59
] included an item recommending that primary outcomes be stated, despite preceding research showing biased modifications of primary outcomes throughout trials [3
]. Similarly, only three [6
] explicitly requested information regarding allocation concealment, the absence of which has been associated with inflated trial effect sizes [84
], although many requested general allocation methods. Conflicts of interest and roles of the sponsor in the trial were explicitly recommended in only one guideline [81
], despite being required by the Declaration of Helsinki [87
] and despite research showing that trials with financial competing interests report positive results more often than other trials [6
]. Only three [71
] guidelines explicitly recommended including or citing a systematic review as part of the trial rationale despite the problems associated with non-systematic literature searches [90
]. Finally, only 4 [46
] of 15 guidelines published after the introduction of trial registration requirements in 2005 [18
] specifically requested registration information. No guideline recommended all of these important concepts.
The reasons for the variation and omissions are unclear. Few of the guideline reports in our sample described their development methods, preventing assessment of the validity of the recommendations. If not properly developed, guidelines could potentially ultimately be of limited use and may not improve the reporting of elements that are important to key users of protocols. Of the eight guidelines that did detail methodology, four seem relatively comprehensive [42
]. Although these four shared many common elements, considerable variation in recommended content was also present.
For a guideline to be widely acceptable, we believe it should be developed using robust methodology that engages key stakeholders during development and is guided by empirical evidence, where possible. In addition, the methodology should be clearly reported and accessible to enable understanding of the process and assessment of its validity. Recommendations for reporting guideline development have recently been proposed [31
] and include a series of steps akin to those recommended for clinical practice guideline development [92
]: involvement of multidisciplinary expert panel for a formal consensus process (for example, Delphi consensus) and consensus meeting(s), literature reviews to identify key evidence, pilot testing, active dissemination and impact evaluation. Recent research conducted by the EQUATOR group on the development of health care reporting guidelines [20
] suggests that such extensive methods are rarely employed. This is congruent with our current findings.
This review has some limitations. Although comprehensive in searching indexed periodicals, our review was not exhaustive in the search for institutional guidelines or books. However, our main findings would not likely substantively change with the inclusion of guidelines from these sources, as most guidelines available outside of journal articles did not describe development methods. Our results are also based on the methodology stated in included reports; we did not contact authors for additional information. Finally, the process of mapping and comparing concepts across guidelines was challenging due to the varied terminology used and the many sub-concepts of general headings that were recommended. To decrease bias we employed a systematic method and a second reviewer verified the process.
Our systematic review highlights some potential limitations of existing clinical trial protocol content guidelines. Given the evidence of protocol deficiencies [11
] and the importance of trial protocols to diverse stakeholders we believe there is a need for standard guidance that is developed using rigorous methods, broad consultation with key stakeholders and is based on empirical evidence, where possible. Development of reporting guidelines requires substantial resources and time [31
], and the conduct of this review is an important inaugural step to justify undertaking such an initiative. Since the initial version of this review, an international collaboration known as the SPIRIT Initiative (S
ecommendations for I
rials) has convened to produce such guidance by systematically developing recommendations for minimum content of clinical trial protocols [93
]. The primary aim of SPIRIT is to improve the content and utility of clinical trial protocols.