Our systematic review found no strong evidence of a harmful or beneficial trial effect of participating in randomised controlled trials (RCTs). The five included RCTs provided limited evidence because of their small sample sizes and the nature of the questions they considered, but they did show that it is possible to consider questions about the effects of participating in RCTs by using randomised designs. Our interpretation of the 50 non-randomised cohort studies was limited by the quality and size of the comparisons and the wide variations in participants, clinical interventions, and outcomes. Most of the 85 non-randomised cohort comparisons found no statistically significant differences, although 10 reported better outcomes for patients in RCTs and four reported better outcomes for patients outside of RCTs.
Previous reviews that considered a less precise question than the one we evaluated drew varied conclusions. For example, one identified 14 articles reporting data from 21 trials and concluded that, if anything, randomised trials tend to have beneficial rather than harmful effects.4
Another review included seven of these 14 articles and 17 additional articles.6
Only eight of the studies compared trial patients with non-trial patients who met the same eligibility criteria, and it was only possible to separate treatment effects from trial effects in three of these trials. A further review found 10 comparisons on survival or quality of life of patients treated in RCTs for life threatening illnesses (eight were cancer treatments) with those treated outside RCTs.5
It found evidence of longer survival in participants, but the authors were not confident of the results.
Our review differs from these reviews in several ways, including the scope and comprehensiveness of our search, our method of analysis, and the question we asked, which controlled for differences in the effects of different interventions and differences between participants and non-participants. (See bmj.com
for an evidence profile of our results according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group.) Our results are based mainly on comparisons of cohorts and are subject to the usual uncertainty associated with observational studies.1
Additionally, we could not explain the significant heterogeneity between studies, which affects our confidence in the results and reduces the overall quality of information. Other relevant studies apart from those included in this review may exist, as indicated by the number of studies awaiting assessment and the difficulty we and others had in searching for this type of study in electronic databases. As we did not find evidence of publication bias, it is unlikely that the studies that we failed to identify would provide strong evidence of either harmful or beneficial effects.
An important corollary of this finding is that it counters suggestions that the results of RCTs cannot be applied to usual clinical practice, because most of the studies found no significant difference in outcomes for participants of RCTs compared with comparable non-participants who received similar treatment.
In most cases, RCTs seem to provide estimates of treatment effects that are applicable to comparable patients who receive similar interventions in usual clinical practice. In addition to being informed about the risks and harms of an intervention when invited to participate in RCTs, patients can be told that, independently of the effects of the interventions being compared, participating in a trial is likely to result in similar outcomes to patients who receive the same or similar treatment outside of the trial.
What is already known on this topic
Some people believe that participation in a randomised controlled trial (RCT) increases a patient's risk of a bad outcome
Some people claim that the results of RCTs are not applicable to usual clinical practice
What this study adds
Participants in RCTs had similar outcomes to comparable patients who received the same or similar treatment outside the trial
The results of RCTs are therefore applicable to comparable patients in usual clinical practice