Summary of Main Findings
The results of the systematic review and survey consistently suggest that trialists’ decisions regarding centre selection in RCTs are driven by pragmatic considerations, such as: the ability to recruit patients, the centre staff’s motivation to participate and ensuring good communication. The findings suggest that generalisability of RCT results does not appear to drive centre selection: 40% (n
51) of the reviewed RCTs did not report any rationale for selecting the participating centres and only 31 studies (n
24%) explicitly acknowledged diversity or representativeness when including centres. Similarly, enrolling centres that ensure the generalisability of results in terms of population characteristics and clinical practice was relevant in current practice for 33% (n
23) and 29% (n
20) of survey respondents, respectively. In optimal practice, however, more than 50% of survey respondents were interested in ensuring generalisability.
Both the systematic review and the survey showed that trialists are primarily interested in centres that they trust to recruit well and meet intervention-related requirements based on prior experience and proven performance. Trialists are keen to recruit highly motivated centres, as illustrated by the focus group and survey results.
There are discrepancies between current and optimal centre selection (). In current practice centres are often enrolled based on pragmatic reasons such as: convenient location in relation to the trial office, or targeting of sites where centre staff are known to the Chief Investigator. Trialists acknowledge that in optimal practice centre selection should consider more factors such as: ensuring the generalisability of trial results; accounting for patient convenience or making centre enrolment decisions on a collective basis within the trial team. For example, the centre staff knowing the Chief Investigator appears to be important in current practice (n
29; 41%), but this is not the case in optimal practice (n
4; 6%). On the other hand, concern for patient convenience when selecting centres is currently rare (n
6; 9%), but should ideally be more prominent (n
Survey: Discrepancies between the current and optimal practice of centre selection for RCTs.
There is little evidence that centres normally included in RCTs are purposively chosen to be representative of their jurisdictions or that the selection process is consistent across RCTs. Only two studies included in our systematic review selected sites randomly from a pre-specified pool.
Strengths and Limitations
A key strength of our study is that it investigated the practice of centre selection from different perspectives and took a mixed methods approach to validate the findings, which appear consistent across methods. Moreover, this research explored the rationale underpinning centre selection, thus contributing to the understanding of conflicting tensions in trial conduct and design, as perceived by trialists. Furthermore, both the focus groups and the survey presented the views of a wide range of professionals involved in the design and conduct of clinical trials, thus contributing to the validity of our findings.
The choice for a mixed methods approach was founded upon the aim to build a comprehensive picture of the current practice of centre selection in trials. We chose to look at trials funded through the NIHR HTA Primary Research programme as this background gives them a clear view to influencing nationwide health policy. For this reason we only included trials with an explicit economic evaluation component, as an economic assessment is currently a pre-requisite for national policy changes in the UK. Furthermore, the potential for centre-induced bias is the greatest for economic outcomes.
One potential limitation of this research is that the systematic review considered RCTs in the UK NIHR HTA portfolio, of which only 7% recruited internationally. Nevertheless, the size of our sample and the coverage of diverse therapeutic areas contribute to the relevance of our findings. It is possible that centre selection is more thoroughly addressed in trials conducted in other countries; however there is little evidence to support this. Furthermore, our meta-summary concentrated on the explicit reporting of including centres and did not aim to generate a standalone theory on centre selection, for which a more in-depth qualitative technique would have been more appropriate. While the RCTs included in the review were publicly-funded, which can be regarded as a limitation, it can equally be argued that generalisability was of interest to the funder given the view to influence nationwide policy. This assumption may not hold for pharmaceutically-sponsored RCTs and generalisability may actually be worse accounted for than our findings show when the entire spectrum of public-private RCTs are considered.
Our systematic review included trials with an explicit economic evaluation component. While this may be viewed as a limitation, two further points must also be considered. Firstly, the UK decision making body i.e. the National Institute for Health and Clinical Excellence (NICE) requires evidence of cost-effectiveness before advising on the nationwide adoption of a medical technology. The economic evaluation component is therefore mandatory for such policy changes, which makes it extremely relevant in the context of generalisability and we attempted to incorporate it accordingly. Furthermore, and lending strength to the previous consideration, only 9 trials out of the 365 trials considered were excluded from our systematic review because they did not have an explicit economic evaluation component (). This suggests that their exclusion is unlikely to have biased our sample and confirms that most UK trials do indeed evaluate economic outcomes.
It can be argued that the potential bias associated with centre selection is of far more relevance for some trials (e.g. primary care and surgery trials) than for others (e.g. drug trials), therefore our meta-summary may have overestimated the extent of centre selection misreporting by pooling together various types of RCTs. However, our sample was dominated by non-pharmacologic trials and an exploratory subgroup analysis () revealed that pharmacologic trials actually did better than non-pharmacologic trials in reporting centre selection considerations (67% vs. 58%), but, as expected, were less concerned with generalisability (15% vs. 27%). We acknowledge that our study sample included a high proportion of non-pharmacologic trials, which may limit the applicability of our findings.
We also performed an exploratory sub-group analysis investigating the differential reporting of centre selection considerations in cluster RCTs and non-cluster RCTs, respectively (). The effect sizes suggest that cluster RCTs perform better than non-cluster RCTs in reporting centre selection considerations (93% vs. 57%), especially in relation to representativeness (43% vs. 22%) and trial participation (57% vs. 25%). While such a finding does not come as a surprise given the obvious interest of accounting for setting-dependent effects in cluster trials, the small number of such RCTs in our sample i.e. 14 out of 129, preclude any strong inferences to be made.
Finally, while a priming effect is possible when comparing current and ideal practice in the survey, the results are not consistent with such an effect: the centre’s ability to recruit patients and staff’s motivation to participate in the RCT were most prominent both in current and optimal practice, which testifies their importance for trialists. On the other hand, the largest relative increase in importance from current to optimal practice was for the three generalisability items. A response rate for the survey could not be calculated as completion relied on trials units’ directors distributing the survey link to their staff, given that individual contact details were not available.
Relation to Other Studies
Results presented here are in line with previous research indicating that reporting generalisability in trials is sub-optimal. The systematic review of Braslow et al 
included 414 randomised and non-randomised studies published in the area of mental health between 1981 and 1996. Using self-constructed external validity indices, the authors found that 75% of the studies did not address sample representativeness and only 3% employed a random or systematic sampling method. Jacquier et al 
reviewed 158 surgical RCTs and found that only 64 of them (41%) reported selection criteria for the surgeons performing the interventions. Eldridge et al 
found in their review of 34 cluster RCTs that 47% of studies did not discuss cluster generalisability.
While previous studies have used original or published checklists to assess generalisability, our study investigated what actually drives the inclusion of centres in RCTs rather than comparing research practice against an external framework. The findings offer a snapshot of what trialists themselves perceive to be important, both in current and optimal practice. As it has been previously suggested 
, the seminal problem with respect to generalisability is that, despite the available assessment frameworks, there is very little guidance as to how the relevant considerations can be incorporated in the trial design and how the degree of generalisability of a trial’s results can be quantified.
Significance and Relevance
Our findings suggest a large discrepancy between trialists’ perception of current and optimal centre selection for RCTs: generalisability of results is rarely incorporated explicitly in trial design, despite acknowledging that this should ideally happen. Several reasons could underlie the discrepancy between current and optimal practice: first, the trial protocols included in the systematic review may be subject to incomplete reporting and thus trialists’ judgements about generalisability may have been masked. Second, there is currently little explicit guidance towards incorporating generalisability when enrolling centres. This applies both to the parameters that should be considered and to the weighting algorithm to be applied. Moreover, it is only the recent CONSORT statement extension to randomised trials of non-pharmacologic interventions 
where generalisability with respect to centre is explicitly required when reporting trial findings; the CONSORT statements for parallel group trials 
and cluster trials 
are relatively vague about discussing generalisability issues. The inclusion of generalisability issues in standardised trial protocols (the SPIRIT Initiative 
) may be a crucial step towards a sustainable improvement in the field. Third, RCTs are primarily driven by clinical outcomes, which usually assume constant treatment effect across settings, and give less importance to economic outcomes, which are most sensitive to geographical variations.
Generalisability is of direct interest to pragmatic RCTs and much less so in explanatory trials. The PRECIS tool states that in a pragmatic trial the interventions are applied “by the full range of practitioners and in the full range of clinical settings, regardless of their expertise” 
. The implications in terms of centre selection are two-fold: from a trial design perspective, neglecting particular types of settings affects the pragmatic nature of a RCT; and from a research perspective, adequately reporting information on centre selection informs the assessment of a trial’s position on the pragmatic-explanatory continuum.
Our results show that most RCTs do not select their centres with a view to the generalisability of results, which leaves two alternatives: 1) the results are largely generalisable purely by chance; or 2) the results are assumed to be generalisable, but researchers and policy makers are unaware if this assumption holds. The problem is that the results of RCTs preferentially recruiting from a limited number of sites when hundreds of others may be available will actually inform policy recommendations for all the centres, which makes bias a legitimate concern, especially in outcomes such as cost-effectiveness. A legitimate future research direction is to develop a tool that can assess the extent of this bias.
It has to be acknowledged that the matter is primarily relevant for therapeutic areas where a choice of centres is available (e.g. GP practices, general hospitals). In the case of specialised care, where few specialist units exist, the issue becomes trivial.
Whilst trialists acknowledge the need for greater consideration of centre selection in order to ensure generalisability in ideal practice, a key concern is how such a change would be accomplished, particularly since the focus of centre selection is currently meeting recruitment targets. Two directions are apparent: first, there is a need for improved reporting of centre selection both in trial protocols and subsequent publications. This would enable readers to consider both the generalisability of the study population compared with the general population, but crucially also centre level characteristics (such as patient throughput) that may influence implementation and outcomes in real-life practice. Second, future research providing insights into quantitative methods of incorporating generalisability at the design stage, potentially leading towards an even more rational centre selection, can only benefit RCTs. Reviewers, journal editors and funding bodies may play a seminal role in facilitating this process.