Randomised trials are at the heart of evidence-based healthcare, but the methods and infrastructure for conducting these sometimes complex studies are largely evidence free. Trial Forge (www.trialforge.org) is an initiative that aims to increase the evidence base for trial decision making and, in doing so, to improve trial efficiency.
This paper summarises a one-day workshop held in Edinburgh on 10 July 2014 to discuss Trial Forge and how to advance this initiative. We first outline the problem of inefficiency in randomised trials and go on to describe Trial Forge. We present participants’ views on the processes in the life of a randomised trial that should be covered by Trial Forge.
General support existed at the workshop for the Trial Forge approach to increase the evidence base for making randomised trial decisions and for improving trial efficiency. Agreed upon key processes included choosing the right research question; logistical planning for delivery, training of staff, recruitment, and retention; data management and dissemination; and close down. The process of linking to existing initiatives where possible was considered crucial. Trial Forge will not be a guideline or a checklist but a ‘go to’ website for research on randomised trials methods, with a linked programme of applied methodology research, coupled to an effective evidence-dissemination process. Moreover, it will support an informal network of interested trialists who meet virtually (online) and occasionally in person to build capacity and knowledge in the design and conduct of efficient randomised trials.
Some of the resources invested in randomised trials are wasted because of limited evidence upon which to base many aspects of design, conduct, analysis, and reporting of clinical trials. Trial Forge will help to address this lack of evidence.
Randomised controlled trials; methodology; efficiency; research waste
Objective To identify the number of drug-disease and drug-drug interactions for exemplar index conditions within National Institute of Health and Care Excellence (NICE) clinical guidelines.
Design Systematic identification, quantification, and classification of potentially serious drug-disease and drug-drug interactions for drugs recommended by NICE clinical guidelines for type 2 diabetes, heart failure, and depression in relation to 11 other common conditions and drugs recommended by NICE guidelines for those conditions.
Setting NICE clinical guidelines for type 2 diabetes, heart failure, and depression
Main outcome measures Potentially serious drug-disease and drug-drug interactions.
Results Following recommendations for prescription in 12 national clinical guidelines would result in several potentially serious drug interactions. There were 32 potentially serious drug-disease interactions between drugs recommended in the guideline for type 2 diabetes and the 11 other conditions compared with six for drugs recommended in the guideline for depression and 10 for drugs recommended in the guideline for heart failure. Of these drug-disease interactions, 27 (84%) in the type 2 diabetes guideline and all of those in the two other guidelines were between the recommended drug and chronic kidney disease. More potentially serious drug-drug interactions were identified between drugs recommended by guidelines for each of the three index conditions and drugs recommended by the guidelines for the 11 other conditions: 133 drug-drug interactions for drugs recommended in the type 2 diabetes guideline, 89 for depression, and 111 for heart failure. Few of these drug-disease or drug-drug interactions were highlighted in the guidelines for the three index conditions.
Conclusions Drug-disease interactions were relatively uncommon with the exception of interactions when a patient also has chronic kidney disease. Guideline developers could consider a more systematic approach regarding the potential for drug-disease interactions, based on epidemiological knowledge of the comorbidities of people with the disease the guideline is focused on, and should particularly consider whether chronic kidney disease is common in the target population. In contrast, potentially serious drug-drug interactions between recommended drugs for different conditions were common. The extensive number of potentially serious interactions requires innovative interactive approaches to the production and dissemination of guidelines to allow clinicians and patients with multimorbidity to make informed decisions about drug selection.
Breast cancer is the most commonly diagnosed cancer and the second cause of cancer deaths amongst women in the UK. The incidence of the disease is increasing and is highest in women from least deprived areas. It is estimated that around 42% of the disease in post-menopausal women could be prevented by increased physical activity and reductions in alcohol intake and body fatness. Breast cancer control endeavours focus on national screening programmes but these do not include communications or interventions for risk reduction.
This study aimed to assess the feasibility of delivery, indicative effects and acceptability of a lifestyle intervention programme initiated within the NHS Scottish Breast Screening Programme (NHSSBSP).
A 1:1 randomised controlled trial (RCT) of the 3 month ActWell programme (focussing on body weight, physical activity and alcohol) versus usual care conducted in two NHSSBSP sites between June 2013 and January 2014. Feasibility assessments included recruitment, retention, and fidelity to protocol. Indicative outcomes were measured at baseline and 3 month follow-up (body weight, waist circumference, eating and alcohol habits and physical activity). At study end, a questionnaire assessed participant satisfaction and qualitative interviews elicited women’s, coaches, and radiographers’ experiences. Statistical analysis used Chi squared tests for comparisons in proportions and paired t tests for comparisons of means. Linear regression analyses were performed, adjusted for baseline values, with group allocation as a fixed effect.
A pre-set recruitment target of 80 women was achieved within 12 weeks and 65 (81%) participants (29 intervention, 36 control) completed 3 month assessments. Mean age was 58 ± 5.6 years, mean BMI was 29.2 ± 7.0 kg/m2 and many (44%) reported a family history of breast cancer.
The primary analysis (baseline body weight adjusted) showed a significant between group difference favouring the intervention group of 2.04 kg (95% CI −3.24 kg to −0.85 kg). Significant, favourable between group differences were also detected for BMI, waist circumference, physical activity and sitting time. Women rated the programme highly and 70% said they would recommend it to others.
Recruitment, retention, indicative results and participant acceptability support the development of a definitive RCT to measure long term effects.
The trial was registered with Current Controlled Trials (ISRCTN56223933).
Electronic supplementary material
The online version of this article (doi:10.1186/s12966-014-0156-2) contains supplementary material, which is available to authorized users.
Breast cancer; Physical activity; Body weight; Alcohol; Sedentary time
Randomized controlled trials play a central role in evidence-based practice, but recruitment of participants, and retention of them once in the trial, is challenging. Moreover, there is a dearth of evidence that research teams can use to inform the development of their recruitment and retention strategies. As with other healthcare initiatives, the fairest test of the effectiveness of a recruitment strategy is a trial comparing alternatives, which for recruitment would mean embedding a recruitment trial within an ongoing host trial. Systematic reviews indicate that such studies are rare. Embedded trials are largely delivered in an ad hoc way, with interventions almost always developed in isolation and tested in the context of a single host trial, limiting their ability to contribute to a body of evidence with regard to a single recruitment intervention and to researchers working in different contexts.
The Systematic Techniques for Assisting Recruitment to Trials (START) program is funded by the United Kingdom Medical Research Council (MRC) Methodology Research Programme to support the routine adoption of embedded trials to test standardized recruitment interventions across ongoing host trials. To achieve this aim, the program involves three interrelated work packages: (1) methodology - to develop guidelines for the design, analysis and reporting of embedded recruitment studies; (2) interventions - to develop effective and useful recruitment interventions; and (3) implementation - to recruit host trials and test interventions through embedded studies.
Successful completion of the START program will provide a model for a platform for the wider trials community to use to evaluate recruitment interventions or, potentially, other types of intervention linked to trial conduct. It will also increase the evidence base for two types of recruitment intervention.
The START protocol covers the methodology for embedded trials. Each embedded trial is registered separately or as a substudy of the host trial.
Randomized controlled trial; Recruitment; Primary care; Community; Trial participation; Intervention; Multimedia; Decision support systems; Protocols; Participant information
Despite significant investment in infrastructure many trials continue to face challenges in recruitment and retention. We argue that insufficient focus has been placed on the development and testing of recruitment and retention interventions.
In this current paper, we summarize existing reviews about interventions to improve recruitment and retention. We report survey data from Clinical Trials Units in the United Kingdom to indicate the range of interventions used by these units to encourage recruitment and retention. We present the views of participants in a recent workshop and a priority list of recruitment interventions for evaluation (determined by voting among workshop participants). We also discuss wider issues concerning the testing of recruitment interventions.
Methods used to encourage recruitment and retention were categorized as: patient contact, patient convenience, support for recruiters, monitoring and systems, incentives, design, resources, and human factors. Interventions felt to merit investigation by respondents fell into three categories: training site staff, communication with patients, and incentives.
Significant resources continue to be invested into clinical trials and other high quality studies, but recruitment remains a significant challenge. Adoption of innovative methods to develop, test, and implement recruitment interventions are required.
Electronic supplementary material
The online version of this article (doi:10.1186/1745-6215-15-399) contains supplementary material, which is available to authorized users.
Trials; Recruitment; Retention
High-risk prescribing in primary care is common and causes considerable harm. Feedback interventions have small/moderate effects on clinical practice, but few trials explicitly compare different forms of feedback. There is growing recognition that intervention development should be theory-informed, and that comprehensive reporting of intervention design is required by potential users of trial findings. The paper describes intervention development for the Effective Feedback to Improve Primary Care Prescribing Safety (EFIPPS) study, a pragmatic three-arm cluster randomised trial in 262 Scottish general practices.
The NHS chose to implement a feedback intervention to utilise a new resource, new Prescribing Information System (newPIS). The development phase required selection of high-risk prescribing outcome measures and design of intervention components: (1) educational material (the usual care comparison), (2) feedback of practice rates of high-risk prescribing received by both intervention arms and (3) a theory-informed behaviour change component to be received by one intervention arm. Outcome measures, educational material and feedback design, were developed with a National Health Service Advisory Group. The behaviour change component was informed by the Theory of Planned Behaviour and the Health Action Process Approach. A focus group elicitation study and an email Delphi study with general practitioners (GPs) identified key attitudes and barriers of responding to the prescribing feedback. Behaviour change techniques were mapped to the psychological constructs, and the content was informed by the results of the elicitation and Delphi study.
Six high-risk prescribing measures were selected in a consensus process based on importance and feasibility. Educational material and feedback design were based on current NHS Scotland practice and Advisory Group recommendations. The behaviour change component was resource constrained in development, mirroring what is feasible in an NHS context. Four behaviour change interventions were developed and embedded in five quarterly rounds of feedback targeting attitudes, subjective norms, perceived behavioural control and action planning (2×).
The paper describes a process which is feasible to use in the resource-constrained environment of NHS-led intervention development and documents the intervention to make its design and implementation explicit to potential users of the trial findings.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0133-9) contains supplementary material, which is available to authorized users.
Feedback; Medication errors; Medication review; Inappropriate medication; Randomised controlled trial; Primary health care; Family practice; Intervention development; Behaviour change; ePrescribing
Delay to presentation with symptoms of acute coronary syndrome (ACS) is common meaning many fail to achieve optimal benefit from treatments. Interventions have had variable success in reducing delay. Evidence suggests inclusion of behaviour change techniques (BCTs) may improve effectiveness of interventions but this has not yet been systematically evaluated. Data from other time-critical conditions may be relevant.
Methods and analysis
A systematic review will be undertaken to identify which BCTs are associated with effective interventions to reduce patient delay (or prompt rapid help-seeking) among people with time-critical conditions (eg, chest pain, ACS, lumps, stroke, cancer and meningitis). A systematic search of a wide range of databases (including Cochrane Library, MEDLINE, EMBASE, CINAHL, PsycInfo) and grey literature will be undertaken to identify all relevant intervention studies (randomised controlled trials, controlled clinical trials and cohort studies). Two independent reviewers will screen abstracts to identify relevant studies, apply inclusion criteria to full papers, assess methodological quality and extract data.
Primary outcome measure
Change in patient decision time BCTs reported in each of the included studies will be categorised and presented according to the latest reliable taxonomy. Results of included studies will be synthesised, exploring relationships between inclusion of each BCT and effectiveness of the overall intervention. Where possible, means and SDs for differences in delay time will be calculated and combined within meta-analyses to derive a standardised mean difference and 95% CI. Analysis of (1) all time-critical and (2) ACS-only interventions will be undertaken.
Ethics and dissemination
No ethical issues are anticipated. Results will be submitted for publication in a relevant peer-reviewed journal.
Quality of Care and Outcomes
Clinical practice guidelines are typically written for healthcare providers but there is increasing interest in producing versions for the public, patients and carers. The main objective of this review is to identify and synthesise evidence of the public’s attitudes towards clinical practice guidelines and evidence-based recommendations written for providers or the public, together with their awareness of guidelines.
We included quantitative and qualitative studies of any design reporting on public, patient (and their carers) attitudes and awareness of guidelines written for providers or patients/public. We searched electronic databases including MEDLINE, PSYCHINFO, ERIC, ASSIA and the Cochrane Library from 2000 to 2012. We also searched relevant websites, reviewed citations and contacted experts in the field. At least two authors independently screened, abstracted data and assessed the quality of studies. We conducted a thematic analysis of first and second order themes and performed a separate narrative synthesis of patient and public awareness of guidelines.
We reviewed 5415 records and included 26 studies (10 qualitative studies, 13 cross sectional and 3 randomised controlled trials) involving 24 887 individuals. Studies were mostly good to fair quality. The thematic analysis resulted in four overarching themes: Applicability of guidelines; Purpose of guidelines for patient; Purpose of guidelines for health care system and physician; and Properties of guidelines. Overall, participants had mixed attitudes towards guidelines; some participants found them empowering but many saw them as a way of rationing care. Patients were also concerned that the information may not apply to their own health care situations. Awareness of guidelines ranged from 0-79%, with greater awareness in participants surveyed on national guideline websites.
There are many factors, not only formatting, that may affect the uptake and use of guideline-derived material by the public. Producers need to make clear how the information is relevant to the reader and how it can be used to make healthcare improvements although there were problems with data quality. Awareness of guidelines is generally low and guideline producers cannot assume that the public has a more positive perception of their material than of alternative sources of health information.
Clinical practice guidelines; Patient; Public; Attitudes; Awareness
Getting the results of research implemented into routine healthcare is often a challenge. The disconnect between the development and implementation of evidence into practice is called the ‘second translational gap’ and is particularly apparent in primary care. To address this gap, we plan to identify, summarise and synthesise currently available evidence by undertaking a systematic review of reviews to: (1) explore barriers and facilitators of implementation of research evidence or complex interventions, and (2) assess the effectiveness of strategies in facilitating implementation of complex interventions in primary care.
Methods and analysis
This is a protocol for a systematic review of reviews. We will search MEDLINE, EMBASE, the Cochrane Library, CINAHL and PsycINFO up until December 2013. We will check reference lists of included studies for further studies. Two authors will independently screen the titles and abstracts identified from the search; any discrepancies will be resolved by discussion and consensus. Full-text papers will be obtained and relevant reviews will be selected against inclusion criteria. Eligible reviews have to be based on predominantly primary care in developed countries and examine either factors to implementation or, the effectiveness of strategies to optimise implementation. Data from eligible reviews will be extracted using standardised data abstraction forms. For barriers and facilitators, data will be synthesised using an interpretative meta-synthesis approach. For implementation strategies, findings will be summarised and described narratively and synthesised using a framework approach. All findings will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.
Ethics and dissemination
Ethical approval is not required. The review findings will inform the work of the design and implementation of future studies and will be of interest to a wide audience including health professionals, researchers, health service or commissioning managers and policymakers.
Trial registration number
Protocol registration number (PROSPERO CRD42014009410).
Primary Care; Public Health
Patients with colorectal cancer report ongoing physical and psychological impairments and a high proportion of these patients are overweight, insufficiently active and high-risk drinkers, putting them at risk of poor recovery and risk of recurrence and comorbidities. A challenge is implementing sustainable and effective rehabilitation as part of routine care for this group.
Methods and analysis
A two-arm pilot randomised controlled trial (RCT) with embedded feasibility study undertaken as a phased programme of work. The intervention involves an existing cardiac rehabilitation programme for cardiac patients accepting colorectal cancer patient referrals. The intervention consists of supervised exercise sessions run by a cardiac physiotherapist and information sessions. Phase 1 will involve one research site enrolling 12 patients to assess intervention and study design processes. Semistructured interviews with patients with colorectal cancer and cardiac patients and clinicians will be used to gather data on acceptability of the intervention and study procedures. Phase 2 will involve three sites enrolling 66 patients with colorectal cancer randomised to control or intervention groups. Outcome measures will be taken preintervention and postintervention, for phases 1 and 2. The primary outcome is accelerometer measured physical activity; secondary outcomes are self-report physical activity, quality of life, anxiety, depression, symptoms including fatigue. The following variables will also be examined to determine if these factors influence adherence and outcomes: self-efficacy, risk perception and treatments.
Ethics and dissemination
Full ethical approval was granted by NRES Committees—North of Scotland (13/NS/0004; IRAS project ID: 121757) on 22 February 2013. The proposed work is novel in that it aims to test the feasibility and acceptability of using an evidence-based and theory driven existing cardiac rehabilitation service with patients with colorectal cancer. Should this model of rehabilitation prove to be clinically and cost effective we aim to conduct a randomised controlled trial of this intervention to measure effectiveness.
Trial registration reference
ISRCTN63510637; UKCRN id 14092.
SPORTS MEDICINE; REHABILITATION MEDICINE
Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item.
We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added.
We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items.
The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.
Postal questionnaires are simple and economical for collecting outcome data for randomised controlled trials (RCTs) but are prone to non-response. In the RECORD trial (a large pragmatic publicly funded RCT in UK) non-responders were sent a reminder and another questionnaire at 1 year, of which 40% were returned. In subsequent years we investigated the effect of an advance telephone call to non-responders on responses rate to reminder questionnaires and the next questionnaire 4 months later.
Non-responders to annual questionnaires were randomised to receive a telephone call from the trial office ahead of the reminder questionnaire in addition to the usual reminder schedule (n = 390) or to a control group that received the usual reminder schedule only (n = 363). The primary outcome was response to the reminder questionnaire within 21 days; secondary outcomes were response to a questionnaire 4 months later; completeness of quality of life instruments; and the number of participants declining further follow-up. Results are presented as odds ratios from a logistic regression intention-to-treat (ITT) analysis and then percentage difference and 95% confidence intervals (CI) for both ITT and average treatment effect on the treated (ATT) analyses.
The proportions that responded were 67.8% (265/390) in the intervention group compared to 62.5% (227/363) in the control group. The ITT estimate was a 5.4% increase (95% CI −1.4 to 12.2). Four months later percentages responding were 51.8% (202) and 42.7% (155). The ITT estimate was a 9.1% increase (95% CI 2.0 to 16.2). In the intervention group 12.3% (48/390) of participants were not telephoned because questionnaires were returned before the scheduled telephone call. ATT estimates adjusting for this were 6.2% (95% CI −1.6 to 14.0) and 10.4% (95% CI 2.2 to 18.5), respectively.
The telephone call resulted in a slight increase in response to the reminder questionnaire, however at 4 months later the proportion in the telephoned group responding was greater. This study suggests that pre-notification telephone calls may only be worthwhile if further questionnaires are to be sent out soon after reminder questionnaires.
Current Clinical Trials ISRCTN51647438
Telephone reminders; Postal questionnaires; Response rates; Average treatment effect on the treated
Recruiting participants to multicentre, community-based trials is a challenge. This case study describes how this challenge was met for the BeWEL trial, which evaluated the impact of a diet and physical activity intervention on body weight in people who had had pre-cancerous bowel polyps.
The BeWEL trial was a community-based trial, involving centres linked to the Scottish National Health Service (NHS) colorectal cancer screening programme. BeWEL had a recruitment target of 316 and its primary recruitment route was the colonoscopy clinics of the Scottish Bowel Screening Programme.
BeWEL exceeded its recruitment target but needed a 6-month no-cost extension from the funder to achieve this. The major causes of delay were lower consent rates (49% as opposed to 70% estimated from earlier work), the time taken for NHS research and development department approvals and the inclusion of two additional sites to increase recruitment, for which there were substantial bureaucratic delays. A range of specific interventions to increase recruitment, for example, telephone reminders and a shorter participant information leaflet, helped to increase the proportion of eligible individuals consenting and being randomized.
Recruitment to multicentre trials is a challenge but can be successfully achieved with a committed team. In a UK context, NHS research and development approval can be a substantial source of delay. Investigators should be cautious when estimating consent rates. If consent rates are less than expected, qualitative analysis might be beneficial, to try and identify the reason. Finally, investigators should select trial sites on the basis of a formal assessment of a site’s past performance and the likelihood of success in the trial being planned.
Current Controlled Trials ISRCTN53033856
Colorectal cancer; Multicentre trial; Recruitment
The prevalence of male obesity is increasing, but men are less likely than women to attend existing weight management programmes. We have taken a novel approach to reducing perceived barriers to weight loss for men by using professional football (soccer) clubs to encourage participation in a weight management group programme, gender-sensitised in content and style of delivery. Football Fans in Training (FFIT) provides 12 weeks of weight loss, physical activity and healthy eating advice at top professional football clubs in Scotland. This pilot randomized trial explored the feasibility of using these clubs as a setting for a randomized controlled trial of 12 month weight loss following men’s participation in FFIT.
A two-arm pilot trial at two Scottish Premier League football clubs (one large, one smaller), with 103 men (aged 35–65, body mass index (BMI) ≥27 kg/m2) individually randomized to the intervention (n=51, received the pilot programme (p-FFIT) immediately) and waitlist comparison (n=52, received p-FFIT after four months) groups. Feasibility of recruitment, randomization, data collection and retention were assessed. Objective physical measurements (weight, waist circumference, blood pressure, body composition) and questionnaires (self-reported physical activity, diet, alcohol consumption, psychological outcomes) were obtained from both groups by fieldworkers trained to standard protocols at baseline and 12 weeks, and from the intervention group at 6 and 12 months. Qualitative methods elicited men’s experiences of participation in the pilot trial.
Following a short recruitment period, the recruitment target was achieved at the large, but not smaller, club. Participants’ mean age was 47.1±8.4 years; mean BMI 34.5±5.0 kg/m2. Retention through the trial was good (>80% at 12 weeks and 6 months; >75% at 12 months), and 76% attended at least 80% of available programme delivery sessions. At 12 weeks, the intervention group lost significantly more weight than the comparison group (4.6% c.f. -0.6%, p<.001) and many maintained this to 12 months (intervention group baseline-12 month weight loss: 3.5%, p<.001). There were also improvements in self-reported physical activity and diet, many sustained long term.
The results demonstrated the feasibility of trial procedures and the potential of FFIT to engage men in sustained weight loss and positive lifestyle change. They supported the conduct of a fully-powered randomized controlled trial.
Overweight; Obesity; Physical activity; Diet; Behaviour change; Men; Gender; Masculinities; Intervention; Sports club
If you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention.
The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters’ subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of pragmatism.
We have concrete suggestions for improving PRECIS and a growing list of enthusiastic individuals interested in contributing to this work. By early 2014 we expect to have a validated PRECIS-2.
Pragmatic; Explanatory; Clinical trials; Trial design; Applicability
Understanding implementation processes is key to ensuring that complex interventions in healthcare are taken up in practice and thus maximize intended benefits for service provision and (ultimately) care to patients. Normalization Process Theory (NPT) provides a framework for understanding how a new intervention becomes part of normal practice. This study aims to develop and validate simple generic tools derived from NPT, to be used to improve the implementation of complex healthcare interventions.
The objectives of this study are to: develop a set of NPT-based measures and formatively evaluate their use for identifying implementation problems and monitoring progress; conduct preliminary evaluation of these measures across a range of interventions and contexts, and identify factors that affect this process; explore the utility of these measures for predicting outcomes; and develop an online users’ manual for the measures.
A combination of qualitative (workshops, item development, user feedback, cognitive interviews) and quantitative (survey) methods will be used to develop NPT measures, and test the utility of the measures in six healthcare intervention settings.
The measures developed in the study will be available for use by those involved in planning, implementing, and evaluating complex interventions in healthcare and have the potential to enhance the chances of their implementation, leading to sustained changes in working practices.
Normalization process theory; NPT; Implementation process; Survey; Instrument development; Complex interventions
This review is an abridged version of a Cochrane Review previously published in the Cochrane Database of Systematic Reviews 2010, Issue 4, Art. No.: MR000013 DOI: 10.1002/14651858.MR000013.pub5 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and Cochrane Database of Systematic Reviews should be consulted for the most recent version of the review.
To identify interventions designed to improve recruitment to randomised controlled trials, and to quantify their effect on trial participation.
The Cochrane Methodology Review Group Specialised Register in the Cochrane Library, MEDLINE, EMBASE, ERIC, Science Citation Index, Social Sciences Citation Index, C2-SPECTR, the National Research Register and PubMed. Most searches were undertaken up to 2010; no language restrictions were applied.
Randomised and quasi-randomised controlled trials, including those recruiting to hypothetical studies. Studies on retention strategies, examining ways to increase questionnaire response or evaluating the use of incentives for clinicians were excluded. The study population included any potential trial participant (eg, patient, clinician and member of the public), or individual or group of individuals responsible for trial recruitment (eg, clinicians, researchers and recruitment sites). Two authors independently screened identified studies for eligibility.
45 trials with over 43 000 participants were included. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (risk ratio (RR) 1.66, 95% CI 1.03 to 2.46; two studies, 1058 participants), use of opt-out rather than opt-in procedures for contacting potential participants (RR 1.39, 95% CI 1.06 to 1.84; one study, 152 participants) and open designs where participants know which treatment they are receiving in the trial (RR 1.22, 95% CI 1.09 to 1.36; two studies, 4833 participants). However, the effect of many other strategies is less clear, including the use of video to provide trial information and interventions aimed at recruiters.
There are promising strategies for increasing recruitment to trials, but some methods, such as open-trial designs and opt-out strategies, must be considered carefully as their use may also present methodological or ethical challenges. Questions remain as to the applicability of results originating from hypothetical trials, including those relating to the use of monetary incentives, and there is a clear knowledge gap with regard to effective strategies aimed at recruiters.
Statistics & Research Methods; Medical Ethics
Process evaluations are recommended to open the ‘black box’ of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials.
We reviewed recent theoretical and methodological literature and selected published process evaluations; these publications identified a need for structure to help design process evaluations. We drew upon this literature to develop a framework through iterative exchanges, and tested this against published evaluations.
The developed framework presents a range of candidate approaches to understanding trial delivery, intervention implementation and the responses of targeted participants. We believe this framework will be useful to others designing process evaluations of complex intervention trials. We also propose key information that process evaluations could report to facilitate their identification and enhance their usefulness.
There is no single best way to design and carry out a process evaluation. Researchers will be faced with choices about what questions to focus on and which methods to use. The most appropriate design depends on the purpose of the process evaluation; the framework aims to help researchers make explicit their choices of research questions and methods.
Process evaluation; Complex intervention; Cluster-randomised controlled trial; Qualitative; Quantitative; Reporting
Healthcare decision makers face challenges when using guidelines, including understanding the quality of the evidence or the values and preferences upon which recommendations are made, which are often not clear.
GRADE is a systematic approach towards assessing the quality of evidence and the strength of recommendations in healthcare. GRADE also gives advice on how to go from evidence to decisions. It has been developed to address the weaknesses of other grading systems and is now widely used internationally. The Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE) consortium (http://www.decide-collaboration.eu/), which includes members of the GRADE Working Group and other partners, will explore methods to ensure effective communication of evidence-based recommendations targeted at key stakeholders: healthcare professionals, policymakers, and managers, as well as patients and the general public. Surveys and interviews with guideline producers and other stakeholders will explore how presentation of the evidence could be improved to better meet their information needs. We will collect further stakeholder input from advisory groups, via consultations and user testing; this will be done across a wide range of healthcare systems in Europe, North America, and other countries. Targeted communication strategies will be developed, evaluated in randomized trials, refined, and assessed during the development of real guidelines.
Results of the DECIDE project will improve the communication of evidence-based healthcare recommendations. Building on the work of the GRADE Working Group, DECIDE will develop and evaluate methods that address communication needs of guideline users. The project will produce strategies for communicating recommendations that have been rigorously evaluated in diverse settings, and it will support the transfer of research into practice in healthcare systems globally.
Guidelines; Recommendations; Communication; Presentation formats