Breast cancer is the most commonly diagnosed cancer and the second cause of cancer deaths amongst women in the UK. The incidence of the disease is increasing and is highest in women from least deprived areas. It is estimated that around 42% of the disease in post-menopausal women could be prevented by increased physical activity and reductions in alcohol intake and body fatness. Breast cancer control endeavours focus on national screening programmes but these do not include communications or interventions for risk reduction.
This study aimed to assess the feasibility of delivery, indicative effects and acceptability of a lifestyle intervention programme initiated within the NHS Scottish Breast Screening Programme (NHSSBSP).
A 1:1 randomised controlled trial (RCT) of the 3 month ActWell programme (focussing on body weight, physical activity and alcohol) versus usual care conducted in two NHSSBSP sites between June 2013 and January 2014. Feasibility assessments included recruitment, retention, and fidelity to protocol. Indicative outcomes were measured at baseline and 3 month follow-up (body weight, waist circumference, eating and alcohol habits and physical activity). At study end, a questionnaire assessed participant satisfaction and qualitative interviews elicited women’s, coaches, and radiographers’ experiences. Statistical analysis used Chi squared tests for comparisons in proportions and paired t tests for comparisons of means. Linear regression analyses were performed, adjusted for baseline values, with group allocation as a fixed effect.
A pre-set recruitment target of 80 women was achieved within 12 weeks and 65 (81%) participants (29 intervention, 36 control) completed 3 month assessments. Mean age was 58 ± 5.6 years, mean BMI was 29.2 ± 7.0 kg/m2 and many (44%) reported a family history of breast cancer.
The primary analysis (baseline body weight adjusted) showed a significant between group difference favouring the intervention group of 2.04 kg (95% CI −3.24 kg to −0.85 kg). Significant, favourable between group differences were also detected for BMI, waist circumference, physical activity and sitting time. Women rated the programme highly and 70% said they would recommend it to others.
Recruitment, retention, indicative results and participant acceptability support the development of a definitive RCT to measure long term effects.
The trial was registered with Current Controlled Trials (ISRCTN56223933).
Electronic supplementary material
The online version of this article (doi:10.1186/s12966-014-0156-2) contains supplementary material, which is available to authorized users.
Breast cancer; Physical activity; Body weight; Alcohol; Sedentary time
Randomized controlled trials play a central role in evidence-based practice, but recruitment of participants, and retention of them once in the trial, is challenging. Moreover, there is a dearth of evidence that research teams can use to inform the development of their recruitment and retention strategies. As with other healthcare initiatives, the fairest test of the effectiveness of a recruitment strategy is a trial comparing alternatives, which for recruitment would mean embedding a recruitment trial within an ongoing host trial. Systematic reviews indicate that such studies are rare. Embedded trials are largely delivered in an ad hoc way, with interventions almost always developed in isolation and tested in the context of a single host trial, limiting their ability to contribute to a body of evidence with regard to a single recruitment intervention and to researchers working in different contexts.
The Systematic Techniques for Assisting Recruitment to Trials (START) program is funded by the United Kingdom Medical Research Council (MRC) Methodology Research Programme to support the routine adoption of embedded trials to test standardized recruitment interventions across ongoing host trials. To achieve this aim, the program involves three interrelated work packages: (1) methodology - to develop guidelines for the design, analysis and reporting of embedded recruitment studies; (2) interventions - to develop effective and useful recruitment interventions; and (3) implementation - to recruit host trials and test interventions through embedded studies.
Successful completion of the START program will provide a model for a platform for the wider trials community to use to evaluate recruitment interventions or, potentially, other types of intervention linked to trial conduct. It will also increase the evidence base for two types of recruitment intervention.
The START protocol covers the methodology for embedded trials. Each embedded trial is registered separately or as a substudy of the host trial.
Randomized controlled trial; Recruitment; Primary care; Community; Trial participation; Intervention; Multimedia; Decision support systems; Protocols; Participant information
Despite significant investment in infrastructure many trials continue to face challenges in recruitment and retention. We argue that insufficient focus has been placed on the development and testing of recruitment and retention interventions.
In this current paper, we summarize existing reviews about interventions to improve recruitment and retention. We report survey data from Clinical Trials Units in the United Kingdom to indicate the range of interventions used by these units to encourage recruitment and retention. We present the views of participants in a recent workshop and a priority list of recruitment interventions for evaluation (determined by voting among workshop participants). We also discuss wider issues concerning the testing of recruitment interventions.
Methods used to encourage recruitment and retention were categorized as: patient contact, patient convenience, support for recruiters, monitoring and systems, incentives, design, resources, and human factors. Interventions felt to merit investigation by respondents fell into three categories: training site staff, communication with patients, and incentives.
Significant resources continue to be invested into clinical trials and other high quality studies, but recruitment remains a significant challenge. Adoption of innovative methods to develop, test, and implement recruitment interventions are required.
Electronic supplementary material
The online version of this article (doi:10.1186/1745-6215-15-399) contains supplementary material, which is available to authorized users.
Trials; Recruitment; Retention
High-risk prescribing in primary care is common and causes considerable harm. Feedback interventions have small/moderate effects on clinical practice, but few trials explicitly compare different forms of feedback. There is growing recognition that intervention development should be theory-informed, and that comprehensive reporting of intervention design is required by potential users of trial findings. The paper describes intervention development for the Effective Feedback to Improve Primary Care Prescribing Safety (EFIPPS) study, a pragmatic three-arm cluster randomised trial in 262 Scottish general practices.
The NHS chose to implement a feedback intervention to utilise a new resource, new Prescribing Information System (newPIS). The development phase required selection of high-risk prescribing outcome measures and design of intervention components: (1) educational material (the usual care comparison), (2) feedback of practice rates of high-risk prescribing received by both intervention arms and (3) a theory-informed behaviour change component to be received by one intervention arm. Outcome measures, educational material and feedback design, were developed with a National Health Service Advisory Group. The behaviour change component was informed by the Theory of Planned Behaviour and the Health Action Process Approach. A focus group elicitation study and an email Delphi study with general practitioners (GPs) identified key attitudes and barriers of responding to the prescribing feedback. Behaviour change techniques were mapped to the psychological constructs, and the content was informed by the results of the elicitation and Delphi study.
Six high-risk prescribing measures were selected in a consensus process based on importance and feasibility. Educational material and feedback design were based on current NHS Scotland practice and Advisory Group recommendations. The behaviour change component was resource constrained in development, mirroring what is feasible in an NHS context. Four behaviour change interventions were developed and embedded in five quarterly rounds of feedback targeting attitudes, subjective norms, perceived behavioural control and action planning (2×).
The paper describes a process which is feasible to use in the resource-constrained environment of NHS-led intervention development and documents the intervention to make its design and implementation explicit to potential users of the trial findings.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0133-9) contains supplementary material, which is available to authorized users.
Feedback; Medication errors; Medication review; Inappropriate medication; Randomised controlled trial; Primary health care; Family practice; Intervention development; Behaviour change; ePrescribing
Delay to presentation with symptoms of acute coronary syndrome (ACS) is common meaning many fail to achieve optimal benefit from treatments. Interventions have had variable success in reducing delay. Evidence suggests inclusion of behaviour change techniques (BCTs) may improve effectiveness of interventions but this has not yet been systematically evaluated. Data from other time-critical conditions may be relevant.
Methods and analysis
A systematic review will be undertaken to identify which BCTs are associated with effective interventions to reduce patient delay (or prompt rapid help-seeking) among people with time-critical conditions (eg, chest pain, ACS, lumps, stroke, cancer and meningitis). A systematic search of a wide range of databases (including Cochrane Library, MEDLINE, EMBASE, CINAHL, PsycInfo) and grey literature will be undertaken to identify all relevant intervention studies (randomised controlled trials, controlled clinical trials and cohort studies). Two independent reviewers will screen abstracts to identify relevant studies, apply inclusion criteria to full papers, assess methodological quality and extract data.
Primary outcome measure
Change in patient decision time BCTs reported in each of the included studies will be categorised and presented according to the latest reliable taxonomy. Results of included studies will be synthesised, exploring relationships between inclusion of each BCT and effectiveness of the overall intervention. Where possible, means and SDs for differences in delay time will be calculated and combined within meta-analyses to derive a standardised mean difference and 95% CI. Analysis of (1) all time-critical and (2) ACS-only interventions will be undertaken.
Ethics and dissemination
No ethical issues are anticipated. Results will be submitted for publication in a relevant peer-reviewed journal.
Quality of Care and Outcomes
Clinical practice guidelines are typically written for healthcare providers but there is increasing interest in producing versions for the public, patients and carers. The main objective of this review is to identify and synthesise evidence of the public’s attitudes towards clinical practice guidelines and evidence-based recommendations written for providers or the public, together with their awareness of guidelines.
We included quantitative and qualitative studies of any design reporting on public, patient (and their carers) attitudes and awareness of guidelines written for providers or patients/public. We searched electronic databases including MEDLINE, PSYCHINFO, ERIC, ASSIA and the Cochrane Library from 2000 to 2012. We also searched relevant websites, reviewed citations and contacted experts in the field. At least two authors independently screened, abstracted data and assessed the quality of studies. We conducted a thematic analysis of first and second order themes and performed a separate narrative synthesis of patient and public awareness of guidelines.
We reviewed 5415 records and included 26 studies (10 qualitative studies, 13 cross sectional and 3 randomised controlled trials) involving 24 887 individuals. Studies were mostly good to fair quality. The thematic analysis resulted in four overarching themes: Applicability of guidelines; Purpose of guidelines for patient; Purpose of guidelines for health care system and physician; and Properties of guidelines. Overall, participants had mixed attitudes towards guidelines; some participants found them empowering but many saw them as a way of rationing care. Patients were also concerned that the information may not apply to their own health care situations. Awareness of guidelines ranged from 0-79%, with greater awareness in participants surveyed on national guideline websites.
There are many factors, not only formatting, that may affect the uptake and use of guideline-derived material by the public. Producers need to make clear how the information is relevant to the reader and how it can be used to make healthcare improvements although there were problems with data quality. Awareness of guidelines is generally low and guideline producers cannot assume that the public has a more positive perception of their material than of alternative sources of health information.
Clinical practice guidelines; Patient; Public; Attitudes; Awareness
Getting the results of research implemented into routine healthcare is often a challenge. The disconnect between the development and implementation of evidence into practice is called the ‘second translational gap’ and is particularly apparent in primary care. To address this gap, we plan to identify, summarise and synthesise currently available evidence by undertaking a systematic review of reviews to: (1) explore barriers and facilitators of implementation of research evidence or complex interventions, and (2) assess the effectiveness of strategies in facilitating implementation of complex interventions in primary care.
Methods and analysis
This is a protocol for a systematic review of reviews. We will search MEDLINE, EMBASE, the Cochrane Library, CINAHL and PsycINFO up until December 2013. We will check reference lists of included studies for further studies. Two authors will independently screen the titles and abstracts identified from the search; any discrepancies will be resolved by discussion and consensus. Full-text papers will be obtained and relevant reviews will be selected against inclusion criteria. Eligible reviews have to be based on predominantly primary care in developed countries and examine either factors to implementation or, the effectiveness of strategies to optimise implementation. Data from eligible reviews will be extracted using standardised data abstraction forms. For barriers and facilitators, data will be synthesised using an interpretative meta-synthesis approach. For implementation strategies, findings will be summarised and described narratively and synthesised using a framework approach. All findings will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.
Ethics and dissemination
Ethical approval is not required. The review findings will inform the work of the design and implementation of future studies and will be of interest to a wide audience including health professionals, researchers, health service or commissioning managers and policymakers.
Trial registration number
Protocol registration number (PROSPERO CRD42014009410).
Primary Care; Public Health
Patients with colorectal cancer report ongoing physical and psychological impairments and a high proportion of these patients are overweight, insufficiently active and high-risk drinkers, putting them at risk of poor recovery and risk of recurrence and comorbidities. A challenge is implementing sustainable and effective rehabilitation as part of routine care for this group.
Methods and analysis
A two-arm pilot randomised controlled trial (RCT) with embedded feasibility study undertaken as a phased programme of work. The intervention involves an existing cardiac rehabilitation programme for cardiac patients accepting colorectal cancer patient referrals. The intervention consists of supervised exercise sessions run by a cardiac physiotherapist and information sessions. Phase 1 will involve one research site enrolling 12 patients to assess intervention and study design processes. Semistructured interviews with patients with colorectal cancer and cardiac patients and clinicians will be used to gather data on acceptability of the intervention and study procedures. Phase 2 will involve three sites enrolling 66 patients with colorectal cancer randomised to control or intervention groups. Outcome measures will be taken preintervention and postintervention, for phases 1 and 2. The primary outcome is accelerometer measured physical activity; secondary outcomes are self-report physical activity, quality of life, anxiety, depression, symptoms including fatigue. The following variables will also be examined to determine if these factors influence adherence and outcomes: self-efficacy, risk perception and treatments.
Ethics and dissemination
Full ethical approval was granted by NRES Committees—North of Scotland (13/NS/0004; IRAS project ID: 121757) on 22 February 2013. The proposed work is novel in that it aims to test the feasibility and acceptability of using an evidence-based and theory driven existing cardiac rehabilitation service with patients with colorectal cancer. Should this model of rehabilitation prove to be clinically and cost effective we aim to conduct a randomised controlled trial of this intervention to measure effectiveness.
Trial registration reference
ISRCTN63510637; UKCRN id 14092.
SPORTS MEDICINE; REHABILITATION MEDICINE
Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item.
We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added.
We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items.
The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.
Postal questionnaires are simple and economical for collecting outcome data for randomised controlled trials (RCTs) but are prone to non-response. In the RECORD trial (a large pragmatic publicly funded RCT in UK) non-responders were sent a reminder and another questionnaire at 1 year, of which 40% were returned. In subsequent years we investigated the effect of an advance telephone call to non-responders on responses rate to reminder questionnaires and the next questionnaire 4 months later.
Non-responders to annual questionnaires were randomised to receive a telephone call from the trial office ahead of the reminder questionnaire in addition to the usual reminder schedule (n = 390) or to a control group that received the usual reminder schedule only (n = 363). The primary outcome was response to the reminder questionnaire within 21 days; secondary outcomes were response to a questionnaire 4 months later; completeness of quality of life instruments; and the number of participants declining further follow-up. Results are presented as odds ratios from a logistic regression intention-to-treat (ITT) analysis and then percentage difference and 95% confidence intervals (CI) for both ITT and average treatment effect on the treated (ATT) analyses.
The proportions that responded were 67.8% (265/390) in the intervention group compared to 62.5% (227/363) in the control group. The ITT estimate was a 5.4% increase (95% CI −1.4 to 12.2). Four months later percentages responding were 51.8% (202) and 42.7% (155). The ITT estimate was a 9.1% increase (95% CI 2.0 to 16.2). In the intervention group 12.3% (48/390) of participants were not telephoned because questionnaires were returned before the scheduled telephone call. ATT estimates adjusting for this were 6.2% (95% CI −1.6 to 14.0) and 10.4% (95% CI 2.2 to 18.5), respectively.
The telephone call resulted in a slight increase in response to the reminder questionnaire, however at 4 months later the proportion in the telephoned group responding was greater. This study suggests that pre-notification telephone calls may only be worthwhile if further questionnaires are to be sent out soon after reminder questionnaires.
Current Clinical Trials ISRCTN51647438
Telephone reminders; Postal questionnaires; Response rates; Average treatment effect on the treated
Recruiting participants to multicentre, community-based trials is a challenge. This case study describes how this challenge was met for the BeWEL trial, which evaluated the impact of a diet and physical activity intervention on body weight in people who had had pre-cancerous bowel polyps.
The BeWEL trial was a community-based trial, involving centres linked to the Scottish National Health Service (NHS) colorectal cancer screening programme. BeWEL had a recruitment target of 316 and its primary recruitment route was the colonoscopy clinics of the Scottish Bowel Screening Programme.
BeWEL exceeded its recruitment target but needed a 6-month no-cost extension from the funder to achieve this. The major causes of delay were lower consent rates (49% as opposed to 70% estimated from earlier work), the time taken for NHS research and development department approvals and the inclusion of two additional sites to increase recruitment, for which there were substantial bureaucratic delays. A range of specific interventions to increase recruitment, for example, telephone reminders and a shorter participant information leaflet, helped to increase the proportion of eligible individuals consenting and being randomized.
Recruitment to multicentre trials is a challenge but can be successfully achieved with a committed team. In a UK context, NHS research and development approval can be a substantial source of delay. Investigators should be cautious when estimating consent rates. If consent rates are less than expected, qualitative analysis might be beneficial, to try and identify the reason. Finally, investigators should select trial sites on the basis of a formal assessment of a site’s past performance and the likelihood of success in the trial being planned.
Current Controlled Trials ISRCTN53033856
Colorectal cancer; Multicentre trial; Recruitment
The prevalence of male obesity is increasing, but men are less likely than women to attend existing weight management programmes. We have taken a novel approach to reducing perceived barriers to weight loss for men by using professional football (soccer) clubs to encourage participation in a weight management group programme, gender-sensitised in content and style of delivery. Football Fans in Training (FFIT) provides 12 weeks of weight loss, physical activity and healthy eating advice at top professional football clubs in Scotland. This pilot randomized trial explored the feasibility of using these clubs as a setting for a randomized controlled trial of 12 month weight loss following men’s participation in FFIT.
A two-arm pilot trial at two Scottish Premier League football clubs (one large, one smaller), with 103 men (aged 35–65, body mass index (BMI) ≥27 kg/m2) individually randomized to the intervention (n=51, received the pilot programme (p-FFIT) immediately) and waitlist comparison (n=52, received p-FFIT after four months) groups. Feasibility of recruitment, randomization, data collection and retention were assessed. Objective physical measurements (weight, waist circumference, blood pressure, body composition) and questionnaires (self-reported physical activity, diet, alcohol consumption, psychological outcomes) were obtained from both groups by fieldworkers trained to standard protocols at baseline and 12 weeks, and from the intervention group at 6 and 12 months. Qualitative methods elicited men’s experiences of participation in the pilot trial.
Following a short recruitment period, the recruitment target was achieved at the large, but not smaller, club. Participants’ mean age was 47.1±8.4 years; mean BMI 34.5±5.0 kg/m2. Retention through the trial was good (>80% at 12 weeks and 6 months; >75% at 12 months), and 76% attended at least 80% of available programme delivery sessions. At 12 weeks, the intervention group lost significantly more weight than the comparison group (4.6% c.f. -0.6%, p<.001) and many maintained this to 12 months (intervention group baseline-12 month weight loss: 3.5%, p<.001). There were also improvements in self-reported physical activity and diet, many sustained long term.
The results demonstrated the feasibility of trial procedures and the potential of FFIT to engage men in sustained weight loss and positive lifestyle change. They supported the conduct of a fully-powered randomized controlled trial.
Overweight; Obesity; Physical activity; Diet; Behaviour change; Men; Gender; Masculinities; Intervention; Sports club
If you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention.
The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters’ subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of pragmatism.
We have concrete suggestions for improving PRECIS and a growing list of enthusiastic individuals interested in contributing to this work. By early 2014 we expect to have a validated PRECIS-2.
Pragmatic; Explanatory; Clinical trials; Trial design; Applicability
This review is an abridged version of a Cochrane Review previously published in the Cochrane Database of Systematic Reviews 2010, Issue 4, Art. No.: MR000013 DOI: 10.1002/14651858.MR000013.pub5 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and Cochrane Database of Systematic Reviews should be consulted for the most recent version of the review.
To identify interventions designed to improve recruitment to randomised controlled trials, and to quantify their effect on trial participation.
The Cochrane Methodology Review Group Specialised Register in the Cochrane Library, MEDLINE, EMBASE, ERIC, Science Citation Index, Social Sciences Citation Index, C2-SPECTR, the National Research Register and PubMed. Most searches were undertaken up to 2010; no language restrictions were applied.
Randomised and quasi-randomised controlled trials, including those recruiting to hypothetical studies. Studies on retention strategies, examining ways to increase questionnaire response or evaluating the use of incentives for clinicians were excluded. The study population included any potential trial participant (eg, patient, clinician and member of the public), or individual or group of individuals responsible for trial recruitment (eg, clinicians, researchers and recruitment sites). Two authors independently screened identified studies for eligibility.
45 trials with over 43 000 participants were included. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (risk ratio (RR) 1.66, 95% CI 1.03 to 2.46; two studies, 1058 participants), use of opt-out rather than opt-in procedures for contacting potential participants (RR 1.39, 95% CI 1.06 to 1.84; one study, 152 participants) and open designs where participants know which treatment they are receiving in the trial (RR 1.22, 95% CI 1.09 to 1.36; two studies, 4833 participants). However, the effect of many other strategies is less clear, including the use of video to provide trial information and interventions aimed at recruiters.
There are promising strategies for increasing recruitment to trials, but some methods, such as open-trial designs and opt-out strategies, must be considered carefully as their use may also present methodological or ethical challenges. Questions remain as to the applicability of results originating from hypothetical trials, including those relating to the use of monetary incentives, and there is a clear knowledge gap with regard to effective strategies aimed at recruiters.
Statistics & Research Methods; Medical Ethics
Process evaluations are recommended to open the ‘black box’ of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials.
We reviewed recent theoretical and methodological literature and selected published process evaluations; these publications identified a need for structure to help design process evaluations. We drew upon this literature to develop a framework through iterative exchanges, and tested this against published evaluations.
The developed framework presents a range of candidate approaches to understanding trial delivery, intervention implementation and the responses of targeted participants. We believe this framework will be useful to others designing process evaluations of complex intervention trials. We also propose key information that process evaluations could report to facilitate their identification and enhance their usefulness.
There is no single best way to design and carry out a process evaluation. Researchers will be faced with choices about what questions to focus on and which methods to use. The most appropriate design depends on the purpose of the process evaluation; the framework aims to help researchers make explicit their choices of research questions and methods.
Process evaluation; Complex intervention; Cluster-randomised controlled trial; Qualitative; Quantitative; Reporting
Healthcare decision makers face challenges when using guidelines, including understanding the quality of the evidence or the values and preferences upon which recommendations are made, which are often not clear.
GRADE is a systematic approach towards assessing the quality of evidence and the strength of recommendations in healthcare. GRADE also gives advice on how to go from evidence to decisions. It has been developed to address the weaknesses of other grading systems and is now widely used internationally. The Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE) consortium (http://www.decide-collaboration.eu/), which includes members of the GRADE Working Group and other partners, will explore methods to ensure effective communication of evidence-based recommendations targeted at key stakeholders: healthcare professionals, policymakers, and managers, as well as patients and the general public. Surveys and interviews with guideline producers and other stakeholders will explore how presentation of the evidence could be improved to better meet their information needs. We will collect further stakeholder input from advisory groups, via consultations and user testing; this will be done across a wide range of healthcare systems in Europe, North America, and other countries. Targeted communication strategies will be developed, evaluated in randomized trials, refined, and assessed during the development of real guidelines.
Results of the DECIDE project will improve the communication of evidence-based healthcare recommendations. Building on the work of the GRADE Working Group, DECIDE will develop and evaluate methods that address communication needs of guideline users. The project will produce strategies for communicating recommendations that have been rigorously evaluated in diverse settings, and it will support the transfer of research into practice in healthcare systems globally.
Guidelines; Recommendations; Communication; Presentation formats
High-risk prescribing in primary care is common and causes considerable harm. Feedback interventions to improve care are attractive because they are relatively cheap to widely implement. There is good evidence that feedback has small to moderate effects, but the most recent Cochrane review called for more high-quality, large trials that explicitly test different forms of feedback.
Methods and analysis
The study is a three-arm cluster-randomised trial with general practices being randomised and outcomes measured at patient level. 262 practices in three Scottish Health Board areas have been randomised (94% of all possible practices). The two active arms receive different forms of prescribing safety data feedback, with rates of high-risk prescribing compared with a ‘usual care’ arm. Sample size estimation used baseline data from participating practices. With 85 practices randomised to each arm, then there is 93% power to detect a 25% difference in the percentage of high-risk prescribing (from 6.1% to 4.5%) between the usual care arm and each intervention arm. The primary outcome is a composite of six high-risk prescribing measures (antipsychotic prescribing to people aged ≥75 years; non-steroidal anti-inflammatory drug (NSAID) prescribing to people aged ≥75 without gastroprotection; NSAID prescribing to people prescribed aspirin/clopidogrel without gastroprotection; NSAID prescribing to people prescribed an ACE inhibitor/angiotensin receptor blocker and a diuretic; NSAID prescription to people prescribed an oral anticoagulant without gastroprotection; aspirin/clopidogrel prescription to people prescribed an oral anticoagulant without gastroprotection). The primary analysis will use multilevel modelling to account for repeated measurement of outcomes in patients clustered within practices.
Ethics and dissemination
The study was reviewed and approved by the NHS Tayside Committee on Medical Research Ethics B (11/ES/0001). The study will be disseminated via a final report to the funder with a publicly available research summary, and peer reviewed publications.
ClinicalTrials.gov, dossier number NCT01602705.
Primary Care; Clinical Pharmacology; Randomised controlled trials; Audit and feedback; Medication review; Medication errors
Trials of complex interventions are criticized for being ‘black box’, so the UK Medical Research Council recommends carrying out a process evaluation to explain the trial findings. We believe it is good practice to pre-specify and publish process evaluation protocols to set standards and minimize bias. Unlike protocols for trials, little guidance or standards exist for the reporting of process evaluations. This paper presents the mixed-method process evaluation protocol of a cluster randomized trial, drawing on a framework designed by the authors.
This mixed-method evaluation is based on four research questions and maps data collection to a logic model of how the data-driven quality improvement in primary care (DQIP) intervention is expected to work. Data collection will be predominately by qualitative case studies in eight to ten of the trial practices, focus groups with patients affected by the intervention and quantitative analysis of routine practice data, trial outcome and questionnaire data and data from the DQIP intervention.
We believe that pre-specifying the intentions of a process evaluation can help to minimize bias arising from potentially misleading post-hoc analysis. We recognize it is also important to retain flexibility to examine the unexpected and the unintended. From that perspective, a mixed-methods evaluation allows the combination of exploratory and flexible qualitative work, and more pre-specified quantitative analysis, with each method contributing to the design, implementation and interpretation of the other.
As well as strengthening the study the authors hope to stimulate discussion among their academic colleagues about publishing protocols for evaluations of randomized trials of complex interventions.
Data-driven quality improvement in primary care trial registration
Complex intervention; Process evaluation; Protocol; Mixed methods; Randomized controlled trial
A number of single case reports have suggested that the context within which intervention studies take place may challenge the assumptions that underpin randomised controlled trials (RCTs). However, the diverse ways in which context may challenge the central tenets of the RCT, and the degree to which this information is known to researchers or subsequently reported, has received much less attention. In this paper, we explore these issues by focusing on seven RCTs of interventions varying in type and degree of complexity, and across diverse contexts.
This in-depth multiple case study using interviews, focus groups and documentary analysis was conducted in two phases. In phase one, a RCT of a nurse-led intervention provided a single exploratory case and informed the design, sampling and data collection within the main study. Phase two consisted of a multiple explanatory case study covering a spectrum of trials of different types of complex intervention. A total of eighty-four data sources across the seven trials were accessed.
We present consistent empirical evidence across all trials to indicate that four key elements of context (personal, organisational, trial and problem context) are crucial to understanding how a complex intervention works and to enable both assessments of internal validity and likely generalisability to other settings. The ways in which context challenged trial operation was often complex, idiosyncratic, and subtle; often falling outside of current trial reporting formats. However, information on such issues appeared to be available via first hand ‘insider accounts’ of each trial suggesting that improved reporting on the role of context is possible.
Sufficient detail about context needs to be understood and reported in RCTs of complex interventions, in order for the transferability of complex interventions to be assessed. Improved reporting formats that require and encourage the clarification of both general and project-specific threats to the likely internal and external validity need to be developed. In addition, a cultural change is required in which the open and honest reporting of such issues is seen as an indicator of study strength and researcher integrity, rather than a symbol of a poor quality study or investigator ability.
RCT; Case study; Context; Complex interventions
High-risk prescribing of non-steroidal anti-inflammatory drugs (NSAIDs) and antiplatelet agents accounts for a significant proportion of hospital admissions due to preventable adverse drug events. The recently completed PINCER trial has demonstrated that a one-off pharmacist-led information technology (IT)-based intervention can significantly reduce high-risk prescribing in primary care, but there is evidence that effects decrease over time and employing additional pharmacists to facilitate change may not be sustainable.
We will conduct a cluster randomised controlled with a stepped wedge design in 40 volunteer general practices in two Scottish health boards. Eligible practices are those that are using the INPS Vision clinical IT system, and have agreed to have relevant medication-related data to be automatically extracted from their electronic medical records. All practices (clusters) that agree to take part will receive the data-driven quality improvement in primary care (DQIP) intervention, but will be randomised to one of 10 start dates. The DQIP intervention has three components: a web-based informatics tool that provides weekly updated feedback of targeted prescribing at practice level, prompts the review of individual patients affected, and summarises each patient's relevant risk factors and prescribing; an outreach visit providing education on targeted prescribing and training in the use of the informatics tool; and a fixed payment of 350 GBP (560 USD; 403 EUR) up front and a small payment of 15 GBP (24 USD; 17 EUR) for each patient reviewed in the 12 months of the intervention. We hypothesise that the DQIP intervention will reduce a composite of nine previously validated measures of high-risk prescribing. Due to the nature of the intervention, it is not possible to blind practices, the core research team, or the data analyst. However, outcome assessment is entirely objective and automated. There will additionally be a process and economic evaluation alongside the main trial.
The DQIP intervention is an example of a potentially sustainable safety improvement intervention that builds on the existing National Health Service IT-infrastructure to facilitate systematic management of high-risk prescribing by existing practice staff. Although the focus in this trial is on Non-steroidal anti-inflammatory drugs and antiplatelets, we anticipate that the tested intervention would be generalisable to other types of prescribing if shown to be effective.
ClinicalTrials.gov, dossier number: NCT01425502
Adverse drug event; Non-steroidal anti-inflammatory drug; Antiplatelet; Medication error; Medication review; Decision support systems; Clinical; Stepped wedge; Randomised controlled trial; Primary healthcare