The prevalence of obesity is increasing globally and will, if left unchecked, have major implications for both population health and costs to health services.
To assess the effectiveness of strategies to change the behaviour of health professionals and the organisation of care to promote weight reduction in overweight and obese people.
We updated the search for primary studies in the following databases, which were all interrogated from the previous (version 2) search date to May 2009: The Cochrane Central Register of Controlled Trials (which at this time incorporated all EPOC Specialised Register material) (The Cochrane Library 2009, Issue 1), MEDLINE (Ovid), EMBASE (Ovid), CINAHL (EBSCO), and PsycINFO (Ovid). We identified further potentially relevant studies from the reference lists of included studies.
Randomised controlled trials (RCTs) that compared routine provision of care with interventions aimed either at changing the behaviour of healthcare professionals or the organisation of care to promote weight reduction in overweight or obese adults.
Data collection and analysis
Two reviewers independently extracted data and assessed study quality.
We included six RCTs, involving more than 246 health professionals and 1324 overweight or obese patients. Four of the trials targeted professionals and two targeted the organisation of care. Most of the studies had methodological or reporting weaknesses indicating a risk of bias.
Meta-analysis of three trials that evaluated educational interventions aimed at GPs suggested that, compared to standard care, such interventions could reduce the average weight of patients after a year (by 1.2 kg, 95% CI −0.4 to 2.8 kg); however, there was moderate unexplained heterogeneity between their results (I2 = 41%). One trial found that reminders could change doctors’ practice, resulting in a significant reduction in weight among men (by 11.2 kg, 95% CI 1.7 to 20.7 kg) but not among women (who reduced weight by 1.3 kg, 95% CI −4.1 to 6.7 kg). One trial found that patients may lose more weight after a year if the care was provided by a dietitian (by 5.6 kg, 95% CI 4.8 to 6.4 kg) or by a doctor-dietitian team (by 6 kg, 95% CI 5 to 7 kg), as compared with standard care. One trial found no significant difference between standard care and either mail or phone interventions in reducing patients’ weight.
Most of the included trials had methodological or reporting weaknesses and were heterogeneous in terms of participants, interventions, outcomes, and settings, so we cannot draw any firm conclusions about the effectiveness of the interventions. All of the evaluated interventions would need further investigation before it was possible to recommend them as effective strategies.
Body Weight; Controlled Clinical Trials as Topic; Delivery of Health Care [organization & administration; standards]; Obesity [psychology; *therapy]; Overweight [psychology; therapy]; Patient Education as Topic; Professional Practice [organization & administration; *standards]; Randomized Controlled Trials as Topic; Weight Loss; Adult; Female; Humans; Male
There is considerable interest in the effectiveness of financial incentives in the delivery of health care. Incentives may be used in an attempt to increase the use of evidence-based treatments among healthcare professionals or to stimulate health professionals to change their clinical behaviour with respect to preventive, diagnostic and treatment decisions, or both. Financial incentives are an extrinsic source of motivation and exist when an individual can expect a monetary transfer which is made conditional on acting in a particular way. Since there are numerous reviews performed within the healthcare area describing the effects of various types of financial incentives, it is important to summarise the effectiveness of these in an overview to discern which are most effective in changing health professionals’ behaviour and patient outcomes.
To conduct an overview of systematic reviews that evaluates the impact of financial incentives on healthcare professional behaviour and patient outcomes.
We searched the Cochrane Database of Systematic Reviews (CDSR) (The Cochrane Library); Database of Abstracts of Reviews of Effectiveness (DARE); TRIP; MEDLINE; EMBASE; Science Citation Index; Social Science Citation Index; NHS EED; HEED; EconLit; and Program in Policy Decision-Making (PPd) (from their inception dates up to January 2010). We searched the reference lists of all included reviews and carried out a citation search of those papers which cited studies included in the review. We included both Cochrane and non-Cochrane reviews of randomised controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time series (ITSs) and controlled before and after studies (CBAs) that evaluated the effects of financial incentives on professional practice and patient outcomes, and that reported numerical results of the included individual studies. Two review authors independently extracted data and assessed the methodological quality of each review according to the AMSTAR criteria. We included systematic reviews of studies evaluating the effectiveness of any type of financial incentive. We grouped financial incentives into five groups: payment for working for a specified time period; payment for each service, episode or visit; payment for providing care for a patient or specific population; payment for providing a pre-specified level or providing a change in activity or quality of care; and mixed or other systems. We summarised data using vote counting.
We identified four reviews reporting on 32 studies. Two reviews scored 7 on the AMSTAR criteria (moderate, score 5 to 7, quality) and two scored 9 (high, score 8 to 11, quality). The reported quality of the included studies was, by a variety of methods, low to moderate. Payment for working for a specified time period was generally ineffective, improving 3/11 outcomes from one study reported in one review. Payment for each service, episode or visit was generally effective, improving 7/10 outcomes from five studies reported in three reviews; payment for providing care for a patient or specific population was generally effective, improving 48/69 outcomes from 13 studies reported in two reviews; payment for providing a pre-specified level or providing a change in activity or quality of care was generally effective, improving 17/20 reported outcomes from 10 studies reported in two reviews; and mixed and other systems were of mixed effectiveness, improving 20/31 reported outcomes from seven studies reported in three reviews. When looking at the effect of financial incentives overall across categories of outcomes, they were of mixed effectiveness on consultation or visit rates (improving 10/17 outcomes from three studies in two reviews); generally effective in improving processes of care (improving 41/57 outcomes from 19 studies in three reviews); generally effective in improving referrals and admissions (improving 11/16 outcomes from 11 studies in four reviews); generally ineffective in improving compliance with guidelines outcomes (improving 5/17 outcomes from five studies in two reviews); and generally effective in improving prescribing costs outcomes (improving 28/34 outcomes from 10 studies in one review).
Financial incentives may be effective in changing healthcare professional practice. The evidence has serious methodological limitations and is also very limited in its completeness and generalisability. We found no evidence from reviews that examined the effect of financial incentives on patient outcomes.
*Motivation; Capitation Fee; Delivery of Health Care [*economics]; Fee-for-Service Plans; Professional Practice [*economics]; Review Literature as Topic; Salaries and Fringe Benefits [*economics]; Treatment Outcome; Humans
Clinical practice is not always evidence-based and, therefore, may not optimise patient outcomes. Opinion leaders disseminating and implementing ‘best evidence’ is one method that holds promise as a strategy to bridge evidence-practice gaps.
To assess the effectiveness of the use of local opinion leaders in improving professional practice and patient outcomes.
We searched Cochrane EPOC Group Trials Register, the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, HMIC, Science Citation Index, Social Science Citation Index, ISI Conference Proceedings and World Cat Dissertations up to 5 May 2009. In addition, we searched reference lists of included articles.
Studies eligible for inclusion were randomised controlled trials investigating the effectiveness of using opinion leaders to disseminate evidence-based practice and reporting objective measures of professional performance and/or health outcomes.
Data collection and analysis
Two review authors independently extracted data from each study and assessed its risk of bias. For each trial, we calculated the median risk difference (RD) for compliance with desired practice, adjusting for baseline where data were available. We reported the median adjusted RD for each of the main comparisons.
We included 18 studies involving more than 296 hospitals and 318 PCPs. Fifteen studies (18 comparisons) contributed to the calculations of the median adjusted RD for the main comparisons. The effects of interventions varied across the 63 outcomes from 15% decrease in compliance to 72% increase in compliance with desired practice. The median adjusted RD for the main comparisons were: i) Opinion leaders compared to no intervention, +0.09; ii) Opinion leaders alone compared to a single intervention, +0.14; iii) Opinion leaders with one or more additional intervention(s) compared to the one or more additional intervention(s), +0.10; iv) Opinion leaders as part of multiple interventions compared to no intervention, +0.10. Overall, across all 18 studies the median adjusted RD was +0.12 representing a 12% absolute increase in compliance in the intervention group.
Opinion leaders alone or in combination with other interventions may successfully promote evidence-based practice, but effectiveness varies both within and between studies. These results are based on heterogeneous studies differing in terms of type of intervention, setting, and outcomes measured. In most of the studies the role of the opinion leader was not clearly described, and it is therefore not possible to say what the best way is to optimise the effectiveness of opinion leaders.
*Leadership; *Policy Making; Evidence-Based Medicine [*standards]; Information Dissemination; Physician’s Practice Patterns; Process Assessment (Health Care); Professional Practice [*standards]; Randomized Controlled Trials as Topic; Humans
Inspection systems are used in health care to promote quality improvements, i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes. These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of health care. However, the benefits of external inspection in terms of organisational, provider and patient level outcomes are not clear.
To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes.
We searched the following electronic databases for studies: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, Scopus, HMIC, Index to Theses and Intute from their inception dates up to May 2011. There was no language restriction and studies were included regardless of publication status. We searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardisation (ISO), regarding any further published or unpublished work.
We included randomised controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time-series (ITSs) and controlled before and after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations.
Data collection and analysis
Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary.
We identified one cluster-RCT involving 20 South African public hospitals (Salmon 2003) and one ITS involving all acute trusts in England (OPM 2009) for inclusion in this review.
Salmon and colleagues (Salmon 2003) showed mixed effects of a hospital accreditation system on the compliance with COHSASA (the Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Significantly improved total mean compliance score with COHSASA accreditation standards was found for 21/28 service elements: mean intervention effect (95% confidence interval (CI)) was 30% (23% to 57%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). A sub-analysis of 424 a priori identified critical criteria (19 service elements) showed significantly improved compliance with the critical standards (P < 0.001). The score increased from 41% (21% to 46%) to 75% (55% to 96%) in intervention hospitals, but was unchanged in control hospitals (37%). Only one of the nine intervention hospitals gained full accreditation status at the end of the study period, with two others reached pre-accreditation status.The median intervention effect (range) for the indicators of hospital quality of care was 2.4 (−1.9 to +11.8) and only one of the eight indicators: ‘nurses perception of clinical quality, participation and teamwork’ was significantly improved (mean intervention effect 5.7, P = 0.03).
Re-analysis of the MRSA (methicillin-resistant Staphylococcus aureus) data showed statistically non-significant effects of the Healthcare Commissions Infection Inspection programme.
We only identified two studies for inclusion in this review, which highlights the paucity of high-quality controlled evaluations of the effectiveness of external inspection systems. No firm conclusions could therefore be drawn about the effectiveness of external inspection on compliance with standards.
Accreditation [*standards]; Commission on Professional and Hospital Activities [*standards]; Cross Infection [epidemiology]; England; Guideline Adherence [standards]; Hospitals [*standards]; Methicillin-Resistant Staphylococcus aureus; Organizational Culture; Outcome Assessment (Health Care) [standards]; Professional Practice [*standards]; Quality Assurance, Health Care [*standards]; Quality Improvement [*standards]; Randomized Controlled Trials as Topic; South Africa; Staphylococcal Infections [epidemiology]
It is becoming increasingly common to release information about the performance of hospitals, health professionals or providers, and healthcare organisations into the public domain. However, we do not know how this information is used and to what extent such reporting leads to quality improvement by changing the behaviour of healthcare consumers, providers and purchasers, or to what extent the performance of professionals and providers can be affected.
To determine the effectiveness of the public release of performance data in changing the behaviour of healthcare consumers, professionals and organisations.
We searched the Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Effective Practice and Organisation of Care (EPOC) Trials Register, MEDLINE Ovid (from 1966), EMBASE Ovid (from 1979), CINAHL, PsycINFO Ovid (from 1806) and DARE up to 2011.
We searched for randomised or quasi-randomised trials, interrupted time series and controlled before-after studies of the effects of publicly releasing data regarding any aspect of the performance of healthcare organisations or individuals. The papers had to report at least one main outcome related to selecting or changing care. Other outcome measures were awareness, attitude, views and knowledge of performance data and costs.
Data collection and analysis
Two review authors independently screened studies for eligibility and extracted data. For each study, we extracted data about the target groups (healthcare consumers, healthcare providers and healthcare purchasers), performance data, main outcomes (choice of healthcare provider and improvement by means of changes in care) and other outcomes (awareness, attitude, views, knowledge of performance data and costs).
We included four studies containing more than 35,000 consumers, and 1560 hospitals. Three studies were conducted in the USA and examined consumer behaviour after the public release of performance data. Two studies found no effect of Consumer Assessment of Healthcare Providers and Systems information on health plan choice in a Medicaid population. One interrupted time series study found a small positive effect of the publishing of data on patient volumes for coronary bypass surgery and low-complication outliers for lumbar discectomy, but these effects did not persist longer than two months after each public release. No effects on patient volumes for acute myocardial infarction were found.
One cluster-randomised controlled trial, conducted in Canada, studied improvement changes in care after the public release of performance data for patients with acute myocardial infarction and congestive heart failure. No effects for the composite process-of-care indicators for either condition were found, but there were some improvements in the individual process-of-care indicators. There was an effect on the mortality rates for acute myocardial infarction. More quality improvement activities were initiated in response to the publicly-released report cards. No secondary outcomes were reported.
The small body of evidence available provides no consistent evidence that the public release of performance data changes consumer behaviour or improves care. Evidence that the public release of performance data may have an impact on the behaviour of healthcare professionals or organisations is lacking.
*Information Dissemination; *Quality Improvement; Canada; Consumer Health Information [*methods]; Evaluation Studies as Topic; Health Maintenance Organizations [standards]; Hospitals [*standards]; Medicaid; Organizational Innovation; Quality Assurance, Health Care [*methods]; Randomized Controlled Trials as Topic; Reproducibility of Results; United States; Humans
One of the greatest challenges in healthcare is how to best translate research evidence into clinical practice, which includes how to change health-care professionals’ behaviours. A commonly held view is that multifaceted interventions are more effective than single-component interventions. The purpose of this study was to conduct an overview of systematic reviews to evaluate the effectiveness of multifaceted interventions in comparison to single-component interventions in changing health-care professionals’ behaviour in clinical settings.
The Rx for Change database, which consists of quality-appraised systematic reviews of interventions to change health-care professional behaviour, was used to identify systematic reviews for the overview. Dual, independent screening and data extraction was conducted. Included reviews used three different approaches (of varying methodological robustness) to evaluate the effectiveness of multifaceted interventions: (1) effect size/dose-response statistical analyses, (2) direct (non-statistical) comparisons of multifaceted to single interventions and (3) indirect comparisons of multifaceted to single interventions.
Twenty-five reviews were included in the overview. Three reviews provided effect size/dose-response statistical analyses of the effectiveness of multifaceted interventions; no statistical evidence of a relationship between the number of intervention components and the effect size was found. Eight reviews reported direct (non-statistical) comparisons of multifaceted to single-component interventions; four of these reviews found multifaceted interventions to be generally effective compared to single interventions, while the remaining four reviews found that multifaceted interventions had either mixed effects or were generally ineffective compared to single interventions. Twenty-three reviews indirectly compared the effectiveness of multifaceted to single interventions; nine of which also reported either a statistical (dose-response) analysis (N = 2) or a non-statistical direct comparison (N = 7). The majority (N = 15) of reviews reporting indirect comparisons of multifaceted to single interventions showed similar effectiveness for multifaceted and single interventions when compared to controls. Of the remaining eight reviews, six found single interventions to be generally effective while multifaceted had mixed effectiveness.
This overview of systematic reviews offers no compelling evidence that multifaceted interventions are more effective than single-component interventions.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0152-6) contains supplementary material, which is available to authorized users.
The opportunity to improve care by delivering decision support to clinicians at the point of care represents one of the main incentives for implementing sophisticated clinical information systems. Previous reviews of computer reminder and decision support systems have reported mixed effects, possibly because they did not distinguish point of care computer reminders from e-mail alerts, computer-generated paper reminders, and other modes of delivering ‘computer reminders’.
To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.
We searched the Cochrane EPOC Group Trials register, MEDLINE, EMBASE and CINAHL and CENTRAL to July 2008, and scanned bibliographies from key articles.
Studies of a reminder delivered via a computer system routinely used by clinicians, with a randomised or quasi-randomised design and reporting at least one outcome involving a clinical endpoint or adherence to a recommended process of care.
Data collection and analysis
Two authors independently screened studies for eligibility and abstracted data. For each study, we calculated the median improvement in adherence to target processes of care and also identified the outcome with the largest such improvement. We then calculated the median absolute improvement in process adherence across all studies using both the median outcome from each study and the best outcome.
Twenty-eight studies (reporting a total of thirty-two comparisons) were included. Computer reminders achieved a median improvement in process adherence of 4.2% (interquartile range (IQR): 0.8% to 18.8%) across all reported process outcomes, 3.3% (IQR: 0.5% to 10.6%) for medication ordering, 3.8% (IQR: 0.5% to 6.6%) for vaccinations, and 3.8% (IQR: 0.4% to 16.3%) for test ordering. In a sensitivity analysis using the best outcome from each study, the median improvement was 5.6% (IQR: 2.0% to 19.2%) across all process measures and 6.2% (IQR: 3.0% to 28.0%) across measures of medication ordering.
In the eight comparisons that reported dichotomous clinical endpoints, intervention patients experienced a median absolute improvement of 2.5% (IQR: 1.3% to 4.2%). Blood pressure was the most commonly reported clinical endpoint, with intervention patients experiencing a median reduction in their systolic blood pressure of 1.0 mmHg (IQR: 2.3 mmHg reduction to 2.0 mmHg increase).
Point of care computer reminders generally achieve small to modest improvements in provider behaviour. A minority of interventions showed larger effects, but no specific reminder or contextual features were significantly associated with effect magnitude. Further research must identify design features and contextual factors consistently associated with larger improvements in provider behaviour if computer reminders are to succeed on more than a trial and error basis.
*Decision Support Systems, Clinical; *Outcome and Process Assessment (Health Care); *Point-of-Care Systems; *Reminder Systems; Decision Making, Computer-Assisted; Humans
Organisational culture is an anthropological metaphor used to inform research and consultancy and to explain organisational environments. Great emphasis has been placed during the last years on the need to change organisational culture in order to pursue effective improvement of healthcare performance. However, the precise nature of organisational culture in healthcare policy often remains underspecified and the desirability and feasibility of strategies to be adopted has been called into question.
To determine the effectiveness of strategies to change organisational culture in order to improve healthcare performance.
To examine the effectiveness of these strategies according to different patterns of organisational culture.
We searched the following electronic databases for primary studies: The Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL, Sociological Abstracts, Web of Knowledge, PsycINFO, Business and Management, EThOS, Index to Theses, Intute, HMIC, SIGLE, and Scopus until October 2009. The Database of Abstracts of Reviews of Effectiveness (DARE) was searched for related reviews. We also searched the reference lists of all papers and relevant reviews identified, and we contacted experts in the field for advice on further potential studies.
We considered randomised controlled trials (RCTs) or well designed quasi-experimental studies, controlled clinical trials (CCTs), controlled before and after studies (CBAs) and interrupted time series analyses (ITS) meeting the quality criteria used by the Cochrane Effective Practice and Organisation of Care Group (EPOC). Studies should be set in any type of healthcare organisation in which strategies to change organisational culture in order to improve healthcare performance were applied. Our main outcomes were objective measures of professional performance and patient outcome.
Data collection and analysis
At least two review authors independently applied the criteria for inclusion and exclusion criteria to scan titles and abstracts and then to screen the full reports of selected citations. At each stage results were compared and discrepancies solved through discussion.
The search strategy yielded 4239 records. After the full text assessment, no studies met the quality criteria used by the EPOC Group and evaluated the effectiveness of strategies to change organisational culture to improve healthcare performance.
It is not possible to draw any conclusions about the effectiveness of strategies to change organisational culture because we found no studies that fulfilled the methodological criteria for this review. Research efforts should focus on strengthening the evidence about the effectiveness of methods to change organisational culture to improve health care performance.
Organizational Culture; Quality of Health Care [*organization & administration; standards]
Reporting of adverse clinical events is thought to be an effective method of improving the safety of healthcare. Underreporting of these adverse events is often said to occur with consequence of missing of opportunities to learn from these incidents. A clinical incident can be defined as any occurrence which is not consistent with the routine care of the patient or the routine operation of the institution.
To assess the effects of interventions designed to increase clinical incident reporting in healthcare settings.
We searched the the following databases: Cochrane Effective Practice and Organisation of Care Group Specialised Register, the Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (OVID), EMBASE (OVID), CINAHL (EBSCO), Social Science Citation Index and Science Citation Index (Web of Knowledge), Healthstar (OVID), INSPEC, DHSS-DATA, SIGLE, ISI Conference Proceedings, Web of Science Conference Proceedings Citation Index (Science), Database of Abstracts of Reviews of Effectiveness (DARE).
Randomised controlled trials (RCT), controlled before-after studies (CBA) and interrupted time series (ITS) of interventions designed to increase clinical incident reporting in healthcare.
Data collection and analysis
At least two review authors assessed the eligibility of potentially relevant studies, extracted the data and assessed the quality of included studies.
Four studies (one CBA and three ITS studies) met our inclusion criteria and were included in the review. The CBA study showed a significant improvement in incident reporting rates after the introduction of the new reporting system. Just one of the ITS studies showed a statistically significant improved effectiveness of the new reporting system from nine months. The other two studies reported no statistically significant improvements.
Because of the limitations of the studies it is not possible to draw conclusions for clinical practice. Anyone introducing a system into practice should give careful consideration to conducting an evaluation using a robust design.
Medical Errors [prevention & control]; Risk Management [organization & administration; *standards; utilization]; Humans
Theory-based process evaluations conducted alongside randomized controlled trials provide the opportunity to investigate hypothesized mechanisms of action of interventions, helping to build a cumulative knowledge base and to inform the interpretation of individual trial outcomes. Our objective was to identify the underlying causal mechanisms in a cluster randomized trial of the effectiveness of printed educational materials (PEMs) to increase referral for diabetic retinopathy screening. We hypothesized that the PEMs would increase physicians’ intention to refer patients for retinal screening by strengthening their attitude and subjective norm, but not their perceived behavioral control.
Design: A theory based process evaluation alongside the Ontario Printed Educational Material (OPEM) cluster randomized trial. Postal surveys based on the Theory of Planned Behavior were sent to a random sample of trial participants two months before and six months after they received the intervention. Setting: Family physicians in Ontario, Canada. Participants: 1,512 family physicians (252 per intervention group) from the OPEM trial were invited to participate, and 31.3% (473/1512) responded at time one and time two. The final sample comprised 437 family physicians fully completing questionnaires at both time points. Main outcome measures: Primary: behavioral intention related to referring patient for retinopathy screening; secondary: attitude, subjective norm, perceived behavioral control.
At baseline, family physicians reported positive intention, attitude, subjective norm, and perceived behavioral control to advise patients about retinopathy screening suggesting limited opportunities for improvement in these constructs. There were no significant differences on intention, attitude, subjective norm, and perceived behavioral control following the intervention. Respondents also reported additional physician- and patient-related factors perceived to influence whether patients received retinopathy screening.
Lack of change in the primary and secondary theory-based outcomes provides an explanation for the lack of observed effect of the main OPEM trial. High baseline levels of intention to advise patients to attend retinopathy screening suggest that post-intentional and other factors may explain gaps in care. Process evaluations based on behavioral theory can provide replicable and generalizable insights to aid interpretation of randomized controlled trials of complex interventions to change health professional behavior.
Electronic supplementary material
The online version of this article (doi:10.1186/1748-5908-9-86) contains supplementary material, which is available to authorized users.
Process evaluation; Theory of planned behavior; Printed educational material; Healthcare professional behavior; Behavior change
The Ottawa Ethics of Cluster Trials Consensus Group sets out 15 recommendations for the ethical design and conduct of cluster randomized trials.
New clinical research findings may require clinicians to change their behaviour to provide high-quality care to people with type 2 diabetes, likely requiring them to change multiple different clinical behaviours. The present study builds on findings from a UK-wide study of theory-based behavioural and organisational factors associated with prescribing, advising, and examining consistent with high-quality diabetes care.
To develop and evaluate the effectiveness and cost of an intervention to improve multiple behaviours in clinicians involved in delivering high-quality care for type 2 diabetes.
We will conduct a two-armed cluster randomised controlled trial in 44 general practices in the North East of England to evaluate a theory-based behaviour change intervention. We will target improvement in six underperformed clinical behaviours highlighted in quality standards for type 2 diabetes: prescribing for hypertension; prescribing for glycaemic control; providing physical activity advice; providing nutrition advice; providing on-going education; and ensuring that feet have been examined. The primary outcome will be the proportion of patients appropriately prescribed and examined (using anonymised computer records), and advised (using anonymous patient surveys) at 12 months. We will use behaviour change techniques targeting motivational, volitional, and impulsive factors that we have previously demonstrated to be predictive of multiple health professional behaviours involved in high-quality type 2 diabetes care. We will also investigate whether the intervention was delivered as designed (fidelity) by coding audiotaped workshops and interventionist delivery reports, and operated as hypothesised (process evaluation) by analysing responses to theory-based postal questionnaires. In addition, we will conduct post-trial qualitative interviews with practice teams to further inform the process evaluation, and a post-trial economic analysis to estimate the costs of the intervention and cost of service use.
Consistent with UK Medical Research Council guidance and building on previous development research, this pragmatic cluster randomised trial will evaluate the effectiveness of a theory-based complex intervention focusing on changing multiple clinical behaviours to improve quality of diabetes care.
Dementia is a growing problem, causing substantial burden for patients, their families, and society. General practitioners (GPs) play an important role in diagnosing and managing dementia; however, there are gaps between recommended and current practice. The aim of this study was to explore GPs’ reported practice in diagnosing and managing dementia and to describe, in theoretical terms, the proposed explanations for practice that was and was not consistent with evidence-based guidelines.
Semi-structured interviews were conducted with GPs in Victoria, Australia. The Theoretical Domains Framework (TDF) guided data collection and analysis. Interviews explored the factors hindering and enabling achievement of 13 recommended behaviours. Data were analysed using content and thematic analysis. This paper presents an in-depth description of the factors influencing two behaviours, assessing co-morbid depression using a validated tool, and conducting a formal cognitive assessment using a validated scale.
A total of 30 GPs were interviewed. Most GPs reported that they did not assess for co-morbid depression using a validated tool as per recommended guidance. Barriers included the belief that depression can be adequately assessed using general clinical indicators and that validated tools provide little additional information (theoretical domain of ‘Beliefs about consequences’); discomfort in using validated tools (‘Emotion’), possibly due to limited training and confidence (‘Skills’; ‘Beliefs about capabilities’); limited awareness of the need for, and forgetting to conduct, a depression assessment (‘Knowledge’; ‘Memory, attention and decision processes’). Most reported practising in a manner consistent with the recommendation that a formal cognitive assessment using a validated scale be undertaken. Key factors enabling this were having an awareness of the need to conduct a cognitive assessment (‘Knowledge’); possessing the necessary skills and confidence (‘Skills’; ‘Beliefs about capabilities’); and having adequate time and resources (‘Environmental context and resources’).
This is the first study to our knowledge to use a theoretical approach to investigate the barriers and enablers to guideline-recommended diagnosis and management of dementia in general practice. It has identified key factors likely to explain GPs’ uptake of the guidelines. The results have informed the design of an intervention aimed at supporting practice change in line with dementia guidelines, which is currently being evaluated in a cluster randomised trial.
Dementia; General practitioners (GPs); Cognitive assessment; Depression assessment; Theoretical Domains Framework (TDF); Guideline implementation
Cluster randomized trials (CRTs) present unique ethical challenges. In the absence of a uniform standard for their ethical design and conduct, problems such as variability in procedures and requirements by different research ethics committees will persist. We aimed to assess the need for ethics guidelines for CRTs among research ethics chairs internationally, investigate variability in procedures for research ethics review of CRTs within and among countries, and elicit research ethics chairs’ perspectives on specific ethical issues in CRTs, including the identification of research subjects. The proper identification of research subjects is a necessary requirement in the research ethics review process, to help ensure, on the one hand, that subjects are protected from harm and exploitation, and on the other, that reviews of CRTs are completed efficiently.
A web-based survey with closed- and open-ended questions was administered to research ethics chairs in Canada, the United States, and the United Kingdom. The survey presented three scenarios of CRTs involving cluster-level, professional-level, and individual-level interventions. For each scenario, a series of questions was posed with respect to the type of review required (full, expedited, or no review) and the identification of research subjects at cluster and individual levels.
A total of 189 (35%) of 542 chairs responded. Overall, 144 (84%, 95% CI 79 to 90%) agreed or strongly agreed that there is a need for ethics guidelines for CRTs and 158 (92%, 95% CI 88 to 96%) agreed or strongly agreed that research ethics committees could be better informed about distinct ethical issues surrounding CRTs. There was considerable variability among research ethics chairs with respect to the type of review required, as well as the identification of research subjects. The cluster-cluster and professional-cluster scenarios produced the most disagreement.
Research ethics committees identified a clear need for ethics guidelines for CRTs and education about distinct ethical issues in CRTs. There is disagreement among committees, even within the same countries, with respect to key questions in the ethics review of CRTs. This disagreement reflects variability of opinion and practices pointing toward possible gaps in knowledge, and supports the need for explicit guidelines for the ethical conduct and review of CRTs.
Cluster randomized trials; Informed consent; Research ethics guidelines; Research ethics review; Web-based survey
The English mass media campaign ‘Act FAST’ aimed to raise stroke awareness and the need to call emergency services at the onset of suspected stroke. We examined the perceived impact and views of the campaign in target populations to identify potential ways to optimise mass-media interventions for stroke.
Analysis of semi-structured interviews conducted as part of two qualitative studies, which examined factors influencing patient/witness response to acute stroke symptoms (n = 19 stroke patients, n = 26 stroke witnesses) and perceptions about raising stroke awareness in primary care (n = 30 clinicians). Both studies included questions about the ‘Act FAST’ campaign. Interviews were content analysed to determine campaign awareness, perceived impact on decisions and response to stroke, and views of the campaign.
Most participants were aware of the Act FAST campaign. Some patients and witnesses reported that the campaign impacted upon their stroke recognition and response, but the majority reported no impact. Clinicians often perceived campaign success in raising stroke awareness, but few thought it would change response behaviours. Some patients and witnesses, and most primary care clinicians expressed positive views towards the campaign. Some more critical participant comments included perceptions of dramatic, irrelevant, and potentially confusing content, such as a prominent ‘fire in the brain’ analogy.
Act FAST has had some perceived impact on stroke recognition and response in some stroke patients and witnesses, but the majority reported no campaign impact. Primary care clinicians were positive about the campaign, and believed it had impacted on stroke awareness and recognition but doubted impact on response behaviour. Potential avenues for optimising and complementing mass media campaigns such as ‘Act FAST’ were identified.
Delay; Stroke; Awareness; Mass-media campaign
Dementia is a common and complex condition. Evidence-based guidelines for the management of people with dementia in general practice exist; however, detection, diagnosis and disclosure of dementia have been identified as potential evidence-practice gaps. Interventions to implement guidelines into practice have had varying success. The use of theory in designing implementation interventions has been limited, but is advocated because of its potential to yield more effective interventions and aid understanding of factors modifying the magnitude of intervention effects across trials. This protocol describes methods of a randomised trial that tests a theory-informed implementation intervention that, if effective, may provide benefits for patients with dementia and their carers.
This trial aims to estimate the effectiveness of a theory-informed intervention to increase GPs’ (in Victoria, Australia) adherence to a clinical guideline for the detection, diagnosis, and management of dementia in general practice, compared with providing GPs with a printed copy of the guideline. Primary objectives include testing if the intervention is effective in increasing the percentage of patients with suspected cognitive impairment who receive care consistent with two key guideline recommendations: receipt of a i) formal cognitive assessment, and ii) depression assessment using a validated scale (primary outcomes for the trial).
The design is a parallel cluster randomised trial, with clusters being general practices. We aim to recruit 60 practices per group. Practices will be randomised to the intervention and control groups using restricted randomisation. Patients meeting the inclusion criteria, and GPs’ detection and diagnosis behaviours directed toward these patients, will be identified and measured via an electronic search of the medical records nine months after the start of the intervention. Practitioners in the control group will receive a printed copy of the guideline. In addition to receipt of the printed guideline, practitioners in the intervention group will be invited to participate in an interactive, opinion leader-led, educational face-to-face workshop. The theory-informed intervention aims to address identified barriers to and enablers of implementation of recommendations. Researchers responsible for identifying the cohort of patients with suspected cognitive impairment, and their detection and diagnosis outcomes, will be blind to group allocation.
Australian New Zealand Clinical Trials Registry: ACTRN12611001032943 (date registered 28 September, 2011).
Determinants of practice are factors that might prevent or enable improvements. Several checklists, frameworks, taxonomies, and classifications of determinants of healthcare professional practice have been published. In this paper, we describe the development of a comprehensive, integrated checklist of determinants of practice (the TICD checklist).
We performed a systematic review of frameworks of determinants of practice followed by a consensus process. We searched electronic databases and screened the reference lists of key background documents. Two authors independently assessed titles and abstracts, and potentially relevant full text articles. We compiled a list of attributes that a checklist should have: comprehensiveness, relevance, applicability, simplicity, logic, clarity, usability, suitability, and usefulness. We assessed included articles using these criteria and collected information about the theory, model, or logic underlying how the factors (determinants) were selected, described, and grouped, the strengths and weaknesses of the checklist, and the determinants and the domains in each checklist. We drafted a preliminary checklist based on an aggregated list of determinants from the included checklists, and finalized the checklist by a consensus process among implementation researchers.
We screened 5,778 titles and abstracts and retrieved 87 potentially relevant papers in full text. Several of these papers had references to papers that we also retrieved in full text. We also checked potentially relevant papers we had on file that were not retrieved by the searches. We included 12 checklists. None of these were completely comprehensive when compared to the aggregated list of determinants and domains. We developed a checklist with 57 potential determinants of practice grouped in seven domains: guideline factors, individual health professional factors, patient factors, professional interactions, incentives and resources, capacity for organisational change, and social, political, and legal factors. We also developed five worksheets to facilitate the use of the checklist.
Based on a systematic review and a consensus process we developed a checklist that aims to be comprehensive and to build on the strengths of each of the 12 included checklists. The checklist is accompanied with five worksheets to facilitate its use in implementation research and quality improvement projects.
In the field of implementation research, there is an increased interest in use of theory when designing implementation research studies involving behavior change. In 2003, we initiated a series of five studies to establish a scientific rationale for interventions to translate research findings into clinical practice by exploring the performance of a number of different, commonly used, overlapping behavioral theories and models. We reflect on the strengths and weaknesses of the methods, the performance of the theories, and consider where these methods sit alongside the range of methods for studying healthcare professional behavior change.
These were five studies of the theory-based cognitions and clinical behaviors (taking dental radiographs, performing dental restorations, placing fissure sealants, managing upper respiratory tract infections without prescribing antibiotics, managing low back pain without ordering lumbar spine x-rays) of random samples of primary care dentists and physicians. Measures were derived for the explanatory theoretical constructs in the Theory of Planned Behavior (TPB), Social Cognitive Theory (SCT), and Illness Representations specified by the Common Sense Self Regulation Model (CSSRM). We constructed self-report measures of two constructs from Learning Theory (LT), a measure of Implementation Intentions (II), and the Precaution Adoption Process. We collected data on theory-based cognitions (explanatory measures) and two interim outcome measures (stated behavioral intention and simulated behavior) by postal questionnaire survey during the 12-month period to which objective measures of behavior (collected from routine administrative sources) were related. Planned analyses explored the predictive value of theories in explaining variance in intention, behavioral simulation and behavior.
Response rates across the five surveys ranged from 21% to 48%; we achieved the target sample size for three of the five surveys. For the predictor variables, the mean construct scores were above the mid-point on the scale with median values across the five behaviors generally being above four out of seven and the range being from 1.53 to 6.01. Across all of the theories, the highest proportion of the variance explained was always for intention and the lowest was for behavior. The Knowledge-Attitudes-Behavior Model performed poorly across all behaviors and dependent variables; CSSRM also performed poorly. For TPB, SCT, II, and LT across the five behaviors, we predicted median R2 of 25% to 42.6% for intention, 6.2% to 16% for behavioral simulation, and 2.4% to 6.3% for behavior.
We operationalized multiple theories measuring across five behaviors. Continuing challenges that emerge from our work are: better specification of behaviors, better operationalization of theories; how best to appropriately extend the range of theories; further assessment of the value of theories in different settings and groups; exploring the implications of these methods for the management of chronic diseases; and moving to experimental designs to allow an understanding of behavior change.
Evidence of variations in red blood cell transfusion practices have been reported in a wide range of clinical settings. Parallel studies in Canada and the United Kingdom were designed to explore transfusion behaviour in intensive care physicians. The aim of this paper is three-fold: first, to explore beliefs that influence Canadian intensive care physicians’ transfusion behaviour; second, to systematically select relevant theories and models using the Theoretical Domains Framework (TDF) to inform a future predictive study; and third, to compare its results with the UK study.
Ten intensive care unit (ICU) physicians throughout Canada were interviewed. Physicians’ responses were coded into theoretical domains, and specific beliefs were generated for each response. Theoretical domains relevant to behaviour change were identified, and specific constructs from the relevant domains were used to select psychological theories. The results from Canada and the United Kingdom were compared.
Seven theoretical domains populated by 31 specific beliefs were identified as relevant to the target behaviour. The domains Beliefs about capabilities (confident to not transfuse if patients’ clinical condition is stable), Beliefs about consequences (positive beliefs of reducing infection and saving resources and negative beliefs about risking patients’ clinical outcome and potentially more work), Social influences (transfusion decision is influenced by team members and patients’ relatives), and Behavioural regulation (wide range of approaches to encourage restrictive transfusion) that were identified in the UK study were also relevant in the Canadian context. Three additional domains, Knowledge (it requires more evidence to support restrictive transfusion), Social/professional role and identity (conflicting beliefs about not adhering to guidelines, referring to evidence, believing restrictive transfusion as professional standard, and believing that guideline is important for other professionals), and Motivation and goals (opposing beliefs about the importance of restrictive transfusion and compatibility with other goals), were also identified in this study. Similar to the UK study, the Theory of Planned Behaviour, Social Cognitive Theory, Operant Learning Theory, Action Planning, and Knowledge-Attitude-Behaviour model were identified as potentially relevant theories and models for further study. Personal project analysis was added to the Canadian study to explore the Motivation and goals domain in further detail.
A wide range of beliefs was identified by the Canadian ICU physicians as likely to influence their transfusion behaviour. We were able to demonstrate similar though not identical results in a cross-country comparison. Designing targeted behaviour-change interventions based on unique beliefs identified by physicians from two countries are more likely to encourage restrictive transfusion in ICU physicians in respective countries. This needs to be tested in future prospective clinical trials.
Implementation Science, like all journals, needs to continue to develop. There will always be changes we need to make as next steps in improving the Journal for readers and improving how it runs. However, we now have our first change in Editors in Chief. We are fortunate to have been able to recruit two experienced academics who are also experienced editors—Professor Michel Wensing and Dr Anne Sales. I hope you will join me in welcoming them and give them, and continue to give Implementation Science, your support.
Healthcare professional response rates to postal questionnaires are declining and this may threaten the validity and generalisability of their findings. Methods to improve response rates do incur costs (resources) and increase the cost of research projects. The aim of these randomised controlled trials (RCTs) was to assess whether 1) incentives, 2) type of reminder and/or 3) reduced response burden improve response rates; and to assess the cost implications of such additional effective interventions.
Two RCTs were conducted. In RCT A general dental practitioners (dentists) in Scotland were randomised to receive either an incentive; an abridged questionnaire or a full length questionnaire. In RCT B non-responders to a postal questionnaire sent to general medical practitioners (GPs) in the UK were firstly randomised to receive a second full length questionnaire as a reminder or a postcard reminder. Continued non-responders from RCT B were then randomised within their first randomisation to receive a third full length or an abridged questionnaire reminder. The cost-effectiveness of interventions that effectively increased response rates was assessed as a secondary outcome.
There was no evidence that an incentive (52% versus 43%, Risk Difference (RD) -8.8 (95%CI −22.5, 4.8); or abridged questionnaire (46% versus 43%, RD −2.9 (95%CI −16.5, 10.7); statistically significantly improved dentist response rates compared to a full length questionnaire in RCT A. In RCT B there was no evidence that a full questionnaire reminder statistically significantly improved response rates compared to a postcard reminder (10.4% versus 7.3%, RD 3 (95%CI −0.1, 6.8). At a second reminder stage, GPs sent the abridged questionnaire responded more often (14.8% versus 7.2%, RD −7.7 (95%CI −12.8, -2.6). GPs who received a postcard reminder followed by an abridged questionnaire were most likely to respond (19.8% versus 6.3%, RD 8.1%, and 9.1% for full/postcard/full, three full or full/full/abridged questionnaire respectively). An abridged questionnaire containing fewer questions following a postcard reminder was the only cost-effective strategy for increasing the response rate (£15.99 per response).
When expecting or facing a low response rate to postal questionnaires, researchers should carefully identify the most efficient way to boost their response rate. In these studies, an abridged questionnaire containing fewer questions following a postcard reminder was the only cost-effective strategy. An increase in response rates may be explained by a combination of the number and type of contacts. Increasing the sampling frame may be more cost-effective than interventions to prompt non-responders. However, this may not strengthen the validity and generalisability of the survey findings and affect the representativeness of the sample.
Although most people with Type 2 diabetes receive their diabetes care in primary care, only a limited amount is known about the quality of diabetes care in this setting. We investigated the provision and receipt of diabetes care delivered in UK primary care.
Postal surveys with all healthcare professionals and a random sample of 100 patients with Type 2 diabetes from 99 UK primary care practices.
326/361 (90.3%) doctors, 163/186 (87.6%) nurses and 3591 patients (41.8%) returned a questionnaire. Clinicians reported giving advice about lifestyle behaviours (e.g. 88% would routinely advise about calorie restriction; 99.6% about increasing exercise) more often than patients reported having received it (43% and 42%) and correlations between clinician and patient report were low. Patients’ reported levels of confidence about managing their diabetes were moderately high; a median (range) of 21% (3% to 39%) of patients reporting being not confident about various areas of diabetes self-management.
Primary care practices have organisational structures in place and are, as judged by routine quality indicators, delivering high quality care. There remain evidence-practice gaps in the care provided and in the self confidence that patients have for key aspects of self management and further research is needed to address these issues. Future research should use robust designs and appropriately designed studies to investigate how best to improve this situation.
Implementation Science has been published for six years and over that time has gone from receiving 100 articles in 2006 to receiving 354 in 2011; our impact factor has risen from 2.49 in June 2010 to 3.10 in June 2012. Whilst our article publication rate has also risen, it has risen much less slowly than our submission rate—we published 29 papers in 2006 and 134 papers in 2011 and we now publish only around 40 % of submissions. About one-half of submitted manuscripts are rejected without being sent out for peer review; it has become clear that there are a number of common issues that result in manuscripts being rejected at this stage. We hope that by publishing this editorial on our common reasons for rejection without peer review we can help authors to better judge the relevance of their papers to Implementation Science.
This article is part of a series of papers examining ethical issues in cluster randomized trials (CRTs) in health research. In the introductory paper in this series, we set out six areas of inquiry that must be addressed if the CRT is to be set on a firm ethical foundation. This paper addresses the sixth of the questions posed, namely, what is the role and authority of gatekeepers in CRTs in health research? ‘Gatekeepers’ are individuals or bodies that represent the interests of cluster members, clusters, or organizations. The need for gatekeepers arose in response to the difficulties in obtaining informed consent because of cluster randomization, cluster-level interventions, and cluster size. In this paper, we call for a more restrictive understanding of the role and authority of gatekeepers.
Previous papers in this series have provided solutions to the challenges posed by informed consent in CRTs without the need to invoke gatekeepers. We considered that consent to randomization is not required when cluster members are approached for consent at the earliest opportunity and before any study interventions or data-collection procedures have started. Further, when cluster-level interventions or cluster size means that obtaining informed consent is not possible, a waiver of consent may be appropriate. In this paper, we suggest that the role of gatekeepers in protecting individual interests in CRTs should be limited. Generally, gatekeepers do not have the authority to provide proxy consent for cluster members. When a municipality or other community has a legitimate political authority that is empowered to make such decisions, cluster permission may be appropriate; however, gatekeepers may usefully protect cluster interests in other ways. Cluster consultation may ensure that the CRT addresses local health needs, and is conducted in accord with local values and customs. Gatekeepers may also play an important role in protecting the interests of organizations, such as hospitals, nursing homes, general practices, and schools. In these settings, permission to access the organization relies on resource implications and adherence to institutional policies.