Audit and feedback interventions in healthcare have been found to be effective, but there has been little progress with respect to understanding their mechanisms of action or identifying their key ‘active ingredients.’
Given the increasing use of audit and feedback to improve quality of care, it is imperative to focus further research on understanding how and when it works best. In this paper, we argue that continuing the ‘business as usual’ approach to evaluating two-arm trials of audit and feedback interventions against usual care for common problems and settings is unlikely to contribute new generalizable findings. Future audit and feedback trials should incorporate evidence- and theory-based best practices, and address known gaps in the literature.
We offer an agenda for high-priority research topics for implementation researchers that focuses on reviewing best practices for designing audit and feedback interventions to optimize effectiveness.
Audit and feedback; Synthesis; Best practice; Implementation; Optimization
Decisions regarding health systems are sometimes made without the input of timely and reliable evidence, leading to less than optimal health outcomes. Healthcare organizations can implement tools and infrastructures to support the use of research evidence to inform decision-making.
The purpose of this study was to profile the supports and instruments (i.e., programs, interventions, instruments or tools) that healthcare organizations currently have in place and which ones were perceived to facilitate evidence-informed decision-making.
In-depth semi-structured telephone interviews were conducted with individuals in three different types of positions (i.e., a senior management team member, a library manager, and a ‘knowledge broker’) in three types of healthcare organizations (i.e., regional health authorities, hospitals and primary care practices) in two Canadian provinces (i.e., Ontario and Quebec). The interviews were taped, transcribed, and then analyzed thematically using NVivo 9 qualitative data analysis software.
A total of 57 interviews were conducted in 25 organizations in Ontario and Quebec. The main findings suggest that, for the healthcare organizations that participated in this study, the following supports facilitate evidence-informed decision-making: facilitating roles that actively promote research use within the organization; establishing ties to researchers and opinion leaders outside the organization; a technical infrastructure that provides access to research evidence, such as databases; and provision and participation in training programs to enhance staff’s capacity building.
This study identified the need for having a receptive climate, which laid the foundation for the implementation of other tangible initiatives and supported the use of research in decision-making. This study adds to the literature on organizational efforts that can increase the use of research evidence in decision-making. Some of the identified supports may increase the use of research evidence by decision-makers, which may then lead to more informed decisions, and hopefully to a strengthened health system and improved health.
The Canadian CT Head Rule was prospectively derived and validated to assist clinicians with diagnostic decision-making regarding the use of computed tomography (CT) in adult patients with minor head injury. A recent intervention trial failed to demonstrate a decrease in the rate of head CTs following implementation of the rule in Canadian emergency departments. Yet, the same intervention, which included a one-hour educational session and reminders at the point of requisition, was successful in reducing cervical spine imaging rates in the same emergency departments. The reason for the varied effect of the intervention across these two behaviours is unclear. There is an increasing appreciation for the use of theory to conduct process evaluations to better understand how strategies are linked with outcomes in implementation trials. The Theoretical Domains Framework (TDF) has been used to explore health professional behaviour and to design behaviour change interventions but, to date, has not been used to guide a theory-based process evaluation. In this proof of concept study, we explored whether the TDF could be used to guide a retrospective process evaluation to better understand emergency physicians’ responses to the interventions employed in the Canadian CT Head Rule trial.
A semi-structured interview guide, based on the 12 domains from the TDF, was used to conduct telephone interviews with project leads and physician participants from the intervention sites in the Canadian CT Head Rule trial. Two reviewers independently coded the anonymised interview transcripts using the TDF as a coding framework. Relevant domains were identified by: the presence of conflicting beliefs within a domain; the frequency of beliefs; and the likely strength of the impact of a belief on the behaviour.
Eight physicians from four of the intervention sites in the Canadian CT Head Rule trial participated in the interviews. Barriers likely to assist with understanding physicians’ responses to the intervention in the trial were identified in six of the theoretical domains: beliefs about consequences; beliefs about capabilities; behavioural regulation; memory, attention and decision processes; environmental context and resources; and social influences. Despite knowledge that the Canadian CT Head Rule was highly sensitive and reliable for identifying clinically important brain injuries and strong beliefs about the benefits for using the rule, a number of barriers were identified that may have prevented physicians from consistently applying the rule.
This proof of concept study demonstrates the use of the TDF as a guiding framework to design a retrospective theory-based process evaluation. There is a need for further development and testing of methods for using the TDF to guide theory-based process evaluations running alongside behaviour change intervention trials.
In the field of implementation research, there is an increased interest in use of theory when designing implementation research studies involving behavior change. In 2003, we initiated a series of five studies to establish a scientific rationale for interventions to translate research findings into clinical practice by exploring the performance of a number of different, commonly used, overlapping behavioral theories and models. We reflect on the strengths and weaknesses of the methods, the performance of the theories, and consider where these methods sit alongside the range of methods for studying healthcare professional behavior change.
These were five studies of the theory-based cognitions and clinical behaviors (taking dental radiographs, performing dental restorations, placing fissure sealants, managing upper respiratory tract infections without prescribing antibiotics, managing low back pain without ordering lumbar spine x-rays) of random samples of primary care dentists and physicians. Measures were derived for the explanatory theoretical constructs in the Theory of Planned Behavior (TPB), Social Cognitive Theory (SCT), and Illness Representations specified by the Common Sense Self Regulation Model (CSSRM). We constructed self-report measures of two constructs from Learning Theory (LT), a measure of Implementation Intentions (II), and the Precaution Adoption Process. We collected data on theory-based cognitions (explanatory measures) and two interim outcome measures (stated behavioral intention and simulated behavior) by postal questionnaire survey during the 12-month period to which objective measures of behavior (collected from routine administrative sources) were related. Planned analyses explored the predictive value of theories in explaining variance in intention, behavioral simulation and behavior.
Response rates across the five surveys ranged from 21% to 48%; we achieved the target sample size for three of the five surveys. For the predictor variables, the mean construct scores were above the mid-point on the scale with median values across the five behaviors generally being above four out of seven and the range being from 1.53 to 6.01. Across all of the theories, the highest proportion of the variance explained was always for intention and the lowest was for behavior. The Knowledge-Attitudes-Behavior Model performed poorly across all behaviors and dependent variables; CSSRM also performed poorly. For TPB, SCT, II, and LT across the five behaviors, we predicted median R2 of 25% to 42.6% for intention, 6.2% to 16% for behavioral simulation, and 2.4% to 6.3% for behavior.
We operationalized multiple theories measuring across five behaviors. Continuing challenges that emerge from our work are: better specification of behaviors, better operationalization of theories; how best to appropriately extend the range of theories; further assessment of the value of theories in different settings and groups; exploring the implications of these methods for the management of chronic diseases; and moving to experimental designs to allow an understanding of behavior change.
The Theoretical Domains Framework (TDF) was developed to investigate determinants of specific clinical behaviors and inform the design of interventions to change professional behavior. This framework was used to explore the beliefs of chiropractors in an American Provider Network and two Canadian provinces about their adherence to evidence-based recommendations for spine radiography for uncomplicated back pain. The primary objective of the study was to identify chiropractors’ beliefs about managing uncomplicated back pain without x-rays and to explore barriers and facilitators to implementing evidence-based recommendations on lumbar spine x-rays. A secondary objective was to compare chiropractors in the United States and Canada on their beliefs regarding the use of spine x-rays.
Six focus groups exploring beliefs about managing back pain without x-rays were conducted with a purposive sample. The interview guide was based upon the TDF. Focus groups were digitally recorded, transcribed verbatim, and analyzed by two independent assessors using thematic content analysis based on the TDF.
Five domains were identified as likely relevant. Key beliefs within these domains included the following: conflicting comments about the potential consequences of not ordering x-rays (risk of missing a pathology, avoiding adverse treatment effects, risks of litigation, determining the treatment plan, and using x-ray-driven techniques contrasted with perceived benefits of minimizing patient radiation exposure and reducing costs; beliefs about consequences); beliefs regarding professional autonomy, professional credibility, lack of standardization, and agreement with guidelines widely varied ( social/professional role & identity); the influence of formal training, colleagues, and patients also appeared to be important factors ( social influences); conflicting comments regarding levels of confidence and comfort in managing patients without x-rays ( belief about capabilities); and guideline awareness and agreements ( knowledge).
Chiropractors’ use of diagnostic imaging appears to be influenced by a number of factors. Five key domains may be important considering the presence of conflicting beliefs, evidence of strong beliefs likely to impact the behavior of interest, and high frequency of beliefs. The results will inform the development of a theory-based survey to help identify potential targets for behavioral-change strategies.
Theoretical domains framework; Focus groups; Content analysis; Social/professional role and identity; Social influence; Chiropractors; Radiography; X-ray guidelines; Back pain
Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this third paper we discuss the issues of: reviewing, reporting, and publishing guidelines; updating guidelines; and the two emerging issues of enhancing guideline implementability and how guideline developers should approach dealing with the issue of patients who will be the subject of guidelines having co-morbid conditions.
Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this first paper we discuss: the target audience(s) for guidelines and their use of guidelines; identifying topics for guidelines; guideline group composition (including consumer involvement) and the processes by which guideline groups function and the important procedural issue of managing conflicts of interest in guideline development.
Clinical practice guidelines are one of the foundations of efforts to improve healthcare. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearinghouses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this second paper, we discuss issues of identifying and synthesizing evidence: deciding what type of evidence and outcomes to include in guidelines; integrating values into a guideline; incorporating economic considerations; synthesis, grading, and presentation of evidence; and moving from evidence to recommendations.
Routine pre-operative tests for anesthesia management are often ordered by both anesthesiologists and surgeons for healthy patients undergoing low-risk surgery. The Theoretical Domains Framework (TDF) was developed to investigate determinants of behaviour and identify potential behaviour change interventions. In this study, the TDF is used to explore anaesthesiologists’ and surgeons’ perceptions of ordering routine tests for healthy patients undergoing low-risk surgery.
Sixteen clinicians (eleven anesthesiologists and five surgeons) throughout Ontario were recruited. An interview guide based on the TDF was developed to identify beliefs about pre-operative testing practices. Content analysis of physicians’ statements into the relevant theoretical domains was performed. Specific beliefs were identified by grouping similar utterances of the interview participants. Relevant domains were identified by noting the frequencies of the beliefs reported, presence of conflicting beliefs, and perceived influence on the performance of the behaviour under investigation.
Seven of the twelve domains were identified as likely relevant to changing clinicians’ behaviour about pre-operative test ordering for anesthesia management. Key beliefs were identified within these domains including: conflicting comments about who was responsible for the test-ordering (Social/professional role and identity); inability to cancel tests ordered by fellow physicians (Beliefs about capabilities and social influences); and the problem with tests being completed before the anesthesiologists see the patient (Beliefs about capabilities and Environmental context and resources). Often, tests were ordered by an anesthesiologist based on who may be the attending anesthesiologist on the day of surgery while surgeons ordered tests they thought anesthesiologists may need (Social influences). There were also conflicting comments about the potential consequences associated with reducing testing, from negative (delay or cancel patients’ surgeries), to indifference (little or no change in patient outcomes), to positive (save money, avoid unnecessary investigations) (Beliefs about consequences). Further, while most agreed that they are motivated to reduce ordering unnecessary tests (Motivation and goals), there was still a report of a gap between their motivation and practice (Behavioural regulation).
We identified key factors that anesthesiologists and surgeons believe influence whether they order pre-operative tests routinely for anesthesia management for a healthy adults undergoing low-risk surgery. These beliefs identify potential individual, team, and organisation targets for behaviour change interventions to reduce unnecessary routine test ordering.
Routine pre-operative testing; Anesthesia management; Anesthesiologists; Surgeons; Chest x-rays; Electrocardiograms; Theoretical domains framework; Semi-structured interviews; Content analysis; Social; Professional role and identity; Social influence
Despite evidence-based recommendations supporting long-term use of cardiac medications in patients post ST-elevation myocardial infarction, adherence is known to decline over time. Discontinuation of cardiac medications in such patients is associated with increased mortality.
This is a pragmatic, cluster-randomized controlled trial with blinded outcome assessment and embedded qualitative process evaluation. Patients from one health region in Ontario, Canada who undergo a coronary angiogram during their admission for ST-elevation myocardial infarction and who survive their initial hospitalization will be included. Allocation of eligible patients to intervention or usual care will take place within one week after the angiogram using a computer-generated random sequence. To avoid treatment contamination, patients treated by the same family physician will be allocated to the same study arm. The intervention consists of recurrent, personalized, paper-based educational messages and reminders sent via post on behalf of the interventional cardiologist to the patient, family physician, and pharmacist urging long-term adherence to secondary prevention medications. The primary outcome is the proportion of patients who report in a phone interview taking all relevant classes of cardiac medications at twelve months. Secondary outcomes to be measured at three and twelve months include proportions of patients who report: actively taking each cardiac medication class of interest (item-by-item); stopping medications due to side effects; taking one or two or three medication classes concurrently; a perfect Morisky Medication Adherence Score for cardiac medication compliance; and having a discussion with their family physician about long-term adherence to cardiac medications. Self-reported measures of adherence will be validated using administrative data for prescriptions filled.
This intervention is designed to be easily generalizable. If effective, it could be implemented broadly. If it does not change medication utilization, the process evaluation will offer insights regarding how such an intervention could be optimized in future.
Randomized trial; Medication adherence; Reminders
One of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?
We suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.
There is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.
In this editorial, we reflect on the arguments for starting a scientific society focused on research on how to improve healthcare. This society would take an inclusive approach to what constitutes healthcare. For instance, it should include mental health healthcare, treatment for substance abuse, the work of allied health professions, and preventive healthcare. The society would be open to researchers from all traditions. Thus, we take an inclusive approach to what constitutes scientific research, as long as it uses rigorous methods, is focused on improving healthcare, and aims at knowledge that can be transferred across settings. The society would primarily target scientific researchers but would invite others with an interest in this area of research, regardless of their discipline, position, field of application, or group affiliation (e.g., improvement science, behavioral medicine, knowledge translation). A society would need fruitful collaboration with related societies and organizations, which may include having combined meetings. Special links may be developed with one or more journals. A website to provide information on relevant resources, events, and training opportunities is another key activity. It would also provide a voice for the field at funding agencies, political arenas, and similar institutions. An organizational structure and financial resources are required to develop and run these activities. Our aim is to start an international debate, to discover if we can establish a shared vision across academics and stakeholders engaged with creating scientific knowledge on how to improve healthcare. We invite readers to express their views in the online questionnaire accessed by following the URL link provided at the end of the editorial.
Psychological models predict behaviour in a wide range of settings. The aim of this study was to explore the usefulness of a range of psychological models to predict the health professional behaviour 'referral for lumbar spine x-ray in patients presenting with low back pain' by UK primary care physicians.
Psychological measures were collected by postal questionnaire survey from a random sample of primary care physicians in Scotland and north England. The outcome measures were clinical behaviour (referral rates for lumbar spine x-rays), behavioural simulation (lumbar spine x-ray referral decisions based upon scenarios), and behavioural intention (general intention to refer for lumbar spine x-rays in patients with low back pain). Explanatory variables were the constructs within the Theory of Planned Behaviour (TPB), Social Cognitive Theory (SCT), Common Sense Self-Regulation Model (CS-SRM), Operant Learning Theory (OLT), Implementation Intention (II), Weinstein's Stage Model termed the Precaution Adoption Process (PAP), and knowledge. For each of the outcome measures, a generalised linear model was used to examine the predictive value of each theory individually. Linear regression was used for the intention and simulation outcomes, and negative binomial regression was used for the behaviour outcome. Following this 'theory level' analysis, a 'cross-theoretical construct' analysis was conducted to investigate the combined predictive value of all individual constructs across theories.
Constructs from TPB, SCT, CS-SRM, and OLT predicted behaviour; however, the theoretical models did not fit the data well. When predicting behavioural simulation, the proportion of variance explained by individual theories was TPB 11.6%, SCT 12.1%, OLT 8.1%, and II 1.5% of the variance, and in the cross-theory analysis constructs from TPB, CS-SRM and II explained 16.5% of the variance in simulated behaviours. When predicting intention, the proportion of variance explained by individual theories was TPB 25.0%, SCT 21.5%, CS-SRM 11.3%, OLT 26.3%, PAP 2.6%, and knowledge 2.3%, and in the cross-theory analysis constructs from TPB, SCT, CS-SRM, and OLT explained 33.5% variance in intention. Together these results suggest that physicians' beliefs about consequences and beliefs about capabilities are likely determinants of lumbar spine x-ray referrals.
The study provides evidence that taking a theory-based approach enables the creation of a replicable methodology for identifying factors that predict clinical behaviour. However, a number of conceptual and methodological challenges remain.
Health-system policy makers need timely access to synthesised research evidence to inform the policy-making process. No efforts to address this need have been evaluated using an experimental quantitative design. We developed an evidence service that draws inputs from Health Systems Evidence, which is a database of policy-relevant systematic reviews. The reviews have been (a) categorised by topic and type of review; (b) coded by the last year searches for studies were conducted and by the countries in which included studies were conducted; (c) rated for quality; and (d) linked to available user-friendly summaries, scientific abstracts, and full-text reports. Our goal is to evaluate whether a "full-serve" evidence service increases the use of synthesized research evidence by policy analysts and advisors in the Ontario Ministry of Health and Long-Term Care (MOHLTC) as compared to a "self-serve" evidence service.
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study in order to explore the findings in greater depth. For the RCT, all policy analysts and policy advisors (n = 168) in a single division of the MOHLTC will be invited to participate. Using a stratified randomized design, participants will be randomized to receive either the "full-serve" evidence service (database access, monthly e-mail alerts, and full-text article availability) or the "self-serve" evidence service (database access only). The trial duration will be ten months (two-month baseline period, six-month intervention period, and two month cross-over period). The primary outcome will be the mean number of site visits/month/user between baseline and the end of the intervention period. The secondary outcome will be participants' intention to use research evidence. For the qualitative study, 15 participants from each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support health-system policy makers in finding and using research evidence.
To support the use of research evidence by community-based organizations (CBOs) we have developed 'Synthesized HIV/AIDS Research Evidence' (SHARE), which is an evidence service for those working in the HIV sector. SHARE consists of several components: an online searchable database of HIV-relevant systematic reviews (retrievable based on a taxonomy of topics related to HIV/AIDS and open text search); periodic email updates; access to user-friendly summaries; and peer relevance assessments. Our objective is to evaluate whether this 'full serve' evidence service increases the use of research evidence by CBOs as compared to a 'self-serve' evidence service.
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study to explore the findings in greater depth. All CBOs affiliated with Canadian AIDS Society (n = 120) will be invited to participate and will be randomized to receive either the 'full-serve' version of SHARE or the 'self-serve' version (a listing of relevant systematic reviews with links to records on PubMed and worksheets that help CBOs find and use research evidence) using a simple randomized design. All management and staff from each organization will be provided access to the version of SHARE that their organization is allocated to. The trial duration will be 10 months (two-month baseline period, six-month intervention period, and two month crossover period), the primary outcome measure will be the mean number of logins/month/organization (averaged across the number of users from each organization) between baseline and the end of the intervention period. The secondary outcome will be intention to use research evidence as measured by a survey administered to one key decision maker from each organization. For the qualitative study, one key organizational decision maker from 15 organizations in each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support CBOs in finding and using research evidence.
Guidelines continue to be underutilized, and a variety of strategies to improve their use have been suboptimal. Modifying guideline features represents an alternative, but untested way to promote their use. The purpose of this study was to identify and define features that facilitate guideline use, and examine whether and how they are included in current guidelines.
A guideline implementability framework was developed by reviewing the implementation science literature. We then examined whether guidelines included these, or additional implementability elements. Data were extracted from publicly available high quality guidelines reflecting primary and institutional care, reviewed independently by two individuals, who through discussion resolved conflicts, then by the research team.
The final implementability framework included 22 elements organized in the domains of adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation. Data were extracted from 20 guidelines on the management of diabetes, hypertension, leg ulcer, and heart failure. Most contained a large volume of graded, narrative evidence, and tables featuring complementary clinical information. Few contained additional features that could improve guideline use. These included alternate versions for different users and purposes, summaries of evidence and recommendations, information to facilitate interaction with and involvement of patients, details of resource implications, and instructions on how to locally promote and monitor guideline use. There were no consistent trends by guideline topic.
Numerous opportunities were identified by which guidelines could be modified to support various types of decision making by different users. New governance structures may be required to accommodate development of guidelines with these features. Further research is needed to validate the proposed framework of guideline implementability, develop methods for preparing this information, and evaluate how inclusion of this information influences guideline use.
Randomised trials of knowledge translation strategies for professional behaviour change can provide robust estimates of effectiveness, but offer little insight into the causal mechanisms by which any change is produced. To illustrate the applicability of causal methods within randomised trials, we undertook a theory-based process evaluation study within an implementation trial to explore whether the cognitions of primary care doctors' predicted their test requesting behaviours and, secondly, whether the trial results were mediated by the theoretical constructs.
The process evaluation comprised a cross-sectional questionnaire survey of a random 50% sample of the randomised groups of primary care practices in Grampian (NHS Grampian), UK, who took part in a trial of the effect of enhanced feedback and brief educational reminders on test requesting behaviour. The process evaluation was based upon the Theory of Planned Behaviour and focussed on three of the test requesting behaviours that were targeted in the trial -- ferritin, follicle stimulating hormone (FSH), and Helicobacter Pylori serology (HPS).
The questionnaire was completed by 131 primary care doctors (56%) from 42 (98%) of the sampled practices. Behavioural intention, attitude, and subjective norm were highly correlated for all the tests. There was no evidence that perceived behavioural control was correlated with any of the other measures. Simple linear regression analysis of the rate of test requests on minimum behavioural intentions had R2 of 11.1%, 12.5%, and 0.1% for ferritin, FSH, and HPS requesting, respectively. Mediational analysis showed that the trial results for ferritin and FSH were partially mediated (between 23% and 78% mediation) through intentions. The HPS trial result was not mediated through intention.
This study demonstrated that a theory-based process evaluation can provide useful information on causal mechanisms that aid not only interpretation of the trial but also inform future evaluations and intervention development.
It is well documented that the translation of knowledge into clinical practice is a slow and haphazard process. This is no less true for dental healthcare than other types of healthcare. One common policy strategy to help promote knowledge translation is the production of clinical guidance, but it has been demonstrated that the simple publication of guidance is unlikely to optimise practice. Additional knowledge translation interventions have been shown to be effective, but effectiveness varies and much of this variation is unexplained. The need for researchers to move beyond single studies to develop a generalisable, theory based, knowledge translation framework has been identified.
For dentistry in Scotland, the production of clinical guidance is the responsibility of the Scottish Dental Clinical Effectiveness Programme (SDCEP). TRiaDS (Translation Research in a Dental Setting) is a multidisciplinary research collaboration, embedded within the SDCEP guidance development process, which aims to establish a practical evaluative framework for the translation of guidance and to conduct and evaluate a programme of integrated, multi-disciplinary research to enhance the science of knowledge translation.
Set in General Dental Practice the TRiaDS programmatic evaluation employs a standardised process using optimal methods and theory. For each SDCEP guidance document a diagnostic analysis is undertaken alongside the guidance development process. Information is gathered about current dental care activities. Key recommendations and their required behaviours are identified and prioritised. Stakeholder questionnaires and interviews are used to identify and elicit salient beliefs regarding potential barriers and enablers towards the key recommendations and behaviours. Where possible routinely collected data are used to measure compliance with the guidance and to inform decisions about whether a knowledge translation intervention is required. Interventions are theory based and informed by evidence gathered during the diagnostic phase and by prior published evidence. They are evaluated using a range of experimental and quasi-experimental study designs, and data collection continues beyond the end of the intervention to investigate the sustainability of an intervention effect.
The TRiaDS programmatic approach is a significant step forward towards the development of a practical, generalisable framework for knowledge translation research. The multidisciplinary composition of the TRiaDS team enables consideration of the individual, organisational and system determinants of professional behaviour change. In addition the embedding of TRiaDS within a national programme of guidance development offers a unique opportunity to inform and influence the guidance development process, and enables TRiaDS to inform dental services practitioners, policy makers and patients on how best to translate national recommendations into routine clinical activities.
There is growing interest in the use of cognitive, behavioural, and organisational theories in implementation research. However, the extent of use of theory in implementation research is uncertain.
We conducted a systematic review of use of theory in 235 rigorous evaluations of guideline dissemination and implementation studies published between 1966 and 1998. Use of theory was classified according to type of use (explicitly theory based, some conceptual basis, and theoretical construct used) and stage of use (choice/design of intervention, process/mediators/moderators, and post hoc/explanation).
Fifty-three of 235 studies (22.5%) were judged to have employed theories, including 14 studies that explicitly used theory. The majority of studies (n = 42) used only one theory; the maximum number of theories employed by any study was three. Twenty-five different theories were used. A small number of theories accounted for the majority of theory use including PRECEDE (Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation), diffusion of innovations, information overload and social marketing (academic detailing).
There was poor justification of choice of intervention and use of theory in implementation research in the identified studies until at least 1998. Future research should explicitly identify the justification for the interventions. Greater use of explicit theory to understand barriers, design interventions, and explore mediating pathways and moderators is needed to advance the science of implementation research.
Blood transfusion is an essential part of healthcare and can improve patient outcomes. However, like most therapies, it is also associated with significant clinical risks. In addition, there is some evidence of overuse. Understanding the potential barriers and enablers to reduced prescribing of blood products will facilitate the selection of intervention components likely to be effective, thereby reducing the number of costly trials evaluating different implementation strategies. Using a theoretical basis to understand behaviours targeted for change will contribute to a 'basic science' relating to determinants of professional behaviour and how these inform the selection of techniques for changing behaviour. However, it is not clear which theories of behaviour are relevant to clinicians' transfusing behaviour. The aim of this study is to use a theoretical domains framework to identify relevant theories, and to use these theories to identify factors that predict the decision to transfuse.
The study involves two steps: interview study and questionnaire study. Using a previously identified framework, we will conduct semi-structured interviews with clinicians to elicit their views about which factors are associated with waiting and further monitoring the patient rather than transfusing red blood cells. Interviews will cover the following theoretical domains: knowledge; skills; social/professional role and identity; beliefs about capabilities; beliefs about consequences; motivation and goals; memory, attention, and decision processes; environmental context and resources; social influences; emotion; behavioural regulation; nature of the behaviour. The interviews will take place independently in Canada and the UK and involve two groups of physicians in each country (UK: adult and neonatal intensive care physicians; Canada: intensive care physicians and orthopaedic surgeons). We will: analyse interview transcript content to select relevant theoretical domains; use consensus processes to map these domains on to theories of behaviour; develop questionnaires based on these theories; and mail them to each group of physicians in the two countries. From our previous work, it is likely that the theories will include: theory of planned behaviour, social cognitive theory and the evidence-based strategy, implementation intention. The questionnaire data will measure predictor variables (theoretical constructs) and outcome variables (intention and clinical decision), and will be analysed using multiple regression analysis. We aim to achieve 150 respondents in each of the four groups for each postal survey.
Complex behaviour change interventions are not well described; when they are described, the terminology used is inconsistent. This constrains scientific replication, and limits the subsequent introduction of successful interventions. Implementation Science is introducing a policy of initially encouraging and subsequently requiring the scientific reporting of complex behaviour change interventions.
Practice guidelines can improve health care delivery and outcomes but several issues challenge guideline adoption, including their intrinsic attributes, and whether and how they are implemented. It appears that guideline format may influence accessibility and ease of use, which may overcome attitudinal barriers of guideline adoption, and appear to be important to all stakeholders. Guideline content may facilitate various forms of decision making about guideline adoption relevant to different stakeholders. Knowledge and attitudes about, and incentives and capacity for implementation on the part of guideline sponsors may influence whether and how they develop guidelines containing these features, and undertake implementation. Examination of these issues may yield opportunities to improve guideline adoption.
The attributes hypothesized to facilitate adoption will be expanded by thematic analysis, and quantitative and qualitative summary of the content of international guidelines for two primary care (diabetes, hypertension) and institutional care (chronic ulcer, chronic heart failure) topics. Factors that influence whether and how guidelines are implemented will be explored by qualitative analysis of interviews with individuals affiliated with guideline sponsoring agencies.
Previous research examined guideline implementation by measuring rates of compliance with recommendations or associated outcomes, but this produced little insight on how the products themselves, or their implementation, could be improved. This research will establish a theoretical basis upon which to conduct experimental studies to compare the cost-effectiveness of interventions that enhance guideline development and implementation capacity. Such studies could first examine short-term outcomes predictive of guideline utilization, such as recall, attitude toward, confidence in, and adoption intention. If successful, then long-term objective outcomes reflecting the adoption of processes and associated patient care outcomes could be evaluated.
Since being identified as a concept for understanding knowledge sharing, management, and creation, communities of practice (CoPs) have become increasingly popular within the health sector. The CoP concept has been used in the business sector for over 20 years, but the use of CoPs in the health sector has been limited in comparison.
First, we examined how CoPs were defined and used in these two sectors. Second, we evaluated the evidence of effectiveness on the health sector CoPs for improving the uptake of best practices and mentoring new practitioners.
We conducted a search of electronic databases in the business, health, and education sectors, and a hand search of key journals for primary studies on CoP groups. Our research synthesis for the first objective focused on three areas: the authors' interpretations of the CoP concept, the key characteristics of CoP groups, and the common elements of CoP groups. To examine the evidence on the effectiveness of CoPs in the health sector, we identified articles that evaluated CoPs for improving health professional performance, health care organizational performance, professional mentoring, and/or patient outcome; and used experimental, quasi-experimental, or observational designs.
The structure of CoP groups varied greatly, ranging from voluntary informal networks to work-supported formal education sessions, and from apprentice training to multidisciplinary, multi-site project teams. Four characteristics were identified from CoP groups: social interaction among members, knowledge sharing, knowledge creation, and identity building; however, these were not consistently present in all CoPs. There was also a lack of clarity in the responsibilities of CoP facilitators and how power dynamics should be handled within a CoP group. We did not find any paper in the health sector that met the eligibility criteria for the quantitative analysis, and so the effectiveness of CoP in this sector remained unclear.
There is no dominant trend in how the CoP concept is operationalized in the business and health sectors; hence, it is challenging to define the parameters of CoP groups. This may be one of the reasons for the lack of studies on the effectiveness of CoPs in the health sector. In order to improve the usefulness of the CoP concept in the development of groups and teams, further research will be needed to clarify the extent to which the four characteristics of CoPs are present in the mature and emergent groups, the expectations of facilitators and other participants, and the power relationship within CoPs.
In the experience of health professionals, it appears that interacting with peers in the workplace fosters learning and information sharing. Informal groups and networks present good opportunities for information exchange. Communities of practice (CoPs), which have been described by Wenger and others as a type of informal learning organization, have received increasing attention in the health care sector; however, the lack of uniform operating definitions of CoPs has resulted in considerable variation in the structure and function of these groups, making it difficult to evaluate their effectiveness.
To critique the evolution of the CoP concept as based on the germinal work by Wenger and colleagues published between 1991 and 2002.
CoP was originally developed to provide a template for examining the learning that happens among practitioners in a social environment, but over the years there have been important divergences in the focus of the concept. Lave and Wenger's earliest publication (1991) centred on the interactions between novices and experts, and the process by which newcomers create a professional identity. In the 1998 book, the focus had shifted to personal growth and the trajectory of individuals' participation within a group (i.e., peripheral versus core participation). The focus then changed again in 2002 when CoP was applied as a managerial tool for improving an organization's competitiveness.
The different interpretations of CoP make it challenging to apply the concept or to take full advantage of the benefits that CoP groups may offer. The tension between satisfying individuals' needs for personal growth and empowerment versus an organization's bottom line is perhaps the most contentious of the issues that make CoPs difficult to cultivate. Since CoP is still an evolving concept, we recommend focusing on optimizing specific characteristics of the concept, such as support for members interacting with each other, sharing knowledge, and building a sense of belonging within networks/teams/groups. Interventions that facilitate relationship building among members and that promote knowledge exchange may be useful for optimizing the function of these groups.
Improved quality of care is a policy objective of health care systems around the world. Implementation research is the scientific study of methods to promote the systematic uptake of clinical research findings into routine clinical practice, and hence to reduce inappropriate care. It includes the study of influences on healthcare professionals' behaviour and methods to enable them to use research findings more effectively. Cluster randomized trials represent the optimal design for evaluating the effectiveness of implementation strategies. Various codes of medical ethics, such as the Nuremberg Code and the Declaration of Helsinki inform medical research, but their relevance to cluster randomised trials in implementation research is unclear. This paper discusses the applicability of various ethical codes to obtaining consent in cluster trials in implementation research.
The appropriate application of biomedical codes to implementation research is not obvious. Discussion of the nature and practice of informed consent in implementation research cluster trials must consider the levels at which consent can be sought, and for what purpose it can be sought. The level at which an intervention is delivered can render the idea of patient level consent meaningless. Careful consideration of the ownership of information, and rights of access to and exploitation of data is required. For health care professionals and organizations, there is a balance between clinical freedom and responsibility to participate in research.
While ethical justification for clinical trials relies heavily on individual consent, for implementation research aspects of distributive justice, economics, and political philosophy underlie the debate. Societies may need to trade off decisions on the choice between individualized consent and valid implementation research. We suggest that social sciences codes could usefully inform the consideration of implementation research by members of Research Ethics Committees.