Background

Guidelines continue to be underutilized, and a variety of strategies to improve their use have been suboptimal. Modifying guideline features represents an alternative, but untested way to promote their use. The purpose of this study was to identify and define features that facilitate guideline use, and examine whether and how they are included in current guidelines.

Methods

A guideline implementability framework was developed by reviewing the implementation science literature. We then examined whether guidelines included these, or additional implementability elements. Data were extracted from publicly available high quality guidelines reflecting primary and institutional care, reviewed independently by two individuals, who through discussion resolved conflicts, then by the research team.

Results

The final implementability framework included 22 elements organized in the domains of adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation. Data were extracted from 20 guidelines on the management of diabetes, hypertension, leg ulcer, and heart failure. Most contained a large volume of graded, narrative evidence, and tables featuring complementary clinical information. Few contained additional features that could improve guideline use. These included alternate versions for different users and purposes, summaries of evidence and recommendations, information to facilitate interaction with and involvement of patients, details of resource implications, and instructions on how to locally promote and monitor guideline use. There were no consistent trends by guideline topic.

Conclusions

Numerous opportunities were identified by which guidelines could be modified to support various types of decision making by different users. New governance structures may be required to accommodate development of guidelines with these features. Further research is needed to validate the proposed framework of guideline implementability, develop methods for preparing this information, and evaluate how inclusion of this information influences guideline use.

The rapid and continuing progress in gene discovery for complex diseases is fuelling interest in the potential application of genetic risk models for clinical and public health practice. The number of studies assessing the predictive ability is steadily increasing, but they vary widely in completeness of reporting and apparent quality. Transparent reporting of the strengths and weaknesses of these studies is important to facilitate the accumulation of evidence on genetic risk prediction. A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines. These recommendations aim to enhance the transparency, quality and completeness of study reporting, and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.

Background

The Canadian CT Head Rule was developed to allow physicians to be more selective when ordering computed tomography (CT) imaging for patients with minor head injury. We sought to evaluate the effectiveness of implementing this validated decision rule at multiple emergency departments.

Methods

We conducted a matched-pair cluster-randomized trial that compared the outcomes of 4531 patients with minor head injury during two 12-month periods (before and after) at hospital emergency departments in Canada, six of which were randomly allocated as intervention sites and six as control sites. At the intervention sites, active strategies, including education, changes to policy and real-time reminders on radiologic requisitions were used to implement the Canadian CT Head Rule. The main outcome measure was referral for CT scan of the head.

Results

Baseline characteristics of patients were similar when comparing control to intervention sites. At the intervention sites, the proportion of patients referred for CT imaging increased from the “before” period (62.8%) to the “after” period (76.2%) (difference +13.3%, 95% CI 9.7%–17.0%). At the control sites, the proportion of CT imaging usage also increased, from 67.5% to 74.1% (difference +6.7%, 95% CI 2.6%–10.8%). The change in mean imaging rates from the “before” period to the “after” period for intervention versus control hospitals was not significant (p = 0.16). There were no missed brain injuries or adverse outcomes.

Interpretation

Our knowledge–translation-based trial of the Canadian CT Head Rule did not reduce rates of CT imaging in Canadian emergency departments. Future studies should identify strategies to deal with barriers to implementation of this decision rule and explore more effective approaches to knowledge translation. (ClinicalTrials.gov trial register no. NCT00993252)

Background

Randomised trials of knowledge translation strategies for professional behaviour change can provide robust estimates of effectiveness, but offer little insight into the causal mechanisms by which any change is produced. To illustrate the applicability of causal methods within randomised trials, we undertook a theory-based process evaluation study within an implementation trial to explore whether the cognitions of primary care doctors' predicted their test requesting behaviours and, secondly, whether the trial results were mediated by the theoretical constructs.

Methods

The process evaluation comprised a cross-sectional questionnaire survey of a random 50% sample of the randomised groups of primary care practices in Grampian (NHS Grampian), UK, who took part in a trial of the effect of enhanced feedback and brief educational reminders on test requesting behaviour. The process evaluation was based upon the Theory of Planned Behaviour and focussed on three of the test requesting behaviours that were targeted in the trial -- ferritin, follicle stimulating hormone (FSH), and Helicobacter Pylori serology (HPS).

Results

The questionnaire was completed by 131 primary care doctors (56%) from 42 (98%) of the sampled practices. Behavioural intention, attitude, and subjective norm were highly correlated for all the tests. There was no evidence that perceived behavioural control was correlated with any of the other measures. Simple linear regression analysis of the rate of test requests on minimum behavioural intentions had R2 of 11.1%, 12.5%, and 0.1% for ferritin, FSH, and HPS requesting, respectively. Mediational analysis showed that the trial results for ferritin and FSH were partially mediated (between 23% and 78% mediation) through intentions. The HPS trial result was not mediated through intention.

Conclusions

This study demonstrated that a theory-based process evaluation can provide useful information on causal mechanisms that aid not only interpretation of the trial but also inform future evaluations and intervention development.

Background

It is well documented that the translation of knowledge into clinical practice is a slow and haphazard process. This is no less true for dental healthcare than other types of healthcare. One common policy strategy to help promote knowledge translation is the production of clinical guidance, but it has been demonstrated that the simple publication of guidance is unlikely to optimise practice. Additional knowledge translation interventions have been shown to be effective, but effectiveness varies and much of this variation is unexplained. The need for researchers to move beyond single studies to develop a generalisable, theory based, knowledge translation framework has been identified.

For dentistry in Scotland, the production of clinical guidance is the responsibility of the Scottish Dental Clinical Effectiveness Programme (SDCEP). TRiaDS (Translation Research in a Dental Setting) is a multidisciplinary research collaboration, embedded within the SDCEP guidance development process, which aims to establish a practical evaluative framework for the translation of guidance and to conduct and evaluate a programme of integrated, multi-disciplinary research to enhance the science of knowledge translation.

Methods

Set in General Dental Practice the TRiaDS programmatic evaluation employs a standardised process using optimal methods and theory. For each SDCEP guidance document a diagnostic analysis is undertaken alongside the guidance development process. Information is gathered about current dental care activities. Key recommendations and their required behaviours are identified and prioritised. Stakeholder questionnaires and interviews are used to identify and elicit salient beliefs regarding potential barriers and enablers towards the key recommendations and behaviours. Where possible routinely collected data are used to measure compliance with the guidance and to inform decisions about whether a knowledge translation intervention is required. Interventions are theory based and informed by evidence gathered during the diagnostic phase and by prior published evidence. They are evaluated using a range of experimental and quasi-experimental study designs, and data collection continues beyond the end of the intervention to investigate the sustainability of an intervention effect.

Discussion

The TRiaDS programmatic approach is a significant step forward towards the development of a practical, generalisable framework for knowledge translation research. The multidisciplinary composition of the TRiaDS team enables consideration of the individual, organisational and system determinants of professional behaviour change. In addition the embedding of TRiaDS within a national programme of guidance development offers a unique opportunity to inform and influence the guidance development process, and enables TRiaDS to inform dental services practitioners, policy makers and patients on how best to translate national recommendations into routine clinical activities.

Background

Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ("recall") and precision of a well-known electronic search strategy ("randomized controlled trial" as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time.

Methods

We manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews.

Results

The existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%).

Conclusions

CRTs should include the phrase "cluster randomized trial" in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization.

Background

There is growing interest in the use of cognitive, behavioural, and organisational theories in implementation research. However, the extent of use of theory in implementation research is uncertain.

Methods

We conducted a systematic review of use of theory in 235 rigorous evaluations of guideline dissemination and implementation studies published between 1966 and 1998. Use of theory was classified according to type of use (explicitly theory based, some conceptual basis, and theoretical construct used) and stage of use (choice/design of intervention, process/mediators/moderators, and post hoc/explanation).

Results

Fifty-three of 235 studies (22.5%) were judged to have employed theories, including 14 studies that explicitly used theory. The majority of studies (n = 42) used only one theory; the maximum number of theories employed by any study was three. Twenty-five different theories were used. A small number of theories accounted for the majority of theory use including PRECEDE (Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation), diffusion of innovations, information overload and social marketing (academic detailing).

Conclusions

There was poor justification of choice of intervention and use of theory in implementation research in the identified studies until at least 1998. Future research should explicitly identify the justification for the interventions. Greater use of explicit theory to understand barriers, design interventions, and explore mediating pathways and moderators is needed to advance the science of implementation research.

Background

Blood transfusion is an essential part of healthcare and can improve patient outcomes. However, like most therapies, it is also associated with significant clinical risks. In addition, there is some evidence of overuse. Understanding the potential barriers and enablers to reduced prescribing of blood products will facilitate the selection of intervention components likely to be effective, thereby reducing the number of costly trials evaluating different implementation strategies. Using a theoretical basis to understand behaviours targeted for change will contribute to a 'basic science' relating to determinants of professional behaviour and how these inform the selection of techniques for changing behaviour. However, it is not clear which theories of behaviour are relevant to clinicians' transfusing behaviour. The aim of this study is to use a theoretical domains framework to identify relevant theories, and to use these theories to identify factors that predict the decision to transfuse.

Methods

The study involves two steps: interview study and questionnaire study. Using a previously identified framework, we will conduct semi-structured interviews with clinicians to elicit their views about which factors are associated with waiting and further monitoring the patient rather than transfusing red blood cells. Interviews will cover the following theoretical domains: knowledge; skills; social/professional role and identity; beliefs about capabilities; beliefs about consequences; motivation and goals; memory, attention, and decision processes; environmental context and resources; social influences; emotion; behavioural regulation; nature of the behaviour. The interviews will take place independently in Canada and the UK and involve two groups of physicians in each country (UK: adult and neonatal intensive care physicians; Canada: intensive care physicians and orthopaedic surgeons). We will: analyse interview transcript content to select relevant theoretical domains; use consensus processes to map these domains on to theories of behaviour; develop questionnaires based on these theories; and mail them to each group of physicians in the two countries. From our previous work, it is likely that the theories will include: theory of planned behaviour, social cognitive theory and the evidence-based strategy, implementation intention. The questionnaire data will measure predictor variables (theoretical constructs) and outcome variables (intention and clinical decision), and will be analysed using multiple regression analysis. We aim to achieve 150 respondents in each of the four groups for each postal survey.

Background

Cluster randomized trials are an increasingly important methodological tool in health research. In cluster randomized trials, intact social units or groups of individuals, such as medical practices, schools, or entire communities – rather than individual themselves – are randomly allocated to intervention or control conditions, while outcomes are then observed on individual cluster members. The substantial methodological differences between cluster randomized trials and conventional randomized trials pose serious challenges to the current conceptual framework for research ethics. The ethical implications of randomizing groups rather than individuals are not addressed in current research ethics guidelines, nor have they even been thoroughly explored. The main objectives of this research are to: (1) identify ethical issues arising in cluster trials and learn how they are currently being addressed; (2) understand how ethics reviews of cluster trials are carried out in different countries (Canada, the USA and the UK); (3) elicit the views and experiences of trial participants and cluster representatives; (4) develop well-grounded guidelines for the ethical conduct and review of cluster trials by conducting an extensive ethical analysis and organizing a consensus process; (5) disseminate the guidelines to researchers, research ethics boards (REBs), journal editors, and research funders.

Methods

We will use a mixed-methods (qualitative and quantitative) approach incorporating both empirical and conceptual work. Empirical work will include a systematic review of a random sample of published trials, a survey and in-depth interviews with trialists, a survey of REBs, and in-depth interviews and focus group discussions with trial participants and gatekeepers. The empirical work will inform the concurrent ethical analysis which will lead to a guidance document laying out principles, policy options, and rationale for proposed guidelines. An Expert Panel of researchers, ethicists, health lawyers, consumer advocates, REB members, and representatives from low-middle income countries will be appointed. A consensus conference will be convened and draft guidelines will be generated by the Panel; an e-consultation phase will then be launched to invite comments from the broader community of researchers, policy-makers, and the public before a final set of guidelines is generated by the Panel and widely disseminated by the research team.

Background

Since September 2005 the International Committee of Medical Journal Editors has required that trials be registered in accordance with the World Health Organization (WHO) minimum dataset, in order to be considered for publication. The objective is to evaluate registries' and individual trial records' compliance with the 2006 version of the WHO minimum data set.

Methods

A retrospective evaluation of 21 online clinical trial registries (international, national, specialty, pharmaceutical industry and local) from April 2005 to February 2007 and a cross-sectional evaluation of a stratified random sample of 610 trial records from the 21 registries.

Results

Among 11 registries that provided guidelines for registration, the median compliance with the WHO criteria were 14 out of 20 items (range 6 to 20). In the period April 2005–February 2007, six registries increased their compliance by six data items, on average. None of the local registry websites published guidelines on the trial data items required for registration. Slightly more than half (330/610; 54.1%, 95% CI 50.1% – 58.1%) of trial records completed the contact details criteria while 29.7% (181/610, 95% CI 26.1% – 33.5%) completed the key clinical and methodological data fields.

Conclusion

While the launch of the WHO minimum data set seemed to positively influence registries with better standardisation of approaches, individual registry entries are largely incomplete. Initiatives to ensure quality assurance of registries and trial data should be encouraged. Peer reviewers and editors should scrutinise clinical trial registration records to ensure consistency with WHO's core content requirements when considering trial-related publications.

Complex behaviour change interventions are not well described; when they are described, the terminology used is inconsistent. This constrains scientific replication, and limits the subsequent introduction of successful interventions. Implementation Science is introducing a policy of initially encouraging and subsequently requiring the scientific reporting of complex behaviour change interventions.

Background

Practice guidelines can improve health care delivery and outcomes but several issues challenge guideline adoption, including their intrinsic attributes, and whether and how they are implemented. It appears that guideline format may influence accessibility and ease of use, which may overcome attitudinal barriers of guideline adoption, and appear to be important to all stakeholders. Guideline content may facilitate various forms of decision making about guideline adoption relevant to different stakeholders. Knowledge and attitudes about, and incentives and capacity for implementation on the part of guideline sponsors may influence whether and how they develop guidelines containing these features, and undertake implementation. Examination of these issues may yield opportunities to improve guideline adoption.

Methods

The attributes hypothesized to facilitate adoption will be expanded by thematic analysis, and quantitative and qualitative summary of the content of international guidelines for two primary care (diabetes, hypertension) and institutional care (chronic ulcer, chronic heart failure) topics. Factors that influence whether and how guidelines are implemented will be explored by qualitative analysis of interviews with individuals affiliated with guideline sponsoring agencies.

Discussion

Previous research examined guideline implementation by measuring rates of compliance with recommendations or associated outcomes, but this produced little insight on how the products themselves, or their implementation, could be improved. This research will establish a theoretical basis upon which to conduct experimental studies to compare the cost-effectiveness of interventions that enhance guideline development and implementation capacity. Such studies could first examine short-term outcomes predictive of guideline utilization, such as recall, attitude toward, confidence in, and adoption intention. If successful, then long-term objective outcomes reflecting the adoption of processes and associated patient care outcomes could be evaluated.

Background

Since being identified as a concept for understanding knowledge sharing, management, and creation, communities of practice (CoPs) have become increasingly popular within the health sector. The CoP concept has been used in the business sector for over 20 years, but the use of CoPs in the health sector has been limited in comparison.

Objectives

First, we examined how CoPs were defined and used in these two sectors. Second, we evaluated the evidence of effectiveness on the health sector CoPs for improving the uptake of best practices and mentoring new practitioners.

Methods

We conducted a search of electronic databases in the business, health, and education sectors, and a hand search of key journals for primary studies on CoP groups. Our research synthesis for the first objective focused on three areas: the authors' interpretations of the CoP concept, the key characteristics of CoP groups, and the common elements of CoP groups. To examine the evidence on the effectiveness of CoPs in the health sector, we identified articles that evaluated CoPs for improving health professional performance, health care organizational performance, professional mentoring, and/or patient outcome; and used experimental, quasi-experimental, or observational designs.

Results

The structure of CoP groups varied greatly, ranging from voluntary informal networks to work-supported formal education sessions, and from apprentice training to multidisciplinary, multi-site project teams. Four characteristics were identified from CoP groups: social interaction among members, knowledge sharing, knowledge creation, and identity building; however, these were not consistently present in all CoPs. There was also a lack of clarity in the responsibilities of CoP facilitators and how power dynamics should be handled within a CoP group. We did not find any paper in the health sector that met the eligibility criteria for the quantitative analysis, and so the effectiveness of CoP in this sector remained unclear.

Conclusion

There is no dominant trend in how the CoP concept is operationalized in the business and health sectors; hence, it is challenging to define the parameters of CoP groups. This may be one of the reasons for the lack of studies on the effectiveness of CoPs in the health sector. In order to improve the usefulness of the CoP concept in the development of groups and teams, further research will be needed to clarify the extent to which the four characteristics of CoPs are present in the mature and emergent groups, the expectations of facilitators and other participants, and the power relationship within CoPs.

Background

In the experience of health professionals, it appears that interacting with peers in the workplace fosters learning and information sharing. Informal groups and networks present good opportunities for information exchange. Communities of practice (CoPs), which have been described by Wenger and others as a type of informal learning organization, have received increasing attention in the health care sector; however, the lack of uniform operating definitions of CoPs has resulted in considerable variation in the structure and function of these groups, making it difficult to evaluate their effectiveness.

Objective

To critique the evolution of the CoP concept as based on the germinal work by Wenger and colleagues published between 1991 and 2002.

Discussion

CoP was originally developed to provide a template for examining the learning that happens among practitioners in a social environment, but over the years there have been important divergences in the focus of the concept. Lave and Wenger's earliest publication (1991) centred on the interactions between novices and experts, and the process by which newcomers create a professional identity. In the 1998 book, the focus had shifted to personal growth and the trajectory of individuals' participation within a group (i.e., peripheral versus core participation). The focus then changed again in 2002 when CoP was applied as a managerial tool for improving an organization's competitiveness.

Summary

The different interpretations of CoP make it challenging to apply the concept or to take full advantage of the benefits that CoP groups may offer. The tension between satisfying individuals' needs for personal growth and empowerment versus an organization's bottom line is perhaps the most contentious of the issues that make CoPs difficult to cultivate. Since CoP is still an evolving concept, we recommend focusing on optimizing specific characteristics of the concept, such as support for members interacting with each other, sharing knowledge, and building a sense of belonging within networks/teams/groups. Interventions that facilitate relationship building among members and that promote knowledge exchange may be useful for optimizing the function of these groups.

Background

Improved quality of care is a policy objective of health care systems around the world. Implementation research is the scientific study of methods to promote the systematic uptake of clinical research findings into routine clinical practice, and hence to reduce inappropriate care. It includes the study of influences on healthcare professionals' behaviour and methods to enable them to use research findings more effectively. Cluster randomized trials represent the optimal design for evaluating the effectiveness of implementation strategies. Various codes of medical ethics, such as the Nuremberg Code and the Declaration of Helsinki inform medical research, but their relevance to cluster randomised trials in implementation research is unclear. This paper discusses the applicability of various ethical codes to obtaining consent in cluster trials in implementation research.

Discussion

The appropriate application of biomedical codes to implementation research is not obvious. Discussion of the nature and practice of informed consent in implementation research cluster trials must consider the levels at which consent can be sought, and for what purpose it can be sought. The level at which an intervention is delivered can render the idea of patient level consent meaningless. Careful consideration of the ownership of information, and rights of access to and exploitation of data is required. For health care professionals and organizations, there is a balance between clinical freedom and responsibility to participate in research.

Summary

While ethical justification for clinical trials relies heavily on individual consent, for implementation research aspects of distributive justice, economics, and political philosophy underlie the debate. Societies may need to trade off decisions on the choice between individualized consent and valid implementation research. We suggest that social sciences codes could usefully inform the consideration of implementation research by members of Research Ethics Committees.

Background

The results of randomised controlled trials can be usefully illuminated by studies of the processes by which they achieve their effects. The Theory of Planned Behaviour (TPB) offers a framework for conducting such studies. This study used TPB to explore the observed effects in a pragmatic cluster randomised controlled trial of a structured recall and prompting intervention to increase evidence-based diabetes care that was conducted in three Primary Care Trusts in England.

Methods

All general practitioners and nurses in practices involved in the trial were sent a postal questionnaire at the end of the intervention period, based on the TPB (predictor variables: attitude; subjective norm; perceived behavioural control, or PBC). It focussed on three clinical behaviours recommended in diabetes care: measuring blood pressure; inspecting feet; and prescribing statins. Multivariate analyses of variance and multiple regression analyses were used to explore changes in cognitions and thereby better understand trial effects.

Results

Fifty-nine general medical practitioners and 53 practice nurses (intervention: n = 55, 41.98% of trial participants; control: n = 57, 38.26% of trial participants) completed the questionnaire. There were no differences between groups in mean scores for attitudes, subjective norms, PBC or intentions. Control group clinicians had 'normatively-driven' intentions (i.e., related to subjective norm scores), whereas intervention group clinicians had 'attitudinally-driven' intentions (i.e., related to attitude scores) for foot inspection and statin prescription. After controlling for effects of the three predictor variables, this group difference was significant for foot inspection behaviour (trial group × attitude interaction, beta = 0.72, p < 0.05; trial group × subjective norm interaction, beta = -0.65, p < 0.05).

Conclusion

Attitudinally-driven intentions are proposed to be more consistently translated into action than normatively-driven intentions. This proposition was supported by the findings, thus offering an interpretation of the trial effects. This analytic approach demonstrates the potential of the TPB to explain trial effects in terms of different relationships between variables rather than differences in mean scores. This study illustrates the use of theory-based process evaluation to uncover processes underlying change in implementation trials.

Background

Evidence generated from reliable research is not frequently implemented into clinical practice. Evidence-based clinical practice guidelines are a potential vehicle to achieve this. A recent systematic review of implementation strategies of guideline dissemination concluded that there was a lack of evidence regarding effective strategies to promote the uptake of guidelines. Recommendations from this review, and other studies, have suggested the use of interventions that are theoretically based because these may be more effective than those that are not. An evidence-based clinical practice guideline for the management of acute low back pain was recently developed in Australia. This provides an opportunity to develop and test a theory-based implementation intervention for a condition which is common, has a high burden, and for which there is an evidence-practice gap in the primary care setting.

Aim

This study aims to test the effectiveness of a theory-based intervention for implementing a clinical practice guideline for acute low back pain in general practice in Victoria, Australia. Specifically, our primary objectives are to establish if the intervention is effective in reducing the percentage of patients who are referred for a plain x-ray, and improving mean level of disability for patients three months post-consultation.

Methods/Design

This study protocol describes the details of a cluster randomised controlled trial. Ninety-two general practices (clusters), which include at least one consenting general practitioner, will be randomised to an intervention or control arm using restricted randomisation. Patients aged 18 years or older who visit a participating practitioner for acute non-specific low back pain of less than three months duration will be eligible for inclusion. An average of twenty-five patients per general practice will be recruited, providing a total of 2,300 patient participants. General practitioners in the control arm will receive access to the guideline using the existing dissemination strategy. Practitioners in the intervention arm will be invited to participate in facilitated face-to-face workshops that have been underpinned by behavioural theory. Investigators (not involved in the delivery of the intervention), patients, outcome assessors and the study statistician will be blinded to group allocation.

Trial registration

Australian New Zealand Clinical Trials Registry ACTRN012606000098538 (date registered 14/03/2006).

Background

Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.

Methodology

AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.

Conclusions

The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were −0.19±1.38. This translates to a minimum detectable difference between reviews of 0.64 ‘AMSTAR points’. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.

Background

A challenge for implementation researchers is to develop principles that could generate testable hypotheses that apply across a range of clinical contexts, thus leading to generalisability of findings. Such principles may be provided by systematically developed theories. The opportunity has arisen to test some of these theoretical principles in the Ontario Printed Educational Materials (OPEM) trial by conducting a sub-trial within the existing trial structure. OPEM is a large factorial cluster-randomised trial evaluating the effects of short directive and long discursive educational messages embedded into informed, an evidence-based newsletter produced in Canada by the Institute for Clinical Evaluative Sciences (ICES) and mailed to all primary care physicians in Ontario. The content of educational messages in the sub-trial will be constructed using both standard methods and methods inspired by psychological theory. The aim of this study is to test the effectiveness of the TheoRY-inspired MEssage ('TRY-ME') compared with the 'standard' message in changing prescribing behaviour.

Methods

The OPEM trial participants randomised to receive the short directive message attached to the outside of informed (an 'outsert') will be sub-randomised to receive either a standard message or a message informed by the theory of planned behaviour (TPB) using a two (long insert or no insert) by three (theory-based outsert or standard outsert or no outsert) design. The messages will relate to prescription of thiazide diuretics as first line drug treatment for hypertension (described in the accompanying protocol, "The Ontario Printed Educational Materials trial"). The short messages will be developed independently by two research teams.

The primary outcome is prescription of thiazide diuretics, measured by routinely collected data available within ICES. The study is designed to answer the question, is there any difference in guideline adherence (i.e., thiazide prescription rates) between physicians in the six groups? A process evaluation survey instrument based on the TPB will be administered pre- and post-intervention (described in the accompanying protocol, "Looking inside the black box"). The second research question concerns processes that may underlie observed differences in prescribing behaviour. We expect that effects of the messages on prescribing behaviour will be mediated through changes in physicians' cognitions.

Trial registration number

Current controlled trial ISRCTN72772651

Background

Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths).

This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs.

Methods/design

We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.

Trial registration number

Current controlled trial ISRCTN72772651

Background

Psychological models can be used to understand and predict behaviour in a wide range of settings. However, they have not been consistently applied to health professional behaviours, and the contribution of differing theories is not clear. The aim of this study was to explore the usefulness of a range of psychological theories to predict health professional behaviour relating to management of upper respiratory tract infections (URTIs) without antibiotics.

Methods

Psychological measures were collected by postal questionnaire survey from a random sample of general practitioners (GPs) in Scotland. The outcome measures were clinical behaviour (using antibiotic prescription rates as a proxy indicator), behavioural simulation (scenario-based decisions to managing URTI with or without antibiotics) and behavioural intention (general intention to managing URTI without antibiotics). Explanatory variables were the constructs within the following theories: Theory of Planned Behaviour (TPB), Social Cognitive Theory (SCT), Common Sense Self-Regulation Model (CS-SRM), Operant Learning Theory (OLT), Implementation Intention (II), Stage Model (SM), and knowledge (a non-theoretical construct). For each outcome measure, multiple regression analysis was used to examine the predictive value of each theoretical model individually. Following this 'theory level' analysis, a 'cross theory' analysis was conducted to investigate the combined predictive value of all significant individual constructs across theories.

Results

All theories were tested, but only significant results are presented. When predicting behaviour, at the theory level, OLT explained 6% of the variance and, in a cross theory analysis, OLT 'evidence of habitual behaviour' also explained 6%. When predicting behavioural simulation, at the theory level, the proportion of variance explained was: TPB, 31%; SCT, 26%; II, 6%; OLT, 24%. GPs who reported having already decided to change their management to try to avoid the use of antibiotics made significantly fewer scenario-based decisions to prescribe. In the cross theory analysis, perceived behavioural control (TPB), evidence of habitual behaviour (OLT), CS-SRM cause (chance/bad luck), and intention entered the equation, together explaining 36% of the variance. When predicting intention, at the theory level, the proportion of variance explained was: TPB, 30%; SCT, 29%; CS-SRM 27%; OLT, 43%. GPs who reported that they had already decided to change their management to try to avoid the use of antibiotics had a significantly higher intention to manage URTIs without prescribing antibiotics. In the cross theory analysis, OLT evidence of habitual behaviour, TPB attitudes, risk perception, CS-SRM control by doctor, TPB perceived behavioural control and CS-SRM control by treatment entered the equation, together explaining 49% of the variance in intention.

Conclusion

The study provides evidence that psychological models can be useful in understanding and predicting clinical behaviour. Taking a theory-based approach enables the creation of a replicable methodology for identifying factors that predict clinical behaviour. However, a number of conceptual and methodological challenges remain.

Background

Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.

Methods

A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.

Results

The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.

Conclusion

A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.

Background

Systematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions.

Methods

We identifed updated Cochrane SRs published in issue 4, 2002 of the Cochrane Library. We assessed the updated and original versions of the SRs using two instruments: the 10 item enhanced Overview Quality Assessment Questionnaire (OQAQ), and an 18-item reporting quality checklist and flow chart based upon the Quality of Reporting of Meta-analyses (QUOROM) statement. At least two reviewers extracted data and assessed quality. We calculated the percentage (with a 95% confidence interval) of 'yes' answers to each question. We calculated mean differences in percentage, 95% confidence intervals and p-values for each of the individual items and the overall methodological quality score of the updated and pre-updated versions using OQAQ.

Results

We assessed 53 SRs. There was no significant improvement in the global quality score of the OQAQ (mean difference 0.11 (-0.28; 0.70 p = 0.52)). Updated reviews showed a significant improvement of 18.9 (7.2; 30.6 p < .01) on the OQAQ item assessing whether the conclusions drawn by the author(s) were supported by the data and/or analysis presented in the SR. The QUOROM statement showed that the quality of reporting of Cochrane reviews improved in some areas with updating. Improvements were seen on the items relating to data sources reported in the abstract, with a significant difference of 17.0 (9.8; 28.7 p = 0.01), review methods, reported in the abstract 35 (24.1; 49.1 p = 0.00), searching methods 18.9 (9.7; 31.6 p = 0.01), and data abstraction 18.9 (11.7; 30.9 p = 0.00).

Conclusion

The overall quality of Cochrane SRs is fair-to-good. Although reporting quality improved on certain individual items there was no overall improvement seen with updating and methodological quality remained unchanged. Further improvement of quality of reporting is possible. There is room for improvement of methodological quality as well. Authors updating reviews should address identified methodological or reporting weaknesses. We recommend to give full attention to both quality domains when updating SRs.

This editorial introduces the new online, open access, peer-reviewed journal Trials. The journal considers manuscripts on any aspect of the design, performance, and findings of randomised controlled trials in any discipline related to health care, and also encourages the publication of protocols. Trialists will be able to provide the necessary detail for a true and complete scientific record. They will be able to communicate not only all outcome measures, as well as varying analyses and interpretations, but also in-depth descriptions of what they did and honest reflections about what they learnt.

Trials also encourages articles covering generic issues related to trials, for example focussing on the design, conduct, analysis, interpretation, or reporting.

Background

There is only limited empirical evidence about the effectiveness of opinion leaders as health care change agents.

Aim

To test the feasibility of identifying, and the characteristics of, opinion leaders using a sociometric instrument and a self-designating instrument in different professional groups within the UK National Health Service.

Design

Postal questionnaire survey.

Setting and participants

All general practitioners, practice nurses and practice managers in two regions of Scotland. All physicians and surgeons (junior hospital doctors and consultants) and medical and surgical nursing staff in two district general hospitals and one teaching hospital in Scotland, as well as all Scottish obstetric and gynaecology, and oncology consultants.

Results

Using the sociometric instrument, the extent of social networks and potential coverage of the study population in primary and secondary care was highly idiosyncratic. In contrast, relatively complex networks with good coverage rates were observed in both national specialty groups. Identified opinion leaders were more likely to have the expected characteristics of opinion leaders identified from diffusion and social influence theories. Moreover, opinion leaders appeared to be condition-specific. The self-designating instrument identified more opinion leaders, but it was not possible to estimate the extent and structure of social networks or likely coverage by opinion leaders. There was poor agreement in the responses to the sociometric and self-designating instruments.

Conclusion

The feasibility of identifying opinion leaders using an off-the-shelf sociometric instrument is variable across different professional groups and settings within the NHS. Whilst it is possible to identify opinion leaders using a self-designating instrument, the effectiveness of such opinion leaders has not been rigorously tested in health care settings. Opinion leaders appear to be monomorphic (different leaders for different issues). Recruitment of opinion leaders is unlikely to be an effective general strategy across all settings and professional groups; the more specialised the group, the more opinion leaders may be a useful strategy.