PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-7 (7)
 

Clipboard (0)
None
Journals
Year of Publication
Document Types
1.  External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR) 
PLoS ONE  2007;2(12):e1350.
Background
Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.
Methodology
AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.
Conclusions
The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were −0.19±1.38. This translates to a minimum detectable difference between reviews of 0.64 ‘AMSTAR points’. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.
doi:10.1371/journal.pone.0001350
PMCID: PMC2131785  PMID: 18159233
2.  Testing a TheoRY-inspired MEssage ('TRY-ME'): a sub-trial within the Ontario Printed Educational Message (OPEM) trial 
Background
A challenge for implementation researchers is to develop principles that could generate testable hypotheses that apply across a range of clinical contexts, thus leading to generalisability of findings. Such principles may be provided by systematically developed theories. The opportunity has arisen to test some of these theoretical principles in the Ontario Printed Educational Materials (OPEM) trial by conducting a sub-trial within the existing trial structure. OPEM is a large factorial cluster-randomised trial evaluating the effects of short directive and long discursive educational messages embedded into informed, an evidence-based newsletter produced in Canada by the Institute for Clinical Evaluative Sciences (ICES) and mailed to all primary care physicians in Ontario. The content of educational messages in the sub-trial will be constructed using both standard methods and methods inspired by psychological theory. The aim of this study is to test the effectiveness of the TheoRY-inspired MEssage ('TRY-ME') compared with the 'standard' message in changing prescribing behaviour.
Methods
The OPEM trial participants randomised to receive the short directive message attached to the outside of informed (an 'outsert') will be sub-randomised to receive either a standard message or a message informed by the theory of planned behaviour (TPB) using a two (long insert or no insert) by three (theory-based outsert or standard outsert or no outsert) design. The messages will relate to prescription of thiazide diuretics as first line drug treatment for hypertension (described in the accompanying protocol, "The Ontario Printed Educational Materials trial"). The short messages will be developed independently by two research teams.
The primary outcome is prescription of thiazide diuretics, measured by routinely collected data available within ICES. The study is designed to answer the question, is there any difference in guideline adherence (i.e., thiazide prescription rates) between physicians in the six groups? A process evaluation survey instrument based on the TPB will be administered pre- and post-intervention (described in the accompanying protocol, "Looking inside the black box"). The second research question concerns processes that may underlie observed differences in prescribing behaviour. We expect that effects of the messages on prescribing behaviour will be mediated through changes in physicians' cognitions.
Trial registration number
Current controlled trial ISRCTN72772651
doi:10.1186/1748-5908-2-39
PMCID: PMC2216024  PMID: 18039363
3.  Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada 
Background
Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths).
This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs.
Methods/design
We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.
Trial registration number
Current controlled trial ISRCTN72772651
doi:10.1186/1748-5908-2-38
PMCID: PMC2213685  PMID: 18039362
4.  Applying psychological theories to evidence-based clinical practice: Identifying factors predictive of managing upper respiratory tract infections without antibiotics 
Background
Psychological models can be used to understand and predict behaviour in a wide range of settings. However, they have not been consistently applied to health professional behaviours, and the contribution of differing theories is not clear. The aim of this study was to explore the usefulness of a range of psychological theories to predict health professional behaviour relating to management of upper respiratory tract infections (URTIs) without antibiotics.
Methods
Psychological measures were collected by postal questionnaire survey from a random sample of general practitioners (GPs) in Scotland. The outcome measures were clinical behaviour (using antibiotic prescription rates as a proxy indicator), behavioural simulation (scenario-based decisions to managing URTI with or without antibiotics) and behavioural intention (general intention to managing URTI without antibiotics). Explanatory variables were the constructs within the following theories: Theory of Planned Behaviour (TPB), Social Cognitive Theory (SCT), Common Sense Self-Regulation Model (CS-SRM), Operant Learning Theory (OLT), Implementation Intention (II), Stage Model (SM), and knowledge (a non-theoretical construct). For each outcome measure, multiple regression analysis was used to examine the predictive value of each theoretical model individually. Following this 'theory level' analysis, a 'cross theory' analysis was conducted to investigate the combined predictive value of all significant individual constructs across theories.
Results
All theories were tested, but only significant results are presented. When predicting behaviour, at the theory level, OLT explained 6% of the variance and, in a cross theory analysis, OLT 'evidence of habitual behaviour' also explained 6%. When predicting behavioural simulation, at the theory level, the proportion of variance explained was: TPB, 31%; SCT, 26%; II, 6%; OLT, 24%. GPs who reported having already decided to change their management to try to avoid the use of antibiotics made significantly fewer scenario-based decisions to prescribe. In the cross theory analysis, perceived behavioural control (TPB), evidence of habitual behaviour (OLT), CS-SRM cause (chance/bad luck), and intention entered the equation, together explaining 36% of the variance. When predicting intention, at the theory level, the proportion of variance explained was: TPB, 30%; SCT, 29%; CS-SRM 27%; OLT, 43%. GPs who reported that they had already decided to change their management to try to avoid the use of antibiotics had a significantly higher intention to manage URTIs without prescribing antibiotics. In the cross theory analysis, OLT evidence of habitual behaviour, TPB attitudes, risk perception, CS-SRM control by doctor, TPB perceived behavioural control and CS-SRM control by treatment entered the equation, together explaining 49% of the variance in intention.
Conclusion
The study provides evidence that psychological models can be useful in understanding and predicting clinical behaviour. Taking a theory-based approach enables the creation of a replicable methodology for identifying factors that predict clinical behaviour. However, a number of conceptual and methodological challenges remain.
doi:10.1186/1748-5908-2-26
PMCID: PMC2042498  PMID: 17683558
5.  A cluster randomised controlled trial of educational prompts in diabetes care: study protocol 
Background
Laboratory services have a central role in supporting screening, diagnosis, and management of patients. The increase in chronic disease management in primary care for conditions such as diabetes mellitus requires regular monitoring of patients' biochemical parameters. This process offers a route for improving the quality of care that patients receive by using test results as a vehicle for delivering educational messages as well as the test result itself.
Aim
To develop and evaluate the effectiveness of a quality improvement initiative to improve the care of patients with diabetes using test report reminders.
Design
A programme of four cluster randomised controlled trials within one population of general practices.
Participants
General practices in Newcastle-upon-Tyne, UK.
Intervention
Brief educational messages added to paper and electronic general practice laboratory test reports introduced over two phases. Phase One messages, attached to Haemoglobin A1c (HbA1c) reports, targeted glycaemic and cholesterol control. Phase Two messages, attached to albumin:creatinine ratio (ACR) reports, targeted blood pressure (BP) control and foot inspection.
Outcomes
General practice mean levels of HbA1c and cholesterol (Phase One) and diastolic and systolic BP and proportions of patients having undergone foot inspections (Phase Two); number of tests requested.
Trial registration
Current Controlled Trial ISRCTN2186314.
doi:10.1186/1748-5908-2-22
PMCID: PMC1973073  PMID: 17650309
6.  A pragmatic cluster randomised controlled trial of a Diabetes REcall And Management system: the DREAM trial 
Background
Following the introduction of a computerised diabetes register in part of the northeast of England, care initially improved but then plateaued. We therefore enhanced the existing diabetes register to address these problems. The aim of the trial was to evaluate the effectiveness and efficiency of an area wide 'extended,' computerised diabetes register incorporating a full structured recall and management system, including individualised patient management prompts to primary care clinicians based on locally-adapted, evidence-based guidelines.
Methods
The study design was a pragmatic, cluster randomised controlled trial, with the general practice as the unit of randomisation. Set in 58 general practices in three Primary Care Trusts in the northeast of England, the study outcomes were the clinical process and outcome variables held on the diabetes register, patient-reported outcomes, and service and patient costs. The effect of the intervention was estimated using generalised linear models with an appropriate error structure. To allow for the clustering of patients within practices, population averaged models were estimated using generalized estimating equations.
Results
Patients in intervention practices were more likely to have at least one diabetes appointment recorded (OR 2.00, 95% CI 1.02, 3.91), to have a recording of a foot check (OR 1.87, 95% CI 1.09, 3.21), have a recording of receiving dietary advice (OR 2.77, 95% CI 1.22, 6.29), and have a recording of blood pressure (BP) (OR 2.14, 95% CI 1.06, 4.36). There was no difference in mean HbA1c or BP levels, but the mean cholesterol level in patients from intervention practices was significantly lower (-0.15 mmol/l, 95% CI -0.25, -0.06). There were no differences in patient-reported outcomes or in patient-reported use of drugs, or uptake of health services. The average cost per patient was not significantly different between the intervention and control groups. Costs incurred in administering the system at the register and in general practice were in addition to these.
Conclusion
This study has shown benefits from an area-wide, computerised diabetes register incorporating a full structured recall and individualised patient management system. However, these benefits were achieved at a cost. In future, these costs may fall as electronic data exchange becomes a reliable reality.
Trial registration: International Standard Randomised Controlled Trial Number (ISRCTN) Register, ISRCTN32042030.
doi:10.1186/1748-5908-2-6
PMCID: PMC1804280  PMID: 17306017
7.  Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews 
Background
Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.
Methods
A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.
Results
The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.
Conclusion
A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.
doi:10.1186/1471-2288-7-10
PMCID: PMC1810543  PMID: 17302989

Results 1-7 (7)