Development of medical test guidelines differs from intervention guideline development. These differences can pose unique challenges in building evidence-based recommendations to guide clinical practice. The aim of our study was to better understand these challenges, explore reasons behind them and identify possible solutions.
Setting and participants
In this qualitative study, we conducted in-depth interviews between February 2012 and April 2013 of a convenience sample of 17 European guideline developers experienced in medical test guideline development.
We used framework analysis with deductive and inductive approaches to generate the themes from the interviews. We kept interpretation grounded in the data.
Guideline developers acknowledged that inclusion of patient important outcomes in their guideline development was necessary but lacking. This and other challenges raised fell into 3 broad and overlapping domains: methodological issues, resource limitations and a lack of awareness on the need for evidence that links testing to patient outcomes. Education was mentioned as a key solution to increase awareness and address the resources limitations mentioned.
Challenges guideline developers face were interlinked across the domains of methodological issues, resource limitations and a lack of awareness. Solutions that addressed these challenges in parallel are needed. Raising awareness, education and training of relevant stakeholders such as medical doctors, funders and regulators to look beyond test accuracy is key to having a long-term resolution to the issues faced in medical test guideline development.
medical tests; guideline development; test accuracy; medical education; in depth interviews; QUALITATIVE RESEARCH
Judgments underlying guideline recommendations are seldom recorded and presented in a systematic fashion. The GRADE Evidence-to-Decision Framework (EtD) offers a transparent way to record and report guideline developers’ judgments. In this paper, we report the experiences with the EtD frameworks in 15 real guideline panels.
Following the guideline panel meetings, we asked methodologists participating in the panel to provide feedback regarding the EtD framework. They were instructed to consider their own experience and the feedback collected from the rest of the panel. Two investigators independently summarized the responses and jointly interpreted the data using pre-specified domains as coding system. We asked methodologists to review the results and provide further input to improve the structure of the EtDs iteratively.
The EtD framework was well received, and the comments were generally positive. Methodologists felt that in a real guideline panel, the EtD framework helps structuring a complex process through relatively simple steps in an explicit and transparent way. However, some sections (e.g., “values and preferences” and “balance between benefits and harms”) required further development and clarification that were considered in the current version of the EtD framework.
The use of an EtD framework in guideline development offers a structured and explicit way to record and report the judgments and discussion of guideline panels during the formulation of recommendations. In addition, it facilitates the formulation of recommendations, assessment of their strength, and identifying gaps in research.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-016-0462-y) contains supplementary material, which is available to authorized users.
Clinical practice guidelines; GRADE; Evidence to decisions framework; GRADEpro; Recommendations
Women with a history of venous thromboembolism (VTE) have an increased recurrence risk during pregnancy. Low molecular weight heparin (LMWH) reduces this risk, but is costly, burdensome, and may increase risk of bleeding. The decision to start thromboprophylaxis during pregnancy is sensitive to women's values and preferences. Our objective was to compare women's choices using a holistic approach in which they were presented all of the relevant information (direct-choice) versus a personalized decision analysis in which a mathematical model incorporated their preferences and VTE risk to make a treatment recommendation.
Multicenter, international study. Structured interviews were on women with a history of VTE who were pregnant, planning, or considering pregnancy. Women indicated their willingness to receive thromboprophylaxis based on scenarios using personalized estimates of VTE recurrence and bleeding risks. We also obtained women's values for health outcomes using a visual analog scale. We performed individualized decision analyses for each participant and compared model recommendations to decisions made when presented with the direct-choice exercise.
Of the 123 women in the study, the decision model recommended LMWH for 51 women and recommended against LMWH for 72 women. 12% (6/51) of women for whom the decision model recommended thromboprophylaxis chose not to take LMWH; 72% (52/72) of women for whom the decision model recommended against thromboprophylaxis chose LMWH.
We observed a high degree of discordance between decisions in the direct-choice exercise and decision model recommendations. Although which approach best captures individuals’ true values remains uncertain, personalized decision support tools presenting results based on personalized risks and values may improve decision making.
Decision making; Decision support techniques; Venous thromboembolism; Heparin; Pregnancy
Meta-analyses of continuous outcomes typically provide enough information for decision-makers to evaluate the extent to which chance can explain apparent differences between interventions. The interpretation of the magnitude of these differences — from trivial to large — can, however, be challenging. We investigated clinicians’ understanding and perceptions of usefulness of 6 statistical formats for presenting continuous outcomes from meta-analyses (standardized mean difference, minimal important difference units, mean difference in natural units, ratio of means, relative risk and risk difference).
We invited 610 staff and trainees in internal medicine and family medicine programs in 8 countries to participate. Paper-based, self-administered questionnaires presented summary estimates of hypothetical interventions versus placebo for chronic pain. The estimates showed either a small or a large effect for each of the 6 statistical formats for presenting continuous outcomes. Questions addressed participants’ understanding of the magnitude of treatment effects and their perception of the usefulness of the presentation format. We randomly assigned participants 1 of 4 versions of the questionnaire, each with a different effect size (large or small) and presentation order for the 6 formats (1 to 6, or 6 to 1).
Overall, 531 (87.0%) of the clinicians responded. Respondents best understood risk difference, followed by relative risk and ratio of means. Similarly, they perceived the dichotomous presentation of continuous outcomes (relative risk and risk difference) to be most useful. Presenting results as a standardized mean difference, the longest standing and most widely used approach, was poorly understood and perceived as least useful.
None of the presentation formats were well understood or perceived as extremely useful. Clinicians best understood the dichotomous presentations of continuous outcomes and perceived them to be the most useful. Further initiatives to help clinicians better grasp the magnitude of the treatment effect are needed.
We conducted a systematic survey of the methodological literature to identify recommended approaches for how and what randomised clinical trial (RCT) authors should report on missing participant data and, on the basis of these approaches, to propose guidance for RCT authors.
We defined missing participant data (MPD) as missing outcome data for trial participants. We considered both categorical and continuous outcome data. We searched MEDLINE and the Cochrane Methodology Register for articles in which authors proposed approaches to reporting MPD from RCTs. We selected eligible articles independently and in duplicate and extracted data in duplicate. Using an iterative process of discussion and revisions, we used the findings to develop guidance.
Of 10 501 unique citations identified, 13 articles reporting on 10 approaches proved eligible. The identified approaches recommend reporting the following aspects (from most to least frequently recommended): number of participants with MPD (n=10), reasons for MPD (n=7), methods used to handle MPD in the analysis (n=4), flow of participants (n=3), pattern of missingness (eg, whether at random) (n=3), differences in rates of MPD between trial arms (n=2), differences between participants with and without MPD (n=2), results of any sensitivity analyses (n=2), implication of MPD on interpreting the results (n=2) and methods used to prevent missing data (n=1). We propose a guide with nine items related to reporting the number, reasons, patterns, analytical methods and interpretation of MPD.
Most identified approaches invite trial authors to report the extent of MPD and the underlying reasons. Fewer approaches focus on reporting missingness patterns, methods for handling MPD and implications of MPD on results. Our proposed guidance could help RCT authors to better report, and readers to better identify participants with missing data.
EPIDEMIOLOGY; Missing participant data; Randomized clinical trials; Systematic reviews
To describe how systematic reviewers are reporting missing data for dichotomous outcomes, handling them in the analysis and assessing the risk of associated bias.
We searched MEDLINE and the Cochrane Database of Systematic Reviews for systematic reviews of randomised trials published in 2010, and reporting a meta-analysis of a dichotomous outcome. We randomly selected 98 Cochrane and 104 non-Cochrane systematic reviews. Teams of 2 reviewers selected eligible studies and abstracted data independently and in duplicate using standardised, piloted forms with accompanying instructions. We conducted regression analyses to explore factors associated with using complete case analysis and with judging the risk of bias associated with missing participant data.
Of Cochrane and non-Cochrane reviews, 47% and 7% (p<0.0001), respectively, reported on the number of participants with missing data, and 41% and 9% reported a plan for handling missing categorical data. The 2 most reported approaches for handling missing data were complete case analysis (8.5%, out of the 202 reviews) and assuming no participants with missing data had the event (4%). The use of complete case analysis was associated only with Cochrane reviews (relative to non-Cochrane: OR=7.25; 95% CI 1.58 to 33.3, p=0.01). 65% of reviews assessed risk of bias associated with missing data; this was associated with Cochrane reviews (relative to non-Cochrane: OR=6.63; 95% CI 2.50 to 17.57, p=0.0001), and the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology (OR=5.02; 95% CI 1.02 to 24.75, p=0.047).
Though Cochrane reviews are somewhat less problematic, most Cochrane and non-Cochrane systematic reviews fail to adequately report and handle missing data, potentially resulting in misleading judgements regarding risk of bias.
EPIDEMIOLOGY; STATISTICS & RESEARCH METHODS
Vitamin K antagonists are commonly used for the prevention of thromboembolic events. Patient self-monitoring of vitamin K antagonists has proved superior to usual care. Dabigatran has been shown, relative to warfarin, to reduce thromboembolic events without increasing bleeding.
We constructed a Markov model to compare vitamin K self-monitoring strategies to dabigatran including effectiveness and costs of monitoring and complications (thromboembolism and major bleeding). The model was used to project the incidence of these complications, life years, quality-adjusted life years, and health system costs with anticoagulant treatment throughout life. The analysis was conducted from the health system perspective and from the societal perspective.
Low quality evidence suggests that self-monitoring is at least as effective as dabigatran for the outcomes of thrombosis, bleeding and death. Moderate quality evidence that patient self-monitoring is more effective than other forms of monitoring degree of anticoagulation with vitamin K antagonists, reducing the relative risk of thromboembolism by 41 % and death by 34 %. The cost per quality adjusted year gained relative to other warfarin monitoring strategies is well below 30,000 € in the short term, and is a dominant alternative from the fourth year. In comparison with dabigatran, the lower annual cost and its equivalence in terms of effectiveness made self-monitoring the dominant option. These results were confirmed in the probabilistic sensitivity analysis.
We have moderate quality evidence that self-monitoring of vitamin K antagonists is a cost-effective alternative compared with hospital and primary care monitoring, and low quality evidence, compared with dabigatran. Our analyses contrast with the available cost analysis of dabigatran and usual care of anticoagulated patients.
Electronic supplementary material
The online version of this article (doi:10.1186/s12913-015-0934-9) contains supplementary material, which is available to authorized users.
Atrial fibrillation; Anticoagulant agents; Self-care; Cost an cost-analysis; Drug monitoring
When potentially associated with the likelihood of outcome, missing participant data represents a serious potential source of bias in randomized trials. Authors of systematic reviews frequently face this problem when conducting meta-analyses. The objective of this study is to conduct a systematic survey of the relevant literature to identify proposed approaches for how systematic review authors should handle missing participant data when conducting a meta-analysis.
We searched MEDLINE and the Cochrane Methodology register from inception to August 2014. We included papers that devoted at least two paragraphs to discuss a relevant approach for missing data. Five pairs of reviewers, working independently and in duplicate, selected relevant papers. One reviewer abstracted data from included papers and a second reviewer verified them. We summarized the results narratively.
Of 9,138 identified citations, we included 11 eligible papers. Four proposed general approaches for handling dichotomous outcomes, and all recommended a complete case analysis as the primary analysis and additional sensitivity analyses using the following imputation methods: based on reasons for missingness (n = 3), relative to risk among followed up (n = 3), best-case scenario (n = 2), and worst-case scenario (n = 3). Three of these approaches suggested taking uncertainty into account. Two papers proposed general approaches for handling continuous outcomes, and both proposed a complete case analysis as the reference analysis and the following imputation methods as sensitivity analyses: based on reasons for missingness (n = 2), based on the mean observed in the same trial or other trials (n = 1), and based on informative missingness differences in means (n = 1). The remaining eligible papers did not propose general approaches but addressed specific statistical issues.
All proposed approaches for handling missing participant data recommend conducting a complete case analysis for the primary analysis and some form of sensitivity analysis to evaluate robustness of results. Although these approaches require further testing, they may guide review authors in addressing missing participant data.
Electronic supplementary material
The online version of this article (doi:10.1186/s13643-015-0083-6) contains supplementary material, which is available to authorized users.
Missing participant data; Systematic reviews; Meta-analysis
Systematic reviews represent one of the most important tools for knowledge translation but users often struggle with understanding and interpreting their results. GRADE Summary-of-Findings tables have been developed to display results of systematic reviews in a concise and transparent manner. The current format of the Summary-of-Findings tables for presenting risks and quality of evidence improves understanding and assists users with finding key information from the systematic review. However, it has been suggested that additional methods to present risks and display results in the Summary-of-Findings tables are needed.
We will conduct a non-inferiority parallel-armed randomized controlled trial to determine whether an alternative format to present risks and display Summary-of-Findings tables is not inferior compared to the current standard format. We will measure participant understanding, accessibility of the information, satisfaction, and preference for both formats. We will invite systematic review users to participate (that is clinicians, guideline developers, and researchers). The data collection process will be undertaken using the online 'Survey Monkey' system. For the primary outcome understanding, non-inferiority of the alternative format (Table A) to the current standard format (Table C) of Summary-of-Findings tables will be claimed if the upper limit of a 1-sided 95% confidence interval (for the difference of proportion of participants answering correctly a given question) excluded a difference in favor of the current format of more than 10%.
This study represents an effort to provide systematic reviewers with additional options to display review results using Summary-of-Findings tables. In this way, review authors will have a variety of methods to present risks and more flexibility to choose the most appropriate table features to display (that is optional columns, risks expressions, complementary methods to display continuous outcomes, and so on).
NCT02022631 (21 December 2013)
Summary-of-Findings table; Systematic review; Knowledge translation; Evidence summary; GRADE approach
Decision aids can help shared decision making, but most have been hard to produce, onerous to update, and are not being used widely. Thomas Agoritsas and colleagues explore why and describe a new electronic model that holds promise of being more useful for clinicians and patients to use together at the point of care
There is no consensus on how authors conducting meta-analysis should deal with trial participants with missing outcome data. The objectives of this study are to assess in Cochrane and non-Cochrane systematic reviews: (1) which categories of trial participants the systematic review authors consider as having missing participant data (MPD), (2) how trialists reported on participants with missing outcome data in trials, (3) whether systematic reviewer authors actually dealt with MPD in their meta-analyses of dichotomous outcomes consistently with their reported methods, and (4) the impact of different methods of dealing with MPD on pooled effect estimates in meta-analyses of dichotomous outcomes.
We will conduct a methodological study of Cochrane and non-Cochrane systematic reviews. Eligible systematic reviews will include a group-level meta-analysis of a patient-important dichotomous efficacy outcome, with a statistically significant effect estimate. Teams of two reviewers will determine eligibility and subsequently extract information from each eligible systematic review in duplicate and independently, using standardized, pre-piloted forms. The teams will then use a similar process to extract information from the trials included in the meta-analyses of interest. We will assess first which categories of trial participants the systematic reviewers consider as having MPD. Second, we will assess how trialists reported on participants with missing outcome data in trials. Third, we will compare what systematic reviewers report having done, and what they actually did, in dealing with MPD in their meta-analysis. Fourth, we will conduct imputation studies to assess the effects of different methods of dealing with MPD on the pooled effect estimates of meta-analyses. We will specifically calculate for each method (1) the percentage of systematic reviews that lose statistical significance and (2) the mean change of effect estimates across systematic reviews.
The impact of different methods of dealing with MPD on pooled effect estimates will help judge the associated risk of bias in systematic reviews. Our findings will inform recommendations regarding what assumptions for MPD should be used to test the robustness of meta-analytical results.
Electronic supplementary material
The online version of this article (doi:10.1186/2046-4053-3-137) contains supplementary material, which is available to authorized users.
Missing participant data; Imputation; Risk of bias; Trials; Systematic reviews; Meta-analysis
Clinical guidelines should be updated to maintain their validity. Our aim was to estimate the length of time before recommendations become outdated.
We used a retrospective cohort design and included recommendations from clinical guidelines developed in the Spanish National Health System clinical guideline program since 2008. We performed a descriptive analysis of references, recommendations and resources used, and a survival analysis of recommendations using the Kaplan–Meier method.
We included 113 recommendations from 4 clinical guidelines with a median of 4 years since the most recent search (range 3.9–4.4 yr). We retrieved 39 136 references (range 3343–14 787) using an exhaustive literature search, 668 of which were related to the recommendations in our sample. We identified 69 (10.3%) key references, corresponding to 25 (22.1%) recommendations that required updating. Ninety-two percent (95% confidence interval 86.9–97.0) of the recommendations were valid 1 year after their development. This probability decreased at 2 (85.7%), 3 (81.3%) and 4 years (77.8%).
Recommendations quickly become outdated, with 1 out of 5 recommendations being out of date after 3 years. Waiting more than 3 years to review a guideline is potentially too long.
Dementia includes a group of neurodegenerative disorders characterized by progressive loss of cognitive function and a decrease in the ability to perform activities of daily living. Systematic reviews of diagnostic test accuracy (DTA) focus on how well the index test detects patients with the disease in terms of figures such as sensitivity and specificity. Although DTA reviews about dementia are essential, at present there is no information about their quantity and quality.
We searched for DTA reviews in MEDLINE (1966–2013), EMBASE (1980–2013), The Cochrane Library (from its inception until December 2013) and the Database of Abstracts of Reviews of Effects (DARE). Two reviewers independently assessed the methodological quality of the reviews using the AMSTAR measurement tool, and the quality of the reporting using the PRISMA checklist. We describe the main characteristics of these reviews, including basic characteristics, type of dementia, and diagnostic test evaluated, and we summarize the AMSTAR and PRISMA scores.
We selected 24 DTA systematic reviews. Only 10 reviews (41.6%), assessed the bias of included studies and few (33%) used this information to report the review results or to develop their conclusions Only one review (4%) reported all methodological items suggested by the PRISMA tool. Assessing methodology quality by means of the AMSTAR tool, we found that six DTA reviews (25%) pooled primary data with the aid of methods that are used for intervention reviews, such as Mantel-Haenszel and separate random-effects models (25%), while five reviews (20.8%) assessed publication bias by means of funnel plots and/or Egger’s Test.
Our assessment of these DTA reviews reveals that their quality, both in terms of methodology and reporting, is far from optimal. Assessing the quality of diagnostic evidence is fundamental to determining the validity of the operating characteristics of the index test and its usefulness in specific settings. The development of high quality DTA systematic reviews about dementia continues to be a challenge.
Electronic supplementary material
The online version of this article (doi:10.1186/s12883-014-0183-2) contains supplementary material, which is available to authorized users.
Diagnosis; Dementia; Alzheimer’s disease dementia; Systematic review; PRISMA checklist; AMSTAR tool
Updating is important to ensure clinical guideline (CG) recommendations remain valid. However, little research has been undertaken in this field. We assessed CGs produced by the National Institute for Health and Care Excellence (NICE) to identify and describe updated recommendations and to investigate potential factors associated with updating. Also, we evaluated the reporting and presentation of recommendation changes.
We performed a descriptive analysis of original and updated CGs and recommendations, and an assessment of presentation formats and methods for recording information. We conducted a case-control study, defining cases as original recommendations that were updated (‘new-replaced’ recommendations), and controls as original recommendations that were considered to remain valid (‘not changed’ recommendations). We performed a comparison of main characteristics between cases and controls, and we planned a multiple regression analysis to identify potential predictive factors for updating.
We included nine updated CGs (1,306 recommendations) and their corresponding original versions (1,106 recommendations). Updated CGs included 812 (62%) recommendations ‘not reviewed’, 368 (28.1%) ‘new’ recommendations, 104 (7.9%) ‘amended’ recommendations, and 25 (1.9%) recommendations reviewed but unchanged. The presentation formats used to indicate the changes in recommendations varied widely across CGs. Changes in ‘amended’, ‘deleted’, and ‘new-replaced’ recommendations (n = 296) were reported infrequently, mostly in appendices. These changes were recorded in 167 (56.4%) recommendations; and were explained in 81 (27.4%) recommendations. We retrieved a total of 7.1% (n = 78) case recommendations (‘new-replaced’) and 2.4% (n = 27) control recommendations (‘not changed’) in original CGs. The updates were mainly from ‘Fertility CG’, about ‘gynaecology, pregnancy and birth’ topic, and ‘treatment’ or ‘prevention’ purposes. We did not perform the multiple regression analysis as originally planned due to the small sample of recommendations retrieved.
Our study is the first to describe and assess updated CGs and recommendations from a national guideline program. Our results highlight the pressing need to standardise the reporting and presentation of updated recommendations and the research gap about the optimal way to present updates to guideline users. Furthermore, there is a need to investigate updating predictive factors.
Clinical practice guidelines; Information dissemination; Evidence-based medicine; Knowledge translation; Methods; Updating
Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item.
We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added.
We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items.
The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.
Clinical guidelines (CGs) are popular for healthcare decision making but their acceptability and use by healthcare providers is influenced by numerous factors. Some of these factors are professional-related, such as knowledge and perceptions of and attitudes toward CGs in general. The aim of our study was to evaluate attitudes and perceptions of Spanish physicians towards CGs.
We coordinated six discussion groups with a total of 46 physicians. The participants were drawn from 12 medical specialties from both specialized and primary care. We recorded the sessions and transcribed the content verbatim. We analyzed the data using an approach based on the grounded theory.
We identified two main constructs that defined the physicians' perceptions towards guidelines: knowledge and usefulness. “Knowledge” defined the theoretical meanings of guidelines, while “Usefulness” referred to the pragmatic approach to guidelines. These constructs were interrelated through a series of categories such as confidence, usability, accessibility, dissemination and formats.
In our study, the constructs that impacted most on physician's attitudes to clinical guidelines were knowledge and usefulness. The tension between the theoretical and the pragmatic constructs determined the attitudes and how physicians use guidelines. Groups developing guidelines should ask relevant clinical questions and develop implementable and context specific recommendations. Developers should be explicit and consistent in the development and presentation of recommendations.
Updating clinical practice guidelines (CPGs) is a crucial process for maintaining the validity of recommendations. Methodological handbooks should provide guidance on both developing and updating CPGs. However, little is known about the updating guidance provided by these handbooks.
We conducted a systematic review to identify and describe the updating guidance provided by CPG methodological handbooks and included handbooks that provide updating guidance for CPGs. We searched in the Guidelines International Network library, US National Guidelines Clearinghouse and MEDLINE (PubMed) from 1966 to September 2013. Two authors independently selected the handbooks and extracted the data. We used descriptive statistics to analyze the extracted data and conducted a narrative synthesis.
We included 35 handbooks. Most handbooks (97.1%) focus mainly on developing CPGs, including variable degrees of information about updating. Guidance on identifying new evidence and the methodology of assessing the need for an update is described in 11 (31.4%) and eight handbooks (22.8%), respectively. The period of time between two updates is described in 25 handbooks (71.4%), two to three years being the most frequent (40.0%). The majority of handbooks do not provide guidance for the literature search, evidence selection, assessment, synthesis, and external review of the updating process.
Guidance for updating CPGs is poorly described in methodological handbooks. This guidance should be more rigorous and explicit. This could lead to a more optimal updating process, and, ultimately to valid trustworthy guidelines.
Clinical practice guidelines; Evidence-based medicine; Handbooks; Methodology; Systematic review
Clinicians, providers and guideline panels use absolute effects to weigh the advantages and downsides of treatment alternatives. Relative measures have the potential to mislead readers. However, little is known about the reporting of absolute measures in systematic reviews. The objectives of our study are to determine the proportion of systematic reviews that report absolute measures of effect for the most important outcomes, and ascertain how they are analyzed, reported and interpreted.
We will conduct a methodological survey of systematic reviews published in 2010. We will conduct a 1:1 stratified random sampling of Cochrane vs. non-Cochrane systematic reviews. We will calculate the proportion of systematic reviews reporting at least one absolute estimate of effect for the most patient-important outcome for the comparison of interest. We will conduct multivariable logistic regression analyses with the reporting of an absolute estimate of effect as the dependent variable and pre-specified study characteristics as the independent variables. For systematic reviews reporting an absolute estimate of effect, we will document the methods used for the analysis, reporting and interpretation of the absolute estimate.
Our methodological survey will inform current practices regarding reporting of absolute estimates in systematic reviews. Our findings may influence recommendations on reporting, conduct and interpretation of absolute estimates. Our results are likely to be of interest to systematic review authors, funding agencies, clinicians, guideline developers and journal editors.
Systematic reviews; Meta-analysis; Statistical data; Evidence-based medicine; Numbers needed to treat; Data reporting; Absolute effect measures
Clinical practice guidelines (CPGs) become quickly outdated and require a periodic reassessment of evidence research to maintain their validity. However, there is little research about this topic. Our project will provide evidence for some of the most pressing questions in this field: 1) what is the average time for recommendations to become out of date?; 2) what is the comparative performance of two restricted search strategies to evaluate the need to update recommendations?; and 3) what is the feasibility of a more regular monitoring and updating strategy compared to usual practice?. In this protocol we will focus on questions one and two.
The CPG Development Programme of the Spanish Ministry of Health developed 14 CPGs between 2008 and 2009. We will stratify guidelines by topic and by publication year, and include one CPG by strata.
We will develop a strategy to assess the validity of CPG recommendations, which includes a baseline survey of clinical experts, an update of the original exhaustive literature searches, the identification of key references (reference that trigger a potential recommendation update), and the assessment of the potential changes in each recommendation.
We will run two alternative search strategies to efficiently identify important new evidence: 1) PLUS search based in McMaster Premium LiteratUre Service (PLUS) database; and 2) a Restrictive Search (ReSe) based on the least number of MeSH terms and free text words needed to locate all the references of each original recommendation.
We will perform a survival analysis of recommendations using the Kaplan-Meier method and we will use the log-rank test to analyse differences between survival curves according to the topic, the purpose, the strength of recommendations and the turnover. We will retrieve key references from the exhaustive search and evaluate their presence in the PLUS and ReSe search results.
Our project, using a highly structured and transparent methodology, will provide guidance of when recommendations are likely to be at risk of being out of date. We will also assess two novel restrictive search strategies which could reduce the workload without compromising rigour when CPGs developers check for the need of updating.
Clinical practice guidelines; Diffusion of innovation; Dissemination and implementation; Evidence-based medicine; Information storage and retrieval; Knowledge translation; Methodology; Updating
Respiratory tract infections are an important burden in primary care and it’s known that they are usually self-limited and that antibiotics only alter its course slightly. This together with the alarming increase of bacterial resistance due to increased use of antimicrobials calls for a need to consider strategies to reduce their use. One of these strategies is the delayed prescription of antibiotics.
Multicentric, parallel, randomised controlled trial comparing four antibiotic prescribing strategies in acute non-complicated respiratory tract infections. We will include acute pharyngitis, rhinosinusitis, acute bronchitis and acute exacerbation of chronic bronchitis or chronic obstructive pulmonary disease (mild to moderate). The therapeutic strategies compared are: immediate antibiotic treatment, no antibiotic treatment, and two delayed antibiotic prescribing (DAP) strategies with structured advice to use a course of antibiotics in case of worsening of symptoms or not improving (prescription given to patient or prescription left at the reception of the primary care centre 3 days after the first medical visit).
Delayed antibiotic prescription has been widely used in Anglo-Saxon countries, however, in Southern Europe there has been little research about this topic. The DAP trial wil evaluate two different delayed strategies in Spain for the main respiratory infections in primary care.
This trial is registered with ClinicalTrials.gov, number http://NCT01363531.
Delayed antibiotic prescription; Respiratory infections; Family medicine; Pharyngitis; Acute tonsillitis; Rhinosinusitis; Acute bronchitis; Chronic obstructive pulmonary disease
Systematic reviewer authors intending to include all randomized participants in their meta-analyses need to make assumptions about the outcomes of participants with missing data.
The objective of this paper is to provide systematic reviewer authors with a relatively simple guidance for addressing dichotomous data for participants excluded from analyses of randomized trials.
This guide is based on a review of the Cochrane handbook and published methodological research. The guide deals with participants excluded from the analysis who were considered ‘non-adherent to the protocol’ but for whom data are available, and participants with missing data.
Systematic reviewer authors should include data from ‘non-adherent’ participants excluded from the primary study authors' analysis but for whom data are available. For missing, unavailable participant data, authors may conduct a complete case analysis (excluding those with missing data) as the primary analysis. Alternatively, they may conduct a primary analysis that makes plausible assumptions about the outcomes of participants with missing data. When the primary analysis suggests important benefit, sensitivity meta-analyses using relatively extreme assumptions that may vary in plausibility can inform the extent to which risk of bias impacts the confidence in the results of the primary analysis. The more plausible assumptions draw on the outcome event rates within the trial or in all trials included in the meta-analysis. The proposed guide does not take into account the uncertainty associated with assumed events.
This guide proposes methods for handling participants excluded from analyses of randomized trials. These methods can help in establishing the extent to which risk of bias impacts meta-analysis results.
Healthcare decision makers face challenges when using guidelines, including understanding the quality of the evidence or the values and preferences upon which recommendations are made, which are often not clear.
GRADE is a systematic approach towards assessing the quality of evidence and the strength of recommendations in healthcare. GRADE also gives advice on how to go from evidence to decisions. It has been developed to address the weaknesses of other grading systems and is now widely used internationally. The Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE) consortium (http://www.decide-collaboration.eu/), which includes members of the GRADE Working Group and other partners, will explore methods to ensure effective communication of evidence-based recommendations targeted at key stakeholders: healthcare professionals, policymakers, and managers, as well as patients and the general public. Surveys and interviews with guideline producers and other stakeholders will explore how presentation of the evidence could be improved to better meet their information needs. We will collect further stakeholder input from advisory groups, via consultations and user testing; this will be done across a wide range of healthcare systems in Europe, North America, and other countries. Targeted communication strategies will be developed, evaluated in randomized trials, refined, and assessed during the development of real guidelines.
Results of the DECIDE project will improve the communication of evidence-based healthcare recommendations. Building on the work of the GRADE Working Group, DECIDE will develop and evaluate methods that address communication needs of guideline users. The project will produce strategies for communicating recommendations that have been rigorously evaluated in diverse settings, and it will support the transfer of research into practice in healthcare systems globally.
Guidelines; Recommendations; Communication; Presentation formats
Scientific knowledge is in constant change. The flow of new information requires a frequent re-evaluation of the available research results. Clinical practice guidelines (CPGs) are not exempted from this phenomenon and need to be kept updated to maintain the validity of their recommendations. The objective of our review is to systematically identify, describe and assess strategies for monitoring and updating CPGs.
Study design and setting
We conducted a systematic review of studies evaluating one or more methods of updating (with or without monitoring) CPGs or recommendations. We searched MEDLINE (PubMed) and The Cochrane Methodology Register (The Cochrane Library) from 1966 to June 2012. Additionally, we hand-searched reference lists of the included studies and the Guidelines International Network book of abstracts. If necessary, we contacted study authors to obtain additional information.
We included a total of eight studies. Four evaluated if CPGs were out of date, three updated CPGs, and one continuously monitored and updated CPGs. The most detailed reported phase of the process was the identification of new evidence. As opposed to studies updating guidelines, studies evaluating if CPGs were out of date applied restricted searches. Only one study compared a restricted versus an exhaustive search suggesting that a restricted search is sufficient to assess recommendations’ Validity. One study analyzed the survival time of CPGs and suggested that these should be reassessed every three years.
There is limited evidence about the optimal strategies for monitoring and updating clinical practice guidelines. A restricted search is likely to be sufficient to monitor new evidence and assess the need to update, however, more information is needed about the timing and type of search. Only the exhaustive search strategy has been assessed for the update of CPGs. The development and evaluation of more efficient strategies is needed to improve the timeliness and reduce the burden of maintaining the validity of CPGs.
Clinical practice guidelines; Diffusion of innovation; Evidence-based medicine; Information storage and retrieval; Methodology; Updating; Implementation science; Dissemination and implementation; Knowledge translation