Health care professionals worldwide attend courses and workshops to learn evidence-based medicine (EBM), but evidence regarding the impact of these educational interventions is conflicting and of low methodologic quality and lacks generalizability. Furthermore, little is known about determinants of success. We sought to measure the effect of EBM short courses and workshops on knowledge and to identify course and learner characteristics associated with knowledge acquisition.
Health care professionals with varying expertise in EBM participated in an international, multicentre before–after study. The intervention consisted of short courses and workshops on EBM offered in diverse settings, formats and intensities. The primary outcome measure was the score on the Berlin Questionnaire, a validated instrument measuring EBM knowledge that the participants completed before and after the course.
A total of 15 centres participated in the study and 420 learners from North America and Europe completed the study. The baseline score across courses was 7.49 points (range 3.97–10.42 points) out of a possible 15 points. The average increase in score was 1.40 points (95% confidence interval 0.48–2.31 points), which corresponded with an effect size of 0.44 standard deviation units. Greater improvement in scores was associated (in order of greatest to least magnitude) with active participation required of the learners, a separate statistics session, fewer topics, less teaching time, fewer learners per tutor, larger overall course size and smaller group size. Clinicians and learners involved in medical publishing improved their score more than other types of learners; administrators and public health professionals improved their score less. Learners who perceived themselves to have an advanced knowledge of EBM and had prior experience as an EBM tutor also showed greater improvement than those who did not.
EBM course organizers who wish to optimize knowledge gain should require learners to actively participate in the course and should consider focusing on a small number of topics, giving particular attention to statistical concepts.
THE GRADE WORKING GROUP IS DEVELOPING and evaluating a common, sensible approach to grading quality of evidence and strength of recommendations in health care. In this article, we discuss the advantages and disadvantages of using letters, numbers, symbols or words to represent grades of evidence and recommendations. Using multiple strategies, we searched for comparative studies of alternative ways of representing ordered categories in any context. In addition, we contacted experts and reviewed theoretical work and qualitative research on how best to communicate grades of any kind quickly and clearly. We were unable to identify health care research that addressed, either directly or indirectly, the best way to present grades of evidence and recommendations. We found examples of symbols used by government, commercial and consumer organizations to communicate quality of evidence or strength of recommendations, but no comparative studies. Although a number of grading systems are used in health care and other fields, there is little or no evidence of how well various presentations are understood. Before promoting the use of specific symbols, numbers, letters or words, the extent to which the intended message is comprehended should be evaluated.
Allergy; Atopy; Eczema; Lactobacillus; Prevention; Probiotic; Supplementation
The prevalence of allergic diseases in infants, whose parents and siblings do not have allergy, is approximately 10 % and reaches 20–30 % in those with an allergic first-degree relative. Intestinal microbiota may modulate immunologic and inflammatory systemic responses and, thus, influence development of sensitization and allergy. Prebiotics – non-digestible oligosaccharides that stimulate growth of probiotic bacteria – have been reported to modulate immune responses and their supplementation has been proposed as a preventive intervention.
The World Allergy Organization (WAO) convened a guideline panel to develop evidence-based recommendations about the use of prebiotics in the prevention of allergy.
The WAO guideline panel identified the most relevant clinical questions about the use of prebiotics for the prevention of allergy. We performed a systematic review of randomized controlled trials of prebiotics, and reviewed the evidence about patient values and preferences, and resource requirements (up to January 2015, with an update on July 29, 2015). We followed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to develop recommendations.
Based on GRADE evidence to decision frameworks, the WAO guideline panel suggests using prebiotic supplementation in not-exclusively breastfed infants and not using prebiotic supplementation in exclusively breastfed infants. Both recommendations are conditional and based on very low certainty of the evidence. We found no experimental or observational study of prebiotic supplementation in pregnant women or in breastfeeding mothers. Thus, the WAO guideline panel chose not to provide a recommendation about prebiotic supplementation in pregnancy or during breastfeeding, at this time.
WAO recommendations about prebiotic supplementation for the prevention of allergy are intended to support parents, clinicians and other health care professionals in their decisions whether or not to use prebiotics for the purpose of preventing allergies in healthy, term infants.
Electronic supplementary material
The online version of this article (doi:10.1186/s40413-016-0102-7) contains supplementary material, which is available to authorized users.
Evidence-based guidelines have undergone an incredible transformation over the last number of years. Significant advances include explicit linkages of systematic evidence summaries to the strength and direction of recommendations, consideration of all patient-important factors, transparent reporting of the recommendation generation process including conflict of interest management strategies and the production of clinical practice guidelines which use simple and clear language. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology provides a framework for guideline development and was employed to produce the recently published ATS/ERS/JRS/ALAT update on treatment for idiopathic pulmonary fibrosis (IPF). Herein we discuss the advantages of using an evidence-based approach for guideline development using the IPF process and resultant document as an example.
Evidence-based; Guideline; Idiopathic pulmonary fibrosis
Latent tuberculosis infection (LTBI) is characterised by the presence of immune
responses to previously acquired Mycobacterium tuberculosis
infection without clinical evidence of active tuberculosis (TB). Here we report
evidence-based guidelines from the World Health Organization for a public health
approach to the management of LTBI in high risk individuals in countries with high or
middle upper income and TB incidence of <100 per 100 000 per year. The
guidelines strongly recommend systematic testing and treatment of LTBI in people
living with HIV, adult and child contacts of pulmonary TB cases, patients initiating
anti-tumour necrosis factor treatment, patients receiving dialysis, patients
preparing for organ or haematological transplantation, and patients with silicosis.
In prisoners, healthcare workers, immigrants from high TB burden countries, homeless
persons and illicit drug users, systematic testing and treatment of LTBI is
conditionally recommended, according to TB epidemiology and resource availability.
Either commercial interferon-gamma release assays or Mantoux tuberculin skin testing
could be used to test for LTBI. Chest radiography should be performed before LTBI
treatment to rule out active TB disease. Recommended treatment regimens for LTBI
include: 6 or 9 month isoniazid; 12 week rifapentine plus isoniazid;
3–4 month isoniazid plus rifampicin; or 3–4 month
Guidelines on LTBI for low TB incidence countries – essential element of
the @WHO #EndTB strategy and TB elimination
Guideline panellists have differing opinions on whether resource use should influence decisions on individual patients. As medical care costs rise, resource use considerations become more compelling, but panellists may find dealing with such considerations challenging
The GRADE system classifies recommendations made in guidelines as either strong or weak. This article explores the meaning of these descriptions and their implications for patients, clinicians, and policy makers
Guideline developers use a bewildering variety of systems to rate the quality of the evidence underlying their recommendations. Some are facile, some confused, and others sophisticated but complex
To develop guidance on what information to include and how to present it in tables summarizing the evidence from systematic reviews of test accuracy following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.
To design and refine the evidence tables, we used an iterative process based on the analysis of data from four rounds of discussions, feedback and user testing. During the final round, we conducted one-on-one user testing with target end users. We presented a number of alternative formats of evidence tables to participants and obtained information about users’ understanding and preferences.
More than 150 users participated in initial discussions and provided their formal and informal feedback. 20 users completed one-on-one user testing interviews. Almost all participants preferred summarizing the results of systematic reviews of test accuracy in tabular format rather than plain text. Users generally preferred less complex tables but found presenting sensitivity and specificity estimates only as too simplistic. Users found the presentation of test accuracy for several values of prevalence initially confusing but modifying table layout and adding sample clinical scenarios for each prevalence reduced this confusion. Providing information about clinical consequences of testing result was viewed as not feasible for authors of systematic reviews.
We present the current formats for tables presenting test accuracy following the GRADE approach. These tables can be developed using GRADEpro guidelines development tool (www.guidelinedevelopment.org or www.gradepro.org) and are being further developed into electronic interactive tables that will suit the needs of different end users. The formatting of these tables, and how they influence result interpretation and decision-making will be further evaluated in a randomized trial.
Emerging health problems require rapid advice. We describe the development and pilot testing of a systematic, transparent approach used by the World Health Organization (WHO) to develop rapid advice guidelines in response to requests from member states confronted with uncertainty about the pharmacological management of avian influenza A (H5N1) virus infection. We first searched for systematic reviews of randomized trials of treatment and prevention of seasonal influenza and for non-trial evidence on H5N1 infection, including case reports and animal and in vitro studies. A panel of clinical experts, clinicians with experience in treating patients with H5N1, influenza researchers, and methodologists was convened for a two-day meeting. Panel members reviewed the evidence prior to the meeting and agreed on the process. It took one month to put together a team to prepare the evidence profiles (i.e., summaries of the evidence on important clinical and policy questions), and it took the team only five weeks to prepare and revise the evidence profiles and to prepare draft guidelines prior to the panel meeting. A draft manuscript for publication was prepared within 10 days following the panel meeting. Strengths of the process include its transparency and the short amount of time used to prepare these WHO guidelines. The process could be improved by shortening the time required to commission evidence profiles. Further development is needed to facilitate stakeholder involvement, and evaluate and ensure the guideline's usefulness.
The authors describe the development and pilot testing of a systematic, transparent approach to develop rapid advice guidelines on pharmacological management of avian influenza.
Several studies investigated the association of anemia with health related quality of life (HRQL) in patients with chronic disease. However, there is little evidence regarding the association of anemia with HRQL in patients with chronic obstructive pulmonary disease (COPD).
This is a post-hoc analysis of a study which enrolled a population of adults aged 35–79 randomly selected from residents of Erie and Niagara Counties, NY, between 1996 and 2000. In addition to demographic information and physical measurements, we obtained spirometry data and hemoglobin levels. We used modified Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria to define COPD, and World Health Organization (WHO) criteria to define anemia. To assess HRQL we used the Short Form-36 (SF-36) to assess physical functioning (PF), physical component summary (PCS) measures and mental component summary (MCS) measures.
In the entire study population (n = 2704), respondents with anemia had lower scores on the physical functioning domain [45.4 (SD10.9) vs. 49.2 (SD 9.1); p < 0.0001]. Among patients with COPD (n = 495) the PF scores (39.9 vs. 45.4) and the PCS (41.9 vs. 45.9) were significantly lower in individuals with anemia compared to those without. In multiple regression analysis, the association between hemoglobin and PCS was positive (regression coefficient 0.02, p = 0.003). There was no significant association of hemoglobin with PF scores or the mental component summary measure after adjusting for covariates in patients with COPD.
In patients with moderate to very severe COPD anemia may be associated with worse HRQL. However, co-morbidities may explain part or all of this association in these patients.
Prevalence of allergic diseases in infants, whose parents and siblings do not have allergy, is approximately 10% and reaches 20–30% in those with an allergic first-degree relative. Intestinal microbiota may modulate immunologic and inflammatory systemic responses and, thus, influence development of sensitization and allergy. Probiotics have been reported to modulate immune responses and their supplementation has been proposed as a preventive intervention.
The World Allergy Organization (WAO) convened a guideline panel to develop evidence-based recommendations about the use of probiotics in the prevention of allergy.
We identified the most relevant clinical questions and performed a systematic review of randomized controlled trials of probiotics for the prevention of allergy. We followed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to develop recommendations. We searched for and reviewed the evidence about health effects, patient values and preferences, and resource use (up to November 2014). We followed the GRADE evidence-to-decision framework to develop recommendations.
Currently available evidence does not indicate that probiotic supplementation reduces the risk of developing allergy in children. However, considering all critical outcomes in this context, the WAO guideline panel determined that there is a likely net benefit from using probiotics resulting primarily from prevention of eczema. The WAO guideline panel suggests: a) using probiotics in pregnant women at high risk for having an allergic child; b) using probiotics in women who breastfeed infants at high risk of developing allergy; and c) using probiotics in infants at high risk of developing allergy. All recommendations are conditional and supported by very low quality evidence.
WAO recommendations about probiotic supplementation for prevention of allergy are intended to support parents, clinicians and other health care professionals in their decisions whether to use probiotics in pregnancy and during breastfeeding, and whether to give them to infants.
Electronic supplementary material
The online version of this article (doi:10.1186/s40413-015-0055-2) contains supplementary material, which is available to authorized users.
Allergy; Prevention; Probiotics; Practice guidelines; GRADE
Chronic lung disease is exacerbated by comorbid psychiatric issues and treatment of depression may improve disease symptoms. We sought to add to the literature as to whether depression is associated with pulmonary function in healthy adults.
In 2,551 healthy adults from New York State, USA, we studied the association of depression via the Center for Epidemiologic Studies Depression scale (CES-D) score and forced expiratory volume (FEV1) and forced vital capacity (FVC) using general linear models and a cross sectional design.
We identified statistically significant inverse trends in FEV1, FVC, FEV1% and FVC% by CES-D category especially in ever smokers and men. When adjusted for covariates, the difference in FEV1 and FEV1% for smokers with >18.5 lifetime pack years from CES-D score 0-3 to ≥16 (depressed) is approximately 0.25 L and 5.0%; adjusted P for trend are <0.001 and 0.019, respectively. In men, we also observed statistically significant inverse trends in pulmonary function with increasing CES-D.
We identified an inverse association of depressive symptoms and pulmonary function in healthy adults especially in men and individuals with a heavy smoking history. Further studies of these associations are essential for the development and tailoring of interventions for the prevention and treatment of chronic lung disease.
pulmonary disease; chronic lung disease; depression; respiratory function tests
Faculty productivity is essential for academic medical centers striving to achieve excellence and national recognition. The objective of this study was to evaluate whether and how academic Departments of Medicine in the United States measure faculty productivity for the purpose of salary compensation.
We surveyed the Chairs of academic Departments of Medicine in the United States in 2012. We sent a paper-based questionnaire along with a personalized invitation letter by postal mail. For non-responders, we sent reminder letters, then called them and faxed them the questionnaire. The questionnaire included 8 questions with 23 tabulated close-ended items about the types of productivity measured (clinical, research, teaching, administrative) and the measurement strategies used. We conducted descriptive analyses.
Chairs of 78 of 152 eligible departments responded to the survey (51% response rate). Overall, 82% of respondents reported measuring at least one type of faculty productivity for the purpose of salary compensation. Amongst those measuring faculty productivity, types measured were: clinical (98%), research (61%), teaching (62%), and administrative (64%). Percentages of respondents who reported the use of standardized measurements units (e.g., Relative Value Units (RVUs)) varied from 17% for administrative productivity to 95% for research productivity. Departments reported a wide variation of what exact activities are measured and how they are monetarily compensated. Most compensation plans take into account academic rank (77%). The majority of compensation plans are in the form of a bonus on top of a fixed salary (66%) and/or an adjustment of salary based on previous period productivity (55%).
Our survey suggests that most academic Departments of Medicine in the United States measure faculty productivity and convert it into standardized units for the purpose of salary compensation. The exact activities that are measured and how they are monetarily compensated varied substantially across departments.
Electronic supplementary material
The online version of this article (doi:10.1186/1472-6920-14-205) contains supplementary material, which is available to authorized users.
Faculty productivity; Salary compensation; Academia; Department of medicine; Survey
Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item.
We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added.
We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items.
The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.
Although even randomization (that is, approximately 1:1 randomization ratio in study arms) provides the greatest statistical power, designed uneven randomization (DUR), (for example, 1:2 or 1:3) is used to increase participation rates. Until now, no convincing data exists addressing the impact of DUR on participation rates in trials. The objective of this study is to evaluate the epidemiology and to explore factors associated with DUR.
We will search for reports of RCTs published within two years in 25 general medical journals with the highest impact factor according to the Journal Citation Report (JCR)-2010. Teams of two reviewers will determine eligibility and extract relevant information from eligible RCTs in duplicate and using standardized forms. We will report the prevalence of DUR trials, the reported reasons for using DUR, and perform a linear regression analysis to estimate the association between the randomization ratio and the associated factors, including participation rate, type of informed consent, clinical area, and so on.
A clearer understanding of RCTs with DUR and its association with factors in trials, for example, participation rate, can optimize trial design and may have important implications for both researchers and users of the medical literature.
Participation rate; Designed uneven randomization trials; Trial participation
Systematic reviews and meta-analyses of randomized trials that include patient-reported outcomes (PROs) often provide crucial information for patients, clinicians and policy-makers facing challenging health care decisions. Based on emerging methods, guidance on improving the interpretability of meta-analysis of patient-reported outcomes, typically continuous in nature, is likely to enhance decision-making. The objective of this paper is to summarize approaches to enhancing the interpretability of pooled estimates of PROs in meta-analyses. When differences in PROs between groups are statistically significant, decision-makers must be able to interpret the magnitude of effect. This is challenging when, as is often the case, clinical trial investigators use different measurement instruments for the same construct within and between individual randomized trials. For such cases, in addition to pooling results as a standardized mean difference, we recommend that systematic review authors use other methods to present results such as relative (relative risk, odds ratio) or absolute (risk difference) dichotomized treatment effects, complimented by presentation in either: natural units (e.g. overall depression reduced by 2.4 points when measured on a 50-point Hamilton Rating Scale for Depression); minimal important difference units (e.g. where 1.0 unit represents the smallest difference in depression that patients, on average, perceive as important the depression score was 0.38 (95% CI 0.30 to 0.47) units less than the control group); or a ratio of means (e.g. where the mean in the treatment group is divided by the mean in the control group, the ratio of means is 1.27, representing a 27% relative reduction in the mean depression score).
Clinicians, providers and guideline panels use absolute effects to weigh the advantages and downsides of treatment alternatives. Relative measures have the potential to mislead readers. However, little is known about the reporting of absolute measures in systematic reviews. The objectives of our study are to determine the proportion of systematic reviews that report absolute measures of effect for the most important outcomes, and ascertain how they are analyzed, reported and interpreted.
We will conduct a methodological survey of systematic reviews published in 2010. We will conduct a 1:1 stratified random sampling of Cochrane vs. non-Cochrane systematic reviews. We will calculate the proportion of systematic reviews reporting at least one absolute estimate of effect for the most patient-important outcome for the comparison of interest. We will conduct multivariable logistic regression analyses with the reporting of an absolute estimate of effect as the dependent variable and pre-specified study characteristics as the independent variables. For systematic reviews reporting an absolute estimate of effect, we will document the methods used for the analysis, reporting and interpretation of the absolute estimate.
Our methodological survey will inform current practices regarding reporting of absolute estimates in systematic reviews. Our findings may influence recommendations on reporting, conduct and interpretation of absolute estimates. Our results are likely to be of interest to systematic review authors, funding agencies, clinicians, guideline developers and journal editors.
Systematic reviews; Meta-analysis; Statistical data; Evidence-based medicine; Numbers needed to treat; Data reporting; Absolute effect measures
To inform clinical guidelines and patient care we need high quality evidence on the relative benefits and harms of intervention. Patient reported outcome (PRO) data from clinical trials can “empower patients to make decisions based on their values” and “level the playing field between physician and patient”. While clinicians have a good understanding of the concept of health-related quality of life and other PROs, evidence suggests that many do not feel comfortable in using the data from trials to inform discussions with patients and clinical practice. This may in part reflect concerns over the integrity of the data and difficulties in interpreting the results arising from poor reporting.
The new CONSORT PRO extension aims to improve the reporting of PROs in trials to facilitate the use of results to inform clinical practice and health policy. While the CONSORT PRO extension is an important first step in the process, we need broader engagement with the guidance to facilitate optimal reporting and maximize use of PRO data in a clinical setting. Endorsement by journal editors, authors and peer reviewers are crucial steps. Improved design, implementation and transparent reporting of PROs in clinical trials are necessary to provide high quality evidence to inform evidence synthesis and clinical practice guidelines.
Quality of life; CONSORT PRO; Reporting; Clinical trials