A peer-reviewed journal would not survive without the generous time and insightful comments of the reviewers, whose efforts often go unrecognized. Although final decisions are always editorial, they are greatly facilitated by the deeper technical knowledge, scientific insights, understanding of social consequences, and passion that reviewers bring to our deliberations. For these reasons, the Editors-in-Chief and staff of the journal warmly thank the 219 reviewers whose comments helped to shape Systematic Reviews, for their invaluable assistance with review of manuscripts for the journal in Volume 3 (2014).
The quality of reporting clinical and preclinical research is not optimal. Reporting guidelines can help make reports of research more complete and transparent, thus increasing their value and making them more useful to all readers. Getting reporting guidelines into practice is complex and expensive, and involves several stakeholders, including prospective authors, peer reviewers, journal editors, guideline developers, and implementation scientists. Working together will help ensure their maximum uptake and penetration. We are all responsible for helping to ensure that all research is reported so completely that it is of value to everybody.
Please see related article: http://dx.doi.org/10.1186/s12916-015-0266-y
Implementation; Preclinical research; Quality of reporting; Reporting guidelines
Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review; few reviews report whether a protocol exists. Detailed, well-described protocols can facilitate the understanding and appraisal of the review methods, as well as the detection of modifications to methods and selective reporting in completed reviews. We describe the development of a reporting guideline, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015). PRISMA-P consists of a 17-item checklist intended to facilitate the preparation and reporting of a robust protocol for the systematic review. Funders and those commissioning reviews might consider mandating the use of the checklist to facilitate the submission of relevant protocol information in funding applications. Similarly, peer reviewers and editors can use the guidance to gauge the completeness and transparency of a systematic review protocol submitted for publication in a journal or other medium.
To evaluate the effectiveness and safety of dipeptidyl peptidase-4 (DPP-4) inhibitors versus intermediate-acting insulin for adults with type 2 diabetes mellitus (T2DM) and poor glycaemic control despite treatment with two oral agents.
Studies were multicentre and multinational.
Ten studies including 2967 patients with T2DM.
Studies that examined DPP-4 inhibitors compared with each other, intermediate-acting insulin, no treatment or placebo in patients with T2DM.
Primary and secondary outcome measures
Primary outcome was glycosylated haemoglobin (HbA1c). Secondary outcomes were healthcare utilisation, body weight, fractures, quality of life, microvascular complications, macrovascular complications, all-cause mortality, harms, cost and cost-effectiveness.
10 randomised clinical trials with 2967 patients were included after screening 5831 titles and abstracts, and 180 full-text articles. DPP-4 inhibitors significantly reduced HbA1c versus placebo in network meta-analysis (NMA; mean difference (MD) −0.62%, 95% CI −0.93% to −0.33%) and meta-analysis (MD −0.61%, 95% CI −0.81% to −0.41%), respectively. Significant differences in HbA1c were not observed for neutral protamine Hagedorn (NPH) insulin versus placebo and DPP-4 inhibitors versus NPH insulin in NMA. In meta-analysis, no significant differences were observed between DPP-4 inhibitors and placebo for severe hypoglycaemia, weight gain, cardiovascular disease, overall harms, treatment-related harms and mortality, although patients receiving DPP-4 inhibitors experienced less infections (relative risk 0.72, 95% CI 0.57 to 0.91).
DPP-4 inhibitors were superior to placebo in reducing HbA1c levels in adults with T2DM taking at least two oral agents. Compared with placebo, no safety signals were detected with DPP-4 inhibitors and there was a reduced risk of infection. There was no significant difference in HbA1c observed between NPH and placebo or NPH and DPP-4 inhibitors.
Trial registration number
PROSPERO # CRD42013003624.
systematic review; meta-analysis
Transparently documenting who and why a person is an author of a clinical trial report, as with any other article, is important since it allows research team members appropriate recognition and also likely helps reduce or avoid problems such as ghost authorship. Marušić and colleagues have previously proposed a five-step framework for attributing authorship of pharmaceutical company-sponsored clinical trials. The process is short, easily implemented, and can be used in addition to the authorship guidance provided by the International Committee of Medical Journal Editors. For the framework to gain optimal traction it is important that a strong implementation plan is developed and carried out across a broad spectrum of stakeholders. Authorship brings with it important responsibilities; authors must ensure that articles baring their names must be fit for purpose. This will help guarantee an increased value for published reports of clinical trials.
Please see related article: http://www.biomedcentral.com/1741-7015/12/197
Authorship; Transparency; Responsibility; Quality
Chronic diseases, such as cardiovascular disease and type 2 diabetes, impose significant burden to public health. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, blood glucose, and lipids, physical inactivity, excessive sedentary behaviours, overweight and obesity, and tobacco usage. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in adults is associated with adverse health outcomes and what the potential moderating factors are.
Of interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case-control studies, and nested case-control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases and grey literature sources will be searched. The processes for selecting studies, abstracting data, and resolving conflicts are described. We will assess risk of bias using design-specific tools. To determine sets of confounding variables that should be adjusted for, we have developed causal directed acyclic graphs and will use those to inform our risk of bias assessments. Meta-analysis will be conducted where appropriate; parameters for exploring statistical heterogeneity and effect modifiers are pre-specified. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used for determining the quality of evidence for outcomes.
Systematic review registration
Sugar-sweetened beverage; Adult; Chronic disease; Obesity; Cardiovascular disease; Metabolic syndrome; Type 2 diabetes; Cancer; Systematic review
Cardiovascular disease and type 2 diabetes are examples of chronic diseases that impose significant morbidity and mortality in the general population worldwide. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, high blood glucose or glucose intolerance, high lipid levels, physical inactivity, excessive sedentary behaviours, and overweight/obesity. The occurrence of intermediate outcomes during childhood increases the risk of disease in adulthood. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in children is associated with adverse health outcomes and what the potential moderating factors are.
Of interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case–control studies, and nested case–control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases and grey literature sources will be searched. The processes for selecting studies, abstracting data, and resolving conflicts are described. We will assess risk of bias using design-specific tools. To determine sets of confounding variables that should be adjusted for, we have developed causal directed acyclic graphs and will use those to inform our risk of bias assessments. Meta-analysis will be conducted where appropriate; parameters for exploring statistical heterogeneity and effect modifiers are pre-specified. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used to determine the quality of evidence for outcomes.
Systematic review registration
Sugar-sweetened beverage; Paediatric; Child; Chronic disease; Academic achievement; Dental caries; Obesity; Type 2 diabetes; Systematic review
A case report is a narrative that describes, for medical, scientific, or educational purposes, a medical problem experienced by one or more patients. Case reports written without guidance from reporting standards are insufficiently rigorous to guide clinical practice or to inform clinical study design.
Develop, disseminate, and implement systematic reporting guidelines for case reports.
We used a three-phase consensus process consisting of (1) premeeting literature review and interviews to generate items for the reporting guidelines, (2) a face-to-face consensus meeting to draft the reporting guidelines, and (3) postmeeting feedback, review, and pilot testing, followed by finalization of the case report guidelines.
This consensus process involved 27 participants and resulted in a 13-item checklist—a reporting guideline for case reports. The primary items of the checklist are title, key words, abstract, introduction, patient information, clinical findings, timeline, diagnostic assessment, therapeutic interventions, follow-up and outcomes, discussion, patient perspective, and informed consent.
We believe the implementation of the CARE (CAse REport) guidelines by medical journals will improve the completeness and transparency of published case reports and that the systematic aggregation of information from case reports will inform clinical study design, provide early signals of effectiveness and harms, and improve healthcare delivery.
Case report; case study; EQUATOR Network; patient reports; meaningful use; health research reporting guidelines
Apply and compare two methods that identify signals for the need to update systematic reviews, using three Evidence-based Practice Center reports on omega-3 fatty acids as test cases.
Study Design and Setting
We applied the RAND method, which uses domain (subject matter) expert guidance, and a modified Ottawa method, which uses quantitative and qualitative signals. For both methods, we conducted focused electronic literature searches of recent studies using the key terms from the original reports. We assessed the agreement between the methods and qualitatively assessed the merits of each system.
Agreement between the two methods was “substantial” or better (kappa > 0.62) in three of the four systematic reviews. Overall agreement between the methods was “substantial” (kappa = 0.64, 95% confidence interval [CI] 0.45–0.83).
The RAND and modified Ottawa methods appear to provide similar signals for the possible need to update systematic reviews in this pilot study. Future evaluation with a broader range of clinical topics and eventual comparisons between signals to update reports and the results of full evidence review updates will be needed. We propose a hybrid approach combining the best features of both methods, which should allow efficient review and assessment of the need to update.
Evidence-based methodology; Systematic reviews; Comparative effectiveness reviews; Omega-3 fatty acids; Cardiovascular disease risk factors; Cancer; Cognitive function
To improve quality of care and patient outcomes, health system decision-makers need to identify and implement effective interventions. An increasing number of systematic reviews document the effects of quality improvement programs to assist decision-makers in developing new initiatives. However, limitations in the reporting of primary studies and current meta-analysis methods (including approaches for exploring heterogeneity) reduce the utility of existing syntheses for health system decision-makers. This study will explore the role of innovative meta-analysis approaches and the added value of enriched and updated data for increasing the utility of systematic reviews of complex interventions.
We will use the dataset from our recent systematic review of 142 randomized trials of diabetes quality improvement programs to evaluate novel approaches for exploring heterogeneity. These will include exploratory methods, such as multivariate meta-regression analyses and all-subsets combinatorial meta-analysis. We will then update our systematic review to include new trials and enrich the dataset by surveying authors of all included trials. In doing so, we will explore the impact of variables not, reported in previous publications, such as details of study context, on the effectiveness of the intervention. We will use innovative analytical methods on the enriched and updated dataset to identify key success factors in the implementation of quality improvement interventions for diabetes. Decision-makers will be involved throughout to help identify and prioritize variables to be explored and to aid in the interpretation and dissemination of results.
This study will inform future systematic reviews of complex interventions and describe the value of enriching and updating data for exploring heterogeneity in meta-analysis. It will also result in an updated comprehensive systematic review of diabetes quality improvement interventions that will be useful to health system decision-makers in developing interventions to improve outcomes for people with diabetes.
Systematic review registration
PROSPERO registration no. CRD42013005165
Diabetes care; Knowledge translation; Quality improvement interventions; Complex Interventions; Health system decision-makers; Systematic review; Meta-analysis; Implementation science; Heterogeneity; Hierarchical modeling
Undertaking a Delphi exercise is recommended during the second stage in the development process for a reporting guideline. To continue the development for the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) a Delphi survey was undertaken to determine the consensus opinion of researchers, journal editors and educators in evidence-based practice (EBP) regarding the information items that should be reported when describing an educational intervention for EBP.
A four round online Delphi survey was conducted from October 2012 to March 2013. The Delphi panel comprised international researchers, educators and journal editors in EBP. Commencing with an open-ended question, participants were invited to volunteer information considered important when reporting educational interventions for EBP. Over three subsequent rounds participants were invited to rate the importance of each of the Delphi items using an 11 point Likert rating scale (low 0 to 4, moderate 5 to 6, high 7 to 8 and very high >8). Consensus agreement was set a priori as at least 80 per cent participant agreement. Consensus agreement was initially calculated within the four categories of importance (low to very high), prior to these four categories being merged into two (<7 and ≥7). Descriptive statistics for each item were computed including the mean Likert scores, standard deviation (SD), range and median participant scores. Mean absolute deviation from the median (MAD-M) was also calculated as a measure of participant disagreement.
Thirty-six experts agreed to participate and 27 (79%) participants completed all four rounds. A total of 76 information items were generated across the four survey rounds. Thirty-nine items (51%) were specific to describing the intervention (as opposed to other elements of study design) and consensus agreement was achieved for two of these items (5%). When the four rating categories were merged into two (<7 and ≥7), 18 intervention items achieved consensus agreement.
This Delphi survey has identified 39 items for describing an educational intervention for EBP. These Delphi intervention items will provide the groundwork for the subsequent consensus discussion to determine the final inclusion of items in the GREET, the first reporting guideline for educational interventions in EBP.
Evidence-based practice; Reporting guideline; Delphi survey
The aim of this systematic review was to identify which information is included when reporting educational interventions used to facilitate foundational skills and knowledge of evidence-based practice (EBP) training for health professionals. This systematic review comprised the first stage in the three stage development process for a reporting guideline for educational interventions for EBP.
The review question was ‘What information has been reported when describing educational interventions targeting foundational evidence-based practice knowledge and skills?’
MEDLINE, Academic Search Premier, ERIC, CINAHL, Scopus, Embase, Informit health, Cochrane Library and Web of Science databases were searched from inception until October - December 2011. Randomised and non-randomised controlled trials reporting original data on educational interventions specific to developing foundational knowledge and skills of evidence-based practice were included.
Studies were not appraised for methodological bias, however, reporting frequency and item commonality were compared between a random selection of studies included in the systematic review and a random selection of studies excluded as they were not controlled trials. Twenty-five data items were extracted by two independent reviewers (consistency > 90%).
Sixty-one studies met the inclusion criteria (n = 29 randomised, n = 32 non-randomised). The most consistently reported items were the learner’s stage of training, professional discipline and the evaluation methods used (100%). The least consistently reported items were the instructor(s) previous teaching experience (n = 8, 13%), and student effort outside face to face contact (n = 1, 2%).
This systematic review demonstrates inconsistencies in describing educational interventions for EBP in randomised and non-randomised trials. To enable educational interventions to be replicable and comparable, improvements in the reporting for educational interventions for EBP are required. In the absence of a specific reporting guideline, there are a range of items which are reported with variable frequency. Identifying the important items for describing educational interventions for facilitating foundational knowledge and skills in EBP remains to be determined. The findings of this systematic review will be used to inform the next stage in the development of a reporting guideline for educational interventions for EBP.
Systematic review; Evidence-based practice; Educational intervention; Reporting guidelines
Social and psychological interventions are often complex. Understanding randomized
controlled trials (RCTs) of these complex interventions requires a detailed description of
the interventions tested and the methods used to evaluate them; however, RCT reports often
omit, or inadequately report, this information. Incomplete and inaccurate reporting
hinders the optimal use of research, wastes resources, and fails to meet ethical
obligations to research participants and consumers. In this article, we explain how
reporting guidelines have improved the quality of reports in medicine and describe the
ongoing development of a new reporting guideline for RCTs: Consolidated Standards of
Reporting Trials-SPI (an extension for social and psychological interventions). We invite
readers to participate in the project by visiting our website, in order to help us reach
the best-informed consensus on these guidelines (http://tinyurl.com/CONSORT-study).
randomized controlled trial; RCT; CONSORT-SPI; reporting guideline; reporting standards
Epilepsy affects about 1% of the general population. Anti-epileptic drugs (AEDs) prevent or terminate seizures in individuals with epilepsy. Pregnant women with epilepsy may continue taking AEDs. Many of these agents cross the placenta and increase the risk of major congenital malformations, early cognitive and developmental delays, and infant mortality. We aim to evaluate the comparative safety of AEDs approved for chronic use in Canada when administered to pregnant and breastfeeding women and the effects on their infants and children through a systematic review and network meta-analysis.
Studies examining the effects of AEDs administered to pregnant and breastfeeding women regardless of indication (e.g., epilepsy, migraine, pain, psychiatric disorders) on their infants and children will be included. We will include randomized clinical trials (RCTs), quasi-RCTs, non-RCTs, controlled before-after, interrupted time series, cohort, registry, and case-control studies. The main literature search will be executed in MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials. We will seek unpublished literature through searches of trial protocol registries and conference abstracts. The literature search results screening, data abstraction, and risk of bias appraisal will be performed by two individuals, independently. Conflicts will be resolved through discussion. The risk of bias of experimental and quasi-experimental studies will be appraised using the Cochrane Effective Practice and Organization of Care Risk-of-Bias tool, methodological quality of observational studies will be appraised using the Newcastle-Ottawa Scale, and quality of reporting of safety outcomes will be conducted using the McMaster Quality Assessment Scale of Harms (McHarm) tool. If feasible and appropriate, we will conduct random effects meta-analysis. Network meta-analysis will be considered for outcomes that fulfill network meta-analysis assumptions.
The primary outcome is major congenital malformations (overall and by specific types), while secondary outcomes include fetal loss/miscarriage, minor congenital malformations (overall and by specific types), cognitive development, psychomotor development, small for gestational age, preterm delivery, and neonatal seizures.
Our systematic review will address safety concerns regarding the use of AEDs during pregnancy and breastfeeding. Our results will be useful to healthcare providers, policy-makers, and women of childbearing age who are taking anti-epileptic medications.
Systematic review registration
Anti-epileptic drug; Breastfeeding; Comparative safety; Congenital malformation; Epilepsy; Fetus; Infant; Network meta-analysis; Pregnancy; Systematic review
Antiretroviral therapy reduces mother-to-child transmission of human immunodeficiency virus (HIV) during pregnancy, delivery, and breastfeeding. However, these agents have been associated with preterm birth, anemia and low birth weight. We aim to evaluate the comparative safety and effectiveness of the use of antiretroviral drugs among HIV-infected women and the effects on their infants and children through a systematic review and network meta-analysis.
Studies examining the effects of six antiretroviral drug classes (nucleoside reverse transcriptase inhibitors, non-nucleoside reverse transcriptase inhibitors, protease inhibitors, integrase inhibitors, fusion inhibitors, co-receptor inhibitors) administered to HIV-infected pregnant women will be included. We will include randomized clinical trials (RCTs), quasi-RCTs, non-RCTs, controlled before-after, interrupted time series, cohort, registry, and case–control studies. No limitations will be imposed on publication status (that is, unpublished studies are eligible for inclusion), duration of follow-up, study conduct period, and language of dissemination. Comprehensive literature searches will be conducted in major electronic databases, including MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials. Gray literature will be identified through searching dissertation databases, trial protocol registries, and conference abstracts.
Two team members will independently screen all citations, full-text articles, and abstract data; conflicts will be resolved through discussion. The risk of bias and methodological quality will be appraised using appropriate tools (for example, Cochrane Collaboration’s tool for assessing risk of bias, Newcastle-Ottawa Scale, and McMaster Quality Assessment Scale of Harms). If feasible and appropriate, we will conduct random effects meta-analysis. Network meta-analysis will be considered for outcomes with the greatest number of treatment comparisons available that fulfill network meta-analysis assumptions (for example, consistency of evidence between direct and indirect data, and low statistical heterogeneity between included studies).
The primary effectiveness outcome is mother-to-child transmission of HIV, and the primary safety outcome is major congenital malformation (overall and specific types) among newborns of HIV-infected women. Secondary safety outcomes include stillbirths, infant/child death, preterm delivery, overall and specific minor congenital malformations, and small for gestational age infants.
Our systematic review will be of utility to healthcare providers, policy-makers, and HIV-positive women regarding the use of antiretroviral drugs.
PROSPERO registry number: CRD42014009071.
antiretroviral therapy; breastfeeding; congenital malformation; human immunodeficiency virus; fetus; mother-to-child-transmission; pregnancy
The purpose of this systematic review is to assess the effectiveness of brief interventions (BIs) as part of the Screening, Brief Intervention, and Referral to Treatment (SBIRT) model for reducing the nonmedical use of psychoactive substances.
Bibliographic databases (including MEDLINE, Embase, The Cochrane Library, CINAHL, and PsycINFO to April 2012) and gray literature sources were searched. We included randomized controlled trials that opportunistically screened adolescents or adults and then provided a one-to-one, verbal BI to those at risk of substance-use harm. Of interest was the nonmedical use of psychoactive substances (for example, drugs prohibited by international law), excluding alcohol, nicotine, and caffeine. Interventions comprised four or fewer sessions and were compared with no/delayed intervention or provision of information only. Studies were assessed for bias using the Cochrane risk of bias tool. Results were synthesized narratively. Evidence was interpreted according to the GRADE framework.
We identified 8,836 records. Of these, five studies met our inclusion criteria. Two studies compared BI with no BI, and three studies compared BI with information only. Studies varied in characteristics such as substances targeted, screening procedures, and BI administered. Outcomes were mostly reported by a single study, leading to limited or uncertain confidence in effect estimates.
Insufficient evidence exists as to whether BIs, as part of SBIRT, are effective or ineffective for reducing the use of, or harms associated with nonmedical use of, psychoactive substances when these interventions are administered to nontreatment-seeking, screen-detected populations. Updating this review with emerging evidence will be important.
brief intervention; drug use; psychoactive substance; SBIRT; screening; substance use; systematic review
Acute respiratory distress syndrome (ARDS) in humans is caused by an unchecked proinflammatory response that results in diffuse and severe lung injury, and it is associated with a mortality rate of 35 to 45%. Mesenchymal stromal cells (MSCs; ‘adult stem cells’) could represent a promising new therapy for this syndrome, since preclinical evidence suggests that MSCs may ameliorate lung injury. Prior to a human clinical trial, our aim is to conduct a systematic review to compare the efficacy and safety of MSC therapy versus controls in preclinical models of acute lung injury that mimic some aspects of the human ARDS.
We will include comparative preclinical studies (randomized and non-randomized) of acute lung injury in which MSCs were administered and outcomes compared to animals given a vehicle control. The primary outcome will be death. Secondary outcomes will include the four key features of preclinical acute lung injury as defined by the American Thoracic Society consensus conference (histologic evidence of lung injury, altered alveolar capillary barrier, lung inflammatory response, and physiological dysfunction) and pathogen clearance for acute lung injury models that are caused by infection. Electronic searches of MEDLINE, Embase, BIOSIS Previews, and Web of Science will be constructed and reviewed by the Peer Review of Electronic Search Strategies (PRESS) process. Search results will be screened independently and in duplicate. Data from eligible studies will be extracted, pooled, and analyzed using random effects models. Risk of bias will be assessed using the Cochrane risk of bias tool, and individual study reporting will be assessed according to the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines.
The results of this systematic review will comprehensively summarize the safety and efficacy of MSC therapy in preclinical models of acute lung injury. Our results will help translational scientists and clinical trialists to determine whether sufficient evidence exists to perform a human clinical trial. These results may also guide future acute lung injury preclinical and clinical research.
Mesenchymal stromal cells; Mesenchymal stem cells; Acute lung injury; Acute respiratory distress syndrome; Preclinical; Systematic review protocol
In 2008, the Cochrane Collaboration introduced a tool for assessing the risk of bias in clinical trials included in Cochrane reviews. The risk of bias (RoB) tool is based on narrative descriptions of evidence-based methodological features known to increase the risk of bias in trials.
To assess the usability of this tool, we conducted an evaluation by means of focus groups, online surveys and a face-to-face meeting. We obtained feedback from a range of stakeholders within The Cochrane Collaboration regarding their experiences with, and perceptions of, the RoB tool and associated guidance materials. We then assessed this feedback in a face-to-face meeting of experts and stakeholders and made recommendations for improvements and further developments of the RoB tool.
The survey attracted 380 responses. Respondents reported taking an average of between 10 and 60 minutes per study to complete their RoB assessments, which 83% deemed acceptable. Most respondents (87% of authors and 95% of editorial staff) thought RoB assessments were an improvement over past approaches to trial quality assessment. Most authors liked the standardized approach (81%) and the ability to provide quotes to support judgements (74%). A third of participants disliked the increased workload and found the wording describing RoB judgements confusing. The RoB domains reported to be the most difficult to assess were incomplete outcome data and selective reporting of outcomes. Authors expressed the need for more guidance on how to incorporate RoB assessments into meta-analyses and review conclusions. Based on this evaluation, recommendations were made for improvements to the RoB tool and the associated guidance. The implementation of these recommendations is currently underway.
Overall, respondents identified positive experiences and perceptions of the RoB tool. Revisions of the tool and associated guidance made in response to this evaluation, and improved provision of training, may improve implementation.
Survey; Focus groups; Bias assessment; Quality assessment; Systematic reviews
Some have suggested the quality of reporting of network meta-analyses (a technique used to synthesize information to compare multiple interventions) is sub-optimal. We sought to review information addressing this claim.
To conduct an overview of existing evaluations of quality of reporting in network meta-analyses and indirect treatment comparisons, and to compile a list of topics which may require detailed reporting guidance to enhance future reporting quality.
An electronic search of Medline and the Cochrane Registry of methodologic studies (January 2004–August 2013) was performed by an information specialist. Studies describing findings from quality of reporting assessments were sought. Screening of abstracts and full texts was performed by two team members. Descriptors related to all aspects of reporting a network meta-analysis were summarized.
We included eight reports exploring the quality of reporting of network meta-analyses. From past reviews, authors found several aspects of network meta-analyses were inadequately reported, including primary information about literature searching, study selection, and risk of bias evaluations; statement of the underlying assumptions for network meta-analysis, as well as efforts to verify their validity; details of statistical models used for analyses (including information for both Bayesian and Frequentist approaches); completeness of reporting of findings; and approaches for summarizing probability measures as additional important considerations.
While few studies were identified, several deficiencies in the current reporting of network meta-analyses were observed. These findings reinforce the need to develop reporting guidance for network meta-analyses. Findings from this review will be used to guide next steps in the development of reporting guidance for network meta-analysis in the format of an extension of the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) Statement.
Patients undergoing renal transplant procedures require multi-agent immunosuppressive regimens both short term (induction phase) and long term (maintenance phase) to minimize the risk of organ rejection. There are several drug classes and agents for immunosuppression. Use of these agents may increase the risk of different harms including not only infections, but also malignancies including post-transplant lymphoproliferative disorder. There is a need to identify which regimens minimize the risk of such outcomes. The objective of this systematic review and network meta-analysis of randomized and observational studies is to explore whether certain modern regimens of immunosuppression used to prevent organ rejection in renal transplant patients are associated with an increased risk of post-transplant lymphoproliferative disorder and other malignancies.
‘Modern’ regimens were defined to be those evaluated in controlled studies beginning in 1990 or later. An electronic literature search of Medline, Embase and the Cochrane Central Register of Controlled Trials has been designed by an experienced information specialist and peer reviewed by a second information specialist. Study selection and data collection will be performed by two reviewers. The outcomes of interest will include post-transplant lymphoproliferative disorder and other incident forms of malignancy occurring in adult renal transplant patients. Network meta-analyses of data from randomized and observational studies will be performed where judged appropriate based on a review of the clinical and methodological features of included studies. A sequential approach to meta-analysis will be used to combine data from different designs.
Our systematic review will include both single-agent and multi-agent modern pharmacotherapy regimens in patients undergoing renal transplantation. It will synthesize malignancy outcomes. Our work will also add to the development of methods for network meta-analysis across study designs to assess treatment safety.
PROSPERO Registration Number: CRD42013006951
Renal transplant; malignancy; post-transplant lymphoproliferative disorder; systematic review; network meta-analysis
Research collaboration contributes to the advancement of knowledge by exploiting the results of scientific efforts more efficiently, but the global patterns of collaboration on meta-analysis are unknown. The purpose of this research was to describe and characterize the global collaborative patterns in meta-analyses of randomized trials published in high impact factor medical journals over the past three decades.
This was a cross-sectional, social network analysis. We searched PubMed for relevant meta-analyses of randomized trials published up to December 2012. We selected meta-analyses (including at least randomized trials as primary evidence source) published in the top seven high impact factor general medical journals (according to Journal Citation Reports 2011): The New England Journal of Medicine, The Lancet, the BMJ, JAMA, Annals of Internal Medicine, Archives of Internal Medicine (now renamed JAMA Internal Medicine), and PLoS Medicine. Opinion articles, conceptual papers, narrative reviews, reviews without meta-analysis, reviews of reviews, and other study designs were excluded.
Overall, we included 736 meta-analyses, in which 3,178 authors, 891 institutions, and 51 countries participated. The BMJ was the journal that published the greatest number of articles (39%), followed by The Lancet (18%), JAMA (15%) and the Archives of Internal Medicine (15%). The USA, the UK, and Canada headed the absolute global productivity ranking in number of papers. The 64 authors and the 39 institutions with the highest publication rates were identified. We also found 82 clusters of authors (one group with 55 members and one group with 54 members) and 19 clusters of institutions (one major group with 76 members). The most prolific authors were mainly affiliated with the University of Oxford (UK), McMaster University (Canada), and the University of Bern (Switzerland).
Our analysis identified networks of authors, institutions and countries publishing meta-analyses of randomized trials in high impact medical journals. This valuable information may be used to strengthen scientific capacity for collaboration and to help to promote a global agenda for future research of excellence.
Authorship; Evidence-based medicine; Meta-analysis; Randomized controlled trial; Scientific collaboration; Social network analysis
Permanent childhood hearing loss affects 1 to 3 per 1000 children and frequently disrupts typical spoken language acquisition. Early identification of hearing loss through universal newborn hearing screening and the use of new hearing technologies including cochlear implants make spoken language an option for most children. However, there is no consensus on what constitutes optimal interventions for children when spoken language is the desired outcome. Intervention and educational approaches ranging from oral language only to oral language combined with various forms of sign language have evolved. Parents are therefore faced with important decisions in the first months of their child’s life.
This article presents the protocol for a systematic review of the effects of using sign language in combination with oral language intervention on spoken language acquisition. Studies addressing early intervention will be selected in which therapy involving oral language intervention and any form of sign language or sign support is used. Comparison groups will include children in early oral language intervention programs without sign support. The primary outcomes of interest to be examined include all measures of auditory, vocabulary, language, speech production, and speech intelligibility skills. We will include randomized controlled trials, controlled clinical trials, and other quasi-experimental designs that include comparator groups as well as prospective and retrospective cohort studies. Case-control, cross-sectional, case series, and case studies will be excluded. Several electronic databases will be searched (for example, MEDLINE, EMBASE, CINAHL, PsycINFO) as well as grey literature and key websites. We anticipate that a narrative synthesis of the evidence will be required. We will carry out meta-analysis for outcomes if clinical similarity, quantity and quality permit quantitative pooling of data. We will conduct subgroup analyses if possible according to severity/type of hearing disorder, age of identification, and type of hearing technology.
This review will provide evidence on the effectiveness of using sign language in combination with oral language therapies for developing spoken language in children with hearing loss who are identified at a young age. The information from this review can provide guidance to parents and intervention specialists, inform policy decisions and provide directions for future research.
PROSPERO registration number
Children; Hearing loss; Spoken language; Sign language; Outcomes; Systematic review
Systematic reviews (SRs) can become outdated as new evidence emerges over time. Organizations that produce SRs need a surveillance method to determine when reviews are likely to require updating. This report describes the development and initial results of a surveillance system to assess SRs produced by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program.
Twenty-four SRs were assessed using existing methods that incorporate limited literature searches, expert opinion, and quantitative methods for the presence of signals triggering the need for updating. The system was designed to begin surveillance six months after the release of the original review, and thenceforth every six months for any review not classified as being a high priority for updating. The outcome of each round of surveillance was a classification of the SR as being low, medium or high priority for updating.
Twenty-four SRs underwent surveillance at least once, and ten underwent surveillance a second time during the 18 months of the program. Two SRs were classified as high, five as medium, and 17 as low priority for updating. The time lapse between the searches conducted for the original reports and the updated searches (search time lapse - STL) ranged from 11 months to 62 months: The STL for the high priority reports were 29 months and 54 months; those for medium priority reports ranged from 19 to 62 months; and those for low priority reports ranged from 11 to 33 months. Neither the STL nor the number of new relevant articles was perfectly associated with a signal for updating. Challenges of implementing the surveillance system included determining what constituted the actual conclusions of an SR that required assessing; and sometimes poor response rates of experts.
In this system of regular surveillance of 24 systematic reviews on a variety of clinical interventions produced by a leading organization, about 70% of reviews were determined to have a low priority for updating. Evidence suggests that the time period for surveillance is yearly rather than the six months used in this project.
Systematic review; Updating; Surveillance
Reporting guidelines can be used to encourage standardised and comprehensive reporting of health research. In light of the global commitment to health equity, we have previously developed and published a reporting guideline for equity-focused systematic reviews (PRISMA-E 2012). The objectives of this study were to explore the utility of the equity extension items included in PRISMA-E 2012 from a systematic review author perspective, including facilitators and barriers to its use. This will assist in designing dissemination and knowledge translation strategies. We conducted a survey of systematic review authors to expose them to the new items in PRISMA-E 2012, establish the extent to which they had historically addressed those items in their own reviews, and gather feedback on the usefulness of the new items. Data were analysed using Microsoft Excel 2008 and Stata (version 11.2 for Mac). Of 151 respondents completing the survey, 18.5% (95% CI: 12.7% to 25.7%) had not heard of the PRISMA statement before, although 83.4% (95% CI: 77.5% to 89.3%) indicated that they plan to use PRISMA-E 2012 in the future, depending on the focus of their review. Most (68.9%; 95% CI: 60.8% to 76.2%) thought that using PRISMA-E 2012 would lead them to conduct their reviews differently. Important facilitators to using PRISMA-E 2012 identified by respondents were journal endorsement and incorporation of the elements of the guideline into systematic review software. Barriers identified were lack of time, word limits and the availability of equity data in primary research. This study has been the first to ‘road-test’ the new PRISMA-E 2012 reporting guideline and the findings are encouraging. They confirm the acceptability and potential utility of the guideline to assist review authors in reporting on equity in their reviews. The uptake and impact of PRISMA-E 2012 over time on design, conduct and reporting of primary research and systematic reviews should continue to be examined.