Background

There are an increasing number of studies reporting the efficacy of educational strategies to facilitate the development of knowledge and skills underpinning evidence based practice (EBP). To date there is no standardised guideline for describing the teaching, evaluation, context or content of EBP educational strategies. The heterogeneity in the reporting of EBP educational interventions makes comparisons between studies difficult. The aim of this program of research is to develop the Guideline for Reporting EBP Educational interventions and Teaching (GREET) statement and an accompanying explanation and elaboration (E&E) paper.

Methods/design

Three stages are planned for the development process. Stage one will comprise a systematic review to identify features commonly reported in descriptions of EBP educational interventions. In stage two, corresponding authors of articles included in the systematic review and the editors of the journals in which these studies were published will be invited to participate in a Delphi process to reach consensus on items to be considered when reporting EBP educational interventions. The final stage of the project will include the development and pilot testing of the GREET statement and E&E paper.

Outcome

The final outcome will be the creation of a Guideline for Reporting EBP Educational interventions and Teaching (GREET) statement and E&E paper.

Discussion

The reporting of health research including EBP educational research interventions, have been criticised for a lack of transparency and completeness. The development of the GREET statement will enable the standardised reporting of EBP educational research. This will provide a guide for researchers, reviewers and publishers for reporting EBP educational interventions.

Background

PROSPERO, an international prospective register of systematic review protocols in health and social care, was launched in February 2011. After one year of operation we describe access and use, explore user experience and identify areas for future improvement.

Methods

We collated administrative data and web statistics and conducted an online survey of users’ experiences.

Results

On 21 February 2012, there were 1,076 registered users and 359 registration records published on PROSPERO. The database usage statistics demonstrate the international interest in PROSPERO with high access around the clock and around the world. Based on 232 responses from PROSPERO users (response rate 22%), almost all respondents found joining and navigation was easy or very easy (99%); turn round time was good or excellent (96%); and supporting materials provided were helpful or very helpful (80%). The registration fields were found by 80% to be relevant to their review; 99% rated their overall experience of registering with PROSPERO as good or excellent. Most respondents (81%) had a written protocol before completing the registration form and 19% did not. The majority, 136 (79%), indicated they completed the registration form in 60 minutes or less. Of those who expressed an opinion, 167 (87%) considered the time taken to be about right.

Conclusions

The first year of PROSPERO has shown that registration of systematic review protocols is feasible and not overly burdensome for those registering their reviews. The evaluation has demonstrated that, on the whole, survey respondents are satisfied and the system allows registration of protocol details in a straightforward and acceptable way. The findings have prompted some changes to improve user experience and identified some issues for future consideration.

Background

Systematic reviews (SRs) are abundant. The optimal reporting of SRs is critical to enable clinicians to use their findings to make informed treatment decisions. Complementary and alternative medicine (CAM) therapies are widely used therefore it is critical that conduct and reporting of systematic research in this field be of high quality. Here, methodological and reporting characteristics of a sample of CAM-related SRs and a sample of control SRs are evaluated and compared.

Methods

MEDLINE® was searched to identify non-Cochrane SRs indexed from January 2010 to May 2011. Control SRs were retrieved and a search filter was used to identify CAM SRs. Citations were screened and publications that met a pre-specified definition of a SR were included. Pre-designed, standardized data extraction forms were developed to capture reporting and methodological characteristics of the included reviews. Where appropriate, samples were compared descriptively.

Results

A total of 349 SRs were identified, of which 174 were CAM-related SRs and 175 were conventional SRs. We compared 131 CAM-related non-Cochrane SRs to the 175 conventional non-Cochrane reviews. Fifty-seven percent (75/131) of CAM SRs specified a primary outcome compared to 21% (37/175) of conventional sample reviews. Reporting of publication bias occurred in less than 5% (6/131) of the CAM sample versus 46% (80/175) of the conventional sample of SRs. Source of funding was frequently and consistently under-reported. Less than 5% (11/306) of all SRs reported public availability of a review protocol.

Conclusion

The two samples of reviews exhibited different strengths and weaknesses. In some cases there were consistencies across items which indicate the need for continued improvements in reporting for all SR reports. We advise authors to utilise the PRISMA Statement or other SR guidance when reporting SRs.

Background

The Consolidated Standards of Reporting Trials (CONSORT) Statement is intended to facilitate better reporting of randomised clinical trials (RCTs). A systematic review recently published in the Cochrane Library assesses whether journal endorsement of CONSORT impacts the completeness of reporting of RCTs; those findings are summarised here.

Methods

Evaluations assessing the completeness of reporting of RCTs based on any of 27 outcomes formulated based on the 1996 or 2001 CONSORT checklists were included; two primary comparisons were evaluated. The 27 outcomes were: the 22 items of the 2001 CONSORT checklist, four sub-items describing blinding and a ‘total summary score’ of aggregate items, as reported. Relative risks (RR) and 99% confidence intervals were calculated to determine effect estimates for each outcome across evaluations.

Results

Fifty-three reports describing 50 evaluations of 16,604 RCTs were assessed for adherence to at least one of 27 outcomes. Sixty-nine of 81 meta-analyses show relative benefit from CONSORT endorsement on completeness of reporting. Between endorsing and non-endorsing journals, 25 outcomes are improved with CONSORT endorsement, five of these significantly (α = 0.01). The number of evaluations per meta-analysis was often low with substantial heterogeneity; validity was assessed as low or unclear for many evaluations.

Conclusions

The results of this review suggest that journal endorsement of CONSORT may benefit the completeness of reporting of RCTs they publish. No evidence suggests that endorsement hinders the completeness of RCT reporting. However, despite relative improvements when CONSORT is endorsed by journals, the completeness of reporting of trials remains sub-optimal. Journals are not sending a clear message about endorsement to authors submitting manuscripts for publication. As such, fidelity of endorsement as an ‘intervention’ has been weak to date. Journals need to take further action regarding their endorsement and implementation of CONSORT to facilitate accurate, transparent and complete reporting of trials.

Background

Systematic reviews (SRs) can provide accurate and reliable evidence, typically about the effectiveness of health interventions. Evidence is dynamic, and if SRs are out-of-date this information may not be useful; it may even be harmful. This study aimed to compare five statistical methods to identify out-of-date SRs.

Methods

A retrospective cohort of SRs registered in the Cochrane Pregnancy and Childbirth Group (CPCG), published between 2008 and 2010, were considered for inclusion. For each eligible CPCG review, data were extracted and “3-years previous” meta-analyses were assessed for the need to update, given the data from the most recent 3 years. Each of the five statistical methods was used, with random effects analyses throughout the study.

Results

Eighty reviews were included in this study; most were in the area of induction of labour. The numbers of reviews identified as being out-of-date using the Ottawa, recursive cumulative meta-analysis (CMA), and Barrowman methods were 34, 7, and 7 respectively. No reviews were identified as being out-of-date using the simulation-based power method, or the CMA for sufficiency and stability method. The overall agreement among the three discriminating statistical methods was slight (Kappa = 0.14; 95% CI 0.05 to 0.23). The recursive cumulative meta-analysis, Ottawa, and Barrowman methods were practical according to the study criteria.

Conclusion

Our study shows that three practical statistical methods could be applied to examine the need to update SRs.

Vivian Welch and colleagues present consensus-based guidelines for reporting equity-focused systematic reviews, the PRISMA-Equity extension.

Background

Pediatric randomized controlled trials (RCTs) are susceptible to a high risk of bias. We examined the barriers and facilitators that pediatric trialists face in the design and conduct of unbiased trials.

Methods

We used a mixed methods design, with semi-structured interviews building upon the results of a quantitative survey. We surveyed Canadian (n=253) and international (n=600) pediatric trialists regarding their knowledge and awareness of bias and their perceived barriers and facilitators in conducting clinical trials. We then interviewed 13 participants from different subspecialties and geographic locations to gain a more detailed description of how their experiences and attitudes towards research interacted with trial design and conduct.

Results

The survey response rate was 23.0% (186/807). 68.1% of respondents agreed that bias is a problem in pediatric RCTs and 72.0% felt that there is sufficient evidence to support changing some aspects of how trials are conducted. Knowledge related to bias was variable, with inconsistent awareness of study design features that may introduce bias into a study. Interview participants highlighted a lack of formal training in research methods, a negative research culture, and the pragmatics of trial conduct as barriers. Facilitators included contact with knowledgeable and supportive colleagues and infrastructure for research.

Conclusions

A lack of awareness of bias and negative attitudes towards research present significant barriers in terms of conducting methodologically rigorous pediatric RCTs. Knowledge translation efforts must focus on these issues to ensure the relevance and validity of trial results.

Background

Recent evidence has highlighted deficiencies in clinical trial protocols, having implications for many groups. Existing guidelines for randomized clinical trial (RCT) protocol content vary substantially and most do not describe systematic methodology for their development. As one of three prespecified steps for the systematic development of a guideline for trial protocol content, the objective of this study was to conduct a three-round Delphi consensus survey to develop and refine minimum content for RCT protocols.

Methods

Panellists were identified using a multistep iterative approach, met prespecified minimum criteria and represented key stakeholders who develop or use clinical trial protocols. They were asked to rate concepts for importance in a minimum set of items for RCT protocols. The main outcome measures were degree of importance (scale of 1 to 10; higher scores indicating higher importance) and level of consensus for items. Results were presented as medians, interquartile ranges, counts and percentages.

Results

Ninety-six expert panellists participated in the Delphi consensus survey including trial investigators, methodologists, research ethics board members, funders, industry, regulators and journal editors. Response rates were between 88 and 93% per round. Overall, panellists rated 63 of 88 concepts of high importance (of which 50 had a 25th percentile rating of 8 or greater), 13 of moderate importance (median 6 or 7) and 12 of low importance (median less than or equal to 5) for minimum trial protocol content. General and item-specific comments and subgroup results provided valuable insight for further discussions.

Conclusions

This Delphi process achieved consensus from a large panel of experts from diverse stakeholder groups on essential content for RCT protocols. It also highlights areas of divergence. These results, complemented by other empirical research and consensus meetings, are helping guide the development of a guideline for protocol content.

Background

All randomized clinical trials (RCTs) require a protocol; however, numerous studies have highlighted protocol deficiencies. Reporting guidelines may improve the content of research reports and, if developed using robust methods, may increase the utility of reports to stakeholders. The objective of this study was to systematically identify and review RCT protocol guidelines, to assess their characteristics and methods of development, and to compare recommendations.

Methods

We conducted a systematic review of indexed literature (MEDLINE, EMBASE and the Cochrane Methodology Register from inception to September 2010; reference lists; related article features; forward citation searching) and a targeted search of supplementary sources, including a survey of major trial funding agencies in six countries. Records were eligible if they described a content guideline in English or French relevant to RCT protocols. Guidelines were excluded if they specified content for protocols for trials of specific procedures or conditions or were intended to assess trial quality. We extracted guideline characteristics and methods. Content was mapped for a subset of guidelines that described development methods or had institutional endorsement.

Results

Forty guidelines published in journals, books and institutional reports were included in the review; seven were specific to RCT protocols. Only eight (20%) described development methods which included informal consensus methods, pilot testing and formal validation; no guideline described all of these methods. No guideline described formal consensus methods or a systematic retrieval of empirical evidence to inform its development. The guidelines included a median of 23 concepts per guideline (interquartile range (IQR) = 14 to 34; range = 7 to 109). Among the subset of guidelines (n = 23) for which content was mapped, approximately 380 concepts were explicitly addressed (median concepts per guideline IQR = 31 (24,80); range = 16 to 150); most concepts were addressed in a minority of guidelines.

Conclusions

Existing guidelines for RCT protocol content varied substantially in their recommendations. Few reports described the methods of guideline development, limiting comparisons of guideline validity. Given the importance of protocols to diverse stakeholders, we believe a systematically developed, evidence-informed guideline for clinical trial protocols is needed.

Aaron Kesselheim and colleagues conduct a systematic review to examine the strengths and weaknesses associated with approaches to medical device regulation in the US and EU.

Background

Policymakers and regulators in the United States (US) and the European Union (EU) are weighing reforms to their medical device approval and post-market surveillance systems. Data may be available that identify strengths and weakness of the approaches to medical device regulation in these settings.

Methods and Findings

We performed a systematic review to find empirical studies evaluating medical device regulation in the US or EU. We searched Medline using two nested categories that included medical devices and glossary terms attributable to the US Food and Drug Administration and the EU, following PRISMA guidelines for systematic reviews. We supplemented this search with a review of the US Government Accountability Office online database for reports on US Food and Drug Administration device regulation, consultations with local experts in the field, manual reference mining of selected articles, and Google searches using the same key terms used in the Medline search. We found studies of premarket evaluation and timing (n = 9), studies of device recalls (n = 8), and surveys of device manufacturers (n = 3). These studies provide evidence of quality problems in pre-market submissions in the US, provide conflicting views of device safety based largely on recall data, and relay perceptions of some industry leaders from self-surveys.

Conclusions

Few studies have quantitatively assessed medical device regulation in either the US or EU. Existing studies of US and EU device approval and post-market evaluation performance suggest that policy reforms are necessary for both systems, including improving classification of devices in the US and promoting transparency and post-market oversight in the EU. Assessment of regulatory performance in both settings is limited by lack of data on post-approval safety outcomes. Changes to these device approval and post-marketing systems must be accompanied by ongoing research to ensure that there is better assessment of what works in either setting.

Please see later in the article for the Editors' Summary.

Editors' Summary

Background

Medical devices—health technologies that are not medicines, vaccines, or clinical procedures—cover a vast range of equipment from the simple to the more complex. Medical devices are essential for patient care, and in the past decade, new devices have offered improved treatment alternatives for many diseases and conditions, leading to substantial growth in the US$350 billion medical device industry. However, new medical devices also pose substantial risks to patients, as shown in recent high-profile product recalls involving breast implants and artificial hip implants.

Why Was This Study Done?

Concerns about the safety of new medical devices have led to calls for greater testing of the safety and effectiveness of new devices before they come on the market and for improved monitoring of their performance after new devices have been approved for use by a regulatory body. In this study, the researchers systematically reviewed evidence about the performance of medical device approval and post-market surveillance systems in two of the most important world markets for medical devices—the United States and the European Union.

What Did the Researchers Do and Find?

The researchers performed a keyword search in Medline (a database of published biomedical literature) for all relevant articles, and supplemented this search with a review of reports on Food and Drug Administration (FDA) device regulation in the US Government Accountability Office's online database. Then they consulted with both US and EU experts and also conducted Google searches to capture reports by management consultant firms. The researchers included only those studies that reported empirical data, either qualitative or quantitative, about the characteristics, performance metrics, or effectiveness of device evaluation or post-market oversight in the US or EU.

Using these methods the researchers identified nine studies that focused on pre-market evaluation and timing, eight studies of device recalls, and three surveys of device manufacturers. Because of the variable quality and lack of outcomes from these studies and reports, the researchers concluded that these studies offered only limited insights into either the US or EU systems. But the available evidence does suggest that in the US, the FDA could improve oversight of device approval, for example, by following up on its commitment to reclassify high-risk medical devices and improve post-market surveillance of devices that are approved on the basis of limited data. The researchers also suggest that using recalls to measure the safety record of individual devices or classes of devices is flawed, as particular devices may be over- or underrepresented in recall data depending on the frequency of their use, design complexity, and the clinical manifestations of malfunction. In the EU, apart from a few studies addressing the timing of approval, the researchers found almost no robust data on device regulation. Some case reports suggested substantial dangers to patients in the EU from devices approved on the basis of limited data, but the researchers could not systematically compare the quality of studies used for device approval or post-approval safety outcomes between the EU and US, mainly because of the lack of transparency among the EU regulators (Notified Bodies).

What Do These Findings Mean?

These findings show that few studies have quantitatively assessed medical device regulation in either the US or EU, but the existing studies examined in this review suggest that policy reforms are necessary for both device approval and post-market evaluation of performance, including improving classification of devices in the US and promoting transparency and postmarket oversight in the EU. However, assessment of regulatory performance in both the US and EU is limited by lack of data on post-approval safety outcomes. Any changes to medical device approval and post-marketing systems should be accompanied by ongoing research and evaluation to ensure that there is an improved assessment of what works in either setting.

Additional Information

Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001276.

This study is further discussed in a PLoS Medicine Perspective by Sanket Dhruva and Rita Redberg

The WHO website has a comprehensive topic section on medical devices

Information on medical devices is also available from the FDA and the European Commission

Background

While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. The objective of this study is to summarize and collate suggested methods for investigating clinical heterogeneity in systematic reviews.

Methods

We searched databases (Medline, EMBASE, CINAHL, Cochrane Library, and CONSORT, to December 2010) and reference lists and contacted experts to identify resources providing suggestions for investigating clinical heterogeneity between controlled clinical trials included in systematic reviews. We extracted recommendations, assessed resources for risk of bias, and collated the recommendations.

Results

One hundred and one resources were collected, including narrative reviews, methodological reviews, statistical methods papers, and textbooks. These resources generally had a low risk of bias, but there was minimal consensus among them. Resources suggested that planned investigations of clinical heterogeneity should be made explicit in the protocol of the review; clinical experts should be included on the review team; a set of clinical covariates should be chosen considering variables from the participant level, intervention level, outcome level, research setting, or others unique to the research question; covariates should have a clear scientific rationale; there should be a sufficient number of trials per covariate; and results of any such investigations should be interpreted with caution.

Conclusions

Though the consensus was minimal, there were many recommendations in the literature for investigating clinical heterogeneity in systematic reviews. Formal recommendations for investigating clinical heterogeneity in systematic reviews of controlled trials are required.

BACKGROUND

The Chronic Care Model is an effective framework for improving chronic disease management. There is scarce literature describing this model for people living with HIV. Decision Support (DS) and Clinical Information Systems (CIS) are two components of this model that aim to improve care by changing health care provider behavior.

OBJECTIVE

Our aim was to assess the effectiveness of DS and CIS interventions for individuals with HIV, through a systematic literature review.

DESIGN

We performed systematic electronic searches from 1996 to February 2011 of the medical (E.g. Medline, EMBASE, CINAHL) and grey literature. Effectiveness was measured by the frequency of statistically significant outcome improvement. Data and key equity indicator extraction and synthesis was completed.

PARTICIPANTS AND INTERVENTIONS

We included comparative studies of people living with HIV that examined the impact of DS or CIS interventions on outcomes.

MAIN MEASURES

The following measures were assessed: outcome (immunological/virological, medical, psychosocial, economic measures) and health care process/performance measures.

KEY RESULTS

Records were screened for relevance (n = 10,169), full-text copies of relevant studies were obtained (n = 123), and 16 studies were included in the review. Overall, 5/9 (55.6%) and 17/41 (41.5%) process measures and 5/12 (41.7%) and 3/9 (33.3%) outcome measures for DS and CIS interventions, respectively, were statistically significantly improved. DS–explicit mention of implementation of guidelines and CIS-reminders showed the most frequent improvement in outcomes. DS-only interventions were more effective than CIS-only interventions in improving both process and outcome measures. Clinical, statistical and methodological heterogeneity among studies precluded meta-analysis. Primary studies were methodologically weak and often included multifaceted interventions that made assessment of effectiveness challenging.

CONCLUSIONS

Overall, DS and CIS interventions may modestly improve care for people living with HIV, having a greater impact on process measures compared to outcome measures. These interventions should be considered as part of strategies to improve HIV care through changing provider performance.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-012-2145-y) contains supplementary material, which is available to authorized users.

Objective

Complementary and alternative medicine (CAM) is commonly used by children, but estimates of that use vary widely partly due to the range of questionnaires used to assess CAM use. However, no studies have attempted to appraise measurement properties of these questionnaires. The aim of this systematic review was to critically appraise and summarize measurement properties of questionnaires of CAM use in pediatrics.

Study design

A search strategy was implemented in major electronic databases in March 2011 and conference websites, scientific journals and experts were consulted. Studies were included if they mentioned a questionnaire assessing the prevalence of CAM use in pediatrics. Members of the team independently rated the methodological quality of the studies (using the COSMIN checklist) and measurement properties of the questionnaires (using the Terwee and Cohen criteria).

Results

A total of 96 CAM questionnaires were found in 104 publications. The COSMIN checklist showed that no studies reported adequate methodological quality. The Terwee criteria showed that all included CAM questionnaires had indeterminate measurement properties. According to the Cohen score, none were considered to be a well-established assessment, two approached the level of a well-established assessment, seven were promising assessments and the remainder (n = 87) did not reach the score’s minimum standards.

Conclusion

None of the identified CAM questionnaires have been thoroughly validated. This systematic review highlights the need for proper validation of CAM questionnaires in pediatrics, which may in turn lead to improved research and knowledge translation about CAM in clinical practice.

Background

Reporting of health research is often inadequate and incomplete. Complete and transparent reporting is imperative to enable readers to assess the validity of research findings for use in healthcare and policy decision-making. To this end, many guidelines, aimed at improving the quality of health research reports, have been developed for reporting a variety of research types. Despite efforts, many reporting guidelines are underused. In order to increase their uptake, evidence of their effectiveness is important and will provide authors, peer reviewers and editors with an important resource for use and implementation of pertinent guidance. The objective of this study was to assess whether endorsement of reporting guidelines by journals influences the completeness of reporting of health studies.

Methods

Guidelines providing a minimum set of items to guide authors in reporting a specific type of research, developed with explicit methodology, and using a consensus process will be identified from an earlier systematic review and from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network’s reporting guidelines library. MEDLINE, EMBASE, the Cochrane Methodology Register and Scopus will be searched for evaluations of those reporting guidelines; relevant evaluations from the recently conducted CONSORT systematic review will also be included. Single data extraction with 10% verification of study characteristics, 20% of outcomes and complete verification of aspects of study validity will be carried out. We will include evaluations of reporting guidelines that assess the completeness of reporting: (1) before and after journal endorsement, and/or (2) between endorsing and non-endorsing journals. For a given guideline, analyses will be conducted for individual and the total sum of items. When possible, standard, pooled effects with 99% confidence intervals using random effects models will be calculated.

Discussion

Evidence on which guidelines have been evaluated and which are associated with improved completeness of reporting is important for various stakeholders, including editors who consider which guidelines to endorse in their journal editorial policies.

Background

There is a significant public health burden associated with substance use in Canada. The early detection and/or treatment of risky substance use has the potential to dramatically improve outcomes for those who experience harms from the non-medical use of psychoactive substances, particularly adolescents whose brains are still undergoing development. The Screening, Brief Intervention, and Referral to Treatment model is a comprehensive, integrated approach for the delivery of early intervention and treatment services for individuals experiencing substance use-related harms, as well as those who are at risk of experiencing such harm.

Methods

This article describes the protocol for a systematic review of the effectiveness of brief interventions as part of the Screening, Brief Intervention, and Referral to Treatment model for reducing the non-medical use of psychoactive substances. Studies will be selected in which brief interventions target non-medical psychoactive substance use (excluding alcohol, nicotine, or caffeine) among those 12 years and older who are opportunistically screened and deemed at risk of harms related to psychoactive substance use. We will include one-on-one verbal interventions and exclude non-verbal brief interventions (for example, the provision of information such as a pamphlet or online interventions) and group interventions. Primary, secondary and adverse outcomes of interest are prespecified. Randomized controlled trials will be included; non-randomized controlled trials, controlled before-after studies and interrupted time series designs will be considered in the absence of randomized controlled trials. We will search several bibliographic databases (for example, MEDLINE, EMBASE, CINAHL, PsycINFO, CORK) and search sources for grey literature. We will meta-analyze studies where possible. We will conduct subgroup analyses, if possible, according to drug class and intervention setting.

Discussion

This review will provide evidence on the effectiveness of brief interventions as part of the Screening, Brief Intervention, and Referral to Treatment protocol aimed at the non-medical use of psychoactive substances and may provide guidance as to where future research might be most beneficial.

Background

In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate P values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression.

Methods

We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final P values between methods.

Results

We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The P values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases.

Conclusions

We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase P values in meta-regression of multiple covariates for relatively small amount of trials.

Background

The CONSORT Statement is a reporting guideline for authors when reporting randomized controlled trials (RCTs). It offers a standard way for authors to prepare RCT reports. It has been endorsed by many high-impact medical journals and by international editorial groups. This study was conducted to assess the endorsement of the CONSORT Statement by high-impact medical journals in China by reviewing their instructions for authors.

Methodology/Principal Findings

A total of 200 medical journals were selected according to the Chinese Science and Technology Journal Citation Reports, 195 of which publish clinical research papers. Their instructions for authors were reviewed and all texts mentioning the CONSORT Statement or CONSORT extension papers were extracted. Any mention of the Uniform Requirements for Manuscripts Submitted to Biomedical Journals (URM) developed by the International Committee of Medical Journal Editors (ICMJE) or ‘clinical trial registration’ was also extracted. For journals endorsing the CONSORT Statement, their most recently published RCT reports were retrieved and evaluated to assess whether the journals have followed what the CONSORT Statement required. Out of the 195 medical journals publishing clinical research papers, only six (6/195, 3.08%) mentioned ‘CONSORT’ in their instructions for authors; out of the 200 medical journals surveyed, only 14 (14/200, 7.00%) mentioned ‘ICMJE’ or ‘URM’ in their instructions for authors, and another five journals stated in their instructions for authors that clinical trials should have trial registration numbers and that priority would be given to clinical trials which had been registered. Among the 62 RCT reports published in the six journals endorsing the CONSORT Statement, 20 (20/62, 32.26%) contained flow diagrams and only three (3/62, 4.84%) provided trial registration information.

Conclusions/Significance

Medical journals in China endorsing either the CONSORT Statement or the ICMJE's URM constituted a small percentage of the total; all of these journals used ambiguous language regarding what was expected of authors.

Background

Rapid reviews have emerged as a streamlined approach to synthesizing evidence - typically for informing emergent decisions faced by decision makers in health care settings. Although there is growing use of rapid review 'methods', and proliferation of rapid review products, there is a dearth of published literature on rapid review methodology. This paper outlines our experience with rapidly producing, publishing and disseminating evidence summaries in the context of our Knowledge to Action (KTA) research program.

Methods

The KTA research program is a two-year project designed to develop and assess the impact of a regional knowledge infrastructure that supports evidence-informed decision making by regional managers and stakeholders. As part of this program, we have developed evidence summaries - our form of rapid review - which have come to be a flagship component of this project. Our eight-step approach for producing evidence summaries has been developed iteratively, based on evidence (where available), experience and knowledge user feedback. The aim of our evidence summary approach is to deliver quality evidence that is both timely and user-friendly.

Results

From November 2009 to March 2011 we have produced 11 evidence summaries on a diverse range of questions identified by our knowledge users. Topic areas have included questions of clinical effectiveness to questions on health systems and/or health services. Knowledge users have reported evidence summaries to be of high value in informing their decisions and initiatives. We continue to experiment with incorporating more of the established methods of systematic reviews, while maintaining our capacity to deliver a final product in a timely manner.

Conclusions

The evolution of the KTA rapid review evidence summaries has been a positive one. We have developed an approach that appears to be addressing a need by knowledge users for timely, user-friendly, and trustworthy evidence and have transparently reported these methods here for the wider rapid review and scientific community.

Welcome to a new age in publishing systematic reviews. We hope the launch of Systematic Reviews will resonate with a broad spectrum of readers interested in using them in a variety of ways, such as providing comprehensive and up to date evidence for patient management, informing health policy, and developing rigorous practice guidelines. Systematic reviews are increasingly popular. Our journal is committed to publishing a wide variety of well conducted and transparently reported systematic reviews and associated research. We are open access and electronic and not confined by space and so offer scope for publishing reviews in detail and providing a modern and innovative approach to publishing. We look forward to participating in the voyage with all of our readers.

Background

Following publication of the PRISMA statement, the UK Centre for Reviews and Dissemination (CRD) at the University of York in England began to develop an international prospective register of systematic reviews with health-related outcomes. The objectives were to reduce unplanned duplication of reviews and provide transparency in the review process, with the aim of minimizing reporting bias.

Methods

An international advisory group was formed and a consultation undertaken to establish the key items necessary for inclusion in the register and to gather views on various aspects of functionality. This article describes the development of the register, now called PROSPERO, and the process of registration.

Results

PROSPERO offers free registration and free public access to a unique prospective register of systematic reviews across all areas of health from all around the world. The dedicated web-based interface is electronically searchable and available to all prospective registrants. At the moment, inclusion in PROSPERO is restricted to systematic reviews of the effects of interventions and strategies to prevent, diagnose, treat, and monitor health conditions, for which there is a health-related outcome.

Ideally, registration should take place before the researchers have started formal screening against inclusion criteria but reviews are eligible as long as they have not progressed beyond the point of completing data extraction.

The required dataset captures the key attributes of review design as well as the administrative details necessary for registration.

Submitted registration forms are checked against the scope for inclusion in PROSPERO and for clarity of content before being made publicly available on the register, rejected, or returned to the applicant for clarification.

The public records include an audit trail of major changes to planned methods, details of when the review has been completed, and links to resulting publications when provided by the authors.

Conclusions

There has been international support and an enthusiastic response to the principle of prospective registration of protocols for systematic reviews and to the development of PROSPERO.

In October 2011, PROSPERO contained 200 records of systematic reviews being undertaken in 26 countries around the world on a diverse range of interventions.

Prospective registration of systematic reviews promotes transparency, helps reduce potential for bias and serves to avoid unintended duplication of reviews. Registration offers advantages to many stakeholders in return for modest additional effort from the researchers registering their reviews.

Emergency department overcrowding is a serious and ongoing issue across Canada. Short stay units (SSUs) have emerged as a potentially useful strategy for managing overcrowding in emergency departments. Members of The Ottawa Hospital senior management team contemplating the introduction of an SSU to help alleviate emergency department overcrowding approached our rapid response service to conduct a rapid review on the safety and effectiveness of SSUs. This paper presents the process for conducting this review, its findings, and the end-user report generated for the senior management team and other stakeholders.

Rafael Dal-Ré and colleagues argue that the recruitment targets and performance of all site investigators in multi-centre clinical trials should be disclosed in trial registration sites before a trial starts, and when it ends.