Search tips
Search criteria

Results 1-5 (5)

Clipboard (0)

Select a Filter Below

more »
Year of Publication
Document Types
1.  Completeness of Reporting of Patient-Relevant Clinical Trial Outcomes: Comparison of Unpublished Clinical Study Reports with Publicly Available Data 
PLoS Medicine  2013;10(10):e1001526.
Beate Wieseler and colleagues compare the completeness of reporting of patient-relevant clinical trial outcomes between clinical study reports and publicly available data.
Please see later in the article for the Editors' Summary
Access to unpublished clinical study reports (CSRs) is currently being discussed as a means to allow unbiased evaluation of clinical research. The Institute for Quality and Efficiency in Health Care (IQWiG) routinely requests CSRs from manufacturers for its drug assessments.
Our objective was to determine the information gain from CSRs compared to publicly available sources (journal publications and registry reports) for patient-relevant outcomes included in IQWiG health technology assessments (HTAs) of drugs.
Methods and Findings
We used a sample of 101 trials with full CSRs received for 16 HTAs of drugs completed by IQWiG between 15 January 2006 and 14 February 2011, and analyzed the CSRs and the publicly available sources of these trials. For each document type we assessed the completeness of information on all patient-relevant outcomes included in the HTAs (benefit outcomes, e.g., mortality, symptoms, and health-related quality of life; harm outcomes, e.g., adverse events). We dichotomized the outcomes as “completely reported” or “incompletely reported.” For each document type, we calculated the proportion of outcomes with complete information per outcome category and overall.
We analyzed 101 trials with CSRs; 86 had at least one publicly available source, 65 at least one journal publication, and 50 a registry report. The trials included 1,080 patient-relevant outcomes. The CSRs provided complete information on a considerably higher proportion of outcomes (86%) than the combined publicly available sources (39%). With the exception of health-related quality of life (57%), CSRs provided complete information on 78% to 100% of the various benefit outcomes (combined publicly available sources: 20% to 53%). CSRs also provided considerably more information on harms. The differences in completeness of information for patient-relevant outcomes between CSRs and journal publications or registry reports (or a combination of both) were statistically significant for all types of outcomes.
The main limitation of our study is that our sample is not representative because only CSRs provided voluntarily by pharmaceutical companies upon request could be assessed. In addition, the sample covered only a limited number of therapeutic areas and was restricted to randomized controlled trials investigating drugs.
In contrast to CSRs, publicly available sources provide insufficient information on patient-relevant outcomes of clinical trials. CSRs should therefore be made publicly available.
Please see later in the article for the Editors' Summary
Editors' Summary
People assume that, when they are ill, health care professionals will ensure that they get the best available treatment. In the past, clinicians used their own experience to make decisions about which treatments to offer their patients, but nowadays, they rely on evidence-based medicine—the systematic review and appraisal of clinical trials, studies that investigate the benefits and harms of drugs and other medical interventions in patients. Evidence-based medicine can guide clinicians, however, only if all the results of clinical research are available for evaluation. Unfortunately, the results of trials in which a new drug performs better than existing drugs are more likely to be published than those in which the new drug performs badly or has unwanted side effects (publication bias). Moreover, trial outcomes that support the use of a new treatment are more likely to be published than those that do not support its use (outcome reporting bias). Both types of bias pose a substantial threat to informed medical decision-making.
Why Was This Study Done?
Recent initiatives, such as making registration of clinical trials in a trial registry (for example, a precondition for publication in medical journals, aim to prevent these biases but are imperfect. Another way to facilitate the unbiased evaluation of clinical research might be to increase access to clinical study reports (CSRs)—detailed but generally unpublished accounts of clinical trials. Notably, information from CSRs was recently used to challenge conclusions based on published evidence about the efficacy and safety of the antiviral drug oseltamivir and the antidepressant reboxetine. In this study, the researchers compare the information available in CSRs and in publicly available sources (journal publications and registry reports) for the patient-relevant outcomes included in 16 health technology assessments (HTAs; analyses of the medical implications of the use of specific medical technologies) for drugs; the HTAs were prepared by the Institute for Quality and Efficiency in Health Care (IQWiG), Germany's main HTA agency.
What Did the Researchers Do and Find?
The researchers searched for published journal articles and registry reports for each of 101 trials for which the IQWiG had requested and received full CSRs from drug manufacturers during HTA preparation. They then assessed the completeness of information on the patient-relevant benefit and harm outcomes (for example symptom relief and adverse effects, respectively) included in each document type. Eighty-six of the included trials had at least one publicly available data source; the results of 15% of the trials were not available in either journals or registry reports. Overall, the CSRs provided complete information on 86% of the patient-related outcomes, whereas the combined publicly available sources provided complete information on only 39% of the outcomes. For individual outcomes, the CSRs provided complete information on 78%–100% of the benefit outcomes, with the exception of health-related quality of life (57%); combined publicly available sources provided complete information on 20%–53% of these outcomes. The CSRs also provided more information on patient-relevant harm outcomes than the publicly available sources.
What Do These Findings Mean?
These findings show that, for the clinical trials considered here, publicly available sources provide much less information on patient-relevant outcomes than CSRs. The generalizability of these findings may be limited, however, because the trials included in this study are not representative of all trials. Specifically, only CSRs that were voluntarily provided by drug companies were assessed, a limited number of therapeutic areas were covered by the trials, and the trials investigated only drugs. Nevertheless, these findings suggest that access to CSRs is important for the unbiased evaluation of clinical trials and for informed decision-making in health care. Notably, in June 2013, the European Medicines Agency released a draft policy calling for the proactive publication of complete clinical trial data (possibly including CSRs). In addition, the European Union and the European Commission are considering legal measures to improve the transparency of clinical trial data. Both these initiatives will probably only apply to drugs that are approved after January 2014, however, and not to drugs already in use. The researchers therefore call for CSRs to be made publicly available for both past and future trials, a recommendation also supported by the AllTrials initiative, which is campaigning for all clinical trials to be registered and fully reported.
Additional Information
Please access these websites via the online version of this summary at
Wikipedia has pages on evidence-based medicine, publication bias, and health technology assessment (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The website is a searchable register of federally and privately supported clinical trials in the US; it provides information about all aspects of clinical trials
The European Medicines Agency (EMA) provides information about all aspects of the scientific evaluation and approval of new medicines in the European Union, and guidance on the preparation of clinical study reports; its draft policy on the release of data from clinical trials is available
Information about IQWiG is available (in English and German); Informed Health Online is a website provided by IQWiG that provides objective, independent, and evidence-based information for patients (also in English and German)
PMCID: PMC3793003  PMID: 24115912
2.  A note on the graphical presentation of prediction intervals in random-effects meta-analyses 
Systematic Reviews  2012;1:34.
Meta-analysis is used to combine the results of several related studies. Two different models are generally applied: the fixed-effect (FE) and random-effects (RE) models. Although the two approaches estimate different parameters (that is, the true effect versus the expected value of the distribution of true effects) in practice, the graphical presentation of results is the same for both models. This means that in forest plots of RE meta-analyses, no estimate of the between-study variation is usually given graphically, even though it provides important information about the heterogeneity between the study effect sizes.
In addition to the point estimate of the between-study variation, a prediction interval (PI) can be used to determine the degree of heterogeneity, as it provides a region in which about 95% of the true study effects are expected to be found. To distinguish between the confidence interval (CI) for the average effect and the PI, it may also be helpful to include the latter interval in forest plots. We propose a new graphical presentation of the PI; in our method, the summary statistics in forest plots of RE meta-analyses include an additional row, ‘95% prediction interval’, and the PI itself is presented in the form of a rectangle below the usual diamond illustrating the estimated average effect and its CI. We then compare this new graphical presentation of PIs with previous proposals by other authors. The way the PI is presented in forest plots is crucial. In previous proposals, the distinction between the CI and the PI has not been made clear, as both intervals have been illustrated either by a diamond or by extra lines added to the diamond, which may result in misinterpretation.
To distinguish graphically between the results of an FE and those of an RE meta-analysis, it is helpful to extend forest plots of the latter approach by including the PI. Clear presentation of the PI is necessary to avoid confusion with the CI of the average effect estimate.
PMCID: PMC3552946  PMID: 22839660
Meta-analysis; Heterogeneity; Random effects model; Forest plot; Prediction interval
3.  First-Line Matched Related Donor Hematopoietic Stem Cell Transplantation Compared to Immunosuppressive Therapy in Acquired Severe Aplastic Anemia 
PLoS ONE  2011;6(4):e18572.
Acquired severe aplastic anemia (SAA) is a rare and progressive disease characterized by an immune-mediated functional impairment of hematopoietic stem cells. Transplantation of these cells is a first-line treatment option if HLA-matched related donors are available. First-line immunosuppressive therapy may be offered as alternative. The aim was to compare the outcome of these patients in controlled trials.
A systematic search was performed in the bibliographic databases MEDLINE, EMBASE, and The Cochrane Library. To show an overview of various outcomes by treatment group we conducted a meta-analysis on overall survival. We evaluated whether studies reported statistically significant factors for improved survival.
26 non-randomized controlled trials (7,955 patients enrolled from 1970 to 2001) were identified. We did not identify any RCTs. Risk of bias was high except in 4 studies. Young age and recent year of treatment were identified as factors for improved survival in the HSCT group. Advanced age, SAA without very severe aplastic anemia, and combination of anti-lymphocyte globulin with cyclosporine A were factors for improved survival in the IST group. In 19 studies (4,855 patients), summary statistics were sufficient to be included in meta-analysis. Considerable heterogeneity did not justify a pooled estimate. Adverse events were inconsistently reported and varied significantly across studies.
Young age and recent year of treatment were identified as factors for improved survival in the transplant group. Advanced age, SAA without very severe aplastic anemia, and combination of anti-lymphocyte globulin with cyclosporine A were factors for improved survival in the immunosuppressive group. Considerable heterogeneity of non-randomized controlled studies did not justify a pooled estimate. Adverse events were inconsistently reported and varied significantly across studies.
PMCID: PMC3081818  PMID: 21541024
4.  High-Dose Chemotherapy Followed by Autologous Stem Cell Transplantation for Metastatic Rhabdomyosarcoma—A Systematic Review 
PLoS ONE  2011;6(2):e17127.
Patients with metastatic rhabdomyosarcoma (RMS) have a poor prognosis. The aim of this systematic review is to investigate whether high-dose chemotherapy (HDCT) followed by autologous hematopoietic stem cell transplantation (HSCT) in patients with metastatic RMS has additional benefit or harm compared to standard chemotherapy.
Systematic literature searches were performed in MEDLINE, EMBASE, and The Cochrane Library. All databases were searched from inception to February 2010. PubMed was searched in June 2010 for a last update. In addition to randomized and non-randomized controlled trials, case series and case reports were included to complement results from scant data. The primary outcome was overall survival. A meta-analysis was performed using the hazard ratio as primary effect measure, which was estimated from Cox proportional hazard models or from summary statistics of Kaplan Meier product-limit estimations.
A total of 40 studies with 287 transplant patients with metastatic RMS (age range 0 to 32 years) were included in the assessment. We identified 3 non-randomized controlled trials. The 3-year overall survival ranged from 22% to 53% in the transplant groups vs. 18% to 55% in the control groups. Meta-analysis on overall survival in controlled trials showed no difference between treatments. Result of meta-analysis of pooled individual survival data of case series and case reports, and results from uncontrolled studies with aggregate data were in the range of those from controlled data. The risk of bias was high in all studies due to methodological flaws.
HDCT followed by autologous HSCT in patients with RMS remains an experimental treatment. At present, it does not appear justifiable to use this treatment except in appropriately designed controlled trials.
PMCID: PMC3044147  PMID: 21373200
5.  Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials 
Objectives To assess the benefits and harms of reboxetine versus placebo or selective serotonin reuptake inhibitors (SSRIs) in the acute treatment of depression, and to measure the impact of potential publication bias in trials of reboxetine.
Design Systematic review and meta-analysis including unpublished data.
Data sources Bibliographic databases (Medline, Embase, PsycINFO, BIOSIS, and Cochrane Library), clinical trial registries, trial results databases, and regulatory authority websites up until February 2009, as well as unpublished data from the manufacturer of reboxetine (Pfizer, Berlin).
Eligibility criteria Double blind, randomised, controlled trials of acute treatment (six weeks or more) with reboxetine versus placebo or SSRIs in adults with major depression.
Outcome measures Remission and response rates (benefit outcomes), as well as rates of patients with at least one adverse event and withdrawals owing to adverse events (harm outcomes).
Data extraction and data synthesis The procedures for data extraction and assessment of risk of bias were always conducted by one person and checked by another. If feasible, data were pooled by meta-analyses (random effects model). Publication bias was measured by comparing results of published and unpublished trials.
Results We analysed 13 acute treatment trials that were placebo controlled, SSRI controlled, or both, which included 4098 patients. Data on 74% (3033/4098) of these patients were unpublished. In the reboxetine versus placebo comparison, no significant differences in remission rates were shown (odds ratio 1.17, 95% confidence interval 0.91 to 1.51; P=0.216). Substantial heterogeneity (I2=67.3%) was shown in the meta-analysis of the eight trials that investigated response rates for reboxetine versus placebo. A sensitivity analysis that excluded a small inpatient trial showed no significant difference in response rates between patients receiving reboxetine and those receiving placebo (OR 1.24, 95% CI 0.98 to 1.56; P=0.071; I2=42.1%). Reboxetine was inferior to SSRIs (fluoxetine, paroxetine, and citalopram) for remission rates (OR 0.80, 95% CI 0.67 to 0.96; P=0.015) and response rates (OR 0.80, 95% CI 0.67 to 0.95; P=0.01). Reboxetine was inferior to placebo for both harm outcomes (P<0.001 for both), and to fluoxetine for withdrawals owing to adverse events (OR 1.79, 95% CI 1.06 to 3.05; P=0.031). Published data overestimated the benefit of reboxetine versus placebo by up to 115% and reboxetine versus SSRIs by up to 23%, and also underestimated harm.
Conclusions Reboxetine is, overall, an ineffective and potentially harmful antidepressant. Published evidence is affected by publication bias, underlining the urgent need for mandatory publication of trial data.
PMCID: PMC2954275  PMID: 20940209

Results 1-5 (5)