We identified 38 publications presenting data from 38 of the 42 studies
submitted to the drug regulatory
authority.
13–50 They were published between 1983 and 1999 and included duplicate publications
and pooled analyses. The sponsoring companies confirmed the completeness of
our search.
Multiple publication
shows the degree of
multiple publication: this varied from no duplicate publication (drug 3) to
extensive multiple publication (drug 1) with three stand alone publications
appearing twice and two subsets of studies published as pooled publications
three times each.
For drug 1, there were no cross references between the pooled analyses of
the same subsets of studies. For each of the subsets, the first author was
different in two of the pooled analyses, and the third publication had a
single author. Many of the studies had appeared previously as stand alone
publications, but reference to these in the pooled publications was given in
two cases only, once for each subset. Some of the analyses were presented as a
pooled analysis of stand alone centres and some as a multicentre study. For
both subsets of studies, the pooled results differed slightly between the
publications.
For drug 2, eight studies resulted in three pooled publications based on
different combinations of studies. The pooled analyses based on two and eight
studies appeared simultaneously (as “a double blind comparison”
and “a large multicentre study” respectively) with one author in
common but without cross reference. Later, the five study analysis was
presented as an intention to treat reanalysis of the per protocol analysis in
the eight study publication without revealing that three studies were omitted.
Nor was it said that two of the included studies had been published earlier as
stand alone publications.
The pooled publication of studies of drug 4 was denoted as a review of
multicentre controlled studies without identification of the included studies.
Two of the studies later appeared as stand alone publications without
acknowledgement of their earlier inclusion in a pooled publication. There was
no author name in common in the pooled and stand alone publications.
For drug 5, the pooled analysis was presented as a meta-analysis of the
five available placebo controlled studies, clearly identified by the name of
the principal investigator. Reference was given to one previous stand alone
publication. The other stand alone publication appeared seven years later
without reference to the pooled publication.
Selective publication
Of the 42 submitted studies, 21 found the test drug to be significantly
more effective than placebo in the primary variable
(). Nineteen of these
studies appeared as stand alone publications. Only six of the 21 studies not
showing significant results were published as stand alone publications. Of the
four studies that never reached the public domain, all showed non-significant
results with respect to the primary variable.
Selective reporting
All but one of the study reports submitted to the regulatory agency
presented results from two or more alternative analyses (intention to treat
and per protocol). Only two of the stand alone publications presented an
intention to treat analysis as well as a per protocol analysis. The remaining
stand alone publications presented only one analysis, which tended to be the
more favourable per protocol analysis. In the 15 stand alone and five pooled
publications reporting differences in percentage response, patients who
withdrew or who could not be evaluated were usually ignored in the
calculations of the response rates. As
shows, this could
result in large overestimates compared with the intention to treat analysis
based on the submitted reports, where patients who withdrew or could not be
evaluated were considered to be non-responders. In one extreme case the
published difference in the percentage of patients responding to treatment was
51%, whereas no difference was seen in the intention to treat analysis. In
five other cases the size of the overestimation was 10-25%. The degree of
overestimation tended to be higher in smaller studies.
Comparison of pooled results from submitted and published
studies
In 41 of the 42 submitted studies data on response rate were provided or
could easily be calculated on an intention to treat basis. In total, 15 stand
alone publications and five pooled publications reported response rates based
on data from 32 studies. For each drug, we compared a pooled analysis of all
studies submitted to the regulatory agency with a pooled analysis of a correct
selection of published studies in which all duplicates were excluded. We also
made a pooled analysis of published studies including those duplicates that
probably could not be identified as such without access to information about
all studies. In this second selection we excluded duplicates with at least one
author in common and only minor differences with respect to patient numbers
and efficacy results but included any duplicates unidentifiable because of
lack of cross reference between pooled publications and stand alone
publications.
The pooled analyses of published data generally gave larger differences in
response rate (drug minus placebo) than did the estimates from all submitted
studies (). The result of
the comparison was conspicuous for two products. The estimate based on
evidence from all available studies on drug 2 indicated a marginal effect,
whereas the pooled analysis of published data gave an estimated effect size
similar to that for most of the other drugs. Similarly, the analyses based on
published data gave the impression that drug 4 was substantially more
effective than the other drugs, whereas the analysis based on submitted
studies did not. Since the estimates based on the published studies for these
two drugs included data from all the submitted studies, the overestimations
are due to selective reporting rather than selective publication. Overall,
there were only minor differences in response rates between the correct
selections of published studies and the plausible selections. Thus, in this
material there is no indication of any major bias due to multiple
publication.
This article has been 
