This study demonstrates for the first time, to our knowledge, a high prevalence of "positive outcome-bias" in the preeminent orthopedic and surgical literature. The disproportion of published positive studies, compared to negative or neutral papers, remained relatively constant over the seven years (2000–2006) and among the twelve journals reviewed in this study. Even more striking was the discrepancy between published original papers with significant findings (positive or negative) compared to purely "neutral" studies which did not report any significant findings (91% vs. 9%).
Our results are consistent with prior investigations which revealed a high prevalence of a "positive-outcome" publication bias in medical journals [
7,
15,
16]. Other groups have examined the likelihood of a subsequent publication of positive versus negative studies from submitted manuscripts, unpublished manuscripts, or manuscripts derived from abstracts presented at scientific meetings [
6,
8,
23-
25]. Easterbrook et al. retrospectively reviewed 487 projects submitted for publication and found a high odds-ratio in favor of publishing articles with a statistically significant outcome compared to those manuscripts which reported no difference between the study groups [
8]. Dickersin and Min performed an analysis of 198 NIH-funded trials [
26]. They reported that those trials with "significant" results were more likely to be published than studies with "non-significant" data, by an adjusted odds-ratio of 12.30 [
26].
In the orthopedic literature, Harris and coworkers reviewed all abstracts presented at the annual meeting American Academy of Orthopaedic Surgeons (AAOS) meeting in 1999 [
25]. They found that articles with a positive outcome and significant results were more likely to be published within the following years [
25]. Callaham and colleagues studied the publication pattern of abstracts presented at a 1991 major annual US research meeting in the field of emergency medicine [
24]. The authors found that articles with a positive outcome were more likely to be published in peer-reviewed journals than negative studies [
24].
Study funding patterns have furthermore been identified as independent factors that influence the publication bias. In particular, the presence of corporate/industrial funding was shown to be associated with a higher prevalence of published studies with positive findings. In the orthopedic literature, Leopold et al. found a significant association between commercially funded studies being more likely to have positive outcomes in publications, compared to unfunded studies [
22]. This industrial funding driven bias has been confirmed in other publications [
5,
8,
10]. Lynch et al. recently challenged this hypothesis by examining articles submitted to the American edition of the
Journal of Bone and Joint Surgery and concluded that publications from commercially funded studies were not more likely to present positive data than non-funded studies [
21]. However, this study was limited in design in that analysis was restricted to articles related to hip and knee arthroplasty during a seventeen-month period [
21].
Finally, there appears to be a significant bias against negative studies in newspaper reports of medical research [
17]. Thus, as for the scientist and clinician, the lay reader is also exposed to filtered information regarding the outcome of clinical studies.
The present study has a number of strengths and weaknesses. To our knowledge, it is the largest analysis of its kind in the medical literature. The 12 selected journals represent relevant sources of clinical knowledge, and are widely used as evidence-based decision-making tools in the fields of general, trauma, and orthopedic surgery. The algorithm used for stratifying articles is widely inclusive for all published original articles in the screened journals. The results were cross-checked and revisited in regular intervals by three different investigators and approved by the senior author. This allowed for a reliable, objective, and reproducible method of scoring.
Some drawbacks and limitations of this study must be addressed. Despite the uniform algorithm used for screening and classification of articles, there remains some degree of inherent inter-observer variability in the assessment of publications. The algorithm was developed by the authors for the present study and has not been externally validated by other groups. Furthermore, study analysis was limited to the last seven years in a limited number of journals in orthopedics and general surgery. Our results are neither representative for other surgical subspecialties and other fields of medicine, nor for papers published prior to the year 2,000.
A further drawback of this study is that we did not stratify between in vitro and experimental studies vs. clinical trials. A bias in the latter group is certainly associated with a higher risk for patients, since clinical treatment trials constitute the main basis for decision making in a clinical setting.
Another limitation is that published articles were assessed exclusively, without accounting for submitted and unpublished manuscripts. Thus, our study design does not determine whether the positive publication bias occurred at the level of manuscript submission or at the editorial decision-making level. The former notion would imply a bias towards a preferential submission of manuscripts with positive outcomes and statistically significant results; the latter would suggest a bias towards preferential publication of positive articles by peer-reviewers and editors. In this regard, Olson and colleagues revealed that the "positive-outcome-bias" does not occur at the editorial decision making stage, based on assessment of manuscripts submitted to the
Journal of the American Medical Association (JAMA) between 1996 and 1999 [
27].
Based on the large and encompassing analysis presented here, it is apparent that research reporting negative outcomes and/or statistically insignificant results is underrepresented in the surgical literature. The clinical implications of this trend are of concern due to the potential impact on patient care. The "positive-publication-bias" may alter the balance of the available evidence-based literature and may negatively affect recommendations and guidelines derived from systematic meta-analyses [
13,
20]. Recently, a new open-access online journal was launched which is devoted exclusively to publishing negative results in biomedicine [
28]. In our opinion, it is imperative to promote the submission and publication of studies with negative outcomes and insignificant results, in order to ensure a balanced availability of evidence-based data for clinical decision-making regarding the best treatment modality for our patients.
1
This article has been 
