PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of iaiPermissionsJournals.ASM.orgJournalIAI ArticleJournal InfoAuthorsReviewers
 
Infect Immun. Mar 2012; 80(3): 1300.
PMCID: PMC3294657
Likelihood of False-Positive Results in High-Impact Journals Publishing Groundbreaking Research
Anurag Agrawalcorresponding author and Abhay Sharma
CSIR-Institute of Genomics and Integrative Biology Delhi, India
R. P. Morrison, Editor
corresponding authorCorresponding author.
Address correspondence to Anurag Agrawal, a.agrawal/at/igib.in.
A recent editorial in this journal has brought retracted science to the fore and provided the “retraction index” as a novel measure of frequency of such articles (2). In their analysis using this measure, the authors found that the probability of retraction of an article published in a higher-impact journal is greater than that published in a lower-impact journal. This is intriguing and in accordance with data previously published on a retraction tracking website (http://pmretract.heroku.com). Here, we would like to point out that apart from various reasons outlined in the editorial, including higher readership and scrutiny, simple Bayesian logic also predicts that the articles published in high-impact journals stand a higher chance of incorporating false-positive results and thereby are more likely to be retracted.
In our understanding of publishing practices, the highest echelon of scientific journals is likely to prioritize publication of extremely novel findings that contradict current thinking; incremental but substantive work that builds upon established facts is more likely to find its way toward respectable but less glamorous society journals. Yet the strength of the data is not likely to be very different, given common high standards. In Bayesian logic, the posterior odds of a hypothesis are equal to the prior odds multiplied by the Bayes factor. The Bayes factor is data dependent and is probably not very different between publications from two quality-oriented journals. However, the prior odds of the results being false positives will always be much higher for work that challenges existing notions. The posterior odds of the results being false are also therefore likely to be much higher, in line with the prior odds. In simpler language, if it appeared unlikely before it was published, it probably still is after it is published (1, 3). That such work appears in the most-cited journals is, at least in part, inherent to the publication process, with the most novel, most exciting, most controversial, most discussed, and therefore most cited work being published in such places. Interestingly, it also emerges from the same logic that while the highest-impact glamour journals may be the best place to read about the new ideas that will change current paradigms, the most reliable and reproducible work may be published elsewhere.
Footnotes
For the author reply, see doi:10.1128/IAI.06330-11.
REFERENCES
1. Duncan LE, Keller MC. 2011. A critical review of the first 10 years of candidate gene-by-environment interaction research in psychiatry. Am. J. Psychiatry 168:1041–1049. [PMC free article] [PubMed]
2. Fang FC, Casadevall A. 2011. Retracted science and the retraction index. Infect. Immun. 79:3855–3859. [PMC free article] [PubMed]
3. Ioannidis JPA. 2005. Why most published research findings are false. PLoS Med. 2(8):e124. [PMC free article] [PubMed]
Articles from Infection and Immunity are provided here courtesy of
American Society for Microbiology (ASM)