Search tips
Search criteria 


Logo of jrsocmedLink to Publisher's site
J R Soc Med. 2000 April; 93(4): 164–167.
PMCID: PMC1297969

A randomized controlled study of reviewer bias against an unconventional therapy.


A study was designed to test the hypothesis that experts who review papers for publication are prejudiced against an unconventional form of therapy. Two versions were produced (A and B) of a 'short report' that related to treatments of obesity, identical except for the nature of the intervention. Version A related to an orthodox treatment, version B to an unconventional treatment. 398 reviewers were randomized to receive one or the other version for peer review. The primary outcomes were the reviewers' rating of 'importance' on a scale of 1-5 and their verdict regarding rejection or acceptance of the paper. Reviewers were unaware that they were taking part in a study. The overall response rate was 41.7%, and 141 assessment forms were suitable for statistical evaluation. After dichotomization of the rating scale, a significant difference in favour of the orthodox version with an odds ratio of 3.01 (95% confidence interval, 1.03 to 8.25), was found. This observation mirrored that of the visual analogue scale for which the respective medians and interquartile ranges were 67% (51% to 78.5%) for version A and 57% (29.7% to 72.6%) for version B. Reviewers showed a wide range of responses to both versions of the paper, with a significant bias in favour of the orthodox version. Authors of technically good unconventional papers may therefore be at a disadvantage in the peer review process. Yet the effect is probably too small to preclude publication of their work in peer-reviewed orthodox journals.

Full Text

The Full Text of this article is available as a PDF (96K).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Ernst E, Resch KL, Uher EM. Reviewer bias. Ann Intern Med. 1992 Jun 1;116(11):958–958. [PubMed]
  • Nieminen P, Isohanni M. Bias against European journals in medical publication Databases. Lancet. 1999 May 8;353(9164):1592–1592. [PubMed]
  • Ernst E, Saradeth T, Resch KL. Drawbacks of peer review. Nature. 1993 May 27;363(6427):296–296. [PubMed]
  • Tattersall MH, Thomas H. Recent advances: oncology. BMJ. 1999 Feb 13;318(7181):445–448. [PMC free article] [PubMed]

Articles from Journal of the Royal Society of Medicine are provided here courtesy of Royal Society of Medicine Press