Journal peer review is a process in which the work of an academic is subject to an examination by other experts in the same field [21
]. Although the first peer review in biomedical journals dates back to 1731 with the publication of “Medical Essays and Observations” by the Royal Society of Edinburgh [14
], the process itself and perception from biomedical academics has scarcely been studied for the past three centuries. In this study, 1,340 academics working for high-ranking universities from 26 countries shared their views on peer review in biomedical publication. The results of this survey do not support our primary hypothesis that most biomedical academics would agree that journal peer review is fair, transparent and scientific. Our survey reports that slightly less than half of respondents agreed that the process is fair and scientific, and only about one-quarter agreed that peer review is transparent. Our findings also suggest that peer review, although being the most prevalent system adopted by biomedical journals to select articles for publication, is perceived differently by academics regarding its fairness, scientific credibility and transparency. Our findings correspond to concerns raised by other researchers that there is a need to improve clarity and transparency of biomedical journal peer review [22
]. This is perhaps not surprising as peer review depends on unquantifiable perceptions [22
]. Decisions on acceptance or rejection of a article should be based on objective selection criteria (e.g. quality scores for an article) and broad criteria (e.g. importance, usefulness, relevance, methods, ethics and accuracy) [22
]. Such criteria should be made transparent to authors, editors, reviewers and readers [23
]. Measures such as publishing reviewers’ reports, authors’ responses and editors’ comments would increase the transparency of the peer review process [24
]. Although some journals have introduced an open peer-review process whereby the identity of authors and peer reviewers are published with the intention to improve transparency [13
], our respondents generally did not support an open peer-review process as about two-third of respondents agreed that reviewers should remain anonymous. Similarly, about two-thirds of respondents agreed that authors should remain anonymous to reviewers. Double anonymity may provide a safe place for reviewers and authors to exchange frank and sensitive comments [12
]. Furthermore, this may have value in concealing the gender of the authors as female gender was a significant factor associated with perception of unfairness in journal peer review. Our findings correspond with the report published by the Publishing Research Consortium (2008) which states that double-blind review was preferred and seen as most effective [19
] although single-blind review is more common in biomedical journals.
The proportion of native English-speaking respondents agreeing that journal peer review is fair was significantly higher than for non-native respondents. It is not surprising because the frequencies of non-native English-speaking respondents encountering personal attacks and imposition of unnecessary references were higher compared to native respondents. Nevertheless, there was no significant difference between the two groups on their views of transparency. Interestingly, the proportion of non-native English speaking respondents who agreed that journal peer review is scientific was significantly higher than for native respondents. This result may reflect a higher expectation about journal peer review adopted by native English-speaking biomedical academics. This interpretation is further supported by the results of this survey, as there was a significantly higher proportion of native English-speaking respondents who urged the editors to give an article a fair hearing without personal bias and to screen for unfair comments, for the reviewers to declare COI, as well as for the journals to establish an appeal system.
Our survey classified the respondents according to type of academic and medical speciality. The proportion of clinicians who agreed that biomedical journal peer review is fair was significantly higher than for basic scientists and clinician-scientists, thus contradicting our hypothesis. A possible explanation for this result is that the proportion of basic scientists and clinician-scientists who reported the occurrence of peer review misconduct such as personal attacks and imposition of unnecessary references was significantly higher than clinicians. Academics involving in basic science research may need more safeguards from journal peer review misconduct when publishing their results. The result concerning our final hypothesis, that most academics would support the establishment of an appeal system within the journal to handle unfairly rejected articles, is supported. About 7 in 10 respondents agreed to the establishment of an appeal system.
The results of this study may generate further discussion to improve biomedical journal peer review. The roles of editors involve assessing the external validity of the article with considerations of the appropriateness of the research to the readership of the journal and appointing experienced experts from the same field to review articles [21
]. It is interesting to note that about one-third of respondents provided a neutral view on whether editors should give an article a fair hearing and avoid personal bias (37.8%) and screen for unfair reviewers’ comments (33.3%). The proportion of subjects remaining neutral on the editor’s role is almost twice that remaining neutral on establishing an appeal system (17.5%). These findings suggest that the respondents do not have a high expectation concerning editors, because editors are mostly perceived as sending articles out for peer review [9
The roles of reviewers include assessing the internal validity of an article, such as accuracy and correctness with considerations of the study design, research methodology, statistical analysis and interpretation of results. Reviewers are also expected to provide constructive feedback to enhance scientific quality of the articles and stimulate further consideration by the authors [21
]. Findings from our survey confirm that misconduct such as breach of confidentiality and unauthorised uses of article information among reviewers was relatively uncommon. Nevertheless, frequency of encountering personal attacks in reviewers’ comments and frequency of imposition of unnecessary references by reviewers were significant factors associated with perception of unfairness in journal peer review. These findings highlight the importance of publication ethics guidelines which should define and prohibit personal attacks by reviewers. Furthermore, editors should investigate such incidents. A structured and standardized training course for reviewers concerning proper criteria for prompt reviewing may avoid some the pitfalls [14
] and enhance the quality of articles. While it is difficult to define unnecessary references, reviewers should try to avoid imposing upon authors to quote the reviewers own publications as it could lead to potential COI. The COI given by reviewers need to be taken seriously in biomedical publication due particularly to the influence of pharmaceutical companies. Not surprisingly, about 8 in 10 respondents disagreed that the reviewers should be exempted from declaring COI. It is interesting to note that a small number of respondents suggested the offering of rewards or financial incentives to reviewers and the report from Peer Review Consortium (2008) also provided limited support for this proposal [19
]. Caution is needed in reacting to this suggestion. Firstly, it could increase the production costs of the journal and the costs would be transferred to the subscribers. Secondly, it may lengthen the review times and increase rejection rates in journals which do not offer financial incentives.
To the best of our knowledge, this study is one of the few examining the views on and experiences with biomedical journal peer review among academics. The study has collected important information which identifies directions for improvement of journal review systems. Nevertheless, our study has some limitations. First, it was not based on random selection of medical schools and academics, it mainly focused on the respondents from high-ranking universities with extensive publication experience. Our study may lead to selection bias because we could not identify academics working in low-ranking universities who may encounter more difficulty in publication and it is difficult to set criteria to define ‘low ranking’. As a result, our findings cannot be generalized to academics with relatively less publication experience, academics working in medical schools which only focus on teaching and academics working in a research institute not affiliated with a university. Second, the medical schools were selected based on the THE-QS World University Rankings which have been criticized for their construct validity and possible selection biases of peer reviewers [26
]. Third, the present survey method may have under-represented those respondents from Africa, South America and southern Asia. Biomedical research in Mainland China has increased substantially in the 21st Century [27
]. Fourth, the response rate (4.8%) was slightly lower than the study conducted by the Publishing Research Consortium (7.7%) in 2008. The low response rate could be due to the fact that academics from high ranking universities are usually successful in getting their manuscripts published and they are less inclined to participate in our survey to express their views. Fifth, the online questionnaire was not able to explore further experiences encountered by participants and focus group interviews would be useful. Furthermore, we did not assess respondents’ views on new review methods such as post-publication peer review and multi-stage open peer review. Jefferson et al. (2002) proposed a randomized controlled study to compare the outcomes of articles undergoing the conventional peer review and an alternative type of assessments [27
]. The multi-stage open peer review is a possible alternative method which should be studied further. The multi-stage open peer review involves three stages [28
]. The first involves rapid pre-screening and publication as a discussion paper in an online forum. The second involves an 8-week interactive public discussion between the authors and the scientific community. The discussion would be published online and the names of referees may be published. The third stage involves the traditional peer review. The multi-stage open peer review may further enhance the transparency of peer review and the quality of articles as compared to the conventional method.