|Home | About | Journals | Submit | Contact Us | Français|
We investigated how often journal articles reporting on human HIV research in four developing world countries mention any institutional review boards (IRBs) or research ethics committees (RECs), and what factors are involved.
We examined all such articles published in 2007 from India, Nigeria, Thailand and Uganda, and coded these for several ethical and other characteristics.
Of 221 articles meeting inclusion criteria, 32.1% did not mention IRB approval. Mention of IRB approval was associated with: biomedical (versus psychosocial) research (P=0.001), more sponsor-country authors (P=0.003), sponsor-country corresponding author (P=0.047), mention of funding (P<0.001), particular host-country involved (P=0.002), journals having sponsor-country editors (P<0.001), and journal stated compliance with International Committee of Medical Journal Editors (ICMJE) (P=0. 003). Logistic regression identified 3 significant factors: mention of funding, journal having sponsor-country editors and research being biomedical.
One-third of articles still do not mention IRB approval. Mention varied by country, and was associated with biomedical research, and more sponsor country involvement. Recently, some journals have required mention of IRB approval, but allow authors to do so in cover letters to editors, not in the article itself. Instead, these data suggest, journals should require that articles document adherence to ethical standards.
Increasing amounts of research in medicine and public health are being conducted in the developing world, yet critical questions remain regarding whether and to what degree these studies adhere to ethical standards. As described more fully below, at present, approximately 5% of the world’s medical journals have stated that they follow guidelines of the International Council Medical Journal Editors (ICMJE) that stipulate that the authors include a statement concerning institutional review board (IRB) review either in the article or in a cover letter to the editors.1 However, several studies have suggested that published articles do not always include this information.2 Many questions remain concerning whether studies conducted in the developing world have in fact obtained IRB review, and whether published articles reporting on that research in fact mention whether IRB approval was obtained. It is important that authors include this information, since science and public trust of science are crucial, especially among groups who may be wary of research.3 Without including this information, readers have no way of knowing whether ethical standards were in fact followed. Given scandals – both documented and alleged – involving violations of patients’ rights in both the developed and the developing world,4 such transparency and assurance are essential. Thus, if a substantial proportion of articles are found currently to fail to disclose this information, arguably, consideration should be given to requiring that all journals mandate that all published articles reporting on human subjects research mention, in-print, whether IRB approval was or was not obtained (and if not, why not).
The amount of research in developing countries continues to rise. More than half of all FDA-regulated clinical trials in 2010 were conducted outside of the US.5 In 2006, of active FDA-regulated researchers, 41% worked outside the US.6 In 2007, of industry funded phase-III trials on Clinicaltrials.gov, only 44% were based in the US.7 Such research may stem from global health priorities, and/or permit researchers to increase sample sizes for studies at lower costs. 8
Yet the globalization of this research raises important ethical questions, such as those concerning the possible exploitation of vulnerable populations.9 International studies may require approval by more than one IRB or research ethics committee (REC), but we know of only one study that has compared IRBs in the US to IRBs or RECs in the developing world.10 Documentation of having obtained IRB or REC approval is important, as it is the ‘final check in ensuring the highest scientific and ethical standards and a necessary step in protecting research subjects and maintaining public trust’ in the process of medical research.11
However, crucial basic information, such as how often research conducted in the developing world is in fact reviewed by an IRB or REC, and, in what way (that is, to what extent and with what quality of review) remain unknown. Unfortunately, to assess these questions requires substantial resources, of time and money, which have proven to be insurmountable obstacles. Yet, there are vital initial questions in this realm that can be readily addressed and, as described below, possibly shed light on critical aspects of these issues – specifically, exploration of how often journal articles reporting on research conducted in various developing countries even mention IRB/REC approval, and what factors may be involved.
The International Committee of Medical Journal Editors (ICMJE), in 1997, altered its requirements for IRB approval reporting in biomedical journals, necessitating that authors indicate if the research adhered to the ethical standards of the appropriate institutional or national body.12 Over 700 journals state online that they follow ICMJE requirements. Editors from 12 journals form the membership itself. A study of the 103 English-language journals included in the Abridged Index Medicus showed IRB approval requirements increased between 1995 and 2005 from 45% to 76% and was associated with higher impact factors of the journals.13
A few articles in the New England Journal of Medicine, the Journal of the American Medical Association and elsewhere have begun exploring whether published journal articles indeed mention IRB approval,14 but many questions about it remain. Just after 1997, in industrialized countries, the failure to report IRB approval and informed consent appeared to decrease somewhat, but then appeared to increase to pre-1997 levels.15 Previously, for example, failure to mention IRB approval was 100% in 1978 among 24 resuscitation research articles16 and 71% among clinical trial articles published in ‘top’ medical journals between 1993 and 1995.17 From immediately before 1997 (1995-1996) to immediately after (1998-1999), failure to mention IRB approval in articles published in the top five medical journals whose editors are members of the ICMJE fell from 31% to 18%.18 However, a 2003 follow up study showed that failure to report IRB approval rose again to 31% of articles in these same top journals.19 Failures to report IRB approval have been documented in not only ‘top’ medical journals,20 but specialty journals, with rates of 29% in 2001 and 24% in 1994 in anesthesia and critical care medicine, respectively.21 In anesthesiology journals, the type of study (for example, trials versus observational studies), and the specific journal were significantly associated with documentation of IRB approval.
Yet this research on documentation of IRB approval has focused on studies conducted in industrialized countries, and has explored to a far less degree research conducted in the developing world. We found only two studies that explicitly examined articles reporting on research in the developing world. A 2008 study reported that 62% of Sri Lankan research articles did not mention IRB/REC approval.22 Yet, this study did not examine other factors that may be associated with reporting of IRB approval – for example, author affiliations or type of journal.23 A 2008 study found that approximately 70% of research papers published in two Indian pediatrics journals did not report IRB/REC approval. But this study had several limitations, studying only two journals from the same country, and did not distinguish between primary investigators from India as opposed to elsewhere in the world.24 In one study of critical care journals, articles on research conducted in the US compared to elsewhere in the world did not differ in mentioning of IRB approval,25 though the authors of this study did not identify the locations outside the US (for example, developed or developing countries), and thus did not address the particular ethical issues faced in developing world research.
In short, we have found no study that examined how often published research articles mention ethical issues in more than one developing country, or assessed several critical factors that may be involved.
Clearly, what is most important is whether IRB/REC approval occurs, not simply whether it is documented in the published article. On the one hand, a critic could potentially argue that published articles do not need to attest to such review, and that nothing more could or should be done to increase the proportion of articles that mention such review. The National Institutes of Health (NIH), Food and Drug Administration (FDA), and World Health Organization (WHO) all state that they require a statement and/or documentation of IRB approval. Hence, one might allege, whether published articles mention it is utterly irrelevant and inconsequential.
Yet, on the other hand, importantly, not all research conducted in the developing world is funded by, and/or submitted to, the NIH, FDA or WHO. Rather, many research studies may be investigator-initiated, funded by private foundations, other governmental agencies, or have little, if any external funding. However, such studies nonetheless involve human subjects whose rights need to be protected.
Moreover, the effectiveness of any policy or guideline depends on not only whether it is enacted by a bureaucratic entity, but whether and how it is implemented by multiple stakeholders, and to what extent and how they in fact follow it. Numerous challenges exist in importing IRBs/RECs into the developing world,26 and critics have questioned whether Western Ethical Imperialism is occurring.27 Some researchers may simply want to check a box that IRB review has or has not occurred, as if it were merely an administrative formality. Yet, the complex political, social, cultural, economic and historical contexts of diverse regions in developing world countries make it imperative to try to shed some light, if possible, on the issues and factors that may be involved.
It is wholly unknown what proportion of published studies are in fact approved by an IRB, and what the extent and quality of the IRB reviews are – for example, who reviews the protocol (that is, a private or for profit IRB, a host country and/or a sponsor country IRB), and of what the reviews consist. IRBs differ significantly in their decisions and interpretations of basic ethical principles.28
Furthermore, researchers do not always follow ethical principles and guidelines. Scandals involving violations of research ethics have occurred in both the US and the developing world.29 Hence, ensuring that research meets the highest possible ethical standards is vital. In this effort, public transparency and documentation are crucial. A paper in Nature entitled ‘Scientists Behaving Badly’ reported that up to 33% of US researchers admitted to engaging in ‘questionable behavior that threatens the integrity of science’.30 Of respondents, 7.6% reported ‘circumventing certain minor aspects of human-subject requirements’.31 The percentage of researchers ‘behaving badly’ in the developing world remains unknown. Hence, explicit documentation of compliance with ethical standards in published articles can be helpful, potentially enhancing such compliance.
In addition, it is crucial to recognize that 95% of medical journals have not signed onto ICMJE’s guidelines. Though 694 journals have listed themselves on ICMJE’s website as following its guidelines, these publications constitute only approximately 5% of the total number of active medical journals in the world.32 Other journals may also follow these ethical standards, but the nature and implementation of these journals’ ethical requirements may vary widely.33 It is also unknown whether journals that have signed onto ICMJE guidelines in fact follow these guidelines, and if so, how and to what degree.
Given these challenges in research ethics, requiring that published articles mention IRB approval may be important, in and of itself, in allowing readers to judge for themselves the scientific and ethical standards followed by the investigators, and in enhancing the likelihood that researchers will indeed obtain appropriate IRB approval.
Clearly, several broad questions remain about IRB review and will require large scale research efforts to address fully. In the meantime, any data that might be available and shed any light on these issues, in any way, may be helpful.
Thus, we decided to explore how often articles on research conducted in four developing world countries mention IRB/REC approval (referred to below as ‘IRB approval’), and what factors may be associated with differences that emerge. We focused on HIV, since it is a vital realm of research in both developed and developing countries, as the pandemic affects approximately 38.6 million people worldwide.34 This research examines vulnerable and stigmatized populations – making human subject protections especially important – and has triggered controversies.35
We have explored elsewhere, whether articles reporting research that was sponsored by one country, but conducted in another country, had listed dual or single IRB/REC approval, and if so, from what country or countries.36 But critical questions emerge as to whether research conducted in the developing world more broadly, regardless of the source of sponsorship, if any, lists any such IRB/REC approval, and what factors might be involved. Such articles often do not list sources of funding,37 but how often they mention any IRB/REC review is unknown.
As we have described elsewhere concerning other research questions,38 we examined all articles published in 2007 identified through the Medline database, using two search terms: i) HIV and ii) the name of one of four countries (India, Nigeria, Thailand and Uganda). We chose these countries since they have among the highest HIV prevalence in their respective geographic regions and the largest numbers of US-sponsored HIV clinical trials.39 We limited the search to papers with human subjects, written in English. We included all articles available online through our university medical library, which carries 117,264 serials.40 To survey original research that involves human subjects specifically recruited to participate in a study, we included only original research articles, and excluded review articles, meta-analyses, letters, communications, brief reports, case reports, retrospective chart reviews, news articles and data obtained from public databases (that is, data collected for purposes other than research – such as state records and national epidemiological surveillance programs). As this research did not involve human subjects, it was exempt from IRB review at Columbia University.
Two research assistants (RAs), closely supervised by the senior author, independently coded the articles meeting the inclusion criteria. They first, each, independently coded ten articles, consisting of at least two from each country. This team then developed a coding manual, and the two RAs subsequently coded ten additional articles independently and compared the results, revising the manual as necessary. They coded further sets of ten articles independently, comparing results and discussing any disagreements until they reached 100% consensus. Using the final codebook, all articles were then recoded. The articles were coded for factors in six main categories concerning ethical and other (that is, study, participant, funding, authorship and journal) characteristics of the article. We developed one composite binary variable of sponsor-country involvement indicating articles in which a sponsor-country was involved in any way (that is, a sponsor-country author, funder or IRB was mentioned).
In this analysis, we selected mention of any IRB approval (that is, that the research was reviewed or exempted) as the primary outcome variable, since it reflected ethical standards most completely. To examine the association between mention of IRB approval and other characteristics of the article we used chi-square tests. We employed logistic regression analysis to further identify important associations of mention of IRB approval. All variables associated with mention of IRB approval at P<0.05 in the chi-square tests were entered in the logistic regression model.
As shown on Table I, of the 590 articles found on Medline searches, 221 (37.5%) met inclusion criteria. As shown in Table II, of the 221 articles, 32.1% did not mention IRB approval in any way; 28.1% did not report any informed consent, 92.8% did not mention either financial compensation or other non-financial benefits to participants, and 80.1% did not mention any conflict of interest (Table II). Non-mention of IRB approval varied by country (P=0.002), with rates as follows: Nigeria (50.0%, n=17), India (40.5%, n=32), Thailand (25.6%, n=11), and Uganda (16.9%, n=11).
We also examined those articles that involved sponsorship from the developed world (n=155), and that thus should have obtained IRB approval from more than one country. Yet, only half of such articles (51.0% [n=79]) did so, while 25.8% (n=40) did not mention IRB approval at all and 20.7% (n=32) stated that IRB approval was obtained only from one country (18.1% [n=28] and 2.6% [n=4] from the host or sponsor, respectively).
We examined associations between reporting of IRB approval and independent variables that reflected other (that is, non-ethical) characteristics of the article. As shown in Table III, studies that did not mention IRB approval were significantly more likely to: be only psychosocial (as opposed to biomedical) (P=0.001), not mention the source of funding (P<0.001), have a host-country corresponding author (P=0.047), have no sponsor involvement (P=0.002), and be in journals with editors from host countries (P<0.001), that are not affiliated with ICMJE (P=0.003). Non-mention of IRB approval is also associated with percentage of host authors (P=0.003).
We entered into a logistic regression, all eight variables that were significant in chi-square analyses. As shown in Table IV, the forward and backward methods each identified threesignificant characteristics: research being only biomedical, mention of funding and journal editors only from sponsor countries.
Approximately one-third of articles reporting on human subject research in the developing world failed to mention whether IRB approval was obtained. This percentage varied between countries (from 17% in Uganda to 50% in Nigeria). This rate of one-third is less than the rates found in the two small studies previously conducted, of articles from the developing world (Sri Lanka41 and India42), but approximately the same as that found in articles from the developed world.43 We examined several factors that other researchers have not previously investigated for associations with the mention of IRB approval, and found that several of these variables were significant: being a biomedical (versus only psychosocial) study, mentioning any funding, research being funded or conducted at least in part by sponsor country researchers (that is, sponsor country involvement), being published in a journal associated with the ICMJE, and having that journal editorial board consisting of members only from sponsor countries. Except for research being psychosocial, these other factors all reflected in some way involvement of sponsor countries. Surprisingly, certain factors (for example, involvement of vulnerable populations, and the study being a clinical trial) were not significant. Of those articles that had both host and sponsor country authors, and that thus appeared to require IRB approval in both host and sponsor countries, less than half mentioned that they in fact had it. This dual approval is critical to ensure that appropriate ethical guidelines are being followed, and to avoid the possibility of researchers adopting lower standards. Yet critical questions arise as to how often such dual country approval in fact occurs.
This study has several potential limitations. We only examined journals available online. Smaller journals may not all thus be available. Yet, even among these larger journals, we still found relatively high rates of failure to mention IRB approval. Thus, these rates may in fact be a ‘floor’, and even higher rates may in fact exist. It is also possible that articles that did not mention IRB approval nonetheless obtained it. We did not contact the authors of these articles to determine whether they in fact did so, though future research can pursue this possibility. However, as discussed earlier, it is important that mention of IRB approval be included in these articles themselves since transparency to readers and the broader public is critical in and of itself in order to assure and communicate that research meets the highest possible scientific and ethical standards.
This paper serves, in part, as an important first step in helping to develop a broader area of research, and to begin to tease out several critical questions. These data highlight crucial needs to explore whether IRB/REC review occurs, and if so, where, who conducts it, what training they have, and of what the review consists. Few such studies have been conducted in the US. Indeed, we know of no published study of how US IRBs actually make decisions. Given the rapidly rising amount of research in the developing world, such research is urgently needed. In the meantime, other data, such as that presented here, can be valuable in shedding light on aspects of these issues, to highlight challenges and areas where problems may emerge.
Several factors and issues that may be involved here can be further analyzed in future research. For example, the finding that psychosocial research was less likely to mention IRB approval is consistent with evidence that many psychosocial researchers dismiss the importance of obtaining IRB approval and thus resist submitting protocols to IRBs.44 Indeed, one Indian research article examined in the present study that included in-depth interviews and accessed HIV and other biological results even stated that ‘ethical review clearance was not necessary, as this was research carried out by students and did not involve human beings.’45 As an additional issue that arose here and can be further probed in the future, while non-mention of IRB approval was associated with the percentage of host authors, this association appears complex and possibly bimodal. An author who did not include any host-country co-authors may also have been less sensitive to cross-cultural concerns and, as such, obligations to obtain and report on ethical review. These findings are disturbing since they suggest that authors may think that IRB approval is not important enough to mention or possibly obtain. Protection of human subjects and documentation of protection remain critical, given possible harms, including violations of privacy and confidentiality and stigmatization of certain groups. These areas need to be further explored in future research with larger samples.
We also found that research articles underreport informed consent. It was not always clear whether consent for medical care mentioned in articles also included consent for research. Though we assumed that it did, this may not always have been the case. Hence, lack of informed consent may occur more frequently than we identified here.
These data thus have several critical policy implications. Importantly, ICJME guidelines appear to leave it to the journal whether to require mention of IRB approval either in the article itself or in a separate correspondence (for example, a cover letter) that the author submits to the journal.46 Clearly, however, a major problem then occurs: if an article does not mention IRB approval, the reader does not know either whether none has been obtained or whether approval was obtained but simply not mentioned in the manuscript. This ambiguity is worrisome because it is vital that scientific and lay communities be able to assess articles as fully as possible, including whether human participants were adequately protected, and if so, how. Hence, documentation that ethical standards were followed is crucial.
Journals’ association with the ICMJE increased the likelihood of mention of IRB approval, but did not guarantee it, with 11.1% of articles still failing to do so. Importantly, our data thus suggest that the ICMJE should consider changing its policies to make authors explicitly state in articles (rather than only in a cover letter, if at all) whether or not IRB approval was obtained or necessary.
While Rowan-Legg, et al. have reported that 76% of journals require IRB approval, and 86% of these require a statement of IRB approval within the manuscript,47 it is not clear how often journals in fact uphold these standards. Our data suggest that the listing of a journal as adhering to ICMJE requirements does not guarantee that the editors, and therefore the authors will necessarily follow it. It is also not clear how many journals require mention of informed consent and how such consent is to be reported.48
Efforts are needed to remedy these problems by making authors, editors, and potentially funders more aware of these issues. In addition, more journals should be encouraged to agree to officially follow ICMJE guidelines. The fact that 95% of all scientific journals do not do so is itself alarming. The barriers to such affiliation need to be explored and addressed.
Hence, there may be advantages to journals all signing onto a mutually agreed upon set of guidelines. Perhaps the involvement of an international organization such as UNESCO, or other international professional scientific organizations can help promulgate use of a universal standard for documentation of research participant protection, through educational or other efforts. As science expands ever-more globally, ethical conduct of scientific research is increasingly vital in ensuring public trust in science throughout the world.
This study was supported by a grant from the National Library of Medicine (5 G13 LM009996-02 – ‘IRBs: Ethics, Context and Interpretations’) and a grant from the National Institutes of Health (5R01HG04214 – ‘Views and Approaches to Research Integrity among IRBs’).
Robert Klitzman is a Professor of Clinical Psychiatry at the Columbia University Department of Psychiatry. He is also the Director of the Ethics and Policy Core of the HIV Center for Clinical and Behavioral Studies at the New York State Psychiatric Institute and Department of Psychiatry and Director of the Masters of Bioethics Program at Columbia University. He conceived of and directed the study. Kelly Kleinert is a medical student at the Columbia University College of Physicians and Surgeons. She was involved with the design and data collection of the study. Hoda Rifai-Bishjawish is a graduate of the Columbia University Fu Foundation School of Engineering and Applied Science with a major in Biomedical Engineering. She is currently a Fulbright Scholar conducting bioethics research in Syria and will be attending Columbia University Law School. She was involved with the design and data collection of the study. Cheng-Shiun Leu is Assistant Professor of Biostatistics at the Columbia University Mailman School of Public Health. He provided statistical support for the study.