|Home | About | Journals | Submit | Contact Us | Français|
This editorial announces requirements for reporting experiments in animals, cells, molecules, or other biological foci that we will term “preclinical” in this editorial. In order for reviewers, editors, and readers to better gauge the quality of research, journals often endorse reporting guidelines developed by consensus methods and promulgated by organizations focused on improving quality of research conduct and reporting, such as Enhancing the QUAlity and Transparency Of health Research (EQUATOR: www.equator-network.org). At their site, you will find consensus recommendations for reporting a wide variety of research designs including randomized clinical trials, observational studies, and systematic reviews. Included in that list, you will find recommendations for reporting preclinical studies as described in Animal Research: Reporting of In Vivo Experiments (ARRIVE).1 Based on the ARRIVE guideline, at Anesthesiology we will require all investigators to:
Following are descriptions of why we require these elements and details for each.
Imagine reading a clinical study where investigators gave patients a drug thought to speed recovery from sedation after anesthesia or a placebo. The description and results of the study in the article includes these statements:
Patients received either study drug (n=22) or placebo (n=23) and sedation was assessed using standard questionnaires and a battery of motor tasks known to be affected by sedation at 30 min after admission to the recovery room. The primary outcome was speed to perform a finger tracking task. Groups were compared using Student’s t-test with p<0.05 considered significant. Results showed that patients receiving the study drug recovered significantly faster after anesthesia by the primary outcome (p=0.048).
What the investigators actually did was the following:
The investigators had not performed these tests before, so decided to give the first 20 patients the placebo. The results were consistent with other studies of recovery from sedation, so they then gave the next 20 patients the active drug. They examined the results and noted that only one of the outcomes, speed of finger tracking, showed a large, but variable drug effect in the anticipated direction, but only at 30 min after surgery (measurements were actually made at 15, 30, 45, and 60 min after surgery). They used several statistical tests to compare groups for this outcome, and the one that was closest to statistical significance showed p = 0.09 after they excluded one patient receiving active study drug who had a longer time than the others. Based on these promising results, the investigators enrolled 2 more patients per group. This resulted in p = 0.06, so they enrolled 1 more patient per group and observed a statistically significant effect (p=0.048).; they rejected the null hypothesis and stopped the study.
Had the investigators completely reported their actual methods, it’s unlikely that a journal would accept such an article, or that a reader would put much stock in its results. To ensure adequate reporting for clinical trials, Anesthesiology requires submitted research to conform to the CONsolidated Standards of Reporting Trials (CONSORT) guidelines. Among many reporting elements, CONSORT requires including 1) an adequate description of the experiments to allow other researchers to replicate them, 2) report of the measures used to reduce bias, including whether and how random allocation and blinding methods were used, 3) how the sample size was determined, and 4) the data analysis plan. These are the same reporting elements that we will now require in all preclinical studies.
It took decades for clinical investigators to embrace these elements as critical to interpretable, reproducible, and actionable science. Given the extent that modern preclinical research lacks rigor regarding these elements, the reporting quality in such studies is “reminiscent of the situation in clinical research about 50 years ago”.2 Reporting these elements is only present a minority of the time (or not at all for sample size calculations), even in journals which strongly endorse the ARRIVE statement.3,4 The lack of translation of reporting quality to preclinical research may reflect many causes, but the consequences of poor reporting can be readily observed. The lack of reporting rigor may underlie the inability of independent industry laboratories to replicate a majority of landmark studies from academic laboratories performing cancer, cardiovascular, and stroke research.2,5,6 Failure of clinical translation and of replication of preclinical research were cited by leaders of the National Institute of Neurologic Diseases and Stroke7 and the National Institutes of Health8 when they called on journals, investigators, and funders to improve education in good scientific design and in transparent reporting of essential research design elements.
Authors are encouraged to review the full ARRIVE guidelines1 (in addition to the citation, they can be directly accessed at www.nc3rs.org.uk/arrive-guidelines) prior to submission of preclinical studies to Anesthesiology. However, the following items will be particularly scrutinized in research submissions.
As noted, despite journal endorsement of these and other elements of the ARRIVE guidelines for reporting preclinical research, articles in these journals report the elements only a small minority of the time. Furthermore, there has been little improvement in reporting practices over the past 3 years and little difference between journals with high or low impact factors.3,4 For the past several years Anesthesiology has scanned all clinical trials with custom designed software to identify elements of CONSORT which are not included, and we will do the same for preclinical research for these elements of ARRIVE. The goal of these efforts is not to reduce the amount of preclinical research we publish or to discourage authors from considering Anesthesiology for publication of their preclinical research. Rather, the goal of these efforts is to enhance trust by our readers in the quality of the science we publish, and to enhance trust by investigators that this published work is more likely to replicated and perhaps translated into improved care of patients.
Supported in part by grant R37-GM48085 from the National Institutes of Health, Bethesda, MD
Competing interest: Dr. Eisenach is the Editor-in-Chief of Anesthesiology and his institution receives salary support from the ASA for this position. Dr. Houle is the statistical Editor of Anesthesiology and his institution receives salary support from the ASA for this position.
James C. Eisenach, Department of Anesthesiology, Wake Forest University School of Medicine, Winston-Salem, NC.
David S. Warner, Duke University School of Medicine, Durham, NC.
Timothy T. Houle, Department of Anaesthesiology and Critical Care, Massachusetts General Hospital, Harvard University Medical Center, Boston, MA.