|Home | About | Journals | Submit | Contact Us | Français|
Quality of reporting (QR) in industry-funded research is a concern of the scientific community. Greater scrutiny of industry-sponsored research reporting has been suggested, although differences in QR by sponsorship type have not been evaluated in weight loss interventions.
To evaluate the association of funding source and QR of long-term obesity randomized clinical trials.
We analyzed papers that reported long-term weight loss trials. Articles were obtained through searches of MEDLINE, HealthStar, and the Cochrane Controlled Trials Register between the years 1966–2003. QR scores were determined for each study based upon expanded criteria from the Consolidated Standards for Reporting Trials (CONSORT) checklist for a maximum score of 44 points. Studies were coded by category of industry support (0=no industry support, 1= industry support, 2= in kind contribution from industry and 3=duality of interest reported). Individual CONSORT reporting criteria were tabulated by funding type. An independent samples t-test compared differences in QR scores by funding source and the Wilcox-Mann-Whitney test and generalized estimating equations (GEE) were used for sensitivity analyses.
Of the 63 RCTs evaluated, 67% were industry-supported trials. Industry funding was associated with higher QR score in long-term weight loss trials compared to non-industry funded studies (Mean QR (SD): Industry = 27.9 (4.1), Non-Industry =23.4 (4.1); p < 0.0005). The Wilcox-Mann-Whitney test confirmed this result (p<0.0005). Controlling for the year of publication and whether paper was published before the CONSORT statement was released in a GEE regression analysis, the direction and magnitude of effect was similar and statistically significant (p=0.035). Of the individual criteria that prior research has associated with biases, industry funding was associated with greater reporting of intent-to-treat analysis (p=0.0158), but was not different from non-industry studies in reporting of treatment allocation and blinding.
Our findings suggest that efforts to improve reporting quality be directed at all obesity RCTs irrespective of funding source.
“Research studies in biomedical journals are increasingly scrutinised, not only for their scientific findings and clinical and public health implications, but also because of … concerns about misleading reporting of industry-sponsored research.”1
The above quotation, the opening line in an editorial in the Journal of the American Medical Association (JAMA), evinces strong concern regarding potential influence of industry-sponsorship in the literature. The editors of JAMA, in reaction to the concern, suggested policies of extra scrutiny for industry-funded research. In response, an editorial in BMJ decried this as unfair and absurd.2
For the scientific community and society at large to benefit maximally from biomedical research, the quality and integrity of each step in the research process must be maximized. Such steps include: (A) Selection of relevant questions; (B) Design of research suitable to address those questions; (C) Rigorous execution of the design; (D) Faithful recording of the data; (E) Appropriate statistical analysis of the data; (F) A decision to publish a report of study; (G) Clear, accurate, and thorough reporting of the study; and (H) Reasonable “second-level” dissemination and interpretation of the findings in press-releases, mass media coverage, scholarly reviews, expert testimony, blue-ribbon panels, and other venues. In each of these steps, there is an opportunity for some factors to inappropriately exert influence.
One such factor that is frequently discussed is financial duality (aka ‘conflict’) of interest (DOI).3 Indeed, at least three papers4–6 suggest that in nutrition and obesity research, published papers in which authors were funded by or had other financial ties to industry were more likely than were other papers to contain results or interpretations that favor the industry or company producing the product or service under study. Similar findings have been observed on multiple occasions in other areas of biomedical research.7,8 The reasons for and appropriate interpretation of these findings remain open to question.9 Nevertheless, concern about inappropriate influences (be they from financial or other DOIs) remains. The most common suggestion for dealing with the potential for inappropriate influence due to financial DOIs is disclosure of the DOI. However, disclosure is likely to be at best ineffectual5 and at worst antithetical to focusing on scientific method,10 derogatory toward industry-funded authors,11 and possibly productive of exactly the biases it is intended to diminish.12
Hence, we believe it will be more fruitful to identify factors that lead to better or poorer performance in each of the stages listed above in the hopes that identification of such factors may lead to ideas about policies and practices that can minimize inappropriate influences and maximize performance. If we are to maintain the public trust, crucial for many reasons,13 we must investigate this area.
In this paper we focus on Step G of the list of research steps presented earlier and ask whether industry funding leads to better or worse reporting of large, long-term randomized clinical trials (RCTs). We focus on RCTs for obesity treatment. This is an especially important area. Obesity is quite prevalent and causes a number of ill-effects including earlier mortality.14 New and better obesity treatments are needed and, as NIDDK wrote, “Well conducted clinical trials are the fastest and safest way to find improved treatments and preventions….”15
The database16–78 of papers evaluated has been described earlier79. In brief “studies were selected by searching MEDLINE (1966 to September 2003), HealthSTAR (1975 to September 2003), and the Cochrane Controlled Trials Register (1990 to September 2003).” Inclusion criteria were: (a) a randomized controlled trial; (b) a sample consisting of overweight or obese adults with a mean body mass index ≥ 25 kg/m2; (c) investigation of a weight loss intervention; (d) duration of patient follow-up ≥ 1 year. Exclusion criteria were: (a) patients were required to attain a weight loss target to qualify for study enrollment; (b) the weight loss intervention was not approved for clinical use; (c) the weight loss intervention was approved only for short-term use.79
The primary outcome measure was quality of reporting (QR). QR was coded and described more fully by Thabane et al.79 Briefly, the authors used the CONSORT Statement80,81 to assess QR. For each trial, the authors, via an independent double review, determined whether each of 44 detailed CONSORT reporting criteria contained within the checklist were satisfied. Eight of the 44 CONSORT criteria associated with methodologic quality from the results and methods sections were examined. The CONSORT statement is considered the medical industry standard of reporting and contains elements that have been used to evaluate methodologic quality of RCTs.82,83 Face and content validity have been evaluated for the CONSORT statement, which has been endorsed by numerous peer-reviewed journals and scientific organizations83.
A full copy of each paper was independently reviewed by each of two authors and placed into one of the 4 categories as defined in Table 1. The two reviewers then compared their codings of funding source, which was determined from financial information disclosed in each publication and an agreement of over 89% (56 of 63 publications) was obtained. The remaining 7 publications were re-examined and consensus was reached by the two reviewers. In all but two cases, the discrepancies were the result of a transcription error or oversight on the part of one coder and corrected. In the final two cases, the disagreement entailed a question regarding whether funding from a for-profit foundation established by but independent of a for-profit pharmaceutical company constituted industry support. Because of the difficulty in reaching a definitive decision on this and because there were only two such studies, we ran all analyses twice, first with these coded as non-industry and then in a sensitivity analysis coding these studies as industry-funded.
Descriptive statistics (mean and standard deviation [SD]) are tabulated (Table 2). The primary inferential analysis consisted of a simple independent samples t-test on the QR scores comparing Category 0 (no industry funding) with Category 1 (industry funded). Subsequently, a number of sensitivity and secondary analyses were conducted including: (A) repeating the primary analysis with a non-parametric Wilcox-Mann-Whitney test; (B) repeating the primary analysis using generalized estimating equations (GEE)84 to account for possible clustering within journals to allow for possible correlation in the residuals by journal; (C) using the GEE regression to control for both whether the paper was published after the 1996 publication of the CONSORT guidelines80,81 on reporting of RCTs and year of publication; (D) repeating all analyses using only non-drug studies (there were no drug studies in Category 0); (E) repeating all analyses after recoding studies in Categories 2 and 3 as in Category 0 (i.e., non-industry funded); (F) repeating all analyses after excluding 4 non-industry studies where funding source was specified. The criterion for statistical significance was set at a 2-tailed α of 0.05. All analyses were done using (SPSS, version 15.0).
Table 2 contains descriptive statistics. As can be seen, the majority (close to 2/3rds) of long-term weight loss studies are industry-funded. The variances in QR scores is almost identical for industry and non-industry-funded studies, but QR scores tend to be higher (i.e., better reporting) for industry funded studies. QR scores for industry-funded drug studies tended to be higher than industry-funded non-drug studies.
The t-test strongly indicated that the average QR was significantly higher for industry-funded than non-industry funded studies (t = 4.02; p < 0.0005; df = 57), with no evidence of heteroscedasticity by Levene’s test (p = .640). This was confirmed by the Wilcox-Mann-Whitney test (p < 0.0005) and the GEE analysis (p < 0.0005). In a GEE regression analysis conditioning on year of publication and an indicator variable for whether the paper was published after the 1996 CONSORT statement was released, the result was in the same direction and remained statistically significant (p = 0.0354). When examining only non-drug studies, the statistical significance was diminished with the much smaller sample size, but again, results were in the same direction (see Table 3). In addition, all analyses were repeated recoding the two aforementioned studies supported by for-profit foundations established by pharmaceutical companies as being industry-supported. Finally, all analyses were repeated after excluding four non-industry studies where the funding source was unspecified. Results from the final two analyses were virtually identical to those of other analysis strategies (data not shown).
The results of this study show that industry funded studies are associated with higher QR on average. In this relatively small sample of weight-loss studies restricted to both those that were relatively large and long-term it was difficult to disentangle the independent effects or association of industry funding from that of whether the study was a drug study, both associated with better reporting.79 Nevertheless, even when examining only non-drug studies, the numerically superior QR was found for industry-funded research, though the statistical significance was diminished. Future research should evaluate whether such associations exist for shorter-term studies and smaller studies.
For the scientific process to proceed effectively, it is important that all studies, both industry-funded and not, be reported with the highest quality possible. This is because it is through the comprehension of published research reports that the scientific community at large can judge the merits and import of the findings.10 If industry-funded reporting was of lower quality than non-industry-funded reporting, this would exacerbate existing concerns about potential biases being created by industry funding.5 In contrast, our results suggest that if anything, industry funding is associated with higher reporting quality. This suggests that, while continued efforts to improve reporting quality are warranted, such efforts should be directed at non-industry-funded research at least as much as at industry-funded research. A benefit of the greater funding offered by industry, the greater scrutiny of industry, or perhaps greater concern or training of industry personnel for rigorous reporting may be an enhancement of the overall reporting quality in the literature, at least for long-term weight loss studies.
We are grateful to Dr. Stacey Cofield for her helpful comments on the statistical analysis.
Contributors: OT participated in the coding of the data, helped to interpret the results, and participated in drafting the manuscript. LT, JD, and RC collected the initial sample of studies, coded the QR scores, advised on the conduct of analyses, and edited the manuscript. DBA conceived the project, participated in the coding of the data, analyzed the data, and led the writing of the manuscript.
Funding: This research was supported in part by NIH grant P30DK056336.
Competing interests: LT is the clinical trials mentor for the Canadian Institutes of Health Research. DBA has received grants, honoraria, consulting fees, and donations from numerous food, pharmaceutical, and other companies as well as on-profit organizations and government agencies with interests in obesity-related issues.
Ethical approval: Not required.