PMCCPMCCPMCC

Search tips
Search criteria 

Advanced


 
Formats
Article sections
Authors
Related links
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Int J Obes (Lond). Author manuscript; available in PMC 2010 July 1.
Published in final edited form as:
PMCID: PMC2815336
NIHMSID: NIHMS155044
Copyright notice and Disclaimer
White Hat Bias: Examples of its Presence in Obesity Research and a Call for Renewed Commitment to Faithfulness in Research Reporting
Mark B Cope, PhD
Department of Pharmacology and Toxicology, School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA. mbcope/at/uab.edu
David B Allison, PhD
Department of Biostatistics; School of Public Health; and Clinical Nutrition Research Center, University of Alabama at Birmingham, Birmingham, AL, USA. Dallison/at/uab.edu
Corresponding Author: David B Allison, PhD, Section of Statistical Genetics, Department of Biostatistics, Ryals Public Health Building, University of Alabama at Birmingham, 1530 3rd Ave S, RPHB 327, Birmingham, Alabama. Phone: 205-975-9169, Fax: 205-975-2541, Dallison/at/uab.edu.
Users may view, print, copy, download and text and data- mine the content in such documents, for the purposes of academic research, subject always to the full Conditions of use: http://www.nature.com/authors/editorial_policies/license.html#terms
Small right arrow pointing to: The publisher's final edited version of this article is available at Int J Obes (Lond)
Small right arrow pointing to: See other articles in PMC that cite the published article.
Abstract
`White hat bias' (WHB) (bias leading to distortion of information in the service of what may be perceived to be righteous ends) is documented via quantitative data and anecdotal evidence from the research record regarding the postulated predisposing and protective effects respectively of nutritively-sweetened beverages and breastfeeding on obesity. Evidence of an apparent WHB is found in a degree sufficient to mislead readers. WHB bias may be conjectured to be fuelled by feelings of righteous zeal, indignation toward certain aspects of industry, or other factors. Readers should beware of WHB and our field should seek methods to minimize it.
Scientific dialogue is dependent on fair and open presentation of data and evidence, yet recent years have raised concerns about bias in research practice. We present data and examples pertinent to a particular bias, a `white hat bias' (WHB), which we define to be bias leading to distortion of research-based information in the service of what may be perceived as righteous ends. We evaluate WHB in the context of two illustrative obesity topics, nutritively-sweetened beverage (NSB) consumption as a postulated risk factor [1] and breastfeeding as a postulated protective factor [8].
EXAMPLE 1 - Data on Citation Bias
If secondary reportings of original research misleadingly cite papers with statements that do inaccurately describe available evidence, then inaccurate beliefs may inappropriately influence clinical practice, public policy, or future research. Previously, [4], we observed that two papers [2,3] had both statistically and non-statistically significant results on body-weight, body mass index (BMI) or overweight/obesity status which allowed future writers to potentially choose which results to cite, and were also widely cited, permitting quantitative analysis of citations.
Cited vs Citing Papers
A Web of Science search (through October, 2008) yielded 195 and 45 papers citing James et al. [2] and Ebbeling et al. [3], respectively. We analyzed those in English (165 and 41, respectively).
James et al. [2] studied an intervention to decrease NSB consumption and adiposity among children. Dichotomized (overweight or obese vs neither overweight nor obese) and continuous (change in BMI) data were analyzed for statistical significance. The authors wrote:
“After 12 months there was no significant change in the difference in body mass index (mean difference 0.13, – 0.08 to 0.34) or z score (0.04, – 0.04 to 0.12). At 12 months the mean percentage of overweight and obese children increased in the control clusters by 7.5%, compared with a decrease in the intervention group of 0.2% (mean difference 7.7%, 2.2% to 13.1%).”
Ebbeling et al. [3] described an RCT of a 25 week NSB reduction program in adolescents and wrote:
“The net difference (in BMI), 0.14 ± 0.21 kg/m2, was not significant overall.” They then report a subgroup finding: “Among the subjects in the upper baseline-BMI tertile, BMI change differed significantly between the intervention…and control…groups, a net effect of 0.75 ± 0.34 kg/m2.”
Ebbeling et al. (p. 676) label the analysis in the total sample the “primary analysis.”
Data Coding and Analysis
Each paper citing either [2] or [3] was categorized (see Tables 1 & 2) based on how citing authors cited results related to body weight, BMI, or overweight/obesity outcomes from these two papers in their report. Papers citing James et al. were independently coded by this paper's authors (DBA or MBC). Any discrepancies were resolved by discussion. Papers citing Ebbeling et al. were scored by DBA and cross-checked by MBC. Proportions (with confidence intervals) were calculated (Tables 1 & 2). Exact binomial calculation tested the null hypothesis that the proportion citing papers in a misleading way that exaggerated the strength of evidence was equal to the proportion citing papers in a misleading way that diminished the strength of evidence; as such an equal proportion would suggest a lack of bias in the overall literature, even if not in any one paper.
Table 1
Table 1
Categorization of 165 papers citing James et al. (2004)
Table 2
Table 2
Categorization of 41 papers citing Ebbeling et al. (2006)
Citation Analysis Results
Results were quite consistent across papers citing either [2] or [3]. The majority, 84.3% for [2] and 66.7% for [3], described results in a misleadingly positive way to varying degrees (i.e., exaggerating the strength of the evidence that NSB reduction showed beneficial effects on obesity outcomes). Some were blatantly factually incorrect in their misleading statements, describing the result as showing an effect for a continuous obesity outcome whereas no statistically significant effect for continuous obesity outcomes was observed. In contrast, only four papers (3.5%) were negatively misleading (i.e., underplayed the strength of evidence) for [2] and none were negatively misleading for [3]. Only 12.7% and 33% of the papers accurately described complete overall findings related to obesity outcomes from [2] and [3], respectively.
To test whether the proportion of misleading reporting in the positive direction was equal to the proportion in the negative direction, we calculated the confidence interval on the proportion of misleading reportings in either direction that were positively misleading. This yields a proportion of 0.96 (95% CI: .903 to .985) for those citing [2] and 1.00 (95% CI: .832 to 1.000) for [3] and is significantly different from ½ for each (p < .0001) indicating a clear bias and potential for readers of the secondary literature to be deceived.
EXAMPLE 2 - Data on Publication Bias
NSB-Consumption
A meta-analysis on NSB consumption and obesity [6] found that estimated adverse associations were significantly smaller (i.e., less adverse) among industry-funded than non-industry-funded studies. One troubling conceivable explanation for this is that industry does something to bias results to make NSBs appear less harmful, but this is not the only conceivable explanation.
To examine this further, we requested, and Dr. Vartanian [6] graciously provided, his meta-analysis datafile. Focusing on cross-sectional studies because a large number had adiposity indicators as outcomes, we conducted publication bias (PB) detection analyses [12]. PB causes the sample of studies published to not constitute a representative sample of the relevant studies that hypothetically could have been published. With PB the probability of a study being published depends on its outcome. Typically, PB involves statistically significant studies having a higher likelihood of being published than non-statistically significant ones. Our analysis (Fig 1) shows a clear inverse association between study precision and association magnitude. This PB hallmark suggests that studies with statistically significant NSB findings are more likely to be published than are non-statistically significant ones. Interestingly, this bias appears to be present only for non-industry-funded research, suggesting that non-industry-funded scientists tend not to publish their non-significant associations in this area. Contrarily, industry-funded studies seem to all exceed some minimal level of precision. Thus, much of the reason for the smaller associations detected by Vartanian et al. [6] for industry funded research seems to be due to PB in non-industry-funded research. However, even after accounting for precision, the mean difference between the association magnitudes of the industry and non-industry funded studies is reduced by 33%, but not eliminated, suggesting that there may be competing biases operating in industry-funded research.
Figure 1
Figure 1
Plot of sample effect sizes from cross-sectional studies of the association between sugar-sweetened beverage consumption and obesity indexes indicating publication bias among non-industry-funded studies.
Breast-Feeding
The World Health Organization [WHO; 7] published a meta-analysis on whether breastfeeding protects against obesity and also found evidence of PB. Figure 2 indicates this strikingly. We retrieved all of the papers from which data were obtained for Figure 2 to evaluate the impact of industry funding on this PB. None of the papers reported any industry funding or were obviously authored by authors employed by the infant formula industry. Thus, as with the NSB literature, there appears to be strong PB that is not apparently fueled by industry interests.
Figure 2
Figure 2
Plot of the relation between association magnitude and study precision indicating publication bias in studies of breastfeeding and obesity (from [7]).
EXAMPLE 3 - Anecdotal Examples of Miscommunications in Press-Releases
Evidence suggests that “Press releases from academic medical centers often promote research that has uncertain relevance to human health and do not provide key facts or acknowledge important limitations” [5]. This is also occurring in the obesity field. For example, Ebbeling et al.'s paper [3] states, “change in body mass index (BMI) was the primary end point…The net difference, 0.14 ± 0.21 kg/m2, was not significant overall,” and then reports the subgroup finding, “Among the subjects in the upper baseline-BMI tertile, BMI change differed significantly between the intervention…and control…groups.” Contrast this modest finding in a sample subset and the circumspect presentation in the original paper with the presentation in the press release issued by the authors' institution1, which states “In randomized trial, a simple beverage-focused intervention led to weight loss” and never states that the primary analysis was not statistically significant.
When James et al. [2] was released, the press release issued on the BMJ website2 stated “Discouraging children from drinking fizzy drinks can prevent excessive weight gain, according to new research available on bmj.com,” despite the facts that no analysis of weight change per se was reported and that the there was no significant effect on BMI change. Neither of these facts was mentioned in the press release.
Finally, in 2009, describing an observational epidemiologic study, UCLA issued a press release3 stating “…research released today provides the first scientific evidence of the potent role soda and other sugar-sweetened beverages play in fueling California's expanding girth.” One of the study authors was quoted in a subsequent news story stating “For the first time, we have strong scientific evidence that soda is one of the – if not the largest – contributors to the obesity epidemic."4 These statements are inaccurate and also unfair to all the authors of observational studies who published such research years before. The press release further stated “"The science is clear and conclusive” [emphasis added], despite the fact that this was correlational research and offered no statement to the reader to interpret the results as indicative of correlation and not necessarily causation.
EXAMPLE 4 – Inappropriate or Questionable Inclusion of Information
Research may also be misleadingly presented by inclusion of incorrect or questionable material in reviews. In our critical review of the WHO report on breastfeeding, we noted several examples (see [8], p. 597) where inspecting the original papers reviewed revealed that the WHO report authors selectively included some values from certain primary papers that led to stronger associations of breastfeeding with reduced obesity risk and excluded less impressive values from the same papers without explanation.
Similarly, Mattes et al. [4] noted that several reviews of NSB consumption and obesity inappropriately included a study [9] that was not actually a test of nutritive sweetener-containing solid food versus beverage nor of NSB consumption versus non-NSB consumption. Sweeteners were presented in both solid and beverage food forms. The original authors [9] wrote, “…subjects who were given supplemental drinks and foods [emphasis added] containing sucrose for 10 wk experienced increases in …body weight” and thus the study should never have been considered as evaluative of NSB effects. Mattes et al. [4] provide other examples of papers being inappropriately included in past reviews of NSB consumption and obesity.
Finding effective methods to reduce obesity is an important goal and appropriate evaluations of the strength of the evidence supporting procedures under consideration are vital. Sound evaluations critically depend on evidence being presented in non-misleading ways. Alarms have been sounded about dramatic rises in obesity levels, not without justification. And yet these alarms may also have roused passions. Certain postulated causes have come to be demonized (e.g., fast food, NSBs, formula feeding of infants) and certain postulated palliatives (e.g., consumption of fruits and vegetables, building of sidewalks and walking trails) seem to have been sanctified. Such demonization and sanctification may come at a cost. Such casting may ignite feelings of righteous zeal.
Some authors compare NSBs, fast-food, and other food and restaurant industry offerings to the tobacco industry [e.g., 10], suggesting for example comparisons between `Joe Camel' and `Ronald McDonald'5. To the extent that such comparisons inform us about important causes of obesity and how to reduce them, this is all to the good. But to the extent that such comparisons and other appeals to the passions inflame rather than inform, they may cloud judgment and decrease inhibitions against breaching ordinary rules of conduct. Historians indicate that during times of war, propagandists demonize (i.e., dehumanize) the enemy to inflame spirits and this facilitates some breaches of codes of conduct such as massacres [11]. Though inflaming the passions of scientists interested in public health is unlikely to provoke bloodshed, scientists have as a discipline our own code of conduct. Central to it is a commitment to faithful reporting, to acknowledging our studies' limitations, to evaluating bodies of evidence without selectively excluding information on the basis of its desirability – in short, a commitment to truthfulness. The demonization of some things and sanctification of others, though perhaps helpful in spurring social action, may be more harmful to us in the long run by giving unconscious permission to breach that code and thereby eroding the foundation of the scientific discipline.
Evidence presented herein suggests that at least one thing has been demonized (NSB consumption) and another sanctified (Breastfeeding), leading to bias in the presentation of research literature to other scientists and to the public at large, a bias sufficient to misguide readers. Interestingly, while many papers point out what appear to be biases resulting from industry funding, we have identified here, perhaps for the first time, clear evidence that white-hat biases can also exist in opposition to industry interests.
Whether WHB is intentional or unintentional, stems from a bias toward anti-industry results, significant findings, feelings of righteous indignation, results that may justify public health actions, or yet other factors is unclear. Future research should study approaches to minimize such distortions in the research record. We suggest that authors be more attentive to reporting primary results from prior studies rather than selectively including only part of the results, to avoiding PB, and to ensuring that their institutional press releases are commensurate with the studies described. Journal editors and peer-reviewers should also be vigilant and seek to minimize WHB. Clinicians, media, public health policy makers, and the public should also be cognizant of such biases and view the literature on NSBs, breastfeeding, and other obesity-related topics more critically.
Acknowledgment
We gratefully acknowledge Dr. Alfred A. Bartolucci for his comments on our data analysis and Dr. Lenny Vartanian for sharing his datafile.
Supported in part by NIH grant P30DK056336. The opinions expressed are those of the authors and not necessarily those of the NIH or any other organization with which the authors are affiliated.
Footnotes
Competing Interests. Drs. Allison and Cope have received grants, honoraria, donations, and consulting fees from numerous food, beverage, dietary supplement, pharmaceutical companies, litigators, and other commercial, government, and nonprofit entities with interests in obesity and nutrition including in interests in breastfeeding and NSBs. Dr. Cope has recently accepted a position with The Solae Company (St Louis, MO.).
References
1. Allison DB, Mattes RD. Nutritively sweetened beverage consumption and obesity: the need for solid evidence on a fluid issue. JAMA. 2009;301(3):318–20. [PMC free article] [PubMed]
2. James J, Thomas JT, Cavan D, Kerr D. Preventing childhood obesity by reducing consumption of carbonated drinks: cluster randomised controlled trial. BMJ. 2004;328(7450):123743. [PMC free article] [PubMed]
3. Ebbeling CB, Feldman HA, Osganian SK, Chomitz VR, Ellenbogen SJ, Ludwig DS. Effects of decreasing sugar-sweetened beverage consumption on body weight in adolescents: a randomized, controlled pilot study. Pediatrics. 2006;117(3):673–80. [PubMed]
4. Mattes RD, Shikany JM, Allison BD. What is the Demonstrated Value of Moderating Nutritively Sweetened Beverage Consumption in Reducing Weight Gain or Promoting Weight Loss? An Evidence-Based Review and Meta-Analysis of Randomized Studies. Submitted for publication.
5. Woloshin S, Schwartz LM, Casella SL, Kennedy AT, Larson RJ. Press Releases by Academic Medical Centers: Not So Academic? Ann Intern Med. 2009;150:613–618. [PubMed]
6. Vartanian LR, Schwartz MB, Brownell KD. Effects of soft drink consumption on nutrition and health: a systematic review and meta-analysis. Am J Public Health. 2007;97(4):667–675. [PubMed]
7. Horta B, Bahl R, Martines J, Victora C. Evidence of the Long- Term Effects of Breastfeeding: Systematic Reviews and Meta- Analysis. World Health Organization Publication; Geneva, Switzerland: 2007.
8. Cope MB, Allison DB. Critical review of the World Health Organization's (WHO) 2007 report on `evidence of the long-term effects of breastfeeding: systematic reviews and meta-analysis' with respect to obesity. Obes Rev. 2008;9(6):594–605. [PubMed]
9. Raben A, Vasilaras TH, Møller AC, Astrup A. Sucrose compared with artificial sweeteners: different effects on ad libitum food intake and body weight after 10 wk of supplementation in overweight subjects. Am J Clin Nutr. 2002;76(4):721–9. [PubMed]
10. Brownell KD, Warner KE. The perils of ignoring history: Big Tobacco played dirty and millions died. How similar is Big Food? Milbank Q. 2009 Mar;87(1):259–94. [PubMed]
11. Levene M, Roberts P, editors. The Massacre in History (Studies on War and Genocide) Berghahn Books; Jul, 1999.
12. Sterne JAC, Egger M. Regression Methods to Detect Publication and Other Bias in Meta-Analysis. In: Rothstein HR, Sutton AJ, Borenstein M, editors. Publication Bias in Meta-Analysis. John Wiley & Sons, Ltd; 2005.