|Home | About | Journals | Submit | Contact Us | Français|
This paper was presented as a poster at the SGIM National Meeting New Orleans, May 2005.
Diet books dominate the New York Times Advice Best Seller list and consumers cite such books as an important source of nutrition information. However, the scientific support for nutrition claims presented as fact (nutrition facts) in diet books is not known.
We assessed the quality of nutrition facts in the best-selling South Beach Diet using support in peer-reviewed literature as a measure of quality. We performed structured literature searches on nutrition facts located in the books' text, and then assigned each fact to 1 of 4 categories (1) fact supported, (2) fact not supported, (3) fact both supported and not supported, and (4) no related papers. A panel of expert reviewers adjudicated the findings.
Forty-two nutrition facts were included. Fourteen (33%) facts were supported, 7 (17%) were not supported, 18 (43%) were both supported and not supported, and 3 (7%) had no related papers, including the fact that the diet had been “scientifically studied and proven effective.”
Consumers obtain nutrition information from diet books. We found that over 67% of nutrition facts in a best-seller diet book may not be supported in the peer-reviewed literature. These findings have important implications for educating consumers about nutrition information sources.
There is significant academic and public interest in the effectiveness of popular diets, particularly low-carbohydrate diets,1–7 but to our knowledge there is little information on the quality of nutrition information contained in these diet books. Several aspects of health information quality have been assessed in other widely accessible non-peer reviewed sources such as the internet.8,9 Assessing the quality of nutrition information provided in diet books is particularly important because they are read by millions of consumers, as evidenced by their frequent presence at the top of the New York Times Best Seller Advice Book list.10 Consumers cite books as an important source of nutrition information;11 but, diet books are not subject to the scientific standards of peer-reviewed journals. Because these books are often presented in a scientific manner, consumers may assume that the nutrition information contained in these books is supported by studies that have met criteria similar to that found in peer-reviewed literature.
Nutrition information that is not supported in the scientific literature may have consequences for consumers, and there are laws protecting consumers against nutrition misinformation.12 The American Dietetic Association position paper on food and nutrition misinformation states that nutrition information presented in non-peer-reviewed publications may “pose risks that include profit at the expense of a trusting but uninformed public and health risks for the individual.”13 There is benefit in educating the public about the difference between nutrition information based on theory and scientifically evaluated hypotheses.
The South Beach Diet has been on the bestseller list since its publication in April 200310 and has more than 7 million copies in print. We chose to evaluate the South Beach Diet as a case study of the quality of nutrition information in a popular diet book because it was outselling all other diet books at the time of this inquiry and remains on bestseller lists. Evidence supporting the nutrition information in the book has not, to our knowledge, been subject to scientific review. We assessed scientific support for nutrition facts available at the time South Beach was published, recognizing that peer-reviewed data related to these facts have since evolved and may be different now than what existed at the time of publication of the South Beach Diet.
We included only nutrition facts in this review. Readers may perceive opinions to be scientifically based as well, but opinions would not be expected to have peer-reviewed support. Fact checking consisted of 4 stages: fact identification, literature review for each fact, scientific support categorization for each fact based on literature review findings, and expert adjudication of fact categorization.
Fact identification was carried out by a subset of the study team. Three members, including an experienced health services researcher/cardiologist (H.K.), an internist/pediatrician with experience in obesity prevention and health services research (S.G.), and a medical student with training in health services research, read the text of the South Beach Diet. Each initially independently identified all statements containing nutrition facts. One hundred eighty-seven facts were identified in this initial phase, with 53% of the facts having been identified by at least 2 of the fact checkers. Fact inclusion was refined after this preliminary fact identification, and all statements containing qualifiers such as “might,” “could,” or “possibly” were excluded as opinion. In addition, all statements of medical facts and definitions such as “carbohydrate digestion begins in the mouth” were excluded, and facts that did not pertain specifically to nutrition were excluded. For example, “statins have been shown to help lower LDL cholesterol” was excluded. Forty-two facts remained after the exclusion process was completed, and 75% of these facts had been identified by at least 2 of the 3 fact checkers. The team resolved differences of opinion in fact exclusion through discussion. Facts were then classified into 1 of 5 major topic areas for organizational purposes by the lead author (S.G.). There was no a priori categorization scheme, but through the process of fact identification, it became clear that the facts fit into specific clinically relevant categories. Once facts were identified and classified, we reviewed the scientific literature pertaining to each fact. The review team consisted of a preventive cardiologist (J.F.) and 2 students at the school of public health studying nutrition-related disease processes in addition to 2 of the 3 team members who identified facts (S.G. and medical student).
We performed a structured literature review system using the British Medical Journal Clinical Evidence search strategies so that searches were standard among the 5-member search team. Details of this search strategy are available elsewhere.14 The Medline database was searched, and initially limited to Core Clinical Journals unless no articles were found, and then the Core Clinical Journal limit was dropped and the search was rerun. If no systematic reviews were found, the Cochrane Systematic Reviews were searched. Searches were limited from 1993 to April 2003 as studies published prior to this would likely be captured in systematic reviews or be outdated. We searched systematic reviews first and stopped with systematic reviews if any were found. If no systematic reviews were identified, we searched randomized-controlled trials, and if no randomized-controlled trials were found, we searched observational studies. We also reviewed consensus statements from national associations such as the American Heart Association15 and American Diabetes Association,16 when available, as a check on our search findings.
Reviewers abstracted information including author, title, journal, year of publication, study design, sample size, findings supporting fact, findings refuting fact, and findings deemed inconclusive by the authors. Each fact was assigned to 1 of 4 categories of support based on search results of: (1) fact supported, (2) fact not supported, (3) fact both supported and not supported, and (4) no related papers identified. If debate over categorization occurred among team members, “benefit of doubt” was given and the fact was categorized as supported.
A team of 4 experts in the following major topic areas reviewed the category assignment for each fact: preventive cardiology, endocrinology, public health/nutrition and cancer, and public health/nutrition and preventive health. If an expert reviewer disagreed with the research team's category of support, they were asked to supply sources of data to support their position and fact category was reassigned if data indicated that reassignment was appropriate. Expert review team members supplied additional evidence for their recommended change from sources not captured by the search strategy, providing a resolution to disagreements on initial categorization.
A total of 42 facts were identified. Facts were classified into 1 of 5 categories: cardiovascular health, cancer prevention, metabolism, satiety, or miscellaneous. Cardiovascular health and metabolism had additional subclasses, including saturated/trans and polyunsaturated fats, micronutrients/phytochemicals/fiber, carbohydrate metabolism/glycemic index, and general metabolism. Fourteen (33%) of the facts were supported in peer-reviewed literature, 7 (17%) were not supported, 18 (43%) were both supported and not supported, and 3 (7%) had no related papers (Table 1).
An example of a supported fact was “Fish oil capsules decrease sudden death.” An example of an unsupported fact was “This diet (South Beach) has been scientifically studied and proven effective.” In some cases, categorization proved difficult. Categorization for 7 (17%) facts was adjusted after expert review. Three facts were changed from “no data” to a different category based on data supplied by reviewer, or to a less supported category, reflecting a leniency on the part of the initial review towards favoring the South Beach position. The other 4 changes were based on supporting evidence provided by the expert.
Obesity is a major public health issue,67,68 and consumers are seeking guidance on weight loss from numerous sources. Based on the high number of books sold and American Dietetic Association information,10,11 diet books are a popular source of guidance. There is potential for the nutrition information in these books to be misleading if the quality of the information is poor. In this study, we demonstrate that the nutrition facts in a popular, heralded diet book authored by a physician are commonly incorrect or still in evolution. It is particularly of interest that the facts regarding the scientific testing and proven effectiveness of the diet and that “people lose 8 to 13 pounds in the first 2 weeks” were not found in peer-reviewed literature and that the facts regarding carbohydrate metabolism and satiety, which are the basis for the diet, were frequently not supported by medical literature.
This study offers only one book as an example and it is not possible to extrapolate the findings to other popular diet books such as the Atkins Diet, the Zone Diet, or Dr. Ornish's Diet. Future work could involve development of a system, such as 1 using a rating scale or Likert scale system, to apply to any diet book.
Completing a full systematic review on each nutrition fact might have strengthened the findings, but we found publications in high-quality journals that gave an indication of the recently reported data using our review method. It would be possible to complete exhaustive reviews, weighting various types of evidence, but we were exploring whether the literature supported each fact, meaning once disagreement was detected, this question of disagreement was answered.
Nutrition experts may not all agree with each categorization, even keeping in mind the authors' caution to include in the literature review only publications available before the publication of the South Beach Diet, but some statements in the book are clearly misleading or incorrect. We considered a fact to be supported if there was debate in the literature, meaning that our findings may actually overestimate the quality of the facts contained in this case study. It is also important to note that we did not imply any findings related to the safety or effectiveness of the South Beach Diet, but looked only at the nutrition facts presented in support of the diet.
Primary care physicians who care for adolescents and adults report feeling unprepared to counsel patients on diet, exercise, and weight loss69–72; yet, because of the substantial morbidity and mortality associated with poor diet, sedentary lifestyle, and obesity, they are called upon to assist patients in designing weight loss strategies, at times even asked their opinions of popular diet books. Assessments such as the one we report here may be helpful to any health practitioner in the position of advising patients on nutrition and weight loss strategies. The findings reported here may also serve to increase public awareness of the difference between peer-reviewed standards and popular publications, and might encourage consumers to scrutinize the facts presented in popular diet books before acting on them. The results of this study may provide material for developing a guideline to diet books for consumers. This guideline might include brief education about the peer review process and a comparison of hypotheses versus facts, with general red flags and positive aspects of diet recommendations, as well as suggested questions to bring to health professionals.
Dr. Goff was supported by a training grant from the Robert Wood Johnson (RWJ) Foundation while this study was conducted. The RWJ Foundation holds no responsibility for the content of this manuscript. We would like to thank Lauren Borchardt, Sharon Gill, and Steven Moore for their assistance with this project.