|Home | About | Journals | Submit | Contact Us | Français|
Recently, localities across the United States have passed laws requiring the mandatory labeling of calories in all chain restaurants, including fast food restaurants. This policy is set to be implemented at the federal level. Early studies have found these policies to be at best minimally effective in altering food choice at a population level. This paper uses receipt and survey data collected from consumers outside fast food restaurants in low-income communities in New York City (NYC) (which implemented labeling) and a comparison community (which did not) to examine two fundamental assumptions necessary (though not sufficient) for calorie labeling to be effective: that consumers know how many calories they should be eating throughout the course of a day and that currently customers improperly estimate the number of calories in their fast food order. Then, we examine whether mandatory menu labeling influences either of these assumptions. We find that approximately one-third of consumers properly estimate that the number of calories an adult should consume daily. Few (8% on average) believe adults should be eating over 2,500 calories daily, and approximately one-third believe adults should eat lesser than 1,500 calories daily. Mandatory labeling in NYC did not change these findings. However, labeling did increase the number of low-income consumers who correctly estimated (within 100 calories) the number of calories in their fast food meal, from 15% before labeling in NYC increasing to 24% after labeling. Overall knowledge remains low even with labeling. Additional public policies likely need to be considered to influence obesity on a large scale.
Over 20 cities and states are attempting to implement mandatory calorie labeling in fast food restaurants, and the initiative was included in the 2010 Health Reform bill passed by Congress and signed by the president. This federal regulation is the first large-scale public policy to change the food environment and subsequently obesity; it requires restaurants with greater than 20 locations nationally to post the caloric content of all regular menu items on the menu board (if a traditional fast food restaurant) or printed on the menu (if a sit-down establishment). The hope of public health advocates is that this additional nutrition information will encourage consumers to make healthier food choices. Of particular concern are those most at risk for obesity and other health problems, the low-income and racial and ethnic minorities. New York City (NYC) was the first locality to successfully implement this legislation in July 2008.
Recent studies have attempted to examine the influence of labeling in both experimental and real world settings. A study by Harnack et al. examined the influence of calorie labels in an experimental setting meant to simulate a fast food menu with a menu board, finding no overall influence of labeling; males ordered more calories in the presence of labeling (1). In another recent experiment utilizing a printed menu, Roberto et al. found that calorie labeling decreased calories consumed over the course of the day, when the labels included a “prompt” indicating how many calorie should be consumed a day, but not without the prompt (2). The Harnack et al. study also included this prompt. In the first “real world” study of the influence of labeling, Elbel et al. examined low-income areas of NYC (utilizing Newark as a comparison city) and found that while labeling did increase the percentage of consumers who saw labels and indicated labels influenced their food choice, this did not translate into any observable change in calorie purchased (3). A related study examining the influence of labeling on the food choices of parents and adolescents found similar results (4), as did a study examining specific purchasing patterns (5). Finally, a recent study examining the influence of labeling at Starbucks found a small, 6% change in calorie purchased (corresponding to a 12 calorie decrease), with likely larger changes found for those who generally purchased a larger than average number of calories (6). While the overall evidence of labeling has been somewhat mixed, one message has been consistent: labeling alone has likely not, in the short term, drastically changed the number of calories purchased at fast food restaurants.
In light of this evidence, it is important to re-examine two assumptions likely necessary (though not sufficient) for calorie labeling to influence food choice. First, consumers must recognize whether a particular caloric content is unhealthy or not. A likely indicator of understanding this is knowledge of the recommended daily caloric intake. In fact, the federal labeling law requires that this information be posted at all labeled restaurants. As such, it is important to understand how well consumers currently understand the recommended caloric content, including whether this knowledge might be influenced by mandatory labeling. This key provision of the federal labeling legislation is different than the law of NYC and elsewhere, and the provision could make labeling more effective, or as described below, possibly less effective. We are aware of only two studies that have directly examined this concept at a population level. The first focused on a community sample of individuals in Vermont. This study found that 64% of respondents were able to identify a number that fell within the range 1,500–2,500; only 10% of respondents identified a number higher than 2,500 (7). The second study used a nationally representative sample from a telephone survey and asked consumers to answer in large ranges (<1,500, 1,500–3,000, or >3,000), the recommended number of calories one of three groups of people should eat; the correct answer was always the middle category. Only 2–15% of respondents ever estimated more than 3,000. Racial and ethnic minorities were more likely to overestimate the recommended number of calories (8). While an important finding, the large categories and closed-ended responses limit our ability to fully understand the truth, unprompted knowledge of caloric requirements.
The second key assumption, more widely discussed, is that individuals poorly judge the caloric content of food items and that this might improve with mandatory labeling. While it is clear individuals tend to under-report their overall energy intake on surveys, much less studied is whether consumers poorly judge the caloric content of the fast food that they purchase for themselves. Seemingly, the most cited article in this regard focuses on individuals’ estimates of a variety of menu items described by researchers in a written format. The meals described were much larger than the typically ordered fast food meals, and consumers dramatically underestimated the number of calories in these meals (9). But, this study does not tell us how consumers might estimate the number of calories in the food they choose to purchase for themselves.
Perhaps more relevant to understanding how consumers estimate fast food calories is a study where subjects were asked for their estimate of the number of calories they consumed after making purchases at fast food restaurants in three medium-sized Midwestern cities. For this study, the actual caloric content consumed was derived from looking at the wrappings left on the tables after eating and by researchers asked clarifying questions to those eating the food (e.g., whether the beverage soda was regular or diet). This study found that consumers at McDonald’s correctly estimated calories for meals defined as small (mean calories consumed = 555), but for medium meals (mean calories consumed = 911), underestimated by 16%; for large meals (mean = 1,327 calories), the underestimation was for 36%. Consumers’ underestimates were even larger at Subway restaurants. However, an additional study in the same paper found that consumers were able to correctly estimate the caloric content of McDonald’s menu items, if not Subway menu items (10). Related, a fairly consistent line of work show that underestimation increases as the size of the meal increases (11,12). Overall, it appears that consumers do generally tend to underestimate the number of calorie for larger meals, though the results across studies with various methodologies are not fully consistent.
Part of the assumption motivating mandatory labeling is that consumers might become better at estimating the calories that they purchase after the introduction of labeling, which could be a precursor to a potential change in calories purchased in the future, or set the stage for an additional policy response. The only data available on this topic of which we are aware come from one of the lab-based studies mentioned above (2). In this experiment, with a printed menu, consumers underestimated the number of calories they consumed by 714 calories in the no-labels condition. With the introduction of labeling, this underestimation decreased between 465 and 509 calories. Though, we note the mean calories purchased at these meals (averages ranged from 1,860 to 2,189 calories) are well over twice as large as the average purchase at fast food restaurants (3). As such, this experimental finding might best resemble what happens at a sit-down restaurant, with a printed menu, as opposed to a fast food restaurant.
In this paper, three basic research questions are addressed, each filling substantial gaps in the prior literature. We focus on the actual, nonexperimental, choices and knowledge of a low-income urban population who has chosen to eat at a chain fast food restaurant. First, we address the extent to which this population knows how many calories they should consume to maintain a healthy weight. Second, we examine how well these same consumers are able to estimate the number of calories in the food and drink they purchase for themselves. Third, we examine how both of the above change as a result of the introduction of calorie labeling. In doing so, we address several of the disadvantages of the above studies to provide evidence directly relevant to calorie labeling policy. We focus on a low-income, racially and ethnically diverse population at greatest risk for obesity. We perform our study in a nonlaboratory setting, examining the choices and caloric estimates that consumers have made for themselves as part of their daily lives. Finally, and importantly, we examine how all of this changes as a result of the introduction of mandatory, city-wide calorie labeling.
The basic premise of our approach is a difference-in-difference design. We collected data in a city where labeling is to be introduced, both before and after it was implemented. At the same time, we collected data in a comparison city. This is a city that is demographically similar to the labeling city, but did not introduce labeling. We then examine any change in our outcomes of interest in the labeling city, and compare them to any changes in the comparison city. This comparison city is critical to our study, and allows us to isolate the actual influence of labeling, as opposed to secular trends in eating or related influences.
While a more detailed explanation of the methods is available elsewhere (3), here we give a basic summary of the data collection methods, details on the specific variables utilized in this paper and the methods by which they are examined. As part of a larger study to examine the influence of calorie labeling in low-income communities of NYC, we collected data from 14 fast food restaurants in NYC (where labeling was implemented) and five restaurants in Newark, New Jersey as a comparison city, where labeling was not implemented. Restaurants included some of the most prominent fast food chains located in NYC and Newark (McDonalds, Burger King, Wendy’s, and KFC). All restaurants were visited during lunch time (generally 12:30 PM–2:30 PM) or early evening hours (generally 5:30 PM–7:30 PM) for ~2.5 h by a research team of 3–4 individuals. Restaurants were visited on Tuesday, Wednesday or Thursday over a 2-week period, directly before calorie labeling began in July 2008 and then 1 month after labeling was instituted, in August 2008. Individuals were approached as they were entering the fast food restaurant and told they could bring their receipt to the research assistant when leaving the restaurant, answer a set of questions, and be compensated with $2. Subjects were not told why the receipts were being collected. While it is difficult to assess cooperation rates with street intercept surveys, a study using the same methodology in NYC found that 55% of people who entered the restaurant participated in the survey, regardless of whether they were initially approached to do so (13). Here we report on the results of those 18 and over, and the food they purchased for their own consumption.
To gather objective nutrition data, research assistants reviewed the receipt with the respondent and confirmed the actual items purchased, as well as any modifications or additions to the food item that would impact caloric content (e.g., mayonnaise, cheese, regular or diet soda). The nutrition data provided on each fast food establishment’s corporate website were utilized to calculate the total calories of the items purchased for the respondent’s consumption.
Subjects were asked on the following open-ended question: “What do you think is the recommended daily calorie intake for an average American to maintain a normal weight?” Immediately following this question, subjects were asked how many calories they thought were in the food and the drink they consumed. These two numbers (food and drink) were recorded separately and then added together for a measure of total estimated caloric content.
A number of other data points were collected and here we focus on: age, sex, race/ethnicity, and whether the food was taken “to go” or not.
We first present basic descriptive statistics, including means, standard deviations, and ranges for demographic variables (Table 1). Then, we present the distribution of consumers’ knowledge of recommended caloric intake, using χ2 statistics to examine differences (Table 2). Next, we calculate differences between the number of calories purchased and the estimates for the sample as a whole and various subgroups. We present these differences via simple subtraction and as a percent change, estimating statistical significance between the means of the estimate and the actual number via a t-test. We also present the unadjusted difference-in-difference results; this examines whether the difference seen in NYC across the two study periods is different than the difference seen in Newark in the same study period. In addition to the unadjusted result, we also perform a difference-in-difference regression model. The outcome of this linear regression is the difference between the estimate and the actual number of calories purchased for each consumer. The model controls for age, sex, race/ethnicity, and whether the food was taken “to go” or not. The key variable of interest in any difference-in-difference model is the interaction term (city × time period), and we present in the table whether this variable is significant (Table 3).
An additional way to examine estimates in calories is to look at the percentage of consumers who overestimate, underestimate, and correctly estimate their purchases. We consider an estimate to be “correct” if the estimate is within 100 calories of the actual number of calories consumed. We then perform another difference-in-difference regression model with the same control variables, this time a logistic regression predicting the probability of having a “correct” estimate, as compared to over or under estimating. Again, we present the unadjusted difference-in-difference value for being correct, and the P value on the interaction term in the full controlled regression model (Table 4). Given the potential difficulty in estimating and interpreting interaction term coefficients in nonlinear models, we also estimate ordinary test squares models, with the results unchanged (results not shown) (14,15).
All regression models in both tables adjust the standard errors for clustering at the restaurant level. Finally, throughout we examine whether the following groups responded differently from the sample as a whole: (i) those who knew the recommended daily allowance of calories, (ii) those who purchased more (or less) than the average number of calories, and (iii) men vs. women.
Table 1 describes our sample as a whole as well as any differences in the NYC vs. Newark samples and the prelabeling vs. postlabeling sample. Overall, our sample is 38% male with a mean age of 38. Approximately 66% of the sample is black, 20% is Hispanic/Latino and the remainder is white or some other race. The sample is equally split between the pre and postlabeling period, and by design is ~70% NYC vs. Newark. The sample is fairly consistent across time and location with the exception of Newark, postlabeling sample being made up of more black respondents. Table 1 also displays the number of calories purchased in each period/city combination. As is discussed elsewhere, we did not detect a change in calories purchased (3).
Table 2 examines consumers’ estimates of the recommended daily caloric intake. The first set of columns indicates responses for the full sample (combining pre and postlabeling in NYC and Newark) and subsequent columns report the results individually for each city-time period combination. Overall, ~11% of subjects simply did not know or did not answer the question of how many calories they should consume. The percentage of consumers who reported not knowing went up pre to postlabeling in NYC (6–13%, P < 0.01) and did not change in Newark (but was higher in Newark prelabeling than NYC prelabeling). Approximately, half of consumers provided an estimate between zero and 1,500 calories. The modal response was generally an estimate of less than 500 calories; averaging across the entire sample, 27% of consumer responses fell into this category. If one consider a “correct” answer something between 1,500 and 2,499 calories, as other have done (16), then just under a third of all respondents are correct in their responses to the number of calories one should consume to maintain a healthy weight; this is unchanged as a result of labeling. Only, ~8% of consumers gave an estimate above 2,499 calories, indicating few are overestimating the total number of calories they believe should be consumed.
Moving to the estimates of the number of calories purchased, Table 3 compares the total number of calories purchased with the consumer’s estimate of calories purchased. Again, data are presented for both NYC and Newark, for the periods before and after labeling began. Approximately, one quarter of individuals did not provide an estimate, and this did not change pre to postlabeling in NYC or Newark. These individuals are excluded from this analysis.
The prelabeling number of calories purchased in NYC was 810 and the estimate was 619, or 24% below the actual number of calories purchased (P < 0.01 for being different than zero). In Newark in the prelabeling period, the difference was 214 calories, or 26% (P < 0.01). After labeling began in NYC, estimates improved in NYC to being 14% lower than the actual number purchased. This value is smaller than the value in Newark post-labeling (21% smaller). The difference-in-difference regression indicated that NYC improved in its estimates by 36 calories over the change in Newark, but the result in the adjusted model was not significant. This indicates that the change in NYC over time was not significantly larger than the change in Newark.
We then examine the differences for only food (for those who purchased food) and only drinks (for those who purchased drinks). For food, consumers in NYC are underestimating the total calories consumed by almost a third before labeling (P < 0.01 different from zero), dropping to 18% after labeling (P < 0.01). Though, again the difference-in-difference results (−48) are not significant. For estimates of drinks, although the difference-in-difference results are not significant, consumers are actually overestimating the number of calories purchased by approximately 21% in NYC before labeling (ns for different than zero), and 33% after labeling (P < 0.01). However, we note that this appears to be driven by some very extreme estimates for beverages, as we see in the below analysis. (Dropping the top 2% of estimates (results not shown) does change the results—a 10% overestimate in NYC and a 6% underestimate in Newark, prelabeling.)
Next, we examine the data on those who provided an estimate deemed to be accurate (between 1,500 and 2,500) for the number of calories that should be consumed to maintain a healthy weight. In NYC before labeling, these consumers are slightly but not significantly overestimating the total number of calories purchased by 10% (ns), and by only 2% after labeling (ns). There is also no noticeable change in Newark, resulting in no overall effect in the difference-in-difference results.
The next two results separate those who purchase meals that were higher than average and lower than average. Here, we see that underestimation of calories is larger in magnitude for those who purchased higher than average number of calories, consistent with prior literature. Finally, it appears that men have larger underestimates than women. The difference-in-difference results were not significant, with the exception of the results for those who purchased less than the mean number of calories, driven by a large difference in Newark (P < 0.10)
Table 4 examines the percentage of consumers overestimating the total calories purchased (believe they are eating more calories than they actually purchased), those who are within 100 calories of the actual calories purchased, and those who are underestimating the total calories purchased (believe they are eating fewer calories than they actually purchased). Similar to Table 3, we run difference-in-difference logistic regressions for being correct in one’s estimate, as compared to over or under estimating. Before labeling in NYC, 15% of consumers are correctly estimating the number of calories they consume; after labeling, this increased to 24%. Over the same period in Newark, this percentage decreased, resulting in a significant difference-in-difference result (15%; P < 0.01). The change was driven by a decrease in those underestimating the number of calories they consume, which dropped to 51% from 61%. The results for food only are slightly smaller in magnitude (9% difference-in-difference), and not significant.
For drinks, Table 3 suggested that on average consumers were overestimating the number of calories in their beverages on average. We now see that this was driven by 20% of consumers in NYC who were overestimating before labeling and 23% after labeling. After labeling, there was an increase in NYC consumers who correctly estimate the calories in their beverages (to 45% from 37%); the difference-in-difference results are significant for a net impact of 24% point increase in correct estimates in NYC. For those who knew the recommended daily allowance of calories, a smaller, but significant percentage of consumers increased their percentage of correct estimates compared to the difference in Newark (10% point net increase, P < 0.05). Similar results were seen for those who purchased above and below the mean number of calories (19% point net increase, P < 0.01), as well as for females (23% point net increase, P < 0.01). The increase in the percentage of males in NYC who improved their estimates was not significant.
We return to the two assumptions necessary for calorie labeling to be effective in altering food choice. First, in terms of knowledge of recommended calorie intake, approximately one-third of our low-income, racially and ethnically diverse sample knew the recommended daily allowance of calories was between 1,500 and 2,500, and 11% did not provide any estimate. However, less than 10% of consumers provided a number above 2,500. About half of all consumers believed the recommended daily amount of calories that should be consumed is less than 1,500. This work is generally consistent with findings from other populations and via other means of data collection. Calorie labeling did not change these estimates.
In terms of the second assumption, we did find that consumers underestimated the total number of calories they purchased. While the difference-in-difference result for the difference between the mean and actual calories purchased was not significant, a significantly higher percentage of consumers were properly estimating the number of calories in their purchases. Before labeling, ~60% of consumers in NYC underestimated the number of calories in their meal and this improved to about 50% after labeling. Consumers particularly improved at estimating the calorie from drinks. While the magnitude of these changes are likely meaningful, half of our sample is still underestimating the number of calories in their food.
In examining these two results, possible improvements and limitations of labeling in a low-income, racially ethnically diverse population are apparent. One potential solution is to enhance nutrition education to improve the effectiveness of labeling. Given that about a quarter of our population did not provide an estimate of the number of calories purchased and some of this nonresponse likely reflects a lack of knowledge, this suggestion has some merit. However, in contrast, less than 10% overestimated the number of calories one should eat in a day to maintain a healthy weight and half of our sample actually underestimated this number. Greater education on this point could raise this estimate for the majority of consumers in this sample, possibly giving consumers license to purchase more calories than they do currently. Given that the recently passed federal labeling legislation requires that information be posted at the point of purchase on how many calories are recommended, this finding should be carefully considered.
Even in the presence of labeling, half of consumers are still underestimating the number of calories they purchase. However, labeling improved these estimates, which is encouraging in terms of the possible effectiveness of labeling. Clearly, some subset of consumers is absorbing this information, particularly women, and it is possible that additional interventions or public policies could take advantage of this and improve the effectiveness of labeling.
This study is not without limitations. We collected our post data only 1 month after labeling. It is possible that consumers might need more time to fully absorb the calorie information. However, the one calorie labeling study that does have data (from Starbucks), that covers a much longer period indicated that any influence of labeling was immediate and sustained over an 11-month period (6). Though, knowledge of calories could still be improving after our 1-month time period. In addition, our data were obtained in a very short intercept survey with a limited set of questions; true knowledge might be better reflected when questions are posed in a different environment. Finally, while we do have objective data on calories purchased from a receipt, we are not able to examine differences in the amount of food actually consumed. And, we did not fully examine the extent to which the method in which the labeling data were presented, open in ranges or in less than ideal font, could have contributed to our results.
Policymakers must look to additional tools to improve food choice and influence obesity. To reach the level of obesity, we currently experience in the United States, individuals are consuming and not expending over 350 calories per day, on average and over many years, over and above the intake required to maintain a healthy weight (17). Even if current studies of the influence of labeling significantly underestimate the influence of labeling on food choice, we are unlikely to solve the obesity epidemic with labeling alone. Of course, labeling was never meant to be the one and only solution, and additional work is needed to identify, implement, and test these additional policy mechanisms. Combined with these additional policy mechanisms, it is possible that labeling could become a powerful policy tool in the overall arsenal of policies and influence obesity in a meaningful way.
I would like to thank Rogan Kersh, Beth Dixon and Tori Brescoll for their help with the larger project associated with this paper, as well as Courtney Abrams and Joyce Gyamfi. Kevin Lyu, Kristin Van Busum and Rebecca DiBennardo provided excellent research assistance. Finally, I gratefully acknowledge funding from the Robert Wood Johnson Healthy Eating Research Program, the Yale Rudd Center and R01HL095935 from the NIH/NHLBI.
The author declared no conflict of interest.