The importance of validation studies has been thoroughly discussed (54
) and the need for validated methods for assessing children's dietary intake has been emphasized repeatedly (12
). However, this is the first validation study to evaluate the accuracy and consistency of children's dietary recalls provided on multiple days without parental assistance. Results indicate that the accuracy of children's school breakfast and school lunch recalls obtained during 24-hour recalls was poor compared with observation; furthermore, accuracy was inconsistent from one recall to another for the same child.
Considering that omission and intrusion rates may both range from 0% to 100%, with 0% as perfect, what constitutes acceptable accuracy? Applying an arbitrary pass or fail criteria for accuracy which establishes omission and intrusion rates of ≤30% as passing and >30% as failing, only 3 of the 79 children (4%) achieved passing accuracy on each of their 3 recalls (). With this criteria, a child's recall would have acceptable accuracy even if the child omitted (ie, failed to report) up to approximately one third of items actually eaten, and if up to approximately one third of items reported by the child were intruded (ie, falsely reported). Taken together, a recall with this level of error would have little utility.
To what extent can recall accuracy vary from one recall to another for a child and still be deemed acceptably consistent? Applying an arbitrary consistent or inconsistent criteria that establishes omission and intrusion rates within 30% across an individual child's 3 recalls as consistent and >30% as inconsistent, only 18 of the 79 children (23%) provided consistent recalls (). With this criteria, recall accuracy would be consistent for a child who omitted 25%, 10%, and 35% of items and intruded 5%, 35%, and 15% of items during each of 3 respective recalls. However, recall accuracy would also be consistent for a child who omitted 80%, 70%, and 50% of items and intruded 50%, 75%, and 60% of items during each of 3 respective recalls. Thus, although acceptable consistency is important, acceptable consistency without acceptable accuracy is useless because children who are consistently inaccurate provide very little information regarding what they actually ate.
Analyzing reporting of items separately from reporting of amounts provides insight about what contributes to inaccurate recall, which in turn provides insight about what improvements can be made. When items are omitted or intruded, amounts cannot be correct. In this study, children omitted more than half of the items they were observed eating at school breakfast and school lunch; furthermore, of what they did report eating at these two meals, almost 40% was not observed eaten. Thus, research to improve the accuracy of children's dietary recalls must first focus on how to improve children's ability to correctly report items eaten. Furthermore, because children report what they eat as foods, not nutrients, unless reporting of items improves, derived measures of intake (eg, energy, nutrients) from children's recalls are not based on accurate reports of actual foods consumed.
When items were correctly reported (ie, matched), children's reported amounts were fairly accurate (ie, one-tenth and one-fourth serving for arithmetic and absolute differences, respectively). This is similar to results from a previous study with fourth-graders (16
). The average omission was more than four-fifths serving; thus, omissions were generally not items for which children ate only small amounts. The average intrusion was four-fifths serving; thus, intrusions were items for which children falsely claimed to have eaten most of the serving.
The total inaccuracy measure for each recall, based on all items observed and/or recalled across breakfast and lunch, captured the dimension of the total error, in servings, of the dietary recall. Across multiple days, this measure had an ICC of 0.29, indicating that there was less variability of total error from child to child than within children. We expected mean total inaccuracy to decrease (and thus accuracy of recall to increase) from the first to the third recall because some children quickly learn tasks and therefore perform better at them. Examining the within-child data, this learning effect was confined to a subset of children, suggesting that similar learning across all children in the population was unlikely; however, our sample size was insufficient to explore this supposition.
There were more omissions than intrusions for 8 of 10 meal components; this suggests that children were not simply substituting intrusions for omissions most of the time. The lowest omission rate by meal component was for beverage, perhaps because milk was regularly included for school breakfast and school lunch. Omissions and intrusions were high among all other meal components, including entree and combination entree. Although each child was asked during the third interview pass whether anything was added to each item (), the omission rate was 67% for condiments, perhaps because children also omitted many items to which condiments were added.
The omission and intrusion rates from this study were higher than those from a previous study (16
). Although both studies included fourth-graders, recalls in the previous study only included school lunch (16
); in this study, however, recalls covered 24 hours. Thus, the cognitive burden of recalling school breakfast and school lunch items in the context of a 24-hour recall seems to be greater than that of recalling lunch items as a single meal, and negatively impacts fourth-graders' ability to accurately recall items eaten. This is similar to a comment by Reynolds et al (17
) that underreporting is less of a problem if recalls are restricted to one meal.
To what extent did the interview protocol used in this study contribute to the low accuracy and low consistency of the children's recalls? This question cannot be answered because this study used the same interview protocol to obtain each recall from each child on each occasion. However, results from 3 validation studies (11
), each with a small number of children, indicated that the structure of the interview protocol seemed to influence children's accuracy regarding daycare snack or school lunch intake compared with observation.
Several studies have compared children's dietary recalls with school lunch observations (eg, 10
); however, to our knowledge, only one other study (14
) also included school breakfast observations. Although certain aspects of school meals (eg, written cycle menu) may enhance memory for the items eaten (8
), other aspects of school meals (eg, menu changes) may impede memory for items eaten (8
An important strength of this study is the use of observation (which did not rely on memory) to validate children's recalls. This strength is especially important in understanding the low accuracy and low consistency of recalls from most children studied. If the recalls had been validated against another self-report method (eg, food diaries), many may have been deemed statistical artifacts rather than actual recalls with low accuracy and low consistency. Dietary recalls are commonly used with children, but research regarding the accuracy and consistency of children's recalls compared with non–self-report methods is scarce. Although there is concern that observations cause reactivity (54
), if reactivity were an issue, the expected levels of accuracy would be higher than our results. Considering the importance of obtaining children's dietary recalls, research is needed to develop improved methods to increase the accuracy and consistency of their recalls. Such research needs to rely on non–self-report methods for validation. Conducting observations of children's consumption during part of their day in an environment natural to them, such as school, provides a gold standard for comparison to their recalls.
Another strength of this study is that accuracy was analyzed several complementary ways. These included accuracy for reporting food items, accuracy for arithmetic and absolute differences per serving in reporting amounts for matches, errors in amounts per serving of omissions and intrusions, and total inaccuracy accounting for errors.
There are several limitations of this study. First, children were recruited from only 6 schools, which were not selected randomly; however, the percentages of AAM, AAF, WM, and WF children who agreed to participate were similar to those of the total population of fourth-graders at the 6 schools, which suggests that our sample is representative of the population (46
). Second, the study was not powered to detect differences between the ethnic/gender groups; instead, the sample was stratified by ethnicity and gender. Third, analyses were limited to the school breakfast and school lunch portion of the 24-hour recall because these were the only meals observed.