|Home | About | Journals | Submit | Contact Us | Français|
To simulate the effect of child-friendly adaptations of the National Cancer Institute's Automated Self-administered 24-hour dietary recall (ASA24) on estimates of nutrient intake.
One hundred twenty children, 8-13 years old entered their previous day's intake using the ASA24 and completed an interviewer-administered recall using the Nutrition Data System for Research (NDSR). Based on a hypothesis that proposed adaptations to the ASA24 will not significantly affect mean nutrient estimates, ASA24 data were manipulated post-administration to simulate a child-friendly version in which two categories of data collection were removed: 1) foods not likely to be consumed by children (45%) based on previous analyses of national dietary data and, 2) food detail questions (probes) to which children are unlikely to know the answers (46%), based on our experience.
Mean estimates of select nutrients between the beta version of ASA24 and the simulated child-friendly recall showed no significant differences, indicating that the food and probe elimination did not significantly affect results. However, a comparison of total sugar and Vitamin C assessments between the original ASA24, the child-friendly version and NDSR showed that the daily nutrient totals for both nutrients were significantly higher in the self-administered methods (both ASA24 and child-friendly version) than in NDSR (interviewer-administered) which warrants a review of different methods for obtaining information about foods that are sources of these nutrients.
The simulation of child-friendly adaptations showed that it is feasible to implement thereby reducing child-friendly response burden without significantly affecting the results.
The National Cancer Institute (NCI) developed the web-based Automated Self-Administered 24-hour dietary recall, ASA24 [1, 2], released as a Beta version in August 2009 and as Version 1 in September 2011. ASA24 is based on the US Department of Agriculture's Automated Multiple Pass Method (AMPM), an interviewer-administered 24-hour recall method used in national surveillance. The AMPM presents multiple passes through the respondent's reporting day, where each pass gathers increasingly detailed information about foods and portion sizes . ASA24 consists of approximately 6800 food terms and 400 detailed questions about food type and preparation (hereafter referred to as “probes”), the answers to which lead to the assignment of a food code  from the Food and Nutrient Database for Dietary Studies (FNDDS) [5, 6]. Food terms map directly to a food code if no probes are required. For example, the food term banana maps to “63107010 Banana, raw”. Alternatively, for the food term Apple (baked), the respondent is asked “Was any type of sweetener added during cooking?” A “yes” answer maps to “63101330 Apple, baked, with sugar” while a “no” maps to “63101320 Apple, baked, unsweetened” and “don't know” maps to “63101310 Apple, baked, not specified as to added sweetener”.
The ASA24 database includes foods, varieties and preparations consumed by people of all ages, some of which are not widely consumed by children, e.g., pickled herring, seaweed, buckwheat pancakes. Children are not generally involved in food preparation, so probes such as type of fat added in cooking are likely to yield “don't know” answers. Further, multiple probes children are unable to answer may contribute to boredom or antagonism, which could reduce completeness, recall, or willingness to complete one or multiple administrations. Because children's self-reported intake is greatly influenced by cognitive abilities and is not comparable to that of adults , the ASA24-Kids was designed to collect food intake from children . ASA24-Kids seeks to make ASA24 child-friendly by modifying the food term, probe and answer choices based on national surveillance data and input from researchers experienced in collecting dietary intake from children. Building upon previous research by the authors assessing children's abilities to self-report intake [8-10], this paper tests the hypothesis that planned child-friendly adaptations to eliminate food terms and probes from the ASA24 would not significantly impact mean nutrient estimates. This would support the value of ASA24-Kids by minimizing reporting burden of ASA24 without affecting accuracy or validity. No such adaptations or tests have been reported.
The Institutional Review Board of the Baylor College of Medicine reviewed and approved the protocol for this study. ASA24 beta version assigned FNDDS1 codes representing approximately 6800 food terms. Methods were developed to reduce the number of food terms and probes presented to children. Food probe pathways from each food term to the resulting FNDDS code (provided by Westat, Rockville, MD) were assessed. Both data from the National Health and Nutrition Examination Survey (NHANES) and our experience were applied as follows.
The number of times each FNDDS code was reported by children, age 8-15, in NHANES 2003-2004  and 2005-2006  was used to evaluate the ASA24 food terms. A food term linked to a food code never reported was eliminated unless it was judged to be a food children likely consume (e.g., Turkey breast, smoked) or one that should be available for special circumstances (e.g., Pedialyte). When similar food terms mapped to the same food code, one instance was eliminated. For example, Blackberries (other kind) and Blackberries (unknown kind) both mapped to “63201010 Blackberries, raw”. Blackberries (unknown kind) was eliminated.
Based on the accumulated experience of six dietitians, each with experience in conducting 24-hour dietary recalls with children, probes most likely to be answered as “don't know” were eliminated by consensus. For example, most children answer “don't know” to the question “What kind of fat was used in cooking?” Although this particular probe is common across food categories, each category was scrutinized to determine which probes children could answer. In the Juice category, the probe, “Was it made according to directions?” was eliminated because children are unlikely to know how juice is prepared, but “Was it 100% juice?” was retained because children often notice such information on labels.
Answers to probes were also evaluated to determine potential probes for elimination. For example, when answer choices mapped to multiple food codes, but only one answer was reported by 8-15 year olds in NHANES, the probe was eliminated. Likewise, when different answer choices were reported but all mapped to the same food code, the probe was eliminated. These changes would naturally lead to more default coding in a child-friendly (CF) version.
A cross-sectional design was employed to test the feasibility of using ASA24 among children. A convenience sample of 120 children was recruited (Table 1).
All participants were asked to report the previous 24-hour's intake by means of ASA24 and an interviewer-administered recall using the Nutrition Data System for Research (NDSR) (NDSR 2008 for collection; NDSR 2009 for analyses to benefit from time-related updates, University of Minnesota, Minneapolis, MN). NDSR interviewers used a food model booklet with 2-D images for portion size assistance; ASA24 used on-screen, portion size images. Participants were randomly assigned to complete either ASA24 or NDSR first. All parents of the children provided informed consent and all children provided assent. The main results have been reported previously .
To test the hypothesis that the planned elimination of food terms and probes from the ASA24 would not significantly impact mean nutrient assessment, data from children who completed the adult beta version were analyzed as if the child modifications had been in place. So, for example, if the child answered a question about food preparation that this research sought to eliminate, the analyses were modified to assume an answer of “don't know” and the nutrients for the default answer were assigned. For example, for Cheese goldfish crackers, the probe “Were they regular, reduced fat, lowfat, fat free or low sodium?” was eliminated, and the nutrients for the default, “don't know”, were assigned for every instance. In addition, if the child chose a food term slated for elimination, the nutrients for the default were inserted, e.g., “Carbonated water, unknown kind” was replaced with “Sparkling water, unknown kind”; “Apple-grape juice” was replaced with “Apple juice”.
Mean daily total nutrient estimates for energy, protein, total fat, carbohydrate, caffeine, total sugar, calcium, iron, sodium, vitamin C, cholesterol, and saturated fat were compared between ASA24, CF and NDSR. These nutrients were chosen based on their importance in children's diets (e.g., macronutrients, calcium, iron), their abundance in foods eaten often by children (e.g., total sugar, Vitamin C, caffeine) and the likelihood that the changes made for the CF might affect their assessments (e.g., sodium, saturated fat). ASA24, CF and NDSR nutrient estimates were also compared to a nationally representative sample, NHANES 07-08 day 1 24-hour dietary intake data (in-person interview) for 8-13 year olds .
Descriptive statistics (means and standard errors) were calculated for all nutrients for the four assessment methods (ASA24, CF, NDSR, NHANES). NHANES was weighted according to the study design using SAS Survey Procedure (SAS 9.2, 2008, SAS Institute Inc, Cary, NC). Outliers (values above or below three standard deviations) were removed. Pearson correlations were calculated for each nutrient across three assessments. To examine whether the mean nutrient estimates were different, repeated measures analysis of covariance (RM ANCOVA), controlling for covariates (age), was conducted for each nutrient separately to account for the dependence among the measures. To investigate a possible moderation effect by age, a method by age interaction term was included in a repeated measures analysis of variance (RM ANOVA). Both RM ANOVA and RM ANCOVA were completed using PROC MIXED (SAS 9.2, 2008, SAS Institute Inc, Cary, NC) which accommodates missing values without deleting cases. To examine possible differences between ASA24, NDSR and NHANES 07-08, a one sample t-test against a known population mean was used. Given twelve nutrients and two comparisons, the level of significance to account for multiple tests was reduced to 0.0016.
Table 2 displays the means and standard errors divided into two age groups and nutrient assessment method. NHANES tended to have higher nutrient estimates than the other three assessments. These differences were statistically significant where noted by superscripts. Except sodium, the mean difference of nutrients and food groups between ASA24 and CF were quite small; and the difference in sodium was not significant.
Significant assessment method effects were found on total sugar and Vitamin C [F=9.27, df=2, p<.001; F=10.82, df=2, p<0.001, respectively]. Follow-up comparisons of adjusted means indicated that significantly larger total sugar and Vitamin C estimates occurred with ASA24 and CF versus NDSR, but no differences were detected between ASA24 and CF. There was no significant interaction effect between assessment method and age.
In addition to differences in foods and details reported to a dietitian versus those reported via computer that result in nutrient differences, several factors might contribute to the higher total sugar found in ASA24 (97.9) and CF (98.2) as compared to NDSR (76.6). NDSR probes for ice used in beverages, which, if reported, reduces the assigned volume by 25%, while ASA24, based on the method used in NHANES, does not. For sugary beverages, ASA24 always records the entire volume reported, whereas NDSR makes the adjustment above for that same volume if ice was reported. A second factor may be portion size reporting differences between the methods. Portion size estimation is prone to error in both adults and children. Food by food comparisons of portion size did not apply to our research question comparing the ASA24 and CF but did apply to comparisons with NDSR. However, portion size grams across sugary foods such as soda, syrups and popsicles were 70% greater in ASA24 than for similar foods reported in NDSR. In contrast, across all foods, grams consumed were only 12% higher in ASA24. This may indicate that there is a social effect on the child's truthfulness when speaking to a person versus a computer or that the process of entering portion sizes for particular food groups is variable between methods. Further research regarding the comparability of portions reported using 2-D models as with NDSR and digital photos as with ASA24 is needed.
The Vitamin C difference (67.4 in ASA24 and CF; 48.5 in NDSR) can be explained by looking at the reports of juice/drinks in ASA24 versus NDSR. NDSR is interviewer-conducted allowing for details such as brand name or label differences to be queried and thus, possibly better disclosure of the type of beverage consumed. Conversely, in ASA24, a child self selects foods consumed and might choose grape juice even though he actually consumed grape drink. In ASA24, 92% of fruit juice/juice drinks were reported as fruit juice compared to 50% in NDSR.
Of note regarding nutrient differences between ASA24/CF and NDSR is that the nutrient databases behind ASA24/CF and NDSR are different. ASA24 beta version utilized FNDDS1, whereas NDSR maintains its own nutrient and recipe database; thus nutrients for similar foods could vary.
Nutrient totals of study participants were compared to those in the NHANES national sample to see if they were reasonable. Table 2 indicates far more significant differences between NDSR and NHANES than ASA24 and NHANES. For each comparison, the trends were similar across both age groups. The significant differences are most likely due to the differences in the populations studied, the databases (i.e., between NDSR and NHANES) or the methods of data collection (i.e., self-completed versus proxy-assisted).
The correlations between any two of these three assessments (CF, ASA24, and NDSR) were all significant at p < 0.05 (Table 3). The strength of the correlations between ASA24 and CF assessments was very strong as would be expected, but the strength of the correlation between ASA24 assessment and NDSR, or CF assessment and NDSR was moderate to large (Table 3). The lowest correlations were for Total Fat and Saturated Fat between ASA24 and NDSR (.39 and .34, respectively) and between CF and NDSR (.36 and .30, respectively). This may be because dietitians using NDSR can modify a reported food by changing the type of fat or milk used or type of spread, cheese or meat in a sandwich. ASA24 allows for this type of modification through multiple probes about the type of meat, cheese or spread in a sandwich or the type of milk in oatmeal, for example, but it may be a difficult task for children, and thus a potential source of error. In addition, when ASA24 prompts for “Fat used in preparation” with answer choices of butter, margarine, animal fat, etc., not all answer choices map to a unique recipe, unlike what occurs with NDSR. The FNDDS database does not often contain a recipe for each type of fat, so many of the answer choices are mapped to the same recipe. Hence, the Saturated Fat nuances among recipes would not be reflected in the nutrient analysis. This same concept showed minor, but statistically significant differences between data analyzed with FNDDS1 and FNDDS3 based on the update of type of fat used in recipes . Between CF and NDSR, the correlations are slightly lower likely due to the fact that all “Fat Used in Preparation” probes were defaulted to “don't know” in CF.
Forty-five percent of the food terms in ASA24 were eliminated in CF. Only 218, or 14.3%, of the 1520 food terms selected by study participants were food terms eliminated in CF. Many of the eliminated food terms were replaced by food terms mapped to the same FNDDS code; hence, they had the same nutrient values. For example, “bacon (meat substitute)” was eliminated, but its surrogate, “bacon, meatless” was retained. Since both food terms mapped to the same FNDDS code, no nutrient differences resulted from this substitution. Of the 218 food terms that were eliminated, 62.4% were replaced by a food term that had the same FNDDS code, thus precluding nutrient changes. Another 37.6% (82 food terms) were replaced by a food term with a different FNDDS code, but not necessarily different nutrients. Twelve of those 82 had the same nutrient profiles since they had identical recipes. The example of Apple (baked), presented earlier, illustrates this. In FNDDS1, “63101330 Apple, baked, with sugar” and “63101310 Apple, baked, NS as to added sweetener” share the same recipe and thus, the same nutrient profile. In summary, only 70 food terms (less than five percent of all food terms reported by study participants) resulted in nutrient differences.
Forty-six percent of the ASA24 probes were eliminated in CF. Of the food terms reported in this study, 49.6% required that one or more probes be asked to assign a food code. Of those, 16.7% were answered, “don't know”. When a probe slated for elimination was answered with a response other than “don't know”, a “don't know” response was substituted thereby leading to the default assignment of a food code. Only 30.1% of the eliminated probes and less than one percent of eliminated answer choices resulted in a nutrient change.
Portion size choices for remaining food terms were studied, but none were eliminated because, in ASA24, a significant effort is being made to make portion size selection easier and more accurate by including over 10,000 images representing different foods and portion sizes [2, 15].
The elimination of food terms and probes did not significantly impact mean estimates of the selected nutrients between ASA24 and CF. Some nutrient differences between ASA24/CF method and NDSR were likely due to the differences between interviewer- and self-administration. This same finding has been reported with an earlier version of ASA24-Kids . However, these nutrient differences are also a result of the difference between databases used by ASA24 and NDSR. A truer comparison would have been obtained using an interviewer-administered recall that utilized the AMPM probes and the FNDDS nutrient database, but this was not available to us. This research did pinpoint areas for potential improvement for probes used in ASA24 such as presenting the fruit juice/drink choices in a clearer manner, continuing to work on methods for portion size reporting and determining the necessity of probing for ice in beverages. In addition, ASA24 Version 1 has been updated from FNDDS1 to FNDDS4.1 (16) as will the CF database thereby allowing more direct comparisons between ASA24-Kids and national surveys using FNDDS4.1. Limitations of the current study include the assumptions that participants would have made the same food selections and responses to probes when fewer and somewhat different options (reflecting our simulation) were available. Lastly, validation studies comparing ASA24-Kids and true intake through observation are needed to better assess method-specific limitations and areas for improvement.
The child-friendly simulation showed that it is feasible to implement child-friendly modifications in the ASA24 database, thereby reducing respondent burden, without significantly affecting the mean nutrient estimates. These proposed adaptations have been incorporated into a child version of ASA24, called ASA24 -Kids, which is available to researchers on the NCI web site .
This research was funded by a grant from the National Cancer Institute (5 U01 CA130762) and institutional support from the US Department of Agriculture, Agricultural Research Service (Cooperative Agreement no. 58-6250-6001). The authors have no conflict of interest.
Conflict of Interest: The authors have none to report.