|Home | About | Journals | Submit | Contact Us | Français|
To examine the agreement in nutrient intake and alternative healthy eating indices (AHEI) between a self-administered Food Frequency Questionnaire (FFQ) and 24-hour recall (24HR) measurements of diet by race, among urban older women.
Cross-sectional observational study.
Urban neighborhoods in Washington, DC, USA.
Community-dwelling White and Black women aged 65 and older.
In 2014 and 2015, 49 White and 44 Black older women were queried on diet using both FFQ and 24-hour recalls. The correlation coefficients of 55 nutrient intake measures and agreements on healthy eating classification between the two instruments were compared overall and by race.
The mean correlation coefficient (rho) was 0.46 for Whites and 0.23 for Blacks. For 47 measures, rho was lower for Blacks. Whites had a strong correlation of ≥0.5 for 28 items, while Blacks had strong correlations for only 3 items. Based on FFQ, the mean (SD) of AHEI were 54.0 (10.3) for Whites and 45.9 (8.8) for Blacks (p<0.001). Based on 24HR, the mean (SD) were 43.9 (10.8) for Whites and 33.2 (9.6) for Blacks (p<0.001). Using 32 as the cutoff (40% of maximum AHEI score), 50% of Blacks and 14% of Whites were classified as eating unhealthy based on the 24HR, versus 2.6% and 0% based on the FFQ.
The FFQ has limited ability to accurately assess nutrient intake among older Black women, and tends to underestimate racial differences in healthy eating. The FFQ should be further improved for use in racial disparities research of healthy eating in older age, using a larger sample of older women with racial and geographic diversities.
Accurate and cost-effective measurement of dietary intake and diet quality is important for evaluation and promotion of healthy eating among older adults. Accurate assessment may help to identify specific unhealthy eating behaviors, inform personalized nutrition counseling, and monitor population trends in adherence to dietary guidelines.
Dietary assessment is prone to errors, regardless of the method of measurement.(1) Commonly used methods include the 24-hour dietary recall interviews (24HR) and food frequency questionnaire (FFQ). The interviewer-administered 24HR is often preferred as the “gold standard”, against which a FFQ is calibrated.(2) It requires a professionally trained interviewer to administer multiple telephone interviews with the respondents to capture the intra-person variability of diet.(3) During a call, the interviewer facilitates portion and ingredient description to improve recall accuracy. However, this method is costly and may impose an unacceptable burden on respondents. In contrast, the self-administered FFQ is a low cost alternative and a frequent measurement of choice for diet when summarizing nutrient intake is acceptable, the funding for the more arduous 24HR is lacking, or reaching participants by telephone is too difficult. A typical FFQ includes a finite list of foods and food groups from which participants quantitatively report how often each item was consumed over a specified period of time. Portion size is collected according to categories of standardized portion sizes. Various forms of the FFQ have been utilized in many large scale studies including the Women’s Health Initiative (WHI),(4) Jackson Heart Study(5) and the National Health and Nutrition Examination Survey,(6) These studies have provided critical data to inform the past and current national dietary guidelines.
Assessment of dietary intake in community settings, however, presents numerous challenges. Accuracy of the assessment can be impeded by a respondent’s personal characteristics, health status and the place where he/she lives. Relevant sociodemographic factors include age, sex, race, culture, education, income, language, home environment and living arrangements. In addition, many older persons experience decline in cognitive function and memory, which affects awareness of intake and accuracy of report.(7) Social desirability or pressure regarding body image and weight can also impede reporting accuracy, particularly among overweight women.(8) Nutrition literacy may play an important role in recall accuracy, such as understanding ingredients when describing and reporting meals, description of portion sizes, and awareness of foods and beverages consumed.(9) The accuracy also may be affected by the availability of specific food items in a geographic area, and a respondent’s proximity to and ability to access food stores.(10, 11) Because these influential factors may differ among individuals with different racial and ethnic backgrounds, reliability and accuracy of dietary measures are of great concern when examining racial disparities in diet and healthy eating.
To assess the utility of the self-administered General Nutrition Assessment (GNA) FFQ of the Fred Hutchinson Cancer Research Center (FHCRC)(12) in community-based racial disparity studies, this paper examined the agreement in nutrient intakes and the alternative healthy eating indices (AHEI)(13) between the FFQ and 24HR measurements of diet by race, among urban older women.
The GNA FFQ was developed by the Nutrition Assessment Shared Resource (NASR) of the FHCRC. The NASR periodically updates its standard FFQ based on current U S. food consumption patterns and major changes in the market place. The GNA FFQ has similar measurement characteristics as the WHI FFQ.(14) The current GNA FFQ was updated in late 2010, and the nutrient database is updated annually in accord with annual updates from the University of Minnesota Nutrition Data Systems for Research (NDSR). This self-administered FFQ asks participants to report the frequency of consumption and portion size of approximately 125 line items over a defined period of time (e.g. the last month). Each line item is defined by a series of foods or beverages. Additional questions on food purchasing and preparation methods enable the analysis software to further refine nutrient calculations. The NDSR 2014(14) was used to analyze the FFQ data from the study.(15, 16) Many studies have used this FFQ or its predecessors to assess nutritional risk factors for chronic diseases and injuries as well as health behaviors among postmenopausal women. We examined the potential utility of the FFQ in a pilot study of racial disparities in healthy aging among urban older women.
In 2014 and 2015, 49 White and 44 Black/African American community-dwelling women and 4 women with undisclosed race, aged 65 years and older, living in the Washington, DC Metro area were enrolled in the study. Recruitment methods are described elsewhere.(17) Briefly, eligibility includes self-identified as Black or White race, aged 65 and older, English speaking, ambulatory, and capable of participating in all aspects of the study. Four women who expressed strong interest but were not willing to classify themselves as White or Black at the time of assessment also were included. Once consented, each participant completed surveys of demographics, health behaviors, physical activity, health and health care, food purchasing habits and diet. Physical activity was also measured objectively by accelerometer (Actigraph GT3X-Plus), worn for 7 consecutive days during all waking hours by each participant. The study protocol was approved by the MedStar Health Institute Review Board (Docket # 2014-261).
The participants were queried about their diet using the GNA FFQ and then 24HR. The FFQ was administered before the 24HR, and all measurements were performed within 3 weeks of each other. Each participant received three unannounced computer-assisted 24HR (NDSR 2011), conducted on randomly selected two weekdays and one weekend within a 1-week period. The 24HR is a comprehensive food and nutrient calculation software largely used for research purposes. Dietitians who are trained and certified in utilizing the software and conducting the interviews administer the calls, through a multiple pass protocol.(17) Annually updated to capture changes in the food supply, the database is designed to allow missing foods to be included on a nutrient level, such that there are virtually no missing values in the NDSR.(15, 18)
Dietary outcomes of primary analytic interest in our study included average daily total caloric intake and measures of diet quality, including consumption of fruits and vegetables, legumes, nuts, averaged daily intake of total protein, fats (saturated, poly- and monounsaturated fats), types of carbohydrates, fibers, and micronutrients (such as sodium and calcium).
An alternate healthy eating index (24HR AHEI) was calculated for each participant based on her 24HR intake, modifying the formula developed by the USDA Center for Nutrition Policy and Promotion.(19) The overall index had a total possible score ranging from 0 to 80 points, with higher scores indicating better overall dietary quality. Dietary component subscores were calculated for 8 components, including vegetables, fruits, nuts and legumes, ratio of white to red meat, cereal fiber, alcohol, trans fats, and ratio of polyunsaturated to saturated fats. Each component score had a scoring range of 0 to 10 points. In parallel, a FFQ-based AHEI score was calculated for each woman using the same 8 component scores. Comparable measures were available in the 24HR and FFQ data for all components except for the nuts and legumes subscore. Servings per day of nuts and legumes were not directly available from the FFQ, so this component score was estimated from the vegetable protein intake reported in the FFQ and scaled to match the point range of the same component in the 24HR AHEI. Total AHEI scores were categorized as “poor” in nutritional value if less than 40% of the maximum score, i.e., <32 points, following the procedure of Rehm and associates.(20)
Participant characteristics were compared by race. White to Black racial differences in sociodemographic, physical and mental health, and lifestyle factors were evaluated using Chi-squared tests for percentages or Wilcoxon rank-sum tests for continuous variables.
Dietary intake values of 55 nutrient items were compared by race within both the FFQ and 24HR measurements using unadjusted linear regression models. For each nutrient, the Pearson’s product moment correlation (rho) between individuals’ FFQ and 24HR measurements was estimated by race, along with its 95% confidence interval based on Fisher’s transformation. Actual discrepancies between the FFQ and 24HR measurements were examined item by item. For each nutrient item and each participant, the raw discrepancy was calculated (24HR measure - FFQ measure) as well as percentage discrepancy [absolute value (100*(24HR-FFQ)/24HR))]. The mean percentage discrepancies were compared by race using linear regression models: 1) unadjusted; 2) adjusted for age; 3) adjusted for gross family income and years of education; and 4) adjusted for age, income, and education. Assumptions of linearity of the associations between absolute discrepancies and age, income, and education were tested and found not to be violated. A sensitivity analysis was performed in which we excluded one woman with only two 24HRs and two women (one White, one Black) with <600 mean daily kcal, and the results were not notably affected, so they are included in the analysis presented here.
AHEI total scores and component subscores were summarized and compared for racial differences using the Wilcoxon rank-sum test. Pearson’s correlations (rho) between the FFQ and 24HR scores also were estimated. The mean differences between the 24HR- and FFQ-based AHEI scores were calculated. These were compared for racial differences using unadjusted linear regression models. Kappa scores and percent agreement were calculated for the classification of total AHEI scores into poor vs. better dietary quality by the FFQ vs. 24HR method. A scatter plot along with fitted linear regression lines and AHEI cutoff points at 32 was drawn to illustrate the racial differences in the distributional characteristics, correlations between FFQ and 24HR based AHEI scores, and misclassification of poor vs. better diet quality.
Of the 97 participants, 49 were White, and 44 were Black women (Table 1). Compared to Whites, Blacks had, on average, younger age, less education and lower household income. Blacks were more likely to have diabetes (34% vs. 12%, p=0.01), less likely to have osteoarthritis (14% vs. 31%, p=0.05) and osteoporosis (2% vs. 18%, p=0.01), fewer outdoor falls in the past 6 months (0.01 vs. 0.03, p=0.01), and a lower level of anxiety (4.8 vs 6.3 points, p=0.05). Black women were also less likely to live alone (32% vs. 63%, p=0.002), used alcohol less frequently (0.7 vs. 2.6 times weekly, p<0.001), and had lower daily step counts (3256 vs. 5457, p<0.001).
Racial differences in nutrient intakes were more evident in the 24HR results than in the FFQ results (Table 2). Based on the FFQ, compared to Whites, Blacks reported substantially lower consumption of alcohol, dietary fiber, lycopene, potassium, and sucrose and somewhat higher intake of trans fatty acids. Other nutrients did not differ between Whites and Blacks to any significant degree (p>0.10).
However, based on 24HR, compared to Whites, Blacks reported a lower daily caloric intake, lower consumption of vegetable proteins, fiber of all types, and micronutrients such as calcium, iron, magnesium, potassium, vitamin B-6, C, and D. They reported much less caffeine and alcohol consumption. Black women reported a higher percentage of intake from total protein and fat, and foods with vitamin K (typically green, leafy vegetables). Blacks had a higher polyunsaturated/saturated fat ratio and higher percentages of calories from monounsaturated, polyunsaturated, and trans fats.
Examination of the correlations (rho) between FFQ and 24HR measurements revealed a remarkable racial difference (Table 3). The mean rho for the 55 dietary items for Whites was 0.46, while for Blacks it was 0.23, and rho for Blacks was less than that for Whites in 47 of the 55 measures. Whites had a "strong correlation"(21) of 0.5 or greater for 28 of the 55 items, while Blacks had strong correlations for only 3 items, and these all concerned the percent of intake from the fats in the diet.
The FFQ showed a greater agreement with the 24HR for Whites than for Blacks (Table 4). For every nutrient in which there was a significant (p<0.05) White-Black difference in average percent discrepancy between the two instruments, Blacks had a greater discrepancy than Whites. A comparison of the means of raw discrepancies does not show the same Black/White difference, but standard deviations of the discrepancies tended to be higher for Blacks, indicating a greater variability among Blacks. Black were not consistently underestimating or overestimating intake, but had larger discrepancies in both directions.
Adjustment for age did not affect the apparent difference in percent discrepancies between Blacks and Whites. Adjustment for income and education, however, accounted for some of the racial differences in 24HR-FFQ discrepancies (Table 4). For example, compared to the White-Black difference in the crude analysis, adjustment for income and education level decreased the racial difference seen for total grams, fiber, lutein/zeaxanthin, magnesium and vitamin A by 20%, 27%, 78%, 34% and 56%, respectively. However, adjustment for income and education did not notably affect the difference for total calories or animal protein, which remained significantly more disparate for Blacks.
By both the FFQ and 24HR measures, total AHEI scores were significantly lower for Blacks than Whites (Table 5). Whites had higher subscores in nuts and vegetable proteins, cereal fiber, alcohol, and percent of calories from trans fat according to both measures. Under the 24HR measure only, Whites also received significantly higher scores for vegetable and fruit intake. FFQ-based AHEI scores and subscores tended to be higher than 24HR-based AHEI scores as shown by their means and by the negative signs for the mean differences (24HR-FFQ) for all subscores except the nuts and legumes subscores of White women.
Correlations between 24HR and FFQ-based AHEI scores were higher for Whites in every subscore category as well as in total score (Table 6). Using 32 points as the cutoff (40% of maximum AHEI score), 50% of Blacks and 14% of Whites were classified as having a poor diet based on 24HR versus 2.6% and 0% based on FFQ, respectively. The percent agreement between FFQ and 24HR AHEIs for poor diet classification, as determined by the kappa test, was 55% for Blacks but 84% for Whites. Varying cutoff values did not substantially improve percent agreement for Blacks. As shown in Figure 1, while all FFQ AHEI scores tended to be higher than the corresponding 24HR AHEI scores (above the equality line), those in the lowest range tended to have the greatest discrepancy (i.e., line of best fit is furthest from the equality line in the lower region).
The GNA FFQ has been of vital importance to understanding older adults’ nutrition and health, and applied in numerous large scale studies.(17, 22–27) Given the significant contribution this cost-efficient measurement of diet provides, it is important to examine its potential utility in racial and geographic disparities research. We successfully engaged 97 community dwelling elderly women in a rigorous evaluation of diet, self-report and objectively measured PA and mobility, neighborhood perceptions, and use of neighborhood resources. Measurements were timed to be less than three weeks apart for the majority of women in order to control for seasonal and typical dietary variation. In addition, the study carefully measured a large number of sociodemographic (e.g., education and income), lifestyle and health characteristics of the participants from a well-defined urban area, which allowed us to explore personal factors that may influence reporting accuracy and calculate covariate-adjusted racial differences between Whites and Blacks.
This analysis demonstrated that the FFQ agrees more with three 24HR in White than in Black older women living in urban neighborhoods in the Washington, DC metro area. Compared to the 24HR, the FFQ nutrient intake measures had weak to modest correlations for most of the energy and nutrient intake measures among Blacks. For assessment of healthy eating status, the FFQ tended to overestimate AHEI scores and the proportion of participants designated as eating healthy, for both Blacks and White, with greater overestimates for Blacks.
Underreporting of energy and protein intake are known to be systematically biased by body mass index (BMI) and to a less extent by race for an FFQ(28, 29) and 24HR and dietary food record.(25) The Nutrient Biomarker Study by Neuhouser and colleagues examined the WHI FFQ against biomarkers.(28) The FFQ was found to be systematically biased related to BMI, age, and ethnicity. Biomarkers were utilized to develop regression calibration equations to correct estimates of energy and protein, and proved essential for providing association with cancer and coronary disease in WHI cohorts.(30, 31) Further examination of the utility of these calibration equations against biomarkers revealed a greater underestimation of energy and protein by women who were in the minority group.(25) The calibrated consumption estimates, however, may be dependent upon BMI which was not collected in the current study.
Overall, White women’s dietary measurements in the current study were commensurate with the original measurement characteristics study results of the WHI FFQ against the 24HR. The current study’s correlation coefficients from the FFQ compared to those obtained by Kristal et al(23) for Whites (rho=0.49), and to Patterson et al(14) for a largely White population (0.45), are nearly identical to our White mean rho of 0.46. While the FFQ showed relatively fewer racial discrepancies in the current study, the 24HR demonstrated significant racial differences in calories, several micro and macronutrients, and overall dietary quality. Consistent with our findings, previous studies have questioned the relative accuracy of the FFQ as measured against the 24HR, particularly in minorities.(23, 32) These discrepancies are concerning because large scale studies based on the FFQ may underestimate racial disparities in diet, which has important implications for research into cardiovascular disease, diabetes, musculoskeletal disorders, and various cancers.(3)
The WHI FFQ was developed in 1995 and geared toward measuring fat in the diet.(4, 14) The subsequent GNA FFQ and 24HR are in close agreement when estimating total fat and saturated fat intake for Black and White women, indicating this may still be a good measure in both populations. However, current dietary research addresses both micro- and macronutrients, as both are important to healthy dietary quality. Given the vital importance of the FFQ in large scale health disparity research, better calibration of this instrument, based on biomarkers and data of larger size and from more geographic locales may be necessary. This is especially important for racial disparity research in older populations, and will empower the FFQ to continue to measure older adults’ diet with confidence regardless of their racial and ethnic background.
The intake measures differed substantially by participant education and socioeconomic status. Such differences helped explain some of the observed racial differences in FFQ- and 24HR-based dietary intake measures. Underreporting is common in dietary assessment is a challenge in (or resistance to) estimating portion sizes.(6, 9) In the current study, Blacks had a much lower FFQ-24HR correlation on the “total grams” item, which seems to point to portion size estimation problems, in addition to specific ingredients in the diet that may have been missed. Future studies may improve the FFQ accuracy by provision of interviewer assistance and food portion visuals.(32)
Aside from errors in portion estimation, causes of nutrient calculation errors are important to identify, particularly since Black women in this study have a higher prevalence of diabetes. There were few differences in nutrient intake identified by the FFQ, yet many differences identified by the 24HR that suggest recommendations of improvement in diet quality for Black women.
When choosing an instrument for nutrition research or evaluation, it is important to consider inherent methodological differences between instruments. The 24HR is usually considered to be the more representative because the FFQ utilizes a fixed food list. Careful understanding of the intended nutrient capture matters as some FFQs may be targeted to general dietary intakes and others may be targeting specific foods or nutrients. The FFQ prompts therefore may not capture certain foods and food preparations unique to racial/ethnic groups or geographic regions. Further study, directed at identification of culturally appropriate particular foods and their preparation for this population may be necessary.
The original WHI FFQ was assessed for measurement characteristics from 24HR and 4-day food record. The percentage of Black women in the sample was proportional to the planned recruitment into WHI. The intent of the measurement characteristics study was not to compare races. However, by recruiting a geographically and racially diverse sample of larger size, it is possible to provide race/ethnic or geographic region specific calibrations against 24HR. Further validation studies using biomarkers may substantially improve accuracy of both macro-and micronutrient measurements and enhance utility of the FFQ to racially diverse older populations.
Because FFQs are also often used in dietary studies in other countries, such as the Hatoyama Cohort Study and Kusatsu Longitudinal Study,(33) cultural and racial/ethnic appropriateness and geographical relevance are important methodologic issues in international dietary studies. More research is needed to assess the comparability and local adaptation of existing FFQs for future cross-cultural, multi-country dietary studies.(34, 35)
In addition to better calibration of FFQ with 24HR among racial and ethnic minorities, contemporary biostatistical models could also be used to lessen the problem, such as error in variable regression models.(36, 37) Such methods have been well developed and implemented in commercial statistical software packages, such as Stata (Stata Corp., College Station, TX). However, dietary measures are often used as predictor variables without error in analysis of nutritional risk factors for diseases and injuries. Large errors associated with dietary measurements may attenuate the associations under analysis, and distort the picture of racial disparities in diet and healthy eating. Greater use of error-in-variable models may reduce the impact of measurement errors, and increase the statistical power and value of the analysis.
Several limitations of the present study should also be noted. The participants were not shown visual aids to assist in estimating portion sizes prior to the FFQ or the 24HR. A packet of printed visual aids is useful when a respondent reports portion sizes, but often of relatively large size. Due to concerns about excessive burdens to frail older adults, a packet of visual aids was not included in the participant survey packets. Both FFQ and 24HR are subject to recall accuracy and bias. The 24HR had dietitians trained to assist women with portion descriptions, but since this generally occurred after the FFQ, any learning of portion reporting from the 24HR would have been too late to aid the accuracy of the FFQ. Both FFQ and 24HR are subject to recall accuracy and social desirability bias. Another limitation is the lack of a BMI measurement. Body weight and BMI have been associated with social desirability bias in other studies.(8, 38) This study could not examine its impact. However, this seems less likely as an explanation for the discrepancies, since the discrepancies for Blacks were not necessarily directional toward healthier choices (Table 3).
Our analysis was limited to community-dwelling older White and Black women living in urban neighborhoods in a large metropolitan area. To fully understand the strengths and limitations of the GNA FFQ, it is important to conduct similar comparative analysis in populations of other racial and ethnic background, women of younger age, men, and individuals living in urban as well as rural neighborhoods in other geographic regions across the country. This is especially important for assessing disparities in healthy eating because racial, socioeconomic and geographic disparities may confound with each other. Proper use of the instrument and a good understanding of its limitations may provide a clearer picture of disparities in healthy eating, and better inform future research and policy decisions.
Although the GNA FFQ has played a vital role in discovery of nutrition-related risk factors for chronic diseases and injuries among older women, its use in analysis of racial disparities in healthy eating may be limited. Further modifications to the FFQ and better calibration with 24HR for Blacks may substantially improve the instrument’s ability to fully capture the dietary patterns of non-White populations, thus making it more effective as a tool for use in future research into racial disparities in healthy eating in older age. Efforts should be made to realize this goal, since a large number of past, existing and future population-based large scale studies will benefit, and their findings can be also applicable to racial and ethnic minorities.
Contribution / authorship statement
Li and Magee conceived the study and obtained the funding. Youssef managed financials, IRB application, recruitment, measurements and data collection in the field. Olendzki managed the 24HR dietary recall data collection and interpreted dietary intake data. Kane was responsible for database management. Li, Olendzki and Procter-Gray conducted the statistical analysis and drafted the manuscript. All authors contributed to the revision, read and approved the manuscript.
This project was supported in part by a grant from the Georgetown-Howard Universities Center for Clinical and Translational Science and MedStar Health Research Institute (Co-PIs: Michelle MaGee and Wenjun Li; from UL1TR000101), and by a Life Science Moment Fund Award (PI: Li) of University Massachusetts Center for Clinical and Translational Sciences which is funded by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1TR000161. Several instruments used in this project were created in a pilot study supported by NIH-funded Women's Health Initiatives (HHSN268201100001C). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
No conflict of interest declared.