PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Public Health Nutr. Author manuscript; available in PMC 2012 August 13.
Published in final edited form as:
PMCID: PMC3417357
NIHMSID: NIHMS398659

Validation of nutrient intake using an FFQ and repeated 24 h recalls in black and white subjects of the Adventist Health Study-2 (AHS-2)

Abstract

Objective

To validate a 204-item quantitative FFQ for measurement of nutrient intake in the Adventist Health Study-2 (AHS-2).

Design

Calibration study participants were randomly selected from the AHS-2 cohort by church, and then subject-within-church. Each participant provided two sets of three weighted 24 h dietary recalls and a 204-item FFQ. Race-specific correlation coefficients (r), corrected for attenuation from within-person variation in the recalls, were calculated for selected energy-adjusted macro- and micronutrients.

Setting

Adult members of the AHS-2 cohort geographically spread throughout the USA and Canada.

Subjects

Calibration study participants included 461 blacks of American and Caribbean origin and 550 whites.

Results

Calibration study subjects represented the total cohort very well with respect to demographic variables. Approximately 33 % were males. Whites were older, had higher education and lower BMI compared with blacks. Across fifty-one variables, average deattenuated energy-adjusted validity correlations were 0·60 in whites and 0·52 in blacks. Individual components of protein had validity ranging from 0·40 to 0·68 in blacks and from 0·63 to 0·85 in whites; for total fat and fatty acids, validity ranged from 0·43 to 0·75 in blacks and from 0·46 to 0·77 in whites. Of the eighteen micronutrients assessed, sixteen in blacks and sixteen in whites had deattenuated energy-adjusted correlations ≥0·4, averaging 0·60 and 0·53 in whites and blacks, respectively.

Conclusions

With few exceptions validity coefficients were moderate to high for macronutrients, fatty acids, vitamins, minerals and fibre. We expect to successfully use these data for measurement error correction in analyses of diet and disease risk.

Keywords: Epidemiological methods, Ethnic groups, Questionnaires, Validation studies

The self-administered FFQ, although a more practical tool for assessing individual diets in large cohorts, has associated measurement errors that usually attenuate relative risk estimates in studies of diet and disease risk(13). One approach to improving effect estimates is through a calibration study where assessment of individual diets by questionnaire is compared with a more precise method, such as repeated 24 h recalls(4). The goal of the present report is to assess the validity of nutrient intake as measured by the FFQ used in the Adventist Health Study-2 (AHS-2) by comparison with repeated 24 h dietary recalls in a sample of black and white Adventists.

The AHS-2, a prospective study of over 96 000 adult Adventists in the USA and Canada, has the goal of relating lifestyle choices, particularly diet, to health outcomes(5). The AHS-2 cohort is relatively unique for its wide range of dietary patterns compared with the general Western population. Specifically, 8 % are vegan, 28 % are lacto-ovo-vegetarian, 16 % are semi- or pesco-vegetarian, and 48 % are non-vegetarian. There is also large variation in the consumption of plant foods, such as nuts, soya, other legumes and grains. The FFQ administered in the study was designed specifically to assess the wide range of dietary intake among a population where a large proportion is vegetarian. A calibration and validation study is an integral part of the AHS-2; the current paper describes that study and examines the validity of macro- and micronutrient intakes estimated from FFQ data.

Experimental methods

Calibration sub-study design

The AHS-2 cohort consists of adult members of Seventhday Adventist churches geographically spread throughout the USA and Canada. Over 25 000 cohort members are blacks; the remaining subjects are of other races, mostly white. Recruitment and selection methods of the parent cohort have been described previously(5). Participants in the calibration study were randomly selected from the parent cohort by church, and then subject-within-church. Because of the special focus on black Adventists as a minority group in the AHS-2, the calibration study was designed such that there would be approximately equal numbers of blacks and whites. No significant differences in the distribution of gender, age, education or vegetarian status between the calibration sub-study sample (n 1011) and the AHS-2 cohort (n 96 592) were observed. When stratifying by ethnicity, only the distributions of mean age and age categories in blacks were nominally significantly different (P = 0·0018 and 0·0032, respectively) between the parent cohort and the calibration study sample (Table 1). This is easily consistent with chance given the multiple testing involved.

Table 1
Comparison of selected characteristics between the Adventist Health Study-2 (AHS-2) cohort and calibration study participants according to race, 2003–8

The duration of the calibration study was 9–12 months for any one subject (see Fig. 1). We obtained a set of three variably timed 24 h dietary recalls (one Saturday, one Sunday and one weekday intake) during the first two months, which then was duplicated approximately six months later. The aim was to obtain two sets of recalls (a total of six 24 h recalls) from each subject. During the 6-month interval, subjects completed a self-administered food FFQ that was identical to the baseline FFQ and attended a local study clinic. The study was approved by the institutional review board of Loma Linda University, and all subjects gave written informed consent.

Fig. 1
Adventist Health Study-2 study design

Dietary assessment

The 24 h dietary recalls were unannounced and obtained by telephone. A two-dimensional food portion visual (2D Food Portion Visual; Nutrition Consulting Enterprises, Framingham, MA, USA) was sent to each participant before the first recall to assist with portion size estimates. Trained research dietitians used standard probes and a multiple-pass approach methodology to collect detailed information on all foods, beverages and supplements consumed by each subject during the previous 24 h. Each recall interview was entered using Nutrition Data System for Research (NDS-R) version 4·06 or 5·0 (The Nutrition Coordinating Center, Minneapolis, MN, USA) and the conversation digitally recorded for subsequent quality check. An experienced research dietitian later evaluated randomly selected recall interviews ( ~5 %) and compared them with the recording, as a quality control measure.

The AHS-2 FFQ is a quantitative and comprehensive 22-page instrument consisting of 204 foods, fifty-four questions about food preparation and forty-six fields for open-ended questions. Frequency categories vary with food type to allow respondents to define their daily intake with greater specificity. Thus, the lowest category for most foods and beverages is never or rarely, and for cold cereals and vegetarian protein products, 1–3 per month. The highest frequency category for vegetables, soups, cereals, pasta, dressing, meats, fish, vegetarian protein products and soya milk is 2 or more per day; for nuts and seeds, 4 or more per day; for breads, eggs, dairy products, snacks and beverages, 6 or more per day. Portion sizes include three levels: standard, equation M1 or less, and equation M2 or more. Standard portions are based on serving sizes using familiar household units such as cup, tablespoon, slice, patty and others. Pictures of common foods or beverages typically served together were included with the questionnaire to assist subjects in estimating portion sizes. For example, to represent a standard portion, spaghetti, broccoli and steak are arranged on an 11-inch dinner plate, each food measured in a standard portion. Representations of equation M3 of standard and equation M4 times the standard portion size are provided as well, using the same foods and dinner plate.

The AHS-2 FFQ was designed to include foods commonly eaten by US Adventists and later modified to accommodate foods specific to black Adventists of US and Caribbean origin(6). The questionnaire was sent to each subject, completed at home, and then mailed back to AHS-2. Respondents were asked to report on their intake over the previous one year. Upon receipt of the questionnaire, study personnel reviewed the questionnaire for completeness and as necessary followed-up by telephone to clarify any ambiguous or incomplete information.

Completed questionnaires were optically scanned using the NCS 5000i Image Scanner with ScanTools Plus software (Pearson NCS, Bloomington, MN, USA). Standardized processing of open-ended questions was done using the Food Write-In Processing software (Adventist Health Study-2, Loma Linda, CA, USA), a network-based application created in Microsoft® Access (Microsoft Corporation, Redmond, WA, USA).

Nutrient composition of foods reported from 24 h recalls and FFQ were based on the NDS-R 5·0_35 database (The Nutrition Coordinating Center), an analytic database of over 20 000 foods updated annually while maintaining nutrient profiles true to the version used for data collection(7). Nutrient profiles of foods and supplements not found in the NDS database were obtained from the US Department of Agriculture, manufacturers, and the Caribbean Food and Nutrition Institute. Considerable attention was given to creating recipes for home-cooked vegetarian dishes (n > 500), home-made and commercial soya and nut milks (n > 180) and commercial meat analogues (n 309) frequently consumed among our study population. For the latter we contacted manufacturers or worked with a senior food technologist with experience in this industry, to create recipes.

Statistical analysis

Within each of the two sets of 24 h dietary recalls, each day was weighted appropriately to produce a synthetic week (Saturday intake + Sunday intake + 5 × weekday intake) and then divided by 7 to obtain mean daily nutrient estimate. Dietary habits among Adventists are often rather different on Saturdays, Sundays and typical weekdays. Thus, for those who completed six 24 h recalls (n 950), mean daily intake was calculated by dividing the total of the two synthetic weeks by 14. For those sixty-one subjects who completed only three, four or five recalls, only the first three recalls (first synthetic week) were used in these analyses. Nutrient estimates from FFQ data were calculated using the product-sum method(8). Thus, nutrient intake = sum [(weighted frequency of use of a food) × (weighted portion size consumed of that food) × (amount of that nutrient in a standard serving size of that food)]. FFQ with estimated energy intake <2093 kJ (<500 kcal) or >18 833 kJ (>4500 kcal) were excluded from analyses. Arithmetic means and standard deviations were calculated for nutrient intakes assessed by 24 h recalls (R) and FFQ (Q). To determine differences, Student’s t tests were used for continuous variables and χ2 tests for categorical variables.

Nutrients were energy-adjusted using the residual method(9). However, for vitamins, minerals and fatty acids also commonly used as supplements, unadjusted supplemental intake was added to energy-adjusted dietary values, resulting in energy-adjusted total intake. Nutrient validity correlations, corr(R,Q), were calculated between averages of the two synthetic weeks of recalls and intake measured from FFQ. Where the linear regression implied by the correlation coefficient had residuals with skew ≥2·0, both dietary quantities were log-transformed.

After energy adjustment, we determined correlations for corr(Q,R) and then corrected the correlation coefficients for attenuation due to within-person variation in the recalls(3). The deattenuated energy-adjusted correlation coefficients, rc, that we report are estimates of equation M5, where var(T) is estimated using the repeated measurements of R. This allowed us to also use the 6 % of subjects who did not complete the second week of recalls, assuming that their within-person variances did not systematically differ from the others.

A bias factor was calculated as the proportional bias, (crude – true)/true, that would be observed in a regression coefficient if the uncorrected FFQ estimate were used as the independent variable in a regression where in fact the mean of a large number of recall estimates is the truth.

We calculated 95 % confidence intervals for all validity coefficients using bootstrap resampling and the BCa method(10). Analyses were performed using the SAS statistical software package version 9·1 (SAS Institute Inc., Cary, NC, USA).

Results

The 1011 subjects in the calibration study represented the total AHS-2 cohort very well, both with respect to demographic variables and dietary practices (Table 1). There were more females than males regardless of race. Whites were older, had higher education and lower BMI compared with blacks. A greater proportion of blacks than whites were semi- and non-vegetarians.

Nutrient intake estimates from the FFQ and the 24 h recalls by race are shown in Table 2. Intakes of nutrients estimated from 24 h recalls were in general higher for whites than blacks, except for trans and very-long-chain fatty acids, protein, animal protein and vitamin D. Nutrient intake estimates from the FFQ were also generally higher in whites; however, blacks had higher intakes of very-long-chain fatty acids, total energy, protein, animal protein, carbohydrate, non-fibre carbohydrate, Fe, K, and vitamins D and C.

Table 2
Comparison of nutrient intake as measured by the FFQ and 24 h recalls in the Adventist Health Study-2 calibration study participants, 2003–8

All uncorrected and deattenuated correlation coefficients reported for selected macro- and micronutrients were energy-adjusted (Table 3 and Fig. 2). Correlations improved after correction for attenuation in the recalls on average by 28 % and 34 % in white and black subjects, respectively. Average deattenuated validity correlations were higher in whites compared with blacks (r = 0·60 and 0·52, respectively). The lowest validity correlations in both races were for energy: r = 0·09 (95 % CI –0·01, 0·18) in blacks and r = 0·24 (95 % CI 0·15, 0·32) in whites. Nutrients with the highest deattenuated correlations were docosapentaenoic acid (22 : 5n-3) in blacks (r = 0·75, 95 % CI 0·70, 0·79) and animal protein in whites (r = 0·85, 95 % CI 0·83, 0·87). Individual components of protein (animal, vegetable, soya and dairy protein) had validity correlations ranging from 0·40 to 0·68 in blacks and from 0·63 to 0·85 in whites, but total protein was much lower in both races (r = 0·39 and 0·18 in whites and blacks, respectively; see further comment in the discussion section). Deattenuated correlations for total fat and fatty acids ranged from 0·43 to 0·75 in blacks and from 0·46 to 0·77 in whites. Regardless of race, fibre and its components had higher validity compared with total carbohydrate. Of the eighteen micronutrients assessed, sixteen in blacks and sixteen in whites had deattenuated correlations ≥0·4, averaging 0·60 and 0·53 in whites and blacks, respectively.

Fig. 2
Comparison of selected deattenuated energy-adjusted correlation coefficients between the FFQ and 24 h dietary recalls in black subjects (An external file that holds a picture, illustration, etc.
Object name is nihms-398659-ig0003.jpg) and white subjects (An external file that holds a picture, illustration, etc.
Object name is nihms-398659-ig0004.jpg) of the Adventist Health Study-2, 2003–8
Table 3
Energy-adjusted validity correlations between the FFQ and 24 h dietary recalls in Adventist Health Study-2 calibration study participants, 2003–8

The average bias factor was greater in blacks compared with whites (–0·70 and –0·58, respectively), but in both ethnic groups it is clear that use of uncorrected FFQ data in a regression would result in severely attenuated effect estimates.

Discussion

Participants of this relatively large (n 1011) calibration study were representative of the parent cohort with respect to gender, age, education, BMI and dietary patterns. Because of our initial selection by church, subjects also were geographically spread throughout the USA and Canada. As a reference measure the present study obtained six 24 h recalls, whereas other cohorts with large calibration samples typically used fewer recalls(4,11,12) or a 4 d food record(13) that did not systematically combine weekdays and weekends.

Intake estimates of most nutrients from both FFQ and recall data generally were higher in whites compared with blacks. However, blacks reported higher intakes of protein, animal protein, very-long-chain fatty acids and vitamin D. This is not surprising, given the greater proportion of non- and semi-vegetarian eating patterns followed by this minority population in the study. As expected, nutrient intake estimates derived from FFQ tended to be higher than those from 24 h recalls. This probably results when people are asked to recall the frequency of intake of a large number of foods in an FFQ and thus tend to overestimate their actual intake(9). In a study that assessed bias of food frequency-based measures of fruit and vegetable intakes, Kristal and colleagues compared intake estimates between recalls (or food records) ν. a brief questionnaire with only seven items or an FFQ with thirty or more items(14). They found that estimated intakes were lower from the brief questionnaire, but higher from the longer questionnaire when compared with recalls or food records.

Despite differences between the recalls and FFQ in the current study, deattenuated correlation coefficients were ≥0·40 for forty-three of fifty-one nutrients examined in both ethnicities combined. Estimates from the FFQ of individual fatty acids, total long-chain fatty acids and fibre were moderately to highly correlated, and the micronutrients (except Na, Cu and dietary vitamin E) were also generally highly correlated with recall data. This shows that the AHS-2 FFQ has the ability to provide relatively good estimates of these nutrients.

Deattenuated correlation coefficients with values less than 0·40 included energy, total protein, total carbohydrate (in blacks), non-fibre carbohydrate (in blacks), Na, dietary vitamin E (in whites) and β-carotene (in blacks). Energy intakes in both races were lower compared with others(4,12,1518). For carbohydrate, our results in blacks and whites were lower or within the range of those observed by other investigators(12,15,17,19).

Our deattenuated validity correlations for total protein were within the range of(4,20) or lower(15,17,21) than those reported in other validation studies. It was particularly noticeable in our study that the individual protein components (animal and vegetable protein) that contribute to this composite nutrient had greatly higher coefficients than total protein. In examining the results from the FFQ and recall data, it appears that the questionnaire over-estimates the amount of vegetable protein intake, but underestimates the amount of animal protein compared with the recalls. The most likely explanation is the differential scaling of animal and plant sources of protein in our FFQ, which includes a long list of plant proteins (in part due to the attention we give to meat analogues) but a comparatively shorter list of animal protein foods. In calculating the deattenuated correlation coefficient for total protein, we noted that the variance of animal protein (APRO) from recalls (315·9) was much greater than the variance of animal protein from the FFQ (211·34), yet the variance of vegetable protein (VPRO) from recalls (165·24) was much smaller than the variance of vegetable protein from the FFQ (325·13).

We resolved this problem by putting the animal and vegetable protein FFQ values on the same scale as each other and the recall values. In a separate analysis, we first calculated for both APRO and VPRO E(recalls|FFQ), summed the two estimates and then correlated this with recall values for total protein. Our correlations for total protein subsequently improved from 0·28 to 0·54 in all subjects combined. This interesting observation suggests that the product-sum method of calculating FFQ intakes may cause distortion at times, depending on the form of the FFQ, and we plan to investigate this issue more fully for other variables (e.g. total energy).

In the current validation study energy-adjusted uncorrected correlations improved after correction for attenuation in the recalls, as others have found. We also observed improved correlations in general when supplements were added, which emphasizes not only the importance of assessing both dietary and supplemental sources of micronutrients, but also that supplements contribute importantly to total nutrient intake in this population.

The λ values reported by Johansson et al.(16) address the same issue as the bias factor (λ – 1) results in the current study. They illustrate quite dramatically the severe proportional biases that result even when validity correlations are well within values commonly held to be acceptable in FFQ work. For example, a univariate regression where uncorrected energy-adjusted PUFA from the FFQ in white subjects (deattenuated validity correlation of 0·72) was used as the independent variable would produce a β coefficient that was biased downwards by 58 %.

When comparing our validation results by race, we found that the deattenuated energy-adjusted validity correlations of several nutrients in our study were smaller in blacks than whites. This is consistent with the reports of others(4,13,18,20,22). Presumably these may be explained by somewhat lesser educational attainment(13,18) on average among blacks or less familiarity with research studies of this sort. In an older biracial sample, investigators also reported lower validity in the very old (aged 79 years and older) compared with those aged 68–78 years(20). In our calibration study, a greater proportion of whites achieved higher education, but they were some-what older compared with blacks.

We provide in this report the validation of an unusually comprehensive list of micronutrients and fatty acids, and with few exceptions find moderate to high validity. Thus we expect to be able to use most of these results to good effect in both traditional analyses of disease risk and analyses incorporating recall- or biomarker-guided measurement error correction.

Acknowledgements

Sources of funding: This work was supported by a grant from the National Cancer Institute RO1 CA94594. Conflict of interest declaration: None of the authors have conflicts of interest. Authors’ contributions: study design, G.E.F.; coordination of data collection, K.J.-S., S.F.K., J.S., W.L.B., J.C., R.P.H., T.L.B., E.H., H.B., S.M.; data analyses, K.O., G.E.F., K.J.-S.; interpretation of data, K.J.-S., S.F.K., J.S., W.L.B.; writing of the manuscript, K.J.-S., G.E.F., S.S.S.; editing of the manuscript, S.F.K., J.S., W.L.B., J.C., R.P.H.

References

1. Fraser GE. A search for truth in dietary epidemiology. Am J Clin Nutr. 2003;78(3 Suppl.):521S–525S. [PubMed]
2. Thomas D, Stram D, Dwyer J. Exposure measurement error: influence on exposure–disease. Relationships and methods of correction. Annu Rev Public Health. 1993;14:69–93. [PubMed]
3. Willett W. Correction for the effects of measurement error. In: Willett W, editor. Nutritional Epidemiology. 2nd ed Oxford University Press; New York: 1998. pp. 302–320.
4. Stram DO, Hankin JH, Wilkens LR, et al. Calibration of the dietary questionnaire for a multiethnic cohort in Hawaii and Los Angeles. Am J Epidemiol. 2000;151:358–370. [PubMed]
5. Butler TL, Fraser GE, Beeson WL, et al. Cohort profile: the Adventist Health Study-2 (AHS-2) Int J Epidemiol. 2008;37:260–265. [PubMed]
6. Akbar JA, Jaceldo-Siegl K, Fraser G, et al. The contribution of soul and Caribbean foods to nutrient intake in a sample of blacks of US and Caribbean descent in the Adventist Health Study-2: a pilot study. Ethn Dis. 2007;17:244–249. [PubMed]
7. Schakel SF, Sievert YA, Buzzard IM. Sources of data for developing and maintaining a nutrient database. J Am Diet Assoc. 1988;88:1268–1271. [PubMed]
8. Willett W. Food frequency methods. In: Willett W, editor. Nutritional Epidemiology. 2nd ed Oxford University Press; New York: 1998. pp. 74–100.
9. Willett W, Stampfer M. Implications of total energy intake for epidemiologic analysis. In: Willett W, editor. Nutritional Epidemiology. 2nd ed Oxford University Press; New York: 1998. pp. 273–301.
10. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. Chapman and Hall; New York: 1993. Better bootstrap intervals; pp. 184–188.
11. Slimani N, Bingham S, Runswick S, et al. Group level validation of protein intakes estimated by 24-hour diet recall and dietary questionnaires against 24-hour urinary nitrogen in the European Prospective Investigation into Cancer and Nutrition (EPIC) calibration study. Cancer Epidemiol Biomarkers Prev. 2003;12:784–795. [PubMed]
12. Subar AF, Thompson FE, Kipnis V, et al. Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: the Eating at America’s Table Study. Am J Epidemiol. 2001;154:1089–1099. [PubMed]
13. Kristal AR, Feng Z, Coates RJ, et al. Associations of race/ethnicity, education, and dietary intervention with the validity and reliability of a food frequency questionnaire: the Women’s Health Trial Feasibility Study in Minority Populations. Am J Epidemiol. 1997;146:856–869. [PubMed]
14. Kristal AR, Vizenor NC, Patterson RE, et al. Precision and bias of food frequency-based measures of fruit and vegetable intakes. Cancer Epidemiol Biomarkers Prev. 2000;9:939–944. [PubMed]
15. Jackson M, Walker S, Cade J, et al. Reproducibility and validity of a quantitative food-frequency questionnaire among Jamaicans of African origin. Public Health Nutr. 2001;4:971–980. [PubMed]
16. Johansson I, Hallmans G, Wikman A, et al. Validation and calibration of food-frequency questionnaire measurements in the Northern Sweden Health and Disease cohort. Public Health Nutr. 2002;5:487–496. [PubMed]
17. Kumanyika SK, Mauger D, Mitchell DC, et al. Relative validity of food frequency questionnaire nutrient estimates in the Black Women’s Health Study. Ann Epidemiol. 2003;13:111–118. [PubMed]
18. Mayer-Davis EJ, Vitolins MZ, Carmichael SL, et al. Validity and reproducibility of a food frequency interview in a multi-cultural epidemiology study. Ann Epidemiol. 1999;9:314–324. [PubMed]
19. Jain MG, Rohan TE, Soskolne CL, et al. Calibration of the dietary questionnaire for the Canadian Study of Diet, Lifestyle and Health cohort. Public Health Nutr. 2003;6:79–86. [PubMed]
20. Morris MC, Tangney CC, Bienias JL, et al. Validity and reproducibility of a food frequency questionnaire by cognition in an older biracial sample. Am J Epidemiol. 2003;158:1213–1217. [PubMed]
21. Marks GC, Hughes MC, van der Pols JC. The effect of personal characteristics on the validity of nutrient intake estimates using a food-frequency questionnaire. Public Health Nutr. 2006;9:394–402. [PubMed]
22. Coates RJ, Monteilh CP. Assessments of food- frequency questionnaires in minority populations. Am J Clin Nutr. 1997;65(4 Suppl.):1108S–1115S. [PubMed]