|Home | About | Journals | Submit | Contact Us | Français|
To assess the accuracy of three self-administered food frequency questionnaires (FFQs) to measure dietary calcium intake in healthy adults.
Estimates of dietary calcium intake from one previously validated and two recently developed FFQs were compared with those from 7-day food records.
Healthy adults enrolled in an outpatient study of calcium supplementation completed the 36-page Dietary History Questionnaire (DHQ), a 3-page Calcium Questionnaire, and a 1-page Short Calcium Questionnaire. Subjects then completed a 7-day food record.
Differences between calcium intake reported on FFQs and calcium intake from food records were compared.
Spearman correlations were used to measure associations among variables; Bland-Altman pairwise comparisons were conducted to assess systematic and magnitude biases.
We studied 341 subjects, 74.5% female, mean (±standard deviation) age of 38±11 years and body mass index (calculated as kg/m2) of 31.8±7.1. Mean (±standard deviation) food record calcium intake was 896±380 mg/day; data from all three FFQs were positively related to food record calcium intake, but accounted for <40% of the variance in food record dietary calcium intake (DHQ: r2=0.21; Calcium Questionnaire: r2=0.33; Short Calcium Questionnaire: r2=0.37; all P<0.001). The DHQ underestimated daily calcium intake (systematic bias: −94 mg/day, P<0.001; magnitude bias r=−0.40; P<0.001), whereas the Calcium Questionnaire overestimated calcium intake (systematic bias +177 mg/day, P<0.001), but had no significant magnitude bias (r=−0.09; P=0.11). The Short Calcium Questionnaire showed minimal systematic bias (+34 mg/day, P=0.09), but had magnitude bias (r=−0.33; P<0.001).
All three FFQs performed reasonably well at estimating dietary calcium intake compared to food records; each may be appropriate for use in select clinical and research settings.
The potential role of calcium intake in the pathogenesis and treatment of osteoporosis (1,2), hypertension (3,4), obesity (5,6), and other diseases (7,8) is of growing interest. Accurate assessment of calcium intake is fundamental for advancing research in these areas. Valid measurements of retrospective calcium intake, as well as the ability to track calcium intake longitudinally, are crucial components for elucidating the role of calcium in health and disease. The methods that can be employed to measure dietary calcium intake will depend upon whether calcium intake is the only nutrient to be studied, or whether calcium intake will be evaluated together with assessments of energy and/or other nutrient consumption.
In clinical settings, practitioners can use estimates of usual calcium intake to determine appropriate treatment for individuals. These recommendations may involve providing dietary guidance and advice on calcium supplementation. Methods are therefore needed to estimate calcium intake of American adults accurately and efficiently, using approaches that are not overly burdensome for respondents, researchers, or clinicians.
Food frequency questionnaires (FFQs) have historically been the method of choice for collection of dietary intake data for epidemiologic studies (9); more recently, their use in other research and clinical settings has become more commonplace. FFQs have often been used to measure usual calcium intake. FFQs are usually designed to be completed independently by subjects and are considered to be a relatively inexpensive assessment technique (10).
Although numerous FFQs for assessing calcium intake have been developed and validated during the past two decades, they may not be appropriate for estimating calcium intake of adults in the United States today, for a number of reasons. FFQs developed during the 20th century may be outdated, because dietary intake patterns change over time (11–15). In addition, older FFQs might not accurately reflect the intake of the ever-increasing number of calcium-fortified foods in the marketplace (7). FFQs that were designed for particular populations (eg, women of specific age ranges) might not be valid for use with other populations (16–18). FFQs developed in other countries might not reflect food eaten by adults in the United States (19–25).
FFQs designed for assessing total diet, such as those of Block and colleagues (26) and Willett and colleagues (27), although well-validated, might not be appropriate for use in all circumstances. Often these FFQs require optical scanning, making them impractical when estimates of intake are needed promptly for study enrollment or treatment decisions. In addition, total diet FFQs can be unnecessarily lengthy when used for estimating only one nutrient, and they might not be as valid for assessing intake of a single nutrient as a trade-off for estimating intake of numerous nutrients in epidemiological studies (9).
In this investigation, we examined the ability of three FFQs of varying length and format to measure dietary calcium intake in healthy adults: a 36-page Dietary History Questionnaire (DHQ), a 3-page Calcium Questionnaire, and a 1-page Short Calcium Questionnaire. We hypothesized that dietary calcium as assessed by each of the three methods would be positively associated with dietary calcium intake as measured by a 7-day food record.
This validation study was conducted as part of a baseline assessment for a randomized clinical trial examining the effects of calcium supplementation on body weight, body composition, and comorbid conditions (28). We recruited both healthy overweight adults, with body mass index (BMI; calculated as kg/m2) ≥25, and healthy normal-weight adult control subjects (BMI range of 18 to 24.9). These studies were carried out in accordance with the Declaration of Helsinki, and were approved by the National Institute of Child Health and Human Development Institutional Review Board. Each participant gave written consent for protocol participation and received compensation.
Based on the requirements of the treatment protocol, eligibility criteria for overweight subjects included age older than 18 years; good general health; no regular use of most prescription or any anorexiant medications; stable weight within 3% over the past 2 months; evidence of normal calcium metabolism, based on relevant serum and urine laboratory tests; no history of renal stones; and, for women, a negative pregnancy test. A history of fractures was not part of the exclusionary criteria. Usual total calcium intake from diet and supplements could not exceed 3.5 g/day as estimated by a 7-day food record augmented by self-reported intake of supplements during the past year. There were also specific limits on supplemental calcium intake (intake required to be ≤300 mg/day) and supplemental vitamin D intake (intake required to be ≤400 IU/day). Eligibility criteria for normal-weight subjects were similar, although total calcium intake was not limited because these subjects would not be participating in the randomized trial of the larger study. An effort was made to recruit subjects in proportion to their ethnic, racial, and socioeconomic representation in the population of the state of Maryland. English-language proficiency was required for this study in order to provide consent and complete all required questionnaires.
The DHQ, developed by the National Cancer Institute for use in epidemiological research to assess total diet, is a validated, 36-page booklet listing 124 separate food items (10,29). Respondents are asked to select a frequency category indicating how often each food item was consumed over the past 12 months, and then choose the portion size from three ranges. The list of foods and portion size ranges were developed from analyses of the US Department of Agriculture’s 1994–1996 Continuing Survey of Food Intakes by Individuals (30). Other questions address seasonal intake, food type (low fat, diet, lean), use of added fats, etc. Questions about dietary supplement use are also included. Data are generally extracted for analysis by use of optical scanning. The DHQ is available at http://riskfactor.cancer.gov/DHQ/ (10).
The Calcium Questionnaire is a 3-page, 87-item FFQ developed by clinical research registered dietitians (RDs) at the Hatfield Clinical Center, National Institutes of Health. It was designed primarily for use with subjects enrolled in observational studies or Phase 1 and 2 clinical trials to assess usual calcium intake for both research and clinical purposes. The format is modeled after the 1987 version of the Health Habits and History Questionnaire (31). Subjects are asked to write in the number of times each food item is eaten by day, week, or month during the past year. Subjects are also asked to indicate whether their portion was small, medium, or large, based on a reference medium portion size listed for each item. Two pages of food model and utensil drawings (32) are provided to aid in portion-size determination. Also included are open-ended sections for writing in product names of any fortified nutritional supplement or meal-replacement drinks and bars consumed, additional calcium-containing foods eaten at least weekly, and calcium-containing supplements and antacids. The food list was developed to reflect foods found to be major contributors of calcium based on the second National Health and Nutrition Examination Survey (NHANES) (33) and later, the 1994–1996 Continuing Survey of Food Intakes by Individuals (34). The Calcium Questionnaire was updated in 2002 (prior to the beginning of this study) to reflect new food products, especially those fortified with calcium. In addition to enabling calculation of daily calcium intake from diet and from dietary supplements, energy intake is estimated by the Calcium Questionnaire as a method of validating subject recording. The Calcium Questionnaire is analyzed using standard spreadsheet software (available upon request from the authors).
The Short Calcium Questionnaire was designed by one of the authors (N.G.S.) to assess usual calcium intake rapidly, either to ascertain eligibility for clinical research studies, or to determine appropriate dosage of supplemental calcium to be prescribed. The Short Calcium Questionnaire is a 1-page, 25-item FFQ developed from the Calcium Questionnaire, reflecting major dietary sources of calcium. For each food item listed, subjects write in the number of servings eaten in a typical week. A reference serving size is included. Foods are grouped by calcium content per reference serving (1,000 mg, 400 mg, 300 mg, 200 mg, and 100 mg). A section on calcium-containing dietary supplements and antacids consumed in the past month is also included. Although the Short Calcium Questionnaire was hand-calculated for this study, it can also be analyzed electronically (available upon request from the authors).
Participants were recruited through flyers and by local newspaper and radio advertisements in the Washington, DC, metropolitan area. Following preliminary screening by telephone, volunteers were mailed a copy of the consent form along with the DHQ and instructed to complete the DHQ at home; these were collected from subjects upon arrival at their initial clinic screening appointment. At this appointment, height was measured to the nearest 0.1 cm using a calibrated stadiometer (Holtain Ltd, Crymych, UK); weight was obtained to the nearest 0.1 kg, with subjects wearing hospital gowns using a calibrated platform digital scale (SR Instruments, Tonawanda, NY). The Hollingshead two-factor scale for social status was administered to assess socioeconomic status (SES) based on self-reported occupation and income level (35); race/ethnicity was self-reported. Medical history, physical examination, fasting blood work, and urine tests were performed. Subjects were then given the Short Calcium Questionnaire and Calcium Questionnaire to complete after they received brief verbal directions provided by research nurses or study coordinators; subjects also were encouraged to read printed instructions on each FFQ. Based on the design of the larger study, participants were asked to fill out the Short Calcium Questionnaire first, followed by the Calcium Questionnaire.
At the end of the initial clinic visit, research nurses provided subjects with both verbal and written instructions for completing a 7-day food record within the next 2 weeks. Subjects were provided with directions for recording all foods and beverages (except water) consumed daily for 7 consecutive days. Instructions for recording both amounts consumed (using number of items, household measures, package weights, or dimensions) and food descriptions, as well as a sample of a completed food record, were provided as a 4-page handout. Subjects were advised to eat as usual and record their intake throughout the day.
Subjects who met criteria for the primary study returned to the clinic approximately 2 weeks later. They met with a research RD who reviewed their DHQ, Short Calcium Questionnaire, and Calcium Questionnaire to confirm comprehension and completeness. In addition, for the Calcium Questionnaire, the research RD clarified any entries written in the open-ended sections and verified any unusual intake frequencies. The research RD then reviewed the food record carefully with subjects for completeness and accuracy of recording, using three-dimensional food models and household measuring utensils to verify portions as needed.
The original target sample size (340 overweight and obese subjects plus 60 nonoverweight subjects) was based on the goals of the larger study (28). For the present study, with an α of .05, there was 100% power to detect a very large-sized relationship (defined as r2≥0.85) between calcium intake from food records and calcium intake by the FFQs, taking into account up to five other covariates such as sex, age, BMI, race/ethnicity, and SES (35).
For inclusion in the present study, subjects were required to have submitted an adequate 7-day food record. By design, subjects were excluded from analysis if their food record: was not kept or was missing, was kept for fewer than 7 days, was illegible, was judged by the research RD to be invalid because of poor recording, or suggested biologically implausible daily energy intakes of <800 or >4,200 kcal for men or <600 or >3,500 kcal for women (10). In addition, subjects were excluded from this analysis if any of the three FFQs were missing or incomplete.
Food records were coded using the Nutrition Data System for Research software versions 4.04 and 4.05 developed by the Nutrition Coordinating Center, University of Minnesota, Minneapolis (36). Efforts were made to code food items to reflect their actual calcium content, using product information from food labels and company Web sites when necessary. Mean daily calcium, macronutrient, and energy intakes were determined.
DHQs were analyzed using Diet*Calc Analysis Program (version 1.3.2. National Cancer Institute, Applied Research Program, June 2003). Calcium Questionnaire dietary calcium contents were entered into a spreadsheet to determine calcium intake. Short Calcium Questionnaires were hand-calculated; daily calcium intakes were determined from weekly intake estimates, adding an adjustment factor of 200 mg/day to reflect the contribution of calcium from foods eaten but not listed on the questionnaire. This adjustment was chosen before initiation of this study. Although all three FFQs allowed for estimation of calcium intake from calcium supplements, calcium-containing multivitamin/mineral supplements, and antacids, these data were not included in this validation study because intake of supplements was not recorded on food records. The analyses performed, therefore, reflect only dietary calcium intake.
Statistical analyses were conducted using StatView (version 5.01, SAS Institute Inc, Cary, NC) software. Methods used to assess agreement were simple regression; Kendall rank correlations; Bland-Altman pairwise comparisons (37); multiple regression analyses with sex, age, race, BMI, season of visit, and SES as independent factors (with no interaction terms); and Student t tests. All tests were two-tailed. The 95% confidence intervals for the Bland-Altman limits of agreement were also calculated. Statistical significance was set at P=0.05. Mean±standard deviation are reported unless otherwise noted.
Preliminary telephone screening of 1,843 potential subjects was performed. Of those meeting preliminary criteria by self-report and expressing interest in enrollment, 485 subjects underwent additional screening at the initial clinic visit, with 398 subjects eventually qualifying for enrollment in the larger study. Of these, 57 subjects were excluded from the dietary validation analysis: 54 subjects were excluded because their food records were not complete or were determined to be inaccurate (see Methods section), and three subjects were excluded because one or more of the three FFQs were not complete. Characteristics of the 341 subjects who are included in this report are shown in the Table. Subjects who were included did not differ by sex, age, BMI, race/ethnicity, or SES from subjects who were excluded for not returning complete data (data not shown). Dietary data collection was equally distributed across all seasons (data not shown).
Mean daily calcium intake from food records was 896±380 mg/day. Mean calcium intake differed significantly from mean daily calcium intake by the DHQ: 803±450 mg/day (P<0.001); and Calcium Questionnaire: 1,073±560 mg/day (P<0.001); but not from the Short Calcium Questionnaire: 930±428 mg/day. Mean daily energy intakes were significantly greater by food records (2,173±567 kcal/day) compared to DHQ (1,971±1015 kcal/day, P<0.001) or Calcium Questionnaire (1,669±709 kcal/day, P<0.001); the format of the Short Calcium Questionnaire did not allow estimation of daily energy intake.
Correlations based on linear regression between dietary calcium intake by food records and calcium intake estimated by each FFQ were significantly related (Figure 1, P<0.001). Correlation coefficients were lowest for the DHQ (r2=0.21), moderate for the Calcium Questionnaire (r2=0.33), and highest for the Short Calcium Questionnaire (r2=0.37), although none of the FFQs accounted for >40% of the variance in food record dietary calcium intake. Kendall rank correlations between dietary calcium intake by food records and calcium intake by each FFQ were also significantly related (P<0.001) with τ values similar to those found using linear regression (DHQ, 0.229; Calcium Questionnaire, 0.383; Short Calcium Questionnaire, 0.373).
The Bland-Altman approach (Figure 2) plots the difference between two methods (calcium intake from of each of the FFQs minus calcium intake from food records) against the mean value for the criterion method (calcium intake by food records). Significant systematic bias for two of the FFQs was revealed, indicating that the DHQ underestimated calcium intake by −94 mg/day (Figure 2A; P<0.001), whereas the Calcium Questionnaire overestimated calcium intake by +177 mg/day (Figure 2B; P<0.001); the Short Calcium Questionnaire showed insignificant systematic bias of +34 mg/day (Figure 2C; P=0.09).
For Bland-Altman comparisons, equations demonstrating significant correlation (or slope) between the difference and the mean calcium intake from food records are described as showing magnitude bias; such correlations indicate that the error of estimation changes when a larger (or smaller) calcium intake is being estimated. Magnitude bias illustrated by negative slopes were significant for the DHQ (Figure 2A; r=−0.40; P<0.001) and for the Short Calcium Questionnaire (Figure 2C; r=−0.33; P<0.001). The Calcium Questionnaire showed no significant magnitude bias (Figure 2B; r=−0.09; P=0.11).
After controlling for age, sex, BMI, race/ethnicity, SES, and season for data collection, multiple linear regression models found that dietary calcium from all three FFQs remained significantly correlated with calcium intake from food records (all P<0.001).
All three FFQs performed reasonably well, although not perfectly, to estimate dietary calcium intake obtained from 7-day food records. Mean calcium intakes from food records and all three FFQs were similar to mean calcium intake from food (892 mg/day) reported for a national sample (NHANES 2001–2002), based on two 24-hour recalls (38). Although the DHQ showed some systematic bias resulting in underestimation of calcium intake, it was relatively small. The DHQ also had a greater magnitude bias than the Calcium Questionnaire or the Short Calcium Questionnaire, suggesting the DHQ may further underestimate calcium intake for those individuals with the highest calcium intakes. One explanation for the magnitude bias exhibited by the DHQ may be that some respondents may be consuming larger food portions of high-calcium foods than those expected by the DHQ database.
Differences among the nutrient databases used for the three FFQs and for the food record analyses may have accounted, in part, for the apparent underestimation of calcium intake by the DHQ. For example, based on package label information, Total brand cereals (General Mills, Inc, Minneapolis, MN) contain 1,000 mg of calcium in a ¾ to 1 cup (30 to 55 g) serving; yet in the DHQ database, a serving of <1 cup of Total and other fortified cereals is assigned a calcium value of only 104 mg for women and 140 mg for men (http://riskfactor.cancer.gov/DHQ/database/current.html). The Calcium Questionnaire and Short Calcium Questionnaire databases, as well as the Nutrition Data System for Research database (used to analyze food records), used the food manufacturer’s figures. The absence of some calcium-fortified food items on the DHQ might also have led to underestimations of calcium intake. For example, calcium-fortified orange juice, which contains >300 mg calcium per cup is not differentiated from nonfortified orange juice on the DHQ. Similarly, no option for calcium-fortified soy milk is available in the DHQ database.
It is understandable then that the DHQ, as a total diet FFQ, may not be as sensitive for measuring calcium intake as a trade-off for measuring a larger number of nutrients in the diet. When intake data for several nutrients is needed in the research setting, use of a total diet FFQ is a logical choice, and estimations of calcium intake from the DHQ may be considered acceptable in this context. The DHQ was developed more than 8 years ago; periodic updating of the DHQ database will be necessary to maintain the integrity of this instrument.
The Calcium Questionnaire demonstrated the largest systematic bias, yet it showed no significant magnitude bias. As FFQs are frequently used to rank order subjects by nutrient intake, the Calcium Questionnaire would appear to perform well in this capacity. The tendency of the Calcium Questionnaire to overestimate calcium intake may be due, in part, to the design of this FFQ; the large number of calcium-containing foods listed and the open-ended format for frequency responses may lead respondents to overestimate their consumption. The absence of magnitude bias for the Calcium Questionnaire suggests that the portion-size categories listed are appropriate. In addition, the ability to enter specific calcium content information for nutritional supplements and meal replacements lessens this potential source of magnitude bias. Because the Calcium Questionnaire is not meant to be a total diet FFQ, it is not surprising that estimated mean daily energy intake was only 77% of that estimated by food records. Researchers and practitioners can use the energy intake estimates generated by the Calcium Questionnaire analysis to gauge validity using cutpoints similar to those used in this study; this is not possible with the Short Calcium Questionnaire.
The Short Calcium Questionnaire showed an insignificant systematic bias, suggesting that use of an adjustment factor of +200 mg/day to reflect the contribution of calcium from foods eaten but not listed on the questionnaire was appropriate. The magnitude bias evidenced by the Short Calcium Questionnaire may be a result of the fact that portion-size selection is not an option on this FFQ; the default portions assigned to food items may underestimate portions consumed for those respondents consuming larger portions. The Short Calcium Questionnaire would appear to be suitable for use as a screening tool to identify those at risk for low calcium intakes in order to make treatment recommendations.
When selecting an appropriate tool for dietary assessment, practical issues related to the administration of FFQs should also be examined. When respondent burden is a concern, the length of time needed for an individual to complete each FFQ should be considered: about 1 hour for the DHQ (10), 15 to 30 minutes for the Calcium Questionnaire, and <10 minutes for the Short Calcium Questionnaire. Analysis times and required resources also vary among the three FFQs. The DHQ requires scanning and/or on-line analysis, the Calcium Questionnaire can be coded in 10 to 15 minutes, and the Short Calcium Questionnaire can be analyzed in <5 minutes. When assessment of only calcium intake is indicated, the shorter Calcium Questionnaire or Short Calcium Questionnaire would thus appear to be better choices.
Possible limitations of using food records as the criterion method should be noted. The act of prospectively recording food intake may influence subjects’ normal consumption resulting in distortion of usual intake, as well as lead to reporting bias; these effects may be more pronounced with obese subjects (39). Of note, our study enrolled proportionally more overweight subjects (BMI >24.9; 86.5%) compared to a national average of 66.3% overweight adults (40). Yet, the food records in this study appeared to be of high quality, as judged by the research RDs, and as evidenced by energy and calcium intake estimates that were similar to national data (38).
Strengths of this validation study include a large, relatively diverse (although geographically limited) sample of adults, with data collection spanning all seasons. Our findings might not be applicable to elderly adults, to those who are not proficient in the English language, or to those with low literacy levels. Weaknesses of this study are primarily related to design and logistical issues. Although it would have been preferable for subjects to have kept food records periodically throughout the year preceding administration of the FFQs (rather than postadministration of the FFQs, as was done in this study), this was not possible because of the design of the primary study. Similarly, logistics of the primary study dictated that the Short Calcium Questionnaire and the Calcium Questionnaire be completed in the same order on the same day; completing these questionnaires 1 or 2 weeks apart in random order would have been preferable. Verbal directions were provided for the Short Calcium Questionnaire and Calcium Questionnaire, which were completed in the clinic; verbal instructions were not provided for the DHQ, which was filled out at home, where other household members may have aided in completing this FFQ. It should also be noted that differences in the time periods reflected by the three FFQs varied somewhat. Directions printed on the DHQ and Calcium Questionnaire were to report intake “during the past year”; instructions printed on the Short Calcium Questionnaire were to indicate servings consumed “during a typical week.” Because the design excluded subjects with calcium intakes that exceeded 3,500 mg/day, the results of the study may not apply to individuals with very high calcium intakes.
Lastly, subjects’ knowledge that they were participating in a study addressing calcium intake and were weighed during their clinic visits may have affected their reports of intake and possibly improved their recording of calcium-containing foods.
In summary, the DHQ, Calcium Questionnaire, and Short Calcium Questionnaire all reflect calcium intake estimated from food records, but none are perfect tools for measuring calcium intake in adults. The DHQ may be a good choice when the intake of several nutrients is being assessed. The Calcium Questionnaire would seem to be a good choice for assessing usual calcium intake in a research setting where relative intake is of primary interest. The Short Calcium Questionnaire appears to work well as a simple-to-use tool for quickly estimating calcium intake for treatment decisions or for screening purposes. This study identified limitations for all three FFQs, many of which can be addressed through revisions in food-list updates, frequency-category design, and portion-size assignments.
This research was supported by the Intramural Research Program of the National Institutes of Health, grant HD-000641 from the National Institute of Child Health and Human Development, National Institutes of Health (J.A.Y.) and Y2-OD-2067 (J.A.Y.) from the Office of Dietary Supplements, National Institutes of Health, Department of Health and Human Services.
Preliminary results were presented at the Endocrine Society Meeting, Philadelphia, PA, June, 2003.
N. Sebring, B. Denkinger, and J. Yanovski are Commissioned Officers in the US Public Health Service, Department of Health and Human Services.