|Home | About | Journals | Submit | Contact Us | Français|
The field of research regarding the effects of habitual caffeine use is immense and frequently utilizes self-report measures of caffeine use. However, various self-report measures have different methodologies, and the accuracy of these different methods has not been compared.
Self-reported caffeine use was estimated from two methods (a retrospective interview of weekly caffeine use and a 7-day prospective diary; n=79). These estimates were then tested against salivary caffeine concentrations in a subset of participants (n=55).
The estimates of caffeine use (mg/day) from the interview- and diary-based methods correlated with one another (r=0.77) and with salivary caffeine concentrations (r=0.61 and 0.68, respectively). However, almost half of the subjects who reported more than 600 mg/day in the interview reported significantly less caffeine use in the diary.
Self-report measures of caffeine use are a valid method of predicting actual caffeine levels. Estimates of high caffeine use levels may need to be corroborated by more than one method.
The field of research regarding the effects of caffeine is immense, both in diversity and volume. The systematic study of caffeine depends on efficient and accurate assessments of dietary caffeine use to study the effects of both habitual and acute caffeine intake. The presence of caffeine in many foods and beverages makes the quantification of daily caffeine use difficult because within every source of caffeine variability exists in the amount of caffeine per volume, the total volume in each serving (i.e., cup size), as well as the frequency of servings within 1 day and across multiple days. The brewing preparation and duration of coffee and tea will affect caffeine content, but even when these conditions are held constant, there may be substantial variations in caffeine concentration between servings; the actual concentration may differ greatly from standardized caffeine contents (Bracken et al. 2002). These factors and others can limit the accuracy of caffeine intake estimates. Despite these challenges, daily caffeine use is routinely estimated with self-report measures. These measures are either retrospective interviews (or pencil-and-paper questionnaires) of past or “typical” caffeine use or prospective diaries in which caffeine use is recorded as it occurs. These two methods of self-reported caffeine use have been shown to agree with one another (Lelo et al. 1986; Kennedy et al. 1991), and both methods have been shown to correlate with caffeine concentrations (James et al. 1988, 1989). However, the accuracy of retrospective reports may be reduced by memory or estimation errors, whereas prospective reports may be inadvertently recorded during a period of atypical caffeine use. Which method is a better predictor of caffeine concentrations is not known.
Another source of variability among investigations of habitual caffeine use is the division of subjects into low, moderate, and high habitual caffeine use groups. Some studies have defined high caffeine use as >100 mg/day (Fine et al. 1994), while others have defined high caffeine use as >750 mg/day (Winston et al. 2005). Inconsistency in these caffeine use boundaries makes interpretation and cross-comparison between studies difficult. This lack of consistency in caffeine use levels has been noted before (Wells 1984), and yet, to our knowledge, no one has proposed standard categories for caffeine use.
The aim of this study was to examine the relationship between self-reported measures of caffeine use and salivary caffeine concentrations during normal caffeine use, in order to identify which self-report method provides the best estimate of caffeine concentrations. With accurate estimates of caffeine use, we can be more rigorous in our categorization of low, moderate, and high caffeine use.
Caffeine and non-caffeine consumers (ages 18–50) were recruited from the Winston-Salem, North Carolina community between 2005 and 2007 as part of a larger study on the effects of habitual caffeine use on cerebral blood flow and functional imaging measures of brain activity. Recruitment was conducted with flyers, newspaper advertisements, and by word of mouth. Recruitment materials asked for volunteers who drink or do not drink coffee for a study on caffeine. Potential volunteers were initially screened for eligibility during a phone interview, and those that reported to be in good health were invited for an in-depth screening visit. Exclusion criteria included current symptoms of an anxiety disorder or depression, and current abuse of alcohol or other illicit drugs. The Institutional Review Board of Wake Forest University School of Medicine approved this study. Participants gave written informed consent before entering this study and were financially compensated for their time.
A retrospective interview of participants' usual caffeine use (Interview) was conducted using a modified version of the caffeine consumption questionnaire (CCQ) (Landrum 1992). The CCQ consists of four time-period columns, for night (2a.m.–6a.m.), morning (6 a.m.–12 noon), afternoon (12 noon–6 p.m.), and evening (6 p.m.–2 a.m.), and separate rows for each different type of caffeine source. Participants were asked to describe the number and volume of caffeinated beverages, foods, medications, and supplements, as well as the brewing method of coffee (e.g., drip brewed versus percolated), across a typical week. Caffeine content of common foods and beverages was taken from the Center for Science in the Public Interest web site (www. cspinet.org; accessed July 1997). This source listed 135 mg of caffeine in 8 oz of drip brewed coffee and 50 mg of caffeine in 8 oz of black tea. Soft drinks vary depending on brand and caffeine content ranges between 35 and 50 mg per 12 oz. Milk chocolate contains 10 mg caffeine per 1.5 oz, and dark chocolate has 31 mg per 1.5 oz. When specific restaurants or brand names of caffeine sources were provided by the participants, caffeine content was estimated from manufacturer's information or other sources of nutritional information available on the Internet.
Participants logged their caffeine use across 1 week with a prospective 7-day caffeine consumption diary (Diary). This diary consisted of seven copies of a modified CCQ (see above). Diaries were collected at the end of the week. Participants were instructed to record caffeine use as it occurred, although exact time of caffeine use was not recorded. At the screening visit, participants were given clear plastic coffee mugs with 4, 6, 8, and 10 oz increments marked along the side, in order to encourage accurate reporting of beverage volume during completion of the diary. Participants were also instructed to record their caffeine use during the study days, prior to the laboratory visit (Same-day Diary).
Participants were instructed to refrain from all food and beverages other than water for 15 min before arriving in the laboratory in order to obtain a clean saliva sample. After rinsing out their mouths with water, a saliva sample was obtained by chewing on a Salivette® cotton swab for 45 s (Sarstedt, Newton, NC, USA). Samples were frozen at −70°C until caffeine concentrations ([CAF]) were assayed using high-performance liquid chromatography (Deroche et al. 1990; Holland et al. 1998; Global Lifescience Solutions, LLC, Ann Arbor, MI, USA). The minimum detection threshold was 0.02 μg/ml. Concentrations below this threshold were recorded as zero. The half-life of 250 mg of caffeine is approximately 6 h, with half-lives up to 11 h among women taking oral contraceptives (Patwardhan et al. 1980). Therefore, saliva likely reflects caffeine intake over the past 30–55 h (5 half-lives).
Participants completed the CCQ as a 1-week caffeine use interview during the initial telephone screen (Interview) then completed a 7-day caffeine use diary (Diary) following the screening visit. Based on the average daily caffeine use calculated from the 7-day diary, a subset of these participants were included in the second part of the study in equal proportions of low (<200 mg/day, n= 18), moderate (200–600 mg/day, n = 19), and high (>600 mg/day, n=18) caffeine users. These assignments into caffeine use groups were part of the design of the larger study. Participants then reported to the laboratory on two occasions (between 6:30 a.m.–1:30 p.m.) during different weeks and were instructed to consume caffeine “as they normally would” prior to the visit and to record their caffeine use in the Same-day Diary. Upon arrival to the laboratory, a saliva sample was collected for a caffeine concentration assay.
The relationships between the self-report measures and [CAF] were compared with Pearson product moment correlations and paired samples t tests. Statistical analyses were conducted with SPSS® version 15.0 (SPSS, Chicago, IL, USA) with alpha set to 0.05. Caffeine use in mg/day was calculated from the Interview and Diary, and this value was converted into mg/kg/day by using participants' weight recorded at the screening visit.
Seventy-nine participants completed the Interview and Diary, and of these, 55 participants completed the two [CAF] study days. Participant characteristics are shown in Table 1. The estimates of daily caffeine use (mg/day) from the Interview and the Diary were correlated (r=0.77, p< 0.001; see Fig. 1a). However, the average caffeine use from the Interview (657±555 mg/day) was greater than the average caffeine use from the Diary (487±393 mg/day). The difference between these two measures was significant (t(78)=4.21, p<0.001), and this difference was driven by the high users. When classified as low, moderate, and high caffeine users based on the Interview estimates, the low (n= 22) and the moderate users (n=15) reported almost identical caffeine use averages in the Diary. Only 6 (16%) of these subjects were classified differently by the Diary results. In contrast, the high users (n=42) reported significantly higher levels of caffeine use in the Interview than in the Diary (1,037±486 versus 725±341 mg/day; t(41)=4.96, p<0.001), and 19 (45%) of these subjects were classified differently by the Diary.
The results of the Same-day Diaries (mg/day) from the two study days were highly correlated (r=0.94, p<0.001) and were not significantly different. Similarly, the [CAF] (μg/ml) from the two study days were also correlated (r= 0.83, p<0.001) and were not significantly different; the mean [CAF] was 2.0±2.4 μg/ml. The average of these two Same-day Diaries and the two [CAF] were used in the analyses. As a post hoc analysis, we compared this average [CAF] with the [CAF] of a saliva sample collected following 90 min of abstention from caffeinated beverages on one of the study days. These measures were strongly correlated (r=0.91, p<0.001) and not significantly different. This indicates that the estimates of caffeine concentration from the earlier saliva samples are not artificially high due to recent consumption of caffeinated beverages.
The average [CAF] correlated with self-reported caffeine use from the Interview, Diary, and the average Same-day Diary in mg/day (r=0.61, 0.68, and 0.83, respectively) and in mg/kg/day (r=0.67, 0.72, 0.86, respectively; see Fig. 1b–d). All of the correlations were significant (p<0.001).
The [CAF] from the saliva sample obtained 90 min post-caffeine produced similar correlations with the Interview, Diary, and the Same-day diary. In addition, removing participants (n=18) who reported cigarette or oral birth control use from the data set did not affect the significance of the correlations.
We conducted a literature search for articles that used mg/day to quantitatively define caffeine use and divided their subjects into low/light, moderate/habitual, or high/heavy caffeine use groups. The search was conducted in PubMed, Web of Science®, and Google™ Scholar. The terms used in the search were “low, moderate, or high” in conjunction with “caffeine use, habitual caffeine use, caffeine intake, or caffeine consumption.” Thirty-three articles were found that met our search criteria, and these articles were published between 1980 and 2007; see Table 2. The median caffeine use is described as ≤100 mg/day for low users, between 170 and 400 mg/day for moderate users, and ≥300 mg/day for high users.
The data demonstrate that while the estimates of caffeine use reported in the Interview and Diary were well correlated (r=0.77), the high users reported greater caffeine use in the Interview than in the Diary. Of the subjects initially identified as high users by the Interview, 45% were classified differently by the Diary. The average daily caffeine use from both the Interview and Diary correlated well with actual salivary caffeine concentrations ([CAF]). These results suggest that both self-report methods are good predictors of actual caffeine concentrations, but if daily caffeine use is used to classify subjects into caffeine use groups, high users may need to be verified by more than one measure.
Previously, daily caffeine use estimated from a retrospective questionnaire correlated (r=0.31) with salivary [CAF] obtained after 5 p.m. (James et al. 1989). Similarly, Lelo et al. (1986) reported that the average caffeine use estimated from a questionnaire and 1-day diary produced similar results, and the calculated caffeine dose correlated with plasma [CAF] obtained during the diary day (r=0.64). In comparison, a correlation between self-reported number of drinks per day and hemoglobin–acetaldehyde was reported as r=0.30 (Hazelett et al. 1998), and a correlation between self-reported number of cigarettes per day and salivary cotinine levels was reported as r=0.60 (Etter et al. 2000). Direct measures of drug concentrations are generally considered more reliable than self-reports, since many drugs in question are illicit, and individuals may feel social or legal pressure to conceal their use. However, caffeine concentrations only reflect use over the past 1 or 2 days; self-reports are still necessary to gather a complete drug use history. Our study and others have shown that self-reported caffeine use is consistent across measures (Lelo et al. 1986; Kennedy et al. 1991; Rapoport et al. 1984) and does predict caffeine concentrations (James et al. 1988, 1989).
There are several reasons for discrepancies between self-reported caffeine use and salivary concentrations. Notably, the estimation of caffeine use may be inaccurate due to standardized caffeine content information, since the actual caffeine content of common beverages can vary considerably (Bracken et al. 2002) and may be due to poor approximation of the volume of the consumed beverage and failure to account for dilution by milk or melted ice. The rate of caffeine metabolism also significantly contributes to this discrepancy. A previous study did not find a relationship between self-reported caffeine use from a questionnaire and [CAF] in plasma drawn following overnight caffeine abstinence (Kennedy et al. 1991) probably due to the short half-life of caffeine. The half-life of caffeine varies between individuals and is also affected by the use of cigarettes and hormonal birth control (Patwardhan et al. 1980; Benowitz et al. 2003). James et al. (1989) reported that at 5 p.m., self-reported caffeine use correlated more strongly with the caffeine metabolite, paraxanthine, than with caffeine itself. In the present study, saliva samples were obtained during the first half of the day, possibly before participants consumed their entire caffeine intake. However, the advantage of early sampling is that caffeine has not yet metabolized, and we report a stronger correlation between the Interview and [CAF] (r=0.61) than the previous study (r=0.31; James et al. 1989).
There are potential sources of error in self-reported drug use. The reporting of some drugs may be biased by social desirability concerns (Johnson and Fendrich 2005), but this is not likely to occur with caffeine since it is an accepted part of American culture. However, self-reports rely on subjective recollection, which can be a source of bias (Johnson and Fendrich 2005). In this study, the high users reported more caffeine use in the Interview than in the Diary. Given the large number and frequent use of caffeinated beverages, it is likely that they overestimated their caffeine consumption in the Interview due to memory error. It is also possible that the Diary was recorded during an atypical week or high users have more variable caffeine use from week to week. Another explanation for the difference between the Interview and Diary is that the monitoring of one's own behavior may alter the observed behavior, but this is more likely to occur if there is a positive or negative value associated with the behavior (Kazdin 1974). For instance, cigarette use has been shown to increase during self-monitoring (McFall 1970), but the self-monitored number of cigarettes used per week did not vary among subjects unmotivated to quit smoking (Lipinski et al. 1975). We did not enquire about participants' motivation to reduce their caffeine intake, but it is unlikely that the diary was biased by a general reactivity to self-monitoring since there was no difference between the Interview and Diary results for the low and moderate users. In future studies, we could determine whether self-monitoring affected caffeine use by obtaining a salivary [CAF] sample prior to the diary to compare with salivary [CAF] samples obtained during the week of the diary.
Of the articles reviewed here that provided information on how subjects were classified into caffeine use groups, 28% used a diary method, while 72% used a questionnaire or interview method. Of these, four studies used both a questionnaire and a diary, and no differences were reported between the two measures (Childs and de Wit 2006; Lane et al. 1990; Rogers et al. 2003; White et al. 1980). The more often used, interview or questionnaire method is undoubtedly faster, and a structured interview may be more systematic than a diary. However, these methods rely on participants' memory or their own perception of average caffeine use. We elected to base our estimates of average daily caffeine use on the 7-day Diary; most importantly because the accuracy of the self-reported diary is less prone to errors of recall or by overgeneralization of normal caffeine use. Seven days were monitored because it is likely that caffeine use is greater during the work week, when individuals feel more pressure to have enhanced concentration and cognitive performance, than during the weekend when there may be fewer cognitive demands. In addition, the average intake over a week is less likely to be biased by an abnormal amount of consumption from a single day than a 1-day Diary.
Many studies categorize participants based on the participants' level of habitual caffeine use. Hypothetically, the utility of a low/light caffeine use group is to identify individuals that are unlikely to experience outcomes associated with habitual caffeine use. Moderate/habitual and high/heavy caffeine users, then, are habitual consumers with a low and high likelihood of outcomes, respectively. While this is a useful strategy for statistical comparison, a consistent definition of these categories is warranted. There are several ways to define these groups; one way is to base definitions on perceived health risks. A comprehensive review on the effects of caffeine on health recommended that up to 400 mg/day is not associated with adverse health effects (Nawrot et al. 2003). For Americans of average adult weight (86 kg for men and 74 kg for women; Ogden et al. 2004), 400 mg/day would be 4.7–5.4 mg/kg/day. A second method would be to define caffeine use groups based on surveys of actual caffeine consumption. According to two recent national surveys, the average caffeine use is 106–170 mg/day, with 227–382 mg/day in the 90th percentile (1.5–2.3 and 3.2–5.2 mg/kg, respectively) for adults aged 20–49 (Knight et al. 2004), and between 166 and 336 mg/day (1.1–1.8 mg/kg) for adults aged 18–54 (Frary et al. 2005). Hughes and Oliveto (1997) reported an average of 222±190 mg/day among a sample of adult Vermont residents, and from this study, we can estimate caffeine use at the 33rd and 66th percentiles to be 138 and 306 mg/day. Thirdly, caffeine use groups could be based on previously published studies, as shown in Table 2. Although unrelated, there is a fair amount of agreement between these three methods. Overall, low caffeine users consume up to 120 mg/day or 1.5 mg/kg/day, moderate users between 120 and 400 mg/day or 1.5–5.0 mg/kg/day, and High users above 400 mg/day or 5.0 mg/kg/day. Reporting caffeine use in mg/kg/day would also improve the consistency in these categories across studies. A recent survey suggests that the average adult weight is around 80 kg (Ogden et al. 2004), instead of the conventional estimate of 70 kg. This change is most likely accompanied by increased body weight variability within and between study samples, possibly making caffeine use in mg/day disproportionate between individuals and studies.
A limitation of our study is the measurement of caffeine from saliva, instead of blood. We elected to use saliva samples to minimize participants' discomfort. Saliva is an adequate measure of caffeine concentrations, the correlation between salivary and serum caffeine concentrations has been reported to be 0.989, and salivary concentrations are approximately 70% of serum concentrations (Biederbick et al. 1997). Biederbick et al. (1997) reported higher caffeine concentrations in saliva than in serum in the first 90 min after caffeine intake from a liquid solution, probably due to gingival contamination. We only requested that the participants abstain from normal caffeine use for 15 min before the saliva sample, so our estimates of [CAF] could have been artificially high. Because of this possibility, we analyzed data from a second saliva sample that was obtained during a single study day following 90 min of caffeine abstinence and found no difference between this and the earlier [CAF]. Another limitation is that we did not record the actual amount of time that passed between the last use of caffeine and the saliva sample. A drawback of the caffeine diary is the 6-h time-bins. If subjects had recorded the exact time of caffeine ingestion instead, we could have included that information to reduce variability in the analyses. Furthermore, we only assayed the saliva sample for caffeine, not for other xanthine metabolites (e.g., James et al. 1988 found a stronger correlation between reported caffeine use and paraxanthine). Lastly, the average caffeine use within our sample is much greater than the national average. A more representative sample may produce smaller correlations with [CAF].
We would like to thank the General Clinical Research Center and Debra Hege for their assistance with the collection of data.
Supported by grants from the National Institutes of Health (R01 EB03880, NS42568, RR07122, and F31DA024950).