Overall, anonymous online reports of marijuana use and related cognitions among young adults demonstrated adequate reliability and validity with a few exceptions. We found strong relationships among items that assessed the prevalence and frequency of marijuana use, with the exception that African-American young adults showed lower inter-item agreement for prevalence of past 30-day marijuana use than other groups. African-American participants may have underreported marijuana use in the screening item at the beginning of the survey because they were concerned about reporting illegal behavior online. A history of misrepresentation in medical research (Harrison, 2001
), coupled with disproportionately high incarceration rates among African-American men (e.g., Pettit & Western, 2004
), could lead to mistrust of the substance abuse research setting, even in an anonymous environment. The ethnic differences in reliability of marijuana use found here suggest that anonymous online reports of substance use should include at least two items to accurately evaluate the validity of data, and be embedded in the middle or end of a survey rather than the beginning..
Internal consistency reliability for measures of marijuana cognitions was generally strong. However, marijuana dependence as measured by the CUDIT was less reliable than other measures. This contrasts with previous reports of reliability reports for the CUDIT and CUDIT-R that were each .91 in previous studies (Adamson et al., 2010
; Adamson & Sellman, 2003
), and could reflect ambivalence toward reporting some marijuana dependence symptoms in a community sample of young adults who may not see marijuana as an addictive substance. A the end of our survey, participants were given a chance to ask questions or make open-ended comments and more than a quarter of past-month marijuana users who chose to do so (28.4% of 331 comments) made a comment negating marijuana’s addiction potential (e.g., “marijuana is not addictive;” “marijuana does not kill, cigarettes do”) or specifically challenged the items assessing cannabis dependence (e.g., “The section about the marijuana use was a bit ridiculous….. Most marijuana smokers that are above the age of 18 are quite responsible in using it…..I don’t think anyone believes that people cannot function without the use of marijuana…”). Further research needs to evaluate the validity of the CUDIT measure in a community sample, as most of the work conducted to date has been with clinical samples of severe substance users (Adamson & Sellman, 2003
). Future work could also include other validated screening measures of problematic marijuana use (e.g., Marijuana Screening Inventory; Alexander, 2003
) or dependence symptoms (e.g., Marijuana Withdrawal Symptoms Checklist; Budney, Novy, & Hughes, 1999
) for comparison.
There were some notable differences between findings reported here and epidemiological data gathered through household interviews. For example, reports of past 30-day marijuana use prevalence were higher in our sample (57%) compared to the 2009 NSDUH reports of past 30 day marijuana use among young adults age 18 to 25 who also used tobacco (34.6%; SAMHSA, 2010a
). In addition, marijuana use frequency was not associated with socio-demographic characteristics found to relate to marijuana use in the NSDUH (SAMHSA, 2010b
). This is likely reflective of the characteristics of our convenience sample recruited using online advertising and recruitment concentration in areas of the United States that have a relatively high prevalence of marijuana use (e.g., California). The sample obtained through these strategies likely differs from samples recruited through other online means or using other sampling techniques that may be more representative of young adult smokers (Chang & Krosnick, 2009
). Further, demographic characteristics of young adult smokers recruited online vary by specific recruitment method (Ramo, Hall, et al., 2010
), and social media website use differs by ethnicity and socioeconomic status (Lenhart et al., 2010
). The present study made use of multiple recruitment strategies and targeted tobacco and marijuana users who may have a different sociodemographic profile than those who only use marijuana. Post-hoc analyses indicated that the three recruitment sources (Facebook, Craigslist, and SurveySampling International) recruited samples that differed by gender, age, ethnicity, employment status, annual family income, region of residence, and number of days using marijuana in the past 30, consistent with our prior online survey studies (Ramo, Hall, et al., 2010
). Those who came from Craigslist used fewer days of marijuana per month than those who came from Facebook (12.9 days vs. 17.1 days, F
(2, 880) = 4.00, p
= .019. Follow-up analyses examined marijuana prevalence for the three marijuana use items and also agreement among these items by recruitment site. Craigslist demonstrated somewhat lower agreement among marijuana prevalence items (kappas: .68, .73) compared to Facebook (kappas: .84, .83) or Survey Sampling International (kappas: .81, .80), which is consistent with socio-demographic differences in reliability found in this study (e.g., there was a larger proportion of African-American participants recruited through Craigslist; Ramo, Hall, et al., 2010
). Online research should consider how recruitment methods may affect both the representativeness of samples and reliability and validity of data gathered.
Construct and concurrent criterion validity were strong for stage of change and abstinence goals, which is consistent with previous tests of the Transtheoretical Model applied to tobacco use (Acton, Prochaska, Kaplan, Small, & Hall, 2001
; Prochaska et al., 2004
). Further, this model has been applied successfully to multiple health risk behaviors including alcohol and drug use (Heather, Hönekopp, & Smailes, 2009
; Prochaska et al., 1994
) and the present study is evidence that it can extend to marijuana use among young adults. Future work should test the central tenets of the TTM to demonstrate the full model validity over time in this population.
Relying on self-report, a study limitation is that respondents may not recall their behaviors accurately. However, this is true for face-to-face modes as well. We did not evaluate marijuana quantity of marijuana consumed, as the potency and route of administration of substances such as cannabis vary widely. Thus we can only draw conclusions about the reliability and validity of frequency reports only. Further, the cross-sectional study design limited an examination of test-retest reliability. We were unable to validate our marijuana use data with biological data due to concerns for anonymity, and the current design did not compare face-to-face to online reports of use. However, the comparisons between multiple measures of marijuana use information in our study suggest validity in reports. In addition, attrition was fairly high in that only 52% of the entire eligible sample completed the survey. However, this is consistent with other online survey studies with young adults (e.g., McCabe et al., 2002
), and methods of tracking participants beyond what were employed here would have compromised a goal of the research to maintain participant anonymity. Finally, study results may not be generalizable to populations other than young adults who report recent tobacco and marijuana use.
The present study demonstrated the validity and reliability of marijuana use, dependence symptoms, stage of change, and other marijuana-related cognitions reported anonymously online in a national sample of young adults who use tobacco and marijuana. Given that the Internet is so broadly used for surveys of substance use behaviors, it is important to know that the Internet yields valid and reliable data from young people. Further, the consistency in hypothesized patterns among theoretical constructs of the Transtheoretical Model supports the use of stage-tailored interventions to this clinical population. Online stage-tailored interventions with young adults have shown promise in changing health behavior and moving participants toward the action stage of change (e.g., Milan & White, 2010
). Findings here underscore the usefulness of these interventions to assist with smoking cessation for young people and support online assessment as a research tool. In addition to being a lower cost method of data collection, the privacy in which research participants can complete assessments may alone reduce social desirability bias from data collection with an interviewer, even in assessments that are not anonymous (e.g., longitudinal or intervention research). Future work should measure social desirability bias and directly compare data gathered confidentially online to data gathered anonymously and through other means (e.g., mail, in-person interviews to better evaluate this. As the Internet is increasingly used for survey research with young adults recruited from online settings other than college email lists (e.g., social networking websites, classified advertising spaces), issues of privacy and non-response bias make it important to evaluate the reliability and validity of online surveys as a research tool. As social media and other public online setting are sources for diverse samples of young adults, rather than limited to those enrolled in colleges, the validity and reliability of data from these sources will ensure this research has maximum impact.