Recruitment results and sample characteristics
The study was run for 16 weeks between June 2005 and September 2005. During that time, 297 individuals were invited to participate. Of those, 288 accepted (97%) and 221 completed the survey for a final recruitment rate of 76.7%. Eight individuals completed the online survey more than 1 week after their telephone assessment and were excluded from analyses, resulting in a final sample of 213.
The majority of participants were female (65.3%), 79.8% were White, 11.7% Black, 4.2% Asian, 3.3% American Indian or Alaskan Native, and 2 individuals (<1%) were Native Hawaiian or other Pacific Islander. The average age of participants was 35.5 (SD=10.2; range 18–70). Almost half of participants had completed 1–3 years of college (49.3%), followed by 31.9% with ≥4 years of college, 16% with a high school degree or GED, and 2.8% with less than a high school degree. Analysis of family income showed 27.1% earning less than US$30,000 per year, 33.3% earning US$30,000 to US$50,000 per year, and 39.5% earning US$50,000 or more.
The majority of participants (92.5%) reported that they were planning to quit in the next 30 days. On average, participants smoked 19.3 cigarettes per day (SD
=9.9, range 5–60), had their first puff of a cigarette at age 14.2 (SD
=3.4, range 7–29), and became daily smokers at age 17 (SD
=3.6; range 8–30). Participants had made an average of 2.9 quit attempts in the past year (SD
=5.2; range 0–50) and reported higher levels of desire to quit (M
=1.54) than confidence in quitting (M
=2.29). Average score on the Fagerström Test for Nicotine Dependence (Heatherton, Kozlowski, Frecker, & Fagerström, 1991
) was 5.03 (SD
=2.37), with 44% of participants scoring 6 or above indicating a high level of nicotine dependence (Fagerström, Kunze, Schoberberger, et al., 1996
). Analyses of body mass index indicated that 32.1% were overweight and 24.4% were obese according to standards of the Centers for Disease Control and Prevention (2004)
. Of those who reported current alcohol use (74.2%, n
=152), 60.8% indicated they had more to drink than they meant to in the past year and 21.5% indicated they wanted or needed to cut down on their drinking.
The majority of participants had used the Internet for more than 5 years (79.8%), accessed the Internet several times a day (76.1%), and used a high-speed Internet connection (85.8%). Almost half (44.1%) of participants used the Internet to communicate with other people through services like an Internet blog, online bulletin board, or instant messaging.
Survey completers vs. noncompleters
Survey completers (n=213) were compared with noncompleters (n=67) to determine if there were group differences in demographic, smoking, and psychosocial characteristics. There were no differences on any of the variables tested, which included age, gender, race, education, employment, income, marital status, smoking rate, nicotine dependence score (Fagerström Test for Nicotine Dependence), desire or confidence in quitting, duration or frequency of Internet use.
Means and descriptive data
Of the continuous variables examined in , the only variable showing differences of moderate effect size between the Internet- and phone administered version is the Negative Affect Situations subscale of Smoking Temptations, with the mean of the phone-administered measures 0.44 standard units higher than the mean of the Internet-administered version (p<.0001). Positive Affect also tends to be higher when reported over the phone, although the observed difference of 0.17 standard units is considerably smaller, albeit statistically significant (p=.0193). As a result, the Total Score of the Smoking Temptations scale shows an overall difference in the “small-to-moderate” range, with the two sample means 0.34 standard units apart (p<.0001). Other measures such as the CES-D show statistically significant differences even though the observed effect sizes are small, a result of the ample power our sample size affords for detecting within-subject differences in continuous outcomes. This lack of systematic bias between the two survey methods is accompanied by strong intraclass correlations, as depicted in .
Although the binary variables listed in also show no significant differences between the two survey methods, the power of McNemar’s test in the present study is quite low for all but large effect sizes because of the small number of discordant pairs (N_D<23 throughout); the only exception is self-reported smoking-related illness, whose prevalence is considerably higher when assessed over the phone (59.91% vs. 49.53%, N_D=32, p<.0001). The ordinal variables listed in (Income, Health Status) also show no significant differences in prevalence (p>.10).
Test–retest reliability and internal consistency results
The test–retest reliabilities for continuous variables measured in these two surveys are uniformly high (above 80%) for the Partner Interaction Questionnaire (PIQ) and the alcohol consumption measures, and moderately strong (in the 70%–80% range) for the Perceived Stress Scale (PSS) and CES-D. As seen in , the results are least satisfactory for the individual subscales of the Smoking Temptations Questionnaire scale, all of which fall below the 70% reliability threshold. Still, the overall scale (Total Score) is more reliable, as would be expected from a composite of three correlated subscales, with its ICC attaining exactly the 70% threshold.
Substantial test–retest agreement has also been obtained for the binary variables in , with kappa values above .70 for all but four variables assessing prior use of quit methods: Individual counseling, nicotine spray, Internet treatment, and telephone counseling. However, the precision of the reliability estimates is lower for binary than for continuous measures, and the 95% confidence intervals appear quite wide, allowing for the possibility that four additional variables show only moderate degrees of agreement between the two survey methods: Use of group counseling, nicotine inhaler, switching to chewing tobacco or snuff as methods to quit smoking, and report of ever having had a smoking-related illness.
In , we find evidence of substantial agreement for health status (weighted κ=.73) and almost perfect agreement for the income measure (weighted κ=.93).
Results for the income measure were not dependent on whether the midpoint of the highest income category used to construct the weights was changed from US$125,000 to US$150,000. Because of the informativeness of ordinal as opposed to binary measures, the confidence intervals are narrower which indicates improved precision in the estimates.
Finally, in we evaluated the internal consistency of four scales and use them to compare the two survey methods. Cronbach’s alpha reliability coefficients exceed 80% for CES-D under both methods, are in the 70% to 80% range for the PIQ and PSS scales, and only fall below 70% for the phone-administered version of the Smoking Temptations scale. Between-method comparisons show no statistically significant differences for PIQ, PSS, and CES-D. Between-survey differences for the Smoking Temptations scale are borderline significant at the 5% level; in this case, as in all others, the Internet version appears to have higher internal consistency than the phone version.