In this study, we showed that enhancement of a smoking cessation Web site with ITEMs, sent via e-mail at strategic times in smokers' quit efforts, increased the rate of early successful quit efforts. There are a number of caveats to this result. The result is based on a historical comparison group and the characteristics of persons visiting the site changed somewhat over the period of data collection, presumably due to changes in how the site was listed with search engines, advertising, etc. Participants were similar in their level of addiction to nicotine, number of cigarettes smoked (though these variables show statistical differences due to the large sample size), duration of smoking, level of depression, and confidence in quitting. Adjusting for these differences using a multivariate statistical model appeared to strengthen differences in outcomes between groups.
One important limitation of this study is that the ITEMs intervention included more components than just reinforcing e-mail messages. The intervention also included a journal and an online tracker for cigarette use. These components could also have contributed to the observed results. These components were added to keep our site competitive with the features of other sites offering smoking cessation services on the Web. Further studies are needed, with larger sample sizes, to elucidate the specific mechanisms of effects, although we believe that the effects of these components were small.
The result is seen at a relatively early period in the process of becoming a nonsmoker. Our prior data from Web studies of smokers9
suggest that this is the maximum time of response and the best time to detect an effect of an intervention on the process. However, these data do not show that e-mail improves long-term quit rates. Rather, the data argue only that it helps individuals undertake more attempts and achieve higher rates of early success. At this stage in the development and evaluation of Web-based smoking cessation interventions, a focus on short-term effects seems appropriate. As in traditional face-to-face interventions, once we have found the most effective short-term interventions, we will need to develop and study interventions that have long-term effects.
The result of this study is based on the use of e-mail and Web technology to elicit outcome data. Previous studies have shown that a substantial proportion of participants in online studies will respond to e-mail requests for follow-up information on quit efforts and complete online questionnaires. For example, in our initial study,9
approximately 42% of participants were willing to respond to e-mail requests for this information. By linking requests to incentives for response, Feil and colleagues17
were able to achieve a 46% follow-up rate in long-term users of a Web site. This approach introduces its own types of biases, including a social acceptability bias. Ultimately, the developers of Web applications for smoking cessation and behavior change need to know whether their applications, in their native contexts without incentives, have effects on behavior. For this reason, our study did not use incentives, and as a result, our response rates were relatively low, especially with the ITEMs intervention group.
Among respondents, rates of seven-day abstinence were relatively high. However, it is unlikely that nonrespondents quit in the same proportion as respondents. We therefore conservatively assumed that nonrespondents continued to smoke—a common assumption in analyses of smoking studies. This produced one-month quit rates of approximately 13.5% in the ITEMs group. The odds ratio for quitting at 30 days was 2.55. This compares favorably with most self-help interventions for smoking cessation. Self-help interventions typically have weak effects—odds ratios that average approximately 1.3 compared with no help.18
The increment in effectiveness observed over the self-help comparison group was large compared with the effort required. For example, a review of group behavior programs found that adding these labor- and time-intensive programs to a self-help intervention increased the odds of quitting by only approximately 1.9.19
The ITEMs intervention Web site is fully computerized and requires no more attention than any other Web site. Therefore, even if our smoking Web site was ultimately effective in only a small percentage of persons who use the site, it could still have a relatively large public health impact because of the low unit cost of delivery of this intervention.20
Two previous studies have shown in randomized trials that e-mail communications can enhance the effectiveness of other medical interventions; however, these studies focused on e-mail messages generated by people. Lorig et al.21
demonstrated in a randomized trial that a closed, moderated e-mail discussion group increased the efficacy of a self-help program with a textbook and other materials in the treatment of chronic back pain. This type of activity requires extensive effort on the part of the leader of the group to filter and respond to messages. Tate and colleagues6
have examined the use of e-mail counseling to enhance a weight loss program. This program also required intensive individual efforts from counselors. Similar to Lorig et al., these investigators found significant enhancements in weight loss when counselors provided individualized feedback via e-mail. Of course, the use of counselors or moderators increases the costs of delivery of the intervention.
This is the first reported study, to our knowledge, of the effects of automated e-mail messaging on quit rates. Etter et al.22
examined the effect of automated e-mail messaging on smokers' motivation to quit. The research group sent e-mail messages to registrants at a Web site for persons interested in smoking cessation. Respondents (25% response rate) indicated that messages encouraging the use of the nicotine replacement to cut down on the number of cigarettes smoked per day increased smokers' intention to quit. Two other messages that advised recipients of the adverse effects of combining nicotine replacement with smoking or urged the use of nicotine replacement in situations in which smoking was forbidden did not have effects of stated intention to quit.