It is important to acknowledge several limitations to the present study. First, we did not design the Enhanced intervention website to track passive Web forum viewing. This limitation prevented us from analyzing the duration of Web forum visits by participants who observed postings but did not post their own comment on the forum posts of others. In addition, although study inclusion criteria required all study participants to be able to access their personal email at least once per week, we did not collect data on participants' previous experience using the Internet or on their computer self-efficacy [38
]. As a result, we cannot report directly on whether there were significant differences between the intervention and control conditions for these dimensions. We believe that random assignment and our large sample size (N = 2375) would tend to mitigate the likelihood of this effect.
It is thought that a key ingredient in determining the impact of any Web-based behavior change program is the extent to which participants are exposed to the program. This assumption is consistent with the finding that the efficacy and intensity of treatment programs tend to be positively related. For example, research on smoking cessation interventions—including self-help approaches—has illustrated the relationship between abstinence rates and program intensity, typically defined as contact time and number of sessions [4
]. Williams et al [41
] have coined the term program thickness
to refer to the "collective intensity, duration, delivery agent, and intervention modality" of an intervention. However, research has also shown that more is not always better when considering which ingredients to include in an intervention [42
] or a website [19
], perhaps because adding features increases the response cost of participation and reduces usage.
Some reviewers of this burgeoning field have recommended that fuller participation in Web-based interventions might be encouraged through the use of a "warm-up period" during which users can demonstrate their commitment by complying with precursor tasks while they become more familiar with what will be asked of them during the course of the program [1
]. The use of intensive treatment approaches and preliminary litmus tests of commitment must be tempered by recognizing the continuum between clinic-based and public-health models for intervention. Specifically, it may be not be a practical goal to provide a highly intensive, population-wide intervention. Moreover, the use of preliminary barriers may help to reduce attrition in efficacy trials but reduce our ability to conduct effectiveness trials that have a broader reach and greater potential to achieve public health impact [46
]. A challenging—and fruitful—line of research lies in identifying the proper program ingredients that provide a balance between sufficient exposure to relevant content and structure on the one hand while encouraging widespread user participation and engagement (both recruitment and follow-through).
We found that the estimated median lifetime website usage (date when 50% of participants stopped using the program) was 11 days for the Enhanced condition and 0 days (ie, the enrollment day) for the Basic condition. We anticipate that some measures of exposure and outcome will likely share a curvilinear (inverted U-shaped) relationship such that those individuals who are least ready to make a meaningful change may be more likely to visit the Web-based program for a short time, while participants who are most prepared to change their behavior may similarly choose to visit the Web-based program for a relatively short time. Those participants who are interested in quitting and decide to learn more about how to do so will spend relatively more time visiting the program. It remains for future research to differentiate characteristics that illuminate the pattern of this relationship among motivation, readiness to quit, and program usage.
Measures of participant exposure can help researchers and program developers determine the extent to which content is viewed. These data can point to needed changes in the information architecture and design features of the website. It is reasonable to assume that program content cannot be helpful if it is never viewed. Exposure measures may have utility in that they inform us about whether certain content—or clusters of content—is related to outcome and thus might be considered to be active ingredients in accomplishing the desired behavioral goals. They enable us to better understand idiosyncratic patterns of program use, highlighting ways we can adapt program structure and content to better accommodate (be tailored to) individual differences in participant interests, needs, and learning styles.
In this regard, we intend to examine a variety of relationships between and among measures of exposure and the smokeless tobacco and tobacco cessation outcome measures in the ChewFree.com research project. For example, we will test whether participants who set a quit date are more successful in quitting, as well as whether, after quitting, there is a relationship between accessing content from the Staying Quit module (number and duration of visits) and lasting abstinence. Similarly, we will examine whether those participants who spend more time reviewing program content after they have lapsed are better able to regain control over their behavior and regain abstinence. We also plan to perform content and text analyses of Web forum postings [eg, 48] to explore whether smokeless tobacco cessation might be related to message types, whether cessation and maintenance strategies shared in postings are consistent with program recommendations, and the extent to which postings convey differing levels of confidence and self-efficacy across participants as well as within participants over time.
There is a significant risk of collecting so much detailed exposure and engagement data that the task of analyzing and interpreting results becomes difficult. We suggest that this task can become more manageable and, thus, more fruitful, by focusing its scope through the use of a rationale that incorporates both theory and pragmatism. Potentially relevant rationales are not difficult to identify. Consider, for example, a rationale that builds on the Web foraging model [49
], which posits that Web users guide their review of online content by quickly identifying interesting information scents
in website materials. This model suggests that websites should foreshadow content even when it is not immediately accessible in order to engage users. It also points to particular usage patterns—brief initial visits followed by later visits of more duration [49
]. The Transtheoretical/Stages of Change model may also hold promise in focusing the analysis of exposure and engagement. Velicer et al [51
] suggest that users in action stage will access a program relatively more than users characterized as being in early stages (precontemplation, contemplation) or the later maintenance stage. Similarly, it might be helpful to consider the behavioral self-management model [17
], which suggests that users who become more confident and capable in their self-management skills would tend to access a program less over time.
We view exposure as representing one of a set of complementary measures of the broader theme of program engagement. Other engagement measures include participant comprehension of program content, practice of that content (especially in the participant's everyday routines outside of interacting with the Web-based program), self-reported satisfaction with the function and content of the website, and measures of self-efficacy. While exposure is obviously important (indeed, it is best viewed as a prerequisite), it represents only one piece of the puzzle in seeking to understand program effectiveness.