Initial trials of web-based smoking-cessation programs have generally been promising. The active components of these programs, however, are not well understood. This study aimed to (1) identify active psychosocial and communication components of a web-based smoking-cessation intervention and (2) examine the impact of increasing the tailoring depth on smoking cessation.
Randomized fractional factorial design.
Two HMOs: Group Health in Washington State and Henry Ford Health System in Michigan.
A web-based smoking-cessation program plus nicotine patch. Five components of the intervention were randomized using a fractional factorial design: high- versus low-depth tailored success story, outcome expectation, and efficacy expectation messages; high- versus low-personalized source; and multiple versus single exposure to the intervention components.
Primary outcome was 7 day point-prevalence abstinence at the 6-month follow-up.
Abstinence was most influenced by high-depth tailored success stories and a high-personalized message source. The cumulative assignment of the three tailoring depth factors also resulted in increasing the rates of 6-month cessation, demonstrating an effect of tailoring depth.
The study identified relevant components of smoking-cessation interventions that should be generalizable to other cessation interventions. The study also demonstrated the importance of higher-depth tailoring in smoking-cessation programs. Finally, the use of a novel fractional factorial design allowed efficient examination of the study aims. The rapidly changing interfaces, software, and capabilities of eHealth are likely to require such dynamic experimental approaches to intervention discovery.
More than a decade of research evaluating the effectiveness of computer-tailored smoking-cessation materials has led to a new class of intervention tools that are well into the early phases of large-scale dissemination. Meta-analyses and reviews of print-based, tailored smoking-cessation interventions have demonstrated a generally positive set of findings, both in significance over comparison conditions and in magnitude of change.1,2 Concepts utilized in these interventions are now migrating to the World Wide Web (“web”), a medium capable of reaching very large numbers of smokers, at any time, for a fraction of the cost of tailored print-based materials. Recent evaluations of web-based tailored smoking-cessation programs have demonstrated outcomes similar to those found in tests of tailored print materials,3–5 although variation in efficacy among web-based interventions has also been found.6
This first generation of research examining the effectiveness of computer-tailored smoking-cessation materials focused on whether “black-box” interventions—what the investigators considered to be an optimal algorithm of various psychosocial and communications components—had a greater impact on cessation than either untailored smoking-cessation materials or the absence of smoking-cessation materials. However, the generally positive but variable results from these interventions suggested the need for a second generation of research that identified the active components of tailored interventions.
In an effort to systematically identify active components of a web-based smoking-cessation intervention, this study used an approach adapted from a framework that has been used successfully in engineering for many years.7 This process, termed Multiphase Optimization Strategy (MOST) by Collins and colleagues,8,9 involves (1) screening potentially active components of a black-box intervention, (2) refining knowledge of the effects of the most relevant components identified in the screening stage, and (3) confirming the optimized set of the components through a trial of the resulting intervention.
In the screening phase, a relatively large number of potentially important components (their selection guided by theory and existing empirical research) are usually examined. These components are typically evaluated through the use of either a factorial or a fractional factorial design. A fractional factorial design employs a systematic approach to reduce the number of study arms to allow a more manageable study, at the cost of allowing only main effects and a pre-specified set of interactions to be tested. In the screening phase, this trade-off is usually acceptable.
This article presents the screening phase of the study, examining five intervention components (factors) using a fractional factorial design. Two of the factors selected for this randomized trial, outcome and efficacy expectations (i.e., self-efficacy), are derived from social cognitive theory10 and are central to most major theories of health-related behavior. For example, within the health belief model,11 perceived-threat and perceived-benefits constructs are associated with outcome expectations, while the perceived-barriers construct is associated with efficacy expectations. Within the transtheoretical model,12 the pros-and-cons constructs are associated with outcome expectations, while the temptations construct is associated with efficacy expectations.
The other three factors—success stories, message source, and message exposure—may be viewed as methods of conveying outcome and efficacy expectation content and are classic constructs examined in the study of persuasive communications.13,14 Success stories communicate a variety of messages: In this study, they communicated both outcome and efficacy expectation messages in a narrative format as opposed to an advice-driven format. Message source focused on the degree of personalization given to the source, which was the HMO. Message exposure focused on the temporal distribution of the web-based sections of the program: with the sections either grouped into a large single entity or distributed over time, as in a correspondence course.
A second issue relevant to computer-tailored smoking-cessation materials is the degree of tailoring depth required to produce a significant outcome. Tailoring depth refers to the degree to which assessment data and connections among data have been utilized to produce the message. In other words, has sufficient feedback been produced from the assessment and utilized in the tailored message? The impact of tailoring depth, or “granularity,” has been discussed frequently in the literature and at scientific conferences,15,16 but to date has not been tested. The questions becomes: How many versions of self-help materials are needed? Adding depth to computer-tailored messages requires more effort to develop concept and message and greater technologic capabilities (e.g., tailoring software), so an understanding of the point at which these efforts can be relaxed would be important to both researchers and developers.
The two aims of this research included (1) identifying active psychosocial and communication components of smoking-cessation interventions and (2) examining the impact of increasing the tailoring depth in the web-based intervention. To accomplish these aims, a fractional factorial design was used to screen and identify promising intervention components from a set of potentially active web-based smoking-cessation components. This design allowed the examination of both main effects of the intervention components as well as pre-specified interactions among intervention components and participant characteristics.
Intervention components developed with high-depth tailoring (e.g., high-depth efficacy expectations, outcome expectations, and success stories) were hypothesized to produce higher subsequent rates of smoking cessation than low-depth tailored versions. Greater personalization of the message source was hypothesized to produce higher cessation rates than a less-personalized source, and distributing the web-based program sections over five weekly installments was hypothesized to result in higher cessation rates than combining the sections into one large installment.
A number of interactions among intervention components, and among intervention components and participants’ characteristics, were also hypothesized. Since self-efficacy is a relatively consistent predictor of subsequent health-related behavior change,10 the efficacy expectation intervention component was of particular interest; the fractional factorial design was constructed to maximize the ability to explore 2-way interactions between a high- versus low-depth tailored efficacy expectation component and other intervention components. It was hypothesized that high-depth efficacy expectation messages—when paired with either a more personalized message source, with high-depth outcome expectation messages, or with high-depth success stories—would produce particularly high rates of cessation. Also examined were potential interactions between the efficacy expectation component and participants’ baseline self-efficacy, between the outcome expectation component and participants’ baseline motivation, and with the success story component according to participants’ baseline level of education; it was hypothesized that more deeply-tailored messages would have particularly strong effects among participants with lower baseline levels of self-efficacy, motivation, and education, respectively.
Participants were recruited from the memberships of two HMOs participating in the National Cancer Institute’s Cancer Research Network: Group Health of Seattle WA and the Henry Ford Health System (HFHS) of Detroit MI. Both Group Health and HFHS are not-for-profit healthcare delivery systems. An individual was eligible to participate if he or she (1) had smoked at least 100 cigarettes in his or her lifetime, currently smoked at least 10 cigarettes per day, and had smoked in the past 7 days; (2) was seriously considering quitting in the next 30 days; (3) was aged 21–70; (4) was a member of either Group Health or HFHS; (5) had home or work access to the Internet and an e-mail account that he or she used at least twice weekly; (6) was not currently enrolled in another formal smoking-cessation program or was not currently using pharmacotherapy for smoking cessation; and (7) had no medical contraindications for nicotine replacement therapy (NRT).
All participants received access, free of charge, to the experimental smoking-cessation program delivered via the web. Specific intervention components received by participants varied according to the experimental group to which they were assigned. All participants also received, free of charge, a 10-week supply of NRT patches. The purpose of NRT provision was to minimize the potential confounding effects of adventitious differences in physiologic addiction and to allow participants to focus on the cognitive–behavioral aspects of smoking cessation. A previous trial combining a web-based behaviorally tailored smoking-cessation program with NRT had demonstrated positive and relatively high rates of cessation 3 months post-quit date.4
The study protocol was reviewed and approved by the IRB of each collaborating institution and of the University of Michigan in January 2004.
Participants were recruited through a combination of individual- and population-level strategies between September 2004 and July 2005. Each of the two HMOs identified likely current smokers via automated smoking status data collected during recent medical appointments, documentation of smoking in electronic medical charts, an internal list of smokers collected during prior research, or lists of patients with smoking-related conditions who had previously been prescribed cessation medications. Thus all invitees were known to have been recent smokers, and there was a high probability of their being current smokers. These likely smokers were prescreened for minimal inclusion criteria (e.g., age) and were mailed a study invitation letter. Individuals who either had not opted out of further contact or had not visited the website at least 4 weeks after their initial invitation were sent a second reminder mailing. Several population-level enrollment strategies were also utilized, including promotion of the study in the HMO newsletter and to HMO staff. Further description of participant recruitment procedures and their results are presented by McClure et al.17
Those invited to participate were given a web address (URL) and an identification code for entering a personalized website. After logging in, invitees were administered the baseline questionnaire. The intervention delivery system controlled the interaction with the participant by running a software script that collected data via an assessment and then immediately produced appropriate (i.e., tailored) cessation feedback based on those data. The baseline questionnaire assessed and stored in a database both the participant’s smoking history and his or her psychosocial, health, and demographic characteristics relevant to smoking-cessation programming.
Randomization was stratified automatically within HMO site by the computer immediately after the assessment, and was invisible to the participant. Follow-up interviews administered 6 months post-quit date were conducted using a computer-assisted telephone interview (CATI). Group Health provided training and oversight of the CATI interviewers.
The overall web-based program and each experimental factor with the program were developed at the University of Michigan’s Center for Health Communications Research (UM-CHCR). The content of the program was based on cognitive–behavioral methods of smoking cessation and relapse prevention, including an appeal to motives for quitting, stimulus control, self-efficacy enhancement, and suggestions for coping with tempting situations and emotions. The intervention components selected for testing within this overall paradigm included outcome expectations, efficacy expectations, use of hypothetical success stories, personalization of the message source, and the timing of message exposure.
Within three of these factors (outcome expectations, efficacy expectations, and success stories), the depth of tailoring was experimentally manipulated. The term “tailoring” refers to a process that consists of (1) an assessment of individual characteristics relevant to smoking cessation, (2) algorithms that use the assessment data to generate intervention messages relevant to the specific needs of the user, and (3) a feedback protocol that delivers these messages to the smoker in a clear, vivid format. The web-based program (www.chcr.umich.edu) utilizes integrated cessation messages from multiple assessment responses to develop sentences and paragraphs written specifically for the user. An example of low- and high-tailored success story feedback is presented in Figure 1.
Study participants received a variation of each of the five 2-level intervention factors described below.
In this factor, the depth of tailored outcome expectation feedback was manipulated. Messages include statements tailored to personal and family health history, perceived health status, functional health status, monetary savings, and appearance, among other outcomes. Participants randomized to the high-depth tailored group received feedback and advice related to their specific motives for quitting. In addition, these participants received an overview of the balance between their intrinsic versus extrinsic reasons for quitting. Participants in the low-depth tailored group received feedback that related to their motives for quitting but did not make as many connections to their existing health or lifestyle characteristics, nor did the feedback address the balance between intrinsic and extrinsic motives.
Tailored efficacy messages addressed relevant barriers to quitting. Responses to high-risk situations, existing skills, and attributions for previous failures in quitting, along with smoking history and current smoking behavior, were used to help build self-efficacy feedback. Those with previous cessation attempts, for example, were asked to consider these experiences in developing coping strategies for specific perceived cessation barriers. Participants randomized to the high-depth tailored group received feedback and advice focusing attention on their two most problematic individual barriers to quitting (e.g., wanting to smoke when drinking coffee, or when feeling stressed, or when spending time with friends and family who smoke). Highly tailored feedback then also utilized information about the participant’s home environment, family life, stress and coping levels, coping skills, and level of physical activity, among other unique characteristic traits, to provide enhanced advice in dealing with the barriers addressed. Participants in the low-depth tailored group received less-tailored content addressing two broader barrier topics cited by the smoker (e.g., daily routines, negative emotion control, social settings).
As part of the intervention, participants received a hypothetic story about an individual who successfully quit smoking. Low-depth success stories were tailored only to the participant’s name and gender. A participant randomized to a high-depth level received a success story tailored not only to name and gender, but also to age, ethnicity, marital status, spouse’s smoking status, number of cigarettes, biggest barrier to quitting, reason for wanting to quit, degree and type of social support, and whether the participant had children in the home, was physically active, or worked outside the home (see Figure 1 for an example of this tailoring manipulation). The tailored success stories used narratives to address both outcome and efficacy expectations.
In the introductory section that welcomed a participant to the program, the highly personalized source version included a photograph of, and supportive text from, the smoking-cessation team of the HMO. It was written in a friendly, personal manner using words like “we” and “our team,” and ended with a signature from the team. The low-personalized version included a photograph of a building representing the HMO institution, used impersonal terms like “this organization,” and did not include a closing signature.
This manipulation compared the impact of providing the smoking-cessation content in a single large set of materials (the equivalent to roughly 16 pages of text in a printed self-help guide) to that of dividing the materials into a series of weekly installments. Participants received the online efficacy, outcome, success story, and source materials either all at one time or distributed over a 5-week period (efficacy messages were separated into 2-week segments) with e-mail reminders to revisit the site when new content was made available. In both exposures, once content was available, it remained available throughout the study period.
This study was used to screen for and identify important intervention components or factors for smoking cessation.8,9 A fractional factorial design with 16 arms allowed the estimation of all main effects and several pre-specified 2-factor interactions among the five intervention factors; in statistical terminology, this is called a Resolution IV design.7 Table 1 describes the specific combinations of 2-level intervention factors in the experimental design. A full factorial design of five factors would have required 25=32 arms; the fractional factorial design reduced the total number of arms by one-half.
(The original design, in fact, included six factors with a 32-arm fractional factorial design to study all six. However, due to a programming error, 1 of the 32 arms was not filled. Data analysis showed that the additional factor was not significant. As a result, this factor was removed, and the design was folded into 16 experimental arms, with one of the arms receiving half the number of subjects. This led to an unbalanced number of subjects in the 16 arms; as a result, regression analyses including all five treatment conditions in the analyses were used.)
The primary abstinence measure in this study was 7-day point prevalence abstinence (“Did you smoke a tobacco cigarette, even a puff, in the past 7 days?”). Abstinence was assessed by self-report during a telephone interview at 6 months post-quit date. Biochemical verification was not collected, as it was considered impractical for this population-based study,18 and there is general consensus that self-report is adequate in minimal-contact treatment studies where there are low demands to misrepresent one’s smoking status.19,20
Data were analyzed in 2007, addressing the two aims of the study: (1) identifying active psychosocial and communication components of smoking-cessation interventions and (2) examining the impact of increasing the tailoring depth in the web-based intervention. Analyses were conducted using logistic regression procedures including relevant design, demographic, and psychosocial covariates. Within each aim, three analyses were conducted: a per-protocol analysis, which was considered the primary analysis in screening for potentially active components of cessation treatment; a complete respondent analysis; and an intent-to-treat (ITT) analysis. The per-protocol analysis focused on participants who (1) answered the smoking-cessation–related questions, and (2) did not report having used other smoking-cessation aids or programs during the 6-month intervention and follow-up period. The complete respondent analysis focused on participants who answered the smoking-cessation–related questions at the 6-month follow-up, regardless of their use of other cessation products and services. In the ITT analysis, participants who failed to provide abstinence data for any reason were considered treatment failures (i.e., current smokers).
To identify potentially active components of cessation, the five experimental factors, along with the hypothesized interactions between factors and factors-by-participant characteristics, were included as independent variables in the model and regressed against the 6-month abstinence variable. Because each factor represents a component of a larger treatment intervention, a traditional αof 0.05 was not used as the criterion for identifying the important factors.8 Instead, it is more meaningful in screening experiments to rank intervention components by standardized effect sizes (or equivalently by p values) and select the important factors following a Pareto analysis. These components are then further examined in follow-up “refining” experiments.
To determine the impact of increasing the tailoring depth, a score was created to represent the number of high-depth tailored components received by the participant. Randomization of the three tailoring depth factors (outcome expectation, efficacy expectation, and success stories) allowed participants to receive a range of 0–3 high-depth tailored components. This score led to a new factor with four levels; the score was then regressed on the 6-month abstinence variable using logistic regression.
During an 11-month recruitment period, 3256 people from both HMOs visited the website; 2651 (81% of website visitors) were screened for eligibility; 2011 (62% of website visitors) were eligible; and 1866 enrolled and were randomized to one of the 16 study arms (57% of website visitors). The primary reasons for ineligibility were: did not smoke enough (26%); medical contraindications for NRT (23%); already enrolled in another smoking-cessation program (16%); lack of adequate Internet/e-mail access (14%); not currently enrolled in the HMO (10%); and currently using pharmacotherapy to quit smoking (8%).
Of these participants, 1415 (76%) responded to the 6-month follow-up CATI and were included in the complete respondent analyses. Of the complete respondent group, 954 reported using no other smoking-cessation aids or programs during the 6-month follow-up period and were included in the per-protocol analyses.
A chi-square test was used to check the association between treatment arm (16 levels) and nonresponse to the 6-month follow-up. There were no significant differences in nonresponse rates between intervention arms (p=0.75).
Demographic data, smoking history, and psychosocial characteristics of enrolled participants by HMO are presented in Table 2.
Possible differences in each of these baseline characteristics were examined by the five experimental conditions. Of the 40 comparisons, significant differences at the p<0.05 level emerged for two baseline participant characteristics, motivation and self-efficacy, both of which were higher in the low- than in the high-tailored success story condition.
Program utilization was measured by a routine of the UM-CHCR data-collection and management software that examined the number of sections opened throughout the course of the study. A total of five sections were available, each focusing on a specific cessation component (efficacy expectation messages were split into two sections). Participants opened an average of 2.8/5 sections; the modal number of sections opened was all five sections (36.7% of participants). Participants receiving the single exposure of sections opened more of the sections (3.1/5) than those receiving the sections over time (2.6/5) (F=37.8; p<0.001).
A total of 10 weeks of nicotine patches were sent to each participant. At the 6-month follow-up, participants were asked how many weeks of nicotine patches they used. Participants reported using an average of 5.1/10 weeks of nicotine patches; the modal patch use was 10 weeks (26.7% of participants).
A logistic regression model was run that included the five experimental factors, along with the hypothesized interactions among factors and factors-by-participant characteristics, and the eight baseline smoker characteristics of Table 2. Table 3 presents adjusted smoking-cessation rates by each intervention component and significant-to-marginally-significant interactions using per-protocol analysis criteria. Two of the intervention components—source personalization and success story depth—were significantly related to 6-month cessation. These two intervention components were also significant using complete respondent and ITT analysis criteria. While in the expected direction, neither efficacy nor outcome expectation components had a significant influence on smoking cessation. Cessation rates at 6 months did not differ among participants assigned to single- versus multiple-exposure conditions.
Two of the hypothesized interactions were at least marginally significant in the per-protocol analyses. As hypothesized, high-tailored success stories had a particularly significant impact on participants with less-than-college-graduate education (adjusted cessation rates of 39.9% for high-tailored success stories versus 25.6% for low-tailored success stories). A marginally significant interaction between source personalization and efficacy depth was found. Any combination of high-personalized source or high-tailored efficacy messages produced significantly higher cessation rates than low-personalized source and low-tailored efficacy messages. However, neither of these interactions was even marginally significant in the complete respondent or ITT analyses. No other two-way interactions were significant.
To test the impact of tailoring depth, participants were assigned to receive between 0 and 3 high-depth tailored intervention components (success stories, outcome expectations, and efficacy expectations). Tailoring depth was significantly related to 6-month smoking cessation using per-protocol analysis (Figure 2), both across the entire range of cumulative high-depth components (OR=1.91; CI=1.18–3.11) and for each high-depth component added (OR=1.24; CI=1.06–1.45). Tailoring depth was marginally related (p<0.08) to smoking cessation in the complete respondent and ITT analyses. Adjusted 6-month cessation rates among participants receiving all three high-depth tailored components were 38.6% in the per-protocol analysis, 37.9% in the complete respondent analysis, and 27.7% in the ITT analysis.
This study used a randomized experimental design to address two aims: (1) identifying active psychosocial and communications elements of a web-based smoking-cessation intervention and (2) testing the impact of tailoring depth on smoking cessation. The study tested these aims in two large, generalizable health delivery systems using recruitment strategies similar to those used in the health promotion programming of these systems.
Two intervention components demonstrated particular promise for further study: success stories and message source. The positive impact of stories or other narrative forms of communication on persuasion was demonstrated decades ago.21 More recently, Green and Brock22 showed that the extent to which an individual is absorbed or “transported” by a story has a strong influence on persuasion. The high-depth tailored versions of the success stories in the current study were an attempt to transport the smoker into a familiar environment while addressing relevant outcome and efficacy expectations within the message of the story. It is noteworthy that participants with lower levels of education were particularly influenced by the high-depth tailored success stories.
Message source, a classic focus of communications research, is rarely examined in smoking-cessation research. In this study, the more personal message source led to significantly higher cessations at the 6-month follow-up, and also appeared to marginally enhance the impact of the high-tailored efficacy expectation messages. Members of HMOs often perceive these organizations as untrustworthy due to a lack of openness and accountability.23 Personalizing the source of the message content may have conveyed greater trustworthiness, in turn leading the message recipient to a stronger commitment to quit smoking.
Two other intervention components—outcome expectations and efficacy expectations—were not strongly related to cessation. These components presented tailored content through a traditional, advice-style format. It should be noted that the success story component used both outcome and efficacy expectation messages, but presented them in a narrative format. The results of this study therefore suggest that “how you say it” and “who says it” are at least as important as “what you say.” As a follow-up to this screening phase, a second randomized trial has been undertaken to test the impact of added source personalization and variations in story design on smoking cessation.
The method of program delivery, whether as a large single installment or spread out into multiple sessions (almost like a correspondence course), had no influence on cessation rates. Interestingly, however, the utilization of web-based sections within the single exposure was significantly higher than the utilization within the multiple-session program.
Increasing the tailoring depth was found to influence subsequent smoking cessation. The combination of the highest tailored intervention components resulted in a relatively high mean quit rate for a low-cost, minimal contact, population-based intervention (38.6% per-protocol analysis; 37.9% complete respondent only; 27.7% ITT) even with the supplemental use of nicotine replacement therapy.1,24 These results provide support for higher-depth tailoring in smoking-cessation programming. Lack of tailoring depth, combined with insufficient attention to relevant psychosocial and communications components, may account for the variation in outcomes from randomized trials of tailored smoking-cessation interventions.
To date, web-based smoking-cessation interventions have been studied as “black boxes”—combinations of potentially active, inactive, and possibly counterproductive components. The multiphase optimization strategy (MOST) used in this study allowed the efficient identification of the active components of these interventions. Rapidly changing interfaces, software, and capabilities of eHealth may require a more dynamic, efficient, experimental approach to intervention discovery than more traditional design strategies. This initial test of screening experiments, using fractional factorial design as a method for understanding underlying components of this emerging field, appears to be promising, and may also be relevant to other intervention modalities such as school- or worksite-based health behavior interventions, where curricula are often multi-component and modularized.
This grant was funded from National Cancer Institute grants P50 CA101451 and R01 CA101843. This project was conducted in collaboration with the Cancer Research Network (CRN). The CRN is a consortium of research organizations affiliated with nonprofit integrated healthcare delivery systems and the National Cancer Institute. Nicotine replacement therapy was provided by GlaxoSmithKline.
VJS is founder and chairman of HealthMedia, Inc., a company that makes computer-tailored behavior change programs.
No other authors reported financial disclosures.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.