|Home | About | Journals | Submit | Contact Us | Français|
Use of computers to promote healthy behavior is increasing. To evaluate the efficacy of these computer-delivered interventions, we conducted a meta-analysis of the published literature.
Studies examining health domains related to the leading health indicators outlined in Healthy People 2010 were selected. Data from 75 randomized controlled trials, published between 1988 and 2007, with 35,685 participants and 82 separate interventions were included. All studies were coded independently by two raters for study and participant characteristics, design and methodology, and intervention content. We calculated weighted mean effect sizes for theoretically-meaningful psychosocial and behavioral outcomes; moderator analyses determined the relation between study characteristics and the magnitude of effect sizes for heterogeneous outcomes.
Compared with controls, participants who received a computer-delivered intervention improved several hypothesized antecedents of health behavior (knowledge, attitudes, intentions); intervention recipients also improved health behaviors (nutrition, tobacco use, substance use, safer sexual behavior, binge/purge behaviors) and general health maintenance. Several sample, study and intervention characteristics moderated the psychosocial and behavioral outcomes.
Computer-delivered interventions can lead to improved behavioral health outcomes at first post-intervention assessment. Interventions evaluating outcomes at extended assessment periods are needed to evaluate the longer-term efficacy of computer-delivered interventions.
Increasing recognition of the importance of behavior for health and the rapidly escalating cost of healthcare conspire to create a strong need for widespread dissemination of interventions to promote health and prevent disease. Traditional public health and clinical interventions cannot fully address this need because of resource constraints, limited access to hard-to-reach populations, and other factors. These conditions have created the impetus for innovative approaches to health education and promotion.
Computer and related information technologies offer an innovative resource for health promotion. Over 263 million people in developed countries use the World Wide Web, with more than one billion computers estimated worldwide by 2008 (Organization for Economic Co-Operation and Development 2007; Yates 2007). According to the U. S. Census Bureau (2005), 62% of U. S. households own at least one personal computer, with ownership steadily growing during the past three decades. Internet use has escalated with nearly 75% of Americans using the internet (Pew Internet and American Life Project 2006) and 57% with home access (Pew Internet and American Life Project 2005). With the growth of ownership of personal computers and 24-hour access to the World Wide Web, health professionals recognize the potential of computers to deliver health promotion messages and interventions.
Benefits of computer-delivered interventions (CDIs) include uniformity of intervention delivery, 24-hour access, and the ability to tailor an intervention to an individual (Budman et al. 2003). Although the first two benefits address the delivery of CDIs, the last feature promises to enhance intervention efficacy. Health behavior change models, such as the information, motivation, and behavioral skills model (IMB) and the transtheoretical model, suggest that tailoring intervention content enhances behavioral change (Fisher and Fisher 1992; Prochaska and DiClemente 1982). Tailoring can increase both the efficacy of an intervention as well as user satisfaction and completion of the program by allowing for a more engaging personalized experience (Ryan and Lauver 2002). Computerized tailoring allows for an individualized experience based on the user’s responses to material in the program. The combined elements of tailoring and open access to CDIs serve important functions. They allow for intervention delivery that is engaging, accessible, and faithfully implemented (Kaufmann 2007). In addition, CDIs can be delivered to individuals living in remote locations (Schopp et al. 2006) or those with physical limitations. CDIs may address antecedents of a health behavior (e.g., knowledge or attitudes) as well as directly addressing the behavior itself. If efficacious, CDIs may also be cost-saving.
Because CDIs vary on many dimensions (e.g., use of tailoring, dose, interactivity, theoretical basis, target behavior and population), a systematic review to identify the components of effective CDIs and their relative impact on behaviors is needed. Previous reviews of CDIs have either been descriptive (Evers et al. 2003) or limited to a single health behavior (e.g., smoking; Walters et al. 2006). A meta-analytic review comparing web-based versus print applications of health interventions showed positive, though widely varying, benefits (Wantland et al. 2004). These reviews suggest that web-based CDIs can be efficacious (Wofford et al. 2005).
This meta-analytic review goes beyond the previous reviews by comparing CDIs across health domains and providing a more comprehensive picture of their benefits. We examine the effects of CDIs on a variety of health constructs and behaviors. Psychosocial, behavioral, and educational CDIs were located through a systematic review of the literature. We examine the extent to which these interventions affect (a) knowledge, (b) hypothesized determinants or mediators of health-behavior change (e.g., attitudes, intentions, norms, self-efficacy), (c) behavioral outcomes (physical activity, nutrition, tobacco use, substance use, safer sexual behavior, and overall health-maintenance), and (d) objective behavioral outcomes (weight loss, diabetes control, binging/purging); we also examine whether sample and intervention characteristics moderate intervention efficacy.
Published (or in press) manuscripts were retrieved through: (a) electronic reference databases (PsycINFO, PubMED, ERIC, CRISP, and the Cochrane Library); (b) reference sections of relevant review or published studies; (c) examining contents of relevant journals (e.g., Journal of the American Medical Informatics Association, American Journal of Public Health), and (d) sending manuscript requests via e-mail listservs (e.g., Games for Health) and to relevant authors.
Studies were included if they (a) examined health domains related to one of the leading health indicators delineated in Healthy People 2010 (U.S. Department of Health and Human Services 2007); (b) implemented a CDI; (c) used a randomized controlled trial (RCT) design with a control condition; (d) assessed hypothesized antecedents of health-behavior change (e.g., knowledge, self-efficacy), the health behavior itself (e.g., physical activity), or an objective health outcomes (e.g., weight loss); and (e) provided sufficient information to calculate between-group effect size estimates. Studies were retained if the only face-to-face contact consisted of an initial basic computer orientation session with no intervention content; and were excluded if the intervention occurred in a group format; or computers were used only to tailor information presented in a non-computerized format (e.g., computer-tailored mailed personalized feedback; Dijkstra et al. 1998). Of the 27 study authors contacted for additional statistical information, 60% responded with the necessary information; the remaining cases were estimated from information in the article when possible.
Studies that fulfilled these criteria available as of May 31, 2007 were included. When several publications provided information about a single study, effect sizes were calculated separately for each measurement occasion. However, due to the small number of studies with multiple follow-ups per health domain, we focused on the first measurement occasion only. When more than one control or comparison condition was used (e.g., standard of care and wait-list), the control condition with the least contact (e.g., wait-list) was used as the comparison condition for ease of interpretation. Using these criteria, 75 RCTs sampling 35,685 participants with a retention rate of 86% (SD = 28%) at first follow-up qualified for the meta-analysis (Figure 1). More details on the search strategies are available upon request from the authors.
For each study, effect size estimates were calculated from the information provided in the report(s). Effect sizes were calculated for the psychological outcomes of health prevention (a) attitudes, (b) intentions, (c) social norms, and (d) self-efficacy; the behavioral outcomes of (a) physical activity, (b) nutrition, (c) tobacco use, (d) substance use (i.e., alcohol and/or drugs), (e) safer sexual behavior (e.g., increased condom use,), (f) reducing binging/purging, and (g) general health maintenance (e.g., adherence to medical advice); and the objective behavioral outcomes of (a) weight loss, (b) diabetes-related outcomes (e.g., blood pressure, glucose, lipids), and (c) weight gain/maintenance (for studies examining eating disorders). Psychosocial outcomes were assessed via self-report, behavioral outcomes were assessed via self-report and/or objective measures.
Effect sizes (d) were calculated as the mean differences between the intervention and control group divided by the pooled standard deviation (SD; Cohen 1988). If the pooled SD was unavailable or could not be derived from the reported statistics, the denominator was replaced with another form of SD (e.g., pretest SD; Lipsey and Wilson 2001). When means and SDs were unavailable, other statistical information (e.g., t- or F-values) was used to calculate d (Johnson and Eagly 2000; Lipsey and Wilson 2001). If a study reported dichotomous outcomes (e.g., frequencies), we calculated an odds ratio and transformed it to d using the Cox transformation (Sánchez-Meca et al. 2003). If statistical information was not available, and could not be obtained from the authors, and the study reported no significant between-group differences, we estimated that effect size to be zero1. In calculating d, we controlled for baseline differences between the intervention and control condition(s) when pre-intervention measure were available (Becker 1988). A positive effect size indicated greater health benefit (e.g., decreased tobacco use) for participants in the intervention vs. control condition; effect sizes were corrected for sample size bias (Hedges 1981). DSTAT 2.0 (Johnson and Wood 2006) was used to calculate effect sizes.
Multiple effect sizes from individual studies resulted when they reported more than one outcome, multiple intervention conditions, or when outcomes were separated by sample characteristics (e.g., gender). When a study contained multiple measures of the same outcome (e.g., daily alcohol use and peak blood alcohol concentration), the effect sizes were averaged. Effect sizes calculated for each intervention and by sample (e.g., when studies separated outcomes by gender) were treated as separate studies to retain independence. Of the 75 RCTs included in the meta-analysis, 82 separate interventions (k) were evaluated.
Two researchers independently coded study information (e.g., theory used), sample characteristics (e.g., ethnicity, gender), risk characteristics (e.g., current alcohol use), design and measurement specifics (e.g., number of follow-ups), and content of intervention and control condition(s) (e.g., number of sessions). Twenty studies were randomly selected to examine reliability. For the categorical dimensions, raters agreed on 43% to 100% of the judgments (mean Cohen’s κ = .61). Reliability for the continuous variables was calculated using the intraclass correlation coefficient (); ranged from 0.18 to 1.00, with an average = 0.90 across categories. Coding disagreements were resolved through discussion.
Weighted mean effect sizes, d+s, were calculated using random-effects procedures (Lipsey and Wilson 2001), such that individual studies’ effect sizes were weighted by the inverse of their random-effects variance. The homogeneity statistic, , was computed to determine whether each set of d+s shared a common effect size. To further assess heterogeneity, the I2 index and its 95% confidence intervals were calculated to assess the inconsistencies in a set of effect sizes (Higgins and Thompson 2002; Huedo-Medina et al. 2006). If the 95% confidence interval around the I2 index includes a zero, the set of effect sizes are considered homogeneous. If was significant and the I2 index indicated medium to high variability, the relation between study characteristics and the magnitude of the effects were examined using a weighted least squares regression analyses with weights equivalent to the inverse of the fixed-effects variance. Analyses were conducted in Stata 10.0 (StataCorp 2007) using macros provided by Lipsey and Wilson (2001).
A detailed summary table of the 75 RCTs included in the meta-analysis is available from the authors (http://digitalcommons.uconn.edu/chip_docs/23). Most studies (73%) used theory to guide their intervention design with 20 studies (36%) using more than one perspective. Studies focused on: overweight and obesity concerns (nutrition, weight, and/or diabetes; 37%), substance use (alcohol and/or other drugs; 23%), tobacco use (11%), sexual behavior (HIV/STD or pregnancy prevention; 8%), physical activity (5%), eating disorders (4%), or other health concerns (e.g., sun exposure; 5%). Several studies (7%) focused on multiple health behaviors (e.g., alcohol, drugs, and tobacco use). All studies randomly assigned participants or groups to conditions, and evaluated participants at both pre- and post-test. The median number of follow-ups was one (M = 1.63; SD = 0.87; range = 1 to 5). The initial post-intervention assessment (the focus of this review) occurred an average of 7.46 weeks (SD = 19.85; range = 0 to 104, Mdn = 0.14) after the intervention.
Participants were predominately female (66%; k = 73), White (69%; k = 49), and adults (M age = 31.05, SD = 14.56; range = 8.28 to 63.00; k = 64). Most of the samples were located within the U. S. (73%) and Europe (15%) and were drawn primarily from a community/clinical (59%) or school/university (38%) environment. Of the 24 studies reporting participant computer skills, 71% of participants had some prior computer experience. Potential risk characteristics of the sample included: alcohol use (k = 16), marijuana use (k = 4,), drug use other than alcohol/marijuana (k = 8), current drug/alcohol treatment (k = 9), and tobacco use (k= 14). For the studies reporting participants’ sexual activity (k= 8) and pregnancy (k= 9), 76% were sexually active and 13% were currently pregnant. Only one study reported sampling participants diagnosed with a severe mental disorder.
Participants were typically recruited via the community (e.g., flyers, community centers, or internet advertisements; 39%), school/university (32%), or clinical contact (26%); two studies recruited participants from drug treatment or prison. Interventions were usually conducted in school/university (21%) or clinic/hospital (21%) settings; some interventions were used at home (9%), community centers (5%), retail settings (4%), worksites (4%), or prison (1%). Most of the studies delivered the intervention using a single computer-based approach (e.g., internet, CD-ROM; 92%) only; some studies (7%) used a computer program and electronic peer support, chat rooms, or individual counseling.
The 75 studies examined 82 separate interventions (i.e., some studies evaluated more than one condition or were divided by group). The single-approach interventions consisted of a median of three (range = 1 to 36) computer-delivered sessions of 21 minutes (range = 4 to 120) each; the multiple approach interventions consisted of a median of 11 sessions (range = 8 to 58) of individual computer-delivered sessions of 45 minutes (range = 15 to 60; k = 6), a median of 42 sessions (range = 3 to 80) of electronic peer support of 6 minutes each (range = 1 to 10; k = 2), and a median of 8 sessions (range = 8 to 24) of electronic chat rooms of 60 minutes each (range = 15 to 60; k = 3). Many interventions (65%) were tailored to the individual (e.g., by readiness-to-change); 18% of the interventions were tailored to the group (e.g., gender, ethnicity). All interventions included health information (e.g., diet/weight, HIV, alcohol), 88% included a motivational component (52% personalized risk assessment, 45% cost/benefits, 37% individual concerns, 36% social norms, 18% sample-specific concerns, and 14% perceived vulnerability), and 89% included a skills training component (71% passive skills training—demonstration or information, 62% planning, 30% self-efficacy, 15% active skills training such as role-playing preventative behaviors). Supplemental materials (e.g., coupons, computer-related equipment) were provided in 15% of the interventions.
The most typical comparison condition (51%), was an assessment-only control. Active comparison conditions (N = 40) (e.g., education-only, brief or altered form of intervention) had a median of 2 sessions of 19 minutes each. Active comparisons consisted of standard education only (26%), irrelevant content that was time matched (10%), brief versions (mailed, printed, or face-to-face) of the intervention (10%), or time-matched relevant content (4%). Supplemental materials (e.g., manuals, brochures) were provided in 15% of the active comparisons conditions.
Because there were few studies (k ≤ 4) that examined knowledge, attitudes, intentions, social norms, and self-efficacy for each health domain, effect sizes are represented as a combination of the various health domains. CDIs improved health-related knowledge, attitudes, and behavioral intentions but did not improve social norms or self-efficacy relative to controls (See Table 1). All of the effects lacked homogeneity, and I2 were significant, except for behavioral intentions. Moderator tests investigated the variability in the effect sizes for knowledge, health-prevention attitudes, social norms, and self-efficacy.
Univariate regression analyses were conducted to examine potential moderators of knowledge, attitudes, social norms, and self-efficacy. Sample (e.g., proportion women, age), study (e.g., type of control or comparison condition), and intervention (e.g., delivery, dose, content) characteristics were individually examined (see Table 2); the small numbers of cases precluded multiple moderator tests.
CDIs were more successful at improving knowledge when interventions sampled more women, younger participants, and people with prior computer experience, the studies did not use an active control condition, and the interventions targeted diet and/or weight, focused on a condition other than diabetes, and were delivered via the internet.
CDIs improved attitudes more when they sampled younger participants, did not use an active comparison condition, and the interventions did not focus on diet and/or weight, were delivered via the internet and not via CD-ROM or on-site, did not include a motivational component, and were presented in greater doses.
CDIs were more successful at improving social norms if they sampled fewer women, did not use an active comparison condition, and the interventions did not focus on diet and/or weight, did not include a motivational or behavioral skills component, presented the content in greater doses, and conducted assessments at later weeks after the intervention.
CDIs were more successful at increasing participants’ self-efficacy if they sampled more women or younger participants and did not deliver the intervention via CD-ROM.
CDIs were successful at improving nutrition (d+ = 0.15, 95% CI 0.07, 0.22; Figure 2), reducing tobacco use (d+ = 0.33, 95% CI 0.08, 0.59; Figure 3), reducing substance use (d+ = 0.24, 95% CI 0.04, 0.43; Figure 4), increasing safer sexual behavior (d+ = 0.35, 95% CI 0.10, 0.60; Figure 5), reducing binge/purging behaviors (d+ = 0.19, 95% CI 0.04, 0.33; Figure 6), and promoting general health maintenance (d+ = 0.18, 95% CI 0.05, 0.30; Figure 7). No improvements were observed for physical activity, weight loss, diabetes control, or weight gain/maintenance. Effect sizes were homogeneous within each domain except for tobacco use, (10) = 372.63, p < .001, and substance use, (10) = 52.90, p < .001; therefore, moderator analyses were conducted only for tobacco and substance use.
Both tobacco and substance use interventions were more successful when investigators sampled more users and were delivered in greater doses (Table 3). Although tobacco interventions showed greater improvements when assessment occurred at a longer interval after the intervention, substance use showed improvement with a shorter interval. Tobacco interventions also were more successful when investigators sampled younger participants, did not use an active comparison condition, when presented via CD-ROM and on-site but not via the internet, and did not include a motivational component. No other sample, study, or intervention characteristics moderated the impact of CDIs on tobacco or substance use.
The current meta-analysis integrates the results of 75 randomized clinical trials that have included more than 35,000 participants and evaluated 82 separate computer-delivered, health promotion interventions. Meta-analysis of these data supports the following conclusions.
First, CDIs can help individuals to make improvements in a variety of health behaviors, including nutrition; tobacco and substance use; sexual behavior; binging/purging episodes; and general health maintenance. The evidence for CDI-stimulated behavior change across six, widely-varying health behaviors is somewhat surprising as one might hypothesize that some behaviors (e.g., substance use) might be more difficult to change. The range of health behaviors targeted by CDIs has been broad, and the sampling suggests the full range of prevention applications (i.e., universal to selected to indicated; Gordon, 1983). It is unlikely that CDIs would be sufficient for a long-standing clinical disorder, as suggested by the lack of improvement for weight loss and diabetes management. Notwithstanding this caveat, the CDIs included in this meta-analysis did help to improve substance use and binging/purging behaviors that are often resistant to change even with therapist-facilitated interventions. There is precedent for self-initiated change for such health habits as smoking, when a modest but influential experience provides the proverbial “straw that broke the camel’s back,” (e.g., Carey et al. 1993). We hypothesize that CDIs may help some participants to organize previously latent motivation and behavioral skills at a critical moment, leading to improved health behaviors. That the CDIs were relatively brief in duration seems consistent with this hypothesis.
Second, as expected the CDIs led to short-term changes in several theoretically-relevant antecedent conditions, which are hypothesized to precede behavior change (see Bandura 1997; Fishbein and Ajzen 1975; Fisher and Fisher 1992). Interventions may prompt change on these elements, which in turn influence behavior. Changes in these psychological antecedents are consistent with current thinking about the determinants of health-behavior change.
Third, the magnitude of the antecedent condition and actual behavior change tends to be small to medium by meta-analytic conventions; nonetheless, these changes compare favorably to other commonly-implemented public health and medical interventions. From a public health perspective, even small changes are meaningful at the population level (Rose 1992).
Fourth, several sample, study, and intervention features explained variability in knowledge, psychological, and behavioral outcomes:
Although intriguing, these findings are based on a relatively small set of studies so continued research is needed to clarify the effects of factors such as age, race/ethnicity, delivery mode, use of motivational components, and dose.
Fifth, a virtue of CDIs is the ability to tailors intervention to individuals. Theories of health behavior change suggest tailoring facilitates behavioral change (Fisher and Fisher 1992; Prochaska and DiClemente 1982) and in non-CDIs appears to have value (e.g., Noar et al. 2007; Skinner et al. 1999). Nonetheless, our meta-analytic review found no support for tailoring activities, whether examined at the individual or the group level. However, we could not fully evaluate the efficacy of tailoring because there was no variability for tailoring in these studies. Future research should more thoroughly evaluate the role of tailoring CDIs on health-behavior change; with appropriate research designs, we predict that tailored interventions will prove more efficacious than non-tailored, “one size fits all” interventions.
Finally, extant data do not address the durability of behavior change: Health behaviors are likely to recur (some would say “relapse”) when environmental conditions change (e.g., during increased stress), and the influence of the intervention wanes. CDIs often exhibited uniform impact on outcomes despite the variability in follow-up duration, suggesting that CDI impact can be durable. Research should more thoroughly investigate the durability of the changes.
As with any meta-analytic investigation, these results should be interpreted mindful of the limitations of both the research literature and our methods. First, there have been relatively few RCTs on this topic; for example, although there have been hundreds of face-to-face intervention trials for sexual risk reduction (see Smoak et al. 2006), we located only four CDI studies that examining it that met our inclusion criteria. The limited number of studies constrained analyses that might identify moderators of intervention success (or failure), a necessary step for the refinement of CDIs. Second, identification of relevant studies may have been hindered by authors’ use of keywords, publication source, and researchers’ non-responses to listservs or our inquiries (see Matt and Cook 1994). Third, we could not parse the literature based on the difficulty of the behavior being changed. As alluded earlier, it is reasonable to expect that some participants (e.g., an addicted smoker) or health problems (e.g., weight loss) might respond less well to CDIs; however, the literature is not sufficiently mature to address this issue. Fourth, we focused on immediate post-intervention efficacy because many studies did not provide data from multiple follow-up assessments. Future research will be most helpful if the follow-up periods extended to one year or longer, to address the stability of health promotion gains. Finally, our analyses also focused exclusively on “stand alone” CDIs; research might explore whether the combination of CDIs with a face-to-face interaction, or other human engagement, might optimize the health promoting benefits.
Computer-delivered interventions can lead to immediate post-intervention improvements in health-related knowledge, attitudes, and intentions as well as modifying health behaviors such as dietary intake, tobacco use, substance use, safer sexual behavior, binge/purging behaviors, and general health maintenance. CDIs do not provide benefits in all contexts; the evidence does not support the use of CDIs to improve physical activity, weight loss, or diabetes self-management. Nonetheless, there is sufficient evidence to continue to investigate the benefits and limits of CDIs, to explore patient- and intervention-characteristics that facilitate health behavior change, and to determine the long-term effects of CDIs.
This work was facilitated by National Institute of Mental Health Grant R01-MH58563 to Blair T. Johnson (PI). We thank Nicole Clark, Jessica M. LaCroix, Jennifer Ortiz, and Anna Switaj for their assistance with this project and the study authors who provided additional intervention or statistical information.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1Although Lipsey and Wilson (2001) and Pigott (1994) discussed a common meta-analytic practice of using conservative methods to estimate an effect size, such as imputing a zero when a study reports no differences between groups (e.g., treatment vs. control) on a particular dependent variable, both suggest that this strategy may lead to biased results but neither suggested an appropriate solution. To retain all possible studies in the current meta-analysis, for 20 (from 14 studies) cases out of the 179 effect sizes, we imputed a zero. We systematically examined each dependent variable of interest with imputed and complete effect sizes only (i.e., excluding all imputed effect sizes); the pattern of results (weighted mean effect sizes and homogeneity tests) remained the same.