|Home | About | Journals | Submit | Contact Us | Français|
We integrated and compared meta-analytic findings across diverse behavioral interventions to characterize how well they have achieved change in health behavior.
Outcomes from 62 meta-analyses of interventions for change in health behavior were quantitatively synthesized, including 1011 primary-level investigations with 599559 participants. Content coding suggested 6 behavioral domains: eating and physical activity, sexual behavior, addictive behaviors, stress management, female-specific screening and intervention behaviors, and behaviors involving use of health services.
Behavior change interventions were efficacious (mean effect sizes=0.08–0.45). Behavior change was more evident in more recent meta-analyses; in those that sampled older interventions and literatures or sampled more published articles; and those that included studies that relied on self-report, used briefer interventions, or sampled fewer, older, or female participants; and in some domains (e.g., stress management) more than others (e.g., sexual behaviors).
Interventions improved health-related behaviors; however, efficacy varied as a function of participant and intervention characteristics. This meta-synthesis provides information about the efficacy of behavioral change interventions across health domains and populations; this knowledge can inform the design and development of public health interventions and future meta-analyses of these studies.
Change in health behavior remains essential for the prevention of premature morbidity and mortality. A recent review suggests that (1) 50% of premature deaths can be attributed to modifiable risk behaviors, (2) many health problems lack curative solutions, and (3) health prevention incurs lower costs and fewer iatrogenic effects than medical solutions.1 As a consequence, public health experts have focused their efforts on health promotion to reduce negative health outcomes. Over several decades, research on change in health behavior has burgeoned, uniting many disciplines in the service of promoting health and preventing disease.2–4 Given the vastness of the literature, continued development of health promotion interventions depends on the ability of researchers to extract insight from prior efforts, a major challenge for researchers and public health officials involved in the development of behavioral health programs.
Public health experts share a strong interest in the efficacy of health promotion interventions. With the growing numbers of intervention studies evaluating the success of behavior change techniques and meta-analytic reviews summarizing these studies, evaluating the efficacy of behavioral interventions can be challenging. Meta-analyses often vary by targeted health domain (e.g., condom vs alcohol use), sample (e.g., adolescents vs adults), intervention setting (e.g., community vs clinic), and assessed outcomes (e.g., single vs multiple outcomes). Moreover, some meta-analyses restrict their samples to published studies whereas others include unpublished reports. Development of health interventions is likely to be challenged by the variations among meta-analytic studies.
We therefore used meta-analytic procedures to examine findings from prior meta-analytic reviews of health promotion literatures. This integration may permit researchers, practitioners, and policymakers to determine the success of behavioral interventions more broadly. By focusing on study, sample, and intervention characteristics reported in meta-analyses of health behaviors, this meta-synthesis may assist researchers in the design and development of behavior change interventions, thus having the potential to improve the science of health promotion; it can serve a similar function for future meta-analyses of related literatures. In our meta-synthesis, we focus on behavioral outcomes, the most definitive gauge of the success of an intervention,5 and examine factors that moderate intervention efficacy.
Meta-analyses were obtained through computerized databases and manual searches of journals. To qualify for inclusion, meta-analyses had to (1) include studies sampling participants only from the general population, rather than institutionalized samples (e.g., inmates); (2) focus on primary or secondary prevention of a behavioral change domain (i.e., appear in one of the Healthy People 20106 overarching categories); (3) review a total of 5 or more studies measuring risk reduction behavior (e.g., exercise) or behavioral tendency (e.g., weight loss); (4) include studies evaluating interventions compared with a comparison or control group (i.e., between-group studies only); (5) review psychological, educational, or behavioral interventions; and (6) provide sufficient statistical information to estimate a weighted mean effect size (d+).7
When meta-analyses lacked sufficient study details or effect sizes, meta-analysts were contacted for additional information. Meta-analyses that (1) used biomedical interventions exclusively,8 (2) focused on mental health outcomes (e.g., depression9), or (3) evaluated mass media campaigns10 were excluded. By the close of our search, 65 meta-analyses met our inclusion criteria; however, as 3 did not fit the 6 targeted categories of health behavior change,11–13 the final sample comprised 62 meta-analyses. (For details, see Technical Summary, available as a supplement to the online version of this article at http://www.ajph.org.)
Two independent raters coded each meta-analysis along several dimensions (κ=0.90). First, the behavioral change domain comprised 6 categories, including female-specific behaviors (e.g., mammography), addictions, eating and physical activity, sexual behavior, stress management, and improving health services or patients’ participation in them. To the extent that each study within the meta-analyses permitted it, we also coded mean values of (1) study characteristics (published studies, randomized controlled trial), self-report [versus objective measures], and number of authors); (2) sample characteristics (sample size, proportion women, and age in years); and (3) intervention characteristics (number of sites, brief interventions—that is, proportion with 1 hour of content or less). Numbers of sites and authors are aspects of organizational span. We also coded the date of the earliest study included in each meta-analysis. We examined linear and logarithmic trends for sample size for the studies within each meta-analysis.14 To gauge the age of the literature, the date of the first study was subtracted from the publication date of the meta-analysis.
To equate the comparisons among meta-analyses, we took the following actions. Most meta-analyses provided a single behavioral index for each study; if more than one was provided and these were highly correlated (e.g., pregnancy rates, sexual activity),15 we averaged these effects. Some syntheses analyzed intervention effects across time by separating their studies’ results into immediate and delayed measures16; in this study, we included only the first follow-up assessment. We used between-group mean effect sizes when possible, because these represent a more conservative estimate of intervention efficacy and because they were most often reported.17 Some meta-analyses included effect sizes from multiple measures of the same outcome (e.g., frequency and last-time condom use) without averaging them,18 thereby violating the meta-analytic assumption of non-independence.19 In these cases, we recalculated relevant statistics from the meta-analyses’ tables, averaging effect sizes from the same participants on the same dependent measures. Finally, when a meta-analysis reported the outcomes of more than 1 type of intervention, we kept the outcomes separate, which resulted in 7 meta-analyses with fewer than 5 studies.
To calculate effect sizes, we used the standardized mean effect size (d+). Positive effect sizes indicate health improvement (e.g., weight loss) among intervention participants whereas negative effect sizes indicate health impairment (e.g., weight gain) compared with controls. We used meta-analytic statistics with conventional weights calculated to reflect the sample sizes of each meta-analysis (i.e., each mean effect size’s inverse variance).7 Our main analyses followed fixed-effects models to examine mean tendencies and moderators of mean effect sizes; these models assume that variation appears only from sampling error.20,21 For each meta-analysis, we report I2, which gauges whether the mean effect size fits its underlying effect sizes well22; inferences from I2 are equivalent to the Q statistic,23 but values no longer depend on the number of studies included in the meta-analysis (k). I2 provides information about homogeneity on a 0 to 100 scale; high values signify more variation in study outcomes than expected by sampling error alone and are interpreted as heterogeneity (see Technical Summary).
Analyses based on coded dimensions of the meta-analyses relied on the observed values for each dimension, considering each dimension as the sole predictor of d+, with 2 exceptions: (1) sample size, for which both linear and log-linear components appeared, and (2) age, for which linear, quadratic, and cubic terms appeared. Estimation of a quadratic trend requires the linear trend be simultaneously modeled, and estimation of a cubic trend requires all 3 trends be modeled simultaneously; these values were zero-centered prior to calculation of the quadratic and cubic terms.
Meta-analyses evaluated cognitive or behavioral interventions, health or resistance education, motivational enhancement, relapse prevention, or skills training, and examined outcomes such as smoking, exercise, diet, weight loss, stress, participation in mammography screening or other medical intervention (e.g., vaccination), sexual behavior, alcohol use, and drug abuse (see Table A, available as a supplement to the online version of this article at http://www.ajph.org). Meta-analyses of individual health behavior domains included as few as 2 and as many as 120 studies (median=12) and typically sampled studies published in journals (89%). In total, 599559 participants were included in the meta-analyses. Average sample size was 9670 participants per meta-analysis, with individual studies sampling as few as 31 or as many as 4427 participants (median=281). Most interventions were brief (86%), were coauthored (mean number of authors=4.57, range=1.79–26.17), investigated using randomized controlled trials (76%), and were conducted at more than 1 site (83%). Study outcomes were predominately assessed by self-report (80%). Fifty-three percent of participants were female and 47% were male; most (74%) were adults (mean age=37 years, SD=15.81, range=10–59). Participants in sexual behavior studies were younger (mean=21.53 years) than those in studies examining other behaviors (mean=46.13 years), F(1, 40)=56.48, P<.001, d=2.34.
Across the 62 meta-analyses and their 1011 studies, intervention participants significantly adopted healthier behaviors (d+=0.21, P<.001); the amount of change varied across the meta-analyses (I2=93%, 95% confidence interval [CI]=92.4, 93.0). No meta-analysis had significantly negative outcomes, but most (89%; k=51) confirmed statistically significant health promotion effects (see Table A1, available as a supplement to the online version of this article at http://www.ajph.org). Nineteen (31%) of the 62 meta-analyses had nonsignificant homogeneity (I2) statistics; for each of these domains, the mean d+ reasonably characterized its effects. For the remaining 46 meta-analyses, outcomes varied more widely around the mean d+ than would be expected by sampling error alone, with some indicating more and others indicating less efficacy. The number of studies in a meta-analysis did not correlate with I2 (r=0.10), but meta-analyses with significant I2 values had larger samples of studies.
A series of moderator models established the extent to which variation in the magnitude of mean effect sizes related to methodological features.
Efficacy depended markedly on health domain (Table 1). The largest effects appeared for meta-analyses addressing stress management (mean d+=0.45). Meta-analyses aimed at improving health services or patients’ participation in them produced a smaller effect (d+=0.35). Health promotion meta-analyses addressing behaviors for women, addictions, and eating and physical activity had d+ of about 0.20, whereas meta-analyses of sexual behavior interventions yielded the smallest (but significantly positive) effects (d+=0.08).
Meta-analyses that included more published studies had larger effect sizes than those with fewer published studies (Table 2). Objective measures of health behavior (e.g., weight change) tended to have smaller effect sizes than meta-analyses relying on self-reports (e.g., physical activity). Meta-analyses that included more studies with randomized controlled trial designs achieved larger mean effect sizes than those that included fewer randomized controlled trials. Finally, meta-analyses of more established literatures obtained larger effects than those reviewing newly emerging interventions. Meta-analyses published more recently documented larger effects, as did those with more studies evaluating adults and females.
Meta-analyses of studies with briefer interventions recorded larger effects; indeed, those sampling only studies with long interventions exhibited an effect size that did not differ from zero. Similarly, meta-analyses were more efficacious when they included studies with fewer research sites; smaller effects were found for studies with multiple sites. Meta-analyses based on studies with more authors documented larger effects than those with fewer authors. Finally, meta-analyses sampling studies with fewer participants recorded larger effects, a negatively accelerated pattern that persisted when the linear effect of sample size was controlled. Although meta-analyses sampling more published studies yielded larger mean effect sizes, simultaneously controlling for this effect left the log-linear pattern intact (P<.001). Similarly, the negatively accelerated curve was still present in meta-analyses based entirely on journal-published studies, suggesting that it does not purely reflect publication or other reporting bias.
Behavior change remains pivotal to disease prevention. Growing numbers of primary-level interventions and meta-analyses of those trials allowed us to gauge the state of the science of health behavior change. Our analyses yielded 3 clear findings. First, behavioral interventions promote healthy behaviors and reduce risky behavior, with effects ranging in size from small to medium. Second, the efficacy of health behavior change varies across behavioral domains: eating and physical activity, sexual behavior, addictive behaviors, stress management, female-specific health behaviors, and health service use. Third, differences in target-specific effect sizes can be explained by the characteristics of the research literatures, including longer history of investigation, more published articles, use of self-report, sample composition (larger, older, more females), and intervention duration; efficacy was also higher in more recent meta-analyses.
Differences in health promotion efficacy varied by targeted behavior (Table 1). Meta-analyses of stress management interventions showed they were more successful than those including interventions for other behaviors. Because stress management often involves thoughts and emotions, change in this domain may be easier to achieve relative to behavior. Changing behavior is typically more difficult than is changing attitudes or stress appraisals, especially when the behavior becomes habitual.24 Further, change in sexual behavior, a domain for which the lowest overall efficacy was achieved, may be the most challenging, partly because it involves another person. However, more recent sexual-risk reduction interventions25–27 have documented a larger impact than the current study showed, suggesting greater impact can be achieved with sexual behaviors.
Age related to the efficacy of behavior change, with efficacy increasing from childhood and adolescence through late middle age (range=10–59 years). The analysis is cross-sectional rather than longitudinal, but observed trends corroborate trends observed by developmental psychologists.28 Younger people may take more risks due to the developmental process (e.g., exploring relationships) and because they have less experience and thus fewer negative experiences than older adults, who have learned to avoid unnecessary risks; developmental experts recognize that, as experience with health risks accumulate across the life span, perceptions of vulnerability increase and healthy behaviors are adopted to minimize aging-related changes.29 The inclusion of HIV and pregnancy prevention interventions helped to create this age-related pattern because participants targeted by such interventions tend to be younger. An investigation of this pattern within the sexual behavior meta-analyses suggests that it matches the overall age findings.
Overall, meta-analyses suggest that interventions succeed better when implemented with female participants, a pattern maintained when other aspects of the meta-analyses are statistically controlled. Because some interventions were implemented only with women (e.g., mammography screening), we examined whether gender influenced outcomes when women-only interventions were omitted. Surprisingly, the relation between gender and efficacy reversed (β=−0.40, P<.001). It is unclear why the pattern would change across the other health behavior domains (Table 1), but explanations include the possibilities that these other domains may be more relevant to men or that the interventions lacked sensitivity to key aspects of gender. Continued research on gender differences in response to health promotion interventions is encouraged.
Contrary to expectations, meta-analyses including briefer interventions had greater efficacy than those sampling longer interventions, even when other study features were controlled for (see Table A2, available as a supplement to the online version of this article at http://www.ajph.org). Intervention dose varied widely, but most meta-analyses included studies with less than 1 hour of intervention time. It is possible that intervention length is an artifact of assessment interval; that is, briefer interventions tended to have shorter follow-up periods. Because initial behavior change is easier to achieve and maintenance is more challenging, shorter interventions may appear more efficacious because of briefer follow-up intervals.30 We could not examine this possibility because duration between the intervention and the measures was not commonly reported. Thus, our findings on the stability of behavior change are less informative than optimal.
That most behavioral interventions are successful suggests that larger-scale interventions based on these models would also prove successful if implemented with fidelity. However, our finding that efficacy decreases as study sample size increases suggests that the impact of health promotion interventions is attenuated in larger group contexts.31 Meta-analyses based on interventions focusing on individuals or small groups may succeed better because participants are more likely to attend to (and benefit from) the intervention, and because facilitators can tailor content to the participant. Thus, small group sizes may influence participants’ motivation and encourage greater participation, a pattern that has been observed in other contexts32,33 and is consistent with research on client characteristics affecting psychotherapy outcomes.34 Most meta-analyses in our sample did not report on studies tailoring their interventions, making this hypothesis difficult to test. Alternatively, to ensure treatment fidelity in larger groups, researchers may forgo tailoring intervention content to the target group, potentially decreasing the intervention’s efficacy.35
Regardless of the interpretation of our group size effect, our results suggest that when behavioral interventions are disseminated, the largest effects may be seen when interventions are implemented in small-group and individualized circumstances such as individual counseling.
Several other factors influenced the efficacy of the interventions we reviewed. First, as found in other studies,36 meta-analyses based on published studies achieved larger effects than those with fewer journal reports. This pattern may reflect a reporting bias, or it may be that unpublished reports are often marked by lower methodological quality.
Meta-analyses based more on self-report measures tended to achieve larger effects than other outcomes, a pattern that may reflect social demand or positive self-appraisals to look better to others or oneself.37 One might expect that such motives are strongest in younger people, who need social acceptance. Indeed, reliance on self-reports had a positive relation to efficacy for meta-analyses with participants aged younger than 30 (β=0.43, P<.001) whereas it had a slight negative relation in those with older participants (β=−0.06, P<.01).
Studies characterized by larger organizational spans (e.g., more research sites) reported reduced efficacy. Overall, this relation was observed, but it disappeared when other dimensions were controlled. Another way to gauge organizational span is by counting the number of authors associated with study reports. Indeed, number of authors positively related to the magnitude of effect sizes, consonant with organizational research.38 We also found larger effect sizes in meta-analyses that depended on randomized controlled trials, but this effect disappeared when other features of the studies were controlled (Table A3, available as a supplement to the online version of this article at http://www.ajph.org).39
We found that meta-analyses based on older literatures had larger effects than more recent meta-analyses. Together, these findings offer optimism that newer research domains with modest behavior change improve over time as interventionists refine their techniques, as suggested by the recent sexual-risk-reduction meta-analyses highlighted in the Results section. Earlier studies may also reflect more self-initiated change, “easier” change, or change from less well-established behaviors.40 For example, in the area of smoking cessation, it is recognized that cessation trials today probably include more dependent smokers than early quitters.41,42 Finally, recent studies may have lower efficacy because they use better control groups whose participants also exhibit positive change; type of comparison group could not be evaluated because meta-analyses seldom focused on the control group. Future meta-analyses should pay greater attention to the recommendations emphasized by research checklists (e.g., CONSORT,43 TREND,44 QUOROM45); commonly examining these dimensions in meta-analytic models of the studies’ effects will likely improve understanding of health behavior change.
Our relatively dated sample of meta-analyses may appear to be a limitation, especially given the growing number of meta-analyses focused on reduction of sexual risk behaviors.27 We did not update our literature search after 2003 because (1) meta-synthesis is complex and time intensive, necessitating the retrieval of information (e.g., study-level effect sizes) not routinely reported, and (2) meta-analyses on health promotion topics are becoming more complex and their numbers are increasing at an average annual rate of 17%, prohibiting a timely cumulative systematic review of the literatures.46 To address this limitation, examination of temporal trends in the current database revealed that the inclusion of publication date did not alter our findings (Table A2, available as a supplement to the online version of this article at http://www.ajph.org). Meta-analyses have found larger effect sizes (by a rate of d=0.0073 per year), yielding the prediction that meta-analyses published in 2008 ought to have an overall d+ of 0.298 (95% CI=0.286, 0.311) compared with those published in 2003 (d+=0.262; 95% CI=0.254, 0.270). Moreover, it is unclear why meta-analyses published after 2003 would be systematically different from those published earlier. Consequently, the current results seem unlikely to change if further meta-analyses are included; future research should test this conclusion.
A related potential limitation, increasing as more meta-analyses appear, is overlapping samples of studies. An advantage of the current meta-synthesis is that samples were largely independent. Only 8 (13%) of the meta-analyses had any studies overlapping with other meta-analyses. Statistical dependencies among samples have no bearing on whether a given meta-analytic mean was significant, but they may affect different moderation results. Future meta-syntheses of meta-analyses of health promotion domains will need to take great care either to examine whether dependencies between samples affect the patterns that emerge or to maintain in their samples only the most recent meta-analyses, which presumably will have larger samples.
Finally, behavior change intervention outcomes are quite variable, especially for larger rather than smaller meta-analyses, which may have low power to reject the null hypothesis of homogeneity (see Technical Summary, available as a supplement to the online version of this article at http://www.ajph.org).23 Across the meta-analyses reviewed, most mean effect sizes represent general trends rather than precise point estimates. Random-effects models of these means produced values similar to the fixed-effects models; therefore, this limitation is unlikely to alter the pattern of the results observed in this review.
Using an unusually large data set, this investigation documents that, on the basis of the studies’ immediate assessments, behavioral interventions reduce health-damaging behaviors and facilitate health-promoting behaviors. Our analysis provides evidence that scientifically based interventions alter behaviors that confer risk for a variety of disease endpoints. These data also show that intervention success varied across literatures, but the inconsistencies could be partially explained by dimensions that we specified in advance of our investigation.
We anticipate that the findings generated by this meta-synthesis approach will stimulate inquiry into the challenges and opportunities associated with health promotion, and help improve behavior change interventions. In addition, continued refinement of methods involved in syntheses of meta-analyses can increase the scientific yield of primary-level and meta-analytic research.
This research was supported by National Institutes of Health grants R01-MH58563 to Blair T. Johnson and K02-MH01582 to Michael P. Carey and facilitated by Centers for Disease Control and Prevention grant P01-CD000237 to Leslie B. Snyder.
We thank the meta-analysts who made their data available for this study; Jessica M. LaCroix and Jennifer Ortiz, who assisted in coding the meta-analyses; and Marcella H. Boynton, Grant Farmer, Tania B. Huedo-Medina, Fritz Ifert-Miller, David B. Portnoy, Judy H. Tan, and Karin Weis, who provided helpful comments on a prior draft of this manuscript.
ContributorsB.T. Johnson conceptualized the study, analyzed the data, and led the writing of the manuscript. L.A.J. Scott-Sheldon assisted with the acquisition, content coding, analysis, and interpretation of the data. M.P. Carey assisted with the conceptualization of the study and interpreted the data. All authors provided critical revisions of the manuscript.
Human Participant Protection
No protocol approval was needed for this study.
Blair T. Johnson, Center for Health, Intervention, and Prevention, University of Connecticut, Storrs.
Lori A. J. Scott-Sheldon, Center for Health and Behavior, Syracuse University, Syracuse, NY.
Michael P. Carey, Center for Health and Behavior, Syracuse University, Syracuse, NY.