PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Prev Med. Author manuscript; available in PMC Mar 1, 2009.
Published in final edited form as:
PMCID: PMC2288581
NIHMSID: NIHMS43500
Methods of Quantifying Change in Multiple Risk Factor Interventions
Judith J. Prochaska, PhD, MPH,1 Wayne F. Velicer, PhD,2 Claudio R. Nigg, PhD,3 and James O. Prochaska, PhD2
1 Department of Psychiatry, University of California, San Francisco
2 Cancer Prevention Research Center, University of Rhode Island
3 Department of Public Health Sciences, University of Hawaii
Corresponding Author: Judith J. Prochaska, PhD, MPH, University of California, San Francisco, 401 Parnassus Ave – TRC 0984, San Francisco, CA 94143-0984. Ph: (415) 476-7695. Fax: (415) 476-7719. Email: JProchaska/at/ucsf.edu
Objective
Risky behaviors such as smoking, alcohol abuse, physical inactivity, and poor diet are detrimental to health, costly, and often co-occur. Greater efforts are being targeted at changing multiple risk behaviors to more comprehensively address the health needs of individuals and populations. With increased interest in multiple risk factor interventions, the field will need ways to conceptualize the issue of overall behavior change.
Method
Analyzing data from over 8,000 participants in four multibehavioral interventions, we present five different methods for quantifying and reporting changes in multiple risk behaviors.
Results
The methods are: (a) the traditional approach of reporting changes in individual risk behaviors; (b) creating a combined statistical index of overall behavior change, standardizing scores across behaviors on different metrics; (c) using a behavioral index; (d) calculating an overall impact factor; and (e) using overarching outcome measures such as quality of life, related biometrics, or cost outcomes. We discuss the methods’ interpretations, strengths, and limitations.
Conclusion
Given the lack of consensus in the field on how to examine change in multiple risk behaviors, we recommend researchers employ and compare multiple methods in their publications. A dialogue is needed to work towards developing a consensus for optimal ways of conceptualizing and reporting changes in multibehavioral interventions.
Keywords: multiple risk factors, multibehavioral change, outcome measurement, intervention, impact
Risky behaviors such as smoking, alcohol abuse, physical inactivity, and poor diet are detrimental to health and often co-occur. The majority of adults meet criteria for two or more risk behaviors (Fine et al., 2004; Poortinga, 2007). Tobacco users, in particular, tend to have poor behavioral profiles, with about 92% of smokers having at least one additional risk (Fine et al., 2004; Pronk et al., 2004). Multiple risk factors also appear more prevalent among men, younger adults, singles, those of lower social class, the economically inactive, the less educated, and chronically ill (Poortinga, 2007; Pronk et al., 2004).
The major causes of morbidity and premature mortality in the United States– heart disease, cancer, stroke, and diabetes–are influenced by multiple health behaviors. Over 80% of cardiovascular events are believed due to the lifestyle factors of smoking, inactivity, poor diet, and alcohol use (Stampfer et al., 2000). When risk behaviors cluster, the negative effects on health are even greater. Having a combination of three risk factors rather than one, for example, more than doubles one’s risk of myocardial infarction (American Hearth Association, 1997). Excess risks also lead to excess medical costs, with the effect believed to be multiplied rather than additive (Edington et al., 1997; Shinton, 1997). Longitudinal data indicate that effectively treating two behaviors reduces health care costs by about $2000 per year (Edington, 2001).
To more comprehensively address the health needs of individuals and populations, greater efforts are being targeted at changing multiple risk behaviors. More effective reductions in morbidity and mortality may be seen if changes in multiple risk factors can occur together. Additionally, changing multiple health behaviors should result in more favorable benefits measured in terms of quality of life outcomes and healthcare costs and utilization. To date, however, the magnitude of the benefits from multiple risk factor interventions, or multibehavioral interventions, remains largely unknown.
To evaluate the effectiveness of interventions targeting multiple behaviors, methods are needed to quantify and report changes across several behaviors. Use of an outcome metric or a standardized assessment to describe change across behaviors would allow those analyzing and considering adoption of health promotion programs -- such as policymakers, organizational and government decision makers, healthcare practitioners and systems, researchers, and individuals -- to decide how to allocate their finite resources (time, money and psychological energy) to maximize health improvement (Woolf, 1999). Such an index would ideally allow for comparison of outcomes from multibehavioral to single behavior interventions as well as allowing for evaluation of the overall impact on health behavior change.
In this special journal issue, the focus is on interventions designed to change multiple risk behaviors within individuals or populations. Briefly, to distinguish the two types of designs:
  • Multibehavioral interventions within individuals aim to deliver all interventions to all participants with participants typically selected to be at-risk for all targeted behaviors.
  • Multibehavioral interventions within populations provide a program of interventions to an entire community, with individuals in that community receiving attention only for the behaviors for which they are identified as at risk. That is, multiple risks are addressed within the community but not all community members receive intervention on all behaviors.
The methods discussed in the current paper are relevant to both designs.
With increased interest in multibehavioral interventions, the field will need ways to conceptualize the issue of overall behavior change. The current paper reviews five different methods for quantifying and reporting change in multibehavioral interventions. We discuss their interpretations, strengths, and limitations. For the less well known approaches, we include numeric examples to illustrate the methodology and interpretation.
The traditional approach to reporting changes in multibehavioral interventions simply analyzes change in each behavior separately. The benefits of this approach are that it allows for the comparison of multibehavioral interventions with single behavior interventions, is easily understood by decision makers, has a long history in the field, is accepted by reviewers, and published scoring protocols exist. The limitations of this approach are that it increases the chances of a type I error due to multiple significance testing or reduced power if alpha-adjustments are made; it may lead to difficulty in interpretation when comparing conditions if changes are significant for some behaviors and not others; and it does not provide an indication of the overall effect of the intervention on behavior change.
Use of this approach in the literature is widespread, so we have not included a numeric example here. All 39 trials in Ebrahim and colleagues’ (2006) Cochrane Review of multibehavioral interventions reported outcomes for each risk factor individually. Despite an estimated 20% net reduction in smoking prevalence, the review concluded little evidence in support of multibehavioral interventions. Changes in dietary and physical activity behaviors were not reviewed. The individual studies targeted different sets of risk behaviors, employed a variety of outcome measures, and reported significant changes in some but not all targeted behaviors (e.g., Emmons et al., 1994; Sorensen et al., 1996). Given the diversity of outcomes and reported effects, it is difficult to synthesize the findings or to conceptualize the overall impact on multiple behavioral risks. The remaining four methods provide potential alternatives to the traditional approach of reporting change in each targeted behavior individually.
A combined index of overall behavior change can be useful for quantifying the overall effect of a multibehavioral intervention. If the behavioral measures to be combined are on different scales -- for example, minutes of physical activity and servings of fruits and vegetables -- a statistical transformation will be necessary. There are several options. Standardized change scores can be created by subtracting baseline scores from the follow-up scores and then dividing by the standard deviation of the difference (i.e., z-score). The scores can then be summed into a combined behavioral index, which indicates the amount of increase or decrease in the combined behaviors from baseline to follow-up.
Another approach uses the standardized residuals from linear regressions of follow-up scores on baseline measures to provide a simple change score adjusted for baseline variance. Residualized change scores are referred to as “base-free” measures of change (Tucker et al., 1966) and are viewed as superior to simple pretest – posttest difference scores (Veldman and Brophy, 1974). Residualized change scores are a standard statistical technique and have been used in examining predictors of change in children’s physical activity (Sallis et al., 1999).
Benefits of combined change scores are: the use of a continuous outcome variable, which should provide greater statistical power; the focus on change; and the assignment of equal weights to each behavior (some may see this as a disadvantage). If data are obtained over more than two time points, an extension of these modeling techniques would be necessary. The limitations of the approach are that it may be difficult to interpret for media and policymakers; may lack meaning in terms of health benefits; treats each behavior equally; and is not widely used and documented. Additionally, residualized change scores are not suited to address the issue of whether there is a significant change across time overall (i.e., ignoring groups) as the mean of a residualized change score is zero.
Figure 1 shows an example using standardized residualized change scores to create a combined index of overall change in physical activity and fruit and vegetable consumption. The data are from a randomized controlled trial (N=138 middle school students) testing the efficacy of a computer-delivered intervention targeting physical activity only versus in combination with fruit and vegetable consumption (Prochaska and Sallis, 2004). The interpretation of the analysis of the combined index is that the physical activity only intervention resulted in greater overall change compared to the combined physical activity and nutrition intervention and the control condition. These findings were not reported in the main outcome paper because the manuscript reviewers were not comfortable with the analytic method used to create an overall index of behavior change. Language is needed in the field to discuss comprehensive changes in multibehavioral interventions.
Figure 1
Figure 1
Example of the use of a combined index for evaluating change in physical activity and fruits and vegetables using standardized residualized change scores. Here, the treatment condition targeting physical activity alone (PA) resulted in greater overall (more ...)
A third approach is to create a multiple behavior change index reflecting the number of behaviors for which an individual has reached criterion and is no longer at risk. Indices require consensus on the “success” criteria for each of the targeted behaviors (e.g., 5 servings per day of fruits and vegetables; 7 day abstinence for tobacco use, 30 minutes per day of physical activity most days of the week).Examples of multibehavioral indices include: the Framingham Heart Study risk score; the Cooper Clinic mortality risk index; cancer risk indices; dietary quality indices; and an index of early problem behaviors (Janssen et al., 2005; McGue et al., 2006; Multiple Risk Factor Intervention Trial Research Group, 1977; Patterson et al., 1994). It is not clear if weighting or straight summative indices are optimal.
Benefits of this approach are that it can be used to directly address Healthy People 2010 goals for the nation, is fairly easy to interpret (though it may lack meaning in terms of health benefits gained for policymakers), and allows for comparison to single behavior interventions using consensus definitions. Limitations of this approach are that it can be difficult to decide on the criteria for success for some behaviors; dichotomizing outcomes usually decreases sensitivity and removes the scale of continuous measurement; and credit is gained only for reaching criteria. That is, this approach does not acknowledge progress along the behavior change continuum.
A statistical approach to creating a behavioral index is multivariate analysis. Multivariate analysis of variance (MANOVA) treats the various behavioral risk factors as multiple dependent variables and yields a weighted linear composite that maximizes between group differences. The MANOVA approach is appropriate only when all of the variables are normally distributed. Models of bivariate outcomes that differ in type have been developed, including jointly modeling of a longitudinal process and time to an event (Guo and Carlin, 2004), dichotomous and continuous outcomes (Gueorguieva and Agresti, 2001), and multivariate longitudinal outcomes (Fieuws and Verbeke, 2001). A limitation is that these approaches do not allow for direct comparability across studies since the statistical weights are data-derived and will therefore differ across studies.
To date, impact -- defined as intervention efficacy (E) times participation (P) or I = E × P (Velicer and Prochaska, 1999) -- has been measured only for the treatment of a single behavior such as smoking. For multibehavioral interventions, impact calculations will need to take into account the number of behaviors treated effectively. A revised formula for impact may be considered as intervention efficacy times participation summed over the multiple behavioral targets, I = Σ# of behaviors(n) (En × Pn) (Prochaska et al., 2006). In multibehavioral interventions within individuals, P would be the study recruitment rate. In multibehavioral interventions within populations, P would be the proportion of at risk individuals participating in the intervention for each behavior. For both designs, E would be the estimate of efficacy for each behavior. Use of a common metric, such as the percent no longer at risk (i.e., the percent reaching action or maintenance), would allow for summation across behaviors. This revised impact equation provides a measure for assessing the impact of interventions in individuals and populations with multiple behavioral risks.
The major strength of this approach is that it provides a fuller indication of the overall impact of an intervention on participants and populations through its incorporation of reach and efficacy. Furthermore, this metric speaks directly to policy and decision makers. The primary limitation of the approach is that there is little guidance in the field as to what constitutes a large impact. Impact values for single risk behaviors can range from 0 to 1. For multiple risk behaviors, impact can exceed 1, so that interventions targeting more behaviors have the potential for greater impact. An additional limitation is that, in multibehavioral interventions for populations, multiplying efficacy by the proportion of the population at risk results in low prevalent behaviors contributing less to the impact score, even if they are major risk factors, as is the case with tobacco use.
To illustrate this approach, Table 1 provides actual data from two recent multibehavioral trials with 5,407 primary care patients and 2,460 parents of high school students that targeted smoking, dietary fat, and sun exposure (Prochaska et al., 2004; 2005). The studies employed the same intervention strategies and used common measures at the same assessment time points, allowing for comparison across trials. The trials were population-based multibehavioral interventions, so not all participants were at risk for all behaviors. Significant treatment differences were reported previously at 12 and 24 months follow up for each of the three behaviors individually (Prochaska et al., 2004; 2005). Table 1 summarizes the proportion at risk and treatment efficacy results for each of the three behaviors. We calculated the overall impact on the sample participants by summing the impact values for each of the three behaviors. We also calculated the impact values for the population, adjusting for the studies’ recruitment rates. In both studies, the dietary intervention component contributed the greatest amount of impact. The overall impact on the target populations for both trials was moderate, with larger impact achieved in the parents (.45) versus the primary care (.30) study due to a higher recruitment rate, a greater prevalence of risk factors at baseline in the sample, and greater efficacy effects for dietary and sun outcomes.
Table 1
Table 1
Quantification of overall impact, across multiple risk behaviors, in three multibehavioral population trials
Measures such as quality of life, morbidity or mortality, biometrics (obesity, cholesterol, lung functioning), or cost outcomes (health care savings, worker productivity) may be used to quantify overall changes in multiple risks due to an intervention. The Centers for Disease Control and Prevention (CDC) recommend health-related quality of life as an outcome (http://www.cdc.gov/hrqol/index.htm). The measure is brief and included in the CDC’s behavioral risk factor surveillance survey allowing investigators to compare study data with national estimates.
The advantages of this approach are that the outcomes closely match policymakers’ interests and are easy to interpret and discuss in terms of health and other benefits. The primary limitation is the overarching measures may be less sensitive to change or require a longer time course to achieve an effect. If studies are powered to these overarching measures, larger samples will likely be necessary.
There are few examples in the literature of multibehavioral interventions reporting significant effects on overarching measures. Ebrahim and colleagues’ (2006) review of multibehavioral interventions, for example, concluded no impact on mortality. A recent exemplary study is Toobert and colleagues’ (2007) randomized, comprehensive lifestyle intervention trial with 279 postmenopausal women with type 2 diabetes that targeted smoking, diet, physical activity, stress management, and social support. The intervention had significant effects on all targeted behaviors, with the exception of smoking due to small numbers, and achieved significant long term effects (12 and 24 months) on a measure of quality of life.
In Figure 2 we present cross-sectional data from a study with 322 clinically depressed smokers (Prochaska et al., 2005). Severity of depressive symptoms, tobacco use, and opiate use contributed individually and collectively to lower health functioning scores on the Medical Outcomes Study Short Form (SF-36) (Ware et al., 1997).
Figure 2
Figure 2
These cross-sectional data are from a study with 322 clinically depressed smokers. Examining risk-related subgroups based on depression scores (BDI-II ≥20), cigarettes per day (≥ 1 pack), and opiate use (yes, no), the severity of risk (more ...)
Interventions that target multiple risk behaviors have the potential to offer greater health benefits, more adequately address participants’ behavioral profiles, maximize health promotion opportunities, and reduce health care costs. To make the case for multibehavioral interventions, however, we will need ways of communicating the complete behavior change effects of these more comprehensive interventions. At the forefront of defining the next generation of health behavior change research, it is of utmost importance that the field is set up for success with fair, meaningful, interpretable, and user-friendly approaches.
We have described several options for conceptualizing change in multibehavioral interventions. Given the lack of consensus in the field on how to examine change in multiple risk behaviors, we recommend researchers employ and compare multiple methods in their publications. A dialogue is needed to work towards developing a consensus for optimal ways of conceptualizing and reporting changes in multibehavioral interventions. Importantly, approaches to conceptualize change in multibehavioral initiatives will need to consider the perspectives of researchers, practitioners, policymakers, individuals and their communities.
Supplementary Material
Acknowledgments
Studies funded by NIH Grants CA 50087, CA 27821, CA85807, CA109941, DA 018691, and DA09253; TRDRP Grant #13KT-0152; and the Hawaii Medical Service Association, an Independent Licensee of the Blue Cross and Blue Shield Association.
Footnotes
Previous Presentation: The study was presented at the Annual Meeting of the Society for Behavioral Medicine in Washington, DC on March 23, 2007.
Precis: Using data from over 8,000 participants in 4 trials, we present five different methods for reporting outcomes from multibehavioral interventions. Strengths and limitations are discussed.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
  • American Heart Association. Heart and Stroke Statistical Update. 1997.
  • Ebrahim S, Beswick A, Burke M, Davey Smith G. Multiple risk factor interventions for primary prevention of coronary heart disease. Cochrane Database Syst Rev. 2006:CD001561. [PubMed]
  • Edington DW. Emerging research: a view from one research center. Am J Health Promot. 2001;15:341–349. [PubMed]
  • Edington DW, Yen LT, Witting P. The financial impact of changes in personal health practices. J Occup Environ Med. 1997;39:1037–1046. [PubMed]
  • Emmons KM, Marcus BH, Linnan L, Rossi JS, Abrams DB. Mechanisms in multiple risk factor interventions: smoking, physical activity, and dietary fat intake among manufacturing workers. Working Well Research Group. Prev Med. 1994;23:481–489. [PubMed]
  • Fieuws S, Verbeke G. Pairwise fitting of mixed models for the joint modeling of multivariate longitudinal profiles. Biometrics. 2006;62:424–431. [PubMed]
  • Fine LJ, Philogene GS, Gramling R, Coups EJ, Sinha S. Prevalence of multiple chronic disease risk factors; 2001 National Health Interview Survey. Am J Prev Med. 2004;27:18–24. [PubMed]
  • Gueorguieva RV, Agresti A. A correlated probit model for joint modeling of clustered binary and continuous responses. J Am Statistical Assoc. 2001;96:1102–1112.
  • Guo X, Carlin BP. Separate and Joint Modeling of Longitudinal and Event Time Data Using Standard Computer Packages. Am Statistician. 2004;58:16–24.
  • Janssen I, Katzmarzyk PT, Church TS, Blair SN. The Cooper Clinic Mortality Risk Index: clinical score sheet for men. Am J Prev Med. 2005;29:194–203. [PubMed]
  • Poortinga W. The prevalence and clustering of four major lifestyle risk factors in an English adult population. Prev Med. 2007;44:124–128. [PubMed]
  • Prochaska JJ, Sallis JF. A randomized controlled trial of single versus multiple health behavior change: promoting physical activity and nutrition among adolescents. Health Psychol. 2004;23:314–318. [PubMed]
  • Prochaska JJ, Sorensen JL, Hall SM, Rossi JS, Redding CA, Rosen AB, Eisendrath SJ, Meisner MR. Predictors of health functioning in two high-risk groups of smokers. Drug Alcohol Depend. 2005;78:169–175. [PubMed]
  • Prochaska JJ, Velicer WF, Prochaska JO, Delucchi K, Hall SM. Comparing intervention outcomes in smokers treated for single versus multiple behavioral risks. Health Psychol. 2006;25:380–388. [PubMed]
  • Prochaska JO, Velicer WF, Redding C, Rossi JS, Goldstein M, DePue J, Greene GW, Rossi SR, Sun X, Fava JL, Laforge R, Rakowski W, Plummer BA. Stage-based expert systems to guide a population of primary care patients to quit smoking, eat healthier, prevent skin cancer, and receive regular mammograms. Prev Med. 2005;41:406–416. [PubMed]
  • Prochaska JO, Velicer WF, Rossi JS, Redding CA, Greene GW, Rossi SR, Sun X, Fava JL, Laforge R, Plummer BA. Multiple risk expert systems interventions: impact of simultaneous stage-matched expert system interventions for smoking, high-fat diet, and sun exposure in a population of parents. Health Psychol. 2004;23:503–516. [PubMed]
  • Pronk NP, Anderson LH, Crain AL, Martinson BC, O’Connor PJ, Sherwood NE, Whitebird RR. Meeting recommendations for multiple healthy lifestyle factors; Prevalence, clustering, and predictors among adolescent, adult, and senior health plan members. Am J Prev Med. 2004;27:25–33. [PubMed]
  • Sallis JF, Alcaraz JE, McKenzie TL, Hovell MF. Predictors of change in children’s physical activity over 20 months. Variations by gender and level of adiposity. Am J Prev Med. 1999;16:222–229. [PubMed]
  • Shinton R. Lifelong exposures and the potential for stroke prevention: the contribution of cigarette smoking, exercise, and body fat. J Epidemiol Community Health. 1997;51:138–143. [PMC free article] [PubMed]
  • Sorensen G, Thompson B, Glanz K, Feng Z, Kinne S, DiClemente C, Emmons K, Heimendinger J, Probart C, Lichtenstein E. Work site-based cancer prevention: primary results from the Working Well Trial. Am J Public Health. 1996;86:939–947. [PubMed]
  • Stampfer MJ, Hu FB, Manson JE, Rimm EB, Willett WC. Primary prevention of coronary heart disease in women through diet and lifestyle. N Engl J Med. 2000;343:16–22. [PubMed]
  • Toobert DJ, Glasgow RE, Strycker LA, Barrera M, Jr, Ritzwoller DP, Weidner G. Long-term effects of the Mediterranean lifestyle program: a randomized clinical trial for postmenopausal women with type 2 diabetes. Int J Behav Nutr Phys Act. 2007;4:1. [PMC free article] [PubMed]
  • Tucker LR, Damarin F, Messick S. A base-free measure of change. Psychometrika. 1966;31:457–473. [PubMed]
  • Veldman D, Brophy J. Measuring teacher effects on pupil achievement. Journal of Educational Psychology. 1974;66:319–324.
  • Velicer WF, Prochaska JO. An expert system intervention for smoking cessation. Patient Educ Couns. 1999;36:119–129. [PubMed]
  • Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 Health Survey Manual and Interpretation Guide. Boston, MA: The Health Institute, New England Medical Center; 1997.
  • Woolf SH. The need for perspective in evidence-based medicine. JAMA. 1999;282:2358–2365. [PubMed]