|Home | About | Journals | Submit | Contact Us | Français|
The development and use of validated processes of change (POC) measures have received little attention in the literature despite their importance in the Transtheoretical Model. Using survey data (n=2909), we examined the construct validity of a 22-item mammography POC scale by testing for factorial validity and factorial invariance across stage of change. We also used MANOVA with tukey post-hoc tests to confirm stage differences in POC use (concurrent validity). Our results confirm the a priori correlated four-factor structure of this scale and provide some support for the measurement equivalence of this scale across stage, supporting comparisons of POC use across stage.
An estimated 192,370 new cases of breast cancer and 40,170 breast cancer deaths are estimated to occur in United States women in 2009 (American Cancer Society, 2009). To reduce breast cancer mortality, screening mammography is recommended on an annual or biennial basis for all women age 40 and older (Humphrey, Helfand, Chan, & Woolf, 2002; Smith, et al., 2003). Following a period of rapid increases in the reported use of mammography between 1987 and 2000, recent estimates from 2005 appear to show a decline (Breen, et al., 2007). Furthermore, adherence rates to mammography guidelines are suboptimal; a review of repeat mammography studies completed between 1995–2001 suggests that only 53% of women receive 2 or more consecutive on-schedule mammograms (Clark, Rakowski, & Bonacore, 2003).
Health promotion interventions based on the Transtheoretical Model (TTM) can increase mammography uptake and repeat mammography (Champion, et al., 2002; Rakowski, et al., 1998). The TTM posits that individuals progress through five stages when making a health behavior change: precontemplation (unaware of the problem), contemplation (considering change), preparation (taking initial steps), action (short-term behavior change), and maintenance (sustained behavior change). Other TTM constructs include decisional balance (pros and cons), self-efficacy, and the processes of change (POC).
The POC represent distinct cognitions, behaviors, or activities that facilitate completion of stage tasks, movement from one stage to another, and promote sustained behavior change. The original 10 POC developed for tobacco cessation (DiClemente & Prochaska, 1982; Prochaska & DiClemente, 1983) have been modified for use among different populations and across different health behaviors such as exercise adoption and eating a low fat diet (Bowen, Meischke, & Tomoyasu, 1994; Marcus, Rossi, Selby, Niaura, & Abrams, 1992). However, comparatively little theoretic or empiric work has been done to extend TTM constructs to mammography behavior. Unlike diet and exercise behaviors, mammography use requires repeated, but infrequent behavior that requires coordination with the healthcare system.
The focus of psychometric research on mammography TTM measures to date has been the validation of constructs such as stage of change, pros, and cons (Rakowski, et al., 1992; Rakowski, Ehrich, et al., 1996; Tiro, et al., 2005). Less attention has been paid to the development and validation of mammography POC. Possibly due to the lack of extensively validated mammography POC scales, mammography intervention studies frequently incorporate stage of change and the pros and cons, but exclude or deemphasize the relevance of POC. As a result of the limited research, little is known about the relationship of mammography POC to other TTM constructs, the role that POC play in predictive and causal models of mammography or if and how mammography POC facilitate behavior and stage change. As a consequence, we know little about which POC to target to move women from precontemplation and contemplation to action and maintenance for mammography.
Although we found one mammography POC scale in the published literature, (Rakowski, Dube, & Goldstein, 1996; Rakowski, Ehrich, et al., 1996) there have not yet been any published psychometric studies of this scale. The scale was developed using exploratory factor analysis on items stemming from both the original POC measures and additional formative research with women aged 40 and older recruited from a worksite. The scale’s 22 items represent four latent constructs, differing from the original conceptualization of POC. Unlike the original POC items that cluster into two higher-order cognitive and behavioral factors, the four constructs in this scale include items that, based on face validity, could be indicators of either cognitive or behavioral processes. Moreover, one factor in the mammography POC scale, Avoids the Health Care System, encompasses a woman’s interaction with the healthcare system in general and is not specific to mammography.
Reliable and valid measures of POC are necessary to understand the success or failure of TTM interventions and for research examining the causal mechanisms of behavior change hypothesized by the TTM. The purpose of this secondary analysis was to assess the construct validity of a previously reported 22-item, correlated four-factor mammography POC scale. Specifically, we tested the factor structure in a new sample of women eligible for mammography (factorial validity), examined the measurement and structural invariance of this measure across women’s stage of change for mammography use (factorial invariance), and examined the distribution of POC across stage of change (concurrent validity).
Construct validity can confirm the extent to which inferences from scale scores can be made in relation to the underlying, latent, theoretical construct of interest. Empirical evidence of factorial validity, factorial invariance, and concurrent validity provide support for construct validity. Typically, POC studies seek to demonstrate construct validity using a test of concurrent validity in which an association is measured between mean POC scores and stage of change (DiClemente, et al., 1991; Fava, Velicer, & Prochaska, 1995; Rakowski, Ehrich, et al., 1996). Evidence of concurrent validity demonstrates that the measures behave as specified by theory, providing some evidence that the items are capturing the intended construct. Concluding that a demonstration of significant mean differences by stage provides evidence of construct validity, however, presupposes factorial validity and invariance across stages of change.
We explicitly test the assumptions of factorial validity, factorial invariance, and concurrent validity by stage of change in this study. Evidence of factorial validity demonstrates that a set of items measures the hypothesized latent variables. The assumption of factorial invariance is that a scale measures the same trait across multiple groups. Factorial invariance is comprised of structural invariance (defined here as the equivalence of factor structure, e.g. number of factors and number of items loading onto each factor) and measurement invariance (defined here as the equivalence of factor loadings and factor variances for each item). Evidence of factorial invariance confirms that any observed between-group mean differences are the result of true attitudinal differences regarding the underlying constructs rather than due to different psychometric responses to the items and/or factors (e.g., biases in the way different subgroups of people interpret and respond to the items) (Nunnally, 1994). Evidence of concurrent validity demonstrates the extent to which a set of items has an expected pattern of association with another, validated measure; in this case, stage of change.
This study is a secondary data analysis of baseline survey data from participants in a intervention trial designed to increase repeat mammography screening. A detailed description of the study design, sampling strategy, and eligibility criteria is available elsewhere (del Junco, et al., 2008; Vernon, et al., 2008). Briefly, women veterans were randomly selected from a national registry and were eligible if they were 52 years of age or older, had no history of a breast cancer diagnosis, were able to be contacted, were physically able to complete the survey, and were not on active military duty. Our sample includes 2909 women who returned the mailed baseline survey, representing 47.7% of the eligible women veterans. All participants provided informed consent.
Stage of change was measured by the combination of three items that asked women to report the month and year of their most recent and second most recent mammograms, if any, and when, if ever, the woman intended to obtain her next mammogram. Women were classified into four stages of change: precontemplation (never had or no recent mammogram within the previous 15 months and no plans to have one in the next year), contemplation (never had or no recent mammogram, but plans to in the next year), action (one recent mammogram and plans to stay on a 1–2 year schedule), and maintenance (2 mammograms on schedule and plans to stay on a 1–2 year schedule). We did not classify women into a preparation stage based on previous research that adapted the stages of change for mammography to suit the inherent periodicity of the behavior (Rakowski, Dube, et al., 1996; Rakowski, Ehrich, et al., 1996).
The four factors of Rakowski’s 22-item POC scale include: Commitment to Regular Screening, Information Sharing and Communication, Thinking Beyond Oneself, and Avoids Contact with the Health Care System (Table 1) (Rakowski, Ehrich, et al., 1996). All 22 items were included and measured on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree” with higher scores indicating greater use of the POC. One item from the Avoids Contact factor, “I keep a record so that I know when to schedule my next doctor’s appointment,” was reverse coded.
The data were analyzed using confirmatory factor analysis (CFA) with full-information maximum likelihood estimation (FIML) in AMOS 7.0 (Arbuckle, 2006). FIML was selected because it is an optimal method for the treatment of missing data in CFA (Enders & Bandalos, 2001). Less than 10% of responses to items in each of the four factors were missing. Before acceptance of the final model with the entire sample, we allowed for improvements and modifications to the model in the form of error covariances.
Factorial validity is indicated when a set of items correlates strongly with the hypothesized latent constructs. We assessed factorial validity by examining model fit and the significance of factor loadings in CFA. Because we performed an initial validation of a previously reported factor structure obtained by exploratory factor analysis, we tested the hypothesized model structure as well as two alternative models representing different conceptualizations of the POC structure to help reduce the possibility of confirmation bias (MacCallum & Austin, 2000). We hypothesized that the previously reported correlated four-factor model would provide the best fit. Using the entire sample, three competing models were evaluated:
To assess model fit, we used multiple fit indices: the chi-square/degrees of freedom test, comparative fit index (CFI), Root Mean Square Error of Approximation (RMSEA) and its associated 90% confidence interval, and the Akaike Information Criterion (AIC). CFI values between 0.90–0.95 or above are considered to provide adequate to good fit (Hu & Bentler, 1995; Hu & Bentler, 1999) and RMSEA values <.06 suggest good model fit (Hu & Bentler, 1999). The AIC adjusts the chi-square statistic to account for model complexity and has been recommended for comparisons between non-nested models. Models with the lowest AIC value are preferred (Kline, 2005). We concluded that models meeting the above criteria indicated support for factorial validity of the POC. We did not expect to obtain non-significant chi-square test statistics due to our large sample size.
The best fitting or preferred model using the entire sample was selected for post-hoc analyses to assess factorial validity within each stage and invariance across stage. First, we conducted independent, stage-specific CFAs to assess model fit for each group (baseline stage models). We did not make any modifications to the stage-specific models. Because we selected the best fitting model using the entire sample, we hypothesized that the stage-specific models, with some loss of fit due to their smaller sample sizes, would also demonstrate acceptable fit. Second, using the chi-square difference test, we examined measurement and structural invariance across adjacent stage groups: precontemplation vs. contemplation (PC-Con), contemplation vs. action (Con-Act), and action vs. maintenance (Act-Maint). We expected that invariance was more likely among adjacent groups than extreme groups (PC-Maint), allowing for the possibility of lack of equivalence between some but not all of the stages. If invariance between multiple adjacent stage groups was found, non-adjacent multigroup comparisons would be conducted. For each pair of comparisons, we began with a fully constrained model in which all factor loadings, factor variances, and factor correlations were specified as equal across stage groups. If the fully constrained model was not invariant, we proceeded to test for partial invariance through a series of nested models. Following the process recommended by Byrne (2001), we successively applied more stringent equality constraints in which we first tested the invariance of factor loadings, followed by factor variances, and finally factor covariances. Once a fully or partially invariant model was identified, we used multiple fit indices to assess overall fit for both stage groups simultaneously. We did not test for the equality of error variances or covariances, as doing so may represent an overly restrictive test of data and is not necessary (Byrne, 2001). Additionally, we did not test for the equality of item intercepts or factor means because we expected substantive differences across stages (Vandenberg & Lance, 2000).
Concurrent validity also provides support for construct validity and is established by confirming an expected association between a set of items and another validated measure. We conducted multivariate analysis of variance (MANOVA) using SPSS version 15.0 (SPSS, Chicago, IL) to determine if overall mammography POC use differed by stage and post-hoc Tukey tests to determine how each POC differed by stage. Based on earlier research demonstrating that mammography POC use increased with stage progression (Rakowski, Ehrich, et al., 1996; Rakowski, et al., 1998), we hypothesized that POC use would increase across the stages of change, that is, that POC use would be highest in maintenance (with the exception of the Avoids subscale which we expected would decrease across stage). While research in cessation behaviors suggests that POC use may begin to decline in maintenance and ceases when the behavior has been terminated (C.C. DiClemente, 2003), there is no literature to date suggesting that use of POCs decline when an adoption behavior such as mammography becomes habitual.
The mean age was 61.8 (SD=9.6) years old and nearly half of the women (41.3%) were married or living with a partner. The sample was predominantly white (86.1%) and well-educated; 44.5% had attended at least some college or technical school. The women in this sample were distributed across the stages of change for regular mammography use as follows: 15.2% in precontemplation, 9.3% in contemplation, 25.1% in action, and 50.4% in maintenance.
The hypothesized correlated four-factor model (Figure 1; CFI=.927, RMSEA=.059, 90% CI: .056–.061; χ2 =2158.7, df=196, AIC=2316.68) provided acceptable fit to the data and an improvement over the alternative models (Alternative Model 1: CFI=.917, RMSEA=.061, 90% CI: .059–.064; χ2 =2416.2, df=202, AIC=2562.20; Alternative Model 2: CFI=.925, RMSEA=.059, 90% CI: .057–.061; χ2 =2207.5, df=198, AIC=2361.47). Because three of the factors were highly correlated (r ≥.91), we also tested two additional alternative post-hoc models. The first post-hoc alternative model was a correlated two factor model that combined these three factors (Thinking, Information, and Commitment) into one factor (CFI=.921, RMSEA=.061, 90% CI: .058–.062; χ2 =2322.0, df=201 [the two factor means had to be fixed to 0 for the model to converge], p<.001; AIC=2470.02). The second post-hoc alternative model was a second-order factor model in which the three highly correlated factors (Thinking, Information, and Commitment) were represented by a second order Mammography factor which was correlated with the forth factor (Avoids) (CFI=.925, RMSEA=.059, 90% CI:.057–.061; χ2 =2207.5, df=198 p<.001; AIC=2361.47). The AIC of the hypothesized correlated four-factor model (AIC=2316.68) was lower than the AIC for either of the alternative models, indicating that it provided the best fit to the data.
The correlated four-factor model was reanalyzed for each stage of change group separately. In the entire sample model and in each of the stage-specific models, all factor loadings, factor correlations, error terms, and error covariances and all but one of the factor variances (Avoids) in one of the stage-specific models (Con) were significantly different from zero. As expected, due to the large sample size, model fit for the entire sample was better than the fit for any of the four individual stage groups. Of the independent stage models, the largest sample was in maintenance, which provided the best fit to the data (Table 2). Only marginal fit was obtained for the other three stage groups.
The correlated four-factor model was then tested for invariance across adjacent stages of change. Tests of model comparison indicated that the fully constrained Model PC-Con was not equivalent (Table 2). After allowing three factor loadings, two factor variances, and three factor correlations to be freely estimated (Figure 1), the model was invariant between precontemplation and contemplation stage groups. Model comparisons indicated that the fully constrained Model Con-Act was invariant. Non-invariance was indicated for Model Act-Maint until two factor variances and two factor correlations were allowed to be freely estimated (Figure 1).
Results of the MANOVA indicated significant differences in mean mammography POC scores by stage of change (F(12, 6643)=3.77 p<.001, multivariate (Wilks) eta-squared=0.129) (Table 3). Post-hoc Tukey tests of significance using mean factor scores indicated that two POC (Information, Commitment) increased across stage (PC < Con < Act < Maint) and one POC (Avoids) decreased across stage (Maint < Act < Con < PC). For the Thinking POC, mean scores increased across stage, but were not significantly different between contemplation and action (PC < Con & Act, < Maint). Alpha coefficients for the four POC subscales ranged from .70–.85 and .86 for the 22 items combined indicating acceptable internal consistency reliability (Table 3).
Our study highlights important issues for cancer prevention and control research and contributes to the scant cancer screening literature of published scales with defined psychometric properties. Our study is one of very few published reports of POC construct validation using confirmatory factor analysis (O’Connor, Carbonari, & DiClemente, 1996) and, to our knowledge, the only study examining POC invariance across stage. As a critical element of the TTM, and the primary construct reflecting how shifts in behavior occur, both the psychometrics and the role of POC in behavior change deserve greater attention in the cancer screening literature. Because psychometric and intervention research studies using the TTM often neglect POC, much remains to be known about the validity, reliability, and utility of POC as mechanisms of change across multiple and varied cancer prevention behaviors. Although we took the first step in exploring the construct validity and invariance across stage of change for one mammography POC scale, continued psychometric research is needed on the multiple existing POC scales for diverse health behaviors and during the development phase of new POC scales.
Our results provide additional support for the construct validity of this previously published scale. We replicated the factor structure and provided evidence of factorial validity in our full sample and among women in the maintenance stage, the largest subsample. We found evidence of full and partial measurement invariance across adjacent stages of change. Examination of mean POC use by stage confirmed the expected pattern of associations and replicated earlier work supporting the concurrent validity of this scale (Rakowski, Ehrich, et al., 1996). Lastly, internal consistency coefficients were adequate to good for each subscale and were higher than previously reported (Rakowski, Ehrich, et al., 1996).
Although the use of MANOVA to support the concurrent validity of POC scales is common in the TTM literature (DiClemente, et al., 1991; Fava, et al., 1995; Rakowski, Ehrich, et al., 1996; Wadsworth & Hallam, 2007), conclusions about observed mean differences are based on an assumption of measurement invariance by stage. Our replication of expected differences in factor means by stage groups (concurrent validity analyses) is supportive, but cannot be accepted as completely unequivocal support for construct validity given the differences in model fit by stage group and only partial measurement invariance in 2 of the 3 adjacent stage comparisons; one of which required the free estimation of factor loadings. There is no consensus regarding the number or percent of items with invariant loadings that are necessary to support measurement invariance across groups nor a single recommended approach for dealing with invariant items. Approaches for dealing with scales lacking invariance include restricting group comparisons to items with equivalent factor loadings and variances or exploring the qualitative meaning of the item or factor to understand meaningful group differences (Gregorich, 2006). Closer examination of the estimates differing by stage indicated that they did not differ qualitatively; that is, they were all positive coefficients and only differed modestly by degree (magnitude). Upon examination of the three factor loadings that were not invariant in the PC-Con comparison, one item (I5) was comparatively longer and more complex than the others and therefore may be a reasonable item to drop from the scale. Four of the items in the Thinking POC referred to women, friends, and people generally, and only the 2 items referring to doctor recommendation (T3, T6) were not invariant in the PC-Con comparison. These items could be considered for deletion because they may be less relevant for women in precontemplation; women in this stage may have less contact with physicians and are therefore less likely to discuss or be offered mammography. However, because this is only the second study to examine this scale, we believe it would be premature to suggest dropping these items prior to analysis of the POC in other samples.
We also found very large factor correlations (r ≥.91) between 3 of the POC suggesting that at a measurement level it is not clear that women perceive these constructs as different. However, alternative models, including a correlated two-factor model combining these three factors and a second-order factor model grouping these three factors into one second-order factor, did not demonstrate improvement in fit compared with the a priori model. Therefore, although the presence of high correlations among factors is not desirable, the a priori model provided the best fit to the data and was selected as the final model.
Ultimately, the relevance of stage-specific model fit and invariance by stage depends on the primary purpose of the POC. If the purpose of using POC scales is to compare mean scores across stage, factorial invariance by stage should be the goal, as it allows for accurate comparisons of true group differences across stage. For this purpose, items should be general enough and relevant to all stages and items lacking equivalence across stage could be candidates for removal. Alternatively, if researchers are using a POC scale to identify stage-specific clinical indicators of change or to develop tailored individual-based behavioral change interventions, then factorial invariance across stages may not be relevant or necessary. Toward this end, within-stage comparisons would be appropriate and stage-specific POC scales may even be useful in the identification and manipulation of processes needed within a particular stage, but they could not be compared across stages as they reflect different POC.
It is not clear whether the mammography POC scales examined in this study are conceptually different from behavioral and cognitive POC or simply a different grouping of these processes. POC are defined as the cognitive and behavioral strategies used by individuals to facilitate behavior change (Prochaska, Velicer, DiClemente, & Fava, 1988) and are typically measured with 10 cognitive/experiential and 10 behavioral items reflecting five POC per subscale. Future research should examine the factorial validity and invariance of 2-factor POC scales and compare different conceptualizations of the POC. However, to our knowledge, no previously published 2-factor mammography POC scales are available; moreover, the convention of measuring 2-factor POC with 2 items per process limits the utility of construct validity testing; as more items are typically recommended for confirmatory factor analysis (Marsh, Hau, Balla, & Grayson, 1998).
It has been suggested that rather than measuring the behaviors and cognitions through which behavior change occurs, these POC may instead represent reactive actions or attitudes that can result from an intervention (Spencer, Pagell, & Adams, 2005). Moreover, the response scale ranges from strongly agree-strongly disagree, similar to that used in attitudinal measures and not the frequency scale typically used for the measurement of 2-factor POC. However, all POC items have been selected in part because they are potentially modifiable factors that can be influenced by intervention and it could be argued that even reactive actions or attitudes represent processes that could influence future action. In a longitudinal or intervention study, we would expect increasing use of processes (with the exception of the Avoids subscale which should decrease) across stage of change progression, as seen in this study and other cross-sectional (Rakowski, Ehrich, et al., 1996) and intervention studies (Rakowski, et al., 1998) of mammography, and in cross-sectional studies of other behaviors (DiClemente, et al., 1991; Fava, et al., 1995; Wadsworth & Hallam, 2007). It would be worthwhile to explore the conceptual overlap between different conceptualizations of POC along with their inter-relations with other theoretical constructs in both cross-sectional and longitudinal research to help clarify and understand their unique roles in supporting health behavior change.
This study was a secondary analysis of self-report data from a large cross-sectional sample of baseline participants in a repeat mammography intervention trial. The women were predominantly white, well-educated, and in the maintenance stage for mammography, which may reduce the generalizability of our results. However, our sample was large and randomly selected from a national population of U.S. veterans that is similar in demographic characteristics to the U.S. female population (del Junco, et al., 2008).
A strength of this study is that it expands on previous research that has applied the TTM to mammography behavior and replicates earlier findings regarding the valid measurement of infrequently measured POC. The results of this study provide some evidence for construct validity, internal consistency reliability, and partial structural and measurement invariance across stages of change for this scale. We have added confidence in our findings because the factor structure was replicated in our full sample, several years after it was first developed and used in a different population. Our study provides a good launching point for additional psychometric and conceptual research on the POC for mammography and other cancer prevention and control behaviors. For example, following recent work on constructs from the Health Behavior Model (Champion, et al., 2008) future researchers should explore the psychometrics of this and other TTM measures among diverse populations. Importantly, our study also examined previously untested assumptions that mean differences across stage of change reflect true stage differences rather than differences in construct measurement and/or response style. Future researchers should consider employing similar analyses prior to comparing mean differences of TTM constructs across stage of change.
POC are a central construct of the TTM and are important targets for intervention; yet they are frequently overlooked in studies of cancer screening behaviors. The availability of valid and reliable POC scales will improve researchers’ ability to discriminate between stages of change and to target stage-specific POC in the design of interventions, thus improving intervention effectiveness.
This research was supported by National Cancer Institute grants RO1-CA-76330 and R25-CA-057712. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health.
Sandi L. Pruitt, PhD, MPH, is a postdoctoral fellow at Washington University School of Medicine, Division of Health Behavior Research.
Amy McQueen, PhD, is an assistant professor at Washington University School of Medicine, Division of Health Behavior Research.
Jasmin A. Tiro, PhD, is an assistant professor in the Department of Clinical Sciences at The University of Texas Southwestern Medical Center.
William Rakowski, PhD, is a professor in the Department of Community Health at Brown University.
Carlo DiClemente, PhD, is a professor in the Department of Psychology at the University of Maryland, Baltimore County.
Sally W. Vernon, PhD, is a professor of Epidemiology and Behavioral Science at the University of Texas School of Public Health