Exploratory factor analysis of the Study 1 baseline data
An initial EFA of the 26 baseline CSE item responses from the Study 1 sample was conducted to identify a probable factor structure. Though only six of 149 participants had not responded to all items, we used the Mplus computer program to perform the analyses because Mplus features full-information maximum likelihood (FIML), which enables the software to make use of incomplete cases’ available data (Muthén & Muthén, 2001
). Simulation studies have shown FIML typically outperforms common ad hoc
missing data-handling methods, such as listwise deletion, across a wide variety of data analysis scenarios (Arbuckle, 1996
; Wothke, 2000
). The 26 items included in the original CSE measure were ‘mapped’ to the curriculum of the CET intervention, in which individuals were being taught how to increase their adaptive coping by selecting the appropriate coping strategy when faced with challenges and threats in their lives. Emphasis was placed on both problem-focused and emotion-focused coping strategies, including seeking social support and engaging in spiritual and/or meditation activities. Therefore, the number of factors to retain was determined by a combination of coping theory and empirical findings. We anticipated extracting, as a minimum, two distinct factors: problem- and emotion-focused coping. However, we recognized that the empirical data might suggest other groups of items that indicate the presence of additional coping factors of interest, such as seeking support or engaging in spirit-focused activities. Items were assigned to the factor on which they had the largest rotated factor loading.
Using Promax rotation to allow for correlated factors, we extracted two to five factors and evaluated the interpretability of each extracted factor. A two-factor solution was rejected because a number of conceptually similar items had split loadings across the two factors. Four- and five-factor solutions yielded several nonsensical doublet factors; that is, pairs of items with no discernable theoretical connection that spuriously loaded on to those factors. The presence of doublet factors may indicate that too many factors were extracted in these solutions. By contrast, the three-factor solution yielded clearly interpretable results. Based upon prior expectations in conjunction with the empirical aggregation of the items with the factors, the three factors were labelled use problem-focused coping, stop unpleasant emotions and thoughts, and get support from friends and family.
The first factor consists of items that measure an individual’s self-efficacy with respect to overcoming problems by analysing the nature of the problem and employing cognitive strategies to make the respondent’s perception of the problem less severe (e.g. ‘break an upsetting problem down into smaller parts’). The second factor measures a respondent’s self-efficacy with respect to altering his emotional response to an unsettling event or problem rather than addressing the characteristics of the problem itself (e.g. ‘take your mind off negative thoughts’). These two factors map on to the existing theoretical domains of problem-focused coping and emotion-focused coping, respectively. The third extracted factor represents a set of items that captures a social dimension by tapping the respondent’s perception of his ability to reach outside himself and seek help from friends and family to cope with problems (e.g. ‘get emotional support from friends and family’).
The factors were moderately related: use problem-focused coping was positively correlated with stop unpleasant emotions and thoughts (r = .67) and with get support from friends and family (r = .60). Stop unpleasant emotions and thoughts was, in turn, positively correlated with get support from friends and family (r = .54). Survey items and standardized loadings for each item on its parent factor are shown in . These factors and their associated items were used as the basis for specifying the first CFA model described below.
Confirmatory factor analyses of the Study 1 and Study 2 baseline data
CFA models may be usefully employed in an exploratory context to further refine the results derived from EFAs because CFAs provide tests of parameter estimates and global model fit. These tests enable investigators to identify items that are weakly related to their parent factors and remove those items from subsequent analyses, which results in a shorter, more pure measurement instrument (Neilands & Choi, 2002
). To evaluate global model fit, we report the chi-squared tests of model fit and several descriptive fit indices, described below. Though preliminary examination of the items’ univariate distributions suggested the items possessed an approximately normal distribution, we opted to report a chi-squared test of model fit and parameter estimate standard errors that are robust to departures from normality (Yuan & Bentler, 2000
). Even with corrections for non-normal data, the chi-squared test of absolute model fit can be sensitive to trivial misspecifications in the model’s structure (Bollen & Long, 1993
). Consequently, we also report the following descriptive measures of model fit that are often used to evaluate the soundness of a model: the standardized root mean residual (SRMR; Bollen, 1989
), the comparative fit index (CFI; Bentler, 1990
), and the root mean square error of approximation (RMSEA; Browne & Cudek, 1993
). Hu and Bentler (1999)
provide recent simulation evidence and guidelines suggesting that CFI values of .95 or higher, RMSEA values of .06 or lower, and SRMR values of .08 or lower indicate good model fit when these fit statistics are considered together. Vandenberg and Lance (2000
, p. 44) note, however, that Hu and Bentler’s simulation study, while extensive, is a single study; its findings should be replicated independently before these cut-off values are canonized. Vandenberg and Lance point to an extensive body of prior simulation research that suggests cut-off values of .90 for the CFI and related incremental fit indices, .08 for the RMSEA, and .10 for the SRMR. Consequently, Vandenberg and Lance suggest treating the Hu and Bentler fit criteria as high confidence values and the previously recommended cut-offs as acceptable lower bounds of good model fit. We follow this approach in evaluating the global model fit tests and indices reported below. All CFAs reported below were fit using Mplus 2.12 (Muthén & Muthén, 2001
Our initial CFA model using the data from Study 1 extended the EFA results reported above by assigning each of the 26 CSE items to the factor with which it was most strongly associated in the EFA results. The overall fit of this model was poor on an absolute basis χ2(296) = 605.83, p < .0001; descriptive model fit statistics also indicated poor fit: CFI = .85, RMSEA = .08, and SRMR = .08. Starting with the item that had the lowest factor loading value, we removed items one at a time, refitting the model after each item’s removal, until we attained satisfactory global model fit. Our final model contained half of the original items. The fit of this model was very good on a descriptive basis: CFI = .95, RMSEA = .07, and SRMR = .05, though it failed the absolute model fit test, χ2(62) = 112.20, p < .0001. Factor loadings from the final CFA model for Study 1 appear in in boldface type. For this model, the factor correlations were nearly identical to those found with the EFA model.
A limitation of the ‘EFA followed by CFA’ model-building approach outlined in the preceding paragraph is that repeated examination of the data under different model scenarios can lead to sample-specific solutions that may not generalize to new samples. To guard against this possibility, we fit the final model from the previous paragraph to the Study 2 data. Results from this analysis indicated satisfactory model fit: the exact test of model fit was rejected, χ2(62) = 113.81, p < .0001, but descriptive model fit statistics indicated satisfactory fit (CFI = .94, RMSEA = .08, SRMR = .06). Factor loadings from the CFA model for Study 2 appear in in boldface type. For this model, the factor correlations were, again, nearly identical to those found with the Study 1 EFA and CFA models. Moreover, as seen in , the factor loadings from the EFA of Study 1’s data, the final CFA of Study 1’s data, and the CFA of Study 2’s data are highly consistent across both studies and their respective factor analyses.
Confirmatory factor analyses of the Study 1 and Study 2 follow-up data
To assess the temporal stability of the model, we refit it to the pooled Study 1 and Study 2 data at 3, 6, and 12 months following baseline. In general, results indicated satisfactory model fit at each follow-up point: 3 months (χ2(62) = 158.68, p<.0001, CFI = .95, RMSEA = .07, SRMR = .06); 6 months (χ2(62)=201.55, p<.0001, CFI = .92,RMSEA = .09, SRMR = .06); and 12 months (χ2(62)=152.36, p<.0001, CFI = .95, RMSEA = .08, SRMR = .05). Factor loadings (not shown to conserve space) were consistent with those reported in for baseline data.
Reliability for the three derived CSE scales was measured via Cronbach’s internal consistency coefficient alpha (Cronbach, 1951
) at baseline using data from all study participants and via test–retest correlations for longitudinal data using data only from control participants (i.e. WLC for Study 1 and MCC for Study 2). Although the sample sizes for the test–retest correlations were reduced by excluding intervention participants, we were able to remove the possible confounding influence of the CET intervention on participants’ behaviours and, in turn, its impact on subsequent assessments of CSE scores. For Study 1, the sample size for the test–retest correlation coefficient from baseline to Month 3 is 38; for Study 2, the sample sizes from baseline to Months 3, 6, and 12 are 61, 57, and 56, respectively. Because the WLC participants in Study 1 were crossed over to receive the CET intervention at the end of the intervention phase (Month 3), they no longer served as controls during the 9-month maintenance phase and, therefore, test–retest correlation coefficients for their 6- and 12-month follow-up assessments were not calculated.
In order to compute test–retest correlation coefficients, CSE scale scores were created by taking the mean of the items in each scale so that they would be in the same metric as the original items (i.e. 0–10). At baseline, sample descriptive statistics for each scale, using data from the combined studies, was as follows: use problem-focused coping (N = 346, mean = 5.6, SD = 2.1), stop unpleasant emotions and thoughts (N = 348, mean = 4.5, stdev = 2.2), and get support from friends and family (N = 348, mean = 5.1, SD = 2.3).
As shown in , the coefficient alpha for each scale was virtually identical using data from either study. Using data from the combined studies, the baseline coefficient alpha for each scale was strong, ranging from .80 for get support from friends and family to .91 for use problem-focused coping and stop unpleasant emotions and thoughts. Although not shown in , for purposes of comparison, we also computed the coefficient alphas for the CSE scales at each of the three follow-up assessments using the data from the combined studies and found that the alphas remained essentially unchanged from those reported at baseline.
Summary of the internal consistency and test–retest reliability analyses for the derived CSE scales
The test–retest correlation coefficients for baseline to Month 3 were relatively similar across the two studies. In addition, all of the correlation coefficients were strong, ranging from .40 to .80 (all p < .005). Within each scale, the strongest correlations were typically between data collected at baseline and then at the 3- and 6-month follow-up assessments. There was some drop-off in the magnitude of the correlations between the baseline and the 12-month data, as might be expected, given the increasing length of time between the baseline and follow-up assessments.
To assess concurrent validity, separate Pearson partial correlations were calculated for each of the three derived CSE scale scores and measures of psychological distress and well-being, ways of coping, and social support using baseline data from the combined studies, while controlling for the effects of the other two CSE scale scores (N = 347 for all analyses). Sample descriptive statistics and Pearson partial correlations for the individual measures are shown in .
Partial correlations evaluating the independent relationships between measures of psychological distress and well-being and each of the three derived CSE scale scores were, on average, largest for feeling able to stop unpleasant emotions and thoughts (absolute partial r’s ranged from .20 to .28, all ps < .0001). Several of the partial correlations between the distress and well-being measures and feeling able to use problem-focused coping skills were also similar in magnitude. For example, when faced with challenges and threats, participants who felt more self-efficacious in being able to stop unpleasant emotions and thoughts and/or using problem-focused coping skills also reported less stress and anxiety and more optimism. On average, the independent relationships between levels of psychological distress and well-being and feeling able to get support from friends and family were somewhat weaker, with the exception of positive states of mind (partial r = .26, p < .001).
The patterns of association between the three derived CSE scales and the ways of coping subscales also provided evidence of convergent and divergent validity. As to how they cope with stressful situations, individuals who reported greater confidence in their ability to use problem-focused coping also tended to report greater use of planful problem solving as a coping style (partial r = .27, p < .001). Individuals who were more confident in their ability to stop unpleasant emotions and thoughts were also less likely to use cognitive escape-avoidance (partial r = −.20, p < .001) as coping styles. Lastly, individuals who were more confident in their ability to get support from friends and family were also more likely to seek out social support (partial r = .21, p < .001) and less likely to use distancing (partial r = −.22, p < .001) as coping styles. In addition, and as would be expected, these individuals also were much more likely to report higher levels of perceived social support (partial r = .60, p < .001) and having more people who provide ‘real’ support to them when needed (partial r = .36, p < .001).
To assess their relative independent predictive validity, baseline to Month 3 residual gain scores for each of the three CSE scale scores were regressed simultaneously on to each outcome, controlling for intervention group assignment and the baseline score of the outcome measure. Separate analyses were conducted using outcome data from the Month 3 (end of the intervention phase; N = 300 or 301) and Month 12 (end of the maintenance phase; N = 223) assessments. As noted previously, the WLC participants in Study 1 were crossed over to receive the CET intervention at the end of the intervention phase (Month 3), so they were not included in the Month 12 analyses because they were no longer control group members. The standardized regression coefficients for the individual measures are shown in .
Table 4 Summary of the predictive validity analyses: standardized regression coefficients for the baseline to Month 3 residual gain in the derived CSE scale scores as predictors of psychological distress and well-being, ways of coping, and social support at Months (more ...)
At Month 3, residualized change in self-efficacy for stopping unpleasant emotions and thoughts was more predictive of decreased levels of psychological distress and increased levels of well-being (absolute βs ranged from 0.21 to 0.35, all ps < .001) than were residualized changes in using problem-focused coping or getting support from friends and family (absolute βs ranged from 0.01 to 0.16). As might be expected, for the ways of coping subscales, residualized changes in using problem-focused coping were associated with higher levels of positive reappraisal and planful problem solving (βs = 0.20 and 0.28, respectively; both p < .001). In addition, individuals who reported increased self-efficacy regarding obtaining support from friends and family also reported higher levels of perceived social support (β = 0.27, p < .001).
In general, by Month 12, the association between changes in each of the three CSE scales and the outcome measures had weakened, with several exceptions. Residualized change in self-efficacy for stopping unpleasant emotions and thoughts was predictive of decreased levels of perceived stress (β = −0.20, p < .01) and increased levels of optimism (β = 0.21, p < .01) and positive states of mind (β = 0.21, p < .01). For the ways of coping subscales, residualized changes in using problem-focused coping were associated with lower levels of cognitive escape avoidance (β = −0.24, p < .01).