The primary goal of these two studies was to evaluate the effectiveness of the CDCU in reducing heavy drinking and alcohol-related problems in heavy drinking college students. Overall, these outcome data provide support for our hypothesis that participants who received the CDCU would reduce their drinking more than participants in the control groups at follow-ups. We find it interesting that the magnitude of change in these heavy drinking college students is comparable to that found in our study of the Drinker’s Check-up (DCU), a CDI for older adults (Hester, Squires, & Delaney, 2005
). The reduction in drinking in that study was 45–55% at 12 months and that group showed a continued reduction in drinking from the 8 week to the 12 month follow-up similar to that in our Experiment 1.
So what could account for the clinically meaningful reductions in drinking? Without dismantling studies, we can only offer speculations. First, the safest assumption to make may be that the CDCU’s effectiveness derives in no small part from its implementation of personalized normative feedback (PNF), which has consistently been found to alter normative perceptions, and is frequently associated with reductions in subsequent drinking behavior (Moreira, Smith & Foxcroft, 2009
; Walters, Vader, & Harris, 2007
; Larimer & Cronce, 2007
). Tailored feedback has been found to promote behavior change across a variety of college drinking interventions, and was further enhanced here with the use of gender-specific as well as university-specific norms (Lewis & Neighbors, 2006
Second, the program incorporates the FRAMES elements found in face-to-face BMIs. While a computer program cannot provide complex empathic responses, it can set an empathic and nonjudgmental tone. As with Motivational Interviewing, one’s choice of words is important. During the development of the CDCU Bill Miller consulted with us on this aspect of the program. Third, the CDCU contains two decisional balance exercises, another common element in face-to-face BMIs. The first exercise precedes the assessments and may reduce defensiveness. That exercise is revisited in the third module, Consider Your Options, and extended in an attempt to get the student to think more deeply about his or her drinking.
Fourth, the Consider Your Options module measures the student’s readiness to change and takes that into account in taking them through the process of setting up a plan for changing their drinking. For instance, if the student is not at all ready to change, the program recommends considering the steps that follow (setting up a change plan) to be hypothetical and something the student could refer to in the future should he or she change his or her mind about changing. Future research (discussed below) can hopefully disaggregate the impact of these elements. Fifth, it is also possible that the CDCU engages the student in thinking longer and more deeply about his or her drinking and that this improves outcomes. That would be consistent with Jouriles and colleagues’ finding in which recalling or re-reading personalized feedback immediately following the eCHUG CDI improved short-term outcomes (Jouriles et al., 2010
Finally, the secondary analyses comparing the control groups in the two studies suggests the role that assessment reactivity may have had in Experiment 1 in reducing typical drinking. Although participants were not randomly assigned to experiments, there was no evidence of important pre-existing differences between the control groups in the two studies. The significant differences in typical drinking in these control groups at 1 month but not at baseline can be most parsimoniously explained by assessment reactivity. The reductions in heavier drinking in the two control groups at 1 month could be a function of “maturation,” regression to the mean, the Hawthorne effect, or some combination of the three.
Assessment reactivity is considered differently depending on whether it occurs in the context of clinical research or treatment. Assessment reactivity is a “problem” when it obscures the interpretation and/or validity of clinical trials of protocols. Some have argued this may have occurred during project MATCH, wherein assessment reactivity masked the true impact of the experimental intervention by reducing the between-group differences (Clifford & Maisto, 2000
; Epstein et al., 2004
). It can also make the intervention appear more effective than it might otherwise be without the benefit of the assessment effect.
On the other hand, many have argued that assessment itself may entail therapeutic benefit either as a prelude (e.g., Bien, Miller & Tonigan, 1993
) or an adjunct to treatment (Finn & Tonsager, 1997
). Given the success of BMIs, there has been renewed interest among alcohol researchers in investigating assessment reactivity as a useful clinical tool. Recent studies have examined the effects of assessment as an intervention
. They include studies of the effect on alcohol consumption simply from answering the questions in the AUDIT (Kypri et al., 2006
; McCambridge & Day, 2008
; Walters et al., 2009
); investigations of the relationship between assessment and other aspects of treatment (Maisto, Clifford, & Davis, 2007
; Carey et al., 2006
; Epstein et al., 2004
); and, research exploring whether some methods of assessment are more effective at changing drinking than others (Clifford, Maisto, & Davis, 2007
Whatever the contribution of assessment reactivity to within-group change, the between-group differences in Experiment 1 show that there is more to CDIs than assessment reactivity. The magnitude of the advantage of those receiving the CDCU over the assessment only control group corresponded to an overall between-group effect size of d = .35, averaging across all dependent variables and across the two follow-ups.
Our study has a number of limitations. First, we recruited students through advertising and those who responded may have been more ready to change their drinking than heavy drinkers who did not respond. Second, the data are based on self-report—the validity of the self-report, however, was improved by requiring the name of an SO to corroborate their self-report of drinking, assuring participants of confidentiality of their responses, and verifying their sobriety before collecting any data. Third, in Experiment 2 we measured the delayed assessment control group’s baseline level of drinking at the 1 month follow-up. Although we took steps to be clear about which period of drinking we were asking about (baseline vs. 1 month.), participants’ recall and self-report of their baseline drinking conceivably may have been influenced by their current (i.e. 1 month.) drinking; however, the lack of differences between the reports of baseline drinking in the contemporaneous (Experiment 1) and delayed (Experiment 2) reports of baseline drinking provides support for the validity of the retrospective pretest. One could also argue that our randomization procedures themselves would have resulted in an equal distribution of drinking levels between the two groups in Experiment 2. Fourth, participants were assessed in a clinical setting, our offices, rather than allowing them to use the CDCU online at their leisure (e.g. their dorm room or apartment). The outcomes may not have been as robust if participants were multi-tasking (e.g. texting, web surfing, watching youtube) while at the same time going through the CDCU. For this reason we consider these studies to be effectiveness studies of the CDCU in clinical settings but efficacy studies of the CDCU in non-clinical settings.
Two aspects of the study contribute to its external validity. First, the study sample was diverse, included a large proportion of women and community college as well as four-year university students. Second, the software program used in this study is the same that will be available to future users of the programs. A computer-based intervention, by its nature, interacts with users in the same way over time; there is no "drift" from the protocol.
We need a better understanding of two aspects of CDIs for college students: the elements of our interventions and how we can maximize assessment reactivity to improve outcomes. Dismantling studies are needed to examine the relative contributions of decisional balance exercises, using university and gender specific personalized feedback about both drinking and alcohol-related problems, exercises that engage the student in thinking about his or her drinking, and making plans for changing. With respect to assessment reactivity, we need to add delayed assessment control groups to control for its confounding impact on the interpretation of results. Solomon (1949)
recommended this approach over 50 years ago but it does not seem to be used often. Assessment reactivity itself may be as important as the other elements of our interventions in promoting change. Experimentation with the wording or phrasing of questions or the prelude to the questions themselves may also prove fruitful. As Moos noted in his commentary on recent reactivity research, “we need to understand the context and mechanisms of reactivity in order to minimize it when we want to obtain a ‘true’ estimate of a construct of interest, but also to maximize it when we want to enhance desired behavior change in response to treatment interventions” (Moos, 2008
, p. 250). With heavy drinking and alcohol-related problems on the increase (Hingson, 2010
), we need to continue to find more effective ways to intervene.