The present study evaluated the outcomes of public registrants using MoodGYM delivered openly on a website by comparing them with trial participants in a RCT using the same site. We found that there were no differences in the initial level of depression or anxiety in the samples, that the gender balance did not differ significantly, and that the rate of change in symptoms was not different for the two samples. Both public registrants and trial participants exhibited significant improvement in symptoms.
Our findings suggest that MoodGYM can be used effectively in the community. If we had found stronger effects in the BlueMood Trial participants, we might have concluded that structured weekly human contact was a necessary condition for the site to deliver effective outcomes. However, we conclude that community users of the site are similar in gender distribution and severity of depression, and that site exposure results in similar significant symptom improvement.
Although there were no differences in outcome among participants who stayed in the program and completed at least 2 modules, we did find that community users were less likely than the trial participants to adhere to the full treatment program. Thus, the community users infrequently progressed beyond the first module of the program. Among the public, only 15.6% completed 2 or more of the modules, while over 66% of the trial participants completed 2 or more of the modules. This finding suggests that the formal structure of the trial may be important for compliance. Merrill et al reported a similar finding for clinic-based CBT: community clients attended substantially fewer sessions than clients in RCTs, yet “still showed similar levels of improvement” [15
]. Only 15% completed 2 or more modules, a finding that may reflect the usability of the site, the acceptability of CBT type interventions, commitment to change, symptom level, user preference, or other factors. Factors such as these are likely to influence the uptake of non-Internet-based services as well, although comparative published data on potential users of standard health care services are not readily available. The low completion rate is not a major problem for free Internet services, which do not have specific costs based on user numbers (few additional costs are incurred for large numbers of non-completers).
There are limitations to the present study that need to be acknowledged. As is the case for studies using a benchmarking strategy, the samples involved are likely to be quite different given the operation of selection biases. The trial participants were Canberra-based, and from a population sample with high education and occupation levels. In contrast, the user sample is international, with participants from more than 62 countries. There was selective attrition, with many participants from the public sample dropping out before completion, and greater retention for the trial sample (although there were many opportunities for trial participants to be excluded or to drop out before randomization). These sample differences are difficult to characterize. However, they should be acknowledged as having the potential to mask differences in outcomes. Other sample characteristics, such as the concurrent use of evidence-based treatments other than psychotherapy (antidepressants, other medications, physical activity), were not measured but did not preclude participation in either sample.
The possibility cannot be ruled out that community users were assisted in the program by clinicians or other counselors. If this occurred for the majority of community users, our claim that the trial participants were the subject of greater assistance/support would be invalid. However, we consider it highly unlikely that the majority of public participants were assisted by a counselor or other person.
Clinicians and researchers have argued that although treatment efficacy
needs to be established, it is crucial to demonstration of the effectiveness
of treatments in real world settings [15
]. Demonstrations of real-world effectiveness often employ benchmarking strategies where RCTs are chosen to compare results to community settings [16
]. We have employed this strategy in the present study using outcomes from our own RCT. Unlike benchmarking in clinical settings where there can be considerable flexibility in application of the clinical therapy, Internet sites have the advantage of transferring the treatment with fidelity, so that differences that may exist between the trial and the real-world site can be more reliably attributed to external factors such as degree of human contact, capacity to maintain compliance, sample characteristics, and intensity of monitoring.
The findings of the present study demonstrate the effectiveness of Internet-based interventions in the early treatment and prevention of depression. Health systems in developed countries are expected to change radically over the next 10 years, with self-help and self-responsibility for health forming a new tier of the health system [17
]. Sites such as MoodGYM are likely to provide both tools for the self-delivery of evidence-based prevention/treatment and resources to be used as adjuncts to professionally managed primary care.