This intervention improves on currently-available traditional bibliotherapy in several respects such as multimedia capabilities, automated scoring of questionnaires, and remote accessibility. Despite these advantages, or perhaps even because of these differences, we were unable to detect a main intervention effect for the Internet program. We conducted a series of exploratory analyses among subgroups of participants, in an attempt to better understand why we were unable to find positive effects in the overall sample. However, we also failed to find positive intervention effects among most of the subgroups, although there was a small benefit for those participants who entered the study with lower levels of depression. Because of this possible subgroup benefit, we believe this intervention requires careful examination in further studies.
There are several possible reasons for this overall lack of effect. Because they are newly developed, the Internet intervention materials themselves may have content shortcomings. That is, curative elements that are present in the hard copy bibliotherapy books and pamphlets [6
] may be missing or inappropriately delivered in this Internet program. Despite these specific issues, the medium itself should improve the accessibility of bibliotherapy for a subgroup of patients likely to benefit from guided self-care. We are examining the intervention materials and improving the content for 2 subsequent trials (see below).
A related issue is whether an unattended, psychoeducational approach to delivering behavioral or lifestyle interventions via the Internet is less potent than conducting person-directed counseling through the same media. The previously conducted studies of Internet-based interventions for eating disorders [16
], weight loss [18
], and panic disorder [19
] employed counselors or therapists who composed e-mail or bulletin board (chat) responses to participant questions and problems. It may be that the general absence of benefit seen in this study is due to the lack of the caring, supportive therapeutic alliance that is an integral part of in vivo psychotherapy. However, this doesn't explain how traditional bibliotherapy materials such as books achieve significant effects [5
], since these materials also lack the personal relationship that is characteristic of live therapy.
Another likely reason for the lack of intervention main effects is that the recruitment procedures attracted a more seriously depressed sample than intended, resulting in participants who may have been too depressed to have benefited from such a low intensity, self-help program. Some evidence supports this explanation. Radloff [37
] suggests a "serious depression" cutoff score of 16 or greater on the CES-D, the depression scale employed in this study. She also notes that only 5% of adults in a representative community sample have CES-D scores 28 or greater. This suggests that our sample, with a mean baseline CES-D of approximately 31, would be among the most depressed persons in a normal population. Further evidence comes from the exploratory analyses for persons with low baseline CES-D scores; experimental participants were significantly less depressed than the relevant control subjects at the 16- and 32-week follow-up points. However, given the large number of exploratory analyses in various subgroups, these findings must be viewed as tentative and hypothesis generating. We are preparing a subsequent randomized trial to test this finding in a more systematic manner, by specifically recruiting a much larger sample of participants with low-grade, subdiagnostic depression.
Yet another explanation for the lack of main effects assumes that both the sample and the intervention content were appropriate but that participants did not return to use the Internet site frequently enough to obtain full benefit. This would be analogous to persons who attend only one or two psychotherapy sessions. However, the dose analyses failed to demonstrate that outcome was associated with frequency of use of the Internet site, even when baseline depression severity was controlled. Nonetheless, to address this issue we have just completed enrolling 259 participants in yet another randomized trial similar to this study, but employing several low-intensity methods (e-mail messages, live telephone calls, postcards) to remind subjects randomized to the Internet program to return to the site, complete more of the content, and potentially obtain more benefit.
This study was limited in several ways that may have contributed to the absence of effects. Most importantly, attrition at follow-up was quite high, with follow-up rates at any one point ranging from 53% to 66% (although three quarters of the participants completed at least one follow-up assessment). While the lost participants were similar to retained participants on almost all baseline characteristics (except for of slightly lower depression scores), this high attrition rate reduces our confidence in the observed results. Nonetheless, given that we relied primarily on e-mail assessment reminders and the assessment was conducted via an Internet-administered questionnaire, these retention/attrition rates may represent what can be expected of this new medium where the norm is surfing(rapidly jumping from one Internet site to another).
Another major limitation of this study was our reliance on a single, self-report measure of depression. In our previous depression randomized trials [30
] we have often employed research psychiatric interviews to ascertain diagnoses. However, this methodology is not amenable to the Internet medium at present. We could have conducted research interviews by telephone or in-person, but did not because of cost and effort limitations in this preliminary study. In addition, we were somewhat concerned that the nonspecific, semi-therapeutic impact of several hours of research diagnostic interviewing over the life of the study would potentially swamp the small therapeutic benefit expected from the Internet intervention. This is not an altogether speculative concern, but has been considered as a possible reason for null findings in another, much larger randomized trial in the alcoholism field [38
All these issues remind us that we are at the very threshold of this burgeoning field, and that we know very little about the circumstances and processes that will optimize the delivery and acceptance of these interventions. We are in an unusual situation relative to the development and testing of in vivo psychotherapies. By the time controlled research studies were mounted to test traditional, person-to-person psychotherapy, therapists had been delivering these services in non-research settings for decades and knew a great deal about the process and parameters of service delivery. In contrast, studies of the use of the Internet to deliver mental health interventions are being conducted before widespread, non-research service delivery has occurred via this medium. Therefore, there is no accumulated clinical lore about how to best provide Internet services; we are blazing this trail as we progress. This study, and the others planned and underway, will help address some of these issues.