Multimedia Appendix 3
shows the characteristics of the included studies (N=27). Studies were categorized according to the symptoms or disorder that the intervention targeted as reported by the study authors. The conditions targeted were depression and anxiety (n=7) [36
], anxiety symptoms (n=4) [31
], examination anxiety (n=4) [46
], specific phobia (n=3) [50
], stress (n=2) [53
], social anxiety (n=1) [55
], computer-related anxiety (n=1) [56
], posttraumatic stress (n=1) [57
], generalized anxiety disorder (n=1) [58
], psychological distress (n=1) [59
], hardiness and acculturation (n=1) [60
], and Internet addiction (n=1) [61
Most studies targeting depression and anxiety were conducted in the United States [36
] with the remaining 2 from Australia [39
]. Studies targeting anxiety disorders and stress were conducted in a wider range of countries with 7 from the United States [44
], 4 from Italy [43
], and 1 each from Australia [31
], United Kingdom [48
], Belgium [52
], Spain [55
], and the Netherlands [57
]. Studies targeting other mental health issues were from the United States [60
], United Kingdom [59
], and China [61
Ten studies employed universal interventions, and fewer studies focused on indicated (n=7), selective (n=7), and treatment (n=3) interventions.
Depression and Anxiety Symptoms (n=7)
Of the studies for depression and anxiety symptoms, 43% (3/7) examined selective interventions [39
], with the remaining studies assessing universal (2/7, 29%) [36
], indicated (1/7, 14%) [38
], and treatment (1/7, 14%) [40
] interventions. The 4 selective or indicated studies [38
] evaluated cognitive behavioral therapy (CBT)-based interventions, the 2 universal intervention studies [36
] focused on relationship skills training, and the treatment study [40
] examined the effectiveness of an intervention based on physical activity and Web-based social cognitive theory (SCT) as an adjunct to mental health counseling.
Anxiety Symptoms (n=4)
Of the 4 studies targeting anxiety symptoms, 3 were universal [44
] and 1 was indicated [31
]. The 3 universal studies delivered either relaxation [43
] or exposure-based interventions [44
], and the indicated study delivered CBT [31
Examination Anxiety (n=4)
Of the 4 studies targeting examination anxiety, 2 were universal [47
], 1 was selective [48
], and 1 was indicated [46
]. The 2 universal studies delivered stress inoculation training [47
], the selective study delivered CBT or education interventions [48
], and the indicated study delivered systematic desensitization and relaxation [46
Specific Phobia (n=3)
Of the 3 studies targeting specific phobias, 2 were indicated studies [50
] and 1 was a treatment study [52
]. The 2 indicated studies targeted spider phobia [50
] and acrophobia [51
], and delivered exposure therapy. The treatment study targeted spider phobia and delivered exposure therapy [52
Of the 2 studies targeting stress, 1 was universal [54
] and 1 was selective [53
]. The universal study delivered real and virtual reality physical activity [54
] and the selective study delivered health information and motivational feedback interventions [53
Other Anxiety Disorders (n=4)
The study targeting computer-related anxiety was universal and delivered hypnosis or biofeedback [56
]. The study targeting posttraumatic stress was selective and delivered structured writing [57
]. The study targeting generalized anxiety disorder was indicated and delivered exposure, expressive writing, or auto-photic stimulation (APS) interventions [58
]. The study targeting social anxiety disorder was a treatment study delivering CBT [55
Other Mental Health Problems and Issues (n=3)
For the 3 studies examining other issues, 1 used a universal intervention targeting hardiness and acculturation [60
], 1 was an indicated study focused on Internet addiction [61
], and 1 was a selective study targeting psychological distress [59
]. Two of the studies delivered education [59
], and the remaining study delivered motivational interviewing [61
The 51 interventions examined in the present review employed a range of broad technology types including the Internet (n=18), audio (n=9), virtual reality (n=6), video (n=4), stand-alone computer programs (n=1), and/or a combination of these (Internet plus computer program, n=5; audio plus video, n=5; computer plus audio, n=1; Internet plus audio, n=1; Internet plus APS, n=1). There were no telephone-only interventions. The interventions were delivered using a range of specific devices, including computer (n=24), mobile phone (n=4), Moving Picture Experts Group Layer-3 (MP3) audio file (n=3), Digital Versatile Disc (DVD; n=3), compact disc (CD; n=2), virtual reality devices (n=6), audiotape player (n=4), video player (n=2), and combinations of these, including computer plus audio player (n=2) and computer plus APS (n=1). CBT interventions tended to be Internet-based and were commonly delivered using websites and in conjunction with therapist support in-person or via email. Email was the most common method of monitoring. Educational interventions tended to be delivered using stand-alone computer-based programs and videos. Interventions involving exposure, stress inoculation training, and relaxation tended to be delivered via audio (audiotape, CD, and MP3), combined audio and video (DVD), mobile phone, or virtual reality.
Intervention Length and Delivery
Intervention length ranged from 15 minutes to 10 weeks. For interventions of less than 1 week in duration, intervention length ranged from 15 to 60 minutes (mean 34.23, SD 13.82). For interventions that were 1 week or longer, the mean intervention length was 4.1 weeks (SD 3.04). Length of time to follow-up ranged from immediately postintervention to 12 months postintervention. Of the 51 technology-based interventions employed, 27 (52.9%) were delivered distally, 18 (35.3%) were delivered nondistally, and 6 (11.8%) contained distal and nondistal components. Of the 25 Internet-based interventions, 13 (52.0%) were completely distal, 6 (24.0%) contained a combined distal and nondistal component, and 6 (24.0%) interventions were not distal [39
Level of Human Contact
Over half of the interventions were self-administered (30/51, 59%), and approximately one-fifth were predominantly self-help (10/51, 20%). For interventions that were predominantly self-help, human contact was most commonly provided in the form of email monitoring or moderation of a discussion forum. Four interventions (8%) involved minimal contact and tended to include more intensive therapist involvement via email. Interventions classified as therapist administered (7/51, 14%) were often face-to-face interventions that served as comparison groups to a technology-based intervention, or were face-to-face interventions with a technology-based component as an adjunct (ie, Internet-based homework) [40
By definition, the mean age of participants fell between 18 and 25 years. Most samples were composed solely of undergraduate university students. Two studies targeted specific groups of students: nursing students [44
] and Asian-Indian students [60
]. Females formed the majority of participants in most studies, with the exception of 5 studies in which males were either the majority (n=3) [46
] or the sample contained equal numbers of males and females (n=2) [43
]. The most common recruitment methods were university-wide emails, advertisements in university publications or during lectures, and flyers posted around university campuses. In 7 studies, samples were recruited through undergraduate psychology or health courses [31
Four of the 7 depression and anxiety studies used the Beck Depression Inventory (BDI) and the Beck Anxiety Inventory (BAI) as their primary outcome measures [36
]. Three of the 4 studies targeting anxiety symptoms used the State Trait Anxiety Inventory (STAI) as their primary outcome measure [43
], as did 2 of the 4 studies targeting examination anxiety [47
]. The 2 remaining studies targeting examination anxiety used the Test Anxiety Inventory (TAI) [46
]. Studies targeting specific phobias used the Fear of Spiders Questionnaire [50
], the Acrophobia Questionnaire [51
], and an 11-point fear rating scale [52
]. The 2 studies targeting stress used either the Perceived Stress Scale [53
] or the Momentary Mood States Checklist [54
]. The single studies targeting social anxiety, computer-related anxiety, posttraumatic stress, generalized anxiety disorder, psychological distress, hardiness, acculturation and social support, and Internet addiction used measures specific to the disorder being targeted.
Sample sizes across all studies ranged from 20 to 283 (median 60). Most studies were RCTs (n=26), and 1 study was a randomized trial [46
]. Of the 26 RCTs, 10 studies employed a no-intervention control, 9 studies used attention control groups, 6 studies used a wait-list control, and in 1 study, participants were assigned to a wait-list control but also received treatment as usual. Quality ratings for the studies employing a control group ranged from 1 to 6, with an overall mean rating of 4.42 of a possible 9 points. The mean quality ratings for categories of studies were stress (mean 5.0, range 4-6); other anxiety disorders, such as seasonal affective disorder, posttraumatic stress disorder, generalized anxiety disorder, and computer-related anxiety (mean 5.0, range 4-6); other issues, such as psychological distress, acculturation, and Internet addiction (mean 4.67, range 3-6); depression and anxiety symptoms (mean 4.43, range 3-6); specific phobias (mean 4.33, range 3-5); anxiety symptoms (mean 4.0, range 3-5); and examination anxiety (mean 3.67, range 1-6). shows the number of studies that satisfied each of the quality rating criteria.
Numbers (and percentages) of studies (with control groups) meeting quality rating criteria of the Cochrane Effective Practice and Organisation of Care (EPOC).
As indicated in , few studies used or reported adequate randomization methods. In terms of baseline outcome measurement, more than half of studies reported that there were no significant differences present across study groups at baseline. Less than half of studies, however, reported that the characteristics of the providers of the intervention and control conditions were similar (criteria 4). Approximately one-third of studies reported that they used methods to adequately address incomplete data. No studies met criteria 6 (Was knowledge of the allocated interventions adequately prevented during the study?).
Of the entire 27 studies, 8 studies undertook ITT analyses, and 13 did not. Six studies did not report this information. Of the 8 ITT studies, half (n=4) reported data from a full sample (no attrition), 2 used maximum likelihood estimation methodology, 1 used the last observation carried forward, and 1 used a mixed models analysis.
Depression and Anxiety Symptoms
Among the 7 studies targeting depression and anxiety symptoms, there were 10 interventions that were compared to a control group (some studies had multiple intervention arms). Six interventions were CBT-based (delivered either online or using a stand-alone computer), 2 interventions involved relationship focused skills training, 1 intervention comprised physical activity and SCT, and 1 intervention involved online peer-support.
Postintervention, 3 of the CBT-based interventions were associated with a significant time × group interaction favoring the intervention group on both depression and anxiety symptom outcomes. The remaining CBT interventions (n=3) only found effects for anxiety symptoms postintervention, as did the online peer-support intervention. Only 1 of the relationship skills training interventions found a significant interaction at posttest for depression symptoms [36
]. The second relationship skills training intervention study found a positive effect for anxiety at 10-month follow-up [37
The physical activity and Web-based SCT intervention did not find a significant group × time interaction postintervention for either depression or anxiety [40
Among the 4 studies targeting anxiety symptoms, 9 interventions were examined. Six interventions were relaxation-based: video plus an audio narrative (n=2), video alone (n=1), audio narrative alone (n=2), and virtual reality headset plus audio narrative (n=1). Two interventions were exposure-based: audiotape alone (n=1) and audiotape plus progressive muscle relaxation (n=1). One intervention was CBT-based.
The 2 exposure-based interventions were effective for reducing anxiety relative to a control condition [44
]. Video and audio relaxation combined was associated with significant within-group decline in anxiety symptoms in 1 study [43
], but data was not compared with a control group and this intervention was also not found to be effective in another study [45
Video alone, audio alone, and a virtual reality headset plus an audio narrative were not found to be effective for reducing anxiety symptoms [43
]. The only online CBT intervention was also not associated with a significant interaction in favor of the intervention [31
Among the 4 studies targeting examination anxiety, 11 interventions were examined. Two interventions examined computer-assisted exposure plus audio relaxation, 8 interventions examined stress inoculation delivered by video and audio (n=3), video alone (n=2), and audio alone (n=3), and 1 intervention examined online CBT.
One study examining 4 stress inoculation interventions (video plus audio vs video only vs audio via MP3 only vs audio via CD only) found that all interventions were effective in reducing anxiety symptoms relative to a no-intervention control condition [47
]. Online CBT was also found to be effective for symptoms of examination anxiety [48
]. The study examining exposure plus audio relaxation found that computer-based delivery was equivalent to group-based delivery of the intervention [46
The remaining study examining stress inoculation interventions did not provide sufficient data to determine the effectiveness of the interventions relative to the control group [49
Among the 3 studies targeting specific phobia, 5 interventions were examined. All interventions were exposure-based. Three were delivered using virtual reality and 2 were delivered using video.
Virtual reality exposure interventions for spider phobia [50
] and acrophobia [51
] were associated with significant reductions in anxiety symptoms relative to a control group. Exposure using video was also effective in the treatment of spider phobia [52
Among the 2 studies examining stress, 4 interventions were examined. Interventions included online education (n=1) or online motivational feedback (n=1) [53
], a virtual reality simulation of the outdoors while walking on a treadmill (n=1), or a virtual reality simulation alone (n=1) [54
None of the interventions were effective in reducing stress.
Other Anxiety Disorders
The study targeting social anxiety disorder examined online CBT [55
]. The study targeting computer-related anxiety examined computer-assisted biofeedback [56
]. The study targeting posttraumatic stress examined online structured writing exercises [57
]. The study targeting generalized anxiety disorder examined 3 interventions: online exposure, online expressive writing, and APS [58
Postintervention, online CBT was found to be effective for treating social anxiety disorder [55
], biofeedback was effective for symptoms of computer-related anxiety [56
], structured writing was effective for symptoms of posttraumatic stress [57
], and online exposure and APS were effective for symptoms of generalized anxiety disorder [58
Online expressive writing was not found to be effective for symptoms of generalized anxiety disorder [58
Other Mental Health Problems and Issues
One study targeting psychological distress examined 2 interventions: online education and online education plus an online support group [59
]. One study targeting hardiness, acculturation, and social support examined online information [60
]. One study targeting Internet addiction examined 3 interventions: online motivational interviewing with feedback in a laboratory setting, online motivational interviewing without feedback in a laboratory setting, and online motivational interviewing without feedback in the participant’s own setting [61
An online education intervention and a social support intervention each demonstrated within-group decline in psychological distress over time, but were not compared with a control group [59
]. However, a combined intervention involving both the online education intervention and the support group was not more effective than education alone [59
]. All of the motivational interviewing interventions targeting Internet addiction were associated with significant within-group decline in symptoms over time [61
], but the interaction effect with the control was not tested. However, the control group did not show significant within-group decline over time.
Online information was not found to be effective in the study targeting hardiness and acculturation [60
For interventions targeting depression and anxiety symptoms with available data (n=8), effect sizes ranged from –0.07 to 3.04 (median 0.54; depression = 0.48, anxiety = 0.77). Across interventions targeting anxiety symptoms and disorders with available data (n=10), effect sizes ranged from 0.07 to 2.66 (median 0.84). Because of insufficient or unavailable data, effect sizes were unable to be calculated for 33 of the 51 interventions (64%) or 14 of the 27 studies (52%), which included all of the interventions targeting stress, computer anxiety, psychological distress, hardiness and acculturation, and Internet addiction.
Less than half of studies provided sufficient data to calculate effect sizes. For interventions that targeted depression and anxiety, effect sizes were as follows for the 1 universal (alpha = –0.74), 6 selective (alpha = 0.81), 1 indicated (alpha = 0.54), and 1 treatment (alpha = 0.18) interventions. For interventions targeting anxiety symptoms and disorders, none of the 16 universal interventions (5 trials) had had sufficient data to calculate effect size. Alpha levels were as follows for the 3 selective interventions (alpha = 0.67), 5 indicated interventions (alpha = 0.49), and 2 treatment interventions (alpha = 1.83).
Association Between Positive Outcomes and Study Characteristics
Mann-Whitney U tests demonstrated no association between study outcome favoring the intervention and the following study characteristics: number of intervention sessions (U=35.0, P=.21), length of intervention (weeks; U=43.5, P=.67), sample size (U=40.5, P=.53), and quality rating (U=41.5, P=.56). Fisher exact tests also demonstrated no association between study outcome favoring the intervention and type of control group (attention placebo: n=9; wait list: n=6; no intervention: n=10, P=.84); type of technology used (Internet, includes all interventions that are Internet-based or involved an Internet component: n=34; other: n=21, P=.57); whether or not the intervention was distal (yes, includes completely and partially distal interventions: n=33; no: n=22; P=.74); and amount of human contact (self-administered: n=30; predominantly self-help: n=10; minimal-contact therapy: n=4; predominantly therapist-administered treatments: n=7; P=.30). The success rates of different types of interventions at achieving a study outcome favoring the intervention appeared dissimilar between the universal (n=25, 56%), selective (n=12, 67%), indicated (n=13, 75%), and treatment (n=5, 80%) trials. However, chi-square tests demonstrated that this difference was not significant (P=.74).