PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jmirEditorial BoardMembershipSubmitCurrent IssueArchiveReviewSearchAboutJMIR PublicationsJMIR
 
J Med Internet Res. 2016 May; 18(5): e103.
Published online 2016 May 24. doi:  10.2196/jmir.4643
PMCID: PMC4897296

Effectiveness of a Web-Based Screening and Fully Automated Brief Motivational Intervention for Adolescent Substance Use: A Randomized Controlled Trial

Monitoring Editor: Gunther Eysenbach
Reviewed by Matthijs Blankers and Severin Haug

Abstract

Background

Mid-to-late adolescence is a critical period for initiation of alcohol and drug problems, which can be reduced by targeted brief motivational interventions. Web-based brief interventions have advantages in terms of acceptability and accessibility and have shown significant reductions of substance use among college students. However, the evidence is sparse among adolescents with at-risk use of alcohol and other drugs.

Objective

This study evaluated the effectiveness of a targeted and fully automated Web-based brief motivational intervention with no face-to-face components on substance use among adolescents screened for at-risk substance use in four European countries.

Methods

In an open-access, purely Web-based randomized controlled trial, a convenience sample of adolescents aged 16-18 years from Sweden, Germany, Belgium, and the Czech Republic was recruited using online and offline methods and screened online for at-risk substance use using the CRAFFT (Car, Relax, Alone, Forget, Friends, Trouble) screening instrument. Participants were randomized to a single session brief motivational intervention group or an assessment-only control group but not blinded. Primary outcome was differences in past month drinking measured by a self-reported AUDIT-C-based index score for drinking frequency, quantity, and frequency of binge drinking with measures collected online at baseline and after 3 months. Secondary outcomes were the AUDIT-C-based separate drinking indicators, illegal drug use, and polydrug use. All outcome analyses were conducted with and without Expectation Maximization (EM) imputation of missing follow-up data.

Results

In total, 2673 adolescents were screened and 1449 (54.2%) participants were randomized to the intervention or control group. After 3 months, 211 adolescents (14.5%) provided follow-up data. Compared to the control group, results from linear mixed models revealed significant reductions in self-reported past-month drinking in favor of the intervention group in both the non-imputed (P=.010) and the EM-imputed sample (P=.022). Secondary analyses revealed a significant effect on drinking frequency (P=.037) and frequency of binge drinking (P=.044) in the non-imputation-based analyses and drinking quantity (P=.021) when missing data were imputed. Analyses for illegal drug use and polydrug use revealed no significant differences between the study groups (Ps>.05).

Conclusions

Although the study is limited by a large drop-out, significant between-group effects for alcohol use indicate that targeted brief motivational intervention in a fully automated Web-based format can be effective to reduce drinking and lessen existing substance use service barriers for at-risk drinking European adolescents.

Trial Registration

International Standard Randomized Controlled Trial Registry: ISRCTN95538913; http://www.isrctn.com/ISRCTN95538913 (Archived by WebCite at http://www.webcitation.org/6XkuUEwBx)

Keywords: substance use, adolescents, brief intervention, web-based intervention, motivational interviewing, randomized controlled trial

Introduction

Early misuse of alcohol and other drugs is widespread in Europe with higher prevalence compared to other regions in the world such as the United States [1, 2]. Although temporary substance misuse is a common and partly normative phenomenon in youth development [3,4], adolescence is a critical period for the development of addiction problems. This period is typical for initiation, and rapid escalation of individual problematic substance use patterns into clinically significant problems can be observed among a substantial proportion of youth in Europe [5, 6]. Early excessive drinking and combined use of alcohol with other psychoactive substances (ie, polydrug use) are of particular relevance [7-11] due to the associated adverse effects on physical, psychological, and social functioning that put youth at a heightened risk for long-lasting disadvantages [12-14].

The widespread use of alcohol and other drugs suggests that current capacities to prevent youth from initiating alcohol and other drug use are limited [15, 16]. Prevention efforts should therefore target at-risk youth with indicated preventive interventions [17, 18]. Effective methods to prevent risky substance use and addiction problems are in principle available, but existing health service provision is limited in accessibility and acceptability [19, 20] with the result that interventions are often provided too late and do not reach the majority of at-risk subjects [21, 22].

Web-based intervention programs have been increasingly acknowledged in their capacity to lessen existing service barriers particularly for at-risk populations [23-25]. Moreover, fully automatic delivery (ie, stand-alone or self-guided with no clinician involvement) allow for standardized delivery and can be disseminated cost-effectively at a large scale [24]. Due to the high Internet access rates in contemporary societies and the fact that youth typically use the Internet when searching for information about alcohol and drugs and also are reluctant to disclose alcohol- and drug-related behavior in face-to-face contacts, Web-based interventions hold promise for younger populations [26, 27]. Evidence indicates that fully automated brief motivational interventions can reduce drinking and related harms for emerging adult at-risk drinkers up to 12 months after the intervention [28-31].

The literature on Web-based interventions for illegal drug use is not as developed as it is for alcohol, but a recent meta-analysis (including 10 studies) suggests that overall the effects are somewhat smaller (g=0.16) compared to drinking (g=0.20-0.39) but significant and (as for alcohol interventions) independent of intervention venue (home vs research setting) and level of guidance through the intervention [32-34].

Although previous studies that have proven the usefulness of Web-based motivational interventions to address substance use and related problems mainly targeted emerging adults [35], the motivational methods that have been studied and found effective are relevant for many risk factors in adolescence, such as their susceptibility to peer influences [36-39]. Motivational interventions are based on the therapeutic style and techniques put forward by Motivational Interviewing (MI) [40], which makes a strong case for conceptual compatibility with adolescent-specific needs for autonomy, subjective perceptions of invulnerability to apparent health risks, and appraisal for short-term benefits at the cost of possible long-term adverse effects [17, 37, 38]. Although effects of MI interventions for alcohol misuse and applied techniques in brief interventions such as normative feedback have recently been summarized as rather small [41, 42], relevant previous studies have shown favorable results from face-to-face [43, 44] as well as Web-based MI-interventions among young adults [29-31]. However, studies targeting mid-to-late adolescents (aged 16-18 years) are lacking [45] despite the fact that this is a critical period for establishing problematic alcohol and other drug use [6, 46] and rapid acceleration for first use of illegal drugs [47, 48].

The purpose of this study was therefore to test the effectiveness of a fully automated Web-based brief MI in a sample of at-risk substance-using adolescents in four European countries. Our primary hypothesis was that participants in the intervention group would report significantly lower levels of past-month drinking (frequency, quantity, and frequency of binge drinking) at 3-month follow-up relative to baseline when compared to an assessment-only control group. Additional hypotheses concerned differences in past-month illegal drug use and combined use of alcohol and illegal drugs.

Methods

A two-armed multisite randomized controlled trial (RCT) design was applied in Sweden, Belgium, the Czech Republic, and Germany. Inclusion criteria were being 16-18 years old, online access, informed consent, and a positive screening for at-risk substance use. Baseline assessment was collected at study entry and a follow-up assessment was collected 3 months after baseline assessment. Figure 1 displays the trial design (see Multimedia Appendix 1[49] for the CONSORT EHEALTH checklist).

Figure 1
Participant flow.

Five university research centers in Europe developed the purely Web-based content of the WISEteens portal between June 2011 and March 2012. The IT platform was established together with GAIA AG, Hamburg. The landing page (see Figure 2) was designed to create an appealing first impression using visual material (eg, pictures, video). It described the main features of the study by highlighting confidentiality, content and source credibility, and provided a brief guided enrollment procedure [50]. Key to developing the content was the integration of MI principles and techniques in a single session together with an open-access delivery format applying a design to match end-user characteristics and preferences. Furthermore, the content should be acceptable, easy to use, and perceived as relevant by the target group [50-53].

Figure 2
WISEteens landing page.

All material was developed in a multidisciplinary team including experts in clinical health promotion, developmental and clinical psychology, and certified behavioral therapists. It was first developed in English and then translated by professional offices into the respective countries’ languages. The Web portal was simultaneously launched in all four countries in June 2012 with recruitment until March 2013. Ethical approval was granted by the responsible Ethics Committees in all participating countries: Chamber of Physicians Hamburg (Germany), Prague Psychiatric Centre (Czech Republic), University Hospital of Antwerp and the University of Antwerp (Belgium), and the Regional Ethics Board in Stockholm (Sweden). The trial design was published [54], and the study was registered in a public database. No content or methodological modifications were made after trial commencement.

Recruitment

We promoted the open-access WISEteens landing page to recruit a convenience sample of potential participants using both online and offline strategies. As offline strategies, we developed print promotion materials (information leaflets and flyer cards) and distributed them in schools, youth-clubs, cafés, bars, stores, and adolescent specific events. We also used a three-fold online recruitment strategy with high rank of our websites’ domain in widely used search engines, advertisements via popular social media, and links on affiliated health promotion sites. To motivate study participants and enhance follow-up rates, we promoted and held a prize draw for tablet computers among participants who provided follow-up assessment.

Procedure and Randomization

The participants were anonymous throughout the study. At the first visit, they were asked to register, which required a user name, email address, and a password that did not contain their name. On the landing page, they could choose their respective language flag for a different language than pre-defined by browser options. After registration, respondents were screened for at-risk substance use. Those fulfilling the inclusion criteria then received study information including confidentiality, voluntariness of participation, and data security, as well as information about the randomization protocol. Participants were not blinded to random allocation. After informed online consent, the baseline assessment was completed, including those items that form the base for tailoring intervention content. Participants in the intervention condition received a login code to enable exit and re-entrance. Randomization was generated automatically by an online computer program without stratification. The envisioned number of participants was sufficient to ensure randomization integrity and a likely balanced distribution among the two parallel groups [55].

Screening

An adapted version of the 6-item CRAFFT (Car, Relax, Alone, Forget, Friends, Trouble) tool was used to screen for at-risk use of alcohol and other drugs (see Multimedia Appendix 2). This tool has proven criterion validity compared to other screening tools [56] and is recommended for identification of at-risk adolescents [17]. A CRAFFT score with at least two positive items was the criterion for study inclusion [57].

Intervention

The WISEteens intervention relied on an interactive system to generate individually tailored content. All system-generated information was presented in small units that combined text and graphics (eg, photos and illustrative drawings) and directly referred to the participant’s statements assessed in the first place (eg, substance use, sex, weight, perceptions of normative drinking). Navigation through the program was designed as a dialogue between the user and a virtual expert with “gates” (ie, choice options) at the end of each page to permit varying degrees of approval or disapproval with page content. The system used these responses to introduce subsequent content on the next page.

Intervention content, dialogue tone, and style was based on MI [40] and consisted of the following 6 components (see Multimedia Appendix 3 for screenshots), with the first three applying to alcohol but not illegal drug use and polydrug use: (1) feedback for individual drinking patterns with information on associated health and developmental risks, (2) normative feedback to descriptive drinking norms about sex- and age-matched peer drinking levels using graphed comparative information, (3) feedback for blood alcohol concentration (BAC) and associated health and other risks (ie, traffic crash, unintended sex) for the peak drinking episode in the last 2 weeks, (4) importance and confidence rulers with a short summary and feedback to encourage change readiness and exploration of personal strengths, resources, and volitional strategies for goal attainment, (5) decisional balance for selection of personal costs and benefits of current substance use and a subsequent graphical display of comparative gains and losses of behavior change in a balance sheet to illustrate ambivalence, and (6) identification and selection of personal high-risk situations for substance use and provision of behavioral strategies, for example, to resist peer pressure (the assumed mechanisms for change are displayed in Multimedia Appendix 4; for a more detailed description, see [54]).

The WISEteens intervention was pilot-tested in two steps. First, 10 adolescents chose their preferred design concept among three options. A preliminary version with the preferred “look & feel” was then pre-tested by 37 other adolescents to ensure ease of registration and navigation use, comprehensibility of intervention content, satisfaction with layout and design, appropriateness of dialogue style (eg, avoiding judgmental and confronting language), overall satisfaction with the program, and time to complete baseline assessment and intervention. Furthermore, open feedback, technical problems, translational ambiguities, and other problems were documented and the program was adapted accordingly. Median for assessment and intervention was 15 minutes, ranging from 5-30 minutes. Most adolescents were satisfied or totally satisfied with the system, design, comprehensibility, and intervention dialogue. Table 1 provides a summary of the pilot-results. The control group received assessment only and was directed to a short information page on where to find help in case of urgent counseling or medical needs.

Table 1
Results pilot-test (N=37a).

Data Collection and Measures

All study measures were administered anonymously and online via self-reports at baseline (t0 before randomization) and at follow-up (t1, 3 months after t0) and required registration with a valid email address. Three months after completing the baseline assessment, participants were automatically invited to participate in the follow-up assessment and guided by an integrated hyperlink in the email invitation with one reminder email after 1 week.

Participants were asked for age, gender, country of residence (Sweden, Germany, Belgium, Czech Republic or other), current school attendance (yes/no), parental educational attainment (low, middle, high), their weight in kilograms for BAC-level feedback, descriptive norms [58, 59], and 12-months scores on the Alcohol Use Disorder Identification Test Consumption subscale (AUDIT-C [60]). To address baseline change motivation as a potential confounder, we included intention to reduce drinking (in the next 30 days) using a single item (“I intend to reduce my drinking during the next 30 days”, 1=“totally disagree” to 7=”totally agree”) and intention to abstain from illegal drug use (“I intend to abstain from taking illegal drugs during the next 30 days”, 1=“totally disagree” to 4=“totally agree”) [57], as a continuous measure for change readiness [61].

Substance Use

All outcome measures concerned use in the past 30 days. Change in alcohol use (frequency, frequency of binge drinking, and quantity) between the two assessments was the primary outcome and measured based on the three items of the AUDIT-C screening tool [60]. This measure provides a widely used and valid index sum score for problem alcohol use of adolescents [62]. The three indicators are drinking frequency (“How often did you have a drink containing alcohol?”; 0=“never” to 4=“four or more times a week ), binge drinking frequency (“How often did you have 5 [4 for girls] or more drinks on one occasion, like during a party or on one night?”; 0=“never” to 4=“four or more times a week”), and drinking quantity (“How many drinks containing alcohol did you have on a typical day when you were drinking?”; 0=“one or two” to 4=“ten or more”). To assess the number of consumed drinks, we used a graphical display of various types of drinks with the indication to select the number of each drink per typical drinking occasion to account for national differences in typical standard drinks. Standard drinks were overall defined as containing 10-12 grams of pure ethanol, and responses were recoded to match the original 0-4 point scale. Using an index for drinking has two advantages over separate measures of alcohol use. First, it allows for modeling one drinking measure to increase the statistical power to detect intervention effects while maintaining several indicators of risky drinking. Second, due to the scale, characteristic outcome data can be analyzed as continuous data, which makes interpretation easier compared to discrete count drinking outcomes [63, 64]. We included the three separate outcomes (frequency, frequency of binge drinking, and quantity) and frequency of illegal drug use (1=“never” to 5=“four or more times a week” [65]) as secondary outcomes and past 30 days prevalence of illegal drug use and polydrug use as additional secondary outcomes. Because most drug use combinations in Europe include alcohol [10], we defined polydrug use as a dichotomous measure for use of alcohol and any illegal drug during the last 30 days, similar to previous studies [9].

Sample Size

Sample size calculation was based on the primary outcome with regard to effect sizes for alcohol use revealed by a recent review on Web-based interventions for young people [31]. According to results from similar studies, we expected a small effect size (Cohen’s d=0.2). To reach power of 80% at a type I error rate of 5% in a two-sided test and expecting a dropout rate of approximately 50% [27, 66], we aimed at N=400 per intervention condition [67]. Possible country dependent clustering effects were not included in the sample size calculation because the study was designed as an individual-based RCT and higher level effects from four clusters were not previsioned [68]. Nevertheless, possible higher order country effects were addressed in all further analyses as described below.

Statistical Analyses

We first analyzed data on sample characteristics using t-tests (for metric data) and chi-square tests (for categorical data) to test for differences between intervention conditions. Next, we performed logistic regressions with completers (ie, those who provided valid follow-up data) versus dropouts as the binary dependent variable to test for possible attrition bias using all available sociodemographic and substance use variables as predictors. Intervention effects for primary and continuous secondary outcomes were tested using Linear Mixed Models (LMM) and binary secondary outcomes were analyzed using Generalized Linear Mixed Models. In all LMMs, we used change in outcome scores from baseline as the dependent variable, intervention condition as the only independent variable, baseline values as covariates (“fixed effects”), and country of residence as a single random effect (“random intercept”). This model controls for the correlation between baseline and follow-up outcome scores and does not require a repeated statement (ie, “time”) and no time x group interaction term to interpret intervention effects. Because we found no higher order effect for country of residence (primary outcome: Wald Z=.77, P=.441), we skipped the random effect and adjusted the analyses for country of residence and possible confounders (ie, variables that were not balanced between intervention and control group at baseline) and predictors for missing data as additional covariates, which resulted in improved model fit (delta Bayesian information criterion=-13.16) (see [69] for a similar approach). Binary secondary outcome analyses (prevalence illegal drug use and polydrug use) focused on follow-up outcome values rather than change scores. All analyses are based on a complete-case dataset and an intention-to-treat (ITT) sample with imputation of missing follow-up data based on expectation maximization (EM). Both results are relevant and commonly reported in Web-based interventions particularly when dropout is large [70]. EM is a single imputation method that was shown to outperform the multiple imputation module available in SPSS in eHealth studies with high dropout rates [66]. Given the huge dropout in this study, we cross-checked the ITT-outcome analysis using a full information maximum likelihood (FIML) estimator for missing follow-up data provided in the structural equation model software AMOS to reduce estimate bias for missing data [71] and increase the robustness of findings (see Multimedia Appendix 5). For all analyses, we report estimated marginal means (percentages) and Cohen’s d effect sizes. Distributions of outcomes (ie, skew and kurtosis) and missing-at-random requirements of missing data were checked prior to the main analyses. Results with a type I error rate of P<.05 in 2-sided tests were considered as statistically significant without adjustment for multiple comparisons but reporting of exact Pvalues [72]. All analyses other than in AMOS were performed using SPSS statistical software package version 22 [73].

Results

Sample Characteristics and Preliminary Analyses

The trial profile is shown in Figure 1. A total of 2673 participants logged on the WISEteens Web portal and participated in the initial screening. We excluded 655 (24.5%) from the study due to a negative CRAFFT screening. This resulted in 2018 (75.5%) adolescents who gave consent to participate in the study and started subsequent baseline assessment (t0). A total of 569 (28.1%) dropped out during the baseline assessment leaving 1449 participants who completed baseline assessment and were randomized to either the intervention (N=715) or control group (N=734). In the intervention group, 453 (63.4%) completed the brief intervention as measured by a log file record whether the last page of the intervention has been visible to the user. A total of 211 adolescents participated in the follow-up assessment after 3 months, corresponding to a valid response rate of 14.5%. In this subsample, the completion rate for the brief intervention was higher than in the full randomized sample (82.4%).

In the randomized sample (ITT population), the mean age was 16.8 years (SD 0.74), nearly half of the participants were women and the majority were currently attending school. Most participants were recruited in the Czech Republic due to a more intense offline recruitment in this country, indicating that adjustment of country of residence as an additional covariate was required in subsequent analyses. Participants in the intervention group tended to have a higher rate of parental educational attainment compared to the control group as indicated by a near significant difference (P=.060). This variable was therefore adjusted as an additional covariate in subsequent analyses [68]. With the exception of binge drinking frequency (P=.048), there were no significant group differences with regard to demographic or assessment data at baseline (see Table 2). Importantly, there were no baseline differences in the intention to change current alcohol and illegal drug use among the groups. Distributions of all continuous outcome variables showed acceptable skew and kurtosis with values well below 1.0 for baseline and follow-up assessments [74]. Regarding characteristics of participants who provided data at follow-up, group comparisons revealed no significant differences in any assessed variable (see Table 3). Overall baseline group comparisons thus indicate that the randomization was successful and that the completer-only subsample appears largely similar to the randomized sample.

Table 2
Baseline values for participant demographic and substance use‒related variables by intervention condition (randomized sample N=1449).
Table 3
Baseline values for participant demographic and substance use‒related variables by intervention condition (completers-only sample N=211).

Response rates were very similar for the intervention group (15%) and the control group (14%). Logistic regression analyses with attrition at follow-up as the dependent variable and all demographic (country, parental educational attainment, sex, age) and substance use‒related variables (all primary and secondary outcomes) as predictors explained 8.4% of the total response variance (Nagelkerke’s R2). Corresponding odds ratios (OR) revealed country of residence as the only significant predictors for dropout. Response rates were significantly lower for participants from the Czech Republic (11%) than for those from Sweden (23%, 58/251; OR 2.42, 95% CI 1.52-3.85, P<.001) and Germany (27%, 31/146; OR 2.514, 95% CI 1.41-4.49; P<.001) but not Belgium (18%, 26/143; OR 1.46, 95% CI 0.76-2.80; P=.26). The analysis thus indicates no biased attrition based on the available variables except for country of residence, which was adjusted in all subsequent analyses as a relevant predictor of missing data [75]. Analogous attrition analyses for intervention completion (as indicated by a record whether a user has “seen” the last page of the intervention dialogue) indicated that 12.2% of the total response variance was predicted by all study variables, with significantly more completers being women (P=.007) and significant differences between participants depending on country of residence (P<.001) with the highest intervention completion in Germany (89.7%), followed by Belgium (71.4%), Sweden (69.8), and the Czech Republic (56.0%).

Primary Outcome: Past Month Drinking Index

Tables 4 and and55 report the primary and secondary intervention outcomes of this trial at follow-up based on the non-imputed completer-sample and Tables 6 and and77 for the EM-imputed intention-to-treat-sample. All analyses concern substance use in the past 30 days and were adjusted for baseline scores, country of residence, and parental educational attainment.

Table 4
Intervention effectsa on primary and (continuous) secondary outcomes (non-imputed datasetb).
Table 5
Intervention effectsa on binary secondary outcomes (non-imputed datasetb).
Table 6
Intervention effects on primary and (continuous) secondary outcomes (EM-imputed dataset).
Table 7
Intervention effectsa on binary secondary outcomes (EM-imputed datasetb).

Based on the non-imputed sample 3 months after the intervention, participants in the intervention group have reduced their drinking as indicated by reduced AUDIT-C based scores relative to baseline with an adjusted mean change of -0.85 (95% CI -1.49 to -0.26) while those in the control group slightly increased their drinking as indicated by a mean increase in drinking of 0.16 (95% CI -0.50 to 0.82). Adjusted mean differences between both groups were 1.02 (95% CI 0.25-1.79) and statistically significant (F 1,134=6.8, P=.010, d=.26). The corresponding between-group effect was smaller in the (imputed) ITT analysis due to significant reductions relative to baseline in the control group. However, the significant between-group effect (0.16, 95% CI 0.02-0.25) was maintained (F 1,1329=5.2, P=.022, d=.04). Additional analysis in AMOS based on an FIML estimation for missing outcome assessments confirmed these results (B=-0.72, β=-0.13, P=.046; see Multimedia Appendix 5).

Secondary Outcomes: Drinking (AUDIT-C‒Based Separate Items)

We conducted identical analyses for the three drinking indicators separately. In the non-imputed sample, we found a significant between-group difference in drinking frequency of 0.25 (95% CI 0.02-0.50) in favor of the WISEteens group (F 1,134=4.4, P=.037, d=.15), which was not maintained in the analysis based on EM-imputation (F 1,1329=3.2, P=.073, d=.11) due to significant reductions relative to baseline in both groups (Ps<.001). We obtained a similar result for binge drinking frequency with a significant adjusted mean difference between groups of 0.31 (95% CI 0.01-0.61; F 1,121=4.2, P=.044, d=.16) in the non-imputed data analysis, which was not maintained in the imputed analysis (F 1,1329=2.3, P=130, d=.01). Additional FIML analysis for both outcomes revealed non-significant and similar results as in the imputed analysis (drinking frequency: B=-0.13, β=-0.08, P=.230 and binge drinking frequency: B=-0.25, β=-0.14, P=.059; see Multimedia Appendix 5). For drinking quantity, we found significant reductions relative to baseline for the intervention group in the non-imputed analysis of -0.39 (95% CI -0.72 to -0.06, P=.024), but these reductions were non-significantly different from the control group (between-group differences: 0.31, 95% CI -0.17 to 0.62; F 1,155=1.3, P=.257, d=.13). This effect was significant in the EM-imputed data-set (F 1,1329=5.3, P=.021, d=.05). However, when cross-checked using the FIML approach employed in AMOS, these differences were no longer significant (B=-0.22, β=-0.08, P=.209, see Multimedia Appendix 5).

Secondary Outcomes: Illegal Drug Use and Polydrug Use

Results for frequency and prevalence of illegal drug use as well as polydrug use prevalence are summarized in Tables 5 and and7.7. With regard to these outcomes, we found no statistically significant between-group effects in the non-imputed (Ps=.138-.311) and the EM-imputed (Ps=.363-.871) datasets. Although overall, both groups show numerical decreases between the measurements that were statistically significant in the intervention group for illegal drug use prevalence (P=.025) and polydrug use prevalence (P=.012) in the non-imputed analyses and statistically significant for all 3 outcomes (ie, frequency of illegal drug use and illegal drug use and polydrug prevalence, Ps<.001) in the imputed analyses.

Discussion

Principal Findings

The purpose of this study was to test the effectiveness of a fully automated Web-based screening and brief motivational intervention targeting adolescents with at-risk substance use in Europe.

We found that self-reported risk drinking as measured by a drinking index (ie, drinking frequency, frequency of binge drinking, and typical quantity of drinks) was significantly reduced for participants in the intervention group. The effect on the primary alcohol use outcome was consistent across imputation and non-imputation (“completers”) based analyses but accented in the non-imputed data analysis, even though statistical power was low for 3-month effects due to large loss to follow-up assessment. Secondary analyses using the three drinking indicators as separate outcomes revealed statistically significant mean differences at follow-up in favor of the WISEteens intervention group for drinking frequency and binge drinking frequency but not quantity when missing follow-up data was not imputed. In contrast, analyses using an EM-imputed dataset revealed drinking quantity as the only significant secondary effect. For illegal drug use or polydrug use, there were no significant intervention effects.

The effect sizes obtained in this study are small but match those summarized in recent systematic reviews for fully automated interventions for young adults [32] and meta-analyses for single session interventions [76, 77]. Moreover, they correspond with effect sizes reported for face-to-face brief interventions for youth who use alcohol and other drugs [17, 78, 79] and indicate that expected effects of MI-based interventions on substance use may indeed be small [41] but can be relevant when a large population can be reached. Overall, there are currently few Web-based interventions targeting adolescents, which limits direct comparisons to prior studies. However, our study contradicts results from one recent RCT that tested the effects of a similar intervention (What Do You Drink [WDYD]). This trial targeted drinking among young people (15-20 years) with low educational background in the Netherlands [80]. While WDYD and WISEteens are comparable in central characteristic (eg, age group, cultural context, single session fully automated delivery mode, intervention duration, applied theory, and outcome measures), there are a number of differences that could account for the divergent effects, such as the school-based study implementation and different follow-up times. Moreover, in the WDYD trial participants with an indication of severe problem drinking at baseline were excluded, while in our study about half of the participants were above the AUDIT-C risk cut-off of 5 points [62] at baseline. Severity of baseline drinking can influence effects of brief motivational interventions with stronger effects among subgroups of heavy drinkers [81].

WISEteens was not effective to address illegal drug use and polydrug use adequately. However, the number of participants with drug and polydrug use was rather low in our sample and meaningful between-group effects might have been undetected due to insufficient statistical power [37]. Note that there were notable decreases in the prevalence rates for polydrug use and illegal drug use in the intervention group, while there were no changes and even slight increases in the control group. There were also no spill-over effects from reduced problem drinking to other substance use, which suggests that effects on targeted outcomes may not translate to untargeted outcomes [82, 83].

In general, effects from comparable Web-based interventions for illegal drugs are typically smaller than for alcohol [32]. Moreover, the main hypothesis and focus of the intervention was on drinking. Some intervention elements, such as decisional balance, importance, and confidence ruler and advice for risk situations were available for alcohol per default and participants with other drug use were explicitly encouraged to take the exercises as templates for use of other substances. Thus, the lack of positive outcomes may also be the result of limited specific intervention content for drugs other than alcohol, which may be a limitation. This notwithstanding, the lack of significant effects in our study corresponds to previous trials among students that failed to promote positive behavior change [84, 85], although such interventions can in principle be effective in the general population [32, 86].

Strengths and Limitations

Our study is among the first to report on a targeted fully automated Web-based brief intervention among at-risk adolescents with excessive alcohol drinking and drug use in a randomized controlled trial. From a public health perspective, the significant effect on drinking is relevant. Notwithstanding the often dramatic consequences associated with illegal drug use, alcohol is the most frequently used psycho-active substance during adolescence, alcohol use disorders are among the most prevalent and costly mental disorders in industrialized countries, and prevention is a public health priority [87]. Although the main burden of alcohol-related diseases and injuries becomes apparent in adulthood, it is well documented that early at-risk alcohol use can lead to persistent problems [88,89]. Considering the magnitude of youth with at-risk alcohol use the need for effective targeted prevention may be particularly high in Europe. For example in Germany, hospital admissions due to acute alcohol intoxication have increased substantially in the past years, although the proportion of youth who drink alcohol is decreasing [7]. With this study, we provide initial support for the effectiveness of Web-based brief interventions for adolescents in Europe—an approach that has proven useful in other critical heavy drinking populations, such as college students [31, 33].

The WISEteens intervention was designed to reflect valid face-to-face motivational strategies in a Web-based format. We extended individual feedback techniques typically used in Web-based brief interventions by other MI techniques from face-to-face interventions, such as decisional balance, confidence and importance ruler, and provision of behavioral and regulatory strategies to resist peer pressure [38, 40, 90]. Moreover, we aimed to mimic MI principles by applying a carefully designed and pilot-tested motivational dialogue to reflect possibly “empathic” language, acknowledgement of ambivalent goals, needs for autonomy and self-directedness, and appreciation of change but not necessarily abstinence. Although our results cannot address this issue directly, we assume that this may have contributed to the favorable effects revealed in this study. Certainly we acknowledge the limits in delivering MI-consistent methods such as relational factors in a fully automated format [91]. However, arguably the ability of such interventions to deliver standardized intervention content on a large scale and at low cost may outweigh these limitations.

Finally, even though response to follow-up assessment was very low, screening and intervention were accessed and completed by a reasonably large number of teenagers, which may indicate acceptance and ecological validity in the target group. The intervention completion rate indicates acceptable user engagement in the “real world” [92] and a good balance between the required amount of program exposure and adherence requirements of a single session Web-based delivery format for adolescents [50, 93].

Our study has a number of limitations. First and foremost, results are limited by the higher than expected dropout rate for follow-up assessment, which is a frequent problem in Web-based trials [66, 70]. Dropout might be partly caused by invalid email addresses used by the participants and the fact that the system sent only one email reminder per participant [94]. Even though we detected no serious attrition bias, this may limit the validity of the study findings. Although in case of large dropout, any approach to missing data imputation such as the EM method employed in this study could be compromised. We approached this problem by cross-check analyses using an FIML approach to missing data estimation, which yielded similar and significant results for the primary outcome. Moreover, it must be noted that attrition was mainly a problem for the evaluation (ie, attrition took place between measurement points) and much less for intervention adherence (ie, intervention completion rates). The fact that we were able to identify intervention effects, even with the fewer than required number of participants needed to detect small effects in the evaluation, could be interpreted as an underestimation of intervention effects. This notwithstanding and although the study was performed in four European countries, the results should be taken only under careful consideration and follow-up should be translated to other countries or even to other regions or to other groups of adolescents in the four involved countries. The inconsistent results of the imputed and non-imputed analyses and substantial between-analyses deviations in obtained effect sizes serve as a quantitative indicator of uncertainty in these results due to the substantial amount of missing follow-up data.

Another limitation is reliance on self-reported data in this study, which are often associated with underreporting of alcohol drinking and other drug use [95]. Measures to avoid underreporting were assurance of and advice for confidentiality and a non-judgmental and non-confronting MI style employed throughout the intervention. Moreover in our study, the self-report was given anonymously and without personal contact, which may add to the reliability of self-reported data. Moreover, we relied on a primary outcome that was based on the AUDIT-C, which was developed as a screening tool for harmful alcohol use in adults. Given the adapted reference time of drinking in the past 30 days, it is difficult to interpret the practical significance of the measured changes in outcome scores in response to the intervention. Furthermore, participants were not blinded to the assigned interventions, which is a common limitation in Web-based trials [96].

Finally, given the focus on intervention effects in this study, we have not systematically included recruitment and reach in our analyses although these are important issues for estimating public health impact [97]. Rather, we aimed for a convenience sample from the general population by employing a pragmatic recruitment procedure. However, the incitement by lottery as an incentive for participation may have increased the reach to a higher level than can be expected in implementation outside a research project. While the self-selected nature of our sample again limits generalization, we consider the realistic setting of this trial a significant strength. In fact, apart from the evaluation requirements at baseline, the actual intervention program was equivalent to a potential real world application. We thus feel confident in stating that our study has realistic public health implications.

Conclusion

Web-based interventions to reduce adolescent at-risk substance use hold promise for accessibility and large scale dissemination but have rarely been tested in randomized controlled trials. Although limited by substantial dropout to follow-up assessment, our findings imply that young adolescents with excessive drinking can benefit from targeted interventions based on MI techniques and counseling style in a fully automated Web-based delivery mode. However, we found no effect on drug use, which calls for further research on effective intervention models, delivery modes, and recruitment strategies.

Acknowledgments

The authors would like to especially thank Sonja Bröning, Magdalena Drechsel, Helena Hansson, and Ulla Zetterlind for their contribution to development of the intervention program; Petr Winkler and Nicklas Kartengren for assistance in data collection; Anne Berman and Veerle Soyez for project advisory; Hamoud Gahl for technical implementation; and Anne Daubmann and Levente Kriston for advice on data analysis.

The study was funded by the Drug Prevention and Information Programme of the European Union (Grant agreement No. JUST/2010/DPIP/AG/0914-30-CE-0379823/00-48).

Abbreviations

AUDIT-C
Alcohol Use Disorder Identification Test Consumption subscale
BAC
blood alcohol concentration
CONSORT
Consolidated Standards of Reporting Trials
EM
expectation maximization
FIML
full information maximum likelihood
ITT
intention-to-treat analyses
LMM
Linear Mixed Model
MI
motivational interviewing
RCT
randomized control trial
WDYD
What Do You Drink

Multimedia Appendix 1

CONSORT-eHEALTH V 1.6.1.

Multimedia Appendix 2

CRAFFT Screening for at-risk substance use (A CRAFFT score of more than one positive item was inclusion criterion).

Multimedia Appendix 3

Screenshots.

Multimedia Appendix 4

Assumed behavior change model and targeted WISEteens content.

Multimedia Appendix 5

Intervention effects at 3-month follow-up, ITT-analysis based on FIML.

Footnotes

Conflicts of Interest:

Conflicts of Interest: The authors were involved in the development of the intervention.

References

1. Bekkering GE, Aertgeerts B, Asueta-Lorente J, Autrique M, Goossens M, Smets K, van Bussel JCH, Vanderplasschen W, Van RP, Hannes K. Practitioner review: evidence-based practice guidelines on alcohol and drug misuse among adolescents: a systematic review. J Child Psychol Psychiatry. 2014 Jan;55(1):3–21. doi: 10.1111/jcpp.12145. [PubMed] [Cross Ref]
2. Lim SS, Vos T, Flaxman AD, Danaei G, Shibuya K, Adair-Rohani H, Amann M, Anderson HR, Andrews KG, Aryee M, Atkinson C, Bacchus LJ, Bahalim AN, Balakrishnan K, Balmes J, Barker-Collo S, Baxter A, Bell ML, Blore JD, Blyth F, Bonner C, Borges G, Bourne R, Boussinesq M, Brauer M, Brooks P, Bruce NG, Brunekreef B, Bryan-Hancock C, Bucello C, Buchbinder R, Bull F, Burnett RT, Byers TE, Calabria B, Carapetis J, Carnahan E, Chafe Z, Charlson F, Chen H, Chen JS, Cheng AT, Child JC, Cohen A, Colson KE, Cowie BC, Darby S, Darling S, Davis A, Degenhardt L, Dentener F, Des Jarlais DC, Devries K, Dherani M, Ding EL, Dorsey ER, Driscoll T, Edmond K, Ali SE, Engell RE, Erwin PJ, Fahimi S, Falder G, Farzadfar F, Ferrari A, Finucane MM, Flaxman S, Fowkes FGR, Freedman G, Freeman MK, Gakidou E, Ghosh S, Giovannucci E, Gmel G, Graham K, Grainger R, Grant B, Gunnell D, Gutierrez HR, Hall W, Hoek HW, Hogan A, Hosgood HD, Hoy D, Hu H, Hubbell BJ, Hutchings SJ, Ibeanusi SE, Jacklyn GL, Jasrasaria R, Jonas JB, Kan H, Kanis JA, Kassebaum N, Kawakami N, Khang Y, Khatibzadeh S, Khoo J, Kok C, Laden F, Lalloo R, Lan Q, Lathlean T, Leasher JL, Leigh J, Li Y, Lin JK, Lipshultz SE, London S, Lozano R, Lu Y, Mak J, Malekzadeh R, Mallinger L, Marcenes W, March L, Marks R, Martin R, McGale P, McGrath J, Mehta S, Mensah GA, Merriman TR, Micha R, Michaud C, Mishra V, Mohd HK, Mokdad AA, Morawska L, Mozaffarian D, Murphy T, Naghavi M, Neal B, Nelson PK, Nolla JM, Norman R, Olives C, Omer SB, Orchard J, Osborne R, Ostro B, Page A, Pandey KD, Parry CDH, Passmore E, Patra J, Pearce N, Pelizzari PM, Petzold M, Phillips MR, Pope D, Pope CA, Powles J, Rao M, Razavi H, Rehfuess EA, Rehm JT, Ritz B, Rivara FP, Roberts T, Robinson C, Rodriguez-Portales JA, Romieu I, Room R, Rosenfeld LC, Roy A, Rushton L, Salomon JA, Sampson U, Sanchez-Riera L, Sanman E, Sapkota A, Seedat S, Shi P, Shield K, Shivakoti R, Singh GM, Sleet DA, Smith E, Smith KR, Stapelberg NJC, Steenland K, Stöckl Heidi, Stovner LJ, Straif K, Straney L, Thurston GD, Tran JH, Van DR, van DA, Veerman JL, Vijayakumar L, Weintraub R, Weissman MM, White RA, Whiteford H, Wiersma ST, Wilkinson JD, Williams HC, Williams W, Wilson N, Woolf AD, Yip P, Zielinski JM, Lopez AD, Murray CJL, Ezzati M, AlMazroa MA, Memish ZA. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012 Dec 15;380(9859):2224–2260. doi: 10.1016/S0140-6736(12)61766-8. http://europepmc.org/abstract/MED/23245609. [PMC free article] [PubMed] [Cross Ref]
3. Masten AS, Faden VB, Zucker RA, Spear LP. Underage drinking: a developmental framework. Pediatrics. 2008 Apr;121 Suppl 4:S235–251. doi: 10.1542/peds.2007-2243A. [PubMed] [Cross Ref]
4. Thomasius R, Stolle M, Sack P. Entwicklungspsychopathologisches Modell. In: Thomasius R, Schulte-Markwort M, Küstner UJ, Riedesser P, editors. Handbuch der Suchtstörungen im Kindes- und Jugendalter. Stuttgart: Schattauer; 2009. pp. 139–146.
5. Heron J, Macleod J, Munafò MR, Melotti R, Lewis G, Tilling K, Hickman M. Patterns of alcohol use in early adolescence predict problem use at age 16. Alcohol Alcohol. 2012;47(2):169–177. doi: 10.1093/alcalc/agr156. http://alcalc.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=22215001. [PMC free article] [PubMed] [Cross Ref]
6. Wittchen H, Behrendt S, Höfler M, Perkonigg A, Lieb R, Bühringer G, Beesdo K. What are the high risk periods for incident substance use and transitions to abuse and dependence? Implications for early intervention and prevention. Int J Methods Psychiatr Res. 2008 Jun;17 Suppl 1:S16–29. doi: 10.1002/mpr.254. [PubMed] [Cross Ref]
7. Diestelkamp S, Arnaud N, Sack P-M, Wartberg L, Daubmann A, Thomasius R. Brief motivational intervention for adolescents treated in emergency departments for acute alcohol intoxication - a randomized-controlled trial. BMC Emerg Med. 2014;14:13. doi: 10.1186/1471-227X-14-13. http://bmcemergmed.biomedcentral.com/articles/10.1186/1471-227X-14-13. [PMC free article] [PubMed] [Cross Ref]
8. Hibell B, Guttormsson U, Ahlström S, Balakireva O, Bjarnason T, Kokkevi A. ESPAD report. Stockholm: The Swedish council for information on alcohol and other drugs (CAN); 2012. [2016-04-18]. The 2011 ESPAD report: Substance use among students in 36 European countries http://www.espad.org/uploads/espad_reports/2011/the_2011_espad_report_full_2012_10_29.pdf[accessed webcite.
9. Olszewski D, Matias J, Monshouwer K, Kokkevi A. Polydrug use among 15- to 16-year olds: Similarities and differences in Europe. Drugs: Education, Prevention and Policy. 2010 Jun 30;17(4):287–302. doi: 10.3109/09687630902806715. [Cross Ref]
10. EMCDDA . European Drug Report. Lisbon: European Monitoring Centre for Drugs and Drug Addiction (EMCDDA); 2009. [2016-04-18]. Annual Report 2009: The State of the Drug Problem http://www.emcdda.europa.eu/publications/annual-report/2009[accessed webcite. [PubMed]
11. Viner RM, Taylor B. Adult outcomes of binge drinking in adolescence: findings from a UK national birth cohort. J Epidemiol Community Health. 2007 Oct;61(10):902–907. doi: 10.1136/jech.2005.038117. http://europepmc.org/abstract/MED/17873228. [PMC free article] [PubMed] [Cross Ref]
12. Van Der Vorst H, Vermulst AA, Meeus WHJ, Deković M, Engels RCME. Identification and prediction of drinking trajectories in early and mid-adolescence. J Clin Child Adolesc Psychol. 2009 May;38(3):329–341. doi: 10.1080/15374410902851648. [PubMed] [Cross Ref]
13. Pharo H, Sim C, Graham M, Gross J, Hayne H. Risky business: executive function, personality, and reckless behavior during adolescence and emerging adulthood. Behav Neurosci. 2011 Dec;125(6):970–978. doi: 10.1037/a0025768. [PubMed] [Cross Ref]
14. Miller JW, Naimi TS, Brewer RD, Jones SE. Binge drinking and associated health risk behaviors among high school students. Pediatrics. 2007 Jan;119(1):76–85. doi: 10.1542/peds.2006-1517. [PubMed] [Cross Ref]
15. Hale DR, Fitzgerald-Yau N, Viner RM. A systematic review of effective interventions for reducing multiple health risk behaviors in adolescence. Am J Public Health. 2014 May;104(5):e19–41. doi: 10.2105/AJPH.2014.301874. [PubMed] [Cross Ref]
16. Bühler A, Thrul J. Expertise zur Suchtprävention - Aktualisierte und erweiterte Neuauflage der Expertise zur Prävention des Substanzmissbrauchs. Köln: BZgA; 2013.
17. Patton R, Deluca P, Kaner E, Newbury-Birch D, Phillips T, Drummond C. Alcohol screening and brief intervention for adolescents: the how, what and where of reducing alcohol consumption and related harm among young people. Alcohol Alcohol. 2014;49(2):207–212. doi: 10.1093/alcalc/agt165. http://alcalc.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=24232178. [PMC free article] [PubMed] [Cross Ref]
18. Meier MH, Caspi A, Houts R, Slutske WS, Harrington H, Jackson KM, Belsky DW, Poulton R, Moffitt TE. Prospective developmental subtypes of alcohol dependence from age 18 to 32 years: implications for nosology, etiology, and intervention. Dev Psychopathol. 2013 Aug;25(3):785–800. doi: 10.1017/S0954579413000175. http://europepmc.org/abstract/MED/23880392. [PMC free article] [PubMed] [Cross Ref]
19. Van HS, Harris SK, Brooks T, Carey P, Kossack R, Kulig J, Knight JR, New England Partnership for Substance Abuse Research The "Six T's": barriers to screening teens for substance abuse in primary care. J Adolesc Health. 2007 May;40(5):456–461. doi: 10.1016/j.jadohealth.2006.12.007. [PubMed] [Cross Ref]
20. Rumpf H. Frühintervention bei problematischem Alkoholkonsum in der medizinischen Versorgung: Auf dem Weg zur Implementierung. SUCHT. 2009 Jan 21;55(6):326–327. doi: 10.1024/2009.06.01. [Cross Ref]
21. Cunningham J, Breslin F. Only one in three people with alcohol abuse or dependence ever seek treatment. Addict Behav. 2004 Jan;29(1):221–223. [PubMed]
22. Flory K, Lynam D, Milich R, Leukefeld C, Clayton R. Early adolescent through young adult alcohol and marijuana use trajectories: early predictors, young adult outcomes, and predictive utility. Dev Psychopathol. 2004;16(1):193–213. [PubMed]
23. Murray E. Web-based interventions for behavior change and self-management: potential, pitfalls, and progress. Med 2 0. 2012;1(2):e3. doi: 10.2196/med20.1741. http://www.medicine20.com/2012/2/e3/ [PMC free article] [PubMed] [Cross Ref]
24. Riper H, Andersson G, Christensen H, Cuijpers P, Lange A, Eysenbach G. Theme issue on e-mental health: a growing field in internet research. J Med Internet Res. 2010;12(5):e74. doi: 10.2196/jmir.1713. http://www.jmir.org/2010/5/e74/ [PMC free article] [PubMed] [Cross Ref]
25. Ritterband LM, Tate DF. The science of internet interventions. Introduction. Ann Behav Med. 2009 Aug;38(1):1–3. doi: 10.1007/s12160-009-9132-5. [PubMed] [Cross Ref]
26. Burns JM, Davenport TA, Durkin LA, Luscombe GM, Hickie IB. The internet as a setting for mental health service utilisation by young people. Med J Aust. 2010 Jun 7;192(11 Suppl):S22–26. [PubMed]
27. Spijkerman R, Roek MAE, Vermulst A, Lemmers L, Huiberts A, Engels RCME. Effectiveness of a web-based brief alcohol intervention and added value of normative feedback in reducing underage drinking: a randomized controlled trial. J Med Internet Res. 2010;12(5):e65. doi: 10.2196/jmir.1465. http://www.jmir.org/2010/5/e65/ [PMC free article] [PubMed] [Cross Ref]
28. Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010;12(1):e4. doi: 10.2196/jmir.1376. http://www.jmir.org/2010/1/e4/ [PMC free article] [PubMed] [Cross Ref]
29. White A, Kavanagh D, Stallman H, Klein B, Kay-Lambkin F, Proudfoot J, Drennan J, Connor J, Baker A, Hines E, Young R. Online alcohol interventions: a systematic review. J Med Internet Res. 2010;12(5):e62. doi: 10.2196/jmir.1479. http://www.jmir.org/2010/5/e62/ [PMC free article] [PubMed] [Cross Ref]
30. Bewick BM, Trusler K, Barkham M, Hill AJ, Cahill J, Mulhern B. The effectiveness of web-based interventions designed to decrease alcohol consumption--a systematic review. Prev Med. 2008 Jul;47(1):17–26. doi: 10.1016/j.ypmed.2008.01.005. [PubMed] [Cross Ref]
31. Tait RJ, Christensen H. Internet-based interventions for young people with problematic substance use: a systematic review. Med J Aust. 2010 Jun 7;192(11 Suppl):S15–21. [PubMed]
32. Tait RJ, Spijkerman R, Riper H. Internet and computer based interventions for cannabis use: a meta-analysis. Drug Alcohol Depend. 2013 Dec 1;133(2):295–304. doi: 10.1016/j.drugalcdep.2013.05.012. [PubMed] [Cross Ref]
33. Riper H, Spek V, Boon B, Conijn B, Kramer J, Martin-Abello K, Smit F. Effectiveness of E-self-help interventions for curbing adult problem drinking: a meta-analysis. J Med Internet Res. 2011;13(2):e42. doi: 10.2196/jmir.1691. http://www.jmir.org/2011/2/e42/ [PMC free article] [PubMed] [Cross Ref]
34. Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, Ebert D, Cuijpers P. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One. 2014;9(6):e99912. doi: 10.1371/journal.pone.0099912. http://dx.plos.org/10.1371/journal.pone.0099912. [PMC free article] [PubMed] [Cross Ref]
35. Carey KB, Scott-Sheldon LAJ, Carey MP, DeMartini KS. Individual-level interventions to reduce college student drinking: a meta-analytic review. Addict Behav. 2007 Nov;32(11):2469–2494. doi: 10.1016/j.addbeh.2007.05.004. http://europepmc.org/abstract/MED/17590277. [PMC free article] [PubMed] [Cross Ref]
36. Fairlie AM, Wood MD, Laird RD. Prospective protective effect of parents on peer influences and college alcohol involvement. Psychol Addict Behav. 2012 Mar;26(1):30–41. doi: 10.1037/a0023879. http://europepmc.org/abstract/MED/21574670. [PMC free article] [PubMed] [Cross Ref]
37. Bernstein J, Heeren T, Edward E, Dorfman D, Bliss C, Winter M, Bernstein E. A brief motivational interview in a pediatric emergency department, plus 10-day telephone follow-up, increases attempts to quit drinking among youth and young adults who screen positive for problematic drinking. Acad Emerg Med. 2010 Aug;17(8):890–902. doi: 10.1111/j.1553-2712.2010.00818.x. http://dx.doi.org/10.1111/j.1553-2712.2010.00818.x. [PMC free article] [PubMed] [Cross Ref]
38. O'Leary TT, Monti P. Motivational enhancement and other brief interventions for adolescent substance abuse: foundations, applications and evaluations. Addiction. 2004 Nov;99 Suppl 2:63–75. doi: 10.1111/j.1360-0443.2004.00855.x. [PubMed] [Cross Ref]
39. Oetting E, Beauvais F. Peer cluster theory, socialization characteristics and adolescent drug use: A path analysis. J Couns Psychol. 1987;34(2):205–213. http://psycnet.apa.org/doi/10.1037/0022-0167.34.2.205.
40. Miller W, Rollnick S. Motivational Interviewing, Second Edition: Preparing People for Change. New York: The Guilford Press; 2002.
41. Foxcroft DR, Coombes L, Wood S, Allen D, Almeida Santimano NML. Motivational interviewing for alcohol misuse in young adults. Cochrane Database Syst Rev. 2014;8:CD007025. doi: 10.1002/14651858.CD007025.pub2. [PubMed] [Cross Ref]
42. Foxcroft DR, Moreira MT, Almeida Santimano NML, Smith LA. Social norms information for alcohol misuse in university and college students. Cochrane Database Syst Rev. 2015;1:CD006748. doi: 10.1002/14651858.CD006748.pub3. [PubMed] [Cross Ref]
43. Spirito A, Sindelar-Manning H, Colby SM, Barnett NP, Lewander W, Rohsenow DJ, Monti PM. Individual and family motivational interventions for alcohol-positive adolescents treated in an emergency department: results of a randomized clinical trial. Arch Pediatr Adolesc Med. 2011 Mar;165(3):269–274. doi: 10.1001/archpediatrics.2010.296. http://europepmc.org/abstract/MED/21383276. [PMC free article] [PubMed] [Cross Ref]
44. Tripodi SJ, Bender K, Litschge C, Vaughn MG. Interventions for reducing adolescent alcohol abuse: a meta-analytic review. Arch Pediatr Adolesc Med. 2010 Jan;164(1):85–91. doi: 10.1001/archpediatrics.2009.235. [PubMed] [Cross Ref]
45. Tønnesen H, Ståhlbrandt H, Pedersen B. Web-based brief interventions for young adolescent alcohol and drug abusers – a systematic review. Clin Health Promot. 2013:3–82.
46. Heron J, Barker ED, Joinson C, Lewis G, Hickman M, Munafò M, Macleod J. Childhood conduct disorder trajectories, prior risk factors and cannabis use at age 16: birth cohort study. Addiction. 2013 Dec;108(12):2129–2138. doi: 10.1111/add.12268. http://dx.doi.org/10.1111/add.12268. [PMC free article] [PubMed] [Cross Ref]
47. Vega WA, Aguilar-Gaxiola S, Andrade L, Bijl R, Borges G, Caraveo-Anduaga JJ, DeWit DJ, Heeringa SG, Kessler RC, Kolody B, Merikangas KR, Molnar BE, Walters EE, Warner LA, Wittchen H. Prevalence and age of onset for drug use in seven international sites: results from the international consortium of psychiatric epidemiology. Drug Alcohol Depend. 2002 Dec 1;68(3):285–297. [PubMed]
48. Chen K, Kandel DB. The natural history of drug use from adolescence to the mid-thirties in a general population sample. Am J Public Health. 1995 Jan;85(1):41–47. [PubMed]
49. Eysenbach G, CONSORT- E. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. 2011;13(4):e126. doi: 10.2196/jmir.1923. http://www.jmir.org/2011/4/e126/ [PMC free article] [PubMed] [Cross Ref]
50. Danaher BG, Seeley JR. Methodological issues in research on web-based behavioral interventions. Ann Behav Med. 2009 Aug;38(1):28–39. doi: 10.1007/s12160-009-9129-0. http://europepmc.org/abstract/MED/19806416. [PMC free article] [PubMed] [Cross Ref]
51. Danaher BG, Smolkowski K, Seeley JR, Severson HH. Mediators of a successful web-based smokeless tobacco cessation program. Addiction. 2008 Oct;103(10):1706–1712. doi: 10.1111/j.1360-0443.2008.02295.x. http://europepmc.org/abstract/MED/18715238. [PMC free article] [PubMed] [Cross Ref]
52. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JEWC. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012;14(6):e152. doi: 10.2196/jmir.2104. http://www.jmir.org/2012/6/e152/ [PMC free article] [PubMed] [Cross Ref]
53. Crutzen R, de NJ, Brouwer W, Oenema A, Brug J, de Vries NK. A conceptual framework for understanding and improving adolescents' exposure to Internet-delivered interventions. Health Promot Int. 2009 Sep;24(3):277–284. doi: 10.1093/heapro/dap018. http://heapro.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=19515716. [PubMed] [Cross Ref]
54. Arnaud N, Bröning S, Drechsel M, Thomasius R, Baldus C. Web-based screening and brief intervention for poly-drug use among teenagers: study protocol of a multicentre two-arm randomized controlled trial. BMC Public Health. 2012;12:826. doi: 10.1186/1471-2458-12-826. http://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-12-826. [PMC free article] [PubMed] [Cross Ref]
55. Lachin JM, Matts JP, Wei LJ. Randomization in clinical trials: conclusions and recommendations. Control Clin Trials. 1988 Dec;9(4):365–374. [PubMed]
56. Levy S, Sherritt L, Harris SK, Gates EC, Holder DW, Kulig JW, Knight JR. Test-retest reliability of adolescents' self-report of substance use. Alcohol Clin Exp Res. 2004 Aug;28(8):1236–1241. [PubMed]
57. Knight JR, Sherritt L, Harris SK, Gates EC, Chang G. Validity of brief alcohol screening tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res. 2003 Jan;27(1):67–73. doi: 10.1097/01.ALC.0000046598.59317.3A. [PubMed] [Cross Ref]
58. Schmiege SJ, Broaddus MR, Levin M, Bryan AD. Randomized trial of group interventions to reduce HIV/STD risk and change theoretical mediators among detained adolescents. J Consult Clin Psychol. 2009 Feb;77(1):38–50. doi: 10.1037/a0014513. [PubMed] [Cross Ref]
59. Wood MD, Fairlie AM, Fernandez AC, Borsari B, Capone C, Laforge R, Carmona-Barros R. Brief motivational and parent interventions for college students: a randomized factorial study. J Consult Clin Psychol. 2010 Jun;78(3):349–361. doi: 10.1037/a0019166. http://europepmc.org/abstract/MED/20515210. [PMC free article] [PubMed] [Cross Ref]
60. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II. Addiction. 1993 Jun;88(6):791–804. [PubMed]
61. Fishbein M, Ajzen I. Predicting and changing behavior: the reasoned action approach. New York: Psychology Press; 2010.
62. Rumpf H, Wohlert T, Freyer-Adam J, Grothues J, Bischof G. Screening questionnaires for problem drinking in adolescents: performance of AUDIT, AUDIT-C, CRAFFT and POSIT. Eur Addict Res. 2013;19(3):121–127. doi: 10.1159/000342331. [PubMed] [Cross Ref]
63. Zhang S, Paul J, Nantha-Aree M, Buckley N, Shahzad U, Cheng J, DeBeer J, Winemaker M, Wismer D, Punthakee D, Avram V, Thabane L. Empirical comparison of four baseline covariate adjustment methods in analysis of continuous outcomes in randomized controlled trials. Clin Epidemiol. 2014;6:227–235. doi: 10.2147/CLEP.S56554. http://dx.doi.org/10.2147/CLEP.S56554. [PMC free article] [PubMed] [Cross Ref]
64. Kypri K. Methodological issues in alcohol screening and brief intervention research. Subst Abus. 2007;28(3):31–42. doi: 10.1300/J465v28n03_04. [PubMed] [Cross Ref]
65. Berman AH, Bergman H, Palmstierna T, Schlyter F. Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample. Eur Addict Res. 2005;11(1):22–31. doi: 10.1159/000081413. [PubMed] [Cross Ref]
66. Blankers M, Koeter MWJ, Schippers GM. Missing data approaches in eHealth research: simulation study and a tutorial for nonmathematically inclined researchers. J Med Internet Res. 2010;12(5):e54. doi: 10.2196/jmir.1448. http://www.jmir.org/2010/5/e54/ [PMC free article] [PubMed] [Cross Ref]
67. Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007 May;39(2):175–191. [PubMed]
68. Maas CJM, Hox JJ. Robustness issues in multilevel regression analysis. Statistica Neerlandica. 2004;58:127–137. doi: 10.1046/j.0039-0402.2003.00252.x. [Cross Ref]
69. Springvloet L, Lechner L, de VH, Candel MJ, Oenema A. Short- and medium-term efficacy of a Web-based computer-tailored nutrition education intervention for adults including cognitive and environmental feedback: randomized controlled trial. J Med Internet Res. 2015;17(1):e23. doi: 10.2196/jmir.3837. http://www.jmir.org/2015/1/e23/ [PMC free article] [PubMed] [Cross Ref]
70. Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11. doi: 10.2196/jmir.7.1.e11. http://www.jmir.org/2005/1/e11/ [PMC free article] [PubMed] [Cross Ref]
71. Little TD, Schnabel KU, Baumert J. Modeling longitudinal and multilevel data: practical issues, applied approaches, and specific examples. Mahwah, NJ: Lawrence Erlbaum Assoc; 2000. Longitudinalmulti-group modelling with missing data.
72. Maher C, Ferguson M, Vandelanotte C, Plotnikoff R, De BI, Thomas S, Nelson-Field K, Olds T. A Web-Based, Social Networking Physical Activity Intervention for Insufficiently Active Adults Delivered via Facebook App: Randomized Controlled Trial. J Med Internet Res. 2015;17(7):e174. doi: 10.2196/jmir.4086. http://www.jmir.org/2015/7/e174/ [PMC free article] [PubMed] [Cross Ref]
73. IBM Corp . http://www-01.ibm.com/support/docview.wss?uid=swg21476197. Armonk, NY: IBM Corp; 2012. IBM SPSS Statistics for Windows, Version 21.0.
74. Kline RB. Principles and practice of structural equation modeling. New York: Guilford Press; 2005.
75. Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med. 2002 Oct 15;21(19):2917–2930. doi: 10.1002/sim.1296. [PubMed] [Cross Ref]
76. Riper H, Spek V, Boon B, Conijn B, Kramer J, Martin-Abello K, Smit F. Effectiveness of E-self-help interventions for curbing adult problem drinking: a meta-analysis. J Med Internet Res. 2011;13(2):e42. doi: 10.2196/jmir.1691. http://www.jmir.org/2011/2/e42/ [PMC free article] [PubMed] [Cross Ref]
77. Riper H, van SA, Keuken M, Smit F, Schippers G, Cuijpers P. Curbing problem drinking with personalized-feedback interventions: a meta-analysis. Am J Prev Med. 2009 Mar;36(3):247–255. doi: 10.1016/j.amepre.2008.10.016. [PubMed] [Cross Ref]
78. Gilvarry E, McArdle P. Alcohol, drugs and young people: clinical approaches. London: Mac Keith Press; 2007. pp. 153–169.
79. Tait RJ, Hulse GK. A systematic review of the effectiveness of brief interventions with substance using adolescents by type of drug. Drug Alcohol Rev. 2003 Sep;22(3):337–346. doi: 10.1080/0959523031000154481. [PubMed] [Cross Ref]
80. Voogt CV, Kleinjan M, Poelen EAP, Lemmers LACJ, Engels RCME. The effectiveness of a web-based brief alcohol intervention in reducing heavy drinking among adolescents aged 15-20 years with a low educational background: a two-arm parallel group cluster randomized controlled trial. BMC Public Health. 2013;13:694. doi: 10.1186/1471-2458-13-694. http://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-13-694. [PMC free article] [PubMed] [Cross Ref]
81. Barnett NP, Apodaca TR, Magill M, Colby SM, Gwaltney C, Rohsenow DJ, Monti PM. Moderators and mediators of two brief interventions for alcohol in the emergency department. Addiction. 2010 Mar;105(3):452–465. doi: 10.1111/j.1360-0443.2009.02814.x. http://europepmc.org/abstract/MED/20402989. [PMC free article] [PubMed] [Cross Ref]
82. Tanner-Smith EE, Steinka-Fry KT, Hennessy EA, Lipsey MW, Winters KC. Can brief alcohol interventions for youth also address concurrent illicit drug use? results from a meta-analysis. J Youth Adolesc. 2015 May;44(5):1011–1023. doi: 10.1007/s10964-015-0252-x. [PMC free article] [PubMed] [Cross Ref]
83. Dutra L, Stathopoulou G, Basden SL, Leyro TM, Powers MB, Otto MW. A meta-analytic review of psychosocial interventions for substance use disorders. Am J Psychiatry. 2008 Feb;165(2):179–187. doi: 10.1176/appi.ajp.2007.06111851. [PubMed] [Cross Ref]
84. Palfai TP, Saitz R, Winter M, Brown TA, Kypri K, Goodness TM, O'Brien LM, Lu J. Web-based screening and brief intervention for student marijuana use in a university health center: pilot study to examine the implementation of eCHECKUP TO GO in different contexts. Addict Behav. 2014 Sep;39(9):1346–1352. doi: 10.1016/j.addbeh.2014.04.025. http://europepmc.org/abstract/MED/24845164. [PMC free article] [PubMed] [Cross Ref]
85. Lee CM, Neighbors C, Kilmer JR, Larimer ME. A brief, web-based personalized feedback selective intervention for college student marijuana use: a randomized clinical trial. Psychol Addict Behav. 2010 Jun;24(2):265–273. doi: 10.1037/a0018859. http://europepmc.org/abstract/MED/20565152. [PMC free article] [PubMed] [Cross Ref]
86. Rooke S, Copeland J, Norberg M, Hine D, McCambridge J. Effectiveness of a self-guided web-based cannabis treatment program: randomized controlled trial. J Med Internet Res. 2013;15(2):e26. doi: 10.2196/jmir.2256. http://www.jmir.org/2013/2/e26/ [PMC free article] [PubMed] [Cross Ref]
87. Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jönsson B, Olesen J, Allgulander C, Alonso J, Faravelli C, Fratiglioni L, Jennum P, Lieb R, Maercker A, van OJ, Preisig M, Salvador-Carulla L, Simon R, Steinhausen H. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011 Sep;21(9):655–679. doi: 10.1016/j.euroneuro.2011.07.018. [PubMed] [Cross Ref]
88. Rohde P, Lewinsohn PM, Seeley JR, Klein DN, Andrews JA, Small JW. Psychosocial functioning of adults who experienced substance use disorders as adolescents. Psychol Addict Behav. 2007 Jun;21(2):155–164. doi: 10.1037/0893-164X.21.2.155. http://europepmc.org/abstract/MED/17563135. [PMC free article] [PubMed] [Cross Ref]
89. Zucker RA, Donovan JE, Masten AS, Mattson ME, Moss HB. Early developmental processes and the continuity of risk for underage drinking and problem drinking. Pediatrics. 2008 Apr;121 Suppl 4:S252–272. doi: 10.1542/peds.2007-2243B. http://europepmc.org/abstract/MED/18381493. [PMC free article] [PubMed] [Cross Ref]
90. Apodaca TR, Longabaugh R. Mechanisms of change in motivational interviewing: a review and preliminary evaluation of the evidence. Addiction. 2009 May;104(5):705–715. doi: 10.1111/j.1360-0443.2009.02527.x. http://europepmc.org/abstract/MED/19413785. [PMC free article] [PubMed] [Cross Ref]
91. Becker J, Haug S, Sullivan R, Schaub MP. Effectiveness of different Web-based interventions to prepare co-smokers of cigarettes and cannabis for double cessation: a three-arm randomized controlled trial. J Med Internet Res. 2014;16(12):e273. doi: 10.2196/jmir.3246. http://www.jmir.org/2014/12/e273/ [PMC free article] [PubMed] [Cross Ref]
92. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999 Sep;89(9):1322–1327. [PubMed]
93. Peters G, de BM, Crutzen R. Everything should be as simple as possible, but no simpler: towards a protocol for accumulating evidence regarding the active content of health behaviour change interventions. Health Psychol Rev. 2015;9(1):1–14. doi: 10.1080/17437199.2013.848409. http://europepmc.org/abstract/MED/25793484. [PMC free article] [PubMed] [Cross Ref]
94. Murray E, Khadjesari Z, White IR, Kalaitzaki E, Godfrey C, McCambridge J, Thompson SG, Wallace P. Methodological challenges in online trials. J Med Internet Res. 2009;11(2):e9. doi: 10.2196/jmir.1052. http://www.jmir.org/2009/2/e9/ [PMC free article] [PubMed] [Cross Ref]
95. Babor TF, Steinberg K, Anton R, Del BF. Talk is cheap: measuring drinking outcomes in clinical trials. J Stud Alcohol. 2000 Jan;61(1):55–63. [PubMed]
96. Voogt C, Kuntsche E, Kleinjan M, Poelen E, Engels R. Using ecological momentary assessment to test the effectiveness of a web-based brief alcohol intervention over time among heavy-drinking students: randomized controlled trial. J Med Internet Res. 2014;16(1):e5. doi: 10.2196/jmir.2817. http://www.jmir.org/2014/1/e5/ [PMC free article] [PubMed] [Cross Ref]
97. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology. Eval Health Prof. 2006 Mar;29(1):126–153. doi: 10.1177/0163278705284445. [PubMed] [Cross Ref]

Articles from Journal of Medical Internet Research are provided here courtesy of Gunther Eysenbach