|Home | About | Journals | Submit | Contact Us | Français|
Attention Bias Modification (ABM) protocols aim to modify attentional biases underlying many forms of pathology. Our objective was to conduct an effect size analysis of ABM across a wide range of samples and psychological problems. We conducted a literature search using PubMed, PsycInfo, and author searches to identify randomized studies that examined the effects of ABM on attention and subjective experiences. We identified 37 studies (41 experiments) totaling 2,135 participants who were randomized to training toward neutral, positive, threat, or appetitive stimuli or to a control condition. The effect size estimate for changes in attentional bias was large for the neutral vs. threat comparisons (g =1.06), neutral vs. appetitive (g =1.41), and neutral vs. control comparisons (g = 0.80), and small for positive vs. control (g =0.24). The effects of ABM on attention bias were moderated by stimulus type (words vs pictures) and sample characteristics (healthy vs. high symptomatology). Effect sizes of ABM on subjective experiences ranged from 0.03 to 0.60 for post-challenge outcomes, −0.31 to 0.51 for post-treatment, and were moderated by number of training sessions, stimulus type and stimulus orientation (top/bottom vs. left/right). Fail-safe N calculations suggested that the effect size estimates were robust for the training effects on attentional biases, but not for the effect on subjective experiences. ABM studies using threat stimuli produced significant effects on attention bias across comparison conditions, whereas appetitive stimuli produced changes in attention only when comparing appetitive vs. neutral conditions. ABM has a moderate and robust effect on attention bias when using threat stimuli. Further studies are needed to determine whether these effects are also robust when using appetitive stimuli and for affecting subjective experiences.
Attention bias, the tendency to selectively attend to disorder-relevant stimuli, is implicated in the etiology and maintenance of psychopathology (Rapee & Heimberg, 1997; Williams, Watts, MacLeod, & Mathews, 1997). A meta-analysis of 172 studies found that attention bias toward threat stimuli is reliably associated with anxiety (Bar-Haim, Dominique, Pergamin, Bakermans-Kranenburg, & van IJzendoorn, 2007). Attention biases have also been associated with depression (Eizenman, 2003; Koster, de Raedt, Leyman, & De Lissnyder, 2010). Moreover, a number of other forms of psychopathology have been characterized by attention bias toward appetitive stimuli rather than threat, i.e., eating disorders (Glauert, Rhodes, Fink, & Grammer, 2010; Shafran, Lee, Cooper, Palmer, & Fairburn, 2007), smoking (Ehrman, 2002), alcohol use (Townshend & Duka, 2001), and sexual dysfunction (Beard & Amir, 2010). Evidence from genetic studies ( e.g., Beevers, Gibb, McGeary, & Miller, 2007; Caspi, Hariri, Holmes, Uher, & Moffitt, 2010; Gibb, Benas, Grassia, & McGeary, 2009) and prospective studies (Fox, Cahill, & Zougkou, 2010; Macleod & Hagan, 1992; van den Hout, Tenney, Huygens, Merckelbach, & Kindt, 1995) suggests that attention bias is not merely associated with psychopathology, but may constitute a vulnerability factor for developing psychopathology in response to stress.
Visual probe tasks are commonly used to assess attention. For example, in a typical dot probe task, two stimuli are presented quickly (e.g., 500 ms) on a computer monitor simultaneously. One of the stimuli is neutral (e.g., the word ‘chair’), whereas the other one is disorder-relevant (e.g., the word ‘disease’). After the brief presentation of these stimuli, the words disappear, and a probe (e.g., the letter ‘E’ or ‘F’) appears in the prior location of one of the words. The participant’s task is to identify the probe quickly and accurately by pressing a corresponding button. Biased attention is inferred from faster reaction times to identify probes replacing disorder-relevant compared to neutral stimuli (MacLeod, et al., 1986).
The original version of the dot probe task was designed to assess bias, and thus probes replaced neutral and disorder-relevant stimuli with equal frequency. By altering the contingency between the location of probes and disorder-relevant stimuli, the dot probe task can also be used to modify attention. If probes always replace neutral or positive stimuli, attention may be directed away from disorder-relevant stimuli. In these neutral or positive training conditions, acquiring an attention bias away from disorder-relevant stimuli will facilitate faster reaction times (i.e., better performance) on the task. Similarly, the task can also be used to induce a bias toward disorder-relevant stimuli when probes always replace the disorder-relevant stimuli. Finally, visual probe tasks have the unique methodological advantage of having an extremely well-matched control condition. In the control condition, probes replace neutral or positive and disorder-relevant stimuli with equal frequency. Thus, the control is simply the assessment version of the dot probe task.
To date, most Attention Bias Modification (ABM) studies have utilized aversive stimuli (e.g., threat words, sad faces) as disorder-relevant stimuli and positive or neutral stimuli (e.g., happy or neutral faces) as the other stimuli in the task1. However, recently a growing number of studies have utilized appetitive stimuli (e.g., alcohol, smoking, food) and examined effects on motivational outcomes (e.g., desire to drink). Additionally, while most ABM studies have utilized probe detection tasks, a number of studies trained attention via a visual search paradigm. In the visual search task, participants repeatedly identify the location of a smiling face among a matrix of angry faces. In both types of tasks, acquiring an attention bias away from disorder-relevant stimuli will facilitate faster reaction times (i.e., better performance) on the tasks.
The aim of all of these ABM trainings is to modify attention via repeated practice on cognitive tasks (for qualitative reviews of ABM, see Bar-Haim, 2010; Browning, Holmes, & Harmer, 2010; Beard, 2011). Thus, ABM training may alter cognitive biases through a more implicit, experiential process compared to the explicit, verbal process of psychotherapy. This training is assumed to alter attentional processes that are not considered to be under volitional control, providing a different method of influencing this stage of information processing compared to existing treatments.
Researchers have mostly used ABM tasks to test the causal relationship between attention bias and emotional vulnerability or motivational states. To this end, researchers attempt to induce biases in healthy individuals in a single experimental session and examine the effect on responses to a laboratory challenge. Challenge tasks are typically related to the specific psychopathology under study. For example, socially anxious participants may undergo an impromptu speech, whereas heavy drinkers may complete an alcohol taste test. Recently, researchers have translated findings from single-session experiments into multi-session treatments, primarily for anxiety disorders.
ABM has grown rapidly over the last decade, and two quantitative reviews of the literature have been published. Hakamata and colleagues (2010) conducted a specific review limiting their examination to the effect of dot probe ABM tasks on attention bias and anxiety. Results revealed that ABM produced a large effect on attention bias (d = 1.16, CI = .82–1.50) and a medium effect on anxiety (d = 0.61, CI = .42–.81). Effect sizes for anxiety were larger on trait versus state measures, words versus face stimuli, and top-bottom versus left-right orientation of stimuli. Additionally, the number of sessions moderated effects on attention bias. These results were promising, yet tentatively based on only 12 experiments. Hallion and Ruscio (2011) recently extended the Hakamata findings by including 21 ABM studies in their meta-analysis and by examining ABM’s effect on depression in addition to anxiety. Results revealed small, but reliable effects on attention (g = .29), anxiety (g = .23), and non-significant effects on depression (g = .12). Effects on anxiety were larger for studies that included more than one training session (g = .40), although this was a non-significant trend.
Thus, the magnitude of ABM’s effects are unclear given the discrepant findings of these prior reviews, with Hakamata suggesting large effects on attention and medium effects on anxiety while Hallion and Ruscio suggesting small effects on both outcomes. Additionally, the Hakamata review concluded that effects on anxiety were reliable, but did not apply standard fail-safe N guidelines when interpreting effect sizes. Moreover, the Hallion and Ruscio review did not examine a number of task characteristics as potential moderators; thus, the moderators revealed by Hakamata await replication in a larger sample. Such findings have direct implications for how future ABM treatment should be delivered. Finally, none of the prior quantitative or qualitative reviews have included the growing number of ABM studies that utilize appetitive stimuli (e.g., alcohol, smoking, food) rather than aversive stimuli (e.g., threat words, sad faces) as disorder-relevant stimuli. Given the inherent differences in approach-avoidance tendencies related to such stimuli, it is possible that ABM may have different effects for appetitive versus threat stimuli. For example, it may be more difficult to train individuals to attend away from alcohol cues, given that the approach system is involved in problematic drinking (e.g., Field, Kiernan, Eastwood, & Child, 2008), whereas anxiety is characterized by behavioral avoidance (e.g., Barlow, 2002).
Thus, the current study extends prior reviews by including 33 additional experiments compared to Hakamata, examining several potentially important moderators (e.g., orientation of stimulus) identified in Hakamata’s review that were not examined in the Hallion and Ruscio review and thus await replication, and including studies which utilized appetitive stimuli. Our objective was to provide a comprehensive quantitative, meta-analytic review of the efficacy of ABM. We reviewed studies examining the effects of ABM on attention bias and subjective experiences (e.g., anxiety, depression, urge to drink alcohol or smoke). To determine the robustness of this new approach to testing cognitive models, it is important to quantitatively examine ABM’s ability to affect attention in healthy individuals, in addition to its ability to alleviate symptoms in analogue and clinical samples. Thus, similar to the Hallion review, we included studies that utilized healthy participants and studies that induced biases toward disorder-relevant stimuli, in addition to studies with samples of high levels of symptomatology (i.e., analogue, clinical).
ABM is not proposed to be an effective mood manipulation, but rather to affect an individual’s vulnerability to respond emotionally or behaviorally to emotional or motivational cues. Thus, we hypothesized that ABM would not have a direct effect on subjective experience (i.e., post-training). We expected ABM to have an effect on responses to challenge tasks (i.e., post-challenge) and following multi-session protocols (i.e., post-treatment). Based on the Hakamata review, we expected word stimuli and top-bottom orientation in training tasks to produce larger effects. However, this was a tentative hypothesis, given that Hakamata et al. (2010) only examined 12 experiments and Hallion & Ruscio (2011) did not examine these potential moderators. Finally, based on prior findings (Hakamta et al., 2010; Hallion & Ruscio, 2011), we expected stronger magnitude of training (i.e., number of sessions) to produce larger effects.
We determined that an intervention qualified as an ABM if it (a) directly targeted attention bias, and (b) modified attention via “extensive practice on a cognitive task designed to encourage and facilitate the desired cognitive change (Koster et al., p. 3).” Additionally, studies meeting the following criteria were eligible for inclusion: (1) included a measure of attention bias or subjective experience; (2) randomized participants; and (3) provided sufficient data to perform effect size analyses (i.e., means and standard deviations, t or F values, change scores, frequencies, or probability levels). Authors were contacted for additional data when needed.
As our primary aim was to provide a comprehensive examination of ABM’s effects on attention and emotional or motivational outcomes, our inclusion/exclusion criteria were broader than those of the Hakamata review. Specifically, Hakamata et al. only examined studies examining the effect of ABM on symptoms of anxiety. Additionally, they excluded studies not utilizing a dot-probe task. Our inclusion/exclusion criteria differed from the Hallion & Ruscio analysis in that they excluded all studies employing stimuli not directly relevant to anxiety or depression (e.g., food, cigarettes, alcohol). These studies were included in our analysis and were of particular interest in comparing the effects of appetitive vs. aversive stimuli. Additionally, the Hallion & Ruscio study included only studies in which the sample population was psychologically healthy or diagnosed with anxiety or depression. We did not employ this criterion, and therefore included studies examining a wider range of pathology (e.g., alcohol dependence, smoking). However, we limited our examination to studies testing ABM, rather than including two different types of interventions as did Hallion and Ruscio (i.e., ABM and interpretation bias modification).
We performed the meta-analysis in accordance with the PRISMA guidelines (Liberati et al., 2009). Searches were conducted in PubMed and PsycInfo to identify studies published between the first available year and February 15, 2011. The following search terms were used: cognitive * bias * modification, attention * bias * modification, and attention * bias* training. Additionally, a manual review of authors identified through database searches was conducted. Articles related to the topic of ABM were selected for further examination.
Two of the authors (CB, ATS) independently extracted numerical data and coded potential moderators for each study. For examining the effect on subjective experience, the primary measure identified in each study report was extracted to examine group differences immediately after a single session of training (i.e., post-training), in response to a challenge task, such as an impromptu speech (i.e., post-challenge), and after a multi-session treatment (i.e., post-treatment). If a study did not identify a primary outcome, the authors (CB, ATS) identified the measure that best assessed the construct targeted by the study.
Meta regression analyses and the Q and I2 statistics were used to determine whether effect sizes varied as a function of clinical characteristics (healthy vs. high symptomatology), type of pathology targeted (anxiety vs. depression vs. alcohol vs. smoking), study year, and training characteristics (number of sessions, number of training trials, stimulus orientation [i.e., top/bottom vs. left/right], stimulus modality [i.e., words vs. pictures]). Please note that stimulus orientation is only relevant for probe detection tasks, whereas the other training characteristics apply to all training tasks. Continuous measures were assessed by meta-regression. The Q and I2 statistics were used to assess heterogeneity and categorical moderators (Huendo-Medina, Sánchez-Meca, Marin-Martinez, & Botella, 2006). The Q statistic indicates whether heterogeneity is present or absent, while the I2 assesses the degree of heterogeneity on a 0 to 1 scale, with 0 representing complete homogeneity and 1 representing complete heterogeneity. Both the heterogeneity within each group (Qwithin), and the heterogeneity between the groups (Qbetween), was assessed. The grouping variable (i.e., the moderator) was considered significant when the between-groups heterogeneity was significant.
Effect sizes were calculated using Hedges’s g and its 95% confidence interval. Hedges’s g is a variation of Cohen's d that corrects for biases due to small sample sizes (Hedges & Olkin, 1985). All effect sizes were calculated using random effects models because the studies included were assumed to be only a sample of the entire population of studies (Hedges & Vevea, 1998).
The effect size estimates for individual studies were combined to obtain a summary statistic. We calculated an average Hedges’s g effect size for attention bias and a separate Hedges’s g effect size for studies that included measures of subjective experience. We calculated effect sizes separately for each type of comparison condition: neutral (e.g., neutral faces) vs. control, positive (e.g., smiling faces) vs. control, and neutral vs. disorder-relevant (e.g., angry faces/alcohol pictures). For those studies that examined changes in subjective experience, we calculated effect sizes at post-training, post-challenge, and post-treatment when applicable. The magnitude of Hedges’s g may be interpreted using Cohen’s (Cohen, 1988) recommendations of small (0.2), medium (0.5), and large (0.8). In cases where the correlation between pre-and post measures was unavailable but necessary to calculate pre-post effect sizes, we followed the recommendation by Rosenthal (Rosenthal, 1993) and assumed a conservative estimation of r = 0.7. All analyses were completed using the software program Comprehensive Meta-Analysis, Version 2 (Borenstein, Hedges, Higgins, & Rothstein, 2005).
To address the file-drawer problem (i.e., the fact that studies with non-significant results are less likely to be published than those reporting significant results and can thereby bias meta-analytic results), we computed the fail-safe N (Rosenthal, 1991; Rosenthal & Rubin, 1988) which is an estimate of the number of unpublished studies reporting effect sizes of zero needed to nullify the significant effect. We used the following formula: , where K is the number of studies included in the meta-analysis and is the mean Z obtained from the K studies. According to Rosenthal (Rosenthal, 1991), if the required number of studies (X) to reduce the overall effect size to a non-significant level exceeds 5K + 10, the effect size can be considered robust. Fail-safe N values were calculated only for effect sizes that were significant.
We examined the following potential moderators: clinical characteristics (healthy vs. high symptomatology); type of pathology targeted (anxiety vs. depression vs. alcohol vs. smoking); stimulus orientation (top/bottom vs. left/right); stimulus modality (words vs. pictures) of training task; stimulus modality (words vs. pictures) of assessment task; publication year; number of sessions; and number of training trials.
Our initial searches identified 921 potentially relevant articles of which 37 studies (41 experiments) and a total of 2,135 participants met our inclusion criteria and were included in the meta-analysis (see Figure 1). In all studies, participants were randomized and blind to training condition. Table 1 details the characteristics of the included studies. For those studies reporting attention bias data, 17 experiments compared neutral vs. control conditions, nine compared neutral vs. disorder-relevant conditions, and six compared positive vs. control conditions (the numbers do not add up to 30 because some studies fell in more than one category). For the studies examining the effect on subjective experience, most experiments examined a neutral vs. control condition (eight studies after training; 11 after a challenge; and eight after treatment), followed by neutral vs. disorder-relevant condition (14 studies at post-training; 12 after a challenge; and none after treatment), and positive vs. control (two studies after training, two after a stressor, and three after treatment); (most experiments included more than one time point). All studies utilized either visual probe tasks or visual search tasks as the ABM method.
Across the experiments, the most frequent type of sample was healthy individuals (19 experiments, n = 898 participants), followed by analogue samples (18 experiments, 993 participants), and clinical samples (7 experiments, 244 participants) (experiment numbers do not add up to 41 because several experiments included more than one type of sample). Within the high symptomatology samples, the most common pathology studied was social anxiety (n = 6), followed by generalized anxiety (n = 5), alcohol use (n = 4), smoking (n = 3), depression (n = 2), low self-esteem (n = 2), fear of spiders (n = 1), and obsessive-compulsive symptoms (n = 1).
As depicted in Table 2, effect sizes were calculated separately for each of the comparison conditions. Within each comparison condition, we also present the effect sizes separately for studies utilizing healthy and high symptomatology samples. The random effects meta-analysis yielded the following average pre-post effect size estimates (Hedges’s g): neutral vs. control condition (g = 0.80; 95% CI: 0.49–1.12, p < .001), positive vs. control condition (g = 0.235; 95% CI: 0.020–0.449, p < .05), and neutral vs. disorder-relevant (g = 1.19; 95% CI: 0.96–1.41 , p< .001). For the neutral vs. control comparison condition, the See et al. (2009) study had a very large effect size (Hedges’s g = 4.76). Eliminating this study from the analysis did not change the general results (Hedges’s g = 0.72; 95% CI: 0.49–0.95, p < .01).
Fail-safe N values were calculated for the pre-post attention bias effect sizes. The fail-safe N value for the attention bias effect size for the neutral vs. control comparison condition was robust at 427 (z-value = 10.00), indicating that 427 unpublished studies with effect sizes of zero would be necessary to nullify this result. When excluding the See et al. (2009) study, the fail-safe N value remains robust at 73 (z-value = 8.59). However, for the positive vs. control condition, the fail-safe N value was 4 (z-value = 2.41), suggesting that this result is unreliable and should be considered preliminary. The fail-safe N value was robust for the neutral vs. disorder-relevant condition (321, z-value = 11.85).
Effect sizes were calculated separately for studies utilizing threat stimuli and those using appetitive stimuli within each comparison condition. For the neutral vs. control comparison condition, the pre-post effect size estimate was large for threat studies (g = 0.96; 95% CI: 0.55–1.37, p < .001), and small for appetitive studies (g = 0.39; 95% CI: 0.13–0.66, p = .004). For the neutral vs. disorder-relevant comparison condition, the pre-post effect size estimate was large for the neutral vs. threat studies (g = 1.06; 95% CI: 0.72–1.40, p < .001), and for the neutral vs. appetitive studies (g = 1.41; 95% CI: 1.09–1.73, p < .001). The fail-safe N value was robust for the threat studies (neutral vs. control = 179, neutral vs. threat = 84). For appetitive studies, the fail-safe N value (N = 73) was robust for the neutral vs. appetitive condition, but it was not robust for the neutral vs. control condition (N= 8).
All meta-regression analyses performed on the neutral vs. control condition excluded the See et al. (2009) study due to its unusually large effect size. For the neutral vs. control condition, when comparing the type of pathology targeted, the largest mean effect size was for the study targeting depression (g =1.54; CI = 0.78–2.29, p < 0.001), followed by studies targeting anxiety (g =0.68; CI = 0.46–0.83, p < 0.001), alcohol (g =0.59; CI = 0.29–0.88, p < 0.001), and smoking (g =0.09; CI = −0.31–0.49, p = 0.66 ). The degree of heterogeneity within the groups was non-significant (Qwithin = 10.84, df = 12, p = 0.54; I2 = 0.0%), whereas the degree of heterogeneity between the groups was significant (Qbetween = 13.18, df = 3, p = 0.004; I2 = 77.2%). However, this effect was driven by the small effects observed in the two smoking studies. When these studies were removed, the type of pathology targeted no longer moderated effects on attention bias.
For the positive vs. control condition, sample type was a significant moderator. The mean effect size for studies with healthy samples was −0.02 (95% CI = −0.27–0.24, p = 0.90, n.s.), and the mean effect size for studies with high symptomatology samples was 0.48 (95% CI = 0.20–0.75, p = 0.001). The degree of heterogeneity within the groups was non-significant (Qwithin = 3.54, df = 6, p = 0.74, n.s.; I2 = 0.0%), whereas the degree of heterogeneity between the groups was significant (Qbetween = 6.51, df = 1, p = 0.011; I2 = 84.6%).
For the neutral vs. disorder-relevant condition, when comparing the stimulus modality (i.e., words vs. pictures) of the training paradigm, results indicated that the mean effect size for studies utilizing pictures was 1.44 (CI = 1.16–1.72, p < 0.001), and the mean effect size for those utilizing words was 0.91 (95% CI = 0.63–1.18, p < 0.001). The degree of heterogeneity within the groups was non-significant (Qwithin = 3.4, df = 7, p = 0.85, n.s.; I2 = 0.0%), whereas the degree of heterogeneity between the groups was significant (Qbetween = 7.12, df = 1, p = 0.008; I2 = 85.9%). This indicates that the effects of ABM on attention bias in studies utilizing pictures are significantly greater than the effects of ABM in studies utilizing words for this comparison condition. This same result emerged when comparing the modality of the test paradigm for this condition (pictures = 1.41 (95% CI = 1.15–1.67, p < 0.001), words = 0.88 (95% CI = 0.59–1.17, p < 0.001). The degree of heterogeneity within the groups was non-significant (Qwithin = 3.42, df = 7, p = 0.84, n.s.; I2 = 0.0%), whereas the degree of heterogeneity between the groups was significant (Qbetween = 7.11, df = 1, p = 0.008; I2 = 85.9%).
As hypothesized, analyses for post-training effects revealed small, non-significant effects. The random effects meta-analysis yielded the following average pre-post effect size estimate (Hedges’s g): neutral vs. control (g = 0.01, 95% CI: −0.17–0.20; p = .89), positive vs. control (g = 0.09, 95% CI: −0.30–0.47; p = .66), and neutral vs. disorder-relevant (g = 0.03, 95% CI: −0.12–0.19, p = .70).
The effect of ABM on subjective experiences at post-challenge and post-treatment is presented in Table 3. For post-challenge effects, the random effects meta-analysis yielded the following average pre-post effect size estimate (Hedges’s g): neutral vs. control (g = 0.22, 95% CI: −0.02–0.45; p = .07), positive vs. control (g = 0.60, 95% CI: −0.08–1.28; p = .08), neutral vs. disorder-relevant (g = 0.40, 95% CI: 0.22–0.57; p < 0.001).
We calculated pre-post effect sizes for studies assessing changes in symptoms following a multi-session ABM. For neutral vs. control conditions, the random effects meta-analysis yielded an average pre-post effect size estimate (Hedges’s g) of 0.41 (95% CI: 0.05–0.78, p = .03). This effect appeared to be driven entirely by studies utilizing sample with high symptomatology (g = 0.51), rather than studies of healthy samples (g = −0.15). For positive vs. control conditions, the analysis yielded an effect size of (0.09 (95% CI: −0.59–0.79, p = .79). None of the studies in the neutral vs. disorder-relevant condition involved multi-session treatments.
Although most post-challenge and post-treatment effects were significant and in the moderate range, none of these effect sizes were robust according to a priori fail-safe N guidelines (Rosenthal, 1991). Thus, all effect sizes for subjective experience outcomes should be interpreted with caution until further evidence is available.
Effect sizes were calculated separately for studies utilizing threat stimuli and those using appetitive stimuli within each comparison condition. For the neutral vs. control condition, the post-challenge effect size estimate for threat studies was not significant (g = 0.30, 95% CI: −0.04–0.64, p = .083). However, the post-treatment effect for threat studies was significant and medium in size (g = 0.48, 95% CI: 0.08–0.88, p = .019). For appetitive studies, neither post-challenge, (g = 0.03, 95% CI: −0.24–0.30, p = .84), nor post-treatment effects (g = −0.031, 95% CI: −0.66–0.60, p = .92) were significant.
For the neutral vs. disorder-relevant comparison condition, the post-challenge effect was significant for both threat (g = 0.41, 95% CI: 0.19–0.64, p < .001) and appetitive studies (g = 0.36, 95% CI: 0.04–0.68, p = .027). However, similar to the overall subjective experience effect sizes, the fail-safe N values were not robust for any of the threat or appetitive subjective experience outcome effect sizes.
For the neutral vs. control condition at post-challenge, the mean effect size for studies utilizing pictures was 0.07 (CI = −0.16–0.31, p = 0.54, n.s.), and the mean effect size for those utilizing words was 0.62 (95% CI = 0.16–1.09, p = 0.009). The degree of heterogeneity within the groups was non-significant (Qwithin = 11.28, df = 8, p = 0.19, n.s.; I2 = 29.0%), whereas the degree of heterogeneity between the groups was significant (Qbetween = 4.27, df = 1, p = 0.039; I2 = 76.6%). This indicates that the effects of ABM utilizing words on subjective experience are significantly greater than those utilizing pictures.
For the neutral vs. control condition at post-treatment, orientation of the stimuli was a moderator. The mean effect size for studies utilizing a top/bottom orientation was 0.88 (CI =0.53–1.23, p < 0.001), and the mean effect size for those utilizing a left/right orientation was −0.02 (95% CI = −0.33–0.30, p = 0.91, n.s.). The degree of heterogeneity within the groups was non-significant (Qwithin = 2.88, df = 6, p = 0.82, n.s.; I2 = 0.0%), whereas the degree of heterogeneity between the groups was significant (Qbetween = 14.09, df = 1, p < 0.001; I2 = 92.9.0%). This indicates that a top/bottom orientation was more effective than a left/right orientation. Finally, Hedges’ g was moderated by the number of training sessions for this condition at post-treatment (β = −0.176, SE = 0.066, p = 0.007), with a greater number of training sessions resulting in larger effect sizes.
For the neutral vs. disorder-relevant condition at post-challenge there were no moderators (there were no studies examining post-treatment effects). There were too few studies in the positive vs. control condition at post-challenge (N = 2) and post-treatment (N = 3) to run moderator analyses.
We examined the effects of ABM on attention bias and subjective experiences across various forms of psychopathology. We identified 37 studies, of which we analyzed 41 experiments with 2,135 participants to derive effect size estimates. The current results confirm that ABM has a reliable effect on attention bias. Similar to Hakamata et al. (2010), we found statistically significant and large effects of ABM on attention with large fail-safe calculations. These large effect sizes are in contrast to small effects reported in Hallion and Ruscio (2011). As expected and similar to prior findings (Hallion & Ruscio, 2011), studies that compared two active trainings yielded larger effects than studies that compared a neutral or positive induction versus a control condition.
Results revealed that effect size estimates for subjective experiences obtained directly following ABM were small and non-significant across all comparison conditions. Thus, ABM is not an effective state emotional or motivational manipulation. In contrast, the effect size estimates for emotional and motivational responses to laboratory and natural challenges were small, but significant. Our effect sizes for these outcomes were in line with Hallion and Ruscio (2011), which also obtained small effect sizes. When applied as a multi-session protocol, ABM effects on symptomatology were also significant. Hakamata et al. (2010) obtained similar effect sizes specifically for changes in anxiety, but those authors interpreted the effect sizes as reliable and supportive of ABM as a treatment for anxiety. However, similar to the current results, the fail-safe N obtained by Hakamata and colleagues was not robust, and thus we conclude that both reviews suggest that there is currently insufficient data to determine ABM’s effect on subjective experiences.
The current review is the first to include studies which utilized appetitive stimuli. When examining effect sizes separately for these studies, results suggest that only studies comparing neutral vs. appetitive conditions (i.e., two active trainings) produced significant effects on attention and subjective experiences. These initial findings suggest that ABM may not be as promising a treatment for some disorders, such as alcohol or nicotine dependence, compared to anxiety. One explanation for these findings is that it may be more difficult to direct attention away from appetitive stimuli than aversive stimuli. However, these findings are based on a small number of appetitive studies per condition. Additionally, no study directly compared the efficacy of ABM with threat stimuli vs. appetitive stimuli. Future reviews are warranted after more studies are completed.
Observed effect sizes were unrelated to publication year, but were related to sample and task characteristics. Consistent with Hakamata et al. (2010), training with top-bottom orientation was more effective than left-right orientation. This effect may be in part due to the fact that the largest effect sizes were obtained from three multi-session treatment studies for clinically anxious individuals, which all utilized top-down orientation. Multi-session studies that used left-right orientation included a variety of samples (healthy, depressed, alcoholics, children) and obtained smaller effects. In Hakamata et al. (2010), all of the studies targeted anxiety, and most of the multi-session studies used top-bottom.
Pictures were superior to words, but only for neutral vs. disorder-relevant condition effects on attention, which were almost exclusively tested in healthy controls. Words were more effective than pictures, but only when comparing neutral vs. control conditions and only for subjective experiences at post-challenge. Thus, there does not appear to be a consistent superior stimulus type across ABM in the current study. Hakamata et al. (2010) found that words were superior to pictures, but only when comparing neutral vs. control condition effects on attention.
Hakamata et al. (2010) found that number of sessions moderated effects on attention. The Hallion and Ruscio (2011) review also obtained this trend, but for emotional outcomes. In the current study, number of sessions moderated subjective experience effects even when the See et al. outlier study was not included in the analyses. This study included substantially more sessions of training (n = 15) than other studies and a much larger effect size, providing further support for a dose-response relationship. Thus, it is clear that future ABM studies should include multiple sessions in order to obtain larger and perhaps more reliable effects on attention and subjective experience.
The results of this study are limited to the meta-analytic technique and, therefore, are dependent on the study selection criteria, the quality of the included studies, expectancy effects, and statistical assumptions about the true values of the included studies. In order to limit any possible biases, we adopted a relatively conservative approach. Following the recommendations by Moses and colleagues (2002) and Hedges and Vevea (1998), we analyzed the effect sizes using a random effect model. Perhaps the most important bias of meta-analyses is the expectancy effect. Cotton and Cook (1982) recommended early on that the investigators of meta-analyses explicitly state their personal view with regards to the outcome in order to acknowledge and possibly avoid the expectancy effect. At the outset of our review, the second and third authors were not involved with ABM. Given the first author’s work in the ABM field and the positive results from prior meta-analyses, we did expect to find significant effects. However, all authors remained skeptical about the size of the effects across different forms of psychopathology and maintained equipoise throughout the review process. Another limitation relates to the small number of studies targeting depression and smoking, which prevented us from drawing conclusions about whether the type of pathology targeted moderates outcomes. Finally, given the few studies that included a follow-up assessment, we were unable to examine duration of effects.
Despite these limitations, the current quantitative review of randomized experiments suggests that ABM is a robust method for modifying attention bias across a wide range of samples and stimuli. However, additional randomized controlled trials are needed before conclusions can be made about ABM’s effect on emotional and motivational outcomes. It is intuitive that effects on attention, the construct being manipulated, would be larger than those on subjective experience. Most studies delivered only a single session of ABM, and this not be an adequate dose of ABM to produce reliable effects on subjective experience. Additional studies are particularly needed to determine the utility of ABM using appetitive stimuli. Given the potential clinical utility (e.g., standardized and computerized delivery, no therapist contact, inexpensive) of this novel approach, larger definitive trials are warranted.
Dr. Beard’s time and effort were supported in part by an NIMH NRSA post-doctoral fellowship (F32 MH083330). Dr. Hofmann is supported by NIMH grants MH-078308 and MH-081116. He is also a paid consultant of Merck/Schering-Plough for work unrelated to this study. We thank Professor Eni Becker for requesting data from some of the authors of the studies that were included in this meta-analysis.
|Heading||Subheading||Descriptor||Reported? (Yes/N)||Page number|
|Title||Identify the report as a systematic review, meta-analysis, or both||Yes||1|
|Abstract||Structured summary||Provide a structured summary including, as applicable, background, objectives, data sources, study eligibility criteria, participants, interventions, study appraisal and synthesis methods, results, limitations, conclusions and implications of key findings, systematic review registration number||Yes||2|
|Introduction||Rationale||Describe the rationale for the review in the context of what is already known||Yes||4–7|
|Objectives||Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS)||Yes||8|
|Methods||Protocol and registration||Indicate if a review protocol exists, if and where it can be accessed (such as web address), and, if available, provide registration information including registration number||N/A|
|Eligibility criteria||Specify study characteristics (such as PICOS, length of follow-up) and report characteristics (such as years considered, language, publication status) used as criteria for eligibility, giving rationale||Yes||9|
|Information sources||Describe all information sources (such as databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched||Yes||10|
|Search||Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated||Yes||10|
|Study selection||State the process for selecting studies (that is, screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis)||Yes||10|
|Data collection process||Describe method of data extraction from reports (such as piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators||Yes||10|
|Data items||List and define all variables for which data were sought (such as PICOS, funding sources) and any assumptions and simplifications made||Yes||10|
|Risk of bias in individual studies||Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis||Yes||11–12|
|Summary measures||State the principal summary measures (such as risk ratio, difference in means).||Yes||11–12|
|Synthesis of results||Describe the methods of handling data and combining results of studies, if done, including measures of consistency (such as I2 statistic) for each meta-analysis||Yes||11–12|
|Risk of bias across studies||Specify any assessment of risk of bias that may affect the cumulative evidence (such as publication bias, selective reporting within studies)||Yes||11–12|
|Additional analyses||Describe methods of additional analyses (such as sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified||Yes||11–12|
|Results||Study selection||Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram||Yes||Figure 1|
|Study characteristics||For each study, present characteristics for which data were extracted (such as study size, PICOS, follow-up period) and provide the citations||Yes||12–13|
|Risk of bias within studies||Present data on risk of bias of each study and, if available, any outcome-level assessment (see item 12).||Yes||12–13|
|Results of individual studies||For all outcomes considered (benefits or harms), present for each study (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot||Yes||Tables 2 & 3|
|Synthesis of results||Present results of each meta-analysis done, including confidence intervals and measures of consistency||Yes||13–18|
|Risk of bias across studies||Present results of any assessment of risk of bias across studies (see item 15)||Yes||13–18|
|Additional analysis||Give results of additional analyses, if done (such as sensitivity or subgroup analyses, meta-regression) (see item 16)||Yes||13–18|
|Discussion||Summary of evidence||Summarise the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (such as health care providers, users, and policy makers)||Yes||19–20|
|Limitations||Discuss limitations at study and outcome level (such as risk of bias), and at review level (such as incomplete retrieval of identified research, reporting bias)||Yes||21|
|Conclusions||Provide a general interpretation of the results in the context of other evidence, and implications for future research||Yes||20–222|
|Funding||Funding||Describe sources of funding for the systematic review and other support (such as supply of data) and role of funders for the systematic review||Yes||23|
1Although one could potentially categorize appetitive stimuli (e.g., alcohol, cigarettes, food) as positive stimuli, we maintain a distinction between these two types of stimuli. Theoretically and clinically, CBM aims to train attention away from disorder-relevant stimuli, which are appetitive or threat-relevant, neither of which are consistently positive in valence. Indeed, it is likely that many individuals would not consider appetitive stimuli (e.g., cigarettes) as positive in valence. Finally, unlike appetitive stimuli, positive stimuli are only used as the comparison emotion when training attention away or toward threat; they are not the disorder-relevant emotion of interest.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*included in meta-analysis