This narrative systematic review has described the range and quality of evidence on second-generation computer-tailored primary prevention interventions targeting dietary behaviour change in adults. It also describes common characteristics of interventions that produced significant between-group effects and interventions with good retention rates. To our knowledge, previous reviews have not attempted to gauge the external validity of such intervention studies although they have included varying measures of external validity in their quality criteria. Doing so is important in determining their generalizability and relevance to health promotion practice [32
The majority of studies that isolated the effect of both the technology and tailoring in their study design reported significant positive outcomes, indicating computer-tailored interventions targeting these behaviours as a promising primary prevention strategy. The efficacy of computer-tailored interventions is dependent on many factors such as the intervention quality, duration, exposure, intensity, use of theory, method of tailoring, source credibility and mode of delivery. The quality, intensity and duration of intervention studies reporting significant positive between-group effects on dietary behaviour outcomes differed widely. However, it appears that tailoring and the use of theory are important factors for success.
Success of the intervention is not dependent on the technology used in its delivery or its intensity. However, this could be due to the small number of heterogeneous studies reviewed and the very small number of studies comparing intervention groups of differing intensity. There seems little evidence that success is more likely in interventions of greater intensity than of lower intensity, and this is the case whether additional support is delivered through the technology or interpersonal communication. Kroeze et al.
] reported similar findings. There is insufficient evidence to determine the optimal intensity for computer-tailored interventions generally and the best way of delivering interventions targeting more than one behaviour. Therefore, more research is needed in this area [21
It has been recommended that studies use a combination of validated self-reports with more objective measures of behaviour change; however, for dietary behaviour change, there is a lack of existing objective measures [4
]. Although valid measurement instruments were used, there was a lack of objective outcome measures. Future studies may benefit from collecting objective outcome measures to determine whether the behaviour changes reported are real.
The real-life effectiveness of such interventions is dependent on the setting, the characteristics and representativeness of the targeted and recruited population sample and the methods of recruitment. These factors influence the external validity and generalizability of findings to practice [32
]. The external validity of reviewed studies was generally poor, resulting in uncertainty about such interventions’ generalizability This finding is not surprising given the majority of studies were RCTs as such designs aim to maximize internal validity and can sacrifice external validity, with results only generalizable to those participants who are willing to accept randomization [32
]. A stronger focus on effectiveness and dissemination may assist in the development of programmes in population-based effectiveness settings. Future RCTs should attempt to increase their external validity by including representative participants and answering real-world questions [32
]. This review found such characteristics of design lacking, with the common use of small, homogenous or unrepresentative samples and for some a lack of comparison conditions relevant to real-world decisions. Such characteristics significantly limit the dissemination of such interventions into practice [17
Although determining cost effectiveness was not the purpose of this review, it is recommended that future studies at the very least report on basic economic measures such as costs. This information is of great relevance to decision makers and can assist in intervention uptake, dissemination and inform more advanced cost-effectiveness studies [17
]. Cost-effectiveness analyses of computer-tailored interventions would also be worthwhile. The potential cost savings gained by participants due to no travel time or costs may be particularly important for those living in rural or isolated areas.
There was a fundamental lack of long-term post-intervention follow-up, with only one study demonstrating that intervention effects were maintained at 2 years post-baseline [21
]. However, the generalizability of this study's findings and application to practice may be limited. More studies with long-term follow-up of 12 months post-intervention are needed [34
Previously noted poor retention rates of computer-tailored interventions, in particular web-based interventions [4
], prompted consideration of characteristics of interventions that might maintain engagement and retention. Such characteristics include the intervention's interactivity, duration and intensity, the length of follow-up, setting and sample population characteristics. However, with the small number of studies comparing retention rates became problematic due to their varied follow-up length and therefore we could not form any definite conclusions. Based on the findings of this review and other published reviews [5
], it seems the following intervention characteristics may be important in enhancing participant retention: ensuring multiple exposures to the intervention material, preferably using controlled programme delivery; the use of incentives; prompts through another medium; interactive and dynamic web components and individualized tailoring. Each of these characteristics may be insufficient on its own to result in good retention and therefore all will need to be considered in intervention design, sample size calculations and probable retention rates in the future.
The limitations of this review must also be acknowledged. Firstly, this review did not actively seek unpublished studies although one such study was included. Therefore, when considering the findings of this review, the possibility of publication bias should be noted, resulting in a bias of studies with positive findings. However, given the fairly high proportion of published studies reviewed that did not have significant findings, it is believed that the likelihood of publication bias is minimal.
Secondly, this review did not include articles in which dietary behaviour was not a primary outcome. This meant articles were excluded in which psychological indicators, behaviour mediators or process measures were the only outcome measures reported. Process measures were not described, limiting this paper's discussion on retention, engagement and acceptability of computer-tailored interventions and their components in different population subgroups and settings. Although this was not the purpose of the review, reviewing computer-tailored intervention research describing process outcomes would be worthwhile as it may indicate different levels of acceptance and the relative effectiveness in different population subgroups. This may be particularly important given the majority of reviewed studies had predominantly female, Caucasian and well-educated samples.
Thirdly, this review has not attempted to estimate a pooled effect size or to calculate and compare effect sizes of different studies due to the heterogeneity of studies in terms of their intervention design, delivery method, exposure and intensity, participants, study design and methods and outcome measures. Such factors make comparisons difficult [35
] and inadequate [12
], and hence a narrative systematic review was conducted. The two previous reviews on computer-tailored or interactive technology health behaviour interventions most relevant to this review reported small to medium effect sizes [4
]. Despite the small effect sizes found, such interventions can have substantial impact at a population health level, with their potential for wide distribution at relatively low cost compared with face-to-face interventions [4
]. However, it will be critical to determine whether such findings are generalizable, can be replicated and to ensure adequate reach and engagement within varied population groups for such interventions.
Lastly, the findings on common characteristics of successful interventions and those with good retention are limited due to the small number of heterogeneous studies included and reliance on varying levels of detail provided in each article. Only a small proportion of the retrieved articles were included in this review. The main reasons for this include: many studies were duplicated in the databases that were searched, broad search terms were used and the exclusion criteria were specific and detailed. For example, the search terms did not distinguish between first- and second-generation interventions, and first-generation interventions which make up a substantial proportion of the literature were not considered in this review.
Future research should endeavour to replicate studies in different populations to indicate effectiveness and generalizability. The work of Vandelanotte and colleagues where the same theory-based intervention was trialed and adapted in different population groups and settings and followed up long term [20
] is important in building the evidence base. Their reports on the acceptability and feasibility of these interventions in individuals of different age, sex, education level and computer literacy [8
] are also valuable.