Implementation intervention effects can only be fully realised and understood if they are faithfully delivered. However the evaluation of implementation intervention fidelity is not commonly undertaken. The IMPLEMENT intervention was designed to improve the management of low back pain by general medical practitioners. It consisted of a two-session interactive workshop, including didactic presentations and small group discussions by trained facilitators. This study aimed to evaluate the fidelity of the IMPLEMENT intervention by assessing: (1) observed facilitator adherence to planned behaviour change techniques (BCTs); (2) comparison of observed and self-reported adherence to planned BCTs and (3) variation across different facilitators and different BCTs.
The study compared planned and actual, and observed versus self-assessed delivery of BCTs during the IMPLEMENT workshops.
Workshop sessions were audiorecorded and transcribed verbatim. Observed adherence of facilitators to the planned intervention was assessed by analysing the workshop transcripts in terms of BCTs delivered. Self-reported adherence was measured using a checklist completed at the end of each workshop session and was compared with the ‘gold standard’ of observed adherence using sensitivity and specificity analyses.
The overall observed adherence to planned BCTs was 79%, representing moderate-to-high intervention fidelity. There was no significant difference in adherence to BCTs between the facilitators. Sensitivity of self-reported adherence was 95% (95% CI 88 to 98) and specificity was 30% (95% CI 11 to 60).
The findings suggest that the IMPLEMENT intervention was delivered with high levels of adherence to the planned intervention protocol.
Trial registration number
The IMPLEMENT trial was registered in the Australian New Zealand Clinical Trials Registry, ACTRN012606000098538 (http://www.anzctr.org.au/trial_view.aspx?ID=1162).
Multidisciplinary team (MDT) meetings are the core mechanism for delivering mental health care but it is unclear which models improve care quality. The aim of the study was to agree recommendations for improving the effectiveness of adult mental health MDT meetings, based on national guidance, research evidence and experiential insights from mental health and other medical specialties.
We established an expert panel of 16 health care professionals, policy-makers and patient representatives. Five panellists had experience in a range of adult mental health services, five in heart failure services and six in cancer services. Panellists privately rated 68 potential recommendations on a scale of one to nine, and re-rated them after panel discussion using the RAND/UCLA Appropriateness Method to determine consensus.
We obtained agreement (median ≥ 7) and low variation in extent of agreement (Mean Absolute Deviation from Median of ≤1.11) for 21 recommendations. These included the explicit agreement and auditing of MDT meeting objectives, and the documentation and monitoring of treatment plan implementation.
Formal consensus development methods that involved learning across specialities led to feasible recommendations for improved MDT meeting effectiveness in a wide range of settings. Our findings may be used by adult mental health teams to reflect on their practice and facilitate improvement. In some other contexts, the recommendations will require modification. For example, in Child and Adolescent Mental Health Services, context-specific issues such as the role of carers should be taken into account. A limitation of the comparative approach adopted was that only five members of the panel of 16 experts were mental health specialists.
Electronic supplementary material
The online version of this article (doi:10.1186/s12888-015-0534-6) contains supplementary material, which is available to authorized users.
Multidisciplinary team; Recommendations; Chronic diseases; Consensus development method; Adult mental health
Despite the availability of evidence-based guidelines for the management of mild traumatic brain injury in the emergency department (ED), variations in practice exist. Interventions designed to implement recommended behaviours can reduce this variation. Using theory to inform intervention development is advocated; however, there is no consensus on how to select or apply theory. Integrative theoretical frameworks, based on syntheses of theories and theoretical constructs relevant to implementation, have the potential to assist in the intervention development process. This paper describes the process of applying two theoretical frameworks to investigate the factors influencing recommended behaviours and the choice of behaviour change techniques and modes of delivery for an implementation intervention.
A stepped approach was followed: (i) identification of locally applicable and actionable evidence-based recommendations as targets for change, (ii) selection and use of two theoretical frameworks for identifying barriers to and enablers of change (Theoretical Domains Framework and Model of Diffusion of Innovations in Service Organisations) and (iii) identification and operationalisation of intervention components (behaviour change techniques and modes of delivery) to address the barriers and enhance the enablers, informed by theory, evidence and feasibility/acceptability considerations. We illustrate this process in relation to one recommendation, prospective assessment of post-traumatic amnesia (PTA) by ED staff using a validated tool.
Four recommendations for managing mild traumatic brain injury were targeted with the intervention. The intervention targeting the PTA recommendation consisted of 14 behaviour change techniques and addressed 6 theoretical domains and 5 organisational domains. The mode of delivery was informed by six Cochrane reviews. It was delivered via five intervention components : (i) local stakeholder meetings, (ii) identification of local opinion leader teams, (iii) a train-the-trainer workshop for appointed local opinion leaders, (iv) local training workshops for delivery by trained local opinion leaders and (v) provision of tools and materials to prompt recommended behaviours.
Two theoretical frameworks were used in a complementary manner to inform intervention development in managing mild traumatic brain injury in the ED. The effectiveness and cost-effectiveness of the developed intervention is being evaluated in a cluster randomised trial, part of the Neurotrauma Evidence Translation (NET) program.
Intervention design; Intervention development; Theory use; Theoretical domains framework; Diffusion of innovations in service organisations
Mobile phone apps have the potential to reduce excessive alcohol consumption cost-effectively. Although hundreds of alcohol-related apps are available, there is little information about the behavior change techniques (BCTs) they contain, or the extent to which they are based on evidence or theory and how this relates to their popularity and user ratings.
Our aim was to assess the proportion of popular alcohol-related apps available in the United Kingdom that focus on alcohol reduction, identify the BCTs they contain, and explore whether BCTs or the mention of theory or evidence is associated with app popularity and user ratings.
We searched the iTunes and Google Play stores with the terms “alcohol” and “drink”, and the first 800 results were classified into alcohol reduction, entertainment, or blood alcohol content measurement. Of those classified as alcohol reduction, all free apps and the top 10 paid apps were coded for BCTs and for reference to evidence or theory. Measures of popularity and user ratings were extracted.
Of the 800 apps identified, 662 were unique. Of these, 13.7% (91/662) were classified as alcohol reduction (95% CI 11.3-16.6), 53.9% (357/662) entertainment (95% CI 50.1-57.7), 18.9% (125/662) blood alcohol content measurement (95% CI 16.1-22.0) and 13.4% (89/662) other (95% CI 11.1-16.3). The 51 free alcohol reduction apps and the top 10 paid apps contained a mean of 3.6 BCTs (SD 3.4), with approximately 12% (7/61) not including any BCTs. The BCTs used most often were “facilitate self-recording” (54%, 33/61), “provide information on consequences of excessive alcohol use and drinking cessation” (43%, 26/61), “provide feedback on performance” (41%, 25/61), “give options for additional and later support” (25%, 15/61) and “offer/direct towards appropriate written materials” (23%, 14/61). These apps also rarely included any of the 22 BCTs frequently used in other health behavior change interventions (mean 2.46, SD 2.06). Evidence was mentioned by 16.4% of apps, and theory was not mentioned by any app. Multivariable regression showed that apps including advice on environmental restructuring were associated with lower user ratings (Β=-46.61, P=.04, 95% CI -91.77 to -1.45) and that both the techniques of “advise on/facilitate the use of social support” (Β=2549.21, P=.04, 95% CI 96.75-5001.67) and the mention of evidence (Β=1376.74, P=.02, 95%, CI 208.62-2544.86) were associated with the popularity of the app.
Only a minority of alcohol-related apps promoted health while the majority implicitly or explicitly promoted the use of alcohol. Alcohol-related apps that promoted health contained few BCTs and none referred to theory. The mention of evidence was associated with more popular apps, but popularity and user ratings were only weakly associated with the BCT content.
alcohol; behaviour change; mHealth; smartphone; iPhone; android; apps; digital; intervention
Digital interventions provide effective and potentially cost-effective models for improving health outcomes as they deliver health information and services that are widely disseminated, confidential, and can be tailored to needs of the individual user. Digital interventions have been used successfully for health promotion, mental health, and for enabling self-management of long-term conditions. However, their effectiveness is limited by low usage rates, with non-engagement a major challenge. Hence, it is crucial to find effective strategies to increase user engagement with digital interventions.
This systematic review will aim to evaluate the effectiveness of technology-based strategies to promote engagement with digital interventions.
We will follow Cochrane Collaboration guidelines on systematic review methodology. The search strategy will be executed across seven e-databases (including MEDLINE, EMBASE, PsycINFO, CINAHL) using the concepts “digital intervention” and “engagement”, limited by study type (randomized controlled trial). Grey literature and reference lists of included studies will be searched. Titles and abstracts will be independently screened by 2 authors. Then the full text of potentially eligible papers will be obtained and double screened. Data from eligible papers will be extracted by 1 author and checked for accuracy by another author. Bias will be assessed using the Cochrane bias assessment tool. Narrative synthesis will report on all included studies, and where appropriate, data will be pooled using meta-analysis. All findings will be reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Sources of heterogeneity will be further investigated if required.
Our research is in progress. The final draft of the systematic review is being written and will be submitted before the end of 2015.
The review findings will inform researchers and digital intervention providers about optimal use of technology-based strategies to promote engagement with digital interventions.
PROSPERO International Prospective Register of Systematic Reviews: CRD42014010164; http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014010164#.VTZmmiFViko (Archived by WebCite at http://www.webcitation.org/6XxQC8fT8).
systematic review; Internet; computers; Web-based interventions; engagement; adherence; attrition; usage
The implementation of research findings into healthcare practice has become increasingly recognised as a major priority for researchers, service providers, research funders and policymakers over the past decade. Nine years after its establishment, Implementation Science, an international online open access journal, currently publishes over 150 articles each year. This is fewer than 30% of those submitted for publication. The majority of manuscript rejections occur at the point of initial editorial screening, frequently because we judge them to fall outside of journal scope. There are a number of common reasons as to why manuscripts are rejected on grounds of scope. Furthermore, as the field of implementation research has evolved and our journal submissions have risen, we have, out of necessity, had to become more selective in what we publish. We have also expanded our scope, particularly around patient-mediated and population health interventions, and will monitor the impact of such changes. We hope this editorial on our evolving priorities and common reasons for rejection without peer review will help authors to better judge the relevance of their papers to Implementation Science.
Implementation research; Methodology
Melioidosis causes more than 1,000 deaths in Thailand each year. Infection occurs via inoculation, ingestion or inhalation of the causative organism (Burkholderia pseuodmallei) present in soil and water. Here, we evaluated public awareness of melioidosis using a combination of population-based questionnaire, a public engagement campaign to obtain video clips made by the public, and viewpoints on these video clips as potential educational tools about the disease and its prevention.
A questionnaire was developed to evaluate public awareness of melioidosis, and knowledge about its prevention. From 1 March to 31 April 2012, the questionnaire was delivered to five randomly selected adults in each of 928 districts in Thailand. A video clip contest entitled “Melioidosis, an infectious disease that Thais must know” was run between May and October 2012. The best 12 video clips judged by a contest committee were shown to 71 people at risk from melioidosis (diabetics). Focus group interviews were used to evaluate their perceptions of the video clips.
Of 4,203 Thais who completed our study questionnaire, 74% had never heard of melioidosis, and 19% had heard of the disease but had no further knowledge. Most participants in all focus group sessions felt that video clips were beneficial and could positively influence them to increase adherence to recommended preventive behaviours, including drinking boiled water and wearing protective gear if in contact with soil or environmental water. Participants suggested that video clips should be presented in the local dialect with simple words rather than medical terms, in a serious manner, with a doctor as the one presenting the facts, and having detailed pictures of each recommended prevention method.
In summary, public awareness of melioidosis in Thailand is very low, and video clips could serve as a useful medium to educate people and promote disease prevention.
Presented in Part
World Melioidosis Congress 2013, Bangkok, Thailand, 18–20 September 2013 (abstract OS VII-04).
The Theoretical Domains Framework (TDF) is an integrative framework developed from a synthesis of psychological theories as a vehicle to help apply theoretical approaches to interventions aimed at behavior change.
This study explores experiences of TDF use by professionals from multiple disciplines across diverse clinical settings.
Mixed methods were used to examine experiences, attitudes, and perspectives of health professionals in using the TDF in health care implementation projects. Individual interviews were conducted with ten health care professionals from six disciplines who used the TDF in implementation projects. Deductive content and thematic analysis were used.
Three main themes and associated subthemes were identified including: 1) reasons for use of the TDF (increased confidence, broader perspective, and theoretical underpinnings); 2) challenges using the TDF (time and resources, operationalization of the TDF) and; 3) future use of the TDF.
The TDF provided a useful, flexible framework for a diverse group of health professionals working across different clinical settings for the assessment of barriers and targeting resources to influence behavior change for implementation projects. The development of practical tools and training or support is likely to aid the utility of TDF.
barriers and enablers; behavioral change; evidence-based practice; implementation; health care; Theoretical Domains Framework
Timely tracking of national patterns of alcohol consumption is needed to inform and evaluate strategies and policies aimed at reducing alcohol-related harm. Between 2014 until at least 2017, the Alcohol Toolkit Study (ATS) will provide such tracking data and link these with policy changes and campaigns. By virtue of its connection with the ‘Smoking Toolkit Study’ (STS), links will also be examined between alcohol and smoking-related behaviour.
The ATS consists of cross-sectional household, computer-assisted interviews of representative samples of adults in England aged 16+. Each month a new sample of approximately 1800 adults complete the survey (~n = 21,600 per year). All respondents who consent to be followed-up are asked to complete a telephone survey 6 months later. The ATS has been funded to collect at least 36 waves of baseline and 6-month follow-up data across a period of 3 years. Questions cover alcohol consumption and related harm (AUDIT), socio-demographic characteristics, attempts to reduce or cease consumption and factors associated with this, and exposure to health professional advice on alcohol. The ATS complements the STS, which has been tracking key performance indicators relating to smoking since 2006. As both the ATS and STS involve the same respondents, it is possible to assess interactions between changes in alcohol and tobacco use. Data analysis will involve: 1) Descriptive and exploratory analyses undertaken according to a pre-defined set of principles while allowing scope for pursuing lines of enquiry that arise from prior analyses; 2) Hypothesis testing according to pre-specified, published analysis plans. Descriptive data on important trends will be published monthly on a dedicated website: www.alcoholinengland.info.
The Alcohol Toolkit Study will improve understanding of population level factors influencing alcohol consumption and be an important resource for policy evaluation and planning.
Alcohol toolkit study; Epidemiology; Smoking toolkit study; Alcohol consumption; AUDIT
The role and value of theory in improvement work in healthcare has been seriously underrecognised. We join others in proposing that more informed use of theory can strengthen improvement programmes and facilitate the evaluation of their effectiveness. Many professionals, including improvement practitioners, are unfortunately mystified—and alienated—by theory, which discourages them from using it in their work. In an effort to demystify theory we make the point in this paper that, far from being discretionary or superfluous, theory (‘reason-giving’), both informal and formal, is intimately woven into virtually all human endeavour. We explore the special characteristics of grand, mid-range and programme theory; consider the consequences of misusing theory or failing to use it; review the process of developing and applying programme theory; examine some emerging criteria of ‘good’ theory; and emphasise the value, as well as the challenge, of combining informal experience-based theory with formal, publicly developed theory. We conclude that although informal theory is always at work in improvement, practitioners are often not aware of it or do not make it explicit. The germane issue for improvement practitioners, therefore, is not whether they use theory but whether they make explicit the particular theory or theories, informal and formal, they actually use.
Evaluation methodology; Social sciences; Complexity; Quality improvement methodologies; Implementation science
Underestimating one's own alcohol consumption relative to others (‘normative misperception’) has been documented in some college student and heavy-alcohol using samples, and may contribute to excessive drinking. This study aimed to assess how far this phenomenon extends to alcohol users more generally in four English-speaking countries and if associations with socio-demographic and drinking variables exist.
A cross-sectional online global survey (Global Drugs Survey-2012) was completed by 9820 people aged 18 + from Australia, Canada, the UK and US who had consumed alcohol in the last year. The survey included the AUDIT questionnaire (which assessed alcohol consumption, harmful drinking and alcohol dependence), socio-demographic assessment and a question assessing beliefs about how one's drinking compares with others. Associations were analysed by linear regression models.
Underestimation of own alcohol use relative to others occurred in 46.9% (95% CI: 45.9%, 47.9%) of respondents. 25.4% of participants at risk of alcohol dependence and 36.6% of harmful alcohol users believed their drinking to be average or less. Underestimation was more likely among those who were: younger (16–24; p < 0.003), male (p < 0.001), from the UK (versus US; p < 0.001), less well educated (p = 0.003), white (p = 0.035), and unemployed (versus employed; p < 0.001).
Underestimating one's own alcohol consumption relative to other drinkers is common in Australia, Canada, the UK and US, with a substantial minority of harmful drinkers believing their consumption to be at or below average. This normative misperception is greater in those who are younger, male, less well educated, unemployed, white, from the UK and high-risk drinkers.
Normative misperception; Alcohol; AUDIT
During the 2009-2010 A(H1N1) pandemic, many people did not seek care quickly enough, failed to take a full course of antivirals despite being authorised to receive them, and were not vaccinated. Understanding facilitators and barriers to the uptake of vaccination and antiviral medicines will help inform campaigns in future pandemic influenza outbreaks. Increasing uptake of vaccines and antiviral medicines may need to address a range of drivers of behaviour. The aim was to identify facilitators of and barriers to being vaccinated and taking antiviral medicines in uncertain and severe pandemic influenza scenarios using a theoretical model of behaviour change, COM-B.
Focus groups and interviews with 71 members of the public in England who varied in their at-risk status. Participants responded to uncertain and severe scenarios, and to messages giving advice on vaccination and antiviral medicines. Data were thematically analysed using the theoretical framework provided by the COM-B model.
Influences on uptake of vaccines and antiviral medicines - capabilities, motivations and opportunities - are part of an inter-related behavioural system and different components influenced each other. An identity of being healthy and immune from infection was invoked to explain feelings of invulnerability and hence a reduced need to be vaccinated, especially during an uncertain scenario. The identity of being a ‘healthy person’ also included beliefs about avoiding medicine and allowing the body to fight disease ‘naturally’. This was given as a reason for using alternative precautionary behaviours to vaccination. This identity could be held by those not at-risk and by those who were clinically at-risk.
Promoters and barriers to being vaccinated and taking antiviral medicines are multi-dimensional and communications to promote uptake are likely to be most effective if they address several components of behaviour. The benefit of using the COM-B model is that it is at the core of an approach that can identify effective strategies for behaviour change and communications for the future. Identity beliefs were salient for decisions about vaccination. Communications should confront identity beliefs about being a ‘healthy person’ who is immune from infection by addressing how vaccination can boost wellbeing and immunity.
Electronic supplementary material
The online version of this article (doi:10.1186/s12889-015-1541-8) contains supplementary material, which is available to authorized users.
Pandemic influenza; Vaccination; Antiviral medicines; Behaviour
Sexually transmitted infections (STI) are a major public health problem. Condoms provide effective protection but there are many barriers to use. Face-to-face health promotion interventions are resource-intensive and show mixed results. Interactive digital interventions may provide a suitable alternative, allowing private access to personally tailored behaviour change support. We have developed an interactive digital intervention (the Men's Safer Sex (MenSS) website) which aims to increase condom use in men. We describe the protocol for a pilot trial to assess the feasibility of a full-scale randomised controlled trial of the MenSS website in addition to usual sexual health clinical care.
Methods and analysis
Participants: Men aged 16 or over who report female sexual partners and recent unprotected sex or suspected acute STI. Participants (N=166) will be enrolled using a tablet computer in clinic waiting rooms. All trial procedures will be online, that is, eligibility checks; study consent; trial registration; automated random allocation; and data submission. At baseline and at 3, 6 and 12 months, an online questionnaire will assess condom use, self-reported STI diagnoses, and mediators of condom use (eg, knowledge, intention). Reminders will be by email and mobile phone. The primary outcome is condom use, measured at 3 months. STI rates will be recorded from sexual health clinic medical records at 12 months. The feasibility of a cost-effectiveness analysis will be assessed, to calculate incremental cost per STI prevented (Chlamydia or Gonorrhoea), from the NHS perspective.
Ethics and dissemination
Ethical approval: City and East NHS Research Ethics Committee (reference number 13 LO 1801). Findings will be made available through publication in peer-reviewed journals, and to participants and members of the public via Twitter and from the University College London eHealth Unit website. Raw data will be made available on request.
Trial registration number
Current Controlled Trials. ISRCTN18649610. Registered 15 October 2013 http://www.controlled-trials.com/ISRCTN18649610.
sexual health; randomized controlled trial; online trial; interactive digital intervention; behavior change; complex intervention
Research is needed on what influences recruitment to smoking reduction trials, and how to increase their reach. The present study aimed to i) assess the feasibility of recruiting a disadvantaged population, ii) examine the effects of recruitment methods on participant characteristics, iii) identify resource requirements for different recruitment methods, and iv) to qualitatively assess the acceptability of recruitment. This was done as part of a pilot two-arm trial of the effectiveness of a novel behavioral support intervention focused on increasing physical activity and reducing smoking, among disadvantaged smokers not wishing to quit.
Smokers were recruited through mailed invitations from three primary care practices (62 participants) and one National Health Stop Smoking Service (SSS) database (31 participants). Six other participants were recruited via a variety of other community-based approaches. Data were collected through questionnaires, field notes, work sampling, and databases. Chi-squared and t-tests were used to compare baseline characteristics of participants.
We randomized between 5.1 and 11.1% of those invited through primary care and SSS, with associated researcher time to recruit one participant varying from 18 to 157 minutes depending on time and intensity invested.Only six participants were recruited through a wide variety of other community-based approaches, with an associated researcher time of 469 minutes to recruit one participant. Targets for recruiting a disadvantaged population were met, with 91% of the sample in social classes C2 to E (NRS social grades, UK), and 41% indicating mental health problems. Those recruited from SSS were more likely to respond to an initial letter, had used cessation aids before, and had attempted to quit in the past year. Overall, initial responders were more likely to be physically active than those who were recruited via follow-up telephone calls. No other demographics or behaviour characteristics were associated with recruitment approach or intensity of effort. Qualitative feedback indicated that participants had been attracted by the prospect of support that focused on smoking reduction rather than abrupt quitting.
Mailed invitations, and follow-up, from health professionals was an effective method of recruiting disadvantaged smokers into a trial of an exercise intervention to aid smoking reduction. Recruitment via community outreach approaches was largely ineffective.
ISRCTN identifier: 13837944, registered on 6 July 2010
Harm reduction; Tobacco control; Physical activity; Recruitment; Smoking cessation; Primary care; Community medicine
Little is known about the effectiveness of mobile apps in aiding smoking cessation or their validity for automated collection of data on smoking cessation outcomes.
We conducted a preliminary evaluation of SF28 (SF28 is the name of the app, short for SmokeFree28)—an app aimed at helping smokers to be smoke-free for 28 days.
Data on sociodemographic characteristics, smoking history, number of logins, and abstinence at each login were uploaded to a server from SF28 between August 2012 and August 2013. Users were included if they were aged 16 years or over, smoked cigarettes at the time of registration, had set a quit date, and used the app at least once on or after their quit date. Their characteristics were compared with data from a representative sample of smokers trying to stop smoking in England. The percentage of users recording 28 days of abstinence was compared with a value of 15% estimated for unaided quitting. Correlations were assessed between recorded abstinence for 28 days and well-established abstinence predictors.
A total of 1170 users met the inclusion criteria. Compared with smokers trying to quit in England, they had higher consumption, and were younger, more likely to be female, and had a non-manual rather than manual occupation. In total, 18.9% (95% CI 16.7-21.1) were recorded as being abstinent from smoking for 28 days or longer. The mean number of logins was 8.5 (SD 9.0). The proportion recording abstinence for 28 days or longer was higher in users who were older, in a non-manual occupation, and in those using a smoking cessation medication.
The recorded 28-day abstinence rates from the mobile app, SF28, suggest that it may help some smokers to stop smoking. Further evaluation by means of a randomized trial appears to be warranted.
smoking cessation intervention; mobile; smartphone; apps; PRIME theory
Behaviour Change Technique Taxonomy v1 (BCTTv1) has been used to detect active ingredients of interventions. The purpose of this study was to evaluate effectiveness of user training in improving reliable, valid and confident application of BCTTv1 to code BCTs in intervention descriptions. One hundred sixty-one trainees (109 in workshops and 52 in group tutorials) were trained to code frequent BCTs. The following measures were taken before and after training: (i) inter-coder agreement, (ii) trainee agreement with expert consensus, (iii) confidence ratings and (iv) coding competence. Coding was assessed for 12 BCTs (workshops) and for 17 BCTs (tutorials). Trainees completed a course evaluation. Methods improved agreement with expert consensus (p < .05) but not inter-coder agreement (p = .08, p = .57, respectively) and increased confidence for BCTs assessed (both p < .05). Methods were as effective as one another at improving coding competence (p = .55). Training was evaluated positively. The training improved agreement with expert consensus, confidence for BCTs assessed, coding competence but not inter-coder agreement. This varied according to BCT.
Behaviour change techniques; Taxonomy; Training methods
Internet-based health behaviour interventions have variable effects on health-related outcomes. Effectiveness may be improved by optimizing the design of interventions. This study examined the specific effect on engagement of providing two different design features – tailoring and self-assessment.
Three versions of an Internet-delivered intervention to support the self-care of mild bowel problems were developed that provided (1) self-assessment without tailored feedback, (2) self-assessment with tailored feedback, and (3) generic information only.
A qualitative study explored participants’ engagement with each version of the intervention (N = 24). A larger quantitative study systematically compared participants’ use of the intervention and self-reported engagement using a partial factorial design (n = 178).
Findings from the qualitative study suggested that self-assessment without tailored feedback appeared to be less acceptable to participants because it was viewed as offering no personal benefit in the absence of personalized advice. In the quantitative study, self-assessment without tailored feedback was associated with greater dropout than when provided in conjunction with tailored feedback. There were significant group differences in participants’ engagement with the intervention and perceptions of the intervention. Self-assessment without tailored feedback was rated as marginally less engaging and was associated with fewer positive perceptions than the generic information condition.
The acceptability of self-assessment or monitoring components may be optimized by also providing tailored feedback. Without tailored feedback, these components do not appear to be any more engaging than generic information provision.
Internet; tailoring; intervention; mixed methods; self-assessment; engagement; health
Advancements in mobile phone technology offer huge potential for enhancing the timely delivery of health behavior change interventions. The development of smartphone-based health interventions (apps) is a rapidly growing field of research, yet there have been few longitudinal examinations of how people experience and use these apps within their day-to-day routines, particularly within the context of a hybrid Web- and app-based intervention.
This study used an in-depth mixed-methods design to examine individual variation in (1) impact on self-reported goal engagement (ie, motivation, self-efficacy, awareness, effort, achievement) of access to a weight management app (POWeR Tracker) when provided alongside a Web-based weight management intervention (POWeR) and (2) usage and views of POWeR Tracker.
Thirteen adults were provided access to POWeR and were monitored over a 4-week period. Access to POWeR Tracker was provided in 2 alternate weeks (ie, weeks 1 and 3 or weeks 2 and 4). Participants’ goal engagement was measured daily via self-report. Mixed effects models were used to examine change in goal engagement between the weeks when POWeR Tracker was and was not available and whether the extent of change in goal engagement varied between individual participants. Usage of POWeR and POWeR Tracker was automatically recorded for each participant. Telephone interviews were conducted and analyzed using inductive thematic analysis to further explore participants’ experiences using POWeR and POWeR Tracker.
Access to POWeR Tracker was associated with a significant increase in participants’ awareness of their eating (β1=0.31, P=.04) and physical activity goals (β1=0.28, P=.03). The level of increase varied between individual participants. Usage data showed that participants used the POWeR website for similar amounts of time during the weeks when POWeR Tracker was (mean 29 minutes, SD 31 minutes) and was not available (mean 27 minutes, SD 33 minutes). POWeR Tracker was mostly accessed in short bursts (mean 3 minutes, SD 2 minutes) during convenient moments or moments when participants deemed the intervention content most relevant. The qualitative data indicated that nearly all participants agreed that it was more convenient to access information on-the-go via their mobiles compared to a computer. However, participants varied in their views and usage of the Web- versus app-based components and the informational versus tracking tools provided by POWeR Tracker.
This study provides evidence that smartphones have the potential to improve individuals’ engagement with their health-related goals when used as a supplement to an existing online intervention. The perceived convenience of mobile access to information does not appear to deter use of Web-based interventions or strengthen the impact of app access on goal engagement. A mixed-methods design enabled exploration of individual variation in daily usage of the app-based tools.
qualitative research; weight loss; behavioral research; mobile apps; Internet; health; program acceptability; behavior; mixed-methods
Fabrizio Faggiano and colleagues discuss how a central, transparent, and evidence-based approval process is needed for behavioral prevention interventions in Europe and propose a way forward.
Please see later in the article for the Editors' Summary
Paul Glasziou and colleagues discuss methods to guide selection of an intervention from individual trials within a systematic review.
Please see later in the article for the Editors' Summary
Audits of blood transfusion demonstrate around 20% transfusions are outside national recommendations and guidelines. Audit and feedback is a widely used quality improvement intervention but effects on clinical practice are variable, suggesting potential for enhancement. Behavioural theory, theoretical frameworks of behaviour change and behaviour change techniques provide systematic processes to enhance intervention. This study is part of a larger programme of work to promote the uptake of evidence-based transfusion practice.
The objectives of this study are to design two theoretically enhanced audit and feedback interventions; one focused on content and one on delivery, and investigate the feasibility and acceptability.
Study A (Content): A coding framework based on current evidence regarding audit and feedback, and behaviour change theory and frameworks will be developed and applied as part of a structured content analysis to specify the key components of existing feedback documents. Prototype feedback documents with enhanced content and also a protocol, describing principles for enhancing feedback content, will be developed. Study B (Delivery): Individual semi-structured interviews with healthcare professionals and observations of team meetings in four hospitals will be used to specify, and identify views about, current audit and feedback practice. Interviews will be based on a topic guide developed using the Theoretical Domains Framework and the Consolidated Framework for Implementation Research. Analysis of transcripts based on these frameworks will form the evidence base for developing a protocol describing an enhanced intervention that focuses on feedback delivery. Study C (Feasibility and Acceptability): Enhanced interventions will be piloted in four hospitals. Semi-structured interviews, questionnaires and observations will be used to assess feasibility and acceptability.
This intervention development work reflects the UK Medical Research Council’s guidance on development of complex interventions, which emphasises the importance of a robust theoretical basis for intervention design and recommends systematic assessment of feasibility and acceptability prior to taking interventions to evaluation in a full-scale randomised study. The work-up includes specification of current practice so that, in the trials to be conducted later in this programme, there will be a clear distinction between the control (usual practice) conditions and the interventions to be evaluated.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0092-1) contains supplementary material, which is available to authorized users.
Audit and feedback; Blood transfusion; Implementation; Health services research; Study protocol; Health professional behaviour change
Internet-based Smoking Cessation Interventions could help pregnant women quit smoking, especially those who do not wish to, or cannot, access face-to-face or telephone support. This study aimed to preliminarily evaluate the effectiveness and usage of a fully automated smoking cessation website targeted to pregnancy, ‘MumsQuit’, and obtain an initial effect-size estimate for a full scale trial.
We recruited 200 UK-based pregnant adult smokers online to a two-arm double-blind pilot RCT assessing the effectiveness of MumsQuit compared with an information-only website. MumsQuit was adapted from a generic internet smoking cessation intervention, ‘StopAdvisor’. The primary outcome was self-reported continuous 4-week abstinence assessed at 8 weeks post-baseline. Secondary outcomes were automatically collected data on intervention usage.
Participants smoked 15 cigarettes per day on average, 73% were in the first trimester of their pregnancy, 48% were from lower socioeconomic backgrounds, and 43% had never used evidence-based cessation support. The point estimate of odds ratio for the primary outcome was 1.5 (95% CI = 0.8–2.9; 28% vs. 21%). Compared with control participants, those in the MumsQuit group logged in more often (3.5 vs. 1.3, p < 0.001), viewed more pages (67.4 vs. 5.7, p < 0.001) and spent more time browsing the website (21.3 min vs. 1.0 min, p < 0.001).
MumsQuit is an engaging and potentially helpful form of support for pregnant women who seek cessation support online, and merits further development and evaluation in a full-scale RCT.
Smoking cessation; Pregnancy; Internet; Self-help
E-cigarettes may be effective smoking cessation aids and their use by smokers has been growing rapidly. It is important to observe and assess natural patterns in the use of e-cigarettes whilst experimental data accumulates. This paper reports the prevalence of e-cigarette awareness, beliefs and usage, including brand choice, and characterises the socio-demographic and smoking profile associated with current use, among the general population of smokers and recent ex-smokers.
Data were obtained from 3538 current and 579 recent ex-smokers in a cross-sectional online survey of a national sample of smokers in Great Britain in November and December 2012. Differences between current and recent ex-smokers in the prevalence of e-cigarette awareness, beliefs and usage were examined and the socio-demographic and smoking profile associated with current use of e-cigarettes was assessed in a series of simple and multiple logistic regressions.
Ninety-three percent of current and recent ex-smokers (n = 3841) were aware of e-cigarettes. Approximately a fifth (n = 884) were currently using e-cigarettes, whilst just over a third (n = 1507) had ever used them. Sixty-seven percent of the sample (n = 2758) believed e-cigarettes to be less harmful than cigarettes; however, almost a quarter (n = 994) remained unsure. Among both current and recent ex-smokers, the most popular reasons for using were health, cutting down and quitting (each > 80%) and 38% used the brand ‘E-lites’. Among current smokers who were aware of but had never used e-cigarettes, approximately half (n = 1040) were interested in using them in the future. Among current smokers, their use was associated with higher socio-economic status (OR = 1.48, 95%CI = 1.25–1.75), smoking more cigarettes (OR = 1.02, 95%CI = 1.01–1.03) and having a past-year quit attempt (OR = 2.82, 95%CI = 2.38–3.34).
There is a near universal awareness of e-cigarettes and their use appears to be common among smokers in Great Britain although a quarter of all smokers are unsure as to whether e-cigarettes are less harmful than cigarettes. E-lites – a brand that delivers a low dose of nicotine – is the most popular. E-cigarette users appear to have higher socio-economic status, to smoke more cigarettes per day and to have attempted to quit in the past year.
•There is now near universal awareness of e-cigarettes.•Use is common among smokers.•Quarter of all smokers unsure as to whether they are less harmful than cigarettes.•E-lites – a brand that delivers a low dose of nicotine – is the most popular.•Users have higher SES, smoke more heavily and have attempted to quit recently.
Smoking; Cessation; Quitting; E-cigarettes; Electronic cigarettes