In terms of the case managers, the study was a randomised controlled trial that compared training against no training. This took place in the South London and Maudsley NHS Trust (now Foundation Trust), located in inner London, and was open to all community mental health teams in the organisation. Ethical approval was granted by the Institute of Psychiatry Ethics Committee (now joint South London and Maudsley and Institute of Psychiatry NHS Research Ethics Committee) and was obtained on 17th September, 1999 (reference 075/99). Case managers within the recruited teams were randomly allocated to either training or no training. Informed consent was obtained from the case managers to participate in the trial. They were informed that all aspects of the trial were voluntary, and they could withdraw at any time.
Case managers who were randomised to the control group were offered the training once the follow-up data had been collected at 18 months post-training. To be eligible for inclusion, participants were expected to be working with their caseloads continuously throughout the 18 month research period.
The sample size was calculated for the broader study on the basis of three service user outcomes using p = 0.05 and 80% power. The largest sample size of the three was adopted (220 patients). In terms of case managers, we assumed that 20% of the community mental health case-loads would be people with co-morbid psychosis and substance misuse. Therefore from an average case-load of 20 we would expect to identify 4 service users per case manager, so that if we aimed for a service user sample of 220, we would need to recruit 55 case managers. The number of case managers actually recruited was 79 which was large enough to detect a standardised effect of 0.65 on the Knowledge about Dual Diagnosis (KADD) questionnaire.
Baseline data were collected from participating staff within those teams prior to knowledge of whether they had been randomly allocated to the training group. In order to minimise bias, the research assistants were blind to randomisation, and the randomisation itself was performed by an independent statistician to maintain concealment.
Primary Outcome Measure
There are to date no valid and reliable measures of attitudes specifically for working with people with co-morbid mental health and substance use. The Alcohol and Alcohol Problems Perception Questionnaire (AAPPQ) [
18] was chosen as it has demonstrated reliability and validity in measuring attitudes towards working with alcohol drinkers by generic workers (including mental health workers). It consists of a series of statements about aspects of working with drinkers. Each statement is rated on a 7 point scale from strongly disagree to strongly agree. In this study, the AAPPQ items were repeated, replacing "drinkers" with "drug users" to obtain a measure of attitudes towards drug users.
There are 6 subscales to the AAPPQ: the agents willingness to work with drinkers/drug users (motivation); their self-reported adequacy in knowledge and skills in working with drinkers/drug users, their self-esteem in working with drinkers/drug users, the extent to which they have the right to work with drinkers/drug users (role legitimacy); expectations of job satisfaction in working with drinkers/drug users; and role support in working with drinkers/drug users. Cartwright and colleagues [
18] reported that there was good test re-test reliability, and Cronbach alpha coefficients for the scales ranged between 0.7 and 0.9. It has been used in previous studies as a measure of attitudes of mental health workers towards working with drinkers. Lightfoot and Orford [
19] used the AAPPQ with community mental health nurses and social workers. They reported an overall Cronbach alpha of 0.83 with the range for the subscales of between 0.7 and 0.9. In this study, good internal consistency was demonstrated; the items for the overall AAPPQ scale had a Cronbach alpha coefficient of 0.9, and the subscales ranged from 0.7– 0.9.
Secondary Outcome Measures
Basic information about professional background, grade, length of time in profession, previous study relevant to dual diagnosis, and previous clinical experience in substance use field was collected from each participant. The secondary outcomes were measured by scales designed specifically for the study and are therefore intended for exploratory data analyses to generate hypotheses.
The Self-Efficacy Scale (SES) Dual Diagnosis Attitudes (DDA) and the Knowledge about Dual Diagnosis Questionnaires (KADD) were developed for the study as a literature search revealed that dual diagnosis specific measures of knowledge and confidence did not exist at that time. These measures were devised by an expert group with experience in working with and training mental health workers in dual diagnosis interventions. The group agreed that the final versions of the SES, the DDA, and the KADD had face and content validity. The measures were administered to 3 dual diagnosis experts (independent of the study) and they all scored highly on the scales.
The SES consisted of a list of key skills which related to working with someone with mental health and substance use problems. This included assessment, health education, and interventions such as motivational interviewing techniques. Each item was rated from 0% which indicated no confidence at all in being able to perform the specific skill, to 100% (totally confident in ones ability). It had a good internal consistency with a Cronbach alpha coefficient of 0.97, and a range of inter-item correlations from 0.537–1.000. Due to the high internal consistency of this scale future work should assess whether the scale could be refined by using a smaller subset of items.
The DDA consisted of a series of statements about dual diagnosis and each item was rated on a four point scale of 1 disagree strongly to 4 agree strongly. There was no neutral option in order to polarise the views. It had a low internal consistency with a Cronbach alpha coefficient of 0.58. The KADD was a 20 item multiple choice questionnaire based on the content of the dual diagnosis training. Trainees were asked to choose the correct response from a choice of 4. The KADD also had a good internal consistency (Cronbach alpha of 0.70).
In addition to the previously mentioned scales, job satisfaction and burn-out were measured by using the Minnesota Job Satisfaction Scale (MSS) [
20], and the Maslach Burn-Out Inventory (MBI) [
21]. Burn-out and job dissatisfaction were measured as it has been suggested that trainees are less likely to implement training if they are feeling burnt-out and unhappy in their work. The MSS is a valid and reliable tool that consists of 20 items, and it assesses aspects of work that are a source of satisfaction or dissatisfaction. Each item is assessed on a 5 point scale ranging from 1 (very dissatisfied) to 5 (very satisfied). A score of 60 indicates a neutral level of job satisfaction, and a score over 80 indicates job satisfaction. The MBI is a valid and reliable tool designed to assess burn-out in people whose occupations involve the delivery of care. It consists of 22 items and respondents are asked to rate how often they experience a range of positive and negative work-related feelings, using a scale ranging from 0 (never experienced job related feelings) to 7 (experiencing job related feelings on a daily basis). Three sub scores are derived from the responses: depersonalisation (feeling emotionally detached and callous), emotional exhaustion (feeling emotionally drained and fatigued), and lack of personal accomplishment (feeling that no matter what they do nothing makes a difference; feeling de-skilled). Maslach and Jackson [
21] calculated the range of scores for burn-out for different professionals. For mental health workers, high burn-out is indicted by a score of 8 or more on depersonalisation; 21 or more for emotional exhaustion and 28 or less for personal accomplishment. Average burn-out for the subscales falls within the following ranges: depersonalisation 5–7; emotional exhaustion 14–20; and personal accomplishment 33–29.
The Training Intervention
The training package was designed to increase mental health case managers' skills and competencies to detect, assess, and intervene with co-morbid substance use problems. The expectation was that case managers would integrate the key skills from the training into their practice, so that they could begin to address substance use. In addition to individual work, the training also addressed issues of referral to other agencies (such as specialist substance use services) or joint working with other professionals if the case was complex. The training course was too brief to produce "experts" in dual diagnosis interventions; the aim (within the limits of time and resources) was to help case managers be more effective in working with co-morbid substance use issues.
The training was based on the Institute of Psychiatry, Kings College London, 12 day accredited dual diagnosis module (which was the first accredited clinical skills course in dual diagnosis interventions in the UK), and comprised of 5 days of classroom based training, delivered one day per week, and an 80 page treatment manual. The training utilised a range of methods including didactic presentations, small group discussion and exercises, clinical case discussions, reflection on practice, and role-play. The teaching methods were designed to link theory to the case managers' previous experience, knowledge and clinical experience. After the training had ended, the trainees were offered monthly one hour supervision sessions for the length of the 18 month period post training. The content of the training included the Integrated Treatment Approach [
22] which included comprehensive assessment, step-wise working (using Osher and Kofoed's Four Stage Model) [
23], taking a flexible and long term view of working with dual diagnosis, Motivational Interviewing techniques (such assessing readiness to change and working with ambivalence) [
24], and cognitive behavioural techniques for psychosis including assessment of problem areas and relapse prevention [
25].
For consistency, the training and supervision were provided by the person who developed the training package (EH). The skill components of the training were demonstrated by video clips, and also described in detail within the manual. Trainees practiced the skills within the training, and were required to try out the skills in clinical practice and discuss progress within the classroom and in follow-up supervision.
Statistical Analysis
The strength of evidence for any differences between the two groups was assessed using analysis of covariance with baseline scores as covariate, and post treatment scores as dependent variable. Data were input to databases using SPSS and analysed using SPSS 14 [
26] and analysis was performed on all participants who had been originally randomised to receive training or waiting list control, and who had follow-up data. Outcomes were analysed irrespective of whether or not the case manager had participated in or completed the training (intention to treat). Statistical significance was evaluated at p = 0.05 level. No adjustment was made for multiple testing as the secondary outcomes were exploratory in nature. In order to address whether membership of different teams affected the results, the analysis of covariance was repeated with "team" as an additional fixed factor, and the results were compared with the first analysis. In addition, analyses were repeated including any baseline variables that appeared to characterise loss to follow up.