We believe this is the first study to apply the rigorous standards of a health promotion model to a mental health population intervention. The program achieved many of its aims despite the relative short duration and moderate intensity of the campaign.
Process evaluation revealed that the website was a far more frequently used source of information than the information line, and was more effective at attracting use by the target age group of young people. Newspapers, schools, posters and General Practitioners were the most effective website referral sources. Schools were also the greatest source of service provider demand for print materials.
Results of the impact evaluation telephone survey revealed significant changes in the target region across the range of impact targets. The overall recall of mental health campaigns was significantly higher in the target region. This suggests that the enabling factor objective of making mental health information more available was achieved. Three key predisposing and reinforcing factors showed significant change. Beliefs regarding the risk of suicide associated with depression and psychosis increased or stabilised in the target region whilst it declined in the comparison region. "Thinking that nothing can help" decreased as a barrier to help seeking in the target region, whilst it either increased or stabilised in the comparison region, suggesting that young people became more positive toward the potential benefits of treatment. Self-identified depression significantly increased in the experimental region and is reported here as a predisposing factor as it is most likely to be an indication of increased awareness of depression rather than an increase in prevalence per se [9
]. This interpretation is supported by the finding that the relative proportion of these self-identifiers who had sought help did not change significantly over time.
In relation to the environmental targets of social supports and social norms, perceived negative evaluation by others regarding help seeking decreased in the experimental area, and perceived prevalence of mental health problems increased. A small but significant increase in the behavioural target of help seeking during the implementation of the campaign was observed in the target area.
The high response rate for the telephone surveys and the comparability of sample characteristics across time and regions strengthens the credibility of the results. The measurement tool itself has established validity [46
], although its validity in telephone interview format has not been established. Observed changes may be an underestimate of the impact of the campaign given that a small amount of media distribution error resulted in some of the print material "leaking" into the comparison region and it was not possible to put geographical restriction on website access.
The use of website and information line referral data was a useful process evaluation tool, but several limitations must be considered. The use of visits per month may be an underestimate of individual actual visitors to the website as firewalls on computer networks within services or institutions will only represent this as one visitor when it may be several people from the same network. Also, the collection of referral source data was at times open to ambiguity where a referral source could be categorised under more than one heading e.g. poster, GP (poster in waiting room) or school (poster on premises). Similarly, referral from "school" to the website could be through recommendation from a school counsellor for personal use or research for a school project on mental health. Furthermore, the limitations of the referral source data as a measure of campaign effectiveness must be considered in relation to major and more expensive media such as radio and cinema advertising. Whilst neither of these media rated well as a referral source, it is difficult to ascertain the degree to which they had a role in priming or reinforcing key messages of the campaign. Indeed, the post hoc analysis revealed a greater impact of the campaign in the Barwon region and this was also the only experimental sub-region where radio advertising was implemented and there was a higher media distribution rate per head of population.
This is one of only a small number of studies that have used a comparison region to evaluate a mental health literacy community awareness campaign regarding depression [9
], and the first study using this methodology in relation to psychosis. Data from the comparison region was critical in establishing the effectiveness of Compass as the strategy was operating concurrently with a range of national mental health community awareness initiatives, including beyondblue: the national depression initiative
], MindMatters school based initiatives [49
], and campaigns by SANE Australia [50
In regard to time taken to have a significant impact on mental health literacy, comparisons are difficult due to differences in study design and campaign intensity and spread. Hegerl and colleagues [47
] reported no significant impact after a 10 month regional campaign, Jorm and colleagues [9
] reported significant changes over a nine year period which included a four year national campaign, whilst Compass demonstrated significant changes after a 14 month regional campaign.
Whilst a number of psychosis studies have examined the impact of a community awareness campaign on help seeking, in particular duration of untreated symptoms [51
], they have not reported on the interim impact of these campaigns on mental health literacy. This may be of particular importance where a community awareness campaign does not demonstrate an impact on duration of untreated illness [52
] or help seeking, as the campaign may not have had enough time to take effect. Timely measures of mental health literacy may provide evidence that some change has occurred, heralding longer term impacts on help seeking, so that campaigns can be continued. However, it must be acknowledged that evidence of this study's effects on help seeking have not been fully evaluated.