|Home | About | Journals | Submit | Contact Us | Français|
Among the noncommunicable diseases, mental ill-health represents the major threat to social and economic progress because it impacts so powerfully on the most critical decades of life. Consequently, mental health reform is increasingly recognized as an urgent priority worldwide. This brings into sharp focus the role of evidence, and more specifically the Cochrane paradigm, in influencing decisions about health system reform. Cochrane clearly still has great value, especially in evidence-based medicine, where the focus is the evaluation of individual treatments. However, it cannot be allowed to be a dominant influence in evidence-based health care (EBHC) policy decisions for health system reform, unless it is modernized or complemented. Health services reform should definitely be as evidence-based as possible; however, the jury should consider its verdict on key reform proposals based on the balance of probabilities and informed by the best “available” evidence from all sources, not only randomized clinical trials, which in many domains may be never be feasible. This is particularly the case when reform is urgent, and the status quo has manifestly failed. So on the one hand, the evidence-based paradigm must not be misused to stifle or paralyze urgent reform. Alternatively, there is a real risk that, if we do not improve the sophistication of EBHC, the whole paradigm will be sidelined and reform will remain reactive, impulsive, and desultory. The recent Cochrane review on early intervention in psychosis provides an opportunity to consider these issues and their wider significance.
Mental health reform is one of the grand challenges of the 21st century.1 In every country, even in the wealthiest societies, people suffering from mental ill-health have considerably worse access to care than those with physical ill-health and that care is of much lower quality. This lop-sided and self-defeating allocation of the health dollar continues despite the fact that 75% of mental ill-health emerges before the age of 25 years, and its disabling and life shortening impact blights the prime productive years of life, causes untold misery, and weakens and diminishes society socially and economically. The recent report on noncommunicable diseases from the World Economic Forum graphically illustrates this in calculating that mental illness will, using 3 different forms of analysis, be the major contributor to the erosion of gross domestic product over the next 20 years.2 We urgently need much more research in prevention and better and safer treatments. However, the most immediate problem is the gap between what we know works and actually delivering this in the most cost-effective manner in a way that consumers find acceptable. This problem has a solution very much within reach. Even with existing knowledge, we could achieve a very substantial reduction in the burden of mental ill-health.3 There is absolutely no doubt that this should be carried out informed by the best available evidence. The publication of the recent Cochrane review on early intervention in psychosis (EIP) provides an opportunity to discuss how this should occur and the appropriate role of the Cochrane approach in the reform of mental health services.
Evidence-based medicine (EBM) originally referred to “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”4 However, it was soon widened to apply to population health and health care systems. “We call the new approach EBM when applied by individual clinicians to individual patients and evidence-based health care (EBHC) when applied by public health professionals to groups of patients and populations.”5 This is a crucial distinction. The Cochrane collaboration is the principal database for EBM, producing reviews of randomized clinical trials (RCTs). The RCT, developed to judge the efficacy of individual treatments, remains, despite its limitations, an essential element in clinical research strategy. Its inherent conservatism provides a reassuring antidote to the potential in psychiatry for pursuing “great and desperate cures”6 that prove to be useless or harmful, though excessive conservatism may also have negative effects. The RCT also has a place, though inevitably much less central, in health services research and EBHC. Crucially, however, the same design standards that were derived for RCTs of individual treatments cannot be simply translated to service level trials, though this is routinely done, including in the Cochrane EIP review. Furthermore, a key question we must answer is what is a valid yet realistic basis for making decisions in mental health reform, in a context of widespread gaps, poor access, and poor quality of care? And where should the onus of proof lie? Should substantially higher standards be applied to proposals for change than for continuing an evidence-poor status quo, when the latter is performing poorly or failing?
Over the past 15 years, early diagnosis and stage-specific treatment, a vital component of health care for potentially serious medical disease, have gained strong support in psychiatry. Ironically, this has been largely limited to what had originally been the seriously unpromising field of schizophrenia and psychosis. For 100 years pessimism had reigned supreme, but from the early 1990s, an international collaboration between researchers and clinicians began to challenge this self-fulfilling prophecy. From the beginning, this challenge has been evidence-based and led by clinical researchers, and it has produced a burgeoning and substantial literature to guide reform. The essence of the EIP paradigm is the contention that late intervention, especially for young people in a highly sensitive developmental period, is inherently damaging, at least psychosocially and possibly neurobiologically. Providing holistic and evidence-based intervention as soon as there is an undeniable need for care, and certainly as soon as frank psychotic symptoms have become sustained and disabling, is held to be not only a clinical and human imperative but also a strategy that should modify the impact of the illness, even if continuing care is needed in many or even most cases to maintain the initial benefits.
There have been 3 principle foci or stages around which research and reform have been conducted. The first is the stage from the onset of persistent symptoms and impairment, which indicates that a line has been crossed from the normal and transient stresses and strains of life and justifies a need for clinical care. In retrospect, after a psychotic or major mental disorder has supervened, this can be termed the prodromal stage. Looking prospectively, it has variously been termed the at-risk mental state, the subthreshold stage, or the ultrahigh risk (UHR) or clinical high-risk mental state, connoting the potential for resolution as well as progression. Since research has unequivocally shown that people in this stage are distressed, impaired, and have a need for care, as well as a markedly elevated risk (though not by any means inevitable) of sustained psychotic illness as well as other serious diagnostic and functional outcomes, there has been a (controversial) proposal to include operational criteria for this clinical phenotype in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition.7 The second stage is the period between the sustained onset of full-threshold positive psychotic symptoms and the provision of effective evidence-based care for first-episode psychosis, which has given rise to the measurable indicator of “duration of untreated psychosis” and corresponds to a second intervention target. Thirdly, the onset of a first episode of psychosis and the subsequent “critical period” of the next 2–5 years represents another opportunity to test disease-modifying strategies, both individual treatments and models of delivery. In summary, there has been widespread acceptance of the value of this approach, intense research activity for over 15 years, substantial innovation and upscaling of new service models in many countries, all supported by a cohesive international effort, an active scientific organization, the International Early Psychosis Association that has held 7 large international conferences, and, since 2007, a peer-reviewed journal, Early Intervention in Psychiatry.8 There have been inevitable controversies and debates, and this is expected and welcomed as a sign that genuine change is being attempted. What is particularly striking is that young people, families, and local organizations have been strongly supportive of these reforms wherever they have been progressed. Moreover, while scientific skepticism is always justified, much of the doubt and resistance has arisen within clinical professional groups and is openly based on other than scientific grounds.
The recent Cochrane review on EIP9 has prompted renewed debate regarding the effectiveness of the EIP model of care, particularly in the context of the increased implementation of this model in Australia and internationally. While acknowledging the need for further studies to expand the evidence base, the review, while encouraging, remained equivocal regarding the effectiveness of EIP. Unfortunately, this stance is largely a function of the RCTs selected for the review, which were fundamentally inadequate “on their own” to answer the review’s central question as to whether EIP is effective and warrants investment. The well-known constraints of the Cochrane methodology inevitably results in the exclusion of crucially relevant evidence—even clozapine has so far failed to receive strong endorsement by Cochrane. In the case of EIP, while 18 RCTs were included, 68 other studies, most of which were relevant to the key question regarding the value and impact of early intervention and should influence the conclusion that the review has sought to comment upon, were considered but excluded. Crucial exclusions were 2 positive health service level RCTs (which it should be noted are notoriously difficult and expensive to conduct, given their evaluation of models of service delivery, in contrast with simpler treatment/placebo trials) as well as a number of positive quasi-experimental10 and historical controlled11 studies. The landmark Early Treatment and Intervention in Psychosis (TIPS) study,12 which showed that reducing the duration of untreated psychosis was feasible and effective in improving long-term outcome (notably negative symptoms), used the best possible (ethical) methodology for that research question yet was excluded by the orthodoxy of Cochrane. Similarly, at least 5 cost-effectiveness studies, with results that clearly favor investment in EIP models, could not be included.
Crucially, the majority of research studies included in the review were not health services research designs: that is, with the exception of the OPUS trial, the review did not include studies comparing EIP with standard mental health care. Rather, the review focused largely on trials comparing the “component interventions” of EIP services, such as cognitive behavioral therapy, suicide prevention, family therapy, vocational intervention, and interventions to improve physical health outcomes, with “standard care” (yet chose to exclude studies of dose and generation of antipsychotic medications). Such interventions were typically studied against the backdrop of the range of care provided within streamed EIP services. It is not surprising that some of these individual trials were “ineffective” given that the control groups in these trials were the recipients of an already comprehensive model of care.13 The review blended RCTs of individual treatment components with health service level RCTs, problematic because the methodological standards cannot be the same, and only the service level studies are relevant to decisions on health service reform. Thus, the review muddied the waters on the salient question.
Despite positive 1- and 2-year outcomes in the OPUS trial,14–16 the Cochrane review focused upon the study’s nonsignificant 5-year outcomes and rather than interpret the partial erosion of benefit as a failure of standard care to maintain real benefits and disease modification, the review appeared to regard this erosion as a limitation of EIP. The recent paper from Norman et al17 illustrates that EIP services, if extended even in dilute form for 5 years, can maintain the initial hard-won gains. Two other positive health services RCTs comparing EIP and standard care were excluded.18,19
Finally, the authors perpetuate the myth that UHR patients are not in need of care of some kind too, despite the fact that they are actively seeking help and manifestly distressed and impaired with average Global Assessment of Functioning scores in the low 50s. Hence the ethical situation is no different from that in first-episode psychosis, with the exception that the content of interventions, in line with the predictions of the clinical staging model,20 should be different because of risk-benefit considerations and because it appears that antipsychotic medications are not needed or appropriate as first line treatment in this clinical stage. The Cochrane authors failed to carry out a meta-analysis of the UHR studies, which, when done, strongly favors intervention of some kind in this group rather than monitoring or neglect,21 at the very least on the basis of need for care but on the secondary basis of delayed transition. Both are valuable and achievable goals.
Cochrane is Cochrane, a fact that is understood by academic researchers. However, in the wider society and the media, these inevitably conservative findings that fail to draw on the complete body of evidence may be misconstrued. This could undermine public confidence in the value of urgently needed policy reforms and put them, and consequently large numbers of young patients, at continued risk of delayed and substandard care as offered by the status quo.22 Just as there are risks of impulsive policy decisions, there are also risks of extreme conservatism. Lack of sufficient scientific evidence can facilitate poor clinical practice; however, excessive conservatism contributes to the long delays in implementing evidence in clinical practice. While different views must always be heard and typically be respected, we now know that some scientific colleagues are willing to misuse the EBM paradigm and spread doubt where it should not exist.23 We must be vigilant to identify this phenomenon when it is occurring in psychiatry.
Finally, a recent Cochrane review on the effects of dietary salt intake24 produced negative conclusions, which have been challenged by many other experts and policy makers, as revealed in a compelling recent BBC radio debate.25 The lead author of the salt review, Professor Rod Taylor, subsequently qualified the negative review’s conclusions. His reasons? The review excluded evidence, and when the full body of evidence was considered a quite different, conclusion was the only reasonable one to embrace. In fact, Cochrane methodology had forced a prematurely negative conclusion. The Cochrane EIP reviewers may be thinking the same way as Professor Taylor because the conclusions of their commentary reveals as much support for the future of EIP as they could manage within the Cochrane constraints. Cochrane clearly still has substantial value, especially in the evaluation of individual treatments; however, it should not address itself to policy decisions on health system reform unless it finds a way to include other forms of relevant evidence. In applying an evidence-based approach in the here and now of policy-making, the jury on health services reform should consider its verdict based on the balance of probabilities and informed by the best available evidence from all sources and not require a unanimous, beyond all reasonable doubt verdict, based on an excessively narrow band of admissible evidence. This is the only feasible approach to EBHC and progress in mental health reform. The alternative is paralysis or impulsive and desultory “reform.”
P.M. currently receives research support from a National Health and Medical Research Council of Australia Program Grant (566529) and the Colonial Foundation. He has also received unrestricted research funding from Astra Zeneca, Eli Lilly, Janssen-Cilag, Pfizer, and Novartis, as well as honoraria for educational and consultancy roles with Astra Zeneca, Eli Lilly, Janssen-Cilag, Pfizer, and BMS.
The authors have declared that there are no conflicts of interest in relation to the subject of this study.