The Schizophrenia Outcomes Workgroup used literature review, expert interviews, and a national expert panel process to increase consensus regarding outcomes monitoring strategies that can be used to support quality improvement and the delivery of evidence-based treatments for schizophrenia. It is clear that outcome measurement is better developed in some domains than others. Good consensus was reached for the domains of psychotic symptoms, medication side-effects, drug and alcohol use, depression, family and caregivers, vocational functioning, community tenure, and housing. In these domains, there are well developed, validated, self-report outcomes assessment instruments, and often multiple excellent choices for routine outcomes monitoring. However, there are areas that would benefit from further research on measurement, including, for example, identifying eligible patients for clozapine or supported employment. It was not possible to develop consensus in other domains, such as negative symptoms, cognitive symptoms, and use of peer support interventions, because efficacy research is not sufficiently developed. National treatment guidelines offer limited guidance regarding treatments that should be provided to patients and, therefore, outcomes to be monitored.
Although there are high quality outcomes assessment instruments, some assessment strategies are much easier than others to implement in routine treatment settings. The most feasible strategies for obtaining outcomes data are the use of administrative or billing data, and self-administered assessments. Administrative data are typically not designed for this purpose, and often contain inaccurate or incomplete information on outcomes and limited clinical content. This is somewhat better in systems that have implemented Electronic Medical Records (EMRs). However, even with EMRs, outcomes data are inconsistently documented or encoded in fields that are readily usable for quality improvement (Young 2007
). To standardize medical records data, healthcare organizations sometimes rely on clinicians to document outcomes, an approach that has not worked well. Although many clinicians could be trained to high levels of reliability in standardized instruments, they typically do not have the time or interest necessary for training, routine performance, and entry of outcomes data into medical records.
Self-administered patient assessments are another strategy for outcome monitoring. They requires less time from busy clinicians, do not require repeated trainings, and alleviate concerns about clinician gaming of assessments. Methods of self-assessment vary and can be determined by the needs of specific projects and healthcare organizations. Methods of self-assessment have included traditional paper and pencil forms, scannable forms, electronic self-entry devices, and internet access by patients. Electronic devices at clinics can include dedicated kiosks or tablet computers with touch-screen and audio interfaces. Self-entry systems can generate reliable and valid data, are well accepted by clinic populations with schizophrenia, and can be feasible to implement in clinic settings (Chinman et al. 2004
For quality improvement, it is necessary to know which patients would benefit from specific treatments, and which of these patients have not been receiving those treatments. The Workgroup did not review methods for monitoring the utilization or appropriateness of services that patients receive. However, there was agreement that, in addition to monitoring outcomes, it will also be important to monitor the provision of effective care. For psychosocial treatments in particular, it is important to monitor the extent to which treatments demonstrate fidelity to practices that are known to improve outcomes.
This project had a number of limitations. First, the outcome strategies and instruments recommended are not intended to replace clinical decision-making in individual patients. Rather, they were developed to support organizational efforts to improve care in populations of patients. Also, the instruments reviewed are not an exhaustive list. There are likely to be other instruments with good psychometric properties that are suitable for monitoring outcomes in these domains. In any specific instance of quality-oriented outcomes monitoring, selection of specific strategies and instruments must be a function of the goals of the project and the characteristics of the healthcare system.
Healthcare organizations have succeeded in improving care by making use of quality improvement that includes routine collection of outcome data. The routine and systematic collection of data allows for identification of unmet needs, and redesign of services to improve access to effective practices and improve outcomes. There have been fewer successes with this approach in schizophrenia than in other chronic medical disorders, and there are specific challenges to collection and encoding of outcomes data in this population. However, the Workgroup believes that there is substantial potential to improve access to effective treatments in populations with schizophrenia. This begins with systematic monitoring of outcomes. Providing outcomes data to clinicians would not by itself be expected to improve the quality of care (Fihn et al. 2004
; Stone et al. 2002
). Rather, this data needs to be used as part of systematic projects to reorganize service delivery. Systematic research will bee needed to evaluate these efforts to improve care.
There was strong consensus regarding viable strategies for routine outcomes monitoring. There are some domains, such as psychotic symptoms, medication side-effects, and medication adherence, where there are existing instruments and data sources that are ready now for widespread implementation and use. There are other domains, such as clozapine, employment, and community reintegration where useful instruments are available, but measurement could be substantially strengthened with additional research. It was not possible, or desirable, to definitively recommend one specific instrument and measurement strategy for all quality improvement efforts. Each measurement strategy and instrument has pros and cons that must be weighed. And, to be successful, quality improvement needs to attend both to evidence and also local context when selecting measurement strategies. Therefore, when there were multiple acceptable instruments, the Workgroup did not select one choice, but left this choice to the quality improvement project that will use the data. With regard to alternative measurement strategies, patient self-report appears to be the most broadly feasible method in many domains. Self-report should be supplemented, where helpful, by clinician documentation and administrative data. Whatever path an organization selects with regard to domains, instruments, and data collection strategies, there are important opportunities to use outcomes data as part of projects to improve care for schizophrenia.