|Home | About | Journals | Submit | Contact Us | Français|
Over the last decade, a consensus has emerged regarding a set of evidence-based practices for schizophrenia that address symptom management and psychosocial functioning. Yet, surveys suggest that the great majority of the population of individuals with schizophrenia do not receive evidence-based care. In this article, we review the empirical literature on implementation of evidence-based practices for schizophrenia patients. We first examine lessons learned from implementation studies in general medicine. We then summarize the implementation literature specific to schizophrenia, including medication practices, psychosocial interventions, information technology, and state- and federal-level interventions. We conclude with recommendations for future directions.
Over the past decade, the US Surgeon General's Report on Mental Health,1 the President's New Freedom Commission Report,2 Schizophrenia Patient Outcomes Research Team (PORT) project,3 the Texas Medications Algorithm Project (TMAP),4 and several other systematic efforts5–10 have identified a variety of evidence-based interventions for persons with schizophrenia. Examples of effective interventions include systematic approaches to medication management, assertive community treatment, relapse prevention programs, and supported employment.
Despite robust evidence on effective interventions, epidemiologic and clinical surveys have established that individuals with schizophrenia in the United States are unlikely to receive these effective treatments.11 Epidemiologic data from the National Comorbidity Study in the early 1990s showed that 60% of persons with serious mental illnesses received no treatment in the past year, 25% received clearly inadequate treatment, and only 15% received minimally adequate (far short of evidence-based) treatment.12 The Schizophrenia PORT study similarly found that patients in 2 large state public mental health programs were unlikely to receive most of the indicated evidence-based practices.13 More recently, the 2005 National Survey on Drug Use and Health14 found that only 8.5% of adults who reported both serious psychological distress and a substance use disorder received any treatment (again far short of evidence-based treatment) for both problems in the past year. Several recent studies indicate that quality of care may be worsening rather than improving.15–17 Thus, even as researchers continue to develop more effective interventions for the treatment of schizophrenia, the preponderance of individuals with this disorder, perhaps as many as 95%, receive either no care or less than optimal care. Previous articles in this special section have reviewed the epidemiology of services and the problems of engagement and retention. In this article, we review efforts to implement effective interventions for schizophrenia in routine mental health treatment settings and offer suggestions for narrowing the gap between science and practice.
Shojania and Grimshaw18 summarized the general medical literature on quality improvement efforts with 4 conclusions: First, the standard approach of passive diffusion of research (ie, publication of research findings in professional journals), including dissemination of findings on effective interventions, has little or no impact on routine practice. Second, more complex efforts to synthesize research evidence in the form of systematic reviews and disseminated guidelines also have little or no effect on practice. Third, adopting total quality management/continuous quality improvement techniques from industry has produced modest but disappointing results. Quality improvement, popularized by Deming,19 is a process that focuses on training, education, and using data to enhance the performance of an organization.20,21 In mental health treatment settings, quality improvement has taken the form of field-based supervision and systematic review of patient outcomes.22–24 Fourth, current attempts at complete systems reengineering using information technology have produced mixed results, including many prominent successes, which need to be understood in greater detail, as discussed below.
Health maintenance organizations have widely adopted disease management systems that emphasize activating patients for self-management, ensuring that clinical teams have the relevant expertise and access to evidence-based guidelines, and monitoring patients without face-to-face contacts.25 Wagner's26–28 chronic care model represents the most clearly articulated and widely studied approach to disease management. It explicitly aims to combine the best available implementation strategies. The Wagner model includes 6 components: (1) addressing “health care organization” at the level of leadership, financing, and removing barriers; (2) improving access to “community resources,” such as self-help, self-management classes, and nurse educator services; (3) enhancing “self-management supports,” such as helping people to set goals, establish action plans, identify barriers, and solve problems; (4) establishing “care monitoring,” often by nurses or pharmacists, to monitor response, self-management skills, and treatment by algorithms; (5) providing “decision supports,” which incorporate evidence-based guidelines into registries, flow charts, and assessments, to clinical teams; and (6) instituting “clinical information systems” that include assessments, treatment planning, services, outcomes, and algorithms. Many disease management programs include all 6 components, but others include only a portion of Wagner's model. The evidence shows that disease management approaches improve quality of care and outcomes across diverse chronic diseases such as diabetes, asthma, cardiovascular disease, and depression.29
Since the original Schizophrenia PORT study in the 1990s, several efforts have targeted implementing effective interventions, or evidence-based practices, for schizophrenia patients on a large scale rather than as small demonstrations. Because changing medication practice involves a different set of providers and mechanisms from changing psychosocial practice, we consider implementing medication guidelines first.
Antipsychotic treatments are a mainstay of treatment for schizophrenia, and most people with schizophrenia who are in treatment have access to antipsychotic medication. In the original Schizophrenia PORT study of adherence to recommendations in routine practice, the likelihood of a person hospitalized with schizophrenia receiving an antipsychotic medication was high (89%), but the quality of the antipsychotic prescribing frequently deviated from the evidence-based recommendations (only 62% of inpatients and 29% of outpatients were prescribed antipsychotic medication either within the recommended dosage range or with chart documentation justifying the dosage prescribed).13 Because prescribing medications is an essential component of treatment of schizophrenia and is relatively easy to implement (compared, eg, with mounting assertive community treatment teams or multifamily psychoeducational groups), administrators, payers, and researchers have focused considerable efforts on improving prescribers’ adherence to treatment guidelines.
The largest of these efforts, the TMAP, began in 1996 with the intent to develop, implement, and evaluate an algorithm-driven treatment approach for adults with major psychiatric disorders treated within the public mental health sector in Texas (http://www.dshs.state.tx.us/mhprograms/tmapover.shtm).30 TMAP incorporated several elements of Wagner's chronic care model, including patient education, systematic assessments, and clear guidelines. The project encompassed 4 phases: (1) creation of evidence-informed algorithms via consensus conferences that clearly indicated (via a flow diagram) the sequence in which the various medication alternatives should be considered (eg, “Consider clozapine before polypharmacy”), (2) a feasibility trial using the algorithms with early adopters to estimate their clinical impact and determine the resources needed to implement them, (3) a prospective comparison of the clinical outcomes and economic costs of using the algorithms vs treatment as usual in outpatient public mental health clinics, and (4) a broad implementation of the algorithms throughout the public mental health system in Texas, called the Texas Implementation of Medication Algorithms. The goals of TMAP were simple but far-reaching: to encourage measurement-based medicine to optimize patients’ outcomes. By combining sequential treatment algorithms with standardized measurements of symptoms and functioning at each visit, the prescriber could track progress or lack thereof over time to determine whether a medication change was indicated.
A wide variety of stakeholders followed TMAP closely. Patients, family members, payers, and clinicians were concerned about wide variations in prescribing practices and lauded TMAP for seeking to reduce unwarranted variation and ensure access to evidence-based decisions regarding medication management. Many other states, counties, and systems of care modeled algorithm-driven interventions after TMAP, even before results of the prospective comparison study were available.
In the prospective comparison study, after 3 months of treatment, patients with schizophrenia who received treatment in the sites that were trained and staffed to use the TMAP algorithms had greater improvement in symptoms (about 3 points on the 18-item Brief Psychiatric Rating Scale, after transformations of the scores to adjust for baseline differences and other factors known to affect health outcomes) than did patients in the comparison sites, but this difference in transformed scores disappeared over time.4 Patients in both algorithm sites and nonalgorithm sites showed improvement over time in test scores measuring cognitive functioning, with the patients in the algorithm sites showing greater improvement that was sustained as of the final (9 mo) measurement of cognitive functioning. Whether this difference in measures of cognition translated into differences in functioning in everyday life is not known. Because the impact of the score transformations used was so profound (eg, using raw scores reported in table 2 of the 2004 report, after 3 mo of treatment the symptom scores for the algorithm sites improved by about 5 points compared with improvements of 6 points at comparison sites that were implementing an algorithm for a disorder other than schizophrenia and improvements of about 6 points at other comparison sites), TMAP analyses may best be considered unfinished. Further exploration of this valuable dataset could provide much needed information on, eg, whether patients of prescribers who became more guideline adherent over time showed greater improvements in outcomes than did patients of prescribers who remained less adherent. Given the large impact in this study of score transformations on outcomes, sensitivity analyses showing the impact of various transformations could inform interpretation.
By design, the TMAP intervention included components to support and encourage the use of the algorithms in addition to the algorithms themselves. The TMAP investigators recognized that passive diffusion of algorithms was unlikely to change practice. The intervention package at a site included the provision of a clinical coordinator to work with physicians and families, treatment manuals, expert consultations through conference calls and site visits, family education programs, standardized assessments of symptoms and side effects, as well as the enhanced record keeping procedures that accompanied use of the algorithms.31,32 The understandable confounding of these intervention components means that one cannot know the extent to which any of the individual components were responsible for TMAP's findings.
What at first blush looks like an expensive intervention (training across sites, implementing measurement tools for prescribers to use to monitor changes over time, and placement of a clinician in each site to work with physicians to help them follow the guidelines) for a modest impact may be too pessimistic a conclusion. No doubt, adherence to the guidelines varied across prescribers, even within the algorithm sites. A stronger test of the payoff of changing prescriber behavior to more closely follow the treatment algorithm would be to determine whether patients of psychiatrists who followed the algorithm more closely had better outcomes than patients of psychiatrists who followed the algorithms less closely. This remains a challenging analysis for a multitude of reasons. Nevertheless, knowing whether and to what extent efforts to enhance adherence to guidelines improves patient outcomes is critical for policy makers who must decide if they should go to the effort of implementing and sustaining the use of such algorithms.
The zeitgeist surrounding TMAP prompted public and private payers to identify questionable patterns of prescribing and to intervene and monitor the impact of such interventions. The Psychiatric Clinical Knowledge Enhancement System in New York provides detailed current medication regimens and histories at the patient, psychiatrist, ward, and facility levels and has been used to decrease rates of antipsychotic polypharmacy.33,34 In another effort aimed at increasing prescriber adherence to evidence-based guidelines, a statewide longitudinal project in Kentucky examined the impact of prescriber training and systematic monitoring of medication practices, using Medication Management Approaches in Psychiatry (MedMAP) (mentalhealth.samhsa.gov/cmhs/communitysupport/toolkits/community/), which is an adaptation of the TMAP schizophrenia module. The Kentucky Project used the MedMAP Fidelity Scale35 as a quality improvement tool and documented modest improvement in prescribing practices over a 16-month period.36
The last decade has also seen the emergence of pharmacy management companies that contract with employers to review patterns of utilization, establish preferred drug formularies to steer patients toward/away from particular costly and/or more effective medications, and send letters to patients or prescribers alerting them to potential concerns about the medication regimens used. The extent to which steering is influenced by cost vs quality concerns typically cannot be determined because the full list of such managed care firms’ algorithms typically is proprietary, making it difficult for the payer or a neutral third party to evaluate the attention paid to cost vs quality of care by such algorithms. Firms may use polypharmacy as an example of a clinically questionable practice their guidelines target but offer few other specifics. While physicians can be offended by receiving “Dear Prescriber” letters from such pharmacy management firms and suspect that the goal is cost containment as opposed to quality improvement, one also can imagine situations in which strong evidence supports the use of timely claims data to flag clinically questionable situations for review (eg, no use of an antipsychotic medication following a hospitalization for schizophrenia). Transparency via independent review and broad opportunity for public comment helps quell understandable concerns that algorithms could be promoting cost savings at the expense of improving clinical outcomes. The business of promoting adherence to particular practices is rife with controversy as well as essential for improving care.37
Dissemination of model psychosocial programs for individuals with schizophrenia has a long history, dating back to the 1970s with the failed effort to promote the use of community lodges for patients discharged from psychiatric hospitals.38 In the late 1970s, the National Institute of Mental Health (NIMH) began funding the dissemination and evaluation of promising models based on Community Support Program principles to promote community integration of individuals with serious mental illness.39 Many of these projects produced disappointing results, in part due to poor model specification,40,41 inadequate implementation plans,42 lack of stakeholder support for the dissemination,43 and inadequate leadership.44 Many projects launched during this era had naive assumptions about the minimum requirements for effective implementation. The challenges of implementing complex psychosocial interventions are much greater than those for medication interventions.45
One positive outcome from the Community Support Program demonstration projects was increasing awareness of the need to define adherence to program models, which in turn required clearly defined models. Building on the work on treatment integrity in the psychotherapy literature,46,47 researchers began developing fidelity measures, defined as methods to assess adherence to the standards of a program model.48 Among the earliest of these efforts were measures to assess adherence to the assertive community treatment model.49–53 Assertive community treatment was the most clearly defined of the psychosocial models at the time.54 In the 1990s, supported employment began accumulating a strong evidence base, and a fidelity scale was developed and validated for this model as well.55,56 For supported employment, fidelity has been found to be associated with higher competitive employment rates.57–63 For assertive community treatment, the correlation between fidelity and improved outcomes has been less consistent.51,64–66
Over the last few years, there has been greater emphasis on the systematic study of the implementation process. In 2006, a new journal was established devoted to this enterprise (http://www.implementationscience.com). Accordingly, the literature on strategies and barriers to implementing evidence-based practices has expanded dramatically.45,67 With regard to schizophrenia, recent projects employing systematic implementation strategies include the National Implementing Evidence-Based Practices Project, the Mental Health Treatment Study, the Department of Veterans Affairs (VA) dissemination of supported employment, several learning communities, and a variety of implementation studies outside the United States.
In 1998, a national panel of experts convened by the Robert Wood Johnson Foundation recommended that 5 psychosocial practices and systematic medication management be offered in every community mental health center.8 The panel also recommended the development of a systematic approach to dissemination. The National Implementing Evidence-Based Practices Project was launched to address the aforementioned deficiencies.68–72 The investigators hypothesized that implementation of evidence-based practices in routine settings minimally required comprehensive, user-friendly information about the practices and their implementation—resources generally unavailable in earlier dissemination efforts. Therefore, the first phase of the National Evidence-Based Practices Project involved creation of toolkits for each practice, consisting of a variety of materials to facilitate practice implementation, such as practitioner workbooks, research articles, introductory and instructional video, and PowerPoint lectures. The toolkits aimed at multiple stakeholders, assuming that success depended on active support from the state mental health authority, agency leadership, practitioners, consumers, and family members.
The investigators also hypothesized that, in addition to the toolkits, another necessary component for successful implementation was systematic training and consultation. Thus, the project researchers developed a training-consultation model that included the following elements71: (1) consultation to the state mental health authority, (2) consultation to community mental health center administrators, (3) a kickoff presentation to the provider agency, (4) provision of the evidence-based practice toolkit to the agency, (5) initial skill training for practitioners, (6) ongoing consultation to sites, and (7) systematic fidelity monitoring. Fidelity monitoring included 2 steps: (a) fidelity assessments based on daylong site visits by a pair of independent fidelity assessors who, guided by fidelity scale criteria, conduct structured interviews, observe team meetings and interventions, and review medical records and (b) fidelity review meetings held with agency leaders in which assessors give oral and written feedback on quality of implementation.73
The second phase of the project was a field test of 5 psychosocial evidence-based practices in 53 sites in 8 states. The sites were studied over a 2-year period. Assessing clinical outcomes would have been prohibitively expensive; therefore, the study focused on fidelity scores at baseline and 6-month intervals. Findings from this project are now appearing in the literature.24,73–84
Overall, 29 (55%) sites showed high-fidelity implementation at 2-year follow-up.79 Most sites achieving high fidelity did so within the first year of implementation. Two of the evidence-based practices (assertive community treatment and supported employment) were more readily implemented than others. Based on qualitative data collected within each site, several factors influenced quality of implementation, including the state mental health authority's provision of leadership, funding, and practice standards76,80,85; skilled mentoring by trainer-consultants24; administrative support and competent clinical supervision at the site level24,74–77,81; systematic monitoring of fidelity and outcomes73,78; and staff turnover.83
In 2006, the Social Security Administration launched a 23-site randomized controlled trial comparing an experimental program to usual services for Social Security Disability Insurance (SSDI) beneficiaries with schizophrenia or affective disorder.86 Beneficiaries in the experimental condition receive a comprehensive package of services including supported employment, systematic medication management, and other behavioral services. A nurse care coordinator assigned in each site helps to ensure that services are individualized and evidence-based. The experimental group also receives supplemental insurance to cover charges for all necessary health care. Building on the experiences of the National Implementing Evidence-Based Practices Project, a quality management team is monitoring fidelity and providing technical assistance to the sites. Many of the same themes have emerged from this project as in earlier implementation studies. Barriers include the maze of state bureaucratic regulations even in the face of a well-funded project, the pivotal role of site leadership, the challenges of integrating supported employment with mental health treatment services, and staff turnover.
Like an earlier dissemination of the assertive community treatment model,42 current efforts by the Veterans Health Administration of the Department of VA to disseminate supported employment are ambitious in their national scope.87–89 Organizational barriers include lack of resources for supported employment, fragmentation of services (with individual clinicians working independently), and VA traditions and policies not aligned with evidence-based supported employment. In addition, longstanding sheltered and transitional work programs within the VA conflict with the principles of supported employment. This project depends on mostly remote supervision and monitoring, which may dilute the impact of technical assistance and quality improvement efforts.90
Borrowing from higher education, some groups of health-care organizations have adopted a learning community approach to quality, forming networks among key staff from these organizations that share common values and beliefs and are actively engaged in learning together from each other.91 Learning communities are especially suited to promoting sustained adherence to a practice as well as continuous quality improvement by creating a culture of peer accountability and sharing among participating organizations. One such application to schizophrenia has been a network of state and local leaders aimed at disseminating evidence-based supported employment.92 First established in 2002, the network has now grown to include participants from 10 states and the District of Columbia. Using familiar strategies such as annual meetings, newsletters, development of videos, on-site trainings, bimonthly teleconferences, and sharing data, this network has stimulated interest in identifying innovative implementation strategies successful in one state that are adapted for use in others.93,94 Program evaluation data suggest growing access to supported employment and achievement of site-level competitive employment rates exceeding 40% for quarterly reporting periods.92
A similar project in New York State, the Wellness Self-management Program (a variation of Illness Management and Recovery95), also uses learning collaboratives.96 Teams from participating sites meet to describe implementation experiences and hear suggestions from colleagues as well as a trainer-facilitator.
Wide-scale dissemination of evidence-based practices has been attempted in Canada,97 various European countries,98–100 Australia,101 and Japan.102 Common barriers found in these projects have included funding issues, lack of cooperation among different services (eg, vocational rehabilitation, mental health, and substance abuse treatment), staff turnover, and insufficient time allocated for program leaders.
Both the Institute of Medicine103 and the New Freedom Commission2 recommended using modern information technology to improve the quality of mental health care in America. The United States lags behind other Westernized countries in the use of health information technology,104 and the US mental health system lags behind other areas of American medicine.11 Several recent reviews conclude that appropriate use of information technology can improve quality of health care.105,106 A simple example of psychiatric knowledge enhancement systems is the use of programs that check for medication interactions and incorrect dosages. Other common components include increasing patient input regarding status, concerns, and goals; increasing patient education regarding evidence-based practices; increasing practitioners’ awareness of evidence-based treatments and algorithms and of patients’ concerns and preferences; identifying and avoiding medical errors; enhancing shared decision making; monitoring outcomes and side effects; and monitoring programs and systems of care.107 Comprehensive electronic decision support systems, which address all these components, are just emerging in mental health. The Veterans Administration has adopted a clinically oriented electronic medical record that facilitates evidence-based care via patient-specific clinical prompts, monitoring of care, and review of outcomes; the system has recently added a patient portal (http://www.myhealth.va.gov/). Enhancements to the Veterans Administration electronic medical record for mental health are being tested,108 and feasibility testing of direct consumer use of computer systems has shown promising results.109 In several European Union countries, a computerized system that enables patients with schizophrenia and their clinicians to compare their perspectives on goals and negotiate plans has demonstrated improvements in satisfaction, reduced unmet need, and increased quality of life.110
Numerous innovative efforts are underway.111 Deegan et al112 have designed a comprehensive electronic decision support system that is used to facilitate shared decisions during medication visits. Similar systems are being created and tested within the Department of VA to support schizophrenia care as well as care more broadly and at the Dartmouth Psychiatric Research Center to support treatment planning, smoking cessation, employment services, co-occurring disorders treatments, and care of comorbid medical illnesses. The New York State Office of Mental Health also is developing a consumer portal to support decision making around medications.
Comprehensive electronic decision support systems must be integrated into real-world contexts to be useful.105,106 Deegan et al112 established a recovery resource center within a routine community mental health center by providing semiprivate computer kiosks, visual and oral information, and peer supports. Patients and clinicians achieved high levels of participation and satisfaction. Of course, much of current community mental health work is done in the community rather than in clinics. Demonstrating that electronic resources can be delivered in other forms, such as through health buddies, in peer support centers, on the web, and on portable information systems that case managers and other clinicians carry with them, will be important.113
In the United States, the role of state government in promoting and inhibiting the growth of evidence-based practices is enormous if not decisive.114–117 The reasons are obvious: State agencies, in conjunction with federal agencies (notably the Centers for Medicare and Medicaid Services) to a large extent determine what services are funded or not funded. In most states, the state mental health authority has sponsored statewide initiatives to promote evidence-based practices. States have aggressively promoted adoption of assertive community treatment,76,80,118 supported employment,93 family psychoeducation,43 integrated dual disorders treatment,119 and medication management approaches.30,33,36 A popular but ineffective approach has been the sponsorship of statewide conferences.120,121 To address the enormous costs of face-to-face meetings, some states are exploring web-based options for training and ongoing supervision. A comprehensive approach to enhancing broad dissemination has often been centered in the formation of technical assistance centers providing consultation, training, and fidelity monitoring.24,118,122,123
Although it is widely assumed that the state mental health authority critically impacts the development of evidence-based practices, most research to date has been anecdotal. To test this hypothesis, a state-level fidelity scale was developed for the National Implementing Evidence-Based Practices Project to measure objective indicators of state mental health authority actions, such as the designation of a point person within the state agency responsible for dissemination, the establishment of a technical assistance center, state-level policies and regulations aligned to support the evidence-based practices, and provision of financial incentives to implement the evidence-based practices. This state-level fidelity scale was strongly correlated with mean fidelity for the evidence-based practices in each state.85
To date, most studies of statewide implementation have examined early stages of dissemination in which enthusiasm and other Hawthorne effects abound. Much less is known regarding sustaining a statewide initiative, especially in the face of the frequent leadership changes in senior staff in many state agencies due to postelection changes in political appointees. From the standpoint of the directors of community agencies, decisions about adopting evidence-based practices are influenced by judgments about the perceived riskiness of the uptake, anticipated resource availability, and exposure to evidence.124,125 Early adopters are willing to mount the initiatives necessary for adoption because they see the risks associated with adopting as lower and more manageable than nonadopters.124,125 Discontinuation is driven in part by staff turnover126; hence, e-learning approaches that are permanently and conveniently accessible can also help deal with this challenge.
At the federal level, the 12% set-aside of the NIMH budget for services research helped launch TMAP and other studies of the effectiveness of various treatment approaches. That set-aside is long gone, but, as this article is being written, the current version of the federal economic stimulus package contains $400 million to the Agency for Healthcare Research and Quality as well as other funds to stimulate comparative effectiveness research. As the largest single payer for health care in the nation, the Federal government can make significant investments in narrowing the quality chasm between what is known and what is practiced. In theory, NIMH generates new knowledge, and the Substance Abuse and Mental Health Services Administration helps such knowledge make its way to real people in real-world settings by promoting ways across this chasm. Knowing how to promote such uptake and where one's efforts are best invested are themselves pressing research issues. The Social Security Administration also has shown an interest in promoting evidence-based employment programs. It sponsored an expert panel to identify a package of evidence-based interventions to help SSDI beneficiaries with mental illness to return to work.127 The work of this panel led to the funding of the Mental Health Treatment Study described earlier. The Social Security Administration has also sought to incentivize employment services through its Ticket to Work program.128 The President's New Freedom Commission was an inspirational product of the executive branch of the federal government, even if the executive branch also required that the Commission's recommendations be revenue neutral. Revenue neutrality, especially in the short term, may be incompatible with implementing evidence-based practices unless a comparable amount of ineffective services can be discontinued.
Implementation research in schizophrenia care has followed the pattern observed in general health care18: moving gradually from passive diffusion to system reengineering based on complex electronic records, decision supports, and Wagner's26 chronic care model. Simple implementation efforts are often fruitless and waste resources, while traditional continuous quality improvement approaches are costly and often only moderately successful. Complex reengineering of systems is needed. However, public mental health systems are currently mired in financing constraints (eg, Medicaid regulations are not aligned with evidence-based practices), economic survival problems (state budget problems have eroded public mental health funding), workforce problems (average tenure in some front-line positions is less than 18 mo), regulatory problems (fears of Medicaid audits prevent systems from innovating), and information technology problems (currently many states have no electronic medical records while others have electronic medical records that emphasize billing and regulatory requirements rather than quality of services and clinical outcomes). Public mental health systems need better alignment between evidence-based practices and payments, sufficient funding to create a sustainable and professional workforce, electronic medical records to monitor process and outcomes, and a systemic commitment to quality. The research community needs much greater attention to the implementation of effective services in order to improve the care of schizophrenia in the United States.