Search tips
Search criteria 


Logo of springeropenLink to Publisher's site
Administration and Policy in Mental Health
Adm Policy Ment Health. 2010 May; 37(3): 205–207.
Published online 2010 March 31. doi:  10.1007/s10488-009-0246-7
PMCID: PMC2877333

Assessment of Evidence-Based Psychotherapy Practices in Usual Care: Challenges, Promising Approaches, and Future Directions

In this special issue of Administration and Policy in Mental Health and Mental Health Services Research, we focus on measuring evidence-based practices in psychotherapy within usual care practice. Measuring evidence-based practices in usual care settings is important for at least three reasons. First, such measurement would help identify practices within usual care settings that are promising. For example, patients in usual care psychotherapy settings tend to have co-morbid mental disorders; whereas, our evidence base for psychotherapy is largely based on single disorders. Mixing strategies from more than one evidence-base may be effective in practice. Second, measuring evidence-based practices is essential for quality improvement interventions. Such measures could provide a baseline, as well as a comparative measure of change following quality improvement interventions that attempt to introduce more evidence-based care to patients. Third, understanding what clinical practices in usual care works best individually or in combination can help tailor graduate training programs to improve the pool of new professionals currently in training. Overall, developing measures of and understanding psychotherapy as it happens in the community is important for improving our mental health service delivery system.

National policy forums focused on improving the mental health delivery system are all noting the need for improved measurement of quality care. The Institute of Medicine Report, Improving The Quality of Health Care for Mental and Substance-Use Conditions (2006), notes, “Measuring the quality of care provided by individuals, organizations, and health plans and reporting back the results is linked both conceptually and empirically to reductions in variations in care and increases in delivery of effective care.” Recommendation 4–3 of this Report focuses on developing better measures of quality. Similarly, The President’s Advisory Commission on consumer Protection and Quality in the Health Care Industry (1998) identified mental health care as an area where quality measures, such as measures of evidence-based practices, are not well addressed.

Managed behavioral health organizations have the clinical and fiduciary responsibility over the quality of care provided to their members; therefore it is in their own interest to have tools that measure quality of care for psychotherapy. Most mental health practitioners belong to multiple managed care organization provider panels where current measures of quality of care for behavioral health care are based on population-level metrics. The Health Plan Employer Data and Information Set (HEDIS) developed by National Committee on Quality Assurance (NCQA) is designed to measure access to care, enrollee satisfaction, use of services and the effectiveness of care. Within behavioral health the metrics for effectiveness, for example, include follow-up after hospitalization rates (FUH), anti-depressant medication management (AMM), and follow-up for children prescribed attention-deficit/hyperactivity medications (ADD). However, these metrics are poor measures of actual quality of care, and primarily apply to health plans rather than managed behavioral health organization. Most individuals receiving outpatient care through managed behavioral health organization provider panels are receiving psychotherapy, yet HEDIS metrics are based on medication management and inpatient care. There are no measures currently available that can provide individual-level metrics of the use of evidence–based psychotherapy to measure overall quality of care. As a proxy, managed behavioral health organizations may measure the quality of care provided by their provider panels by assessing the use of clinical-practice guidelines (Azocar et al. 2001, 2003) or by monitoring outcomes and in some circumstances, providing feedback reports (Lambert et al. 2005; Brodey et al. 2005; Brown et al. 2005); neither of which look at the actual use of evidence-based practices and how that relates to quality of care. The use of outcomes-informed care is growing but still a rarity among behavioral health providers and difficult to implement within managed behavioral health organizations because of its association to pay-for-performance issues. The development of measures of evidence-based practices in outpatient psychotherapy will help engage the provider and the consumer at the individual level as a stakeholder in improving quality of care.

The need for measuring evidence-based psychotherapy practices have also been called for by those individuals developing policies to improve our mental health workforce. The Annapolis Coalition on Workforce Development, in their report “An Action Plan for Behavioral Health Workforce Development (DHHS 2007)” discuss the recurrent finding of lack of reliable and valid data on the status of workforce development strategies, including evaluation of the sustained adoption of newly learned skills in real-world service settings. In this special issue, we begin to answer these important policy recommendations by presenting new evidence in this seminal study of evidence-based practices in clinical settings.

In this special issue, four papers help develop methods for moving ahead in this important field of measuring evidence-based practices in clinical settings. Garland et al. (2010) offer an important overview of the methodological issues to be considered when measuring usual care as practiced in clinical settings. Hepner et al. (2010a) provide data on a short, efficient measure of evidene-based practices for depression care completed by providers of adult services and Kelley et al. (2010) provide data on a similar measure for youth services. Hurlburt et al. (2010) then present compelling data regarding therapists self-rating of psychotherapy techniques and goals as compared with ratings by observers. Finally, Miranda et al. (2010) present initial data on a self-report measure of psychotherapy techniques completed by patients following usual care therapy in managed health care settings.

Following these initial papers addressing important methods issues in measuring psychotherapy practice, four papers present state-of-the-art data on psychotherapy in clinical settings. Brookman-Frazee et al. (2010) discuss characteristics that predict the likelihood of evidence-based practices occurring in child therapy sessions. Hepner et al. (2010b) present data on evidence-based practices in psychotherapy among practitioners in a large, cross-national managed behavioral healthcare organization. Finally, Landry et al. (2010) present data on evidence-based practices in a nationally representative sample of patients receiving mental health care services.

Together, these articles define the state of the art for measuring psychotherapy practices in clinical settings. This series provides a clear overview of this seminal work and helps shape new efforts to carefully understand the care we provide to our nations vulnerable populations with mental health care needs.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.


  • Azocar F, Cuffel B, Goldman W, McCarter L. The impact of evidence-based guideline dissemination for the assessment and treatment of major depression in a managed behavioral health care organization. Journal of Behavioral Health Services & Research. 2003;30:09–18. [PubMed]
  • Azocar F, Cuffel BD, Goldman W, McCulloch J. Best practices: Dissemination of guidelines for the treatment of major depression in a managed behavioral health care network. Psychiatic Services. 2001;52:1014–1016. doi: 10.1176/ [PubMed] [Cross Ref]
  • Brodey BB, Cuffel BJ, McCulloch J, Tani S, Maruish M, Brodey I, et al. Acceptability and effectiveness of patient-reported assessments and feedback in a managed behavioral healthcare setting. American Journal of Managed Care. 2005;11:774–780. [PubMed]
  • Brookman-Frazee, L., Haine, R. A., Baker-Ericzen, M., Zoffness, R., & Garland, A. F. (2010). Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0244-9. [PMC free article] [PubMed]
  • Brown GS, Jones E, Lambert MJ, Minami T. Evaluating the effectiveness of psychotherapist in a managed care environment. American Journal of Managed Care. 2005;2(8):513–520. [PubMed]
  • DHHS. Annapolis Coalition (2007). An action plan for behavioral health workforce development: A framework for discussion by the Annapolis Coalition on the Behavioral Health Workforce. (SAMHSA/DHHS Publication No. 280-02-0302). Rockville, MD: Department of Health and Human Services.
  • Garland, A. F., Hurlburt, M., Brookman-Frazee, L., Taylor, R. M., & Accurso, E. C. (2010). Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0237-8. [PMC free article] [PubMed]
  • Hepner, K. A., Azocar, F., Greenwood, G. L., Miranda, J., & Burnam, M. A. (2010a). Development of a clinician report measure to assess psychotherapy for depression in usual care settings. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0249-4. [PMC free article] [PubMed]
  • Hepner, K. A., Greenwood, G. L., Azocar, F., Miranda, J., & Burnam, M. A. (2010b). Usual care psychotherapy for depression in a large managed behavioral health organization. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0247-6. [PubMed]
  • Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0251-x. [PMC free article] [PubMed]
  • Quality Chasm Series. Committee on crossing the quality chasm: Adaptation to mental health and addictive disorders. Washington, DC: The Institute of Medicine, The National Academies Press; 2006.
  • Kelley, S. D., de Andrade, A. R. V., Sheffer, E., & Bickman, L. (2010). Exploring the black box: Measuring youth treatment process and progress in usual care. Administration and Policy in Mental Health and Mental Health Services Research, 37(3). [PubMed]
  • Lambert MJ, Harmon C, Slade K, Whipple JL, Hawkins EJ. Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology. 2005;61:165–174. doi: 10.1002/jclp.20113. [PubMed] [Cross Ref]
  • Landry, C., Klap, R., Tang, L., Liao, D., Miranda, J., & Wells, K. B. (2010). The content of substance abuse and mental health counseling reported by patients in a national survey. Administration and Policy in Mental Health and Mental Health Services Research, 37(3). doi:10.1007/s10488-009-0250-y. [PubMed]
  • Miranda, J., Hepner, K. A., Azocar, F., Greenwood G., Ngo V., & Burnam, M. A. (2010). Development of a patient-report measure of psychotherapy for depression. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-009-0238-7. [PMC free article] [PubMed]
  • The President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. (1998). Quality First: Better Health Care for All Americans.

Articles from Springer Open Choice are provided here courtesy of Springer