|Home | About | Journals | Submit | Contact Us | Français|
This study evaluates a large demonstration project of collaborative care in community health centers by examining the role of clinic site on measures of the implementation process and on clinical outcomes that are not accounted for by characteristics of the patients served.
This quasi-experimental study examines data on the treatment of 2821 patients over three years at six organizations that implemented collaborative depression care. Outcome data included two quality indicators (receipt of early follow-up or appropriate pharmacotherapy) and depression improvement (50% reduction in PHQ-9 score or PHQ-9 score ≤ 5).
Multivariate logistic regression models revealed significant differences across clinics in the probability of receiving early follow-up (.34 to .88) or appropriate pharmacotherapy (.27 to .69); or experiencing improvement (.36 to .84) after adjustment for patient characteristics. Similarly, Cox proportional hazards models revealed that time to improvement differed significantly across clinics (p ≤ 0.0001) after adjusting for patient characteristics.
Across all sites, a plurality of patients achieved meaningful improvement in depression and in many sites improvement occurred rapidly. Despite receiving similar training and resources, organizations exhibited substantial variability in their ability to enact change in clinical care systems, as evidenced by both quality indicators and outcomes. Although we cannot conclude that performance on quality indicators caused improved outcomes, those sites that performed better had better outcomes, differences that were not attributable to patient characteristics.
A robust literature has emerged supporting the value of quality improvement initiatives, such as collaborative care models, that provide integrated care within primary care settings for common mental disorders such as depressive and anxiety disorders.(1–6) The IMPACT trial, one exemplar of collaborative care, demonstrated significant improvement in depression outcomes among elderly patients with cost savings in the long-term.(3) These promising results have engendered considerable national interest in the dissemination of integrated models, and the corresponding need to understand how to implement such models in real-world settings.(7, 8) Presently, little is known about the care provided or the outcomes achieved when such models are adopted by community-based practices, particularly those in resource-poor settings.
To address this gap in knowledge and gain understanding of the process of implementing a collaborative care model into community health centers, the Hogg Foundation for Mental Health sponsored a demonstration project. Separately, we have reported that patients who were older, more depressed, and more anxious were more likely to receive high-quality care, whereas patients with Spanish language preference and lower anxiety had the best outcomes.(9) Here, we analyze the role of site on specific process measures of depression treatment (early follow-up and appropriate pharmacotherapy) and on clinical outcomes (depression improvement and remission).
We report a quantitative evaluation of quality indicators and patient outcomes across the participating organizations, as described below. Our analysis is also informed by qualitative observations derived from the extensive contact of the investigators with the organizations, including loosely structured site visits. Although this latter component does not qualify as a rigorous mixed-methods approach, these observations inform the interpretation of our quantitative findings in our Discussion. The institutional review board at Harvard Medical School approved this evaluation.
Six community health organizations [described in detail in an online supplement at ps.psychiatryonline.org] in Texas treating predominantly uninsured patients were selected through a competitive review process in response to a request for proposals issued by the Hogg Foundation for Mental Health. Key selection criteria included the organizations’ demonstrated grasp of the collaborative care model, their preparedness to implement the model, and their ability to demonstrate their organization’s investment and commitment to participation and sustainability after the grant period. Because one organization had two participating clinics, data from seven clinic sites were analyzed in total.
Patients were identified by their primary care teams as having symptoms of depression. A total of 2821 patients age 18 and above were treated between 2006 and 2009.
Organizations adopted a collaborative care model which included the following key elements: the use of a web-based disease registry to track patients; care management to support management by primary care providers and provide proactive follow-up; and organized psychiatric consultation. The care manager conducted initial assessments of patients using standardized instruments (see Outcome Measures below), provided patient education, tracked patients’ treatment response longitudinally in clinic and via telephone with standardized instruments, coordinated medication management with primary care providers, and assisted with referrals to psychotherapy or other specialty care where indicated. Most sites had at least one care manager who provided services in Spanish and most sites employed licensed care managers or separate licensed therapists who provided brief cognitive behavioral therapy. The consulting psychiatrist provided weekly supervision to care managers either in-person or via telephone, reviewed treatment plans, performed consultations on select patients who were not improving, and was available by telephone for primary care providers’ questions. The disease registry included prompts to assist care managers in identifying patients needing follow-up and tools to facilitate identification of patients not responding to treatment.(10) Primary care providers received trainings on evidence-based depression management (described below) and were responsible for prescribing psychotropic medications, maintaining primary responsibility for patients’ care.
A technical assistance team from the University of Washington provided training for care managers, consulting psychiatrists, and primary care providers, access to the registry, and logistical support for implementing the collaborative care program through annual in-person trainings and monthly conference calls. During a kick-off meeting, the University of Washington team provided the sites with an orientation to the collaborative care model. Following the kick-off meeting, the sites began individual monthly calls with a University of Washington coach. The calls covered clinical and operational/program issues (e.g., engaging hard-to-reach patients). Annual in-person trainings included full-group training sessions on program issues such as triaging patients, as well as separate sessions by site or by team role. Care manager training sessions covered topics like managing caseloads and communicating with primary care providers. Psychiatrist and primary care provider training sessions covered topics like evidence-based depression treatment and managing treatment-resistant depression. All sites had access to the same training and technical assistance resources, except that Organization 6 started the grant program approximately a year later, thus missing the kick-off meeting and the first year of coaching calls.
All data was obtained from the disease registry which included data on patient demographics, type and timing of follow-up contacts, and when applicable, the type and dose of psychotropic medication prescribed. All data was collected in the routine course of delivering care. Sites were not asked to collect additional data for the evaluation.
To assess an organization’s performance, care process measures can serve as indicators of healthcare quality.(11) We identified two indicators: early follow-up [a follow-up with the care manager within 3 weeks of treatment initiation] and appropriate pharmacotherapy [an antidepressant medication at a therapeutic dose based on published guidelines (12–14) in the treatment plan at the visit preceding outcome assessment].
At every care manager contact, patients completed the Patient Health Questionnaire – 9 (PHQ-9), a widely used measure of depression severity and treatment response.(15–19) Based on baseline PHQ-9 scores, depression severity was categorized as mild (0–9), moderate (10–14), or severe (15–27).(15) Patients who achieved a 50% reduction in PHQ-9 score at follow-up or scored 5 or lower were considered to have improved, whereas only patients in the latter category were classified as remitted. Acute phase outcomes were obtained from follow-ups that occurred at least 6 weeks but no more than 12 weeks following the initial assessment because the benefits of both antidepressant medications and psychotherapy necessitate at least 4–8 weeks of treatment. (12–14)
Demographic information (age, gender, preferred language) was obtained from the registry. The Overall Anxiety Severity and Impairment Scale (OASIS), a 5-item self-report measure of anxiety symptoms that has been validated in primary care patients, was used to assess anxiety symptoms. (6, 20)
To examine the impact of site on the probability of receiving quality indicators and of depression improvement, logistic regression models were specified. To estimate the effect of site on time to improvement, Cox proportional hazards models were constructed. Because the patients served in each clinic differed in their demographic and clinical characteristics, factors that are associated with depression care and outcomes, (21–24) all analyses accounted for potential confounding by adjusting for patient demographic and clinical characteristics (age, gender, preferred language, baseline depression and anxiety).
Patients were predominantly female, ranged widely in age, spoke English or Spanish, and had moderate-to-severe depression and anxiety symptoms (Table 1). Of 2821 patients enrolled, 271 met criteria for remission (PHQ-9 ≤ 5) at baseline and were excluded from analyses of outcomes. Due to missing values for some predictor variables, the valid sample for the regression models was 2010. Table 1 presents data for each clinic on the caseload, patient characteristics, and details of the treatment and outcomes.
Both quality indicators were strongly associated with site (Table 1), differences that were not attributable to characteristics of the patients served (Table 2). Based on multivariate logistic regression models, the adjusted probability that a patient received an early follow-up contact ranged from 0.34 to 0.88, whereas the adjusted probability for receipt of appropriate pharmacotherapy ranged from 0.27 to 0.69.
Depression outcomes at 12 weeks, 4 months, and 6 months varied substantially across clinics in unadjusted analyses (Table 1). The multinomial logistic regression model revealed that after adjustments for patient characteristics, the probability of discontinuing treatment differed significantly across clinics (0.29 to 0.75), as did the probability of improvement (.36 to .84) among patients with a valid PHQ-9 score (Table 2). Sensitivity analyses examined whether the quality indicators could account for the observed differences in outcomes across site. The inclusion of the quality indicators in the multinomial model did not attenuate the effects of site on outcome (data not shown). Patients with early follow-up were less likely to drop out (OR=0.50, p<0.001) and more likely to improve (OR=1.64, p<0.01), whereas patients who received appropriate pharmacotherapy were less likely to drop out (OR=0.73, p<0.05).
Survival analysis was utilized to estimate the time to improvement across sites while taking into account censoring of individuals who drop out of care. The unadjusted survival curve, Figure 1, illustrates that the proportion of patients who experienced improvement across clinics diverged as early as 4 weeks and that these differences persisted over time. Due to censoring, these results cannot be compared directly to rates of improvement reported in Tables 1 and and2,2, although the findings are similar. Site differences in time to improvement remained significant following adjustment for patient characteristics (age, gender, preferred language, baseline depression and anxiety) in a Cox proportional hazards model (not shown; available from the authors).
Overall, this demonstration project reveals that organizations with markedly differing characteristics were able to integrate mental health care into primary care for disadvantaged and underserved patients. In all sites, a plurality of patients experienced meaningful improvement in depression, and in some sites, the majority experienced rapid improvement following initiation of treatment. Despite these broad successes, the organizations differed dramatically in indicators of quality and in outcomes, differences that were not attributable to measured characteristics of the patients served. Moreover, the variability in quality measures and outcomes indicates that substantial room for improvement in quality of care remained.
Retention in depression treatment is poor for primary care patients, particularly low-income minorities.(25, 26) Outcome measures were collected exclusively in the course of clinical treatment and therefore, were obtained at different times for different patients. Consequently, many patients who did not have a PHQ-9 score between 6 and 12 weeks were counted as drop-outs for acute phase outcomes, but fewer patients (18%) were truly lost to follow-up. This rate closely approximates retention in collaborative care research trials.(1, 3, 4, 27) Moreover, our survival models accounted for attrition in constructing estimates of outcomes over time.
In research settings, around 60% of patients treated in a collaborative model experience improvement and 25% achieve remission during the acute phase compared to around 40% of patients who improve and 10% who remit in usual care. (3, 4, 27, 28) Depression outcomes are worse for complex patients, including those who are elderly, socioeconomically disadvantaged, anxious, or treatment-resistant.(3, 4, 6, 27–32) Although one might expect that trial results could not be reproduced by organizations in economically-deprived, minority communities serving heterogeneous patients, our results refute this notion. Several participating organizations are federally qualified health centers in poor urban border towns with high concentrations of Spanish-speakers and are safety net providers for uninsured individuals, including undocumented immigrants who are ineligible for Medicaid. At six of seven sites, half of patients improved, and at five sites more than 40% of patients achieved remission, a more challenging clinical target. Thus, these results provide robust support in demonstrating that collaborative care, when adapted by community health centers, can meet or surpass the outcomes achieved in research settings even in poorly resourced settings. This finding is highly relevant in light of strong interest in integrated healthcare in the context of national health reform.
Although all sites received the same training, guidance, and support, and all used the same web-based registry, the care provided and the outcomes achieved differed markedly. Outcomes diverged across sites as early as 4 weeks, highlighting the critical importance of the early course of treatment. For example, loss to follow-up occurred for only 4% of patients at organization 2 compared to over one-third of patients at organization 6. Organization 2 was successful in all metrics of care: 87% of patients received early follow-up; 66% had 4 or more follow-ups within 12 weeks; 68% received appropriate pharmacotherapy; and patients received an average of 4.8 contacts during the first 12 weeks. Patients treated at this clinic had excellent acute-phase outcomes, with two-thirds improving and half achieving remission. In contrast, the site with the worst acute-phase outcomes (organization 1, clinic B) performed relatively poorly on process measures, with 19% of patients lost to follow-up, one-third of patients receiving early follow-up, 5% of patients receiving 4 or more follow-ups within 12 weeks, and approximately half of patients receiving appropriate pharmacotherapy. Having conducted site visits, we have the impression that some sites implemented more effectively than others. Those sites that appeared to implement well demonstrated better performance on both quality indicators and patient outcomes. These differences were not attributable to patient characteristics. Although early follow-up was significantly associated with improved outcomes, we are not able to conclude that more patients improved because they received more intensive follow-up. It is possible that the indicators we measured tap into broader constructs of the quality of services provided across organizations and are not responsible for improved outcomes, but nevertheless associated with such outcomes.
In multivariate analyses adjusting for quality indicators, site remained strongly associated with outcomes, suggesting that the effect of being treated in a particular clinic is at least as salient as receiving (or not receiving) early follow-up or appropriate pharmacotherapy. On the surface this may appear surprising; however, this is consistent with our impression that the quality measures we quantified were indicators of broader processes occurring within the clinics. We speculate that similar patterns may have emerged if we had quantified other quality indicators. It is possible that clinics with better implementation were more likely to provide treatment intensification for patients not responding to initial treatments, effects that were not captured by our analyses. Similarly, it is possible that clinics that achieved superior outcomes were more effective in tailoring treatment to individuals’ needs, for example, by accounting for patient preferences, availability of services, and adaptation of treatment to individual outcomes.
Because a small number of organizations participated in this demonstration project, we were not able to quantitatively evaluate how characteristics of the organizations were associated with outcomes. However, with great interest in replicating collaborative care models in community settings, we speculate on some potential contributors. Several organizations located in impoverished communities with no pre-existing mental health services, little or no experience with integrated care, and few resources were largely successful in implementing collaborative care. In contrast, the organization that had the least success in program implementation was a large organization with a significant pre-existing structure of mental health providers both within primary care and at specialty sites. This existing system of separating mental health and primary care appeared to impede implementation. Despite training and additional resources, little practice change was apparent and care remained fragmented between a wide array of providers such that many patients “fell through the cracks”. Thus, building an integrated delivery model de novo may be more straightforward than re-engineering a system with existing services. Organization 4 provides an example of a success story. Although the leadership at this organization was unsupportive and at times antagonistic to the initiative, the organization had strong support from capable primary care providers and care managers who embraced the model. In this case, it appears that the clinicians were successful in enacting substantial practice change despite a lack of strong support from the administration. They engaged patients early on and provided appropriate pharmacotherapy to a majority. These examples suggest that attention to the pre-existing system of care may be an important element for organizations considering implementing an integrated model and that buy-in from on-the-ground clinical providers may be particularly crucial.
In considering the implications of these findings, several limitations are important. First, unmeasured differences in the patient populations treated across sites may account for some of the observed variability. Information was not available on individual race/ethnicity, education or income, factors that may be associated with depression outcomes. The organizations predominantly treated uninsured patients in impoverished communities, so these results may not represent organizations treating patients under more advantaged circumstances. Without data on treatment patterns and outcomes at these organizations prior to the initiative, we are unable to indicate to what extent the observed treatment and outcomes are the result of the introduction of this model. The patients treated represented a heterogeneous mixture of clinical presentations and data on diagnoses was not available for further description. Analyses were also limited by a lack of detail regarding patients’ use of psychotherapy and psychiatric consultation. Finally, medication data was obtained from the registry and may not reflect patients’ actual medication use.
Several lessons from this demonstration project are instructive as health centers throughout the country grapple with the process of providing integrated health services. Successful implementation of integrated care occurred in a variety of settings and achieved admirable outcomes among a heterogeneous population of disadvantaged primary care patients. Moreover, organizations without pre-existing mental health personnel were particularly successful in creating well-functioning integrated mental health care teams. Organizations interested in implementing an integrated model may wish to pay particular attention to a few of our observations on the challenges these organizations faced. First, in settings with extensive existing mental health services, this pre-existing structure may impede practice change and integration. Second, insufficient follow-up may result if staffing is inadequate. Third, the effectiveness of the model may be limited if there is inadequate participation by a psychiatrist for supervision of care managers and for consultation. Successful sites engaged patients early in treatment and had multiple contacts with patients during the first 12 weeks. Thus, organizations striving to provide collaborative care should focus their efforts on developing a system that allows for early and intensive follow-up, thereby sending a strong message to patients and engaging them in their care.
We gratefully acknowledge financial support from the Hogg Foundation for Mental Health through its Integrated Health Care Initiative. In addition, Author 1 was supported by a National Research Service Award.
Disclosures: The Hogg Foundation provided a grant to Author 5 to provide technical assistance to participating clinics and funded the evaluation of the program.