PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Am J Psychiatry. Author manuscript; available in PMC 2014 April 1.
Published in final edited form as:
PMCID: PMC3816374
NIHMSID: NIHMS522806

Practice Based Versus Telemedicine Based Collaborative Care for Depression in Rural Federally Qualified Health Centers: A Pragmatic Randomized Comparative Effectiveness Trial

John C. Fortney, PhD,1,2,3 Jeffrey M. Pyne, MD,1,2,3 Sip B. Mouden, MS, CRC,4 Dinesh Mittal, MD,1,2,3 Teresa J. Hudson, PharmD,1,2,3 Gary W. Schroeder, PhD,1 David K. Williams, PhD,5 Carol A. Bynum, PhD,6 Rhonda Mattox, MD,7 and Kathryn M Rost, PhD8

Abstract

Objective

Practice Based Collaborative Care is a complex evidence-based practice that is difficult to implement in smaller primary care practices lacking on-site mental health staff. Telemedicine Based Collaborative Care virtually co-locates and integrates mental health providers into primary care settings. The objective of this multi-site randomized pragmatic comparative effectiveness trial was to compare the outcomes of patients randomized to Practice Based versus Telemedicine Based Collaborative Care.

Method

From 2007–2009, patients at Federally Qualified Health Centers serving medically underserved populations were screened for depression, and 364 patients screening positive were enrolled and followed for 18 months. Those randomized to Practice Based Collaborative Care received evidence-based care from an on-site primary care provider and nurse care manager. Those randomized to Telemedicine Based Collaborative Care received evidence-based care from an on-site primary care provider and off-site telephone nurse care manager, telephone pharmacist, tele-psychologist and tele-psychiatrist. The primary clinical outcomes were treatment response, remission and changes in depression severity

Results

There were significant group main effects for both response (OR=7.74, CI95=3.94–15.20, p<0.0001) and remission (OR=12.69, CI95=4.81–33.46, p<0.0001) and a significant overall group by time interaction effect for Hopkins Symptom Checklist depression severity (χ23=40.51, p<0.0001) with greater reductions in depression severity observed over time for those randomized to Telemedicine Based Collaborative Care. Improvements in outcomes appeared to be attributable to higher fidelity to the collaborative care evidence-base in the Telemedicine Based group.

Conclusions

Contracting with an off-site Telemedicine Based Collaborative Care team yields better outcomes than implementing Practice Based Collaborative Care with locally available staff.

INTRODUCTION

Complex evidence-based practices are difficult to implement with fidelity in routine care. One such evidence-based practice, Practice Based Collaborative Care (PBCC), involves co-located primary care (PC) providers, mental health (MH) specialists and care managers working together on-site in the PC setting. PBCC has been shown to improve depression outcomes in numerous randomized effectiveness trials.17 However, implementation in smaller PC practices presents challenges where it is typically not feasible to employ on-site MH specialists or full-time care managers. Only 25% of U.S. PC practices have on-site MH specialists.8 Two previous studies have demonstrated that depression outcomes can be improved in small PC clinics lacking on-site MH specialists by training on-site nurses to be depression care managers, however the effect sizes have been small to medium.3, 9 Moreover, based on the results of a meta analysis of randomized trials of PBCC, Gilbody concluded that collaborative care interventions with more MH specialist involvement have larger effect sizes.10

Telemedicine technologies now make possible the virtual co-location of MH specialists and PC providers. Telemedicine Based Collaborative Care (TBCC) involves an off-site team of MH specialists collaborating with on-site PC providers, from a centralized location, using telephones, interactive video and electronic health records. A multi-site randomized trial conducted in the Department of Veterans Affairs demonstrated that TBCC is more effective than usual care in small satellite PC clinics.11, 12 TBCC has also been adopted and sustained as part of routine care in small satellite PC clinics within the Department of Veterans Affairs.13 Compared to PBCC, TBCC has the potential to be implemented with higher fidelity in smaller PC clinics because the off-site team can include an array of MH specialists, and full-time care managers can practice under more intensive clinical supervision and dedicate their time to care-coordination activities (resulting in higher fidelity to care manager protocols). However, there are also potential disadvantages to TBCC. Off-site care managers cannot build upon established relationships with patients and may have difficulty establishing therapeutic alliances from a distance. Likewise, if off-site care managers lack access to on-site medical records, they may have difficulties integrating behavioral and physical aspects of care and face barriers communicating with on-site PC providers. In addition, the stepped care elements of the collaborative care model (e.g., psychiatric consultation) may be less effective when delivered via telemedicine. However, patients and providers uniformly report high levels of satisfaction with interactive video,1417 and there is good evidence documenting the clinical equivalency of psychiatric1821 and psychological2226 treatments delivered via interactive video compared to face to face.

Given that: 1) PBCC without the involvement of on-site MH specialists improves outcomes, 2) PBCC with involvement from on-site MH specialists improves outcomes more, and 3) the potential disadvantages of TBCC, an important policy relevant question is whether it is more effective for small remote PC clinics to implement PBCC without the involvement of on-site MH specialists or to implement TBCC in partnership with an off-site team of tele-MH specialists. Therefore, the objective of this Pragmatic 27 Comparative Effectiveness Trial 28 (NCT00439452) was to compare the process outcomes and clinical outcomes of patients randomized to TBCC versus PBCC in small remote PC clinics lacking on-site MH specialists, thereby comparing two organizational approaches (with clinical equipoise) to delivering collaborative care for depression.29 We chose to conduct a pragmatic trial design in order to compare two viable competing strategies to delivering an evidence-based practice rather than an explanatory trial designed to determine why one approach was superior to the other. Compared to patients randomized to PBCC, we hypothesized that patients randomized to TBCC would receive higher fidelity care management, more specialty MH services (e.g., tele-psychiatry and tele-psychology), and higher quality pharmacotherapy, and therefore would experience greater symptom improvement. Secondary outcomes examined included health status, quality of life, and satisfaction with care.

We partnered with five Federally Qualified Health Centers (FQHCs) serving medically underserved populations in Arkansas' Mississippi delta and Ozark highlands. With federal oversight from Health Resources and Services Administration (HRSA), FQHCs make up the nation’s largest and fastest growing network of PC providers, with 8,000 clinics providing services to 20 million Americans.30, 31 Three quarters of FQHC patients live in poverty, half live in rural areas, a third are uninsured, and a third are from minority populations. MH problems are the most commonly reported reasons for visits to FQHCs.32 Yet, only 5.5% of encounters are with on-site MH specialists.33 Research conducted in partnership with FQHCs has direct applicability to a large segment of the US population at risk for experiencing health disparities.

METHODS

Design Overview

This multi-site randomized trial employed a comparative effectiveness research design. 28 Patients were randomized to one of two active treatment arms, both of which represented potentially feasible approaches to adapting the evidence-based collaborative care model for routine delivery in medically underserved areas. The study design also included many elements of pragmatic trials27 including: 1) applying relatively few exclusion criteria, 2) enrolling a diverse sample of patients, 3) monitoring but not controlling intervention fidelity, 4) defining the primary outcome as changes in patient-reported symptoms , and 5) using intent-to-treat analyses to examine group differences.

Setting and Participants

We approached six FQHCs, and five (83.3%) agreed to participate. Participating FQHCs employed between 1.3 and 9.7 PC physician FTEs, served between 5,362 and 13,050 unique PC patients, and operated one to six clinics across multiple locations. None of the participating clinic locations had an on-site MH specialist. Between November 2007 and June 2009, 19,285 patients were screened for depression by FQHC staff using the PHQ9 (see Consort Diagram). Fifteen percent of patients (n=2,863) screened positive (PHQ9 ≥10); FQHC staff obtained informed consent from 62% of potentially eligible patients (n=829); and 55% (n=364) were found to be eligible and completed the baseline telephone interview. We excluded patients that PC providers would not be comfortable treating. These exclusion criteria included pregnancy, schizophrenia, acute suicide ideation, substance dependence, bipolar disorder, recent bereavement and already receiving specialty mental health treatment. We also excluded patients who were unable to participate in research. These exclusion criteria included cognitive impairment, court-appointed guardian, non-English speaking, no telephone, or life event preventing participation. Screening and eligibility results were entered into the medical record.

Randomization and Interventions

Randomization

Patients were the unit of randomization. Eligible patients were computer randomized using a 2×2 Latin Square Design (stratified by clinic) to either TBCC or PBCC. It was not feasible to blind patients or providers to randomization status.

Practice-Based Collaborative Care (PBCC)

The PBCC intervention was designed to be the same as the model of care supported by HRSA, known as the Depression Health Disparities Collaborative,34, 35 and represents more intensive treatment than usual care. PBCC involved two types of providers: on-site PC providers and on-site nurse depression care managers (DCM). Each clinic location employed a half-time Depression Care Manager (DCM) funded by the study. DCMs were RNs or LPNs who had no prior MH experience. They received one day of training in depression care management, a care manager training manual, and access to a web-based patient registry and DCM decision support system (https://www.netdss.net/).36 Patients could choose either watchful waiting or antidepressant treatment. DCM encounters were conducted either face-to-face or by telephone depending on the preference of the patient. The initial DCM encounter included: 1) PHQ9 symptom monitoring; 2) education/activation; 3) barrier assessment/resolution; and 4) establishing self-management goals, including planning physical, rewarding, and social activities. Follow-up encounters, scheduled every two weeks during acute treatment and every four weeks during continuation treatment, included the monitoring of: 1) PHQ9 symptoms, 2) medication adherence, 3) side-effects and 4) engagement in planned self-management activities. DCMs received no clinical supervision from a MH specialist. Progress notes were entered into the patients' paper medical record. A trial was considered to have failed in the acute phase if the patient did not respond (i.e., 50% decrease in PHQ9 score) after eight weeks of treatment. No additional on-site MH support was available for patients failing treatment, although patients could be referred to off-site MH providers (e.g., Community Mental Health Centers). Patients received the intervention for up to 12 months.

Telemedicine-Based Collaborative Care (TBCC)

TBCC involved five types of providers: on-site PC providers, and off-site DCM (RN), pharmacist (PharmD), psychologist (PhD) and psychiatrist (MD). The off-site team was funded by the study and located at the University of Arkansas for Medical Sciences. The full-time DCM was an RN who had no prior MH experience and received the same training and tools as the on-site DCMs. All DCM encounters were conducted by telephone and followed the protocol described above. Progress notes were faxed to the clinic. During weekly meetings, the DCM received clinical supervision and the off-site team discussed new patients and patients failing treatment, and offered treatment recommendations to PC providers via the DCM progress notes. Patients received stepped-care, whereby treatment intensity was increased for patients failing treatment. If the patient did not respond to the initial antidepressant, the telephone pharmacist conducted a medication history and provided medication management as needed. If the patient did not respond to two trials, a psychiatry consultation via interactive video was scheduled. At any time, patients had access to cognitive behavioral therapy (CBT) delivered via interactive video, and patients failing an antidepressant trial were specifically encouraged to initiate and complete CBT.

Outcomes and Follow-up

Fidelity Assessment

Fidelity to the DCM protocol was measured from chart review. Five (1.4%) of the charts could not be located during the site visits. The following fidelity measures were abstracted: number of DCM encounters with documented PHQ9 scores, self-management activities, antidepressant adherence assessments, and side-effect assessments (for those prescribed antidepressants) and counseling adherence assessments (for those referred to psychotherapy). DCM fidelity was also measured from patient self-report during the 6 and 12 months telephone interviews using items that addressed: education, self-management, symptom monitoring, adherence monitoring and collaboration among providers. To measure fidelity to the stepped care protocol, we examined what proportion of patients randomized to TBCC and failing at least one medication trial had a telephone pharmacist encounter, and what proportion of those failing two trials had a tele-psychiatry consultation. We also examined what proportion of those randomized to TBCC attended at least one tele-psychotherapy session and completed ≥8 sessions.

Baseline Interviews

Data were collected at baseline via blinded telephone interview. At baseline, socio-demographic and clinical casemix factors were collected using the Depression Outcomes Module,37, 38 Mini International Neuropsychiatric Interview,39, 40 Duke Social Support and Stress Scale,41, 42 Quality Improvement for Depression Treatment Acceptability scale,3, 4 and the Depression Health Beliefs Inventory.43 Zip codes were used to categorize patient’s residence as rural or urban according to Rural Urban Commuting Area (RUCA) classification scheme C.

Follow-up Interviews

Blinded follow-up telephone interviews were completed for 87% (n=318) at 6 months, 79% (n=287) at 12 months and 78% (n=283) at 18 months. In addition to the fidelity measures described above, the other primary process measures included self-reported MH service utilization, antidepressant prescriptions, antidepressant dose (categorized as starting, usual or high44) and antidepressant adherence (taking the full prescribed dosage at least 80% of days in previous month). The primary clinical outcomes were changes in depression severity, and treatment response and remission. Depression severity was measured continuously using the Hopkins Symptom Checklist SCL-20.45, 46 Response was measured dichotomously as a 50% improvement in SCL-20 score between baseline and follow-up. Remission was measured dichotomously as a SCL-20 score <0.5. Secondary outcomes included health status (PCS and MCS scores from the SF12),47, 48 quality of life (Quality of Well Being - QWB),4952 and satisfaction with care (CAHPS).53

Statistical Analysis

Patients were the unit of the intent-to-treat analysis. Clinic level intraclass correlation coefficients for SCL-20 change scores were not significant. Provider level intraclass correlation could not be calculated because patients could have multiple PC providers. For the hypotheses examining fidelity, separate logistic regressions were specified for the first six months and the second six months. For the hypotheses examining other outcomes, we used mixed models and included data from all completed research assessments.54 Casemix variables were selected using the method of purposeful selection.55, 56 Casemix variables with missing values were imputed using the PROC MI procedure in SAS9.3. PROC MIXED and PROC GLIMMIX were used with the PROC MIANALYZE procedure to model outcomes with linear (e.g., SCL20), binomial (e.g., response), negative binomial (e.g., visits) or ordinal (dosage categories) distributions. All models specified clinic as a random effect to control for ICC. Time was included as a fixed effect. The model specifications included main effects for group and time (with PBCC assigned as the reference group), and interaction effects for group by time. The main group effect was used to test the hypotheses for dependent variables not measured at baseline (e.g., response). For dependent variables measured at baseline (e.g., SCL-20), group by time interaction effects were used to test the hypotheses that the rate of improvement differed across the two groups. Because there were multiple group by time interaction terms, an omnibus test was used to determine whether these variables collectively explained a significant amount of the variance in the dependent variable.57 If the omnibus test was significant at the conservative alpha <0.10 level, we report group differences and significance tests for each time period. Otherwise, we report group differences averaged across the time periods and one significance test. There was 85% power to detect a 15% difference (e.g., 30% versus 45%) in response rates.

RESULTS

The socio-economic and clinical characteristics of study participants are provided in Table 1. Two thirds (64.5%) were unemployed, 69.7% had annual household incomes <$20,000, 50.8% were uninsured, and 68.1% lived in a rural area. At baseline, 83.2% met diagnostic criteria for major depressive disorder, and the mean SCL-20 score was 1.9, indicating moderately severe depression. The mean number of chronic physical health disorders was 4.6 (sd=2.6), and psychiatric comorbidity was common. Mean PCS and MCS scores were nearly two standard deviations below the general population. Nearly half (48.4%) were already receiving depression treatment at enrollment, indicating treatment resistance.

Table 1
Group Differences in Baseline Patient Socio-Demographic and Clinical Characteristicsa

Care Manager Fidelity

Table 2 describes care manager fidelity. At the 6 and 12 month follow-ups, significantly more patients randomized to TBCC compared to PBCC reported that a health care professional other than their PC provider: 1) gave them helpful information about depression or depression treatment (6 months: OR=2.77, CI95=1.67–4.61, p=<0.0001; 12 months: OR=2.32, CI95=1.37–3.94, p=0.0018), 2) made helpful suggestions about things they could do to help depression, such as exercise or becoming more socially active (6 months: OR=3.47, CI95=2.15–5.62, p<0.0001; 12 months: OR=2.50, CI95=1.48–4.23, p=0.0006), 3) asked them about their depression symptoms (6 months: OR=3.60, CI95=2.21–5.86, p<0.0001; 12 months: OR=2.63, CI95=1.54–4.52, p=0.0004), 4) asked them whether there were taking antidepressant medications as prescribed or attending scheduled counseling sessions (6 months: OR=4.70, CI95=2.82–7.84, p<0.0001; 12 months: OR=3.96, CI95=2.22–7.05, p<0.0001), and 5) their PC provider worked collaboratively with a MH specialist (6 months: OR=4.63, CI95=2.08–10.30, p=0.0002; 12 months: OR=9.05, CI95=3.04–26.93, p<0.0001).

Table 2
Group Differences in Fidelity to the Care Manager Protocol

Compared to patients randomized to PBCC, patients randomized to TBCC had significantly more DCM encounters with PHQ9 depression severity scores documented in the medical record during the first six months (Incidence Rate Ratio (IR)=4.10, CI95=3.41–4.92, p<0.0001) and second six months (IR=4.64, CI95=3.19–6.74, p<0.0001). Patients randomized to TBCC also had significantly more DCM encounters with documented self-management goals during the first six months (IR=5.62, CI95=4.46–7.07, p<0.0001). Among those patients prescribed an antidepressant, those randomized to TBCC had significantly more DCM encounters with medication adherence documented compared to those randomized to PBCC during the first six months (IR=2.69, CI95=2.20–3.28, p<0.0001) and second six months (IR=2.49, CI95=1.69–3.67, p<0.0001). Likewise, TBCC patients prescribed an antidepressant had significantly more DCM encounters with presence/absence of side effects documented during the first six months (IR=4.22, CI95=3.32–5.36, p<0.0001) and second six months (IR=4.46, CI95=2.75–7.25, p<0.0001). Among those patients referred to counseling, there were no significant group differences in the number of DCM encounters with session attendance documented in the medical record.

Stepped Care Fidelity

Among patients randomized to TBCC and failing at least one medication trial (n=73), 8.2% (n=6) had a telephone encounter with the tele-pharmacist. Among those with at least two failed trials (n=29), 48.3% (n=14) had a tele-psychiatry consultation. Another seven patients had an ad hoc tele-psychiatry consultation. Also, 16.6% (n=30) attended at least one tele-psychotherapy session and 7.8% (n=14) completed ≥8 sessions.

Treatment Process Outcomes

Table 3 describes the treatment provided to the two groups. There were no significant group differences at baseline, and no significant group by time interactions for number of PC visits (χ23=5.11, p=0.16) or number of depression-related PC visits (χ23=3.74, p=0.29), or MH visits (χ23=5.19, p=0.16). With respect to antidepressant treatment, there were no significant group main effects for being prescribed an antidepressant (OR=1.64, CI95=0.75–3.58, p=0.21), number of prescribed antidepressants (Incidence Ratio=1.19, CI95=0.92–1.56, p=0.19), dose level (starting, usual, high) (OR=1.84, CI95=0.77–4.38, p=0.17), or adherence (OR=1.22, CI95=0.38–3.89, p=0.74).

Table 3
Group Differences in Services Utilization and Antidepressant Treatment

Clinical Outcomes

Clinical outcomes are presented in Table 4. There was a significant group main effect for both response (OR=7.74, CI95=3.94–15.20, p<0.0001) and remission (OR=12.69, CI95=4.81–33.46, p<0.0001) with patients randomized to TBCC having better outcomes. There was also a significant overall group by time interaction effect for SCL-20 (χ23=40.51, p<0.0001), with greater reductions in severity for those randomized to TBCC (Figure 2). There was also a significant overall group by time interaction effect for MCS (χ23=11.46, p=0.01) and QWB (χ23=6.55, p=0.09), with greater improvements among patients randomized to TBCC. There were no significant overall group by time interaction effects for PCS (χ23=2.61, p=0.46, although this finding is likely an artifact resulting from the orthogonal factor rotation and negative weights used to score the MCS and PCS.58 There were no group differences in satisfaction at baseline, but the omnibus test of the group by time interactions was significant at the alpha <0.10 level (χ23=6.69, p=0.08), with the TBCC group having higher satisfaction.

Figure 2
Adjusted Depression Severity SCL_20 Scores
Table 4
Group Differences in Clinical Outcomes

DISCUSSION

Study participants were recruited from small remote PC clinics associated with the largest publically-funded healthcare system in the country. The sample was predominantly rural, unemployed, and uninsured. Patients had numerous comorbidities and were treatment resistant. The high degree of treatment resistance likely contributed to the low response/remission rates among those randomized to the PBCC group. Compared to patients randomized to PBCC, patients randomized to TBCC had significantly and substantially greater treatment response rates, remission rates, reductions in depression severity and increases in mental health status and quality of life. Improved outcomes were achieved in the TBCC group without increasing the number of PC visits. Improved outcomes appear to be due to higher fidelity to the care manager protocol in the TBCC model, despite the fact that off-site and on-site DCMs had similar levels of clinical experience at baseline and underwent identical training. The higher fidelity to the care manager protocol in the TBCC model may have been due to the fact that the off-site DCM practiced under more intensive clinical supervision and dedicated 100% of her time to DCM activities.

The greater fidelity to the care manager protocol did not translate into improvements in the quality of pharmacotherapy in the TBCC group. The majority of patients in both groups initiated antidepressant treatment at therapeutic dosages and adherence was high. Likewise, the psychotherapy available to patients in the TBCC group via interactive video was not highly utilized, and was not likely to have contributed substantially to improved outcomes. Our findings were essentially unchanged when patients receiving tele-psychotherapy were excluded from the analytical sample. Because group differences in outcomes are not likely to be attributable to either pharmacotherapy or psychotherapy , we hypothesize that patients randomized to TBCC were more likely to engage in self-management activities such as physical, rewarding, and social activities. This hypothesis is based on the finding that TBCC patients received more encouragement from the DCM to engage in self-management activities. Previous research has demonstrated that behavioral activation is a clinically effective stand-alone treatment for depression.59, 60 Also, patients randomized to the PBCC intervention in Project Impact were found to have significantly better treatment response rates if the DCM documented in the medical record that the patient scheduled physical, rewarding, and social activities.61 This hypothesis needs to be tested using an experimental dismantling study specifically designed to estimate the incremental treatment effect of scheduling self-management activities. Another possibility is that the more frequent DCM encounters provided to the TBCC group resulted in greater social support, which in turn reduced depressive symptoms. This was the conclusion of Hunkeler who similarly reported that a DCM program at Kaiser Permanente did not improve antidepressant management, but did reduce depression symptoms.62

An inherent limitation of this pragmatic trial (designed to inform policy), is that results are not conclusive with respect to identifying treatment mechanisms. Explanatory trial designs are needed to determine why patients randomized to TBCC had better outcomes than those randomized to PBCC. However, for policy makers at HRSA and FQHCs lacking on-site MH personnel, these results clearly indicate that contracting with an off-site depression care team yields better depression outcomes than implementing collaborative care with staff available on-site. Future research should also examine whether having on-site nurse care managers supported by off-site tele-psychiatrists, tele-psychologists and tele-pharmacists is an effective organizational approach to delivering collaborative care.

Figure 1
Consort Diagram

Acknowledgements

This research was supported by a grant from the National Institute of Mental Health, (R01 MH076908, MH076908-04S1) to Dr. Fortney. The clinical trials # is NCT00439452. We would like to gratefully acknowledge the patients and staff at the Boston Mountain Rural Health Center, Inc., Community Clinic NWA, Corning Area Healthcare Inc., East Arkansas Family Health Center, Inc., Jefferson Comprehensive Care Systems, Inc., as well as staff at the Community Health Centers for Arkansas Inc. We would also like to acknowledge the important contributions of project staff including Amanda Davis, Loretta Ducker, Debbie Hodges, Choi Lai, Liya Lu, Michael McCarther, Camille Mack, Jennifer Stephens and Vera Tate, as well as the contributions of two anonymous manuscript reviewers. Dr. Fortney had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Footnotes

Conflict of Interest: The authors report no conflicts of interest.

Reference List

1. Katon W, Von Korff M, Lin E, Simon G, Walker E, Unützer J, Bush T, Russo J, Ludman E. Stepped collaborative care for primary care patients with persistent symptoms of depression: A randomized trial. Arch Gen Psychiatry. 1999;56(12):1109–1115. [PubMed]
2. Katon W, Robinson P, Von Korff M, Lin E, Bush T, Ludman E, Simon G, Walker E. A multifaceted intervention to improve treatment of depression in primary care. Arch Gen Psychiatry. 1996;53(10):924–932. [PubMed]
3. Rost K, Nutting P, Smith J, Werner J, Duan N. Improving depression outcomes in community primary care practice: A randomized trial of the QuEST intervention. J Gen Intern Med. 2001;16(3):143–149. [PMC free article] [PubMed]
4. Wells KB, Sherbourne C, Schoenbaum M, Duan N, Meredith L, Unützer J, Miranda J, Carney MF, Rubenstein LV. Impact of disseminating quality improvement programs for depression in managed primary care: A randomized controlled trial. J Am Med Assoc. 2000;283(2):212–220. [PubMed]
5. Unützer J, Katon W, Callahan CM, Williams JW, Jr, Hunkeler E, Harpole L, Hoffing M, Della Penna RD, Noël PH, Lin EH, Areán PA, Hegel MT, Tang L, Belin TR, Oishi S, Langston C. IMPACT Investigators. Improving Mood-Promoting Access to Collaborative Treatment. Collaborative care management of late-life depression in the primary care setting: A randomized controlled trial. J Am Med Assoc. 2002;288(22):2836–2845. [PubMed]
6. Alexopoulos GS, Katz IR, Bruce ML, Heo M, Have TT, Raue P, Bogner HR, Schulberg HC, Mulsant BH, Reynolds CF., III Remission in depressed geriatric primary care patients: A report from the PROSPECT Study. Am J Psychiatry. 2005;162(4):718–724. [PMC free article] [PubMed]
7. Bruce ML, Ten Have TR, Reynolds CF, III, Katz II, Schulberg HC, Mulsant BH, Brown GK, McAvay GJ, Pearson JL, Alexopoulos GS. Reducing suicidal ideation and depressive symptoms in depressed older primary care patients: A randomized controlled trial. J Am Med Assoc. 2004;291(9):1081–1091. [PubMed]
8. Williams JW, Jr, Rost K, Dietrich AJ, Ciotti MC, Zyzanski SJ, Cornell J. Primary care physicians' approach to depressive disorders. Effects of physician specialty and practice structure. Arch Fam Med. 1999;8:58–67. [PubMed]
9. Gensichen J, Von KM, Peitz M, Muth C, Beyer M, Guthlin C, Torge M, Petersen JJ, Rosemann T, Konig J, Gerlach FM. PRoMPT (PRimary care Monitoring for depressive Patients Trial) Case management for depression by health care assistants in small primary care practices: A cluster randomized trial. Ann Intern Med. 2009;151:369–378. [PubMed]
10. Gilbody S, Whitty P, Grimshaw J, Thomas R. Educational and organizational interventions to improve the management of depression in primary care: A systematic review. J Am Med Assoc. 2003;289(23):3145–3151. [PubMed]
11. Fortney J, Pyne JM, Edlund MJ, Williams DK, Robinson DE, Mittal D, Henderson KL. A Randomized Trial of Telemedicine-Based Collaborative Care for Depression. J Gen Intern Med. 2007;22(8):1086–1093. [PMC free article] [PubMed]
12. Pyne JM, Fortney JC, Tripathi S, Maciejewski ML, Edlund MJ, Williams DK. Cost effectiveness analysis of a rural telemedicine collaborative care intervention for depression. Arch Gen Psychiatry. 2010;67(8):812–821. [PubMed]
13. Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, Curran G. Implementation outcomes of evidence-based quality improvement for depression in VA community-based outpatient clinics. Implementation Science. 2012;7:30. [PMC free article] [PubMed]
14. Rohland BM, Saleh SS, Rohrer JE, Romitti PA. Acceptability of telepsychiatry to a rural population. Psychiatr Serv. 2000;51(5):672–674. [PubMed]
15. Callahan EJ, Hilty DM, Nesbitt TS. Patient satisfaction with telemedicine consultation in primary care: Comparison of ratings of medical and mental health applications. Telemed J. 1998;4(4):363–369. [PubMed]
16. Germain V, Marchand A, Bouchard S, Guay S, Drouin MS. Assessment of therapeutic alliance in face-to-face or videoconference treatment for posttraumatic stress disorder. Cyberpsychology, Behavior and Social Networking. 2010;13(1):29–35. [PubMed]
17. Simpson S. The provision of a telepsychology service to Shetland: Client and therapist satisfaction and the ability to develop a therapeutic alliance. Journal of Telemedicine & Telecare. 2001;7(Suppl 1):34–36. [PubMed]
18. Ruskin PE, Silver-Aylaian M, Kling MA, Reed SA, Bradham DD, Hebel JR, Barrett D, Knowles F, III, Hauser P. Treatment outcomes in depression: Comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry. 2004;161(8):1471–1476. [PubMed]
19. Ruskin PE, Reed S, Kumar R, Kling MA, Siegel E, Rosen M, Hauser P. Reliability and acceptability of psychiatric diagnosis via telecommunication and audiovisual technology. Psychiatr Serv. 1998;49(8):1086–1088. [PubMed]
20. O'Reilly R, Bishop J, Maddox K, Hutchinson L, Fisman M, Takhar J. Is telepsychiatry equivalent to face-to-face psychiatry? Results from a randomized controlled equivalence trial. Psychiatr Serv. 2007;58(6):836–843. [PubMed]
21. Hailey D, Roine R, Ohinmaa A. The effectiveness of telemental health applications: A review. Can J Psychiatry. 2008;53(11):769–778. [PubMed]
22. Nelson EL, Barnard M, Cain S. Treating childhood depression over videoconferencing. Telemedicine Journal and e-Health. 2003;9(1):49–55. [PubMed]
23. Bouchard S, Paquin B, Payeur R, Allard M, Rivard V, Fournier T, Renaud P, Lapierre J. Delivering cognitive-behavior therapy for panic disorder with agoraphobia in videoconference. Telemedicine Journal & e-Health. 2004;10(1):13–25. [PubMed]
24. Frueh BC, Monnier J, Yim E, Grubaugh AL, Hamner MB, Knapp RG. A randomized trial of telepsychiatry for post-traumatic stress disorder. J Telemed Telecare. 2007;13:142–147. [PubMed]
25. Germain V, Marchand A, Bouchard S, Drouin MS, Guay S. Effectiveness of cognitive behavioural therapy administered by videoconference for posttraumatic stress disorder. Cognitive Behaviour Therapy. 2009;38(1):42–53. [PubMed]
26. Morland LA, Greene CJ, Rosen CS, Foy D, Reilly P, Shore J, He Q, Frueh BC. Telemedicine for anger management therapy in a rural population of combat veterans with posttraumatic stress disorder: A randomized noninferiority trial. J Clin Psychol. 2010;71(7):855–863. [PubMed]
27. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–475. [PubMed]
28. Institute of Medicine. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press; 2009.
29. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. J Am Med Assoc. 2003;290(12):1624–1632. [PubMed]
30. National Association of Community Health Centers. Expanding Health Centers under Health Care Reform: Doubling Patient Capacity and Bringing Down Costs. 2010. Jun,
31. National Association of Community Health Centers. America's Health Centers. 2010. Aug,
32. Druss BG, Bornemann T, Fry-Johnson YW, McCombs HG, Politzer RM, Rust G. Trends in Mental Health and Substance Abuse Services at the Nation's Community Health Centers: 1998–2003. Am J Public Health. 2006;96:1779–1784. [PubMed]
33. National Association of Community Health Centers. US Health Center Fact Sheet 2006. 2007. Ref Type: Pamphlet.
34. Landon BE, Hicks LS, O'Malley AJ, Lieu TA, Keegan T, McNeil BJ, Guadagnoli E. Improving the management of chronic disease at community health centers. N Engl J Med. 2007;356:921–934. [PubMed]
35. Meredith LS, Mendel P, Pearson M, Wu S, Joyce G, Straus JB, Ryan G, Keeler E, Unutzer J. Implementation and maintenance of quality improvement for treating depression in primary care. Psychiatr Serv. 2006;57(1):48–55. [PubMed]
36. Fortney JC, Pyne JM, Steven CA, Williams JS, Hedrick RG, Lunsford AK, Raney WN, Ackerman BA, Bonner LM, Smith JL. A web-based clinical decision support system for depression care management. American Journal of Managed Care. 2010;16(11):849–854. [PMC free article] [PubMed]
37. Smith GR, Jr, Burnam A, Burns BJ, Cleary P, Rost KM. Depression Outcomes Module (DOM) In: American Psychiatric Association, editor. Handbook of Psychiatric Measures. Washington, DC: 2000. pp. 213–215.
38. Kramer TL, Smith GR, D'Arezzo KW, Card-Higginson P. The Guide to Behavioral Health Outcomes Management Systems. Little Rock AR: 2000. Depression Outcomes Module; pp. 71–83.
39. Lecrubier Y, Sheehan DV, Weiller E, Amorim P, Bonora I, Sheehan KH, Janavs J, Dunbar GC. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: Reliability and validity according to the CIDI. Eur Psychiatry. 1997;12(5):224–231.
40. Sheehan DV, Lecrubier Y, Sheehan KH, Janavs J, Weiller E, Keskiner A, Schinka J, Knapp E, Sheehan MF, Dunbar GC. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry. 1997;12(5):232–241.
41. Parkerson GR, Jr, Michener JL, Wu LR, Finch JN, Muhlbaier LH, Magruder-Habib K, Kertesz JW, Clapp-Channing N, Morrow DS, Chen AL, Jokerst E. Associations among family support, family stress, and personal functional health status. J Clin Epidemiol. 1989;42(3):217–229. [PubMed]
42. Parkerson GR, Jr, Broadhead WE, Tse CJ. Quality of life and functional health of primary care patients. J Clin Epidemiol. 1992;45(11):1303–1313. [PubMed]
43. Edlund MJ, Fortney JC, Reaves CM, Pyne JM, Mittal D. Belief about depression and depression treatment among depressed veterans. Med Care. 2008;46(6):581–589. [PubMed]
44. Simon GE, Lin EHB, Katon W, Saunders K, VonKorff M, Walker E, Bush T, Robinson P. Outcomes of "inadequate" antidepressant treatment. J Gen Intern Med. 1995;10(12):663–670. [PubMed]
45. Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. The Hopkins symptom checklist (HSCL): A measure of primary symptom dimensions. Pharmacopsychiatry. 1974;7:79–110. [PubMed]
46. Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. The Hopkins symptom checklist (HSCL): A self-report symptom inventory. Behav Sci. 1974;19:1–15. [PubMed]
47. Jones D, Kazis L, Lee A, Rogers W, Skinner K, Cassar L, Wilson N, Hendricks A. Health status assessments using the Veterans SF-12 and SF-36: methods for evaluating outcomes in the Veterans Health Administration. J Ambulatory Care Manage. 2001;24(3):68–86. [PubMed]
48. Kazis LE, Miller DR, Clark J, Skinner K, Lee A, Rogers W, Spiro A, III, Payne S, Fincke G, Selim A, Linzer M. Health-related quality of life in patients served by the Department of Veterans Affairs: Results from the Veterans Health Study. Arch Intern Med. 1998;158(6):626–632. [PubMed]
49. Kaplan RM, Anderson JP. The general health policy model: An integrated approach. In: Spiker B, editor. Quality of Life and Pharmacoeconomics in Clinical Trials. Second ed. Philadelphia: Lippincott-Raven Publishers; 1996. pp. 309–321.
50. Kaplan RM, Bush JW. Health-related quality of life measurement for evaluation research and policy analysis. Health Psychol. 1982;1(1):61–80.
51. Pyne JM, Patterson TL, Kaplan RM, Gillin JC, Koch WL, Grant I. Assessment of the quality of life of patients with major depression. Psychiatr Serv. 1997;48(2):224–230. [PubMed]
52. Pyne JM, Patterson TL, Kaplan RM, Ho S, Gillin JC, Golshan S, Grant I. Preliminary longitudinal assessment of quality of life in patients with major depression. Psychopharmacol Bull. 1997;33(1):23–29. [PubMed]
53. Beebe TJ, Harrison PA, McRae JA, Jr, Asche SE. Evaluating behavioral health services in Minnesota's Medicaid population using the Experience of Care and Health Outcomes (ECHO) Survey. Journal of Health Care for the Poor & Underserved. 2003;14(4):608–621. [PubMed]
54. Little RC, Milliken GA, Stroup WW, Wolfinger RD. SAS System for Mixed Models. Cary, NC: SAS Institute Inc.; 1996.
55. Bursac Z, Gauss CH, Williams DK, Hosmer DW. Purposeful selection of variables in logistic regression. Source Code for Biology and Medicine. 2008;3(17) [PMC free article] [PubMed]
56. Bursac Z, Gauss H, Williams DK, Hosmer D. A purposeful selection of variables macro for logistic regression; Proceedings at the SAS Global Forum 2007 Conference; Cary, NC. 2007.
57. Hogan JW, Laird NM. Intention-to-treat analyses for incomplete repeated measures data. Biometrics. 1996;52:1002–1017. [PubMed]
58. Simon GE, Revicki DA, Grothaus L, Von Korff M. SF-36 summary scores: Are physical and mental health truly distinct? Med Care. 1998;36(4):567–572. [PubMed]
59. Spates CR, Pagoto S, Kalata A. A qualitative and quantitative review of behavioral activation treatment of major depressive disorder. The Behavior Analyst Today. 2006;74(4):508–518.
60. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, Addis ME, Gallop R, McGlinchey JB, Markley DK, Gollan JK, Atkins DC, Dunner DL, Jacobson NS. Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. J Consult Clin Psychol. 2006;74(4):658–670. [PubMed]
61. Ribe G, Fan M, Unützer J, Vannoy S. Activity scheduling as a core component of effective care management for late-life depression. International Journal of Geriatric Society. 2012 epub ahead of print. [PMC free article] [PubMed]
62. Hunkeler EM, Meresman JF, Hargreaves WA, Fireman B, Berman WH, Kirsch AJ, Groebe J, Hurt SW, Braden P, Getzell M, Feigenbaum PA, Peng T, Salzer M. Efficacy of nurse telehealth care and peer support in augmenting treatment of depression in primary care. Arch Fam Med. 2000;9:700–708. [PubMed]