A Marginal Structural Model to Estimate the Causal Effect of Antidepressant Medication Treatment on Viral Suppression among Homeless and Marginally Housed Persons Living with HIV

doi:10.1001/archgenpsychiatry.2010.160

Arch Gen Psychiatry. Author manuscript; available in PMC 2011 December 1.

Published in final edited form as:

Arch Gen Psychiatry. 2010 December; 67(12): 1282–1290.

doi: 10.1001/archgenpsychiatry.2010.160

PMCID: PMC3208399

NIHMSID: NIHMS329976

A Marginal Structural Model to Estimate the Causal Effect of Antidepressant Medication Treatment on Viral Suppression among Homeless and Marginally Housed Persons Living with HIV

Alexander C. Tsai, MD, PhD,¹ Sheri D. Weiser, MD, MPH,² Maya L. Petersen, MD, PhD,³ Kathleen Ragland, PhD,⁴ Margot B. Kushel, MD,⁴ and David R. Bangsberg, MD, MPH^5,^6,^7,^8,⁹

¹Langley Porter Psychiatric Institute, University of California at San Francisco (San Francisco, California)

²Division of HIV/AIDS, San Francisco General Hospital, University of California at San Francisco(San Francisco, California)

³Division of Biostatistics, School of Public Health, University of California at Berkeley(Berkeley, California)

⁴Division of General Internal Medicine, San Francisco General Hospital, University of California at San Francisco (San Francisco, California)

⁵Harvard Medical School (Boston, Massachusetts)

⁶Harvard Initiative for Global Health (Cambridge, Massachusetts)

⁷Center for Global Health, Massachusetts General Hospital (Boston, Massachusetts)

⁸Mbarara University of Science and Technology (Mbarara, Uganda)

⁹Phillip T. and Susan M. Ragon Institute of Massachusetts General Hospital, Massachusetts Institute of Technology, and Harvard University(Charlestown, Massachusetts)

Corresponding Author: David Bangsberg, MD, MPH, Harvard Initiative for Global Health, 104 Mt. Auburn Street, Cambridge, Massachusetts, 02138 (Email: david_bangsberg/at/harvard.edu)

The publisher's final edited version of this article is available at Arch Gen Psychiatry

See other articles in PMC that cite the published article.

Abstract

context

Depression strongly predicts non-adherence to HIV antiretroviral therapy, and adherence is essential to maintaining viral suppression. This suggests that pharmacologic treatment of depression may improve virologic outcomes. However, prior longitudinal observational analyses have inadequately adjusted for time-varying confounding by depression severity, which could yield biased estimates of treatment effect. Application of marginal structural modeling to longitudinal observation data can, under certain assumptions, approximate the findings of a randomized controlled trial.

Objective

To determine whether antidepressant medication treatment increases the probability of HIV viral suppression.

Design

Community-based prospective cohort study with assessments conducted every three months.

Setting

Community-based research field site in San Francisco, California.

Participants

One hundred and fifty-eight homeless and marginally housed persons living with HIV who met baseline immunologic (CD4+T-lymphocyte cell count <350 cells/mm³)and psychiatric (Beck Depression Inventory-II score >13) inclusion criteria, followed from April 2002 through August 2007.

Main Outcome Measures

Probability of achieving viral suppression to <50 copies/mL. Secondary outcomes of interest were probability of being on antiretroviral therapy, seven-day self-reported percent adherence to antiretroviral therapy, and probability of reporting complete (100%) adherence.

Results

Marginal structural models estimated a 2.03 greater odds of achieving viral suppression(95% CI, 1.15–3.58; P=0.025) resulting from antidepressant medication treatment. In addition, antidepressant medication use increased the probability of antiretroviral uptake(weighted odds ratio, 3.87; 95% CI, 1.98–7.58; P<0.001). Self-reported adherence to antiretroviral therapy increased by 25% percentage points (95% CI, 14–36%; P<0.001), and the odds of reporting complete adherence nearly doubled (weighted odds ratio, 1.94; 95% CI, 1.20–3.13; P=0.006).

Conclusions

Antidepressant medication treatment increases viral suppression among persons living with HIV. This effect is likely attributed to improved adherence to a continuum of HIV care, including increased uptake and adherence to antiretroviral therapy.

INTRODUCTION

Depression is common among people living with HIV/AIDS. In a nationally representative probability sample of adults receiving care for HIV in the United States, the 12-month prevalence of major depressive disorder(MDD)using the Composite International Diagnostic Interview Short Form was 36 percent ¹. This exceeds the 5–7 percent 12-month prevalence of MDD in the general population ^2–4.

Among persons living with HIV, depression has been associated with reduced uptake ^{5, 6} of and adherence ^7–9 to antiretroviral therapy (ART), as well as CD4+ T-lymphocyte count decline ¹⁰ and progression to AIDS ¹¹. However, little research exists on whether pharmacologic treatment of depressed mood can improve HIV outcomes ¹². One analysis of electronic medical record data from persons living with HIV enrolled in two large health maintenance organizations showed that depression was associated with reduced odds of achieving HIV-1 RNA suppression to <500 copies/mL and that treatment with serotonin specific reuptake inhibitor (SSRI) medications was associated with improved ART adherence and viral suppression ⁸. This analysis, however, did not adjust for depression severity, which could have confounded the observed relationship between treatment and outcome. Specifically, patients with more severe symptoms of depression are more likely to be prescribed treatment with antidepressant medication, and antidepressant medication treatment may improve subsequent depression severity (Figure 1). Confounding arises because depression severity is associated with the outcome.

Figure 1

Model of the causal pathway between antidepressant medication use and viral suppression, with time-varying confounding by indication

While observational studies using conventional statistical methods can adjust for baseline confounding by indication(to the extent that confounders are measured without error), they are unable to adjust for time-dependent confounding that arises in longitudinal treatment settings. Conventional statistical adjustment, i.e., including depression severity as a time-dependent variable in a regression model, may bias the estimated treatment effect by conditioning on part of the effect of interest. The statistical methodology of marginal structural models provides a means to account for this time-dependent confounding by indication. Under certain assumptions(the validity of which are examined below), marginal structural modeling aims to use observational data to approximate the findings of a randomized controlled trial ^13–16. Therefore, we fit a marginal structural model to data from a longitudinal cohort of homeless and marginally housed persons living with HIV to estimate the effect of treatment with antidepressant medications on ART adherence and viral suppression.

METHODS

THE REACH COHORT

Data for this analysis were drawn from participants followed from April 2002 through August 2007 in the Research on Access to Care in the Homeless (REACH) study, which is an observational, prospective cohort of homeless and marginally housed adults living with HIV in San Francisco, California ^{17, 18}. In brief, study participants in the parent cohort were recruited from homeless shelters, free-lunch programs, and low-income, single-room occupancy hotels. Participants signed a written consent form upon entry into the study and were reimbursed $10–15 per assessment, which occurred approximately every three months at the UCSF Clinical and Translational Science Institute Tenderloin Clinical Research Center and included a structured interview and blood collection. This yielded quarterly-updated information on socio-demographics, depression severity, alcohol and drug use, health services utilization, overall health status, and medications. Depression severity was measured using the Beck Depression Inventory II (BDI-II) ¹⁹. The Committee on Human Research at the University of California at San Francisco approved all study procedures.

Participants were eligible for inclusion in this analysis if they had (1) a CD4+T-lymphocyte cell count <350 cells/mm³ at baseline, and (2) symptoms of depression at baseline, defined as a BDI-II >13. Our choice of a CD4+ count <350 was based on a threshold, widely used at the time of the study, for deciding when to initiate ART in asymptomatic HIV-infected patients ²⁰. The psychiatric inclusion criterion represents a reasonable clinical threshold at which many psychiatrists would choose to recommend starting psychopharmacologic or psychotherapeutic treatment for depressed mood. We decided not to limit the sample solely to participants with formal DSM diagnoses, because sub-syndromal symptoms are commonly experienced during the course of mood disorders and are associated with significant psychosocial impairment ^21–23.

For this study, the primary outcome of interest was probability of HIV-1 RNA viral suppression to <50 copies/mL. Plasma was processed and stored at −40°C within 6 hours of collection. HIV-1 viral load determinations were made using the HIV-1 Amplicor Monitor Version 1.5 ultrasensitive assay (Roche Molecular Systems, Alameda, California, USA), with a lower detection limit of 20 copies/mL. Secondary outcomes of interest were: (1)probability of being on ART;(2) self-reported ART adherence, defined as the percentage of prescribed ART doses taken within a seven-day recall period ^{24, 25}; and (3)probability of reporting complete (i.e., 100%) ART adherence. Zero adherence was assigned to participants who were eligible for but were not on ART, consistent with an expanded concept of adherence to a continuum of HIV care including ART uptake, persistence, and dose-taking adherence (or execution) ^{26, 27} that has been used in prior research ²⁸.

STATISTICAL ANALYSIS

We used weighted regression modeling to estimate the parameters of a marginal structural model ^13–16. That is, rather than adjust for time-dependent confounding by including depression severity as a covariate in the regression model, each patient received a weight inversely proportional to the estimated probability of having her own observed antidepressant medication treatment history. Intuitively, this approach corrects for the non-random assignment of antidepressant medication treatment by up-weighting individuals whose treatment and covariate histories are under-represented compared to what would have been observed if treatment had been randomized. This approach accounts for confounding without stratifying or conditioning on factors in the postulated causal pathway and has been successfully applied in the field of HIV medicine, yielding results that have more closely approximated the findings from randomized controlled trials than have other statistical adjustment methodologies ^{13, 15}. Marginal structural modeling has also been used to estimate the effects of other time-varying exposures, such as methotrexate in patients with rheumatoid arthritis ²⁹ and aspirin among middle-aged men ³⁰.

The model used to estimate the denominator of the weights was a pooled logistic regression model ³¹ for the probability of receipt of antidepressant medication at a given visit. Included in this logistic model were variables, measured at baseline, that have been previously studied as potential correlates of psychotropic medication use among persons with HIV ^{32, 33}: age (years), sex, education (high school graduate and some college vs. no diploma), self-identified race (white, black, other), presence of one of five chronic medical conditions (heart disease, hypertension, diabetes, emphysema, or asthma), CD4+ count nadir, substance use (alcohol, crack cocaine, methamphetamines, heroin, or any injection drug) in the 30 days prior to baseline, and BDI-II score. We also included time-varying BDI-II score, measured at the prior visit, and cumulative number of days of follow-up, modeled as a restricted cubic spline with knots at the 5^th, 25^th, 50^th, 75^th, and 95^th centiles. The model used to estimate the numerator of the weights was similar, except that terms depending on the time-varying covariates were eliminated.

Each person-visit was treated as an observation, and the model was fit on the subsample of person-visits for which no exposure to antidepressant medication had yet occurred through the prior visit. We conducted the analysis using a conservative “intention-to-treat” assumption ^{15, 34}, which is necessary to avoid generating over inflated estimates of treatment effect ^{35, 36}. In the context of our study, the observational analog of this assumption meant that once participants were started on antidepressant medication they were assumed to remain on it thereafter (i.e., probability weights were unaffected by subsequent depression severity scores or weights). To adjust for potential selection bias by measured factors due to loss to follow-up, a second set of censoring weights was obtained using a similar procedure, where participants who died were designated failures and censoring was defined as loss to follow-up for any other reason ^{13, 14}. The overall inverse probability of treatment and censoring (IPTC) weights were computed as the product of the treatment and censoring weights, and then stabilized to increase efficiency ^{13, 14}.

To estimate the effect of antidepressant medication treatment on viral suppression, the IPTC weights were used in a weighted pooled logistic regression model with viral suppression to <50 copies/mL as the outcome. We re-assessed the statistical significance of the treatment estimate when self-reported ART adherence was included in the regression model. We interpreted an attenuated treatment estimate as suggestive that the effect was mediated by adherence, although additional assumptions would be necessary to make a definitive conclusion. For the secondary outcomes, we estimated the effect of antidepressant medication treatment on self-reported adherence by using the same IPTC weights in a weighted pooled linear regression model with self-reported adherence as the outcome, and in weighted pooled logistic regression models with being on ART and complete adherence as the outcomes. These regression models included the same baseline covariates as were used in estimation of the weights but did not include the time-varying covariate. The primary regressor of interest was receipt of antidepressant medication treatment at or before the prior (quarterly) visit. We used a twelve-week lag period because this has been considered a duration of antidepressant medication treatment sufficient to produce a robust therapeutic effect ^{37, 38}. All analyses were censored at the last time the participant remained under follow-up. Standard errors were based on robust variance estimates to account for clustering of observations within participants over time ^39–42.

SENSITIVITY ANALYSIS

We undertook a number of sensitivity analyses to assess the robustness of our findings ⁴³. First, in light of prior research showing that the efficacy of antidepressant medication in improving mood is greater among those with more severe depression ^44–46, we stratified our analyses by baseline depression severity. We compared the effect of antidepressant medication treatment on viral suppression among those with minimal or mild depression at baseline (BDI-II<20)vs. moderate-to-severe depression at baseline (BDI-II≥20). Second, we examined the sensitivity of our estimates to different model specifications. We included different configurations of additional baseline and time-varying covariates, including alcohol and substance use (prior 30 days), Short Form-36 (SF-36) Mental Component Summary (MCS) and Physical Component Summary (PCS) scores, self-reported overall health, emergency department and hospital utilization (prior 90 days), homelessness status (prior 90 days), and representative payeeship (prior 90 days). Third, in order to explore bias-variance tradeoffs, we progressively trimmed ⁴⁷ the IPTC weights at the 1st and 99th percentiles, the 5th and 95th percentiles, and the 10th and 90th percentiles. And fourth, we re-fit all models using treatment with SSRI medication (vs. no SSRI) as the exposure. We examined the effect of this specific class of antidepressant medication because SSRIs are generally regarded, due to safety and tolerability considerations ⁴⁸, as first-line agents for pharmacologic treatment of depression in patients with a substance abuse comorbidity profile similar to the participants in the REACH cohort. Furthermore, SSRIs are the class of antidepressant medication most commonly prescribed to HIV-infected persons with mood disorders ³³.

RESULTS

CHARACTERISTICS OF THE SAMPLE

A total of 158 participants (out of 551 in the parent cohort) met inclusion criteria and contributed a total of 1,782 person-quarters of observation. The average length of follow up was 2.9 years(median, 3.0 years; range, 0.2–5.3 years). During the follow up period, 38 participants died(24%), and 17 were lost to follow up(11%). An additional 8 completed 12 months of follow up according to a prespecified protocol for a related randomized controlled trial (but then exited the cohort)(5%), and one left the study due to incarceration (1%).

At baseline, 92 participants (58%) were on ART despite being eligible to receive it. There were 750 person-quarters of observation contributed prior to antidepressant medication initiation, with serotonin-specific reuptake inhibitor medications being the most frequently prescribed type of antidepressant medication (85%). Among the 119 participants who ultimately received antidepressant medication treatment at some point during follow up, the average percent time actually on antidepressant medication after initiation was 67% (median, 73%; interquartile range [IQR], 42–100%). In terms of total treatment time, 763 of 1,259 (61%) person-quarters of observation after antidepressant medication initiation were spent on treatment. Many subjects experienced one or more subsequent interruptions of antidepressant medication treatment, suggesting that our “intention-to-treat” assumption would yield conservative estimates of treatment effect ^{35, 36}. The median duration of uninterrupted antidepressant medication treatment was 251 days (IQR, 85–432 days).

Baseline summary statistics for the sample are displayed in Table 1. One-third to one-half of the sample reported alcohol or drug use. Participants who had been ever treated with antidepressant medications appeared to have greater severity of illness at baseline. The ever-treated group had a lower mean baseline SF-36 MCS score, and higher proportion had a chronic medical condition The ever-treated and never-treated groups were relatively balanced with regards to other baseline characteristics such as CD4+ count, log viral load, self-reported overall health, alcohol use, and socioeconomic indicators.

Table 1

Baseline characteristics of study participants (N=158)

Mean depression severity as measured by the BDI-II was greater among those who had ever initiated treatment with antidepressant medications (23.9 vs. 20.2; P=0.06). This was consistent with what was observed in the multivariable probability-of-treatment model used to construct the weights (Table 2): each one-point increase in the BDI-II at the prior visit was associated with a 4% increased odds of initiating treatment with antidepressant medication (adjusted odds ratio [AOR]=1.04; 95% CI, 1.01–1.08; P=0.02), even after adjusting for baseline severity of depression. Participants were also more likely to start antidepressant medication treatment if they were male or chronically ill. Stabilized IPTC weights based on the resulting model fit had a mean of 1.001(SD=0.12). Further details on the distribution of both stabilized and unstabilized weights are available in the appendix(Web Appendix Figure 1).

Table 2

Factors associated with starting antidepressant medication treatment (N=158)

EFFECT OF ANTIDEPRESSANT MEDICATION TREATMENT

Without any adjustment for confounding, antidepressant medication treatment was associated with a 1.55 greater odds (95% CI, 1.03–2.31; P=0.034) of achieving viral suppression(Table 3). Using conventional multivariable logistic regression adjustment strategy for confounding, antidepressant medication treatment was associated with a 1.58 greater odds (95% CI, 1.07–2.31; P=0.02) of achieving viral suppression. However, because depression severity is affected by past treatment with antidepressant medication, these estimates may not carry a causal interpretation as the overall effect of antidepressant medication treatment. Marginal structural models estimated a 2.03 greater odds (95% CI, 1.15–3.58; P=0.025) of achieving viral suppression. When self-reported ART adherence was included in the regression model, the estimated effect declined in magnitude and statistical significance (weighted OR=1.32; 95% CI, 0.73–2.40; P=0.36).

Table 3

Estimates of the effect of antidepressant medication treatment on viral suppression, using conventional statistical adjustment vs. inverse probability of treatment and censoring (IPTC) weighting (N=158)

Antidepressant medication treatment seemed to have larger effects among participants with more severely depressed mood. Among those with minimalor mild depression severity at baseline(BDI-II<20), antidepressant medication treatment did not result in a statistically significant increased odds of achieving viral suppression (weighted OR=1.75; 95% CI, 0.71–4.33; P=0.23). However, the confidence interval does not rule out the possibility of a reasonably large benefit in this subgroup. Among those with moderate-to-severe depression at baseline (BDI-II≥20), however, the effect on viral suppression was statistically significant(weighted OR=2.77; 95% CI, 1.26–6.09; P=0.011).

In supplemental analyses, we sought to determine whether the effect of antidepressant medication treatment on viral suppression could be attributable to its effects on a continuum of HIV care. Weighted regression showed that antidepressant medication use resulted in a 3.87 greater odds of being on ART (95% CI, 1.98–7.58; P<0.001). Additionally, antidepressant medication treatment increased self-reported adherence by 25% (95% CI, 14–36%; P<0.001) and nearly doubled the odds of achieving complete adherence (weighted OR=1.94; 95% CI, 1.20–3.13; P=0.006).

SENSITIVITY ANALYSIS

To explore the sensitivity of our estimates to alternative model specifications, we added more baseline and time-varying covariates to the regression models in different configurations (Web Appendix Table 1). Under these alternate specifications, the estimated odds ratio for achieving viral suppression ranged from 1.52–2.19 (P-values 0.19 to 0.01). Because these alternate specifications did not produce qualitatively dissimilar estimates, we reported the results of the original model as our primary findings. Next, we progressively truncated the IPTC weights at the 1st and 99th, 5th and 95th, and 10th and 90th percentiles (Web Appendix Table 2). The estimated odds ratios were qualitatively similar to (i.e., within +/−2% of) the original estimates, indicating that, despite relatively larger weights, outlier participants did not exert overt influence on the results. Finally, we re-fit all models to determine the effect of SSRI medication treatment on viral suppression. Marginal structural models estimated qualitatively similar effects of SSRIs on probability of achieving viral suppression (weighted OR=1.73; 95% CI, 0.84–3.55; P=0.14).

COMMENT

Using a marginal structural model to account for time-varying confounding by depression severity, we found that antidepressant medication treatment increased the probability of achieving viral suppression among a cohort of homeless and marginally housed persons living with HIV. In supplemental analyses, we found evidence of improved adherence along a continuum of HIV care: antidepressant medication treatment increased the probability of ART uptake by nearly fourfold and also resulted in a 25% percentage point increase in self-reported ART adherence and a nearly twofold-increased probability of achieving complete adherence. These results are consistent with prior studies linking depressive symptoms to reduced uptake ^{5, 6} and adherence ^7–9 to ART.

While changes in behavior are the most plausible explanation for our findings ^{49, 50}, some researchers have hypothesized that biological pathways may directly link depression to poorer HIV outcomes ⁵¹. This is consistent with prior studies showing that, even after adjusting for ART adherence, depression is associated with worsened HIV outcomes, including CD4+ count decline ⁵², incident AIDS-defining illness ⁵³, and AIDS-related mortality ⁵⁴. One study showed that resolution of a major depressive episode was associated with increased natural killer cell activity ⁵⁵. And more recently, a cross-sectional analysis of data from 658 HIV-positive men and women showed that participants taking serotonin reuptake inhibitors were less likely to have detectable cerebrospinal fluid HIV-1 RNA levels ⁵⁶. This relationship held even among those not concurrently taking ART, suggestive of a biologic effect and leading some to suggest that psychotropic medications could be useful as adjunctive treatment for persons living with HIV ⁵⁷. In our marginal structural model analysis, the estimated effect of antidepressant medication treatment became non-statistically significant when adjusted for ART adherence, suggesting that the effect of antidepressants on HIV treatment response is at least partially mediated by adherence. However, the attenuation of the treatment effect once adherence was added to the model could also have been due to the limitations of our relatively small sample size. Additional assumptions would be required to fully interpret the adjusted effect as a direct (non-mediated) effect of antidepressant medication treatment ^{58, 59}. In particular, we would need to assume that the baseline covariates alone capture all of the confounding from the effect of adherence on viral suppression, which is unlikely to be the case. Distinguishing the relative contributions of the two mechanisms, direct vs. indirect (biological vs. behavioral), through which antidepressant medication treatment could affect virologic outcomes was beyond the scope of our study and remains an important area for future work.

We observed greater effects of antidepressant medication on viral suppression among participants with more severe depressive symptoms at baseline. This finding is potentially analogous to results from a recently published meta-analysis of randomized controlled trials showing that the efficacy of antidepressant medication on mood is greater among those with more severe depressive symptoms at baseline ^44–46. In other clinical contexts, marginal structural models have also estimated treatment effects that closely approximate the findings from randomized controlled trials ^{16, 60–62}. Even though our estimates have a causal interpretation under certain assumptions, randomized controlled trial evidence is needed in order to definitively conclude that pharmacologic treatment of depression has beneficial effects on HIV treatment adherence and HIV treatment outcomes.

Despite these caveats, our study contributes to a sparse literature on how treatment of depression can result in improved HIV outcomes. No randomized controlled trials of antidepressant medication treatment alone have shown improvements in virologic outcomes. Safren et al. ⁶³ studied the effect of individual cognitive behavioral therapy (CBT) among persons living with HIV and also diagnosed with a depressive mood disorder. The CBT intervention explicitly incorporated adherence training and improved ART adherence by more than 20% percentage points at 12 month follow up, but the small sample size limited the investigators’ ability to detect differences in viral load. Two randomized studies of group-based cognitive behavioral stress management for persons living with HIV have yielded mixed results, one positive ⁶⁴ and one negative ⁶⁵, but those studies enrolled participants with minimal depressive symptoms(i.e., mean BDI<14 at baseline). Our study is notable in that it suggests that antidepressant medication treatment can improve HIV care and HIV treatment outcomes among persons with significant depressive symptoms.

Also in contrast to these studies, the participants in the REACH cohort are drawn from a population whose frequently changing living situations and medical and psychiatric comorbidities can make controlled study difficult. All REACH participants were either homeless or marginally housed, approximately one-half reported alcohol or illicit drug use, and more than one-third had been assigned to representative payeeship. Due to these complex comorbidities, many of our study participants would have been excluded from most randomized controlled trials of antidepressant efficacy ^{66, 67}. The clinical and public health importance of our work is further underscored by nationally representative evidence of underdiagnosis ⁶⁸ and undertreatment ³² of depression among persons living with HIV/AIDS, as well as the fact that even incremental (e.g., 10%) increases in ART adherence can improve virologic ^{69, 70} and immunologic ⁷¹ outcomes in this population.

Despite these strengths, interpretation of our findings is subject to a number of limitations. Most participants in our study were female, which limits generalizability to the HIV epidemic in the United States ^{68, 72}. However, while not formally a random sample of HIV-infected, homeless and marginally housed persons, the parent cohort (REACH) was drawn from a systematic and reproducible venue-based sample ⁷³ of homeless and marginally housed persons living with HIV. The REACH cohort was comprised of mostly men with a high prevalence of drug use, alcohol use, and mental illness and is roughly generalizable to the HIV-infected urban poor ^{17, 18}. The preponderance of females in our analytic sample may reflect the overall epidemiology of major depressive disorder in the general population ^{3, 74}. Although our sample may not represent patients seen in most clinical settings, it does reflect a population that has variable access to medical and mental health care services and which remains an important part of the national HIV epidemic ^{75, 76}. A second limitation is that our statistical analyses group antidepressant medications together into a single category, implicitly assuming equivalent treatment effects across medication classes. However, there is recent meta-analytic evidence to support this simplifying assumption ^77–79. Lack of power prevented us from studying individual drugs, but we conducted a sensitivity analysis for the most frequently prescribed medication class in our study (SSRIs) and observed qualitatively similar effects on viral suppression. And finally, our data did not permit us to account for dose escalation. Drug metabolism and clearance varies widely between individuals, and psychiatrists frequently compensate for this pharmacokinetic variability by tailoring antidepressant medication dosage to individual patients’ responses. Our marginal structural model analysis can be conceptualized as analogous to a flexible randomized controlled trial in which subjects are randomized to receive antidepressant medication treatment (or not), but the drug and dose are left up to physician and patient discretion. As noted previously, marginal structural models require several assumptions. First, consistency implies that each participant’s potential outcome under her observed antidepressant medication exposure history is precisely her observed outcome ⁸⁰. Although consistency may be problematic when the exposure is a feature such as obesity, it is plausible (although not empirically verifiable) in observational studies of medical treatments. Second, with regards to positivity, or the experimental treatment assumption ¹⁴, there were no structural zeroes ⁴³ in the setting of our data, i.e., factors that would be deterministic of either treatment or non-treatment with antidepressant medication. We were able to identify exposed and unexposed participants at each level of depressive severity as measured by the BDI-II, thereby ruling out the presence of potential random zeroes. In addition, we fitted a regression model using all of the baseline covariates and the time-varying covariate to compute predicted probabilities of treatment. We then visually inspected a plot of the log odds of treatment against both the observed treatment and predicted probabilities of treatment to ensure that there was an acceptable degree of variation of observed values across all levels of the predicted ^{81, 82}. Third, we assumed that conditioning on several baseline covariates and recent values of depression severity was sufficient to achieve exchangeability between those who did and did not initiate treatment with antidepressant medication during the follow-up period ^{83, 84}. This is not an empirically verifiable assumption, but we relied on prior studies to guide our inclusion of the most important confounders. Furthermore, we included a broad range of other covariates in an exhaustive sensitivity analysis, and our findings remained robust to these alternate specifications. Nonetheless, some unmeasured confounding could remain, e.g., receipt of adherence counseling. Subjects who received adherence counseling may be more likely to initiate antidepressant medication treatment due to greater interaction with the care team and greater awareness of depression severity, and they may also be more likely to adhere to ART. And fourth, we made the conservative “intent-to-treat” assumption, long recognized as the preferred approach to analysis of data from randomized controlled trials ^{35, 36, 85}. Thus, we anticipate some bias towards the null in our treatment estimates. Participants in the study cohort remained on antidepressant medications an average of 67% (median 73%) of the time following treatment intiation, which compares favorably with completion rates observed in long-term (i.e., 6–8 months in duration) randomized controlled trials of SSRIs ⁸⁶ and is similar to completion rates observed in short-term trials of both SSRIs ⁸⁷ and tricyclic antidepressant medications ⁸⁸.

In summary, we introduced the method of marginal structural modeling to the psychiatric literature to estimate the causal effect of antidepressant medication treatment on viral suppression among a longitudinal cohort of homeless and marginally housed persons living with HIV. Antidepressant medication treatment resulted in a twofold greater probability of achieving viral suppression, and this effect was likely due to improved adherence along a continuum of HIV care. The estimated effects are clinically meaningful and(under certain assumptions) have a causal interpretation, yet randomized controlled trials are needed to conclude definitively that antidepressant medication increases viral suppression in this population. Given the relatively high prevalence of under diagnosed and undertreated depressive mood disorders among persons living with HIV, our findings suggest that improved diagnosis and treatment of depression may have an important contribution to improving HIV treatment outcomes.

Supplementary Material

Appendix Figure 1

Click here to view.^{(105K, tif)}

Appendix Table 1

Click here to view.^{(13K, docx)}

Appendix Table 2

Click here to view.^{(12K, docx)}

Acknowledgments

Funding Disclosure:

The REACH study was funded by U.S. National Institute of Mental Health (NIMH) R01 MH-054907. Members of the research team also acknowledge the following additional funding sources: NIMH Institutional Training Award R25 MH-060482 (Dr. Tsai), U.S. National Institutes of Health(NIH)/National Center for Research Resources UCSF-Clinical and Translational Science Institute Grant UL1 RR-024131 (Dr. Tsai), NIMH K23 MH-079713-01 (Dr. Weiser), and NIMH K24 MH-087227 (Dr. Bangsberg). HIV RNA kits were donated by Roche. The study contents are solely the responsibility of the authors and do not necessarily represent the official views of the sponsors.

The authors wish to thank the REACH participants who made this study possible by sharing their experiences, as well as the REACH staff who conducted the interviews. Dr. Robert Daroff provided helpful comments. While these individuals are acknowledged for their assistance, no endorsement of manuscript contents or conclusions should be inferred.

All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

References

1. Bing EG, Burnam MA, Longshore D, et al. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch Gen Psychiatry. 2001 Aug;58(8):721–728. [PubMed]

2. Hasin DS, Goodwin RD, Stinson FS, Grant BF. Epidemiology of major depressive disorder: results from the National Epidemiologic Survey on Alcoholism and Related Conditions. Arch Gen Psychiatry. 2005 Oct;62(10):1097–1106. [PubMed]

3. Kessler RC, Berglund P, Demler O, et al. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R) JAMA. 2003 Jun 18;289(23):3095–3105. [PubMed]

4. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005 Jun;62(6):617–627. [PMC free article] [PubMed]

5. Fairfield KM, Libman H, Davis RB, Eisenberg DM. Delays in protease inhibitor use in clinical practice. J Gen Intern Med. 1999 Jul;14(7):395–401. [PMC free article] [PubMed]

6. Tegger MK, Crane HM, Tapia KA, Uldall KK, Holte SE, Kitahata MM. The effect of mental illness, substance use, and treatment for depression on the initiation of highly active antiretroviral therapy among HIV-infected individuals. AIDS Patient Care STDS. 2008 Mar;22(3):233–243. [PubMed]

7. Chesney M. Adherence to HAART regimens. AIDS Patient Care STDS. 2003 Apr;17(4):169–177. [PubMed]

8. Horberg MA, Silverberg MJ, Hurley LB, et al. Effects of depression and selective serotonin reuptake inhibitor use on adherence to highly active antiretroviral therapy and on clinical outcomes in HIV-infected patients. J Acquir Immune Defic Syndr. 2008 Mar 1;47(3):384–390. [PubMed]

9. Kacanek D, Jacobson DL, Spiegelman D, Wanke C, Isaac R, Wilson IB. Incident depression symptoms are associated with poorer HAART adherence: a longitudinal analysis from the Nutrition for Healthy Living Study. J Acquir Immune Defic Syndr. 2009;00(0):0. [PMC free article] [PubMed]

10. Burack JH, Barrett DC, Stall RD, Chesney MA, Ekstrand ML, Coates TJ. Depressive symptoms and CD4 lymphocyte decline among HIV-infected men. JAMA. 1993 Dec 1;270(21):2568–2573. [PubMed]

11. Page-Shafer K, Delorenze GN, Satariano WA, Winkelstein W., Jr Comorbidity and survival in HIV-infected men in the San Francisco Men’s Health Survey. Ann Epidemiol. 1996 Sep;6(5):420–430. [PubMed]

12. Ferrando SJ, Freyberg Z. Treatment of depression in HIV positive individuals: a critical review. Int Rev Psychiatry. 2008 Feb;20(1):61–71. [PubMed]

13. Hernan MA, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology. 2000 Sep;11(5):561–570. [PubMed]

14. Robins JM, Hernan MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000 Sep;11(5):550–560. [PubMed]

15. Hernan MA, Brumback B, Robins JM. Marginal structural models to estimate the joint causal effect of nonrandomized treatments. J Am Stat Assoc. 2001;96(454):440–448.

16. Hernan MA, Brumback BA, Robins JM. Estimating the causal effect of zidovudine on CD4 count with a marginal structural model for repeated measures. Stat Med. 2002 Jun 30;21(12):1689–1709. [PubMed]

17. Zolopa AR, Hahn JA, Gorter R, et al. HIV and tuberculosis infection in San Francisco’s homeless adults. Prevalence and risk factors in a representative sample. JAMA. 1994 Aug 10;272(6):455–461. [PubMed]

18. Robertson MJ, Clark RA, Charlebois ED, et al. HIV seroprevalence among homeless and marginally housed adults in San Francisco. Am J Public Health. 2004 Jul;94(7):1207–1217. [PubMed]

19. Beck A, Steer R, Brown G. Manual for Beck Depression Inventory II (BDI-II) San Antonio, Texas: Psychology Corporation; 1996.

20. Hammer SM, Eron JJ, Jr, Reiss P, et al. Antiretroviral treatment of adult HIV infection: 2008 recommendations of the International AIDS Society-USA panel. JAMA. 2008 Aug 6;300(5):555–570. [PubMed]

21. Wells KB, Burnam MA, Rogers W, Hays R, Camp P. The course of depression in adult outpatients. Results from the Medical Outcomes Study. Arch Gen Psychiatry. 1992 Oct;49(10):788–794. [PubMed]

22. Judd LL, Akiskal HS, Zeller PJ, et al. Psychosocial disability during the long-term course of unipolar major depressive disorder. Arch Gen Psychiatry. 2000 Apr;57(4):375–380. [PubMed]

23. Judd LL, Akiskal HS, Maser JD, et al. A prospective 12-year study of subsyndromal and syndromal depressive symptoms in unipolar major depressive disorders. Arch Gen Psychiatry. 1998 Aug;55(8):694–700. [PubMed]

24. Chesney MA, Ickovics JR, Chambers DB, et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG) AIDS Care. 2000 Jun;12(3):255–266. [PubMed]

25. Moss AR, Hahn JA, Perry S, et al. Adherence to highly active antiretroviral therapy in the homeless population in San Francisco: a prospective study. Clin Infect Dis. 2004 Oct 15;39(8):1190–1198. [PubMed]

26. Urquhart J. Pharmionics: research on what patients do with prescription drugs. Pharmacoepidemiol Drug Saf. 2004 Sep;13(9):587–590. [PubMed]

27. Giordano TP, Suarez-Almazor ME, Grimes RM. The population effectiveness of highly active antiretroviral therapy: are good drugs good enough? Curr HIV/AIDS Rep. 2005 Nov;2(4):177–183. [PubMed]

28. Kushel MB, Colfax G, Ragland K, Heineman A, Palacio H, Bangsberg DR. Case management is associated with improved antiretroviral adherence and CD4+ cell counts in homeless and marginally housed individuals with HIV infection. Clin Infect Dis. 2006 Jul 15;43(2):234–242. [PubMed]

29. Choi HK, Hernan MA, Seeger JD, Robins JM, Wolfe F. Methotrexate and mortality in patients with rheumatoid arthritis: a prospective study. Lancet. 2002 Apr 6;359(9313):1173–1177. [PubMed]

30. Cook NR, Cole SR, Hennekens CH. Use of a marginal structural model to determine the effect of aspirin on cardiovascular mortality in the Physicians’ Health Study. Am J Epidemiol. 2002 Jun 1;155(11):1045–1053. [PubMed]

31. D’Agostino RB, Lee ML, Belanger AJ, Cupples LA, Anderson K, Kannel WB. Relation of pooled logistic regression to time dependent Cox regression analysis: the Framingham Heart Study. Stat Med. 1990 Dec;9(12):1501–1515. [PubMed]

32. Burnam MA, Bing EG, Morton SC, et al. Use of mental health and substance abuse treatment services among adults with HIV in the United States. Arch Gen Psychiatry. 2001 Aug;58(8):729–736. [PubMed]

33. Vitiello B, Burnam MA, Bing EG, Beckman R, Shapiro MF. Use of psychotropic medications among HIV-infected patients in the United States. Am J Psychiatry. 2003 Mar;160(3):547–554. [PubMed]

34. Hotopf M, Lewis G, Normand C. Putting trials on trial--the costs and consequences of small trials in depression: a systematic review of methodology. J Epidemiol Community Health. 1997 Aug;51(4):354–358. [PMC free article] [PubMed]

35. Bollini P, Pampallona S, Tibaldi G, Kupelnick B, Munizza C. Effectiveness of antidepressants. Meta-analysis of dose-effect relationships in randomised clinical trials. Br J Psychiatry. 1999 Apr;174:297–303. [PubMed]

36. Newell DJ. Intention-to-treat analysis: implications for quantitative and qualitative research. Int J Epidemiol. 1992 Oct;21(5):837–841. [PubMed]

37. Quitkin FM, Rabkin JG, Stewart JW, McGrath PJ, Harrison W. Study duration in antidepressant research: advantages of a 12-week trial. J Psychiatr Res. 1986;20(3):211–216. [PubMed]

38. Rush AJ, Fava M, Wisniewski SR, et al. Sequenced treatment alternatives to relieve depression (STAR*D): rationale and design. Control Clin Trials. 2004 Feb;25(1):119–142. [PubMed]

39. Froot KA. Consistent covariance matrix estimation with cross-sectional dependence and heteroskedasticity in financial data. J Fin Quant Analysis. 1989;24(3):333–355.

40. Williams RL. A note on robust variance estimation for cluster-correlated data. Biometrics. 2000;56(2):645–646. [PubMed]

41. Rogers WH. Regression standard errors in clustered samples. Stata Tech Bull. 1993;13:19–23.

42. Wooldridge JM. Econometric analysis of cross section and panel data. Cambridge, Massachusetts: MIT Press; 2002.

43. Cole SR, Hernan MA. Constructing inverse probability weights for marginal structural models. Am J Epidemiol. 2008 Sep 15;168(6):656–664. [PMC free article] [PubMed]

44. Khan A, Warner HA, Brown WA. Symptom reduction and suicide risk in patients treated with placebo in antidepressant clinical trials: an analysis of the Food and Drug Administration database. Arch Gen Psychiatry. 2000 Apr;57(4):311–317. [PubMed]

45. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med. 2008 Feb;5(2):e45. [PMC free article] [PubMed]

46. Fournier JC, DeRubeis RJ, Hollon SD, et al. Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA. 2010 Jan 6;303(1):47–53. [PubMed]

47. Kish L. Weighting for unequal p. J Off Stat. 1992;8(2):183–200.

48. Thase ME, Salloum IM, Cornelius JD. Comorbid alcoholism and depression: treatment issues. J Clin Psychiatry. 2001;62( Suppl 20):32–41. [PubMed]

49. Paterson DL, Swindells S, Mohr J, et al. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Intern Med. 2000 Jul 4;133(1):21–30. [PubMed]

50. Bangsberg DR. Less than 95% adherence to nonnucleoside reverse-transcriptase inhibitor therapy can lead to viral suppression. Clin Infect Dis. 2006 Oct 1;43(7):939–941. [PubMed]

51. Leserman J. HIV disease progression: depression, stress, and possible mechanisms. Biol Psychiatry. 2003 Aug 1;54(3):295–306. [PubMed]

52. Ironson G, O’Cleirigh C, Fletcher MA, et al. Psychosocial factors predict CD4 and viral load change in men and women with human immunodeficiency virus in the era of highly active antiretroviral treatment. Psychosom Med. 2005 Nov–Dec;67(6):1013–1021. [PMC free article] [PubMed]

53. Anastos K, Schneider MF, Gange SJ, et al. The association of race, sociodemographic, and behavioral characteristics with response to highly active antiretroviral therapy in women. J Acquir Immune Defic Syndr. 2005 Aug 15;39(5):537–544. [PubMed]

54. Cook JA, Grey D, Burke J, et al. Depressive symptoms and AIDS-related mortality among a multisite cohort of HIV-positive women. Am J Public Health. 2004 Jul;94(7):1133–1140. [PubMed]

55. Cruess DG, Douglas SD, Petitto JM, et al. Association of resolution of major depression with increased natural killer cell activity among HIV-seropositive women. Am J Psychiatry. 2005 Nov;162(11):2125–2130. [PubMed]

56. Letendre SL, Marquie-Beck J, Ellis RJ, et al. The role of cohort studies in drug development: clinical evidence of antiviral activity of serotonin reuptake inhibitors and HMG-CoA reductase inhibitors in the central nervous system. J Neuroimmune Pharmacol. 2007 Mar;2(1):120–127. [PubMed]

57. Ances BM, Letendre SL, Alexander T, Ellis RJ. Role of psychiatric medications as adjunct therapy in the treatment of HIV associated neurocognitive disorders. Int Rev Psychiatry. 2008 Feb;20(1):89–93. [PubMed]

58. Robins JM, Greenland S. Identifiability and exchangeability for direct and indirect effects. Epidemiology. 1992 Mar;3(2):143–155. [PubMed]

59. Pearl J. Direct and indirect effects. Paper presented at: 17th Conference on Uncertainty in Artificial Intelligence; 2001; San Francisco.

60. Cole SR, Hernan MA, Robins JM, et al. Effect of highly active antiretroviral therapy on time to acquired immunodeficiency syndrome or death using marginal structural models. Am J Epidemiol. 2003 Oct 1;158(7):687–694. [PubMed]

61. Sterne JA, Hernan MA, Ledergerber B, et al. Long-term effectiveness of potent antiretroviral therapy in preventing AIDS and death: a prospective cohort study. Lancet. 2005 Jul 30-Aug 5;366(9483):378–384. [PubMed]

62. Delaney JA, Daskalopoulou SS, Suissa S. Traditional versus marginal structural models to estimate the effectiveness of beta-blocker use on mortality after myocardial infarction. Pharmacoepidemiol Drug Saf. 2009 Jan;18(1):1–6. [PubMed]

63. Safren SA, O’Cleirigh C, Tan JY, et al. A randomized controlled trial of cognitive behavioral therapy for adherence and depression (CBT-AD) in HIV-infected individuals. Health Psychol. 2009 Jan;28(1):1–10. [PMC free article] [PubMed]

64. Ironson G, Weiss S, Lydston D, et al. The impact of improved self-efficacy on HIV viral load and distress in culturally diverse women living with AIDS: the SMART/EST Women’s Project. AIDS Care. 2005 Feb;17(2):222–236. [PubMed]

65. Antoni MH, Carrico AW, Duran RE, et al. Randomized clinical trial of cognitive behavioral stress management on human immunodeficiency virus viral load in gay men treated with highly active antiretroviral therapy. Psychosom Med. 2006 Jan-Feb;68(1):143–151. [PubMed]

66. Posternak MA, Zimmerman M, Keitner GI, Miller IW. A reevaluation of the exclusion criteria used in antidepressant efficacy trials. Am J Psychiatry. 2002 Feb;159(2):191–200. [PubMed]

67. Zimmerman M, Chelminski I, Posternak MA. Exclusion criteria used in antidepressant efficacy trials: consistency across studies and representativeness of samples included. J Nerv Ment Dis. 2004 Feb;192(2):87–94. [PubMed]

68. Asch SM, Kilbourne AM, Gifford AL, et al. Underdiagnosis of depression in HIV: who are we missing? J Gen Intern Med. 2003 Jun;18(6):450–460. [PMC free article] [PubMed]

69. Liu H, Miller LG, Hays RD, et al. Repeated measures longitudinal analyses of HIV virologic response as a function of percent adherence, dose timing, genotypic sensitivity, and other factors. J Acquir Immune Defic Syndr. 2006 Mar;41(3):315–322. [PubMed]

70. Petersen ML, Wang Y, van der Laan MJ, Guzman D, Riley E, Bangsberg DR. Pillbox organizers are associated with improved adherence to HIV antiretroviral therapy and viral suppression: a marginal structural model analysis. Clin Infect Dis. 2007 Oct 1;45(7):908–915. [PMC free article] [PubMed]

71. Bangsberg DR, Perry S, Charlebois ED, et al. Non-adherence to highly active antiretroviral therapy predicts progression to AIDS. AIDS. 2001 Jun 15;15(9):1181–1183. [PubMed]

72. Bozzette SA, Berry SH, Duan N, et al. The care of HIV-infected adults in the United States. HIV Cost and Services Utilization Study Consortium. N Engl J Med. 1998 Dec 24;339(26):1897–1904. [PubMed]

73. Burnam MA, Koegel P. Methodology for obtaining a representative sample of homeless persons: the Los Angeles Skid Row Study. Eval Rev. 1988;12(2):117–152.

74. Hirschfeld RM, Cross CK. Epidemiology of affective disorders. Arch Gen Psychiatry. 1982 Jan;39(1):35–46. [PubMed]

75. Weiser SD, Wolfe WR, Bangsberg DR. The HIV epidemic among individuals with mental illness in the United States. Curr HIV/AIDS Rep. 2004 Dec;1(4):186–192. [PubMed]

76. Fischer PJ, Breakey WR. The epidemiology of alcohol, drug, and mental disorders among homeless persons. Am Psychol. 1991 Nov;46(11):1115–1128. [PubMed]

77. MacGillivray S, Arroll B, Hatcher S, et al. Efficacy and tolerability of selective serotonin reuptake inhibitors compared with tricyclic antidepressants in depression treated in primary care: systematic review and meta-analysis. BMJ. 2003 May 10;326(7397):1014. [PMC free article] [PubMed]

78. Hansen RA, Gartlehner G, Lohr KN, Gaynes BN, Carey TS. Efficacy and safety of second-generation antidepressants in the treatment of major depressive disorder. Ann Intern Med. 2005 Sep 20;143(6):415–426. [PubMed]

79. Gartlehner G, Gaynes BN, Hansen RA, et al. Comparative benefits and harms of second-generation antidepressants: background paper for the American College of Physicians. Ann Intern Med. 2008 Nov 18;149(10):734–750. [PubMed]

80. Cole SR, Frangakis CE. The consistency statement in causal inference: a definition or an assumption? Epidemiology. 2009 Jan;20(1):3–5. [PubMed]

81. Mortimer KM, Neugebauer R, van der Laan M, Tager IB. An application of model-fitting procedures for marginal structural models. Am J Epidemiol. 2005 Aug 15;162(4):382–388. [PubMed]

82. Tager IB, Haight T, Sternfeld B, Yu Z, van Der Laan M. Effects of physical activity and body composition on functional limitation in the elderly: application of the marginal structural model. Epidemiology. 2004 Jul;15(4):479–493. [PubMed]

83. Robins JM. A new approach to causal inference in mortality studies with a sustained exposure period --application to control of the healthy worker survivor effect. Math Modelling. 1986;7(9–12):1393–1512.

84. Robins JM. Addendum to ‘A new approach to causal inference in mortality studies with a sustained exposure period --application to control of the healthy worker survivor effect’ Comput Math Applic. 1987;14(9–12):923–945.

85. Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer. 1976 Dec;34(6):585–612. [PMC free article] [PubMed]

86. Deshauer D, Moher D, Fergusson D, Moher E, Sampson M, Grimshaw J. Selective serotonin reuptake inhibitors for unipolar depression: a systematic review of classic long-term randomized controlled trials. CMAJ. 2008 May 6;178(10):1293–1301. [PMC free article] [PubMed]

87. Gartlehner G, Hansen RA, Carey TS, Lohr KN, Gaynes BN, Randolph LC. Discontinuation rates for selective serotonin reuptake inhibitors and other second-generation antidepressants in outpatients with major depressive disorder: a systematic review and meta-analysis. Int Clin Psychopharmacol. 2005;20(2):59–69. [PubMed]

88. Storosum JG, Elferink AJ, van Zwieten BJ, et al. Short-term efficacy of tricyclic antidepressants revisited: a meta-analytic study. Eur Neuropsychopharmacol. 2001 Apr;11(2):173–180. [PubMed]