We used data from a large national cohort of VHA patients receiving depression treatment to complete the first comprehensive assessment of monitoring visits and suicide deaths. We show that in this observational dataset, patient-level characteristics associated with higher risks of suicide (Spettell et al. 2003
) were also associated with higher levels of monitoring, suggesting there are likely substantial treatment selection biases when assessing the relationship between suicide and clinical monitoring.
We used two analytic approaches to examine the link between monitoring and suicide while addressing these treatment selection biases, the traditional case–control approach and an IV approach—and we found that these analytic approaches resulted in different conclusions.
Although widely used, traditional case–control analyses with the covariate data available in large administrative databases appeared unsuccessful in addressing treatment selection biases. Case–control analyses indicated that visits with clinicians were significantly associated with a slight increase in suicide deaths among patients during high-risk periods. This would seem an unlikely scenario unless one believes that clinical contacts increase patient distress or impulsivity. Instead, this finding is likely due to residual treatment indication biases because of insufficient patient information in administrative data to adjust for factors associated with both monitoring and suicide.
Although some IV analyses based on facility of most use suggested no impact of monitoring on suicide risk, several IV analyses, particularly those using the distance-weighted IV, produced results more in line with expert opinion, suggesting that increased monitoring may lead to reduced suicide risks. However, none of the IV results were statistically significant.
IV estimates may be less subject to treatment indication biases than either the naïve single equation estimate or the case–control study estimates, and they may more closely address policy questions of whether to increase or decrease monitoring for defined populations during high-risk periods. IV analyses assess the marginal
impact of increases in monitoring visits—the impact of increased monitoring for individuals who receive this closer monitoring only because of differences in facility practices (Newhouse and McClellan 1998
). Some patients, because of high-risk behaviors, will receive high levels of monitoring regardless of usual facility practices, and IV estimates do not reflect the benefit these individuals receive from close monitoring. However, because IV approaches focus only on treatment variation that can be explained by the IV, these analyses tend to produce larger standard errors than more direct methods. Likely due to this imprecision and low base rate of completed suicide, IV analyses did not provide a definitive answer regarding the relationship between closer monitoring and suicide despite the use of a large administrative database (Sturm 1998
We note that while insignificant findings in IV analyses might be due to residual bias or imprecision, it is also possible that clinical visits as currently practiced are not effective in reducing suicide (though they may be effective in addressing other concerns). Routine clinical visits may fail to include a careful assessment of suicidal ideation, plans, or access to lethal means. Alternatively, suicidal ideation or other clinical indicators of risk may not be present during routinely scheduled visits. Suicide attempts are often impulsive and are frequently planned for <30 minutes before being enacted (Simon et al. 2001
). Therefore, even large increases in monitoring (e.g., more than four contacts) over the course of 84 days may not be sufficient to detect these short periods of acute risk. Finally to date, even if acute suicide risk is detected during a visit, few clinical interventions have been shown to be effective in reducing these risks (Mann et al. 2005
). These possibilities, in combination with our lack of a robust finding for a protective effect, highlight a need to further refine and evaluate additional clinical approaches for reducing suicide risks.
We have reported previously that increasing monitoring from current levels to FDA suggested levels would mean a substantial reorganization of health services along with substantial incremental costs (Valenstein et al. 2009
). As noted previously, RCTs of sufficient size to demonstrate a clear link between monitoring and suicide deaths are impractical, and we now show that using observational data (in which clinicians are following individuals they deem at greater risk more closely) to demonstrate a clear link
between higher levels of monitoring and reduced suicide mortality is also difficult—even when using case–control and IV analyses to address treatment biases. Given the difficulty in demonstrating this link, we believe health care organizations and clinicians will remain unwilling to change current behaviors and press forward with implementing broad policy recommendations regarding monitoring—unless a public consensus develops that these activities should proceed without firm evidence. Indeed, because of the lack of evidence for effectiveness and the large investment that would be required to implement a blanket policy of close monitoring during high-risk periods for all patients, an argument could be made that treatment resources might be better used for mental health interventions with stronger evidence for effectiveness. Future research using large datasets with more detailed information on potential confounders and the development of new methodologies to address treatment selection biases in observational data are clearly needed.
Diagnoses, demographics, and cause of death may not be completely accurate in administrative databases. However, VHA administrative data quality is generally considered good, with high levels of concordance between VHA administrative data and medical record data (Kashner 1998
; Cowper et al. 1999
;). The NDI is also considered the “gold standard” of U.S. mortality databases (Cowper et al. 2002
). Findings for VHA patients also may not generalize to non-VHA patient populations, and findings within the VHA may change as greater numbers of younger veterans enter the health care system following their return from Iraq or Afghanistan.
In summary, although expert and governmental recommendations have urged closer monitoring for depressed patients during high-risk periods to prevent suicide, strong evidence for this recommendation may be difficult to generate. When using observational datasets to address this issue, IV analyses appear less biased than case–control approaches; however, even with a very large database, the application of the IV method appears limited due to low rate of suicide and the resulting increased variability of its estimate, making it difficult to arrive at definite answers regarding this relationship.