PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Affect Disord. Author manuscript; available in PMC 2017 March 15.
Published in final edited form as:
PMCID: PMC4744514
NIHMSID: NIHMS748871

Identifying a clinical signature of suicidality among patients with mood disorders: a pilot study using a machine learning approach

Ives Cavalcante Passos, MD,1,2,* Benson Mwangi, PhD,1,* Bo Cao, PhD,1 Jane E Hamilton, PhD,1 Mon-Ju Wu, PhD,1 Xiang Yang Zhang, MD, PhD,1,3 Giovana B. Zunta-Soares, MD,1 Joao Quevedo, MD, PhD,1 Marcia Kauer-Sant'Anna, MD, PhD,2 Flávio Kapczinski, MD, PhD,2 and Jair C. Soares, MD, PhD1

Abstract

Objective

A growing body of evidence has put forward clinical risk factors associated with patients with mood disorders that attempt suicide. However, what is not known is how to integrate clinical variables into a clinically useful tool in order to estimate the probability of an individual patient attempting suicide.

Method

A total of 144 patients with mood disorders were included. Clinical variables associated with suicide attempts among patients with mood disorders and demographic variables were used to ‘train’ a machine learning algorithm. The resulting algorithm was utilized in identifying novel or ‘unseen’ individual subjects as either suicide attempters or non-attempters. Three machine learning algorithms were implemented and evaluated.

Results

All algorithms distinguished individual suicide attempters from non-attempters with prediction accuracy ranging between 65%-72% (p<0.05). In particular, the relevance vector machine (RVM) algorithm correctly predicted 103 out of 144 subjects translating into 72% accuracy (72.1% sensitivity and 71.3% specificity) and an area under the curve of 0.77 (p<0.0001). The most relevant predictor variables in distinguishing attempters from non-attempters included previous hospitalizations for depression, a history of psychosis, cocaine dependence and post-traumatic stress disorder (PTSD) comorbidity.

Conclusion

Risk for suicide attempt among patients with mood disorders can be estimated at an individual subject level by incorporating both demographic and clinical variables. Future studies should examine the performance of this model in other populations and its subsequent utility in facilitating selection of interventions to prevent suicide.

Keywords: suicide, bipolar disorder, depression, machine learning, big data, personalized medicine

Introduction

When a patient with a depressive episode and no known previous history of suicide attempts first presents for treatment, what is the likelihood that this individual patient will attempt suicide? This question is a major unmet challenge in mood disorder treatment and also an important question for society at large, as suicide is a tragic but highly preventable event (Mann et al., 2005). Suicide accounted for 4.8% and 5.7% of total global deaths in female and male subjects aged 15-49 years respectively in 2010 (Lozano et al., 2012). In the same year, suicide was reported as the sixth leading cause of life years lost in North-America (Lozano et al., 2012). Furthermore, it has been reported that approximately 90% of individuals who die through suicide are diagnosed with a mental disorder prior to their death (Arsenault-Lapierre et al., 2004). However, there is little awareness in medical practice on objective stratification of suicide risk, which has led suicide to be referred to as “the quiet epidemic” (Turecki, 2014). The current situation is particularly worrisome in major depressive disorder (MDD) and bipolar disorder (BD), given the high prevalence of these disorders and the strong association between suicide and depressive symptoms.

Suicide is usually viewed as an extreme response to a catastrophic event, such as loss of a close relative (M A Oquendo et al., 2014). However, many individuals, including patients with mood disorders, go through these kinds of stressors and yet they do not attempt suicide (Maria A Oquendo et al., 2014). Consequently, a growing body of knowledge has put forward several risk factors associated with patients that attempt suicide (Brundin et al., 2015; Mann and Currier, 2010; Maria A Oquendo et al., 2014; Turecki, 2014). They are developmental and distal factors (as opposed to precipitating factors, such as loss of a close relative) associated to the suicidal behavior (Turecki, 2014; Turecki et al., 2012). These efforts have largely reported average group-level differences between suicide attempters and non-attempters. However, what is not known is how to integrate these variables to build a signature of suicide attempt and estimate the probability of an individual patient with mood disorder attempting suicide while faced with a stressful event.

In this study we set out to investigate whether a predictive clinical signature derived from easily accessible clinical and demographic variables can objectively identify individual patients likely to attempt suicide. Predictor variables were selected using a priori knowledge. Notably, the majority of clinical variables selected were related to psychiatric comorbidities (Almeida et al., 2012; Bhui et al., 2012; Blackmore et al., 2008; Blanco et al., 2012; Bostwick and Pankratz, 2000; Bottlender et al., 2000; de Araújo et al., 2015; Foley et al., 2006; Galfalvy et al., 2006; Goes et al., 2012; Goldstein et al., 2012; Gonzalez-Pinto et al., 2006; Holma et al., 2014; Isometsä et al., 2014; Johnson et al., 1991; Katz et al., 2011; Lenze et al., 2000; Morina et al., 2013; Neves et al., 2009; Oquendo et al., 2010; Schaffer et al., 2014a, 2014b; Simon et al., 2004; Soloff et al., 2000; Stein, 2006; Torres et al., 2011; Webb et al., 2014), given the recent findings that effects of mental disorders on the risk of suicide attempt were exerted almost exclusively through a general psychopathology factor representing the shared effect across all mental disorders (Hoertel et al., 2015). As a result, we implemented a set of algorithms that are able to integrate the information from multiple variables to subsequently identify an individual patient's probability or risk of being a suicide attempter.

Machine learning in psychiatric research is an emerging field with a great potential for innovation and paradigm shift as these algorithms facilitate integration of multiple measurements as well as allows objective predictions of previously ‘unseen’ observations. Typically, these algorithms are implemented in three key stages. First, subjects' data are separated into algorithm ‘training’ and ‘testing’ sets with the former being used to ‘train’ a machine learning model. Second, the predictive accuracy of the ‘learnt’ model is evaluated using the ‘testing’ set that consists of observations previously ‘unseen’ by the model, and algorithm accuracy, sensitivity and specificity reported. In this study, we implemented three machine learning algorithms namely; least absolute shrinkage and selection operator (LASSO) (Robert Tibshirani, 2011), support vector machines (SVM) (Vapnik, 1998) and relevance vector machine (RVM) (Tipping, 2001). The LASSO technique was implemented using a generalized linear model (GLM) with a LASSO regularization penalty. A detailed discussion of these algorithms in the context of psychiatric research is given elsewhere (Mwangi et al., 2012).

In summary, the main objective of the current study was to establish a clinically useful predictive signature able to accurately determine individual patients' likely to committ suicide. We used easily accessible clinical variables to achieve our aim. Of note, this is a proof-of-concept study due to a small sample size.

Methods

The Institutional Review Board of the University of Texas Health Science Center at Houston approved the study. All subjects signed informed consent before any study-related procedures with ample time for questions. Participants included in the current study were recruited from January 2006 to June 2010.

Participants

A total of 144 subjects were recruited from the community and psychiatric clinics through flyers, radio, and newspaper advertisements. Inclusion criteria were subjects with MDD or BD types I or II (DSM-IV), and aged between 18 and 65. Exclusion criteria were head trauma with residual effects, neurological disorder, and uncontrolled major medical conditions.

Assessments

Subjects were evaluated using a socio-demographic history form to assess age, gender, years of education, and occupational status. Axis-I diagnoses and clinical characteristics were assessed with the Structured Clinical Interview for DSM-IV axis-I Disorders (SCID-I), which was administered by fully trained staff. The criterion for suicidality was one or more documented actual suicide attempts assessed by the suicide history form from the Conte Center for the Neuroscience of Mental Disorders (New York State Psychiatric Institute, Columbia University). Current dimensional mood and anxiety symptoms were assessed with the Hamilton Depression Rating Scale (HDRS), the Young Mania Rating Scale (YMRS), and Hamilton Anxiety Rating Scale (HARS). Data from all instruments were collected regardless of treatment status.

Selection of predictor variables and machine learning

Selection of predictor variables to be utilized in ‘training’ an algorithm is a challenge in machine learning. However, a recommended method of selecting relevant predictor variables is the use of expert domain knowledge – largely from previously published literature (Mwangi et al., 2013; Perlis, 2013). We performed a structured search on Pubmed to find relevant studies that report association of clinical and demographic variables included in this study with suicide attempts (Table 1). Of note, we chose patients with mood disorders because a) the developmental variables associated with suicide in both BD and MDD are somewhat similar (see table 1); and b) our goal was to create a tool that could be used with the large number of patients with mood disorders treated in primary care settings (Table 1).

Table 1
Clinical and demographic variables selected to develop to ‘train’ the model

Three machine learning algorithms (LASSO, SVM, and RVM) were implemented in MATLAB (The Mathworks, Inc., Natick, Massachusetts, United States). Specifically, predictor variables (see Table 1) were normalized by z-scoring and together with corresponding categorical labels (1 – suicide attempt, 0 – non-attempt) input into the machine learning algorithm. Here, we briefly describe these machine learning algorithms but detailed and technical discussions are given elsewhere (Robert Tibshirani, 2011; Tipping, 2001; Vapnik, 1998). SVM and RVM are ‘kernel-learning’ algorithms which transform a set of input variables (feature vectors) into a ‘kernel’ or similarity matrix through a ‘kernel function’ and subsequently used in the algorithm ‘learning’ process. These algorithms can utilize multiple types of kernel functions (e.g. linear, polynomial, Gaussian) but in the current study a linear function was utilized in training both SVM and RVM algorithms as described elsewhere (Mwangi et al., 2012). In contrast, the LASSO utilizes a typical linear regression formulation but introduces a ‘penalization’ procedure by assigning some predictor variables zero coefficients or weighting factors - a process also referred to as regularization. The penalization or regularization process reportedly improves the linear regression algorithm prediction accuracy and generalization ability but entails identifying an optimal regularization parameter (Robert Tibshirani, 2011). In the current study, the regularization parameter was identified through a 10-fold cross-validation process using training data only.

Class imbalance problem

Class imbalance is a common problem in machine learning where observations (subjects) in one class (e.g. suicide non-attempters) exceed observations in the other class (e.g. suicide attempters). A typical machine learning algorithm trained using an imbalanced data set assigns new observations to the majority class (e.g. suicide non-attempters) (Dubey et al., 2014). In this study, the class imbalance problem was circumvented by ‘under-sampling’ the majority class (suicide non-attempters) followed by training an algorithm with a balanced sample – a process which was repeated until all observations in the majority class were selected at least once and predictions aggregated as shown in Figure 1.

Figure 1
Flow diagram showing algorithm training and testing process

Model testing and validation

A unique characteristic of machine learning algorithms is the ability to utilize a cross-validation approach to separate data into model ‘training’ and ‘testing’ sets. In this study, a leave-one-out cross-validation (LOOCV) approach was used. LOOCV involves training a machine learning algorithm with all subjects except one while the ‘left out’ subject is used for algorithm testing. This iterative process is repeated until all subjects are left out for algorithm testing at least once as shown in Figure 1 and described elsewhere (Johnston et al., 2014). Cross-validation is used in machine learning to establish the generalization ability of an algorithm to new or previously ‘unseen’ subjects. The validity of the algorithms in predicting individual suicide attempters from non-attempters was evaluated using prediction accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). In this study, sensitivity and specificity represented correctly predicted suicide attempters (true positives) and correctly predicted non-attempters (true negatives) respectively. A ‘classical’ prediction accuracy was calculated as the sum of true positives, true negatives and divided by the total sample. In contrast, to account for the class imbalance problem, a ‘balanced’ accuracy was calculated as the average of predicted sensitivity (true positives) and specificity (true negatives) as reported elsewhere (Feis et al., 2013). In addition, PPV was calculated as the proportion of subjects predicted as suicide attempters that were actual positives (attempters) and NPV as the proportion of subjects predicted as non-attempters but were actual non-attempters (true negatives). Receiver operating characteristic curves and the corresponding area under curve were computed and reported. Lastly, chi-square statistical test between actual and machine learning predicted labels were calculated and the results reported. The outcome of a chi-square test was considered significant at p<0.05.

Results

A total of 144 subjects with BD or MDD were included in the study. Table 2 summarizes subjects' clinical and demographic characteristics. All algorithms (RVM, SVM and LASSO) distinguished individual suicide attempters from non-attempters with prediction accuracy ranging (64.7%-72%) and all models were significant (chi-square p<0.05) as detailed in Table 3. In particular, the RVM algorithm correctly identified 103 out of 144 as either attempters or non-attempters. This translated into 72% accuracy (72.1% sensitivity and 71.3% specificity) and significant at χ2 p<0.0001. The RVM receiver operating characteristic (ROC) curve and the ‘confusion matrix’, which were used in calculating the sensitivity, specificity and area under ROC curve values, are shown in Figure 2. To calculate ‘concentration of risk’ – RVM predicted subjects' probabilities were divided into 20 groups of equal sizes (ventiles) and number of subjects correctly predicted within the top two ventiles calculated. In both top two ventiles, 5 out of 7 subjects were correctly predicted – translating into 71.4 % accuracy in both ventiles. Individual subjects' predicted probabilities are shown in Table S1 of supplementary materials.

Figure 2
‘Confusion matrix’ and receiver operating characteristic (ROC) curve
Table 2
Demographic and clinical characteristics
Table 3
Algorithm performance in identifying individual suicide attempters from non-attempters among patients with mood disorders. The balanced accuracy (sensitivity + specificity/2) was used to identify the best performing algorithm due to the class imbalance ...

The most relevant predictor variables in distinguishing suicide attempters from non-attempters that were assigned positive coefficients or weighting factors by RVM – indicating an increase in suicide attempters included; 1) a high number of previous hospitalizations for depression, 2) a history of psychosis, 3) cocaine dependence, and 4) PTSD. In contrast, patients’ age and mood diagnosis (0 - bipolar disorder or 1 - unipolar depression) were assigned negative weighting factors indicating a reduction of these variables was relevant in identifying suicide attempters. Figure 3 shows predictor variables with corresponding weighting factors.

Figure 3
Clinical signature

An additional RVM algorithm was evaluated by removing previous number of hospitalizations in the analysis and the algorithm was able to identify individual subjects with accuracy = 68.9%, sensitivity = 74%, specificity = 63%, p<0.05 and area under ROC curve =0.71. This is an indication that past hospitalization variable was not the only contributor to the algorithm performance.

Discussion

Although risk factors for suicide are widely known, synthesizing easily accessible clinical information to optimize suicide risk identification in subjects with mood disorders has been an elusive goal up to now. This is the first study to evaluate the feasibility of using a clinical tool developed with advanced machine learning algorithms to identifying an individual's risk of being a suicide attempter among patients with mood disorders. All algorithms achieved greater than chance (>50%) accuracy in distinguishing attempters from non-attempters. Most notably, the RVM algorithm achieved the best prediction accuracy 72% and an area under receiver operating characteristic curve of 0.77. Notably, the prediction accuracy, sensitivity and specificity of the machine learning algorithm in identifying previously ‘unseen’ suicide attempters was determined through a robust cross-validation approach. The main outcome of the machine learning algorithm was a probability score able to quantify the risk of a subject being a suicide attempter.

Most relevant predictor variables distinguishing suicide attempters from non-attempters by RVM included number of prior hospitalizations for depression, lifetime psychotic symptoms, and comorbidity with cocaine dependence or PTSD. Previous average group level studies showed that these variables are associated with suicide attempts in patients with BD. In this sense, a prospective study with 72 bipolar I disorder patients showed that suicidal risk was higher with more hospitalization (Gonzalez-Pinto et al., 2006), while two studies showed that psychotic symptoms were associated with suicidality in patients with MDD and BD (Bottlender et al., 2000; Johnson et al., 1991). In addition, a cross-sectional study with 500 patients with BD reported that patients with BD and comorbid PTSD presented more suicide attempts compared against patients with BD and without PTSD (Simon et al., 2004). Finally, a review suggested that there is cross-sensitization between cocaine use and number of mood episodes in patients with bipolar disorder leading to faster illness progression, which may potentially increase the number of suicide attempts (Post and Kalivas, 2013).

In contrast with other branches of medicine (e.g. cardiology, oncology) – the lack of risk stratification approaches in psychiatric clinical practice is notable (Kapczinski and Passos, 2015). Noteworthy risk stratification applications include, prediction of cardiac arrest risk, cancer screening, and prediction of mortality risk in intensive care units (Anothaisintawee et al., 2012; Lee et al., 1999). Indeed, in personalized medicine, risk stratification plays an important role in treatment planning from the time of diagnosis. For instance, in acute care of myocardial infarction efforts to develop treatment algorithms on the basis of clinical presentation are increasingly integrated in clinical practice (5).

One important question is how this suicide risk stratification tool can be applied in a clinical practice to optimize individual patient treatment? For example, it is possible that individual patients with a high risk of attempting suicide as identified by the algorithm would benefit from preventive strategies or antisuicidal drugs. For instance, a randomized clinical trial of suicidal patients reported that a selective serotonin-reuptake inhibitor led to a greater acute improvement in suicidal ideation compared to a noradrenergic-dopaminergic drug. Notably, this therapeutic advantage was greater in patients with the most severe suicidal ideation (Grunebaum et al., 2012). Furthermore, evidence from meta-analysis reported that lithium reduces suicide attempts (Cipriani et al., 2013). In addition, clozapine recently emerged as one option for treatment-resistant patients with BD and also showed a protective effect against suicidal behavior compared with olanzapine (Meltzer et al., 2003). These studies further highlight the need to develop novel suicide stratification algorithms as presented in this study and allow individualized treatments and care for patients predicted with a high risk of attempting suicide.

The current study has some potential limitations. The study sample is small and fromthe Texas region which may not fully represent the entire United States population. This mayexplain the reason why comorbid generalized anxiety disorder (GAD) and obsessive-compulsive disorder (OCD) findings were not fully in agreement with previous studies.However, multivariate machine learning techniques may have found unexpected interactionsamong predictive variables and therefore unexpected results on GAD and OCD. Therefore,future studies should confirm or reject these results. In this sense, the current study serves as aproof-of-concept and future longitudinal studies should collect larger samples from multiplecenters to replicate current findings. In line with our hypothesis, our findings are related topatients with mood disorder and may not be generalized to patient with other mentaldisorders. However, future studies may utilize a similar approach to accommodate othermental disorders. Of note, borderline personality disorder that is also shown to be associatedwith suicide attempts in patients with mood disorders (Schaffer et al., 2014a) was notincluded in the algorithm due to missing data in our database. Notwithstanding theselimitations, this study is the first to describe and implement a risk stratification tool usingmachine learning algorithms to identify an individual's probability of attempting suicideamong patients with mood disorders. This algorithm could be used in a primary care settingby general practitioners to identify the risk of a mood disorder patient attempting suicide.Future work will focus on validating the algorithm using a larger sample as well asdeveloping a web-based suicide risk calculator. Importantly, the strengths of our studyinclude the application and testing of a multivariate machine learning algorithm in suicidestratification leading to a high area under the ROC curve of 0.77. Notably, a recent studyreported that most breast cancer prediction algorithms, including the widely-studied ‘Gailscore’ report area under ROC curves below .70 (Anothaisintawee et al., 2012). Indeed, machine learning is a fascinating field that may change the traditional doctor-patient relationship in psychiatry. We hypothesize that highly accurate predictive models will support important clinical decisions such as selection of treatment options, preventive strategies, and prognosis orientations. For instance, several studies recently utilized machine learning techniques to predict treatment response using functional brain scans and neurocognitive data (Hahn et al., 2015; Johnston et al., 2015; van Waarde et al., 2015). Moreover, a recent study reported a prediction model able to estimate the risk of renal failure in patients with BD treated with lithium (Castro et al., 2015). Therefore, future studies should develop web-based calculators and medical devices based on validated models to allow translation of these predictive models into clinical practice.

In summary, we report a highly accurate algorithm able to identify suicide attempts in patients with mood disorders using clinical and demographic data. These results suggest that it is possible to utilize clinical measures in identifying individual patients at greater risk of attempting suicide. Lastly, we hypothesize that the combination of genetics, blood markers, neuroimaging, and the clinical signature together with a machine learning algorithm could generate an even more accurate multi-modal signature.

Highlights

  • We estimated risk for suicide attempt among patients with mood disorders
  • Prior hospitalizations, history of psychosis, cocaine use and PTSD comorbidities were the most relevant predictor variables
  • Risk for suicide attempt can be estimated at an individual subject level
  • Risk for suicide attempt can be estimated using demographic and clinical variables

Supplementary Material

supplement

Acknowledgments

Our study was supported in part by grants NIH grant MH68766, NIMH grant R01 085667, NIH grant RR2057, and the John S. Dunn Foundation from United States. Dr Jair C. Soares received these grants. ICP was supported by scholarship from Coordenacao de Aperfeicoamento de Pessoal de Nivel Superior (CAPES) and JQ, MKS, and FK were supported by Conselho Nacional de Desenvolvimento (CNPq).

Role of Funding Source: The funders had no role in the design, data collection, data management, data analysis and data interpretation.

Footnotes

Contributors: Dr Passos participated in the study design, data collection, data analysis, interpretation of findings, literature search, writing, implementation and approval of final manuscript. Dr Mwangi participated in the study design, writing, figures, tables, and approval of final manuscript. Dr Cao participated in the study design, literature search, figures, tables, and approval of final manuscript. Dr Hamilton participated in the literature search, writing, and approval of final manuscript. Dr Wu participated in literature search, figures, tables, and approval of final manuscript. Dr Zhang participated in the literature search, writing and approval of final manuscript. Dr Zunta-Soares participated in literature search, writing, and approval of final manuscript. Dr Quevedo participated in the study design, interpretation of findings, writing, and approval of final manuscript. Dr Kauer-Sant'Anna participated in the study design, interpretation of findings, writing, and approval of final manuscript. Dr Kapczinski participated in the study design, data analysis, interpretation of findings, literature search, writing, and approval of final manuscript. Dr Soares participated in the study design, data analysis, interpretation of findings, literature search, writing, and approval of final manuscript.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • Almeida OP, Draper B, Snowdon J, Lautenschlager NT, Pirkis J, Byrne G, Sim M, Stocks N, Flicker L, Pfaff JJ. Factors associated with suicidal thoughts in a large community study of older adults. Br J Psychiatry. 2012;201:466–72. doi: 10.1192/bjp.bp.112.110130. [PubMed] [Cross Ref]
  • Anothaisintawee T, Teerawattananon Y, Wiratkapun C, Kasamesup V, Thakkinstian A. Risk prediction models of breast cancer: a systematic review of model performances. Breast Cancer Res Treat. 2012;133:1–10. doi: 10.1007/s10549-011-1853-z. [PubMed] [Cross Ref]
  • Arsenault-Lapierre G, Kim C, Turecki G. Psychiatric diagnoses in 3275 suicides: a meta-analysis. BMC Psychiatry. 2004;4:37. doi: 10.1186/1471-244X-4-37. [PMC free article] [PubMed] [Cross Ref]
  • Bhui KS, Dinos S, McKenzie K. Ethnicity and its influence on suicide rates and risk. Ethn Health. 2012;17:141–8. doi: 10.1080/13557858.2011.645151. [PubMed] [Cross Ref]
  • Blackmore ER, Munce S, Weller I, Zagorski B, Stansfeld SA, Stewart DE, Caine ED, Conwell Y. Psychosocial and clinical correlates of suicidal acts: results from a national population survey. Br J Psychiatry. 2008;192:279–84. doi: 10.1192/bjp.bp.107.037382. [PubMed] [Cross Ref]
  • Blanco C, Alegría AA, Liu SM, Secades-Villa R, Sugaya L, Davies C, Nunes EV. Differences among major depressive disorder with and without co-occurring substance use disorders and substance-induced depressive disorder: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2012;73:865–73. doi: 10.4088/JCP.10m06673. [PubMed] [Cross Ref]
  • Bostwick JM, Pankratz VS. Affective disorders and suicide risk: a reexamination. Am J Psychiatry. 2000;157:1925–32. [PubMed]
  • Bottlender R, Jäger M, Strauss A, Möller HJ. Suicidality in bipolar compared to unipolar depressed inpatients. Eur Arch Psychiatry Clin Neurosci. 2000;250:257–61. [PubMed]
  • Brundin L, Erhardt S, Bryleva EY, Achtyes ED, Postolache TT. The role of inflammation in suicidal behaviour. Acta Psychiatr Scand. 2015;132:192–203. doi: 10.1111/acps.12458. [PMC free article] [PubMed] [Cross Ref]
  • Castro VM, Roberson AM, McCoy TH, Wiste A, Cagan A, Smoller JW, Rosenbaum JF, Ostacher M, Perlis RH. Stratifying Risk for Renal Insufficiency Among Lithium-Treated Patients: An Electronic Health Record Study. Neuropsychopharmacology. 2015 doi: 10.1038/npp.2015.254. [PMC free article] [PubMed] [Cross Ref]
  • Cipriani A, Hawton K, Stockton S, Geddes JR. Lithium in the prevention of suicide in mood disorders: updated systematic review and meta-analysis. BMJ. 2013;346:f3646. [PubMed]
  • De Araújo RMF, Mazzochi L, Lara DR, Ottoni GL. Thinking about dying and trying and intending to die: results on suicidal behavior from a large Web-based sample. J Clin Psychiatry. 2015;76:e359–65. doi: 10.4088/JCP.13m08864. [PubMed] [Cross Ref]
  • Dubey R, Zhou J, Wang Y, Thompson PM, Ye J. Analysis of sampling techniques for imbalanced data: An n = 648 ADNI study. Neuroimage. 2014;87:220–41. doi: 10.1016/j.neuroimage.2013.10.005. [PMC free article] [PubMed] [Cross Ref]
  • Feis DL, Schoene-Bake JC, Elger C, Wagner J, Tittgemeyer M, Weber B. Prediction of post-surgical seizure outcome in left mesial temporal lobe epilepsy. NeuroImage Clin. 2013;2:903–11. doi: 10.1016/j.nicl.2013.06.010. [PMC free article] [PubMed] [Cross Ref]
  • Foley DL, Goldston DB, Costello EJ, Angold A. Proximal psychiatric risk factors for suicidality in youth: the Great Smoky Mountains Study. Arch Gen Psychiatry. 2006;63:1017–24. doi: 10.1001/archpsyc.63.9.1017. [PubMed] [Cross Ref]
  • Galfalvy H, Oquendo MA, Carballo JJ, Sher L, Grunebaum MF, Burke A, Mann JJ. Clinical predictors of suicidal acts after major depression in bipolar disorder: a prospective study. Bipolar Disord. 2006;8:586–95. doi: 10.1111/j.1399-5618.2006.00340.x. [PubMed] [Cross Ref]
  • Goes FS, McCusker MG, Bienvenu OJ, Mackinnon DF, Mondimore FM, Schweizer B, Depaulo JR, Potash JB. Co-morbid anxiety disorders in bipolar disorder and major depression: familial aggregation and clinical characteristics of co- morbid panic disorder, social phobia, specific phobia and obsessive-compulsive disorder. Psychol Med. 2012;42:1449–59. doi: 10.1017/S0033291711002637. [PMC free article] [PubMed] [Cross Ref]
  • Goldstein TR, Ha W, Axelson DA, Goldstein BI, Liao F, Gill MK, Ryan ND, Yen S, Hunt J, Hower H, Keller M, Strober M, Birmaher B. Predictors of prospectively examined suicide attempts among youth with bipolar disorder. Arch Gen Psychiatry. 2012;69:1113–22. doi: 10.1001/archgenpsychiatry.2012.650. [PMC free article] [PubMed] [Cross Ref]
  • Gonzalez-Pinto A, Mosquera F, Alonso M, López P, Ramírez F, Vieta E, Baldessarini RJ. Suicidal risk in bipolar I disorder patients and adherence to long-term lithium treatment. Bipolar Disord. 2006;8:618–24. doi: 10.1111/j.1399-5618.2006.00368.x. [PubMed] [Cross Ref]
  • Grunebaum MF, Ellis SP, Duan N, Burke AK, Oquendo MA, John Mann J. Pilot randomized clinical trial of an SSRI vs bupropion: effects on suicidal behavior, ideation, and mood in major depression. Neuropsychopharmacology. 2012;37:697–706. doi: 10.1038/npp.2011.247. [PMC free article] [PubMed] [Cross Ref]
  • Hahn T, Kircher T, Straube B, Wittchen HU, Konrad C, Ströhle A, Wittmann A, Pfleiderer B, Reif A, Arolt V, Lueken U. Predicting Treatment Response to Cognitive Behavioral Therapy in Panic Disorder With Agoraphobia by Integrating Local Neural Information. JAMA Psychiatry. 2015;72:68. doi: 10.1001/jamapsychiatry.2014.1741. [PubMed] [Cross Ref]
  • Hoertel N, Franco S, Wall MM, Oquendo MA, Kerridge BT, Limosin F, Blanco C. Mental disorders and risk of suicide attempt: a national prospective study. Mol Psychiatry. 2015;20:718–26. doi: 10.1038/mp.2015.19. [PubMed] [Cross Ref]
  • Holma KM, Haukka J, Suominen K, Valtonen HM, Mantere O, Melartin TK, Sokero TP, Oquendo MA, Isometsä ET. Differences in incidence of suicide attempts between bipolar I and II disorders and major depressive disorder. Bipolar Disord. 2014;16:652–61. doi: 10.1111/bdi.12195. [PubMed] [Cross Ref]
  • Isometsä E, Sund R, Pirkola S. Post-discharge suicides of inpatients with bipolar disorder in Finland. Bipolar Disord. 2014;16:867–74. doi: 10.1111/bdi.12237. [PubMed] [Cross Ref]
  • Johnson J, Horwath E, Weissman MM. The validity of major depression with psychotic features based on a community study. Arch Gen Psychiatry. 1991;48:1075–81. [PubMed]
  • Johnston BA, Coghill D, Matthews K, Steele JD. Predicting methylphenidate response in attention deficit hyperactivity disorder: a preliminary study. J Psychopharmacol. 2015;29:24–30. doi: 10.1177/0269881114548438. [PubMed] [Cross Ref]
  • Johnston BA, Mwangi B, Matthews K, Coghill D, Konrad K, Steele JD. Brainstem abnormalities in attention deficit hyperactivity disorder support high accuracy individual diagnostic classification. Hum Brain Mapp. 2014;35:5179–89. doi: 10.1002/hbm.22542. [PubMed] [Cross Ref]
  • Kapczinski F, Passos IC. Wet minds, dry minds, and the future of psychiatry as a science. Acta Psychiatr Scand. 2015;131:7–8. doi: 10.1111/acps.12357. [PubMed] [Cross Ref]
  • Katz C, Yaseen ZS, Mojtabai R, Cohen LJ, Galynker II. Panic as an independent risk factor for suicide attempt in depressive illness: findings from the National Epidemiological Survey on Alcohol and Related Conditions (NESARC) J Clin Psychiatry. 2011;72:1628–35. doi: 10.4088/JCP.10m06186blu. [PubMed] [Cross Ref]
  • Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF, Sugarbaker DJ, Donaldson MC, Poss R, Ho KK, Ludwig LE, Pedan A, Goldman L. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100:1043–9. [PubMed]
  • Lenze EJ, Mulsant BH, Shear MK, Schulberg HC, Dew MA, Begley AE, Pollock BG, Reynolds CF. Comorbid anxiety disorders in depressed elderly patients. Am J Psychiatry. 2000;157:722–8. [PubMed]
  • Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, Abraham J, Adair T, Aggarwal R, Ahn SY, Alvarado M, Anderson HR, Anderson LM, Andrews KG, Atkinson C, Baddour LM, Barker-Collo S, Bartels DH, Bell ML, Benjamin EJ, Bennett D, Bhalla K, Bikbov B, Bin Abdulhak A, Birbeck G, Blyth F, Bolliger I, Boufous S, Bucello C, Burch M, Burney P, Carapetis J, Chen H, Chou D, Chugh SS, Coffeng LE, Colan SD, Colquhoun S, Colson KE, Condon J, Connor MD, Cooper LT, Corriere M, Cortinovis M, de Vaccaro KC, Couser W, Cowie BC, Criqui MH, Cross M, Dabhadkar KC, Dahodwala N, De Leo D, Degenhardt L, Delossantos A, Denenberg J, Des Jarlais DC, Dharmaratne SD, Dorsey ER, Driscoll T, Duber H, Ebel B, Erwin PJ, Espindola P, Ezzati M, Feigin V, Flaxman AD, Forouzanfar MH, Fowkes FGR, Franklin R, Fransen M, Freeman MK, Gabriel SE, Gakidou E, Gaspari F, Gillum RF, Gonzalez-Medina D, Halasa YA, Haring D, Harrison JE, Havmoeller R, Hay RJ, Hoen B, Hotez PJ, Hoy D, Jacobsen KH, James SL, Jasrasaria R, Jayaraman S, Johns N, Karthikeyan G, Kassebaum N, Keren A, Khoo JP, Knowlton LM, Kobusingye O, Koranteng A, Krishnamurthi R, Lipnick M, Lipshultz SE, Ohno SL, Mabweijano J, MacIntyre MF, Mallinger L, March L, Marks GB, Marks R, Matsumori A, Matzopoulos R, Mayosi BM, McAnulty JH, McDermott MM, McGrath J, Mensah GA, Merriman TR, Michaud C, Miller M, Miller TR, Mock C, Mocumbi AO, Mokdad AA, Moran A, Mulholland K, Nair MN, Naldi L, Narayan KMV, Nasseri K, Norman P, O'Donnell M, Omer SB, Ortblad K, Osborne R, Ozgediz D, Pahari B, Pandian JD, Rivero AP, Padilla RP, Perez-Ruiz F, Perico N, Phillips D, Pierce K, Pope CA, Porrini E, Pourmalek F, Raju M, Ranganathan D, Rehm JT, Rein DB, Remuzzi G, Rivara FP, Roberts T, De León FR, Rosenfeld LC, Rushton L, Sacco RL, Salomon JA, Sampson U, Sanman E, Schwebel DC, Segui-Gomez M, Shepard DS, Singh D, Singleton J, Sliwa K, Smith E, Steer A, Taylor JA, Thomas B, Tleyjeh IM, Towbin JA, Truelsen T, Undurraga EA, Venketasubramanian N, Vijayakumar L, Vos T, Wagner GR, Wang M, Wang W, Watt K, Weinstock MA, Weintraub R, Wilkinson JD, Woolf AD, Wulf S, Yeh PH, Yip P, Zabetian A, Zheng ZJ, Lopez AD, Murray CJL, AlMazroa MA, Memish ZA. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2095–128. doi: 10.1016/S0140-6736(12)61728-0. [PubMed] [Cross Ref]
  • Mann JJ, Apter A, Bertolote J, Beautrais A, Currier D, Haas A, Hegerl U, Lonnqvist J, Malone K, Marusic A, Mehlum L, Patton G, Phillips M, Rutz W, Rihmer Z, Schmidtke A, Shaffer D, Silverman M, Takahashi Y, Varnik A, Wasserman D, Yip P, Hendin H. Suicide prevention strategies: a systematic review. JAMA. 2005;294:2064–74. doi: 10.1001/jama.294.16.2064. [PubMed] [Cross Ref]
  • Mann JJ, Currier DM. Stress, genetics and epigenetic effects on the neurobiology of suicidal behavior and depression. Eur Psychiatry. 2010;25:268–71. doi: 10.1016/j.eurpsy.2010.01.009. [PMC free article] [PubMed] [Cross Ref]
  • Meltzer HY, Alphs L, Green AI, Altamura AC, Anand R, Bertoldi A, Bourgeois M, Chouinard G, Islam MZ, Kane J, Krishnan R, Lindenmayer JP, Potkin S. Clozapine treatment for suicidality in schizophrenia: International Suicide Prevention Trial (InterSePT) Arch Gen Psychiatry. 2003;60:82–91. [PubMed]
  • Morina N, Ajdukovic D, Bogic M, Franciskovic T, Kucukalic A, Lecic-Tosevski D, Morina L, Popovski M, Priebe S. Co-occurrence of major depressive episode and posttraumatic stress disorder among survivors of war: how is it different from either condition alone? J Clin Psychiatry. 2013;74:e212–8. doi: 10.4088/JCP.12m07844. [PubMed] [Cross Ref]
  • Mwangi B, Ebmeier KP, Matthews K, Steele JD. Multi-centre diagnostic classification of individual structural neuroimaging scans from patients with major depressive disorder. Brain. 2012;135:1508–21. doi: 10.1093/brain/aws084. [PubMed] [Cross Ref]
  • Mwangi B, Tian TS, Soares JC. A Review of Feature Reduction Techniques in Neuroimaging. Neuroinformatics. 2013;12:229–244. doi: 10.1007/s12021-013-9204-3. [PMC free article] [PubMed] [Cross Ref]
  • Neves FS, Malloy-Diniz LF, Corrêa H. Suicidal behavior in bipolar disorder: what is the influence of psychiatric comorbidities? J Clin Psychiatry. 2009;70:13–8. [PubMed]
  • Oquendo MA, Currier D, Liu SM, Hasin DS, Grant BF, Blanco C. Increased risk for suicidal behavior in comorbid bipolar disorder and alcohol use disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) J Clin Psychiatry. 2010;71:902–9. doi: 10.4088/JCP.09m05198gry. [PMC free article] [PubMed] [Cross Ref]
  • Oquendo MA, Perez-Rodriguez MM, Poh E, Sullivan G, Burke AK, Sublette ME, Mann JJ, Galfalvy H. Life events: a complex role in the timing of suicidal behavior among depressed patients. Mol Psychiatry. 2014;19:902–9. doi: 10.1038/mp.2013.128. [PMC free article] [PubMed] [Cross Ref]
  • Oquendo MA, Sullivan GM, Sudol K, Baca-Garcia E, Stanley BH, Sublette ME, Mann JJ. Toward a biosignature for suicide. Am J Psychiatry. 2014;171:1259–77. doi: 10.1176/appi.ajp.2014.14020194. [PMC free article] [PubMed] [Cross Ref]
  • Perlis RH. A clinical risk stratification tool for predicting treatment resistance in major depressive disorder. Biol Psychiatry. 2013;74:7–14. doi: 10.1016/j.biopsych.2012.12.007. [PMC free article] [PubMed] [Cross Ref]
  • Post RM, Kalivas P. Bipolar disorder and substance misuse: pathological and therapeutic implications of their comorbidity and cross-sensitisation. Br J Psychiatry. 2013;202:172–6. doi: 10.1192/bjp.bp.112.116855. [PMC free article] [PubMed] [Cross Ref]
  • Robert Tibshirani. Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc. 2011;73:273–282.
  • Schaffer A, Isometsä ET, Tondo L, H Moreno D, Turecki G, Reis C, Cassidy F, Sinyor M, Azorin JM, Kessing LV, Ha K, Goldstein T, Weizman A, Beautrais A, Chou YH, Diazgranados N, Levitt AJ, Zarate CA, Rihmer Z, Yatham LN. International Society for Bipolar Disorders Task Force on Suicide: meta-analyses and meta-regression of correlates of suicide attempts and suicide deaths in bipolar disorder. Bipolar Disord. 2014a doi: 10.1111/bdi.12271. [PubMed] [Cross Ref]
  • Schaffer A, Sinyor M, Reis C, Goldstein BI, Levitt AJ. Suicide in bipolar disorder: characteristics and subgroups. Bipolar Disord. 2014b;16:732–40. doi: 10.1111/bdi.12219. [PubMed] [Cross Ref]
  • Simon NM, Otto MW, Wisniewski SR, Fossey M, Sagduyu K, Frank E, Sachs GS, Nierenberg AA, Thase ME, Pollack MH. Anxiety disorder comorbidity in bipolar disorder patients: data from the first 500 participants in the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) Am J Psychiatry. 2004;161:2222–9. doi: 10.1176/appi.ajp.161.12.2222. [PubMed] [Cross Ref]
  • Soloff PH, Lynch KG, Kelly TM, Malone KM, Mann JJ. Characteristics of suicide attempts of patients with major depressive episode and borderline personality disorder: a comparative study. Am J Psychiatry. 2000;157:601–8. [PubMed]
  • Stein MB. An epidemiologic perspective on social anxiety disorder. J Clin Psychiatry. 2006;67(Suppl 1):3–8. [PubMed]
  • Tipping ME. Sparse Bayesian Learning and the Relevance Vector Machine. J Mach Learn Res. 2001:211–244.
  • Torres AR, Ramos-Cerqueira ATA, Ferrão YA, Fontenelle LF, do Rosário MC, Miguel EC. Suicidality in obsessive-compulsive disorder: prevalence and relation to symptom dimensions and comorbid conditions. J Clin Psychiatry. 2011;72:17–26. doi: 10.4088/JCP.09m05651blu. quiz 119–20. [PubMed] [Cross Ref]
  • Turecki G. The molecular bases of the suicidal brain. Nat Rev Neurosci. 2014;15:802–16. doi: 10.1038/nrn3839. [PMC free article] [PubMed] [Cross Ref]
  • Turecki G, Ernst C, Jollant F, Labonté B, Mechawar N. The neurodevelopmental origins of suicidal behavior. Trends Neurosci. 2012;35:14–23. doi: 10.1016/j.tins.2011.11.008. [PubMed] [Cross Ref]
  • Van Waarde JA, Scholte HS, van Oudheusden LJB, Verwey B, Denys D, van Wingen GA. A functional MRI marker may predict the outcome of electroconvulsive therapy in severe and treatment-resistant depression. Mol Psychiatry. 2015;20:609–14. doi: 10.1038/mp.2014.78. [PubMed] [Cross Ref]
  • Vapnik VN. Wiley Statistical Learning Theory. wiley; 1998. p. 768.
  • Webb RT, Lichtenstein P, Larsson H, Geddes JR, Fazel S. Suicide, hospital-presenting suicide attempts, and criminality in bipolar disorder: examination of risk for multiple adverse outcomes. J Clin Psychiatry. 2014;75:e809–16. doi: 10.4088/JCP.13m08899. [PMC free article] [PubMed] [Cross Ref]