|Home | About | Journals | Submit | Contact Us | Français|
Determining risk for a suicide attempt in psychiatric patients requires assessment of multiple risk factors and knowledge of their relative importance. Classification and regression tree (CART) analysis generates decision trees that select the variables that perform best in identifying the group of interest and model clinical decision making. Hypothetical decision trees to identify recent and remote suicide attempters, weighted to increase sensitivity, were generated for psychiatric patients using correlates of past suicidal behavior.
Correlates of past suicide attempts were identified in 408 patients with mood, schizophrenia spectrum, or personality disorders (DSM-IV). Correlated variables were entered into recursive partitioning statistical models to generate equally weighted and unequally weighted hypothetical decision trees for distinguishing recent (≤ 30 days prior to study) and remote (> 250 days prior to study) suicide attempters from nonattempters. The study was conducted from December 1989 to November 1998.
In equally weighted trees, a recent past suicide attempt was best predicted by current suicidal ideation (sensitivity = 56%, specificity = 91%, positive predictive value = 69%), and no adequate model was found for remote attempts. In unequally weighted models, recent attempters were identified by suicidal ideation and comorbid borderline personality disorder (sensitivity = 73%, specificity = 80%, positive predictive value = 58%). Remote attempters were identified by lifetime aggression and current subjective depression (sensitivity = 89%, specificity = 36%, positive predictive value = 44%).
Current suicidal ideation is the best indicator of a recent suicide attempt in psychiatric patients. Indicators of a remote attempt are aggressive traits and current depression. Weighted decision trees can improve sensitivity and miss fewer attempters but with a cost in specificity.
Suicide is a complication of psychiatric disorders, which are present in over 90% of suicide victims at death.1,2 Suicide attempt rates range from 15% to 50% in psychiatric disorders.3–6 Suicide attempts result in considerable morbidity and cost, and those who make a suicide attempt are at greater risk for both further attempts and suicide.7,8 However, because most psychiatric patients do not attempt suicide or die by suicide,9–12 additional causal factors must be involved. We have proposed a stress-diathesis model of suicidal behavior wherein state-related risk factors or stressors, such as a major depressive episode or life events, trigger suicidal acts in those with the diathesis, or predisposition, to suicidal behavior.13 Putative characteristics of the diathesis include aggressive-impulsive traits and a tendency for pessimism.13,14 This model organizes the “laundry list” of often related risk factors into independent domains with potential explanatory and predictive properties. Within this model, however, it is still necessary to determine the relative importance of contributory risk factors.
One method for assessing the relative importance of risk factors, and simulating the process of clinical assessment of risk, is to generate decision trees using a classification technique called recursive partitioning (classification and regression tree [CART]).15,16 This statistical technique constructs classification trees by dividing a sample into nonoverlapping subsamples that differ in terms of the response variable; in this case, the subsamples are presence or absence of a suicide attempt. The first partition into subsamples is based on the best correlate of a history of a suicide attempt, and then subsequent steps partition those subsamples into further nonoverlapping subsamples using the best correlate of the response variable for each subsample. This partitioning process is repeated as often as will yield a meaningful result, and then the trees are “pruned” to the most optimal level using a cost-complexity criterion.15
The advantages of tree-based models over other multivariate methods such as generalized linear models include (1) ease of interpretation: the resulting tree model seeks to resemble clinical decision-making processes, whereas generalized linear models use linear combinations of variables that can be difficult to interpret in practice; and (2) flexibility: due to the large number of tree-structures possible, and because no assumptions are made about the distribution of the response variable, recursive partitioning is a nonparametric procedure. Moreover, in CART, different cost weightings, reflective of clinical priorities, can be assigned to the outcome variable. For example, we believe that clinical practice is based on the view that the “cost” of misclassifying a suicide attempter as a nonattempter is greater than classifying a nonattempter as at-risk, and a tree can be generated that considers the different costs of false negatives and false positives when selecting the optimal predictor variables.
In this retrospective study of 408 patients with mood, schizophrenia spectrum, or personality disorders, we use CART techniques to generate equally and unequally weighted hypothetical decision trees to identify clinical correlates that best distinguish recent and remote suicide attempters from nonattempters. A previous suicide attempt is the best predictor of future suicidal acts, presumably because of shared risk factors. We have demonstrated that correlates of past suicide attempt identified in retrospective analyses, including both state- and trait-dependent variables, were predictive of future suicidal behavior in a prospective study.13,14 However, prospective studies often lack statistical power due to too few outcome events, and therefore this initial study compares recent and more remote past attempters as a proof of principle study.
Four hundred fifty-seven patients (47% male, 70% white) aged 14 to 72 years, with an IQ > 80 (assessed using the Wechsler Adult Intelligence Scale), were recruited in the Western Psychiatric Institute and Clinic (Pittsburgh, Pa.) and the New York State Psychiatric Institute (New York), using the same protocol. All patients gave written informed consent as required by the applicable institutional review board. From this sample, 408 subjects were retained who had not made a suicide attempt (N = 210), had made a suicide attempt within 30 days of assessment (N = 80), or had made a suicide attempt more than 250 days prior to assessment (N = 118). Three hundred forty-seven subjects (75.9%) participated in an earlier study.13 The study was conducted from December 1989 to November 1998.
Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) Axis I psychiatric disorders were diagnosed using the Structured Clinical Interview for DSM-IV (SCID-I).17 Axis II diagnoses were made in nonschizophrenic patients using the Personality Disorder Examination18 in the first 221 cases and with the SCID-II19 in the subsequent 84 cases. Psychiatric symptoms were assessed by the Brief Psychiatric Rating Scale (BPRS),20 the Scales for the Assessment of Positive and Negative Symptoms in Schizophrenia,21,22 the 17-item Hamilton Rating Scale for Depression (HAM-D),23 the Beck Depression Inventory (BDI),24 and the Beck Hopelessness Scale.25 Lifetime aggression, hostility, and impulsiveness were rated with the Brown-Goodwin Lifetime History of Aggression (BGLHA),26 the Buss-Durkee Hostility Inventory,27 and the Barratt Impulsivity Scale.28 Life events were assessed using the St. Paul-Ramsey Life Experience Scale (A. E. Lumry, unpublished, 1978) and potential protective factors by the Reasons for Living Inventory.29 Participants, and, for about half the patients, one other family member, were interviewed regarding family history of suicidal behavior and psychiatric disorder, the latter ascertained using the modified DSM-IV checklist. Raters were nurses, psychologists, or social workers having a master’s degree or Ph.D.
A suicide attempt was defined as a self-destructive act carried out with intent to end one’s life. The Scale for Suicide Ideation30 rated suicidal ideation during the 2 weeks prior to hospitalization and after admission to hospital. Characteristics of suicide attempts were documented using the Columbia Suicide History Form,31 which records all past suicide attempts and the method and lethality of the attempts, and were rated using Beck’s Medical Lethality Scale,32 a semi-structured interview with the patient report supplemented by all available medical records. We did not rely on past medical records for suicidal behavior because they underreport previous suicidal behavior, particularly suicidal acts, compared with a semi-structured questionnaire.33 One hundred ninety-eight patients (48.6%) reported a previous suicide attempt, and 59% of these reported multiple suicide attempts. Suicide attempt methods for most recent attempt were overdose/poison (69%), cutting (19%), jumping or drowning (6%), firearms (3.3%), and hanging (3.3%). Attempters and nonattempters had comparable diagnoses (mood disorders, 54% vs. 58%; schizophrenia spectrum disorders, 34% vs. 37%; other Axis I or Axis II disorders, 22% vs. 16%). Eighty-eight percent of subjects were inpatients.
Suicide attempters were divided into recent (≤ 30 days prior to assessment) and remote (> 250 days prior to assessment) attempters on the basis that suicidal behavior within a short-term period of 30 days is most relevant to clinical practice in terms of assessing suicide risk. Suicide attempts that occurred in the longer term were also examined (> 250 days from assessment), as they might be informative about trait predictors while, in contrast, state-dependent psychopathology should be more closely correlated to a recent suicide attempt. Suicide attempters who had made an attempt from 31 to 249 days prior to the study (N = 50) were omitted from further analyses. The omitted suicide attempter group had more male subjects, had more borderline personality and fewer schizophrenia subjects, reported fewer reasons for living, and had fewer suicide attempts than the included suicide attempters (data not shown).
Group comparisons of state and trait variables were conducted between nonattempters and remote and recent attempters. Three group comparisons used χ2 for categorical variables, and analysis of variance and Kruskal-Wallis for continuous variables with normal and non-normal distribution, respectively. Comparisons between recent and remote attempter groups used χ2 for categorical variables and Student t test and Mann-Whitney U test for continuous variables with normal and non-normal distributions, respectively. All tests were 2-tailed. Results are reported as mean and SD.
Classification and Regression Tree (CART) analysis is a “recursive partitioning” algorithm (see Breiman et al.15 and Ellis et al.16) that provides a backward elimination method for rigorous model selection in predictive model building. Model selection is accomplished by “pruning” a large initial tree until the sum
is minimized. The “complexity” of the tree is the number of terminal nodes or “leaves,” and the coefficient, α, is the “complexity parameter.” Larger values of α correspond to more pruning. The point is to balance quality of fit with complexity of the model.
We used the recursive partitioning (RPART) implementation of CART in R.34 Two sets of CART analyses were undertaken, one for recent attempt versus no attempt (excluding remote attempters from the analysis), and one for remote attempt versus no attempt (excluding recent attempters). In both, the response variable is the dichotomous variable “prior history of attempt.” Other variables entered into the analyses were the state and trait variables detailed in Table 1. Diagnosis, age, and study site were also included.
When the response variable is dichotomous, the “cost” of classification errors using a given tree is simply the number of false positives in the training data multiplied by the “cost” of a false positive + the number of false negatives multiplied by the “cost” of a false negative. The default values of these 2 different costs are both 1, so the default cost of classification errors is merely the number of misclassified cases. We used this equal cost in fitting the initial versions of our tree models and then applied an unequal cost weighting for a second set of trees in order to explore potential for improvement gained by differential weighting.
Given a choice of costs for the 2 types of errors (false negatives and false positives) in binary classification, CART offers cross-validation (CV) to estimate the best value of α. The data are divided into tenths. Each tenth is held out in turn, a model is fitted on the remaining data (using a given value of α), and the misclassification cost for that tree on the held out tenth is calculated. That process is repeated for all 10 subsamples, and the weighted mean cost over all 10 hold out samples is an estimate of the weighted mean cost of predicting the attempter status of a randomly chosen subject from the population using trees computed using the given value of α and fitted to samples comparable to the sample at hand. One chooses the largest value of α that “nearly” minimizes these CV estimates of cost.
To estimate sensitivity and specificity with minimal bias, we used a “leave out 1” CV. One case is left out from the dataset and the above process is performed on the remaining data. The resulting tree is used to classify the held out case. The proportion of positive cases classified correctly when held out is a good estimate of sensitivity. The proportion of negative cases classified correctly when held out is a good estimate of specificity. Sensitivity and specificity estimated by this method will be more accurate than that calculated from the raw data used to construct the tree. These cross-validated estimates do not apply to any specific tree, but rather to a tree building method as applied to samples of the given size from the population. “Tree building method” is the choice of costs. Thus, CV enters into the calculation at 2 levels: to estimate the sensitivity and specificity of a tree constructed for a given cost ratio, and, given a cost ratio, the construction of the tree itself involves a CV to choose the complexity parameter α.
By varying the cost ratio (cost of false negative:cost of false positive) a receiver operating characteristic (ROC) curve35 can be plotted. That is, a plot of the sensitivity versus 1 – specificity. An ROC curve that coincides with the 45-degree x = y line corresponds to classifying subjects at random, i.e., without regard to the subject attributes. An ROC curve that arcs high above the x = y line indicates that the classification method extracts much useful information from the attributes. Based on ROC curves (Figures 1A and 1B), we generated a second set of trees with unequal false negative:false positive cost weightings of 2.1:1 for recent attempts and 2.875:1 for remote attempts. Weightings were chosen based on the ROC curves (Figures 1A and 1B) where the ROC curve was farthest from the 45-degree line. However, other points could be selected depending on clinical or other priorities.
Table 1 gives details of group comparisons. Suicide attempters had more severe current suicidal ideation, perceived fewer reasons for living, had more subjective depression (BDI), and had fewer positive symptoms than nonattempters. Compared to nonattempters, suicide attempters had a greater incidence of comorbid borderline personality disorder, and were more likely to report a childhood history of abuse, past head injury, and comorbid alcoholism or substance use disorder; be smokers; have a family history of suicide or suicide attempt; and have more severe lifetime aggression, hostility, and impulsivity scores. No group differences were observed on clinician ratings of depression severity (HAM-D) and general psychopathology (BPRS). Age, sex, and ethnicity had no effect on differences (data not shown).
Recent attempters had higher suicidal ideation scores and fewer positive symptoms compared with remote attempters and did not differ on current clinician-rated illness severity.
Figure 2 shows the equal weight pruned tree generated by CART analysis. The first split was considered optimal by the program; suicidal ideation score was the only variable used, classifying the sample into attempters (Scale for Suicide Ideation score ≥ 17.5) and nonattempters (Scale for Suicide Ideation score < 17.5). This model had a positive predictive value (PPV) of 69.7%, cross-validated sensitivity of 56%, and cross-validated specificity of 91%.
The area under the estimated ROC curve is 0.80 for recent attempters (Figure 1A), and Figure 3 shows the pruned tree weighted with a false negative:false positive cost ratio of 2.1:1. Two splits were considered optimal. Suicidal ideation score remains the initial predictor variable. The high suicidal ideation score (Scale for Suicide Ideation score ≥ 17.5) group is classified as suicide attempters, and the low ideation (Scale for Suicide Ideation score < 17.5) group further split into those with borderline personality disorder who are classified as recent attempters and those without borderline personality disorder, classified as nonattempters. This model had a PPV of 57.8%, cross-validated sensitivity of 72.5%, and cross-validated specificity of 80%.
Efforts to produce the equally weighted pruned tree for remote attempts failed, showing no practical predictive utility.
For remote attempters, the area under the estimated ROC curve is 0.65 (Figure 1B), and Figure 4 shows the pruned tree for remote attempters in which the false negative:false positive cost ratio was set at 2.875:1. Life-time aggression was the initial predictor, with the high aggression score (BGLHA score ≥ 15.5) group classified as remote attempters. The low aggression group (BGLHA score < 15.5) was then split according to self-rated depression (BDI) scores, with the low depression score (BDI score < 20.5) group being classified as nonattempters. The high depression (BDI score ≥ 20.5) group was split again on the basis of aggression score, and the high aggression (BGLHA score ≥ 13.5) group classified as attempters. This model had a PPV of 43.8%, cross-validated sensitivity of 89%, and cross-validated specificity of 36%.
Consistent with previous studies, we found multiple state- and trait-related variables that distinguished suicide attempters from nonattempters in univariate analysis.36,37 Given the multiplicity of correlates of suicidal behavior, clinicians must gauge relative importance of individual factors to assess suicide risk. Classification and regression tree analysis produces classification trees that identify the most salient predictive variables for the outcome in question and ranks them in order of relative importance. In this retrospective study, CART analyses identified current suicidal ideation as the strongest predictor of recent suicide attempter status, and lifetime aggression as the strongest predictor of remote suicide attempter status. Weighting the models to attach a greater cost to a missed suicide attempter (a false negative) resulted in improved sensitivity and slightly altered the number and order of variables in the trees. This improvement in sensitivity was offset by a decline in specificity and reflected in a decrease of PPV from 70% to 58% in the recent attempter tree, as the number of false positives increases as specificity declines.
Suicidal ideation distinguished recent from remote attempters in a univariate analysis and was the strongest predictor of recent suicide attempter status in both the equally and unequally weighted CART analyses (Figures 2 and and3).3). The weighted tree with suicidal ideation and comorbid borderline personality disorder had 73% sensitivity and 80% specificity, providing a reasonable level of predictive utility.
Suicidal ideation has been associated with suicide attempt in retrospective studies3,13,38,39 and with repeat suicide attempts40,41 and suicide42–44 in prospective studies. Few studies have evaluated the predictive capacity of current suicidal ideation with respect to time period of risk. Two studies found that current or worst lifetime suicidal ideation was associated with suicide in the longer-term (2–4 years)42–45; however, neither reported on suicide attempts. Others found that suicide attempters who reat-tempted within 12 months of baseline assessment had higher suicidal ideation at the 1-month follow-up assessment point.46 Here, suicidal ideation was not useful in identifying remote attempters, not surprising for a state-dependent measure. Suicidal ideation was nevertheless relevant for identifying individuals at short-term risk. This is consistent with current clinical practice of hospitalizing, or more closely monitoring, individuals with more severe suicidal ideation, particularly when ideation includes a specific plan or method for suicide.47 However, given the 30% to 40% false positive rate in these models, clearly decisions about risk and treatment cannot be made solely on the basis of a single instrument.
More severe lifetime externally directed aggression distinguishes past suicide attempters from nonattempters regardless of specific psychiatric diagnosis in this study and others13,48 and also predicts future suicide attempters.14 In univariate analysis, lifetime aggression did not differentiate remote and recent attempters, as might be expected for a trait that is associated with the diathesis for suicidal behavior. However, in CART analysis lifetime aggression was the strongest predictor of remote attempter status. Interestingly, the lifetime aggression score above which an individual was classified as a remote suicide attempter was below both recent and remote attempter mean aggression scores (Table 1). This suggests that the risk posed by higher lifetime aggression is present in both the short- and long-term periods, although the absence of aggression in the recent attempter tree indicates that other factors are more salient predictors in the short term.
Borderline personality disorder was more common in suicide attempters than nonattempters and did not distinguish recent from remote attempters in univariate analysis. In CART analysis, borderline personality disorder appeared in the unequal cost recent attempt tree (Figure 3) as an additional variable to predict suicide attempt in the lower suicidal ideation subgroup. While that split misclassified more attempters than it correctly classified (12 misclassified and 8 correctly classified), it did identify additional attempters, improving the sensitivity of the overall tree from 56% to 73%.
Borderline personality is characterized by high levels of suicide attempt and other self-harm behaviors.49 Suicidal behavior in borderline personality disorder often occurs in the context of comorbid major depression or substance use disorder,50 and, in this sample, the majority (80%) of individuals with borderline personality disorder also had a comorbid Axis I disorder, predominantly major depressive disorder (64%). Suicidality in borderline personality disorder is thought to be, in part, related to the aggressive/impulsive traits that characterize the disorder,51,52 although other factors such as pronounced mood lability and susceptibility to interpersonal conflict may play a role.53 Thus, the inclusion of borderline personality disorder in the recent attempt model suggests that, in the short term, aggressive/impulsive traits pose a risk for suicide attempt even in the absence of high levels of suicidal ideation.
Self-rated depression was more severe in suicide attempters compared with nonattempters, but no different between recent and remote attempters in univariate analysis. In the unequal cost tree for remote attempts, among the lower lifetime aggression group, those with higher subjective depression and greater lifetime aggression were identified as suicide attempters (Figure 4). Self-rated, or subjective, depression can be state-related; however, we have also suggested that it may be indicative of a trait-like construct of pessimism. We have proposed that pessimism, characterized by an excess of subjective depression, hopelessness, suicidal ideation, and the perception of fewer reasons for living, relative to severity of clinician-rated depression, is an element of the diathesis for suicidal behavior.13,14 In a prospective study, pessimism was predictive of suicidal behavior in follow-up.13,14 Suicide over a 10-year period was predicted by hopelessness and the pessimism item of the BDI.54 The presence of greater subjective depression in the remote attempter tree suggests it has some predictive utility, perhaps as an indicator of trait pessimism and the diathesis for suicidal behavior, for ongoing and not just acute risk for suicide attempt.
In this study, determining how to cost weight the analyses was done on the basis of the assumption that a missed attempter is more costly than a misclassified nonattempter. This reflects the common clinical concern about missing a potentially life-threatening outcome; however, there are other factors that also need to be considered, for example, the burden on resources and patients of unwarranted hospitalization of false positives. However, medications such as clozapine and lithium and psychotherapies such as dialectical behavior therapy or cognitive therapy have been shown to prevent suicide attempts in randomized controlled trials.55 Therefore, there is value to assigning false negatives a greater cost than false positives, as the availability of treatments that work make it worthwhile identifying at-risk patients, and the problem of lower specificity, or more false positives, is mitigated because there is meaningful benefit to treating mood disorders even if no additional benefit accrues in terms of suicide prevention. The risk of suicide attempts is 6-fold greater during an episode of major depression,7 and so treating the major depression with or without additional antisuicidal effects is valuable. Determining appropriate cost weightings is a complex matter, and beyond the scope of this analysis; however, for the purposes of demonstrating the flexibility of the CART method in terms of incorporating clinical judgment into the classification process, with potential improvement in sensitivity, the weightings selected here demonstrated some benefit. Future studies should systematically address the impact of different cost weightings.
This is a retrospective study and can only identify correlates of past suicidal behavior by CART. As such, it does not inform us directly about the predictive salience of risk factors for future suicidal behavior, although our previous studies have shown that suicidal behavior correlates identified in retrospective study had predictive value for suicide attempts in a prospective study.13,14 A prospective study is needed to determine if the CART method can build trees of adequate sensitivity and specificity for predicting future suicidal behavior. Such a prospective study would provide more precise estimates of the specificity and sensitivity of predictors and combinations of predictors. The study sample has a high proportion of inpatients and thus represents a more severely impaired cohort. As such, results may not be generalizable to milder illness.
As with any retrospective study, there is a possibility of recall bias but there is no evidence that the factors that predict suicide attempts such as pessimism or aggressive/impulsive traits should affect recall of previous suicide attempts, and, for recent attempters, the period of recall was only 1 month. The size of subsamples in each major psychiatric disease category was large compared to many published studies of suicidal behavior but modest given the number of variables involved.
Evaluating suicide risk involves assessing multiple psychosocial and psychopathological factors. Ranking the relative importance of risk factors, or combinations of risk factors, is necessary for prediction and prevention. Using retrospective data, CART analysis generated classification trees, with promising sensitivity. Short-term prediction was optimized at 73% sensitivity and 80% specificity. Improvements in sensitivity come at the cost of specificity, and the reduction in specificity is reflected in a decline in the positive predictive value of the models, as it indicates a decrease in the ability to correctly detect true negatives. Minimizing false positives is a clinically desirable outcome of any suicide risk assessment method.
The predictors identified by the CART analysis are consistent with a stress-diathesis model, whereby the state-dependent factor of suicidal ideation was the strongest putative predictor of recent attempter status, and the trait-related factor of lifetime aggression the best putative predictor of remote attempter status. Instruments used to assess these risk factors are amenable to clinical practice and include inquiries as to frequency and intensity of suicidal thoughts, specific plans for suicide attempts, and questions on lifetime aggression, hostility, and impulsiveness. The usefulness of such screening approaches must be tested in prospective studies with the goal of improving prevention.
Drug names: clozapine (Clozaril, FazaClo, and others), lithium (Eskalith, Lithobid, and others).
Supported by National Institute of Mental Health grants MH62185 and MH48514 and by the Audrey Wallace Otto Fund of the St. Louis Community Foundation and The Diane Goldberg Foundation.
Patient assessments were performed by Donna Abbondanza, R.N.; Tom Kelly, Ph.D.; Diane Dolata, R.D.; and Elizabeth Radomsky, Ph.D.
The authors report no additional financial or other relationships relevant to the subject of this article.