For the period 1999 through 2001, we identified 177,618 hospital discharges of patients ages 10 and over in the BC dataset that had a valid E-code and a primary or secondary ICD-9-CM diagnosis code for an injury other than an adverse effect of medical care, late effect, or adverse food reaction. In the 2003 NIS, we identified 527,798 injury hospitalizations meeting these criteria. After deleting 4,110 with missing age or sex, 254,910 from states with E-code reporting rates less than 85%, and 13,431 hospitalizations without E-codes from remaining states, we were left with a sample size of 245,164.
Characteristics of the hospital discharges randomly allocated to the development sample -- 88,808 of 177,618 BC hospitalizations and 122,574 of 254,910 NIS hospitalizations -- are presented in . The mean patient age was 55 in BC and 57 in the NIS, with the population roughly split between males and females. Race was not reported in the BC data; in the NIS sample 75.9% of patients were white.. In-hospital death resulted from 1.1% of BC admissions and 2.7% of NIS admissions. Fractures were the most common injury type, accounting for 34.0% of BC admissions and 51.4% of NIS admissions. Substance abuse was the most commonly recorded psychiatric diagnosis (8.0% in BC, 13.7% in NIS), followed by depression (4.1% in BC, 12.2% in NIS) and dementia (3.6% in BC, 8.5% in NIS). A greater proportion of injury hospitalizations in younger subjects were due to intentional self-harm (in BC, 13.8% in age 10 – 25, 10.2% in age 25 – 64; in the NIS, 15.8% in age <25, 12.2% in age 25 – 64) than were in the elderly (0.8% in BC, 0.7% in the NIS).
Patient characteristics in the 50% development sample of hospital discharges with a primary or secondary injury diagnosis
The performance characteristics of possible algorithms to identify intentional self-harm hospitalizations are presented in . In the BC data, the presence of a substance abuse diagnosis had the greatest sensitivity (30.7%), indicating that 30.7% of intentional self-harm hospitalizations had a substance abuse diagnosis recorded. Other diagnoses that were commonly assigned to intentional self-harm hospitalizations were depression (sensitivity = 30.4%), mania (19.5%), personality disorder (18.5%), unspecified psych (6.2%), adjustment reaction (6.6%), other psych disorder (6.2), and psychotic disorder (5.9%). The specificity – i.e. the probability that hospitalizations for injuries other than intentional self-harm didn’t have the diagnosis coded – was above 96% for all psychiatric diagnoses excluding substance abuse (specificity = 94%). The positive predictive value – i.e. the probability that a hospitalization assigned a psychiatric diagnosis of interest is an intentional self-harm hospitalization – was highest for unspecified non-psychotic mental disorders (78.1%), personality disorder (67.0%), adjustment reaction (66.6%), mania (57.7%) and depression (50.5%). Similar patterns were observed in the NIS data, although depression was recorded with a greater frequency among NIS intentional self-harm cases (sensitivity = 61.1%) and in the overall population (12.2% in the NIS versus 4.2% in BC). Algorithms based on a combination of psychiatric diagnoses had sensitivities ranging from 62.9% to 69.3% in the BC data and 77.6% to 82.1% in the NIS; PPVs ranged from 43.9% to 55.1% in the BC data and 35.5% to 39.4% in the NIS.
Operating characteristics of alternative algorithms to identify intentional self-harm cases among injury hospital discharges – development sample
Certain types of injuries were frequently recorded among intentional self-harm hospitalizations. In the BC data, 38.3% of intentional self-harm hospitalizations had a diagnosis of poisoning by a psychotropic agent, 53.4% had a diagnosis of poisoning by another drug, 9.2% had a diagnosis of toxic effects of non-medicinal substances, and 7.6% had a diagnosis of an open wound to the wrist, forearm, or elbow. While asphyxiation was an uncommon diagnosis (sensitivity was 1.2%), the PPV for this diagnosis was high – 84.5% -- indicating that the majority of asphyxiation cases resulted from intentional self-harm. Similar patterns were observed in the NIS, although PPVs were lower. An algorithm defining intentional self-harm as a hospitalization for poisoning, toxicity of substances chiefly non-medical in nature, or asphyxiation had a sensitivity of 84.3% in the BC development sample (82.3% in the NIS) and a PPV of 64.9% in the BC development sample (50.8% in the NIS). Including open wound to wrist, elbow or forearm increased the sensitivity to 90.3% (89.4% NIS) but reduced the PPV to 62.0% (47.4% NIS). Algorithms based on type of injury and the presence of psychiatric diagnoses provided an improvement in PPV at the loss of some sensitivity. Defining intentional self-harm hospitalizations as those with a diagnosis of depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder plus a diagnosis of poisoning, toxicity of a substance chiefly non-medical in nature, asphyxiation, or open wound to the elbow, wrist, or forearm yielded a sensitivity of 59.8%, a specificity of 99.4%, and PPV of 88.3% in the BC data; values in the NIS were 71.1%, 98.0%, and 74.1%. Adding ADHD, psychotic disorder, and other mental disorders to the list of allowed psychiatric diagnoses increased the sensitivity to 65% but reduced the PPV to 85.8% in the BC data. In the NIS, the values were 74.2 and 72.3%. In the interest of maintaining a high specificity at the expense of some sensitivity, we elected to omit the ADHD, psychotic disorder, and other mental disorder diagnoses from our final algorithm.
Thus, our final algorithm classifies a hospitalization due to injury as resulting from intentional self-harm if a diagnosis of depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder is recorded as well as a diagnosis of poisoning, toxicity of a substance chiefly non-medical in nature, asphyxiation, or open wound to the elbow, wrist, or forearm.
The performance of the derived algorithms in the validation sample varied by patient characteristics, as shown for the more restrictive algorithm above in . In both BC and the NIS, the specificity was highest for hospitalizations among patients age <25, resulting in the highest PPV in this group (92.2% in BC, 83.4% NIS). The low prevalence of intentional self-harm hospitalizations among all injury hospitalization in subjects aged > 65 resulted in a low PPV in this group (65.8% in BC, 48.6% in NIS). The PPV was slightly lower in males. In the BC sample, which included history of depression diagnosis or antidepressant prescription in the past 180 days, PPV was slightly lower in subjects with no antidepressant use (87.5% versus 88.9% for those with past antidepressant use in BC) and differed little by prior history of depression diagnosis. Results from a secondary analysis including children younger than 10 were essentially identical.
Table 3 Operating characteristics of the preferred algorithm by patient characteristics in the 50% validation sample. The preferred algorithm defines intentional self-harm hospitalizations as those with a diagnosis of depression, personality disorder, mania, (more ...)
summarizes the bias in estimated relative rates of intentional self-harm that might result from using the algorithm to identify intentional self-harm hospitalizations in a hypothetical study of antidepressant safety conducted under three scenarios. In the first example () based on data from the general BC population where intentional self-harm hospitalizations accounted for 10% – 23% of injury hospitalizations depending on age and antidepressant use, intentional self-harm rate ratios comparing antidepressant use to non-use were underestimated by about 0.07 (1.83 versus 1.90 for adolescents, 1.50 versus 1.57 for non-senior adults). In the second example, based on patients initiating antidepressants in BC where rates of self-harm are substantially higher and intentional self-harm hospitalizations are highly prevalent among injury hospitalizations (62 – 76% of injury hospitalizations in those under 25, 24 – 33% in non-senior adults), bias was reduced to −0.007 and −0.03. depicts the results from an analysis based on rates of intentional self-harm and other injury hospitalizations observed in the general US population9
and the sensitivity and specificity values calculated from the NIS. Although the prevalence of intentional self-harm hospitalizations among all injury hospitalizations was comparable to that in the BC general population, the reduced specificity of the algorithm in the US data led to greater bias. Rate ratios were underestimated by 0.12 – 0.13.
Bias in Rate Ratios Calculated Using the Algorithm to Identify Outcomes