shows the sensitivity, specificity, and overall accuracy for the three standardized instruments using a spread of cutoff points. The MHSF demonstrated the highest overall accuracy (73%) and second highest sensitivity (90%), with a cutoff score of 3 or higher; specificity was less satisfactory at 47.5%. A cutoff point of 6 produced a better balance between sensitivity (75%) and specificity (68%), with an equivalent degree of overall accuracy (72%). The GSS produced similar overall accuracy with a cutoff score of 2 or higher (71%), based primarily on the sensitivity score (83%) while achieving relatively low specificity (45%). The MINI, with a cutoff score of 5 or 6, closely approximated the other instruments in overall accuracy (69%). At a cutoff score of 6, the MINI was similar to the MHSF with a cutoff score of 6 in both sensitivity (73%) and specificity (63%). The MINI produced the highest level of specificity (80%) at a cutoff score of 8 with sensitivity of 50% and overall accuracy of 62%.
Comparison of Screening Instruments
ROC curves shown in demonstrate a higher discriminant value for the MHSF followed by the MINI and the GSS short screener with areas under the curve of .805, .741, and .731, respectively.
Receiver Operating Characteristics Curve
In an attempt to explore the possibility of developing a more powerful screener using selected items from each of the three standard instruments, all items from the three instruments were correlated with a positive diagnosis on the SCID. Items with a statistically significant (p < .05) correlation were then tested in regression analysis against the positive SCID diagnosis to remove items accounting for the same variance. The regression results are shown in .
Results of Regression Analysis: Relationship of Individual Items to SCID-IV Diagnosis of a Mental Disorder
Six items with standardized coefficients ranging from 0.194 to 0.389 (disregarding the signs) and p values ranging from p < .000 to p < .02 were significant in the regressions. These items and the instruments from which they were drawn included (a) “Have you ever felt you needed help with your emotional problems or have other people told you that you should get help for your emotional problems?” (MHSF), (b) “Have you ever talked to a psychiatrist, psychologist, therapist, social worker, or counselor about an emotional problem?” (MHSF), (c) “Have you been told by teachers, guidance counselors, or others that you have a special learning problem?” (MHSF, reverse scored), (d) “Have you ever been advised to take medication for anxiety, depression, hearing voices, or for any other emotional problem?” (MHSF, reverse scored), (e) “Have you had one or more occasions when you felt intensely anxious, frightened, uncomfortable, or uneasy even when most people would not feel this way?” “Do you feel anxious or uneasy in a place or situations where you might have the panic like symptoms we just spoke about? Or do you feel anxious or uneasy in a situations where help might not be available or escape might be difficult?” (MINI), (f) “During the past 12 months, have you been a bully or threatened other people two or more times?” (GSS). These items formed a 6-item screener, the Co-occurring Disorder Screening Instrument for Mental Disorders (CODSI-MD). Although the reverse scoring for 2 of these items appears unusual, the reverse scores compensate for an overestimation of mental disorder by the 4 positively scored items. unusual, the reverse scores compensate for an overestimation of mental disorder by the 4 positively scored items.
The area under the ROC curve for the 6-item CODSI-MD was .808, indicating good discriminant value. shows sensitivity and specificity at several potential cutoff scores. At a score of 3 or higher, the CODSI-MD achieved an overall accuracy of 81.0%, with a sensitivity of 86.7% and a specificity of 72.5%.
Precision of Six-Item Co-Occurring Disorders Screening Instrument for Mental Disorders
shows the results by gender. The CODSI-MD performed better for men than for women, but the overall accuracy was acceptable for both genders. Disaggregating the results by gender gave ROC curve values of .833 for men and .733 for women. With a cutoff score of 3 or higher, sensitivity for men was 84.4%, specificity was 80.0%, and overall accuracy was 82.7%. The same cutoff score of 3 also produced the highest overall accuracy for women with sensitivity of 93.3%, specificity of 50.0% and overall accuracy of 76.0%.
Precision of Co-Occurring Disorders Screening Instrument for Mental Disorders by Gender
Screening for Severe Mental Disorders
A secondary aim that emerged in the course of conducting this study was to determine the precision of instruments for detecting severe mental disorders (major depression, schizophrenia, and bipolar disorders) and risk for suicide. Along with the three standard instruments, the study tested a 3-item screener for severe mental disorders (the CODSI-SMD), which was developed using a regression analysis to determine the items in the standard instruments that were most associated with severe disorders. The first 3 items selected in the regression, the instrument in which they appeared, and the standardized coefficients were (a) “Have you felt sad, low, or depressed most of the time for the past two years?” (MINI, β = .266, t = 3.053, p < .01), (b) “Did you ever attempt to kill yourself?” (MHSF, β = .515, t = 3.921, p < .01), and (c) “Have you ever had a period of time when you were so full of energy and your ideas came very rapidly, when you talked nearly non-stop, when you moved quickly from one activity to another, when you needed little sleep, and believed you could do almost anything?” (MHSF, β = .205, t = 2.668, p < .01).
shows the overall sensitivity, specificity and overall accuracy for CODSI-SMD, the MHSF, MINI and GSS Internal Disorder Screener scale (GSS-IDS; the subsection of the GSS that pertains to severe mental disorder) using the cutoff score reflecting the highest overall accuracy. The CODSI-SMD with a cutoff score of 2, and the MHSF with a cutoff score of 11, showed the best scores for overall accuracy (82.0% and 76.0%, respectively), specificity (90.3% and 88.9%, respectively), and sensitivity (60.7% and 42.9%, respectively). Although these two instruments were similar in specificity, the CODSI-SMD showed higher sensitivity (the ability to detect severe mental illness when it is present). shows the results of the ROC curves for the four instruments with the cutoff scores specified in the table. Areas under the curve, indicative of the relative discriminant value over chance of each instrument, were .755 for the CODSI-SMD, .659 for the MHSF, .642 for the MINI, and .639 for the GSS-IDS.
Identification of Presence of Severe Mental Disorder