The data provide strong evidence for the inter-rater reliability of prototype diagnosis of PDs in clinical practice. Whereas field trials and inter-interview studies comparing diagnoses made by different structured interviews and questionnaires for the same PDs administered days or a few weeks apart have shown low correlations and even lower kappa coefficients indicating convergent diagnoses (Clark, Livesley, & Morey, 1997
; Pilkonis, et al., 1995
; Skodol, Oldham, Rosnick, Kellman, & Hyler, 1991
), we found high correlations in the range of r =
.70 between two assessments made by independent assessors from naturalistic clinical hours. Prototype diagnoses also demonstrated strong differentiation across disorders (what might be called discriminant inter-rater reliability), something not seen in previous research. Although elsewhere we have discussed whether prototype matching might be useful for research diagnosis or for diagnosis of axis I disorders as well (Westen & Bradley, 2005
; Westen, et al., 2002
), what the data here suggest is that prototype diagnosis offers a promising alternative method for personality diagnosis in clinical practice.
The major limitation of the study is that the sample was relatively small and the clinicians were relatively inexperienced and drawn from the same clinical training pool. Two considerations, however, mitigate these limitations. First, the limitations would favor null findings. For example, inexperienced clinicians would likely have more difficulty using diagnoses other than the more familiar DSM-IV categories as well as the DSM-IV diagnostic approach, and their limited clinical experience would render them less likely to converge on diagnostic impressions following an interviewing procedure that is relatively open-ended, focusing on the patient’s life history as a way of exploring ongoing and enduring personality dynamics. Second, the effects were large and significant, even with this sample size, and the small magnitude of the correlations off the diagonal (i.e., between unrelated or minimally related diagnoses) relative to those on the diagonal (showing diagnostic agreement) clearly demonstrated that even inexperienced clinicians could make highly specific diagnostic judgments when evaluating the same patient using the kinds of data experienced clinicians collect over the course of initial interviews in clinical practice (which the assessment procedure was intended to simulate and standardize).
A second limitation is that clinicians were rating empirically derived diagnoses rather than prototypes of the current axis II disorders. Three considerations, however, limit this concern. First, as with the first limitation, this one would reduce inter-clinician agreement, rendering the findings more conservative, given that clinicians were matching patients to diagnoses with which they were unfamiliar. Second, prior research has found that the four empirically derived disorders tested here that resemble Cluster B disorders have similar correlates as the DSM-IV Cluster B PDs and hence are likely to be a reasonable proxy for them.
Third, although DSM-V appears increasingly likely to include prototype matching as an approach to dimensionalizing PD diagnosis, it is unlikely to use the current diagnoses precisely as they are configured now, given their many limitations, such as comorbidity (Skodol & Bender, 2009
). Thus, the approach tested here, with rich, empirically derived prototypes, is at least as likely as a version of the current diagnoses woven into prototype form to resemble prototype diagnosis in DSM-V. Indeed, the prototypes tested here provide a model of the kind of prototypes that might be useful in DSM-V, in that they are empirically derived rather than clinically constructed by committee. Spitzer and colleagues (Spitzer, et al., 2008
) used these empirically derived PD prototypes in their comparative study of clinical utility and found that clinicians preferred even these unfamiliar prototypes over prototypes derived from the current axis II disorders because of their clinical richness of description. Westen and colleagues (Westen, et al., 2006
) compared the four of these prototypes most comparable to the DSM-IV Cluster B disorders (antisocial-psychopathic, emotionally dysregulated, histrionic, and narcissistic) as well as prototypes of the four Cluster B disorders to DSM-IV diagnoses and found similar results. One of the advantages of prototype diagnosis is that it can also include more, and clinically richer, criteria than the eight to nine criteria per disorder included in the current diagnostic system, because clinicians do not have to make independent judgments on each criterion; rather, they make a single prototypicality judgment on each diagnosis taken as a gestalt.
Finally, a major question often raised about a prototype matching approach to diagnosis is whether it is a “throwback” to DSM-II (i.e., a return to paragraph length diagnoses). That question, however, misses the point. Prototype diagnosis has the parsimony of DSM-II diagnosis but lacks its disadvantages. Although the format of the diagnostic prototypes may superficially resemble the format of the diagnostic paragraphs in the first two editions of the DSM, this approach to diagnosis differs from the early diagnostic manuals in several key respects: (1) the diagnostic criteria (and in this case, the diagnoses themselves) are entirely empirically derived, not rationally or clinically derived, as in DSM-I and – II; (2) the diagnoses are not laden with causal clinical hypotheses of the 1930s and 1940s; and (3) most importantly, clinicians are not making idiosyncratic dichotomous characterizations of patients as either having or not having a disorders, which would likely be as unreliable as dichotomous judgments about prototypes. Rather, clinicians are taking into account all available data and making a judgment of the extent to which the patient matches an empirically derived prototype.
Clinicians are reluctant to implement the existing axis II diagnostic system with its laundry list of symptoms, cumbersome algorithms, overlapping criteria, and descriptive vagaries. Prototype matching, on the other hand, allows for rich descriptions of personality constructs without an exorbitant clinical effort. Using a prototype system, clinicians could briefly and efficiently (within one or two minutes) make an axis II diagnosis, generating a diagnostic profile that indicates for each disorder both the extent to which the patient resembles the prototype and whether the patient matches the prototype strongly enough to receive a categorical diagnosis. Empirically, the results of this study generated extremely high estimates of cross-clinician reliability and lower cross-correlations with unrelated disorders than we have seen in any PD study to date. The prototype diagnostic system utilized in this study offers clinically-rich diagnostic descriptions which are not only reliably observable across clinicians but also highly discriminative. Narrative diagnostic descriptions allow for improved treatment planning and clinical training while reliable and distinctive diagnoses increase the efficiency of clinical communication and co-ordination across providers. Indeed, clinicians find prototype diagnosis preferable to alternative approaches across a range of clinical utility variables including comprehensiveness, ease of implementation, enhancement of treatment planning, and clarity of communication with mental health providers as well as patients (Rottman, et al., 2009
; Spitzer, First, & Skodol, 2006
At this point, given the consistent evidence of the validity, clinical utility (First, et al., 2004
), and now inter-clinician reliability of prototype diagnosis for PDs, we would recommend that DSM-V incorporate prototype matching as the primary method of diagnosing personality constellations, given the likelihood that the Work Group appears headed toward maintaining a constellational approach that is likely to be supplemented by other approaches, such as trait diagnosis (Skodol & Bender, 2009
). We would also recommend, based on these and other data on Axis I disorders (Ortigo, et al., 2010
), that the ICD-11 consider prototype diagnosis for all disorders for clinical practice, and that the architects of both the ICD-11 and DSM-V undertake research to test whether prototype matching may be a workable approach for clinical practice and research for all clinical disorders, not only PDs.