PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Psychosom Med. Author manuscript; available in PMC 2009 April 15.
Published in final edited form as:
PMCID: PMC2669490
NIHMSID: NIHMS88597

THE DIAGNOSTIC ACCURACY OF PREDICTING SOMATIZATION FROM PATIENTS’ ICD-9 DIAGNOSES

Robert C. Smith, MD, MS,1 Joseph C. Gardiner, PhD,2 Zhehui Luo, PhD,2 and Kathryn Rost, PhD3

Abstract

OBJECTIVES

Our earlier study demonstrated that ICD-9 codes and other data from the administrative database (ADB) effectively identified somatization. To further develop this simple screening method, we hypothesized, in a new and different population, that ADB screening would identify somatizing patients by increasing numbers of visits, female gender, and greater percent of ICD-9 primary diagnosis codes in musculoskeletal, nervous, gastrointestinal, and ill-defined body systems; we labeled these codes as having “somatization potential.”

METHODS

Using a prospective observational design in a staff model HMO, we evaluated 1364 patients from 18–65 years old who had 8 or more visits yearly in the two years before study. Clinician raters applied a reliable method of medical chart review to identify patients meeting criteria for somatization. We randomly selected 2/3 for the derivation set (N=901) for logistic regression to evaluate the contribution of potential ADB correlates (age, gender, all encounters, primary diagnosis codes [ICD-9], revenue codes, and charges) of a diagnosis of somatization. This prediction rule was then applied to the remaining 1/3 of subjects, the validation set (N=463).

RESULTS

Patients averaged 47.1 years, 12.8 visits per year, and 71.6% were female; 319 had somatization. Age, visits, and somatization potential were associated with clinician-rated somatization, with a c-statistic 0.719 [95% CI: 0.679, 0.760] in the derivation set and 0.679 [95% CI: 0.625, 0.734] in the validation set.

CONCLUSIONS

These data support our earlier findings that selected ICD-9 diagnoses in the ADB predict somatization, suggesting their potential in identifying a common, costly, and usually unrecognized problem. The demonstrated stability of ICD-9 codes for diagnosing somatization indicates that the next step in research be taken, as we outline here.

Keywords: administrative database screening, somatization, medically unexplained symptoms, sensitivity & specificity, diagnostic accuracy

BACKGROUND

Patients with somatization are common, and they are defined as having physical symptoms with little or no documented basis in underlying organic disease; when organic disease exists, the symptoms are inconsistent with or out of proportion to it [13]. The prevalence of primary care patients with one or more somatizing symptoms ranges from 33% upwards in outpatient settings [1, 4], while the prevalence of patients with three of more concurrent somatic symptoms (multisomatoform disorder) is estimated to be 8% [5, 6]. Although the field has begun to define effective, cost-neutral primary care treatment, somatizing patients seldom receive it because they are not recognized by clinicians or health care systems [79]. The treatment they do receive often occasions safety and cost problems, as ill-advised lab testing and ‘trial treatments’ lead to iatrogenic complications and substantially increased costs [1014].

To improve system-wide diagnosis/identification, our research team [15] and Barsky et al. [16] reported that administrative databases (ADB) can provide useful information in identifying somatizing patients. The performance of such ADB screening programs needs replication in different populations before they can be incorporated into system-wide efforts to improve identification (and treatment) of somatization.

From data used to identify subjects for a RCT -- to treat somatization in primary care [8] -- we report in this paper the test of our earlier ADB screener [15]. The initial screening study demonstrated that selected ICD-9 primary diagnosis codes identified somatization: all codes in the musculoskeletal, nervous, gastrointestinal, and ill-defined body systems, which we labeled as having “somatization potential.” Based on results from this study [15], we hypothesized in the study reported here that greater somatization potential would identify somatizing patients in addition to female gender and greater health care utilization. We defined somatization as one or more unexplained somatic symptoms identified by clinician raters to test the screening method’s capacity to identify patients with any somatization. The validation of ADB screeners against Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) [17], somatoform definitions (full and abridged) was avoided because DSM-IV diagnoses miss >75% of somatizing primary care patients with high utilization and significant impairment [18, 19].

DESIGN

We employed a prospective observational design to study patients in a large HMO in Lansing, MI and obtained the Institutional Review Board (IRB) approvals from the HMO and from Michigan State University to conduct the study.

SUBJECTS

From all the plan’s enrolled patients on the ADB (constructed for clinical and administrative purposes only), we selected (in May 2000) all patients 18–65 years old who had at least 8 visits per year for each of the two years prior to the study. We selected this high-utilizing population because somatization is most prevalent in such populations [2], and they also are the logical targets for cost-effective intervention. We did not exclude subjects with ICD-9 codes for depression and anxiety because such codes are erratically recorded, if at all, in primary care. We selected patients who were members of the same health care system for the previous two years to generate a comprehensive list of diagnoses, recognizing that this approach precluded somatization identification in new patients and patients using multiple health care systems. Visits were defined as all outpatient clinical encounters with physicians, nurse practitioners, or physician assistants. Visits to nurses were not included; e.g., for blood pressure checks or allergy shots.

MEASUREMENT

Operational Definition of Major Constructs

Screening Variables

Gender – Gender was collected from the ADB and coded with males as the referent.

Age – Age was collected from the ADB and coded as a continuous variable.

Number of Visits – The total number of visits to all physician, nurse practitioner, or physician assistant providers across the two years was collected from the ADB and coded as a continuous variable.

Somatization Potential – Somatization potential was derived from the ADB by examining ICD-9 primary diagnosis codes during the preceding year in the following ranges: 320–389 (nervous system); 520–579 (gastrointestinal system); 710–739 (musculoskeletal system); and 780–799 (ill-defined body systems), as detailed in Appendix A. Somatization potential was measured by the percent of all diagnoses in the above ranges, capturing both organic and potentially somatization diagnoses. Post hoc, we explored other definitions of somatization potential and could demonstrate no benefit over our a priori definition above; e.g., using codes from additional body systems added nothing, likely due to the greater proportions of organic diseases in these systems. Definitions and data for these exploratory evaluations are available from the authors on request. Post hoc, we also removed all ICD-9 codes referring to depression or anxiety in testing somatization potential.

Only predictors available on the ADB were used in this study.

Clinician Ratings of Somatization

In contrast to most studies, which use structured patient interviews, this study used a reliable, clinician-conducted method for rating patients’ medical records to identify somatization symptoms. Five clinicians (3 senior internal medicine residents, one primary care nurse practitioner faculty, one general medicine faculty) were trained to rate these patients’ medical charts. All patient charts were rated, but each clinician rated randomly selected samples of the paper charts of the high-utilizers identified from the ADB to establish criterion standard ratings of somatization for this study [19]. To offset diagnostic and suspicion biases [20], the raters were blinded to the hypothesis and predictors, and they did not use gender, age, number of visits, or specific symptom complexes in their diagnostic evaluations. Instead, the patient’s medical record was evaluated entirely for laboratory and other objective data indicating the presence or absence of organic diseases. Using a reliable procedure [19], raters rated visits to any primary care provider in the clinic (physician, physician assistant, nurse practitioner) over a one-year period; all data in the paper medical record, including laboratory and non-primary care visits, were available to inform ratings. Applying a scoring algorithm to these ratings, we identified patients whose primary problem was rated as somatization, defined for this study (to be most comprehensive, guided by the least stringent definition in DSM-IV[17]) as no documented organic disease to explain one or more symptoms of at least 6 months duration [19]. We found that patients classified as somatization had at least 50% of primary care visits in the previous year designated as having physical symptoms that could not be explained by organic disease. Symptom syndromes identified in charts by patients’ physicians (e.g., irritable bowel syndrome, fibromyalgia, chronic fatigue syndrome), were evaluated by our raters according to the above criteria rather than interpreting the diagnosis given in the chart as evidence of medically unexplained symptoms. Inter-rater agreement on characterizing symptoms as documented organic, documented somatization, or undocumented ranged from 92% – 96% [19].

Data Analysis

A logistic regression model for somatization status was developed on a randomly selected derivation dataset of a 2/3 sample of patients (N=901). The remaining 1/3 of the sample of 463 patients served as a validation dataset. Patient characteristics in the derivation and validation datasets were statistically equivalent. We tested age, gender, past visits, and somatization potential as predictors of somatization status in the derivation sample. From the final model, we estimated the c-statistic, or Area Under the Curve, from an empirical receiver-operator curve (ROC) analysis. The c-statistic measures the ability of the model to discriminate somatization from non-somatization. A c-statistic of 0.68 or greater was deemed acceptable [21]. Sensitivity, specificity, and positive/negative predictive values were computed from the final model in the derivation data. The final model was prospectively validated in the validation dataset and the c-statistic was reported as well. We also tested the goodness-of-fit of the data to the estimated model by use of the Hosmer-Lemeshow test [22].

To compare the predictive power of the newly generated ADB screening device to our previous screener [15], we applied the latter (1) in the current validation sample:

equation M1
(1)

where π is the predicted probability of somatization, and SomaPot represents somatization potential.

RESULTS

The parent RCT [8] initially identified 1646 patients and eliminated 246 subjects because they had obvious organic diseases. We fully rated the remaining 1400 paper medical records, and eliminated 36 additional subjects because they had incomplete medical records. The remaining 1364 patients are the subjects of this research. As Table 1 displays, patients averaged 47.1 years of age, 12.8 visits per year, and 71.6% were female; 319/1646 (19.4% prevalence) of these high utilizers met criterion standard for somatization. Somatizers differed from non-somatizers in age (p<.003), gender (p<.002), total visits (p<.001), and somatization potential (p<.001).

TABLE 1
Distribution of Patient Characteristics by Medical Record-Based Somatizing Status

Table 2 shows the percent of all patients who had somatization potential diagnoses in the four main body systems (left side). For example, 21.2% of patients had gastrointestinal system diagnoses (n=289), and somatizing patients had a higher proportion with these diagnoses as compared with nonsomatizing patients (25.7% versus 19.8%, p-value=0.02). Table 2 also shows (right side) the mean percent of the somatization potential diagnoses among all diagnoses. For example, among patients with GI diagnoses, the mean percent of these diagnoses of the total diagnoses was 18.9 percent among somatizers and 19.1 percent among nonsomatizers (p-value=0.94). The mean percent of musculoskeletal system diagnoses and ill-defined body system diagnoses among all diagnoses were significantly different between the two groups (p-value=0.002 and 0.004, respectively).

TABLE 2
Distribution of All Visit Diagnoses by Medical Record-Based Somatizing Status

Derivation Set

A randomly selected, two-thirds sample of 901 patients formed the derivation set for constructing a logistic model for assessing the correlates of somatization. The remaining patients had statistically similar characteristics to the derivation sample patients. The logistic model using information from the ADB contained age, age squared, gender, the number of total visits, and somatization potential (Table 3). There were no significant interactions. The probability of somatization π is estimated by:

equation M2
(2)

The c-statistic was 0.719 [95% CI: 0.679, 0.760]. In this model, the higher order term for age was statistically significant (p<0.01), albeit the magnitude was small. It captures an important feature of the relationship between somatizing and age, i.e., the probability of somatizing first increased by age, peaked at age 38 and then decreased by age.

TABLE 3
Predictive Logistic Model of Somatizing Status Based on Administrative Database (ADB) ICD-9 Codes in the Derivation Sample (N=901)

When we explored restricting the dataset to just subjects from 20–55 years of age, to better match the 2001 dataset, the c-statistic was the same (0.72) and, as expected, age and age squared were no longer statistically significant in the prediction equation (p-value= 09 and p-value=.06, respectively); gender still was not significant. This age restriction does not make the new sample comparable on age distribution and the association of age and somatization.

Validation Set

When we computed the c-statistic using the estimated parameters in Model (2) in the validation dataset, we found a smaller c-statistic of 0.679 [95% CI: 0.625, 0.734], as expected. Applying Model (1) to the validation dataset, we found that the previously published model had a very similar predictive power (c-statistic is 0.70). Applying Model (2) to the original dataset also gave a similar c-statistic of 0.76. Sensitivity and specificity estimates based on Model (2) in the derivation dataset are presented in Table 4. To illustrate, using a cutoff of 0.3 in Model (2) means that a patient is predicted to be a somatizer if the model gave an estimated probability π ≥ 0.3. In this case, the sensitivity and specificity in Model (2) were 46.5% and 82.5% respectively. The corresponding positive predictive value and negative predictive value were 38.9% and 86.5%, respectively, based on a prevalence of 19.4%.

TABLE 4
Sensitivity, Specificity, PPV and NPV Based on “Somatization Potential” in ADB: Derivation Dataset

In a post hoc analysis, we dropped all patients with a primary ICD-9 code indicating depression or anxiety prior to the enrollment into the study. In the derivation dataset, a total of 103 patients had a primary diagnosis of depression or anxiety. We re-estimated the model in equation (2) and the resulting c-statistic was 0.706 and all estimates were similar to the estimates in Table 3.

DISCUSSION

In a cohort where 19.4% of patients met reliable clinician rater criteria for somatization, age, number of visits, and somatization potential were associated with clinician-rated somatization, with a c-statistic 0.72 in the derivation set and 0.68 in the validation set. C-statistics from 0.68 and higher have been considered to have good discriminative ability [21].

There is encouraging clinical comparability between the prediction equations in the current and previous study. As in the previous study, the likelihood of somatization increases with increasing percentages of all ICD-9 diagnosis codes in the nervous (320–389), gastrointestinal (520–579), musculoskeletal (710–739), and ill-defined (780–799) body systems – those with “somatization potential.” We suspect the greater stability of the current prediction rule (c-statistic drop of 0.04 between the derivation and validation set) compared to the previous study (drop of 0.12 between the derivation and validation set) reflects the more reliable clinician rating of our criterion definition of somatization in the current study.

There are sociodemographic differences in the prediction equations between the current and previous study. The new inclusion of age squared and the exclusion of female gender in the current prediction rule suggest that the research team has not determined the sociodemographic variables and their associated weights in an optimal prediction equation for use across health care systems. This task will require further research in sociodemographically heterogeneous populations. In addition to providing generalizable estimates of sociodemographic predictors, such research can also rigorously test the key assumption that diversely trained clinicians practicing across a range of health care systems comparably code potential somatization and other diagnoses in their ADB diagnoses.

There are other potential limitations for ADB screening. We studied a select, high-utilizing group who are likely to be more severe than many somatizing patients in primary care and, therefore, our data do not apply to screening less severe somatization [2]. Nevertheless, our screening does identify those primary care patients likely to be most in need of treatment.

At the other end of the severity spectrum, the present study population differs from populations identified from structured interviews for diagnosing DSM-IV somatoform disorders [17]. Of the 206 somatization subjects identified for the RCT from the population reported here, another report demonstrated that 124 (60.2%) had a nonsomatoform DSM-IV diagnosis of any type, mostly anxiety and depression [18]. But, only 9 (4.4%) had any full DSM-IV somatoform diagnosis, and only 39 (18.9%) had abridged somatization disorder; i.e., the remaining 158 (76.7%) had neither a full nor a minor DSM-IV somatoform diagnosis [18]. Therefore, this psychologically distressed, high-utilizing primary care somatizing population we report does not resemble a population with DSM-IV somatoform disorders and more closely reflects DSM-IV anxiety and depression patients. This is consistent with other findings that most primary care depression and anxiety patients present, often exclusively, with medically unexplained symptoms [2331]. We suggest that DSM-IV somatoform diagnoses reflect the even more severe disease seen in specialty settings, and that they are far less applicable in primary care [18, 19]. We believe the population reported here represents a typical high-utilizing primary care population with somatization of the type for which one would most beneficially screen – and target for treatment. Screening for major or minor DSM-IV disorders would miss over 75% of psychologically distressed, high-utilizing primary care patients with unexplained symptoms [18].

Further, we required continuous enrollment for 2 years in one system, which would preclude ADB screening in patients new to the system and/or using multiple systems. We also note the potential limitation of retrospective chart review to identify the criterion standard for somatization. However reliable our raters may have been, the reliability and validity of the chart itself depends on how aggressively physicians sought organic diseases, how completely they actually recorded their findings and patients’ symptoms, how consistent they were over time, and how similar they were to other physicians in recording their behaviors [32]. Nevertheless, our chart rating procedure does overcome many shortcomings of reliability and validity resulting from using structured clinical interviews to identify DSM-IV somatization [18, 19]. The problems with present definitions of somatization (ICD-9 as well as DSM-IV) and recommended changes presently are being vigorously debated [2, 18, 33].

At this point, used alone, our results are insufficient to support wide scale screening. For example, from Table 4, an HMO might want to screen at the 0.30 cutoff to identify approximately one-half (46.5%) of their somatization patients. But, while the positive predictive value doubles from 19.4% (prevalence; pre-test probability) to 38.9%, this posttest probability is not sufficient for wide scale screening, identifying more false than true positives.

It is important to note that the cut-off point for ADB screening can be adjusted to be highly sensitive (e.g., 90%), which makes this approach an ideal first-stage screener identifying at-risk somatizers. To more accurately identify just somatization cases, however, will require second-stage screening. Rost et al. recently identified a modification of the 15 question Patient Health Questionnaire (PHQ-15) [34], called the PHQ-15-R [revised]. It has a sensitivity and specificity for identifying somatization of .99 and .98, respectively [35]. As a second stage of screening, applying the PHQ-15-R to the ADB screen-positives (first stage) would remove nearly all false positives and retain almost all somatizers; e.g., at the 0.30 cut point, two-stage screening would identify nearly 1/2 of all somatizing patients and include almost no false positives. These subjects could then be referred for treatment of somatization. While the PHQ-15-R could be administered to all subjects in the first place, the need for a questionnaire would impede simple, inexpensive population-based screening. Rigorous testing of this 2-stage screening will first be required, however, to demonstrate satisfactory positive predictive values. Two-stage screening could then be used by any system that employs the ICD-9 diagnosis codes required for first stage screening.

We conclude that the somatization potential of primary diagnosis ICD-9 codes from the ADB, combined with utilization characteristics, can help identify somatizing patients. We have corroborated this finding in a new data set of representative primary care patients using a reliable criterion standard definition of somatization. To most effectively screen for somatization on a population basis, however, will require combining administrative database screening with more precise second-stage screening – which should be the next phase of study.

ACKNOWLEDGEMENTS

This work was funded in part by NIMH grant MH50799 and AHRQ grant HS 14206.

Acronyms Used in Text

ADB
administrative database
ICD-9
International Classification of Diseases, 9th edition
HMO
health maintenance organization
RCT
randomized controlled trial
DSM-IV
Diagnostic and Statistical Manual of Mental Disorders, 4th edition
PHQ-15
Patient Health Questionnaire, 15 items
PHQ-15-R
PHQ-15, revised

Appendix A

The 17 Body Systems and their ICD-9 Code Ranges

  1. Infectious and parasitic diseases (001–139);
  2. Neoplasms (140–239);
  3. Endocrine, nutritional and metabolic diseases, and immunity disorders (240–279);
  4. Diseases of the blood and blood forming organs (280–289);
  5. Mental disorders (290–319);
  6. Diseases of the nervous system and sense organs (320–389);
  7. Diseases of the circulatory system (390–459);
  8. Diseases of the respiratory system (460–519);
  9. Diseases of the digestive system (520–579);
  10. Diseases of the genitourinary system (580–629);
  11. Complications of pregnancy, childbirth, and the puerperium (630–677);
  12. Diseases of the skin and subcutaneous tissue (680–709);
  13. Diseases of the musculoskeletal system and connective tissue (710–739);
  14. Congenital anomalies (740–759);
  15. Certain conditions originating in the perinatal period (760–779);
  16. Symptoms, signs, and ill-defined conditions (780–799);
  17. Injury and poisoning (800–999).

Body systems above numbered 6, 9, 13, and 16 are the four systems with somatization potential.

REFERENCES

1. Katon W, Russo J. Somatic symptoms and depression. J. Fam. Pract. 1989;29:65–69. [PubMed]
2. Smith RC, Dwamena FC. Classification and diagnosis of patients with medically unexplained symptoms. J Gen Intern Med. 2007;22(5):685–691. [PMC free article] [PubMed]
3. Lyles JS, Hodges A, Collins C, Lein C, Given CW, Given B, D'Mello D, Osborn GG, Goddeeris J, Gardiner JC, Smith RC. Using nurse practitioners to implement an intervention in primary care for high utilizing patients with medically unexplained symptoms. Gen. Hosp. Psychiatry. 2003;25:63–73. [PubMed]
4. deGruy F, Columbia L, Dickinson P. Somatization disorder in a family practice. J. Fam. Pract. 1987;25:45–51. [PubMed]
5. Kroenke K, Spitzer RL, deGruy FV, 3rd, Hahn SR, Linzer M, Williams JB, Brody D, Davies M. Multisomatoform disorder. An alternative to undifferentiated somatoform disorder for the somatizing patient in primary care. Arch Gen Psychiatry. 1997;54(4):352–358. [PubMed]
6. Jackson JL, Kroenke K. Prevalence, impact, and prognosis of multisomatoform disorder in primary care: a 5-year follow-up study. Psychosom Med. 2008;70(4):430–434. [PubMed]
7. Smith RC, Lein C, Collins C, Lyles JS, Given B, Dwamena FC, Coffey J, Hodges A, Gardiner JC, Goddeeris J, Given CW. Treating patients with medically unexplained symptoms in primary care. J. Gen. Intern. Med. 2003;18:478–489. [PMC free article] [PubMed]
8. Smith RC, Lyles JS, Gardiner JC, Sirbu C, Hodges A, Collins C, Dwamena FC, Lein C, Given CW, Given B, Goddeeris J. Primary Care Clinicians Treat Patients with Medically Unexplained Symptoms -- A Randomized Controlled Trial. J. Gen. Intern. Med. 2006;21:671–677. [PMC free article] [PubMed]
9. Luo Z, Goddeeris J, Gardiner J, Lyles J, Smith RC. Costs of an intervention for primary care patients with medically unexplained symptoms -- a randomized controlled trial. Psychiatr Serv. 2007;58:1079–1086. [PMC free article] [PubMed]
10. Deyo RA. Pain and public policy. N. Engl. J. Med. 2000;342:1211–1213. [PubMed]
11. Fink P. Surgery and medical treatment in persistent somatizing patients. J. Psychosom. Res. 1992;36:439–447. [PubMed]
12. Hoffman RM, Wheeler KJ, Deyo RA. Surgery for herniated lumbar discs: a literature synthesis. J. Gen. Intern. Med. 1993;8:487–496. [PubMed]
13. Lightfoot JRW, Luft BJ, Rahn DW, Steere AC, Sigal LH, Zoschke DC, Gardner P, Britton MC, Kaufman FL. Empiric parenteral antibiotic treatment of patients with fibromyalgia and fatigue and a positive serologic result for Lyme disease. Ann. Intern. Med. 1993;119:503–509. [PubMed]
14. Adams K, Corrigan J, editors. Institute of Medicine: Priority Areas for National Action -- Transforming Health Care Quality. Washington,DC: The National Academies Press; 2003. 20001.
15. Smith RC, Gardiner JC, Armatti S, Johnson M, Lyles JS, Given CW, Lein C, Given B, Goddeeris J, Korban E, Haddad R, Kanj M. Screening for high utilizing somatizing patients using a prediction rule derived from the management information system of an HMO -- a preliminary study. Med. Care. 2001;39:968–978. [PubMed]
16. Barsky AJ, Orav EJ, Bates DW. Distinctive patterns of medical care utilization in patients who somatize. Med. Care. 2006;44:803–811. [PubMed]
17. American Psychiatric A: Diagnostic and Statistical Manual of Mental Disorders -- DSM-IV. 4th ed. Washington, DC: American Psychiatric Association; 1994.
18. Smith RC, Gardiner JC, Lyles JS, Sirbu C, Dwamena FC, Hodges A, Collins C, Lein C, Given CW, Given B, Goddeeris J. Exploration of DSM-IV Criteria in Primary Care Patients with Medically Unexplained Symptoms. Psychosom. Med. 2005;67:123–129. [PMC free article] [PubMed]
19. Smith RC, Korban E, Kanj M, Haddad R, Lyles JS, Lein C, Gardiner JC, Hodges A, Dwamena FC, Coffey J, Collins C. A method for rating charts to identify and classify patients with medically unexplained symptoms. Psychother. Psychosom. 2004;73:36–42. [PMC free article] [PubMed]
20. Sackett DL, Haynes RB, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. Boston: Little, Brown and Co; 1985.
21. Thompson IM, Ankerst DP, Chi C, Lucia MS, Goodman PJ, Crowley JJ, Parnes HL, Coltman CA., Jr Operating characteristics of prostate-specific antigen in men with an initial PSA level of 3.0 ng/ml or lower. Jama. 2005;294(1):66–70. [PubMed]
22. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York: John Wiley & Sons,Inc; 2000.
23. Henningsen P, Zimmermann T, Sattel H. Medically unexplained physical symptoms, anxiety, and depression: a meta-analytic review. Psychosom. Med. 2003;65:528–533. [PubMed]
24. Katon WJ, Walker EA. Medically unexplained symptoms in primary care. J Clin Psychiatry. 1998;59 suppl 20:15–21. [PubMed]
25. Kisely S, Goldberg D, Simon G. A comparison between somatic symptoms with and without clear organic cause: results of an international study. Psychol. Med. 1997;27:1011–1019. [PubMed]
26. Kroenke K. The interface between physical and psychological symptoms. Primary Care Companion J Clin Psychiatry. 2003;5 suppl 7:11–18.
27. Kroenke K, Jackson JL, Chamberlin J. Depressive and anxiety disorders in patients presenting with physical complaints: clinical predictors and outcome. Am. J. Med. 1997;103:339–347. [PubMed]
28. Kroenke K, Spitzer RL, Williams JBW, Linzer M, Hahn SR, deGruy Iii FV, Brody D. Physical symptoms in primary care -- predictors of psychiatric disorders and functional impairment. Archives of Family Medicine. 1994;3:774–779. [PubMed]
29. Russo J, Katon W, Sullivan M, Clark M, Buchwald D. Severity of somatization and its relationship to psychiatric disorders and personality. Psychosomatics. 1994;35:546–556. [PubMed]
30. Simon G, Gater R, Kisely S, Piccinelli M. Somatic symptoms of distress: an international prmary care study. Psychosom. Med. 1996;58:481–488. [PubMed]
31. Simon GE, VonKorff M. Somatization and psychiatric disorder in the NIMH Epidemiologic Catchment Area Study. American Journal of Psychiatry. 1991;148:1494–1500. [PubMed]
32. Aaronson LS, Burman ME. Use of health records in research: reliability and validity issues. Research in Nursing and in Health. 1994;17:67–73. [PubMed]
33. Rief W, Rojas G. Stability of somatoform symptoms--implications for classification. Psychosom Med. 2007;69(9):864–869. [PubMed]
34. Kroenke K, Spitzer RL, Williams JB. The PHQ-15: validity of a new measure for evaluating the severity of somatic symptoms. Psychosom Med. 2002;64(2):258–266. [PubMed]
35. Rost KM, Dickinson WP, Dickinson LM, Smith RC. Multisomatoform disorder: agreement between patient and physician report of criterion symptom explanation. CNS Spectr. 2006;11(5):383–388. [PubMed]