Search tips
Search criteria 


Logo of schbulschizophrenia bulletinsubscriptionscontact uscurrent issuemy basketarchivemy accountsearchcontact this journaloxford journalsabout this journal
Schizophr Bull. 2012 March; 38(2): 231–238.
Published online 2011 September 9. doi:  10.1093/schbul/sbr129
PMCID: PMC3283149

Phenotypically Continuous With Clinical Psychosis, Discontinuous in Need for Care: Evidence for an Extended Psychosis Phenotype



Rates of self-reported psychotic experiences (SRPEs) in general population samples are high; however the reliability against interview-based assessments and the clinical significance of false-positive (FP) ratings remain unclear. Design: The second Netherlands Mental Health Survey and Incidence Study-2, a general population study.


Trained lay interviewers administered a structured interview assessing psychopathology and psychosocial characteristics in 6646 participants. Participants with at least one SRPE (N = 1084) were reassessed by clinical telephone interview.


Thirty-six percent of participants with SRPEs were confirmed by clinical interview as true positive (TP). SPREs not confirmed by clinical interview (FP group) generated less help-seeking behavior and occurred less frequently compared with TP experiences (TP group). However, compared with controls without psychotic experiences, the FP group more often displayed mood disorder (relative risk [RR] 1.7, 1.4–2.2), substance use disorder (RR 2.0, 1.6–2.6), cannabis use (RR 1.5, 1.2–1.9), higher levels of neuroticism (RR 1.8, 1.5–2.2), affective dysregulation, and social dysfunction. The FP group also experienced more sexual (RR 2.0, 1.5–2.8) and psychological childhood trauma (RR 2.1, 1.7–2.6) as well as peer victimization (RR 1.5, 1.2–2.0) and recent life events (RR 2.0, 1.6–2.4) than controls without psychotic experiences. Differences between the FP group and the TP group across these domains were much smaller and less conclusive.


SRPEs not confirmed by clinical interview may epresent the softest expression of an extended psychosis phenotype that is phenotypically continuous with clinical psychosis but discontinuous in need for care.

Keywords: diagnosis, schizophrenia, trauma, cannabis, epidemiology, false positive


Extended phenotypes of psychotic disorder in the general population are thought to reflect the behavioral expression of distributed population risk.1 Although psychometric expressions of risk in the general population are transitory in the majority of individuals, progression to clinical outcome may occur depending on the rate of persistence2; degree of “comorbid” admixture of dimensions of negative symptoms, affective dysregulation and reality distortion3,4; level of coping5; number, frequency, severity and associated distress of psychotic experiences68; and level of premorbid social functioning.6 Poulton and colleagues demonstrated that an assessment of “definite” psychotic experience carried a higher likelihood of transition to clinical psychotic disorder over a 16-year follow-up period than a rating of “likely” psychotic experience.7 These data suggest that accurate assessment of psychotic experiences is important in relation to the degree of associated risk. A systematic review of 285 rates of prevalence or incidence of psychotic experiences showed that half of the considerable heterogeneity in rates of subclinical psychotic experiences across studies is due to study cohort and design factors.8 Particularly, rates were found to be much higher in studies using smaller n, convenience sampling, and self-report assessment.

Self-reports of psychotic experiences generate “false-positive” (FP) ratings. Depending on how data are analyzed, the rate of FP self-reported psychotic experiences (SRPEs) when verified by clinical interview may vary from 7%9 to 61%.10 There is evidence, however, that FP in this context does not indicate absence of risk. Thus, Bak and colleagues found that FP psychotic experiences (ie, the presence of Composite International Diagnostic Interview [CIDI] SRPEs that were not confirmed by clinical interview) nevertheless were strongly associated with future psychotic disorder, albeit at a lower level than confirmed psychotic experiences.11 These findings echo those by Poulton and colleagues regarding definite and likely psychotic symptoms and suggest that SRPEs do not come as either “true positive” (TP) or FP. Instead, they may index risk as a continuum reflecting the level of certainty as to what degree the experience that an individual reports can be regarded as “psychotic.”7 Understanding the determinants of FP experiences may help to identify factors related to the expression of mild psychotic experiences and how they eventually lead to need for care.

In the present study, the relationship between SRPEs and assessment of psychosis by clinical interview was therefore investigated. First, positive predictive values (PPVs) of several different psychotic experiences were established. Low PPVs were hypothesized, with higher PPVs for hallucinations than for delusions.10 In line with Bak and colleagues,11 it was further hypothesized that individuals with FP psychotic experiences would be more similar to the group with confirmed psychotic symptoms than to the control group in terms of psychopathology and exposure to environmental risk factors associated with psychotic disorder.12


This study forms part of the recent second Netherlands Mental Health Survey and Incidence Study (NEMESIS), an entirely new longitudinal study of the prevalence, incidence, course, and consequences of psychiatric disorders in the Dutch general population. NEMESIS-2 replicates and extends the first NEMESIS-1 study, conducted from 1996 to 1999,13 in an independent, nonoverlapping sample. The study was approved by the Medical Ethics Review Committee for Institutions on Mental Health Care. For a more detailed description of the NEMESIS-2 method, see De Graaf et al.14,15

Instruments—First Interview

The participants were interviewed at home by trained interviewers who are not clinicians with the CIDI version 3.0.16,17 Demographics, somatic health, life events, treatment-seeking, and different vulnerability factors were also assessed, such as neuroticism, using the Eysenck Personality Questionnaire (the revised short scale)18,19 and childhood trauma (emotional, physical, psychological, and sexual abuse and peer victimization), using self-constructed questionnaires. Cannabis use was assessed in the section Illegal Substance Use of the CIDI 3.0, and analyzed, conform previous analyses in NEMESIS-1,20 as two dichotomous variables indicating lifetime use and regular use. Continuous ratings of general mental and physical health and social functioning were assessed by the Medical Outcomes Study Short-form Health Survey (SF-36).21,22 For a full overview of the assessment instruments in addition to the CIDI 3.0, see De Graaf et al.14

Studies on earlier CIDI versions concluded that the CIDI assesses disorders with generally acceptable reliability and validity, with the exception of psychosis.23,24 As CIDI methodology to assess psychotic experiences in versions of CIDI 1 and CIDI 2 was not included in CIDI 3.0, a psychosis add-on instrument was constructed, based on the section of psychotic symptoms in CIDI versions 1 and 2. This part of the interview consisted of 20 psychotic experiences, each rated “yes,” “no,” “don’t know,” or “refuse,” over the lifetime period. Whenever a psychotic experience was endorsed, the subject was asked to state, on a 1 (rarely) to 4 (almost always) scale, how often this experience occurred (Frequency), how much it bothered them (Distress), and to what extent the experience had an influence on their daily professional and social activities (Impact). The sum scores for frequency and impact of psychotic experiences, as well as distress by psychotic experiences were calculated as the mean of the sum scores of these items across the 20 psychotic experiences. Psychotic experiences were considered secondary if all endorsed psychotic items were caused by use of drugs/alcohol or physical illness. Because clinical relevance of psychotic experiences may be difficult to diagnose by lay interviewers,25,26 and the interviewers made no clinical judgment about participants’ answers, the reported experiences may be considered an extension of “self-report.”

Consistent with work in NEMESIS-127 and other CIDI-based population work,4 a lifetime depression score was obtained by adding up 28 symptom items (present/not present) from the CIDI 3.0 Depression section. Lifetime and past-year mania scores were calculated similarly.4,27

Lastly, participants were asked about help seeking in the context of any psychopathology (help from psychiatrists/psychologists for any psychiatric problem including drug or alcohol problems) and help seeking specifically for psychotic experiences.


Individuals who endorsed at least one lifetime psychotic experience (1078 out of 6646 participants) were contacted for reinterview over the telephone by an experienced clinician at the level of psychologist or psychiatrist within 8 weeks after the initial interview, as in NEMESIS-1.11 Reinterviews were conducted using questions from the Structured Clinical Interview for DSM-IV (SCID-I), an instrument with proven reliability and validity in diagnosing psychotic disorders.28 Findings from all reinterviews were discussed with a second clinician (M.B.), who also conducted and supervised the clinical reinterview in NEMESIS-1.

Sample and Prevalences

The total sample consisted of 6646 participants (response rate 65%; see table 1 for characteristics). As this was a study of relative rather than absolute risk, data were not weighted, and therefore, proportions may be different from weighted estimates of absolute risk presented elsewhere.14,15

Table 1.
NEMESIS-2 Baseline Sample and Nonweighted Prevalences—First Interview and Reinterview

The sample that participated in the reinterview consisted of 792 participants (participation rate: 74%, see table 1). No significant differences existed between those that participated in the clinical reinterview and those that did not with regard to age, lifetime Axis I disorders, gender, educational level, or employment status. However, nonparticipants more often had self-reports of psychotic experiences (mean = 2.5 [95% CI: 2.3–2.8]) compared with participants (mean = 1.9 [95% CI: 1.8–2.0]).


PPVs of SRPEs were calculated with clinical ratings as gold standard, using STATA, version 10.29 Participants were divided into three groups, based on their symptom profiles: (1) participants who did not report any psychotic experience (control group); (2) participants who reported one or more psychotic experiences, none of which were confirmed at clinical interview (FP group); (3) participants who reported one or more psychotic experiences and for whom at least one psychotic experience was confirmed during clinical interview (TP group). Multinomial logistic regression was applied using group as dependent variable, a priori controlling for age and gender. Associations with categorical predictors were expressed as relative risk (RR) ratio, while associations with continuous variables were expressed as B-coefficient, comparing the FP group and the TP group to the controls (the reference group) and comparing the TP group to the FP group (using FP as reference).


Psychotic experiences

The prevalence of at least one SRPE over the lifetime was 16% (1078 out of 6646), similar to the lifetime rate of 18% in NEMESIS-1.30 The prevalence of at least one lifetime true psychotic experience (confirmed by the clinician) was 6% (384 out of 6360), which could not be compared with NEMESIS-1 due to methodological differences.30 The PPVs for delusions were generally lower (20%–50%) than for hallucinations (45%–60%; table 2). The prevalence of psychotic disorder was 0.7% (43 out of 6646). Of the 43 participants with a diagnosis of psychotic disorder, 22 had been available for clinical reinterview with the SCID; the remaining 21 participants received the diagnosis of psychotic disorder based on CIDI interview data only.

Table 2.
Positive Predictive Values for Self-reported Psychotic Experiences in Baseline Assessment of the NEMESIS-2 Sample, Compared Wth Clinical Interview

Group comparisons

The prevalence rates and means of the various psychopathological and psychosocial variables are shown in tables 3 and and4.4. The FP group, similar to the TP group, had significantly higher RRs than the control group for a lifetime mood disorder, a lifetime substance use disorder, as well as for level of neuroticism, regular cannabis use, and childhood sexual trauma (table 5). In the comparison between TP and FP, no significant differences were found for exposure to childhood sexual trauma, negative life events in the past year, lifetime cannabis use, and whether psychotic experiences were secondary to drug or alcohol use or a somatic condition. Even for variables that were significantly different in the comparison between FP and TP groups, RRs were smaller than for the corresponding comparisons between the FP group and controls (table 5). The TP group more often reported help-seeking behavior for psychotic experiences (RR 3.79 [2.27–6.34]) and also more often sought help in the context of any psychopathology than the FP group (RR 1.86 [1.22–2.83]).

Table 3.
NEMESIS-2 Baseline Assessment—Prevalence Rates for Dichotomous Variables, Psychopathology, and Environmental Risk Factors
Table 4.
NEMESIS-2 Baseline Assessment—Means, Minimum, Maximum, and SDs for Continuous Variables; Psychopathology, General Health and Social Functioning, and Severity of Psychotic Experiences
Table 5.
Comparisons of Control Group, FP Group and TP Group in Terms of Psychopathology and Environmental Risk Factors—Relative Risk Ratios and B Coefficients

Compared with the control group, both the FP and the TP groups displayed higher levels of psychopathology, including lifetime depression and lifetime mania (table 5). In addition, they had worse physical health and social functioning (table 5). No significant FP-TP between-group differences were found for manic symptoms in the past year, general mental health, general social functioning, and impact of or distress by the psychotic experiences. The TP group displayed poorer general physical health and higher lifetime depression and mania scores compared with the FP group (table 5). The TP group scored higher on the frequency scale (B FP vs TP: .36 [0.17–0.55], meanFP: 1.65, SDFP: 0.75, meanTP: 1.85, SDTP: 0.76) compared with the FP group. There was no significant difference in age at onset (mean: 26.72, SD: 13.83), but the TP group had more recent experience of psychosis than the FP group (B FP vs TP: .28 [0.19–0.37], meanFP: 1.64, SDFP: 1.44, meanTP: 2.48, SDTP: 1.97). No participants in the FP group met criteria for psychotic disorder, whereas 22 participants in the TP group (6%) did.


The current study found a high proportion of formally FP psychotic experiences in a large general population sample, comparing self-report with clinical interview. The present study also confirms earlier work11 that FP psychotic experiences have clinical and prognostic relevance. Compared with the control group, the FP group was more likely to have mood, anxiety, or substance use disorders, as well as higher levels of neuroticism. They also had higher rates of childhood trauma and peer victimization, were more likely to have experienced a negative life event in the past year, and to have ever used cannabis. They had worse physical and mental health, worse social functioning, and more symptomatic expression of depression and mania. Compared with those with confirmed psychotic experiences, however, associations with psychopathology, social functioning, environmental risk factors, and help seeking were generally weaker.

To the best of our knowledge, this is the first study to investigate the characteristics of individuals with FP psychotic experiences. It confirms the findings of earlier studies that self-report questionnaires for psychotic experiences yield high rates of formally FPs.9,10,31 The PPVs for hallucinations were higher than for delusions, which is also in line with previous research,10 possibly because questions about hallucinations may be less ambiguous than questions about delusions. Importantly, however, the current results also indicate that reporting a FP psychotic experience on a self-report measure may carry relevant risk-related psychometric information about an extended psychosis phenotype that is more likely to remain subclinical but may eventually also lead to need for care, as suggested by Bak and colleagues, who found that FP psychotic experiences predicted the subsequent development of psychotic disorder 3 years later.11 The reported effect sizes may therefore point to a continuum of behavioral expression of risk, as suggested by Poulton and colleagues.7 Indeed, probabilities of other psychopathology and environmental risk factors for psychotic disorder became successively higher with increasing certainty about the presence and nature of the reported experiences. The current study is an important addition to the NEMESIS-1 findings reported by Bak and colleagues as they found that FPs predicted future psychotic disorder but reported no additional characteristics of their FP group.

Compared with the FP group, the psychotic experiences of the TP group were more frequent and more recent. Furthermore, the TP group was about 4 times more likely to seek help for their psychotic experiences. This might represent the crucial difference between the FP and the TP group: the SRPEs were more likely to be confirmed by clinicians when they were more frequent and the individual had sought help for this experience. In a study by Brett and coworkers, evidence was found that underlying unhelpful metacognitive beliefs (that in turn were associated with anxiety and depression) were positively associated with help seeking for psychotic experiences32. In line with these findings, the present study showed that the TP group, compared with the FP group, more often had anxiety and mood disorder diagnoses, which could account for the increased help-seeking behavior.

The results of this study should be interpreted in the context of its strengths and limitations. Strengths are the size and representativeness of the sample. The most important limitation is the lack of information on whether the FP group also has a higher risk of transition to psychotic disorder compared with the control group. This information should become available as part of the ongoing follow-up of the current sample. A further limitation is that the clinical interviews were only conducted in those individuals reporting a possible psychotic experience and that the interview only included questions about these experiences. The consequence of this choice was that there was only information on FP experiences, not false negatives. Therefore, it was not possible to investigate the sensitivity and specificity of the various psychotic experiences. Another limitation was the chosen method of reinterviews: because it was not feasible to visit the participants with SRPEs on a second occasion for the purpose of a clinical interview, it was chosen to conduct the reinterview over the telephone. It is possible that subtle nonverbal cues have been missed that would have been picked up in a face-to-face interview. However, this method was also used in NEMESIS-1 and findings from these telephone11 interviews reliably predicted future psychotic disorder 11, thus supporting its reliability.

In spite of these limitations, the present findings provide important clues about the characteristics of a subgroup of individuals presenting with experiences that may represent the mildest subthreshold expression of psychosis. Further study of this group may help to identify biological, psychological, and social processes underlying the first expression of psychotic symptoms, the persistence of these over time, and eventually, development of need for care.


Ministry of Health, Welfare and Sport (310253); Netherlands Organization for Health Research and Development (ZonMw), Genetic Risk and Outcome of Psychosis (GROUP) investigators.


The Netherlands Mental Health Survey and Incidence Study-2 (NEMESIS-2) is conducted by the Netherlands Institute of Mental Health and Addiction (Trimbos Institute) in Utrecht. The reinterviews are conducted by the University of Maastricht, Department of Psychiatry and Psychology.


1. Van Os J, Linscott RJ, Myin-Germeys I, Delespaul P, Krabbendam L. A systematic review and meta-analysis of the psychosis continuum: evidence for a psychosis-proneness-persistence-impairment model of psychotic disorder. Psychol Med. 2009;39:179–195. [PubMed]
2. Dominguez MD, Wichers M, Lieb R, Wittchen HU, van Os J. Evidence that onset of clinical psychosis is an outcome of progressively more persistent subclinical psychotic experiences: an 8-year cohort study. Schizophr Bull. 2011;37:84–93. [PMC free article] [PubMed]
3. Dominguez MD, Saka MC, Lieb R, Wittchen H, van Os J. Early expression of negative/disorganized symptoms predicting psychotic experiences and subsequent clinical psychosis: a 10-year study. Am J Psychiatry. 2010;167:1075–1082. [PubMed]
4. Tijssen MJA, van Os J, Wittchen HU, et al. Prediction of transition from common adolescent bipolar experiences to bipolar disorder: a 10 year study. Br J Psychiatry. 2010;196:102–108. [PubMed]
5. Bak M, Myin-Germeys I, Hanssen M, et al. When does experience of psychosis result in need for care? A prospective general population study. Schizophr Bull. 2003;29:349–358. [PubMed]
6. Werbeloff N, Drukker M, Dohrenwend BP, et al. Self-reported psychotic symptoms in the community are associated with increased risk of later hospitalization for non-affective psychotic disorders (conference abstract) Schizophr Bull. 2009;35(suppl 1):74.
7. Poulton R, Caspi A, Moffitt TE, Cannon M, Murray R, Harrington H. Children's self-reported psychotic symptoms and adult schizophreniform disorder: a 15 year longitudinal study. Arch Gen Psychiatry. 2000;57:1053–1058. [PubMed]
8. Linscott RJ, Van Os J. Systematic reviews of categorical versus continuum models in psychosis: evidence for discontinuous subpopulations underlying a psychometric continuum. Implications for DSM-V, DSM-VI, and DSM-VII. Annu Rev Clin Psychol. 2010;6:391–419. [PubMed]
9. van Os J, Hanssen M, Bijl RV, Vollebergh W. Prevalence of psychotic disorder and community level of psychotic symptoms. Arch Gen Psychiatry. 2001;58:663–668. [PubMed]
10. Kelleher I, Harley M, Murtagh A, Cannon M. Are screening instruments valid for psychotic-like experiences? A validation study of screening questions for psychotic-like experiences using in-depth clinical interview. Schiophr Bull. 2009;37:362–369. [PMC free article] [PubMed]
11. Bak M, Delespaul P, Hanssen M, de Graaf R, Vollebergh W, van Os J. How false are “false” positive psychotic symptoms? Schizophr Res. 2003;62:187–189. [PubMed]
12. van Os J, Kenis G, Rutten BP. The environment and schizophrenia. Nature. 2010;468:203–212. [PubMed]
13. Bijl RV, van Zessen G, Ravelli A, de Rijk C, Langendoen Y. The Netherlands Mental Health Survey and Incidence Study (NEMESIS): objectives and design. Soc Psychiatry Psychiatr Epidemiol. 1998;33:581–586. [PubMed]
14. De Graaf R, ten Have M, van Dorsselaer S. The Netherlands Mental Health Survey and Incidence study-2 (NEMESIS-2): design and methods. Int J Method Psychiatr Res. 2010;19:125–141. [PubMed]
15. De Graaf R, Ten Have M, van Gool C, Van Dorsselaer S. Prevalence of mental disorders and trends from 1996 to 2009. Results from the Netherlands Mental Health Survey and Incidence Study-2. Soc Psychiatry Psychiatr Epidemiol. doi:10.1007/s00127-010-0334-8. [PubMed]
16. Alonso J, Angermeyer M, Bernert S, et al. Sampling and methods of the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr Scand. 2004;109(suppl 420):8–20. [PubMed]
17. De Graaf R, Ormel J, Ten Have M, Burger H, Buist-Bouwman M. Mental disorders and service use in the Netherlands. Results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) In: Kessler RC, Üstün TB, editors. The WHO World Mental Health Surveys: Global Perspectives on the Epidemiology of Mental Disorders. Cambridge, UK: Cambridge University Press; 2008. pp. 388–405.
18. Eysenck SBG, Eysenck HJ, Barrett P. Revised version of the psychoticism scale. Person Ind Diff. 1985;6:21–29.
19. Eysenck SBG, White O, Eysenck HJ. Personality and mental illness. Psychol Rep. 1976;39:1011–1022. [PubMed]
20. Van Os J, Bak M, Hanssen M, Bijl RV, De Graaf R, Verdoux H. Cannabis use and psychosis: a longitudinal population-based study. Am J Epidemiol. 2002;156:319–327. [PubMed]
21. Stewart AL, Hayes RD, Ware JE. The MOS short form general health survey. Med Care. 1988;26:724–735. [PubMed]
22. Ware JE, Sherbourne CD. The RAND-36 Short-form Health status Survey: 1: conceptual framework and item-selection. Med Care. 1992;30:473–481. [PubMed]
23. Andrew G, Peters L. The psychometric properties of the Composite International Diagnostic Interview. Soc Psychiatry Psychiatr Epidemiol. 1998;33:80–88. [PubMed]
24. Wittchen HU. Reliability and validity studies of the WHO-Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res. 1994;28:57–84. [PubMed]
25. Helzer JE, Robins LN, McEvoy LT, et al. A comparison of clinical and diagnostic interview schedule diagnoses. Physician reexamination of lay-interviewed cases in the general population. Arch Gen Psychiatry. 1985;42:657–666. [PubMed]
26. Cooper SA, Collacott RA. Clinical features and diagnostic criteria of depression in Down's syndrome. Br J Psychiatry. 1994;165:399–403. [PubMed]
27. Krabbendam L, Myin-Germeys I, de Graaf R, et al. Dimensions of depression, mania and psychosis in the general population. Psychol Med. 2004;34:1177–1186. [PubMed]
28. Spitzer RL, Williams JB, Gibbon M, First MB. The Structured Clinical Interview for DSM-III-R (SCID). I: history, rationale, and description. Arch Gen Psychiatry. 1992;49:624–629. [PubMed]
29. Stata Corporation. Statistical Software: Release 10.0. [computer program]. Version. College Station, TX. 2007
30. van Os J, Hanssen M, Bijl RV, Ravelli A. Strauss; revisited: a psychosis continuum in the general population? Schizophr Res. 2000;45:11–20. [PubMed]
31. Hanssen MSS, Bijl RV, Vollebergh W, van Os J. Self-reported psychotic experiences in the general population: a valid screening tool for DSM-III-R psychotic disorders? Acta Psychiatr Scand. 2003;107:369–377. [PubMed]
32. Brett CMC, Johns LC, Peters EP, McGuire PK. The role of metacognitive beliefs in determining the impact of anomalous experiences: a comparison of help-seeking and non-help-seeking groups of people experiencing psychotic-like anomalies. Psychol Med. 2009;39:939–950. [PubMed]

Articles from Schizophrenia Bulletin are provided here courtesy of Oxford University Press