|Home | About | Journals | Submit | Contact Us | Français|
The Israeli National Psychiatric Hospitalization Registry is a nationwide list of all psychiatric hospitalizations in the country and has been widely used as a source of data for psychiatric research. This study assessed the sensitivity of the diagnosis of psychotic disorders (International Statistical Classification of Diseases, 10th Revision [ICD-10] F20.0–F29.9) and schizophrenia (ICD-10 F20.0–F20.9) in the Registry.
Registry discharge diagnoses of psychotic disorders (ICD-10 F20.0–F29.9) and schizophrenia (ICD-10 F20.0–F20.9) were compared with research diagnoses derived from best-estimate procedures based on Research Diagnostic Criteria (RDC) using structured clinical research interviews, hospital records, and family information.
Out of 169 patients meeting RDC for psychotic disorder, 150 also had a diagnosis of psychotic disorders in the Registry, yielding a sensitivity of 0.89.
Re-running this analysis for the narrow definition of schizophrenia identified 94 patients who were diagnosed with schizophrenia using RDC; 82 of those patients also had a diagnosis of schizophrenia in the Registry, yielding a sensitivity of 0.87.
In 87% to 89% of cases with psychotic disorders or with schizophrenia, Registry diagnoses agreed with RDC diagnoses, a rate of agreement comparable with those of other, similar registries. Because a large number of analyses derived from this and similar national registries will be published in the coming years, this constitutes relevant information.
Research using clinical databases can be a valuable source of information, and psychiatric hospitalization registries have been a particularly rich source of research data [1–5]. The Israeli Ministry of Health maintains a national psychiatric hospitalization registry that includes diagnostic information for all psychiatric hospital admissions in the country since 1950. This registry is a rich source of psychiatric data and has been used in many studies [6–19].
One obvious caveat that arises when using clinical registries for research purposes is the accuracy of the diagnoses that they contain. Because the diagnoses are made on the basis of signs, symptoms, and history and not as research diagnoses, concerns regarding the validity and reliability of the diagnoses are pertinent.
Because most patients hospitalized suffer from schizophrenia and psychotic disorders, and as most of the studies using the Registry focused on these disorders, the purpose of this study was to evaluate the sensitivity of the diagnoses of schizophrenia and psychotic disorders. This was done by comparing best-estimate research diagnoses (obtained using a structured diagnostic research interview and treated as the gold standard) with Registry diagnoses for 169 cases.
The subjects were 169 patients who had been hospitalized in psychiatric wards throughout Israel, who as part of clinical research projects had undergone a structured research diagnostic assessment. The patients were drawn from 2 separate clinical samples, the first being a sample of Jewish Israeli patients that was cross-sectionally recruited from admissions to the Department of Psychiatry, Hadassah-Hebrew University Medical Center and from referrals from other centers. The second sample consisted of Arab Israeli patients who were systematically recruited from the rosters of regional mental health centers. Only those patients who had been hospitalized were included in the current study. The research diagnostic assessments were conducted by the research team of the Biological Psychiatry Laboratory, Hadassah-Hebrew University Medical Center as part of their ongoing studies on the genetics of major psychiatric disorders. The studies had been approved by the Helsinki Committee (Internal Review Board) of the Hadassah-Hebrew University Medical Center and all subjects gave written informed consent after the project had been explained to them . Subjects were 112 men (mean age at interview = 34.5 ± 11.1 years, mean duration of illness = 10.0 years) and 57 women (mean age at interview = 37.5 ± 10.0 years, mean duration of illness = 10.7 years). The subjects were 89 Israeli Jews and 78 Israeli Arabs. Data regarding the ethnic group of 2 subjects were missing.
Subjects had been interviewed with the Schedule for Affective Disorders and Schizophrenia—Lifetime Version (SADS-L)  and were questioned about psychiatric symptoms in the family according to the Family History Research Diagnostic Criteria . Medical records of hospitalizations and clinic care were obtained. The completed SADS-L interview form, Family History Research Diagnostic Criteria information, and medical records were reviewed by 2 psychiatrists according to a best-estimate procedure  and in cases where consensus was not achieved, the conflict was resolved by a senior psychiatric researcher (BL). Lifetime diagnoses were established according to the Research Diagnostic Criteria (RDC)  using a best-estimate consensus procedure .
Registry diagnoses were compared with research diagnoses twice: first the Registry's last discharge diagnosis was compared with the research diagnosis. The second comparison of Registry diagnoses with research diagnoses was done by comparing the research diagnosis with the Registry discharge diagnosis that was assigned to the patient in the hospitalization that was closest in date to the research assessment. Because of missing values of the dates of the research assessment, this comparison was done for 148 subjects.
After obtaining approval from the local Internal Review Board, the Israeli identification numbers (similar to American Social Security numbers) of the subjects who had undergone the SADS-L assessment and were assigned a RDC diagnosis were sent to the Registry, where their ICD-10 Registry diagnosis was obtained. Before the file containing the Registry diagnoses and the RDC diagnoses was transferred from the managers of the Registry to the investigators, the identification numbers were removed, thus keeping the identification of the subjects confidential. For those patients who had been hospitalized more than once, discharge diagnoses from all hospitalizations were obtained.
To compare the diagnoses of psychotic disorders from the Registry with the RDC research diagnoses, we took 169 patients who met the criteria for RDC diagnosis of psychotic disorders (schizophrenia, schizo-affective disorder or unspecified functional psychosis) and recoded their diagnoses into yes/no for having psychotic disorders. For these patients, the Registry diagnoses were also recoded as yes/ no for having psychotic disorders, using ICD-10 categories of F20.0 to F29.9. One patient from the Registry with a diagnosis of schizotypal personality disorder (ICD-10 F21) was recoded as “no.”
In addition, we compared the RDC diagnosis of schizophrenia with the Registry diagnoses of schizophrenia. The research diagnosis of schizophrenia included all patients who after a SADS-L interview met RDC for schizophrenia, and the Registry diagnosis of schizophrenia included patients with ICD-10 diagnoses F20.0 to F20.9.
We used the RDC diagnosis as the “gold standard” to which the Registry diagnoses were compared, and the measure of sensitivity is used as the index of agreement between the RDC and the Registry's diagnosis. The sensitivity of the diagnosis was calculated as the proportion of the patients who were diagnosed by RDC as suffering from psychotic disorders or from schizophrenia, who received the same diagnosis from the Registry.
To rule out the presence of bias related to the presence of different ethnic groups in the sample, we compared the sensitivity of the Registry diagnoses between Arab Israelis and Jewish Israelis using Pearson's χ2 test.
Nineteen patients were diagnosed as suffering from a psychotic disorder according to RDC, but had not received a diagnosis of psychotic disorder in the Registry. We compared these patients with those who were diagnosed with a psychotic disorder both using RDC, and in the Registry; differences in gender were compared using Pearson's χ2 test and age at first hospitalization using independent-sample t tests.
Out of 169 patients diagnosed as suffering from psychotic disorders using RDC, 150 also had a last discharge diagnosis of psychotic disorders in the Registry, yielding a sensitivity of 0.89.
The data were reanalyzed for the narrow diagnosis of schizophrenia, defined as ICD-10 diagnoses F20.0 to F20.9. Ninety-four out of 169 patients were diagnosed as suffering from schizophrenia using RDC, and 82 out of 169 also had a Registry last discharge diagnosis of schizophrenia, yielding a sensitivity of 0.87.
Out of the 75 participants who were diagnosed as suffering from psychotic disorders but not with schizophrenia using RDC, 71 participants were diagnosed with schizo-affective disorder, and the 4 additional participants were diagnosed with unspecified functional psychosis.
The sensitivity of diagnoses was computed again for the hospitalization that was closest to the date that the research assessment was performed and yielded similar results: 0.89 for psychotic disorders and 0.90 for the narrow diagnosis of schizophrenia.
To rule out the presence of bias related to the presence of different ethnic groups in the sample, we compared the sensitivity of the Registry diagnoses between Arab Israelis (psychotic disorders, 0.93; schizophrenia, 0.88) and Jewish Israelis (psychotic disorders, 0.89; schizophrenia, 0.86). These differences were not significant (Pearson's χ2 = 1.45, P = .229).
Nineteen patients were diagnosed as suffering from a psychotic disorder according to RDC, but had not received a diagnosis of psychotic disorder in the Registry. Fourteen of these patients were diagnosed as suffering from affective disorders (ICD diagnosis F30), 4 from substance abuse disorders (ICD diagnosis F10), and 1 patient was diagnosed as suffering from a personality disorder (ICD diagnosis F60).
Compared with the entire cohort, these patients did not differ in their mean age at first hospitalization (25.07 ± 8.53 vs 24.67 ± 14.61, t = 0.175, P = .862) or gender distribution (31.6% female and 68.4% male vs 33.7% female and 66.3% male in the entire cohort, χ2 = 0.044, P = .833).
The main finding of this study is that the diagnoses of psychotic disorders and schizophrenia which appear in the Israeli National Psychiatric Hospitalization Registry exhibit high sensitivity, reaching 0.89 for psychotic disorders and 0.90 for the narrow definition of schizophrenia. These results are congruent with research on the reliability of hospitalization Registry diagnoses from other registries. A UK-based study showed an 85% concordance between psychiatric case register and International Statistical Classification of Diseases, Ninth Revision, diagnoses . A study based on the Swedish National Inpatient Registry yielded an 86% concordance with the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria of the broadly defined schizophrenia syndrome and 76% for the “narrow” definition of the syndrome . Isohanni et al studied the diagnostic reliability of the Finnish National Hospital Discharge Register and discovered a high percentage of true positives (87%–100%) . A study very similar to ours  evaluated the agreement between discharge diagnosis and best-estimate RDC diagnosis. The sensitivity for diagnosis of schizophrenia was found to be 0.62 and the specificity 0.84. However, this study focused on patients in their first psychotic episode, from which diagnoses are known to change. Another study  comparing hospital diagnoses with research diagnoses reported sensitivity of 0.53 to 0.66 and specificity of 0.69 to 0.96 for schizophrenia, schizophreniform, and schizo-affective disorders. Thus, the accuracy of the diagnoses in the Israeli Hospitalization Registry seems to be very much in line with other clinical diagnoses.
A study design that might be preferable to the design used in this study would take a random selection of patients diagnosed with psychotic disorders or schizophrenia in the Registry and diagnose them using RDC. However, there is no reason to think that the patients that had undergone the diagnostic assessments as part of other research protocols would be effected by any particular bias, and their mean ages and duration of illness are compatible with those of the chronic mental patients which comprise most patients in the Registry. Moreover, the results of this study point to the fact that the Israeli Hospitalization Registry is not affected by an ethnic group bias.
Another potential limitation is that it is possible that the 169 cases that made up the research sample were unusually “classic” because they were ascertained as part of ongoing studies of the genetics of major psychiatric disorders. Furthermore, the agreement might have been somewhat inflated because the best-estimate diagnosticians had access to hospital records as well as the SADS-L interviews and family history information.
The use of face-to-face diagnostic interviews, family history information, medical records, and a best-estimate diagnostic procedure is a strength of this study, compared to others that assessed Registry diagnoses relying on chart review alone.
Another strength is the use of the RDC best-estimate procedure as a gold standard. Previous studies have shown that the reliability and validity of the best-estimate diagnosis procedure are very high [30,31].
The study's design prevented us from using additional measures of concordance that are common to many validation studies, namely, the measure of specificity and the k statistic. The main reason for using sensitivity as our only concordance measure was that the participating subjects were selected with an a priori RDC diagnosis of psychotic disorders. Specificity, which measures the proportion of subjects not diagnosed by both raters as suffering from each disorder, was not an appropriate measure in this case. Similarly, k, which is influenced by the unequal proportions of base-rate diagnoses and the resulting high proportion of chance agreement , was not used in this study because of the biased low results it would have yielded.
The diagnosis of psychotic disorders and schizophrenia in the Israeli National Psychiatric Registry is comparable to those of other hospitalization registries.
Because a large number of analyses derived from this and similar national registries will be published in the coming years, this constitutes relevant information.
This study was supported in part by the Stanley Medical Research Foundation (MD and MW); NARSAD (MD and MW); the German Israeli Foundation for Scientific Research, Jerusalem, Israel and Bonn, Germany (GIF, No. I-579-069.02/98) and the Israel Ministry of Science (Korean-Israeli Research Cooperation) (BL); the German Federal Ministry of Education and Research (BMBF) within the framework of German Israeli Project Cooperation (DIP) (MD), Bonn, Germany; and NIMH R01 MH59114 (MD), Bethesda, MD.