|Home | About | Journals | Submit | Contact Us | Français|
Stroke patients frequently experience medical complications; yet, data on incidence, causes, and consequences are sparse.
To examine the data validity of medical complications among patients with stroke in a population-based clinical registry and a hospital discharge registry.
We examined the predictive values, sensitivity and specificity of medical complications among patients admitted to specialized stroke units and registered in the Danish National Indicator Project (DNIP) and the Danish National Registry of Patients (NRP) between January 2003 and December 2006 (n = 8,024). We retrieved and reviewed medical records from a random sample of patients (n = 589, 7.3%).
We found substantial variation in the data quality of stroke-related medical complication diagnoses both within the specific complications and between the registries. The positive predictive values ranged from 39.0%–87.1% in the DNIP, and from 0.0%–92.9% in the NRP. The negative predictive values ranged from 71.6%–98.9% in the DNIP and from 63.3% to 97.4% in the NRP. In both registries the specificity of the diagnoses was high. The sensitivity ranged from 23.5% (95% confidence interval [CI]: 14.9–35.4) for falls to 62.9% (95% CI: 54.9–70.4) for urinary infection in the DNIP, and from 0.0 (95% CI: 0.0–4.99) for falls to 18.1% (95% CI: 2.3–51.8) for pressure ulcer in the NRP.
The DNIP may be useful for studying medical complications among patients with stroke.
Stroke patients frequently experience medical complications such as pneumonia, urinary infection, pressure ulcer, falls, venous thromboembolism, and severe constipation. These complications may hinder optimum rehabilitation and have been associated with increased mortality.1–6 However, the available data on incidence, causes, and consequences of serious medical complications are sparse and inconclusive.1–5
Large-scale population-based studies, reflecting real-life conditions in modern stroke care, are much needed. Primary data collection is often time-consuming and costly; consequently, it is often only done on a smaller scale. Furthermore, there is a likelihood of bias due to recall, nonresponse, and effects on the diagnostic process as a result of the research question.6–8 Clinical and administrative registries are possible alternatives to primary data collection, they have the advantage of readily available data and often contain complete registration of information on people in the target population. Registries are generally valuable tools for answering clinical, administrative, and research questions and may also be useful for studying medical complications in patients with stroke; however, documented reasonable data validity is a prerequisite for using such data sources.
To our knowledge, information on the validity of registry data on medical complications among stroke patients has not been reported previously. Therefore, we examined the data validity of medical complications among stroke patients in two population-based Danish registries: the Danish National Indicator Project (DNIP) and the Danish National Registry of Patients (NRP).
The Danish healthcare system provides tax-supported health care to the country’s 5.5 million residents, all of whom have free access to hospital care and general practitioners. Patients with acute medical conditions, including stroke, are exclusively admitted to public hospitals. The entire population is covered by a large number of administrative and medical registries, which are used for monitoring and regulating all central aspects of the public sector including the health care system. Through the use of a civil registry number, which is unique to every Danish citizen and encodes sex and date of birth, unambiguous linkages between the population-based registers can be made.9
In 2000, the DNIP was established as a nationwide initiative to monitor and improve the quality of treatment and care provided by the Danish health care system for specific diseases, including stroke. The project develops evidence-based quality criteria related to the structure, process, and outcome of health care and monitors the fulfillment of these criteria. Project participation is mandatory for all Danish hospitals, relevant clinical departments, and units treating patients with stroke.10 Upon hospital admission, data on care and patient characteristics (eg, age, sex, marital status, housing, Scandinavian Stroke Scale score, history of stroke or myocardial infarction, previous and/or current atrial fibrillation, hypertension, diabetes mellitus or intermittent claudication, smoking habits, and alcohol intake) are collected for all patients admitted with stroke. Detailed written instructions are available to the staff to ensure the validity of the data collected and completeness of patient registration in the DNIP. After hospital discharge the data are entered into a central database. A structured audit process is carried out regularly (every year) on a national, regional, and local basis to assess critically the quality of the dataset and results. After the audit process is completed the data are released publicly, including comments on the results from the audit groups. To ensure completeness of patient registration in DNIP, its enrollees are compared with local hospital discharge registries.10
The NRP is an administrative nationwide public registry that covers all discharges from somatic hospitals in Denmark since January 1, 1977. The data include the civil registry number, which is unique to every Danish citizen, the dates of admission and discharge, the surgical procedure(s) performed, and up to 20 diagnoses for every discharge classified, since 1994, according to the Danish version of the International Classification of Diseases, 10th edition (ICD-10). All discharge diagnoses are assigned by the physician who discharges the patient. Reporting of all hospital contacts to the NRP is mandatory by law. Data reported to the NRP are also used for classifying the patients in relation to Diagnosis-related groups (DRG), which determines the financial compensation given to the hospital for treating the individual patients.11
We identified all admissions for acute stroke, from 10 specialized stroke units in the former Copenhagen Hospital Corporation and Aarhus County, registered in the DNIP from January 13, 2003 to December 31, 2006. All patients (≥18 years) admitted to Danish hospitals with stroke, as defined by WHO criteria, ie, an acute disturbance of focal or global cerebral function with symptoms lasting more than 24 hours or leading to death of presumed vascular origin, are eligible for inclusion in the DNIP database. This includes patients with intracerebral hemorrhage and ischemic stroke (ICD-10: I63, infarction; I61, hemorrhage; and I64, unspecified). Patients with subarachnoidal or epidural hemorrhage, subdural hematoma, retinal infarct, and infarct caused by trauma, infection, or an intracranial malignant process are excluded. Patients with diffuse symptoms, such as isolated vertigo or headache, and asymptomatic patients with infarct detected only by computed tomography (CT) or magnetic resonance imaging (MRI) scans are also excluded.
Although some patients had multiple events, we included only the first stroke event registered in the DNIP during the study period. Furthermore, only patients with a valid civil registry number (a unique personal identification number issued to all Danish citizens that allows unambiguous linkage between public registries) were included in the study. In total, we identified 8,024 (99.5%) admissions in the registry which were eligible for inclusion in our study population.
During hospital admission, detailed data on each patient are prospectively registered using a standardized form. The data includes the presence (and if so, date of diagnosis) or absence of the following seven medical complications: pneumonia, urinary infection, pressure ulcer, falls, deep venous thrombosis (DVT), pulmonary embolism (PE), and constipation. Only complications that occurred after hospital admission are registered.
Information on medical complications during the hospital admission was obtained through linkage to the NRP. We retrieved data on the following ICD-10 discharge diagnoses: pneumonia (DJ12–DJ18), urinary infection (DN30.0, DN30.8, DN30.9, and DN10), pressure ulcer (DL899), falls (DR297 and EUHE), DVT (DI82.9, DI82.9A-E), PE (DI26), and constipation (DK590). Both primary diagnoses, the condition, which is the principal reason for the hospital admission, and secondary diagnoses, all other clinically relevant diagnoses, including any significant complications and symptoms that appear during the admission, were identified from the registry.
We aimed to retrieve and review medical records from a random sample of approximately 5% of patients in the defined study population (n = 417). A computer-generated 5% random sample was obtained from the entire study population, stratified according to the 10 stroke units serving the study population. Additionally, we retrieved and reviewed 25 medical records for each of the seven types of complications registered in the DNIP. The 25 extra medical records were selected randomly among all patients registered with medical complications in the DNIP. The later records were added to ensure a reasonable statistical precision of the estimated positive predictive value for relatively rare complications. In total, 589 records were retrieved and reviewed.
All medical records were retrieved and reviewed by a single reviewer (AI) using a standardized form. The form was developed and tested in close collaboration with 2 consultants in neurology and clinical epidemiology (GA, SPJ). The criteria used to define the presence of a complication were in accordance with existing international defined criteria or, if such criteria were not available, defined a priori by consensus among the authors.
The diagnostic criteria are presented in Table 1. All cases with uncertain diagnosis, based on the available information, were discussed with the consultants. The review was based on all available information in the medical records including written radiology reports, results from laboratory tests, and the nurse records. The actual brain imaging films were not re-interpreted. The study was approved by The Danish Data Protection Agency (J.no. 2007-41-0563) and The National Board of Health (J.no. 7-604-04-2/26/EHE).
The positive and negative predictive values of the registered complications in the DNIP and the NRP were assessed using the review of the medical records as the gold standard. Positive predictive values of the complications in the registries were computed as proportions with the number of patients with a verified complication, after review of medical records using the diagnostic criteria in Table 1, as the numerator and the total number of patients registered in the DNIP/NRP with this specific complication as the denominator. Negative predictive values of the complications in the registries were computed as proportions with the number of patients verified to not have the specific complication, after review of medical records using the diagnostic criteria in Table 1, as the numerator and the total number of patients registered in the DNIP/NRP without this specific complication as the denominator. We computed the sensitivity of the complications as the proportion of patients with a verified complication registered in the DNIP/NRP divided by the total number of patients in our random sample with a complication in the medical record. Specificity of the complications was similarly defined as the number of patients without any verified complication registered in the DNIP/NRP divided by the total number of patients in our random sample without a complication in the medical record.
Positive and negative predictive values, the sensitivity, and the specificity were calculated for all complications combined and for the specific complications (pneumonia, urinary infection, pressure ulcer, falls, DVT, PE, and constipation). Finally, we estimated the proportion of patients who had at least one symptom of a complication registered in their medical record among patients registered in the DNIP with the complication.
For all estimates, a 95% confidence interval (CI) was calculated. Data were analyzed using STATA® version 10.0 (StataCorp, College Station, TX, USA).
Table 2 shows demographic and clinical characteristics of the 8,024 patients with acute stroke registered in the DNIP.
Table 3 shows the number of medical complications registered in the DNIP and NRP among the patients. During hospitalization, a total of 25.4 % (n = 2,039) of the patients were registered with at least one medical complication in the DNIP. The most common complications in the DNIP were urinary infection in 15.5% (n = 1,240), pneumonia in 8.4% (n = 670), and constipation in 6.8%, (n = 544) of the patients; whereas DVT and PE were only registered in 0.4% (n = 32 and n = 28, respectively) of the patients.
Table 4 presents the positive and negative predictive values of all the reviewed medical complications in the DNIP and NRP. We found the overall positive predictive value for the medical complications registered in the DNIP was 71.7% (95% CI: 67.4–75.8) compared to 76.1% (95% CI: 75.9–84.6) in the NRP. The positive predictive values in the DNIP ranged from 39.0% (95% CI: 24.2–55.5) for pressure ulcer to 87.1% (95% CI: 78.0–93.4) for pneumonia. In the NRP, there was even more variation in the positive predictive values, which ranged from 0.0 % (95% CI: 0–97.5) for falls to 92.9% (95% CI: 66.1–99.8) for pneumonia. The overall negative predictive value registered in the DNIP was 91.3% (95% CI: 90.3–92.2). The negative predictive values in the DNIP ranged from 71.6% (95% CI: 67.2–75.7) for pneumonia to 98.9% (95% CI: 97.5–99.6) for pulmonary embolism. The overall negative predictive value for the medical complications registered in the NRP was 85.1% (95% CI: 83.9–86.1), the values ranged from 63.3% (95% CI: 59.2–67.3) for pneumonia to 97.4% (95% CI: 95.8–98.5) for pulmonary embolism.
Table 5 shows the sensitivity and specificity of the medical complications in the DNIP and NRP. The estimates are based on a 5% random sample of all patients with stroke registered in DNIP in the study period.
The overall sensitivity of the seven medical complications in the DNIP and NRP was 40.9% (95% CI: 36.6–45.4) and 7.7% (95% CI: 5.6–10.3), respectively. The lowest sensitivity in both registries was found for falls (ie, 23.5% [95% CI: 14.9–35.4] in the DNIP and 0.0% [95% CI: 0.0–4.99] in the NRP). The highest sensitivity in the DNIP was for urinary infection, 62.9% (95% CI: 54.9–70.4); whereas, the highest in the NRP was for pressure ulcer, 18.1% (95% CI: 2.3–51.8).
The overall specificity of the seven medical complications was 97.3% (95% CI: 96.7–97.8) in the DNIP and 99.5% (95% CI: 99.2–99.7) in the NPR. The specificity ranged from 93.4% (95% CI: 91.0–95.8) for urinary infection to 99.8% (95% CI: 98.9–100) for PE in the DNIP. In the NRP, the specificity ranged from 98.0% (95% CI: 96.1–99.1) for urinary infection to 100% (95% CI: 99.9–100) for PE. In a sub analysis (data not shown), we found that of the 130 complications registered in DNIP that were not verified during the review of the medical records, 57% (95% CI: 48.7–66.3) (n = 75) of these patients had a least one or more symptoms registered in their medical record which indicated the presence of a medical complication, but was not sufficient to fulfill the diagnostic criteria listed in Table 1.
We found substantial diagnosis- and register-specific variation when validating the data quality of seven stroke-related medical complications registered in public Danish registries. The positive and negative predictive values were generally moderate to high for most complications in the two registries. In addition, the specificity of the medical complication diagnoses was high for all complications in the two registries. In contrast, major differences were found in the two registries for the sensitivity of the medical complication diagnoses. The sensitivity for all types of complications was moderate to low in the DNIP, whereas it was extremely low in the NRP.
The strengths of our study included the population-based design, the access to medical records with detailed clinical data, and the standardized evaluation of the records including the use of well-defined diagnostic criteria developed in accordance with available existing literature and/or international consensus. The use of a single reviewer, who was not blinded for the diagnosis in DNIP, of the medical records was a potential methodological limitation as there exist a possibility of error in extracting information from the medical records. The use of retrospective review of medical records may also be a limitation as the collection and the quality of the data are not under control of the researcher (eg, the required information to make a valid conclusion about the presence of specific medical complications were always available in the medical records).
However, detailed diagnostic criteria and a standardized form were used to reduce any risk of bias in the data collection. Furthermore, all cases with uncertain diagnosis, based on the available information, were discussed with senior consultants. Although the number of reviewed medical records was relatively large compared to other validation studies based on detailed review of medical records, some of the complications (eg, deep venous thrombosis and pulmonary embolism) were rare with relatively few cases available for assessment. Consequently, the precision of the estimated predictive values, sensitivity, and specificity for these complications were only moderate. Finally, it is not known to what extent our findings can be extrapolated to other settings, where the prevalence of complications and the diagnostic strategy may be different.
We are unaware of other studies validating the quality of diagnoses of medical complications among patients with stroke in administrative and clinical registries. There may be different possible explanations for the low sensitivity we found in the NRP. However, the fact that there is no financial incentive for the stroke units to register medical complications in NRP due to the DRG system used in Denmark (ie, medical complications do not result in additional compensation to the hospital), is likely to be a major factor. Furthermore, registration of data to the NRP is done by physicians, whereas nurses are much more involved in the reporting of data to the DNIP. It is our impression that the nurses in general are much more aware of registration of medical complications, which may explain the higher sensitivity found in DNIP.
However, our finding of moderate to high positive predictive values in both registries are generally in accordance with the results from a number of validation studies concerning medical complications in different patient populations (eg, unselected internal medicine patients, pregnant women, patients from geriatric wards, general practice, or with a former cancer diagnosis).12–22 The positive predictive values reported from these studies have ranged from 20.7% for pressure ulcer to 96.2% for pneumonia.12–15,17–22 Only a few studies have previously reported the sensitivity and specificity of diagnoses of medical complications in registries. Quan et al examined diagnoses of complications in Canadian administrative hospital discharge data and found that the sensitivity ranged from 0% to 57.1% (higher than 50% for only two conditions). In contrast, specificity was generally high (range: 99.0%–100%).16 These results are very consistent with our findings and also in line with findings from validation studies on other diagnoses not related to medical complications.23,24 The implications of the often low sensitivity and high specificity of diagnoses in administrative and clinical registries depend on the research question and the design of the study.8 A low sensitivity would result in misleading rates in a study determining incidence (eg, of medical complications among patients with stroke), whereas analytical studies examining associations would likely produce unbiased relative risk estimates due to the high specificity.25 Thus, in analytical studies, a low specificity would be far more troublesome in most situations than a low sensitivity.23 The fact that half of the complications not verified had a least one or several symptoms of the specific complications registered in the medical record supports this and indicates data on complications in the DNIP are valid, although the international criteria of the specific complication were not completely fulfilled.
In conclusion, we found moderate to high predictive values for medical complication diagnoses among patients with stroke in two population-based registries. However, the sensitivity varied substantially between the two registries with the stroke registry having a much higher sensitivity compared to the hospital discharge registry. The specificity of the diagnoses was high in both registries. These findings indicate that data from the DNIP may be useful for studying medical complications in patients with stroke, particularly for analytical studies.
This study was supported by The Danish Heart Foundation and the Central Denmark Research Foundation. The authors report no conflicts of interest in this work.