PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmchsrBioMed Centralsearchsubmit a manuscriptregisterthis articleBMC Health Services Research
 
BMC Health Serv Res. 2017; 17: 666.
Published online 2017 September 18. doi:  10.1186/s12913-017-2612-6
PMCID: PMC5604304

Moderate sensitivity and high specificity of emergency department administrative data for transient ischemic attacks

Abstract

Background

Validation of administrative data case definitions is key for accurate passive surveillance of disease. Transient ischemic attack (TIA) is a condition primarily managed in the emergency department. However, prior validation studies have focused on data after inpatient hospitalization. We aimed to determine the validity of the Canadian 10th International Classification of Diseases (ICD-10-CA) codes for TIA in the national ambulatory administrative database.

Methods

We performed a diagnostic accuracy study of four ICD-10-CA case definition algorithms for TIA in the emergency department setting. The study population was obtained from two ongoing studies on the diagnosis of TIA and minor stroke versus stroke mimic using serum biomarkers and neuroimaging. Two reference standards were used 1) the emergency department clinical diagnosis determined by chart abstractors and 2) the 90-day final diagnosis, both obtained by stroke neurologists, to calculate the sensitivity, specificity, positive and negative predictive values (PPV and NPV) of the ICD-10-CA algorithms for TIA.

Results

Among 417 patients, emergency department adjudication showed 163 (39.1%) TIA, 155 (37.2%) ischemic strokes, and 99 (23.7%) stroke mimics. The most restrictive algorithm, defined as a TIA code in the main position had the lowest sensitivity (36.8%), but highest specificity (92.5%) and PPV (76.0%). The most inclusive algorithm, defined as a TIA code in any position with and without query prefix had the highest sensitivity (63.8%), but lowest specificity (81.5%) and PPV (68.9%). Sensitivity, specificity, PPV, and NPV were overall lower when using the 90-day diagnosis as reference standard.

Conclusions

Emergency department administrative data reflect diagnosis of suspected TIA with high specificity, but underestimate the burden of disease. Future studies are necessary to understand the reasons for the low to moderate sensitivity.

Electronic supplementary material

The online version of this article (10.1186/s12913-017-2612-6) contains supplementary material, which is available to authorized users.

Keywords: Transient ischemic attack, Diagnosis, Administrative data, Emergency department, Epidemiology

Background

Over half of all ischemic cerebrovascular disease patients present with transient or mild symptoms [1]. Patients with transient ischemic attack (TIA) and minor stroke are at high risk of stroke recurrence or progression in the following days to weeks [24]. Further, despite mild or resolved symptoms, 15% of TIA and minor stroke patients develop permanent disability [5]. Accurate surveillance of TIA and minor stroke allows for monitoring disease burden and temporal trends in the population. It assists in planning health resource allocation and conducting future stroke prevention studies.

While active surveillance is defined as case ascertainment in real-time, passive surveillance is a time and cost-effective method to identify cases using existing data, such as administrative data [6]. Prior studies have shown that the coding of hospital charts using the International Classification of Diseases (ICD) has moderate to high predictive value for cerebrovascular diseases [7, 8]. However, the few validation studies that included TIA as a separate diagnostic category have found that codes for TIA in hospital discharge abstract data have lower predictive value compared to those for ischemic or hemorrhagic strokes [912].

Patients with TIA are increasingly being evaluated on an outpatient basis, from the emergency department (ED) to rapid-access clinics without requiring an inpatient admission [1315]. Chart records are more sparse in the outpatient setting, which may decrease the sensitivity and specificity of passive surveillance [16, 17]. The diagnosis of TIA in the ED is challenging [18, 19] and subject to disagreement even among neurologists [20]. Although the ED is the most relevant setting to validate TIA coding, prior studies have focused on discharge diagnoses after admission to acute-care hospital and therefore are vulnerable to a specific selection bias [9, 10, 2123].

In a cohort of patients presenting with acute and minor neurological symptoms, we aimed to determine the validity of the ICD 10th Canadian Revisions (ICD-10-CA) codes for TIA in the National Ambulatory Care Reporting System (NACRS). We hypothesized that coding for the diagnosis of TIA has lower accuracy in the ED compared to coding after inpatient hospitalization and that ED administrative data codes reflect the ED diagnosis more accurately than the 90-day diagnosis. A key secondary objective was to determine the accuracy of the ED working diagnosis compared to the final 90-day diagnosis.

Methods

In this administrative data validation study for identifying TIA in the ED, we compared the ICD-10-CA coding algorithms to two reference standards: 1) the ED diagnosis from chart re-abstraction and 2) the final 90-day neurologist-determined diagnosis.

National Ambulatory Care Reporting System (NACRS)

In Alberta, ED discharge diagnoses are reported in the NACRS, the Canadian national ambulatory administrative database, containing information on ED visits, day surgery admissions, and selected outpatient clinic visits [24]. The database contains one ICD-10-CA code for the “main problem” per ED visit, up to 10 codes for “other problems”, and has the option to identify queried diagnoses with a prefix “Q”. In Canada, ICD codes are assigned by non-health professional health records technicians who undergo a 2-year technical degree training. Coding is performed according to Canadian standards published and updated by the Canadian Institute for Health Information [25]. We tested four algorithms to identify TIA in NACRS: 1) TIA codes (G45.x except G45.4) in the main position without querying prefix Q (MP); 2) TIA codes in any position without Q (AP); 3) TIA codes in the main position with or without Q (MP + Q); and 4) TIA codes in any position with or without Q (AP + Q). We defined ischemic cerebrovascular disease as coding for either TIA or ischemic stroke (H34.1, H34.2, I63.x, I64.x) [10].

Study population

The study population was obtained from two ongoing Canadian studies: SPECTRA (Spectrometry for Transient Ischemic Attack Rapid Assessment) and DOUBT (Diagnosis Of Uncertain-origin Benign Transient neurological symptoms). SPECTRA aims to identify a blood biomarker to differentiate TIA and minor strokes from mimics and enrolls patients within 24 h after symptom onset. DOUBT is a neuroimaging study of patients with TIA, minor strokes, and stroke mimics and enrolls patients within 7 days after symptom onset. Both studies prospectively enroll consenting patients and require a clinical evaluation by a neurologist at least once between symptom onset and 90 days. Study inclusion and exclusion criteria are outlined in Additional file 1: Table S1. Minor stroke is defined as National Institute of Health Stroke Scale (NIHSS) score ≤ 3. A key feature of both studies is that patients with stroke mimics are actively recruited. These are patients with acute and focal neurological symptoms in whom the working diagnosis is non-ischemic in etiology, such as syncope, seizure, migraine, or other. Recruitment for both studies largely took place in a tertiary-care teaching hospital with an ED volume of about 78,000 visits per year, including 905 annual acute stroke consults. Residents of Alberta, Canada enrolled between December 1st 2013 and October 30th 2015 with at least one ED visit prior to the enrolment date of SPECTRA (within 24 h) and DOUBT (within 7 days) were included into the current study. The Alberta unique personal healthcare number was used for deterministic linkage with NACRS. When linkage to multiple ED visits occurred, including between-ED transfers, the last visit prior to enrolment was retained for analysis. Patients who did not have a 90-day SPECTRA or DOUBT diagnosis were excluded.

Sample size estimation

Prior studies have reported moderate to high accuracy of administrative data to identify TIA after hospital admission (77% to 97%) [9, 10]. Thus, we determined that a sensitivity or specificity below 0.85 would be a clinically meaningful cut-off. We expected a third of patients enrolled into SPECTRA and DOUBT to be true TIAs. With a sample size of 400 patients, we expected estimating sensitivity and specificity with a confidence interval width of 0.10 around the proportion estimates.

Reference standards: ED chart re-abstraction and 90-day neurologist diagnosis

A stroke neurologist (AY) performed chart abstractions to retrospectively adjudicate an ED diagnosis, the first reference standard. The abstractor was blinded to the NACRS code and only evaluated documentation dated and timed prior to the disposition from the ED to replicate the Canadian coding standards [25]. The ED charts of four patients were missing and the SPECTRA and DOUBT case report forms were abstracted instead. The diagnosis of TIA was adjudicated based on the World Health Organization time-based criteria [26]. For example, a patient with symptoms lasting less than 24 h but a magnetic resonance imaging showing evidence of tissue ischemia, such as a small diffusion weighted imaging lesion, was adjudicated as a TIA and specifically not a stroke. TIA was further classified into “possible” or “definite,” depending on whether the adjudicator judged if there was a high likelihood of a competing non-vascular diagnosis. A senior stroke neurologist (SBC) independently reviewed a random sample of thirty charts to determine inter-abstractor reliability.

The second reference standard was the final 90-day diagnosis from the DOUBT and SPECTRA studies. In both studies, a final diagnosis was determined by a stroke neurologist at 90 days, with knowledge of all investigations and clinical follow-up completed in clinical routine.

Given the purpose of this study was to determine the validity of coding for mild or transient ischemic events, we categorized the rare intracranial hemorrhage into the stroke mimic category. These cases had a specific neuroimaging finding, heterogeneous underlying pathologies (e.g. cortical subarachnoid hemorrhage due to reversible cerebral vasoconstriction syndrome, multifocal parenchymal hemorrhage due to cerebral hyperperfusion post carotid revascularization), and all were clinically mild, NIHSS ≤3 as per inclusion criteria of the SPECTRA and DOUBT studies.

Statistical methods

Descriptive statistics were used to determine the frequency of baseline characteristics and prevalence of disease. We first used the diagnosis from ED chart re-abstraction to determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the four TIA case definition algorithms. The analyses were repeated with the final 90-day diagnosis. The categories of definite and possible TIA were combined for the primary analysis. Sensitivity analyses were performed to determine the validity of the algorithms for definite TIA as well as for ischemic cerebrovascular diseases. Accuracy of the working ED diagnosis was compared to the final 90-day diagnosis. All analyses were performed using STATA 14.0 (STATA Corp, College Station, Tex).

Results

Among 417 patients included (n = 375 from SPECTRA and n = 42 from DOUBT), the median age was 67 years (IQR 22). Most patients, 390 (93.5%) were discharged from a tertiary comprehensive stroke centre and 27 patients were discharged from one of the four community hospitals. Re-abstracted diagnoses from ED charts showed 163 (39.1%) TIA, including 121 definite and 42 possible TIA, 155 (37.2%) ischemic strokes, and 99 (23.7%) stroke mimics. Inter-abstractor agreement was 76.7% (κ = 0.50) for the diagnosis of TIA. There were eight intracranial hemorrhages (two subarachnoid and six parenchymal hemorrhages).

Table 1 shows the baseline characteristics as well as the final 90-day diagnosis according to the adjudicated ED diagnosis. The sensitivity and specificity of the ED working diagnosis to accurately determine the 90-day final diagnosis were 61.4% [CI95 52.9, 69.3] and 72.8% [CI95 67.1, 78.0] for TIA; 77.3% [CI95 69.1, 84.3] and 80.6% [CI95 75.6, 85.0] for ischemic stroke; and 56.9% [CI95 48.4, 65.2] and 93.8% [CI95 90.2, 96.3] for stroke mimic.

Table 1
Baseline characteristics and 90-day diagnosis per ED chart adjudicated diagnosis

Table 2 presents the sensitivity, specificity, PPV, and NPV with 95% confidence intervals of the four administrative data coding algorithms using the re-abstracted ED diagnosis. Algorithm MP identified 79 (18.9%) TIA in the cohort. It had the highest specificity (92.5%) and PPV (76.0%), but lowest sensitivity (36.8%). In comparison, algorithm AP + Q identified 151 (36.2%) TIA and had the highest sensitivity (63.8%) at the cost of lower specificity (81.5%) and PPV (68.9%). Sensitivity increased when considering all cerebral ischemia, including TIA and minor stroke, with similar specificity. The sensitivity, specificity, PPV, and NPV were overall lower when comparing administrative data with the final 90-day diagnosis as reference standard (Table 3). The results remained similar when evaluating definite TIA exclusively (Tables 4 and and55).

Table 2
Sensitivity, specificity, PPV, and NPV of ICD-10-CA with adjudicated ED diagnosis as reference standard
Table 3
Sensitivity, specificity, PPV, and NPV of ICD-10-CA with 90-day diagnosis as reference standard
Table 4
Sensitivity, specificity, PPV, and NPV of ICD-10-CA with ED definite TIA as reference standard
Table 5
Sensitivity, specificity, PPV, and NPV of ICD-10-CA with 90-day definite TIA as reference standard

The most specific algorithm MP found 19 (4.6%) false positive and 103 (24.7%) false negative cases when compared to the re-abstracted ED diagnosis. Table 6 outlines the diagnosis in the main position for the false negative cases and shows that 66% of these codes are still related to cerebrovascular diseases.

Table 6
First position diagnoses amongst the false negatives (n = 103)

Discussions

Using the Canadian NACRS database, we showed that suspected cases of acute TIA can be identified with high specificity in the ED, but the sensitivity was low to moderate. Even the most inclusive algorithm AP + Q, evaluating codes in any position and including queried codes, only achieved a sensitivity of 64%, meaning that more than a third of suspected TIA cases discharged from the ED will be inappropriately identified as a stroke or non-stroke syndrome.

Administrative data coding can only be as good as the attending physician’s clinical diagnostic abilities. We found that administrative data reflect the ED diagnosis better than the final 90-day diagnosis, indicating that the coding reflects the episode of care. The process of translating diagnoses during the health encounter into ICD codes is a static snapshot in time, while a physician’s diagnostic impression is fluid and changes based on new information from the clinical course, collateral history, and investigation results. Empirically, transient neurological events are described and diagnosed by the clinical history and show high diagnostic variability. As a result, we show an ED diagnosis of TIA carries only a 61% sensitivity and 73% specificity compared to the final diagnosis, a finding that is consistent with prior reports [1820].

Our results have implications for the use of administrative data for TIA surveillance and research. Suspected acute TIAs can be identified with high specificity and used to determine the temporal trends of disease, but the low sensitivity will underestimate the burden of disease. The most inclusive algorithm AP + Q has moderate sensitivity 64% and PPV 69% and may be useful to evaluate patterns of healthcare utilization in suspected acute TIA or to determine adherence rates to secondary stroke prevention guidelines. However, because the coding is less accurate when compared to the 90-day diagnosis, if administrative data were used to select a cohort of suspected TIAs for a prospective study to determine outcomes of interest, such as rates of stroke recurrence or mortality after TIA, misclassification errors must be addressed by chart review or other case ascertainment method. Finally, there are implications for the use of administrative data for other conditions primarily managed in the ED. Outpatient cases, for example angina or a single seizure, should be validated separately from their inpatient counterparts, acute myocardial infarction or complex seizure disorder [27, 28].

We show that all four algorithms yielded a similar PPV (68.9 to 76.0%), which is lower than the estimates reported by prior TIA validation studies on discharge diagnostic codes after hospital admission (88.9 to 97%) [9, 10], but higher than other studies on TIA patients discharged from the ED (48.8 to 56.1%) [12, 29]. Our study used different methods compared to prior publications. By including patients with TIA mimics, we were able to determine both true negative and false positive cases, allowing for the calculation of sensitivity and specificity. The majority of prior studies evaluating the validity of administrative data in cerebrovascular diseases identified a priori patients with high-yield stroke codes and then verified the coding accuracy or agreement [9, 10, 23, 30, 31]. By design, these studies had a high prevalence of disease, which may lead to an overestimation of the agreement [32] and PPV [33]. Others examined the validity of cerebrovascular disease coding as a comorbidity of another condition (e.g. diabetes, hypertension), such that the prevalence of cerebrovascular diseases is low, less than 5%, yielding estimates with wide confidence intervals and prevalent disease is not differentiated from incident cases [34, 35]. Finally, a third type of study validated administrative data against a registry or other active case finding methods [11, 12, 16, 36]. Stroke mimics are excluded by design, limiting the ability to evaluate true negative and false positive cases.

Our study has several strengths. By using two reference standards, we addressed the question of accuracy of administrative data in two different ways: how well do the codes reflect 1) the clinical encounter and 2) the patient’s actual condition? By including stroke mimics, we evaluated true negative and false positive rates in order to report specificity estimates. Some limitations are worth mentioning. SPECTRA and DOUBT prospectively enrolled patients who consented to participation, resulting in a sample of non-consecutive patients presenting to the ED with acute neurological symptoms. The TIA prevalence in our study is higher than that of a randomly selected sample from the ED; because prevalence impacts the predictive value, our results may not be directly applicable to a specific ED population. Nevertheless, the prevalence of TIA and stroke mimics in this study is similar to prior publications on patients with acute neurological presentations referred to a stroke prevention clinic [19, 37]. The majority of the patients were discharged from a single center. This may limit the generalizability of our results because administrative data coding is variable between jurisdictions. Finally, although chart abstractors were blinded to NACRS codes and the 90-day diagnosis, the retrospective derivation of the ED diagnosis may be subject to residual bias.

Conclusions

We show that administrative data identify suspected TIA in the ED with high specificity while underestimating the incidence of acute TIA cases. We also highlight the diagnostic challenges that are presented to the physician for patients with TIA in the ED. Understanding the detailed reasons for the moderate sensitivity of administrative data for this diagnosis and whether the miscoded patients are systematically different from the accurately coded ones will require prospective diagnostic studies.

Funding

AYXY holds a Clinician Fellowship Award from Alberta Innovates Health Solutions. MDH reports funding from Heart & Stroke Foundation of Alberta, Northwest Territories, Nunavut and from Alberta Innovates Health Solutions. SPECTRA was funded by Genome Canada and Genome Alberta and DOUBT was funded by the Canadian Institutes of Health Research.

Availability of data and materials

The dataset supporting the conclusions of this article is available upon request.

Abbreviations

DOUBT
Diagnosis Of Uncertain-origin Benign Transient neurological symptoms
ED
Emergency Department
ICD
International Classification of Diseases
ICD-10-CA
International Classification of Diseases, 10th Canadian iteration
NACRS
National Ambulatory Care Reporting System
NPV
Negative Predictive Value
PPV
Positive Predictive Value
SPECTRA
Spectrometry for Transient Ischemic Attack Rapid Assessment)
TIA
Transient Ischemic Attack

Additional file

Additional file 1: Table S1.(22K, docx)

Inclusion and exclusion criteria for the DOUBT and SPECTRA studies. Table listing the inclusion and exclusion criteria for the DOUBT and SPECTRA studies. (DOCX 22 kb)

Authors’ contributions

Authors’ contributions

AYXY contributed to the study concept and design, acquisition, analysis, and interpretation of the data, as well as drafting and revising the manuscript. HQ contributed to the study concept and design, data acquisition and interpretation, critical revision of the manuscript, and supervision. AM contributed to the study concept and design, data interpretation, critical revision of the manuscript, and supervision. GOW contributed to the data acquisition, administrative support, and manuscript revision. MDH contributed to the study concept and design, data analysis and interpretation, critical revision of the manuscript, and supervision. SBC contributed to the study concept and design, data interpretation, critical revision of the manuscript, and supervision. All authors read and approved the final manuscript.

Notes

Ethics approval and consent to participate

The SPECTRA and DOUBT studies received approval from the University of Calgary institutional review board for research using human subjects and written informed consent was obtained from all patients enrolled. The current study received approval from the University of Calgary institutional review board for research and a waiver of consent was granted (REB15–2943).

Consent for publication

N/A

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Footnotes

Electronic supplementary material

The online version of this article (10.1186/s12913-017-2612-6) contains supplementary material, which is available to authorized users.

Contributor Information

Amy Y. X. Yu, Phone: 1 (403) 210-8116, ac.yraglacu@uy.yma.

Hude Quan, ac.yraglacu@nauqh.

Andrew McRae, ac.yraglacu@earcma.

Gabrielle O. Wagner, ac.yraglacu@rengawog.

Michael D. Hill, ac.yraglacu@llih.leahcim.

Shelagh B. Coutts, ac.yraglacu@sttuocs.

References

1. Reeves M, Khoury J, Alwell K, Moomaw C, Flaherty M, Woo D, Khatri P, Adeoye O, Ferioli S, Kissela B, et al. Distribution of National Institutes of Health stroke scale in the Cincinnati/northern Kentucky stroke study. Stroke. 2013;44(11):3211–3213. doi: 10.1161/STROKEAHA.113.002881. [PMC free article] [PubMed] [Cross Ref]
2. Johnston SC, Gress DR, Browner WS, Sidney S. Short-term prognosis after emergency department diagnosis of TIA. JAMA. 2000;284(22):2901–2906. doi: 10.1001/jama.284.22.2901. [PubMed] [Cross Ref]
3. Hill MD, Yiannakoulias N, Jeerakathil T, Tu JV, Svenson LW, Schopflocher DP. The high risk of stroke immediately after transient ischemic attack: a population-based study. Neurology. 2004;62(11):2015–2020. doi: 10.1212/01.WNL.0000129482.70315.2F. [PubMed] [Cross Ref]
4. Amarenco P, Lavallee PC, Labreuche J, Albers GW, Bornstein NM, Canhao P, Caplan LR, Donnan GA, Ferro JM, Hennerici MG, et al. One-year risk of stroke after transient ischemic attack or minor stroke. N Engl J Med. 2016;374(16):1533–1542. doi: 10.1056/NEJMoa1412981. [PubMed] [Cross Ref]
5. Coutts SB, Modi J, Patel SK, Aram H, Demchuk AM, Goyal M, Hill MD. What causes disability after transient ischemic attack and minor stroke?: results from the CT and MRI in the triage of TIA and minor cerebrovascular events to identify high risk patients (CATCH) study. Stroke. 2012;43(11):3018–3022. doi: 10.1161/STROKEAHA.112.665141. [PubMed] [Cross Ref]
6. Yu AY, Holodinsky JK, Zerna C, Svenson LW, Jette N, Quan H, Hill MD. Use and utility of administrative health data for stroke research and surveillance. Stroke. 2016;47(7):1946–1952. doi: 10.1161/STROKEAHA.116.012390. [PubMed] [Cross Ref]
7. Andrade SE, Harrold LR, Tjia J, Cutrona SL, Saczynski JS, Dodd KS, Goldberg RJ, Gurwitz JH. A systematic review of validated methods for identifying cerebrovascular accident or transient ischemic attack using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):100–128. doi: 10.1002/pds.2312. [PMC free article] [PubMed] [Cross Ref]
8. McCormick N, Bhole V, Lacaille D, Avina-Zubieta JA. Validity of diagnostic codes for acute stroke in administrative databases: a systematic review. PLoS One. 2015;10(8):e0135834. doi: 10.1371/journal.pone.0135834. [PMC free article] [PubMed] [Cross Ref]
9. Benesch C, Witter DM, Jr, Wilder AL, Duncan PW, Samsa GP, Matchar DB. Inaccuracy of the international classification of diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease. Neurology. 1997;49(3):660–664. doi: 10.1212/WNL.49.3.660. [PubMed] [Cross Ref]
10. Kokotailo RA, Hill MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005;36(8):1776–1781. doi: 10.1161/01.STR.0000174293.17959.a1. [PubMed] [Cross Ref]
11. Ives DG, Fitzpatrick AL, Bild DE, Psaty BM, Kuller LH, Crowley PM, Cruise RG, Theroux S. Surveillance and ascertainment of cardiovascular events. The cardiovascular health study. Ann Epidemiol. 1995;5(4):278–285. doi: 10.1016/1047-2797(94)00093-9. [PubMed] [Cross Ref]
12. Hall R, Mondor L, Porter J, Fang J, Kapral MK. Accuracy of administrative data for the coding of acute stroke and TIAs. Can J Neurol Sci. 2016:1–9. [PubMed]
13. Ranta A, Barber PA. Transient ischemic attack service provision: a review of available service models. Neurology. 2016;86(10):947–953. doi: 10.1212/WNL.0000000000002339. [PubMed] [Cross Ref]
14. Rothwell PM, Giles MF, Chandratheva A, Marquardt L, Geraghty O, Redgrave JN, Lovelock CE, Binney LE, Bull LM, Cuthbertson FC, et al. Effect of urgent treatment of transient ischaemic attack and minor stroke on early recurrent stroke (EXPRESS study): a prospective population-based sequential comparison. Lancet. 2007;370(9596):1432–1442. doi: 10.1016/S0140-6736(07)61448-2. [PubMed] [Cross Ref]
15. Lavallee PC, Meseguer E, Abboud H, Cabrejo L, Olivot JM, Simon O, Mazighi M, Nifle C, Niclot P, Lapergue B, et al. A transient ischaemic attack clinic with round-the-clock access (SOS-TIA): feasibility and effects. Lancet Neurol. 2007;6(11):953–960. doi: 10.1016/S1474-4422(07)70248-X. [PubMed] [Cross Ref]
16. Piriyawat P, Smajsova M, Smith MA, Pallegar S, Al-Wabil A, Garcia NM, Risser JM, Moye LA, Morgenstern LB. Comparison of active and passive surveillance for cerebrovascular disease: the brain attack surveillance in Corpus Christi (BASIC) project. Am J Epidemiol. 2002;156(11):1062–1069. doi: 10.1093/aje/kwf152. [PubMed] [Cross Ref]
17. Brown DL, Lisabeth LD, Garcia NM, Smith MA, Morgenstern LB. Emergency department evaluation of ischemic stroke and TIA: the BASIC project. Neurology. 2004;63(12):2250–2254. doi: 10.1212/01.WNL.0000147292.64051.9B. [PubMed] [Cross Ref]
18. Ferro JM, Falcao I, Rodrigues G, Canhao P, Melo TP, Oliveira V, Pinto AN, Crespo M, Salgado AV. Diagnosis of transient ischemic attack by the nonneurologist. A validation study. Stroke. 1996;27(12):2225–2229. doi: 10.1161/01.STR.27.12.2225. [PubMed] [Cross Ref]
19. Prabhakaran S, Silver AJ, Warrior L, McClenathan B, Lee VH. Misdiagnosis of transient ischemic attacks in the emergency room. Cerebrovasc Dis. 2008;26(6):630–635. doi: 10.1159/000166839. [PubMed] [Cross Ref]
20. Castle J, Mlynash M, Lee K, Caulfield AF, Wolford C, Kemp S, Hamilton S, Albers GW, Olivot JM. Agreement regarding diagnosis of transient ischemic attack fairly low among stroke-trained neurologists. Stroke. 2010;41(7):1367–1370. doi: 10.1161/STROKEAHA.109.577650. [PubMed] [Cross Ref]
21. Ghia D, Thomas PR, Cordato DJ, Worthington JM, Cappelen-Smith C, Griffith N, Hanna I, Hodgkinson SJ, McDougall A, Beran RG. Validation of emergency and final diagnosis coding in transient ischemic attack: south western Sydney transient ischemic attack study. Neuroepidemiology. 2010;35(1):53–58. doi: 10.1159/000310338. [PubMed] [Cross Ref]
22. Goldstein LB. Accuracy of ICD-9-CM coding for the identification of patients with acute ischemic stroke: effect of modifier codes. Stroke. 1998;29(8):1602–1604. doi: 10.1161/01.STR.29.8.1602. [PubMed] [Cross Ref]
23. Leone MA, Gaviani P, Ciccone G. Inter-coder agreement for ICD-9-CM coding of stroke. Neurol Sci. 2006;27(6):445–448. doi: 10.1007/s10072-006-0721-9. [PubMed] [Cross Ref]
24. National Ambulatory Care Reporting System (NACRS) Metadata [https://www.cihi.ca/en/national-ambulatory-care-reporting-system-metadata].
25. Canadian Coding Standards for Version 2015 ICD-10-CA and CCI [https://secure.cihi.ca/estore/productSeries.htm?pc=PCC189].
26. Investigators WMPP. The World Health Organization MONICA project (monitoring trends and determinants in cardiovascular disease): a major international collaboration. J Clin Epidemiol. 1988;41(2):105–114. doi: 10.1016/0895-4356(88)90084-4. [PubMed] [Cross Ref]
27. Austin PC, Daly PA, Tu JV. A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario. Am Heart J. 2002;144(2):290–296. doi: 10.1067/mhj.2002.123839. [PubMed] [Cross Ref]
28. Nagurney JT, Brown DF, Chae C, Chang Y, Chung WG, Cranmer H, Dan L, Fisher J, Grossman S, Tedrow U, et al. Disagreement between formal and medical record criteria for the diagnosis of acute coronary syndrome. Acad Emerg Med. 2005;12(5):446–452. doi: 10.1111/j.1553-2712.2005.tb01546.x. [PubMed] [Cross Ref]
29. Johnsen SP, Overvad K, Sorensen HT, Tjonneland A, Husted SE. Predictive value of stroke and transient ischemic attack discharge diagnoses in the Danish National Registry of patients. J Clin Epidemiol. 2002;55(6):602–607. doi: 10.1016/S0895-4356(02)00391-8. [PubMed] [Cross Ref]
30. Goldstein LB, Jones MR, Matchar DB, Edwards LJ, Hoff J, Chilukuri V, Armstrong SB, Horner RD. Improving the reliability of stroke subgroup classification using the trial of ORG 10172 in acute stroke treatment (TOAST) criteria. Stroke. 2001;32(5):1091–1098. doi: 10.1161/01.STR.32.5.1091. [PubMed] [Cross Ref]
31. Tirschwell DL, Longstreth WT., Jr Validating administrative data in stroke research. Stroke. 2002;33(10):2465–2470. doi: 10.1161/01.STR.0000032240.28636.BD. [PubMed] [Cross Ref]
32. Chen G, Faris P, Hemmelgarn B, Walker RL, Quan H. Measuring agreement of administrative data with chart data using prevalence unadjusted and adjusted kappa. BMC Med Res Methodol. 2009;9:5–2288. doi: 10.1186/1471-2288-9-5. [PMC free article] [PubMed] [Cross Ref]
33. Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16(9):981–991. doi: 10.1002/(SICI)1097-0258(19970515)16:9<981::AID-SIM510>3.0.CO;2-N. [PubMed] [Cross Ref]
34. Newton KM, Wagner EH, Ramsey SD, McCulloch D, Evans R, Sandhu N, Davis C. The use of automated data to identify complications and comorbidities of diabetes: a validation study. J Clin Epidemiol. 1999;52(3):199–207. doi: 10.1016/S0895-4356(98)00161-9. [PubMed] [Cross Ref]
35. Humphries KH, Rankin JM, Carere RG, Buller CE, Kiely FM, Spinelli JJ. Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review? J Clin Epidemiol. 2000;53(4):343–349. doi: 10.1016/S0895-4356(99)00188-2. [PubMed] [Cross Ref]
36. Appelros P, Terent A. Validation of the Swedish inpatient and cause-of-death registers in the context of stroke. Acta Neurol Scand. 2011;123(4):289–293. doi: 10.1111/j.1600-0404.2010.01402.x. [PubMed] [Cross Ref]
37. Calanchini PR, Swanson PD, Gotshall RA, Haerer AF, Poskanzer DC, Price TR, Conneally PM, Dyken ML, Futty DE. Cooperative study of hospital frequency and character of transient ischemic attacks. IV. The reliability of diagnosis. JAMA. 1977;238(19):2029–2033. doi: 10.1001/jama.1977.03280200041015. [PubMed] [Cross Ref]

Articles from BMC Health Services Research are provided here courtesy of BioMed Central