Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Pharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2012 June 1.
Published in final edited form as:
PMCID: PMC3365595

An Automated Database Case Definition for Serious Bleeding Related to Oral Anticoagulant Use



Bleeding complications are a serious adverse effect of medications that prevent abnormal blood clotting. To facilitate epidemiologic investigations of bleeding complications, we developed and validated an automated database case definition for bleeding-related hospitalizations.


The case definition utilized information from an in-progress retrospective cohort study of warfarin-related bleeding in Tennessee Medicaid enrollees 30 years of age or older. It identified inpatient stays during the study period of January 1990 through December 2005 with diagnoses and/or procedures that indicated a current episode of bleeding. The definition was validated by medical record review for a sample of 236 hospitalizations.


We reviewed 186 hospitalizations that had medical records with sufficient information for adjudication. Of these, 165 (89% [95% CI, 83%-92%]) were clinically confirmed bleeding-related hospitalizations. An additional 19 hospitalizations (10% [7%-15%]) were adjudicated as possibly bleeding-related. Of the 165 clinically confirmed bleeding-related hospitalizations, the automated database and clinical definitions had concordant anatomical sites (gastrointestinal, cerebral, genitourinary, other) for 163 (99% [96%-100%]). For those hospitalizations with sufficient information to distinguish between upper/lower gastrointestinal bleeding, the concordance was 89%(76%-96%) for upper gastrointestinal sites and 91%(77%-97%) for lower gastrointestinal sites.


A case definition for bleeding-related hospitalizations suitable for automated databases had a positive predictive value of between 89% and 99% and could distinguish specific bleeding sites.

There is frequent and growing use of medications that prevent abnormal blood clotting, including anticoagulants, antifibrinolytics, and platelet inhibitors. However, their therapeutic benefits must be balanced against the risk of bleeding complications, which for some of these drugs are relatively frequent. Warfarin, the most commonly used oral anticoagulant, substantially increases the risk of major bleeding complications, including hemorrhagic stroke and gastrointestinal hemorrhage,1 to as high as 3%-8% per year.2 Similarly, serious bleeding is a frequent complication of platelet P2Y12 adenosine diphosphate receptor antagonists 3-5 such as clopidogrel. Indeed, the elevated incidence of serious bleeding for prasugrel may deter use of this newer drug in high-risk patients.5

Thus, optimal clinical use of anti-clotting agents requires accurate information on the risk of serious bleeding complications. The clinical trials that establish efficacy provide valuable information. However, trial patients often have lower risk of adverse effects than do those in actual practice. Trials often exclude the highest risk patients, include procedures that are not part of routine clinical practice to reduce the occurrence of adverse effects, and may have insufficient power to examine drug-drug or drug-disease interactions.6 Epidemiologic studies of patients in actual clinical practice thus are essential to quantify the risk of bleeding complications for commonly prescribed anti-clotting agents.

Automated databases of medical care encounters for defined populations are a potentially valuable resource for epidemiologic studies of bleeding complications. These databases include records of prescriptions written by clinicians, filled by patients or administered in institutions, records that provide a reliable measure of drug exposure that would be difficult or very expensive to obtain in other ways.7 These prescription records permit classification of anti-clotting drug exposure on a day-to-day basis, which is important because increased bleeding generally is an acute effect of these drugs. Similarly, the use of other medications that may alter the risk of bleeding8 can be closely tracked.

A prerequisite for the study of bleeding complications with automated databases is a reliable definition for bleeding endpoints. One could identify potential endpoints from the computerized files and review medical records to determine ultimate endpoint status.9 However, this is not always possible and, even when possible, may be expensive and time-consuming. We thus developed a case definition for serious bleeding designed for use in automated databases. We report here its validation relative to a definition based upon review of medical records, utilizing a sample from an in-progress cohort study of warfarin-related bleeding in a Tennessee Medicaid population.


Sources of Data

The automated database case definition is based upon information from an in-progress retrospective cohort study of the effect of antimicrobials on the risk of warfarin-related bleeding. The study utilizes computerized files from the Tennessee Medicaid program,10 including an enrollment file as well as files recording prescriptions filled at a pharmacy, hospital admissions, outpatient visits, and long-term care residence. The Medicaid files have been augmented by linkage with computerized death certificates11 and, since 1998, with the State Hospital Discharge File, an “all payers” database of hospital discharges and emergency department visits, which provides information occasionally missing from Medicaid files. These files permitted identification of study populations, tracking of current use of study medications, classification of subjects according to baseline risk factors for bleeding, and ascertainment of potential bleeding complications.7,10

Persons eligible for the underlying cohort included current users of warfarin 30 years of age or older during the study period of 1 January 1990 through 31 December 2005. The eligible population was further restricted to those with at least 1 year of prior Medicaid enrollment, and, in that year, at least one outpatient visit. From this group, we identified the study cohort, which consisted of warfarin users with episodes of antimicrobial use and comparable warfarin users without antimicrobial use. Cohort followup included the interval following the filling of the antibiotic prescription during which there plausibly was an interaction between the antibiotic and warfarin or a comparable interval for the no-antibiotic-use controls.

Automated Database Definition for Bleeding-Related Hospitalization

The definition identified inpatient stays with diagnoses and/or procedures that indicated the hospitalization was related to a current episode of bleeding. We focused on hospitalizations because these are unambiguous and generally represent serious events. The types of serious bleeding events considered included gastrointestinal bleeding, hemorrhagic strokes and other intracranial bleeds, genitourinary bleeding, and bleeding at other sites.

The algorithm identified bleeding-related hospitalizations from the primary discharge diagnosis. The specific diagnosis codes were based upon those presented by Arnason and colleagues,12 modified according to our experience and an extensive review of the computerized records for the hospitalization and related medical care (see Appendix). The diagnosis codes were also the basis for determining the probable site of the bleeding, classified as gastrointestinal, cerebral, genitourinary, or other. Hospitalizations for which the bleed was deemed likely to be due to major trauma were excluded. The complete algorithm is presented in the Appendix.

The algorithm did not consider hospitalizations in which only a secondary diagnosis indicated bleeding. Although serious bleeding may have occurred during these hospitalizations, our experience indicates the bleeding was more likely to have begun in the hospital.

Clinical Validation Study

To assess the quality of the automated database case definition, a sample of computer-identified cases of bleeding-related hospitalizations was selected from the study cohort. The sample included all of the cases from two large metropolitan areas in Tennessee: Davidson and adjacent counties (Cheatham, Robertson, Sumner, Wilson, Rutherford, Williamson) and Knoxville and adjacent counties (Anderson, Jefferson, Sevier, Blount, Loudon, Roane).

For each of the hospitalizations in the sample, trained nurse-abstractors reviewed the hospital chart (when available) and completed a structured abstract form. These forms were adjudicated by two of the study investigators (AC, CMS) to determine if the case met the clinical definition for a bleeding-related hospitalization; cases where there was ambiguity (all were gastrointestinal sites) were also reviewed by a gastroenterologist (WES). The investigators were blinded to the discharge diagnoses at the time of the adjudication.

Clinically confirmed bleeding-related hospitalizations were those with objective evidence in the hospital record indicating that bleeding, unrelated to major trauma, had occurred. This category included both definite and probable bleeding. Definite bleeding required one or more of three criteria to be met (see Appendix):

  1. Direct visualization of blood by a physician or health care provider that was documented in the chart
  2. An investigation or imaging procedure that demonstrated bleeding
  3. An investigation or imaging procedure that identified a potential bleeding source in conjunction with a history of bleeding.

Patients with probable bleeding did not meet the criteria for definite bleeding, but had a history and clinical evidence in the medical record (see Appendix) indicating bleeding was likely to have occurred.

Possible bleeding-related hospitalizations were those for which the clinical history was consistent with bleeding, even though objective evidence was not present (Appendix).

Clinically confirmed bleeding-related hospitalizations were further classified according to severity as either major or minor, based upon the criteria used in several anticoagulant clinical trials. In brief, major bleeds were those that were fatal, involved a critical site (intracranial, retroperitoneal, intraspinal, intraocular, pericardial or intraarticular), led to a reduction in hemoglobin of at least 2 g/dl, or required transfusion of two or more units of blood or packed red cells. 13-19

Statistical Analysis

All statistical analysis was performed with SAS 9.2. The proportions of bleeding-related hospitalizations for which the automated database definition was concordant with clinical adjudication were calculated, with 95% confidence intervals derived using the method of Wilson.


The study sample included a total of 236 hospitalizations the automated database definition identified as bleeding-related (Table 1). Of these, there were 186 (78.8%) for which medical records were reviewed and found to include sufficient information for adjudication. The most common reason for inability to adjudicate was that we were unable to obtain the medical record from the hospital (16.5% of charts, Table 1).

Table 1
Clinical validation study: medical record adjudication process.

Of the 186 hospitalizations adjudicated, there were 165 (88.7% [95% CI, 83.4%-92.5%]) clinically confirmed bleeding-related hospitalizations, of which 133 were definite (71.5% [64.6%-77.5%]) or and 32 were probable (17.2% [12.5%-23.3%]) (Table 2). An additional 19 hospitalizations (10.2% [6.6%-15.4%]) were adjudicated as possibly bleeding-related, with a clinical history consistent with bleeding, but no objective evidence noted in the hospital record. Two of the hospitalizations were for patients with a prior history of bleeding, but there was no evidence of bleeding related to the current admission (Appendix). For the 35 cerebral bleeding-related hospitalizations identified by the automated database definition, 33 (94.3% [82.4%-98.4%]) were classified as probable or definite according to the clinical validation criteria (Table 2).

Table 2
Clinical validation study adjudication results according to site for bleeding assigned by the computer definition.

Table 3 shows the concordance between the automated database-defined site of the bleeding and that ascertained in the clinical validation study for the 165 clinically confirmed bleeding-related hospitalizations. Only two hospitalizations had the site misclassified: one hospitalization that according to the automated database definition was related to gastrointestinal bleeding was found by clinical validation to have an “other” site and one that the automated database definition assigned to the “other” category was adjudicated as gastrointestinal.

Table 3
Clinically confirmed bleeding hospitalizations: concordance between computer-assigned and clinically confirmed bleeding sites.

We assessed the capacity of the automated database definition to distinguish between specific gastrointestinal sites (Table 4). There were 44 hospitalizations with a database-identified upper gastrointestinal site that had clinically confirmed bleeding, of which 38 had sufficient information in the medical record to assign a specific gastrointestinal site. Of these, 34 (89.5% [75.9%-95.8%]) had an adjudicated upper gastrointestinal site. There were 39 with a database-identified lower gastrointestinal site with clinically confirmed bleeding, of which 34 had a specific adjudicated gastrointestinal site; of these, 31 (91.2% [77.0%-96.9%]) were adjudicated to have lower gastrointestinal bleeding.

Table 4
Computer-identified gastrointestinal bleeding-related hospitalizations that were clinically confirmed: concordance between computer-assigned and clinically adjudicated specific gastrointestinal bleeding sites.

Of the 165 clinically confirmed bleeding-related hospitalizations, 137 (83.0% [76.6%- 88.0%]) were judged to have major bleeding (Table 5). For those with a cerebral site of bleeding, 97% (84.7%-99.5%) were classified as having major bleeding.

Table 5
Clinically confirmed bleeding-related hospitalizations: severity of bleeding according to site.

We assessed the positive predictive value for the individual ICD-9-CM diagnosis codes present for hospitalizations in the study sample (Table 6). For those codes for which there were at least 5 hospitalizations adjudicated, the positive predictive values ranged from 80% to 100%. However, for many of the diagnosis codes, the number of adjudicated cases was small.

Table 6
Positive predictive values for individual ICD-9-CM diagnosis codes


We developed and validated a case definition for bleeding-related hospitalizations designed for use in automated databases. Our motivation was to improve the efficiency and quality of epidemiologic studies of bleeding complications of anti-clotting medications by reducing the time and expense required for medical record review. Of 186 hospitalizations with a primary database discharge diagnosis indicating bleeding that were adjudicated, only 2 (1%) were inconsistent with bleeding occurring during the current hospitalization. Both of these were patients with a past history of bleeding, but no evidence of bleeding related to the current admission. There were an additional 19 hospitalizations (10%) for which the hospital record was consistent with a bleeding-related admission, but which lacked the objective evidence we required to consider the bleeding as clinically confirmed. The lack of data often was related to the decision not to perform invasive diagnostic procedures (endoscopy, colonoscopy) on frail patients in a precarious state of health. Thus, our estimate of the positive predictive value of the automated database case definition is between 89% (clinically confirmed bleeding) and 99% (clinically confirmed or possible bleeding). Furthermore, there was excellent agreement with regard to broad anatomical site, with only two instances of misclassification.

We also assessed the potential of the automated database case definition to differentiate between upper and lower gastrointestinal bleeding. This is important because the upper gastrointestinal tract is a common location for bleeding caused by anti-clotting drugs and for this reason proton-pump inhibitors often are recommended as co-therapy.20 Evaluation of the effectiveness of such co-therapy would require identifying bleeding at upper gastrointestinal sites. Determination of the site of gastrointestinal bleeding can be challenging in clinical practice. Patients with bleeding from upper or lower GI sites can present with identical symptoms of blood loss accompanied by the passage of altered blood per rectum. Endoscopic or radiological investigations may not find the bleeding sources or by the time they are performed bleeding may not be present. Thus, given these challenges it was important to evaluate the performance of an automated database algorithm that sought to define the source of bleeding. We found that for database-identified cases of upper gastrointestinal bleeding for which the medical record review could identify a site, 89% were adjudicated to have an upper gastrointestinal site.

Our findings are generally similar to those of Arnason and colleagues,12 who reviewed a sample of 361 discharges from a Canadian tertiary care hospital with a discharge diagnosis indicating warfarin-related bleeding and found a positive predictive value of 91%. The similarity of findings from different settings suggests that automated database case definitions for warfarin-related bleeding may have utility in multiple settings.

There were several study limitations. The automated database definition requires a primary hospital discharge diagnosis indicating bleeding and thus will fail to detect some bleeding events. Cases would thus be missed if the bleeding was coded as a secondary diagnosis (e.g., myocardial infarction as the primary diagnosis with bleeding gastric ulcer as a secondary diagnosis). Our review of database profiles suggests that this scenario is most likely to occur for bleeding that begins in the inpatient setting (e.g., stress ulcer related to ICU stay), but we lack confirmatory data.

Because we did not review all hospitalizations in the study population we could not calculate sensitivity, which would quantify the extent to which bleeding events were undetected. The finding of Arnason et al that their list of codes had a sensitivity of 93%,12 the more extensive set of codes employed in our study, and the straightforward clinical presentation of more serious bleeding complications provide indirect evidence that the sensitivity of our algorithm is adequate.

We were unable to adjudicate 21% of the records in the sample. The primary reason was that the hospitals could not locate the chart, most often because the admission occurred several years in the past and the records had not been retained.

The automated database case-definition did not consider deaths with an underlying cause coded as bleeding-related if these occurred in the absence of a hospital admission. In our experience, these are much less frequent than hospitalizations and subject to considerable misclassification.21

The study sample contained a limited number of individual ICD-9-CM codes with adequate sample size for stable estimates of the positive predictive value. Additional validation studies with larger sample sizes for individual diagnosis codes would be useful.

The sample of hospitalizations considered may limit generalizability. The study cohort consisted of current users of warfarin who resided in a specific geographic region. Further study of bleeding in patients taking other anti-clotting drugs and of different populations would be useful. The sample consisted entirely of Tennessee Medicaid enrollees. However, performance of automated database case definitions has been broadly similar in Medicaid and non-Medicaid populations for other diseases, including gastroduodenal ulcers,22 stroke,23 and myocardial infarction.24 The limited sample size precluded detailed analysis for specific diagnostic codes. Further work in other populations with larger sample sizes would be useful.

In conclusion, we developed and validated an automated database case definition for bleeding-related hospitalizations suitable for automated databases. The definition had a positive predictive value of between 89% and 99% and could distinguish specific bleeding sites, which should make it a useful tool for pharmacoepidemiologists.

Supplementary Material



Supported by grants from the Vanderbilt Center for Education and Research on Therapeutics (CERTs, cooperative agreement # HS-016974)

We gratefully acknowledge the Tennessee Bureau of TennCare and the Department of Health, which provided study data.

Reference List

(1) Singer DE. Overview of the randomized trials to prevent stroke in atrial fibrillation. Ann Epidemiol. 1993;3:563–567. [PubMed]
(2) Beyth RJ, Landefeld CS. Anticoagulants in older patients. A safety perspective. Drugs and Aging. 1995;6:45–54. [PubMed]
(3) Bhatt DL, Fox KAA, Hacke W, et al. Clopidogrel and aspirin versus aspirin alone for the prevention of atherothrombotic events. N Eng J Med. 2006;354:1706–1717. [PubMed]
(4) Eshaghian S, Kaul S, Amin S, Shah P, Diamond GA. Role of clopidogrel in managing atherothrombotic cardiovascular disease. Ann Intern Med. 2007;146:434–441. [PubMed]
(5) Wiviott SD, Braunwald E, McCabe CH, et al. Prasugrel versus clopidogrel in patients with acute coronary syndromes. N Engl J Med. 2007;357:2001–2015. [PubMed]
(6) Ray WA, Griffin MR, Avorn J. Evaluating drugs after their approval for clinical use. N Engl J Med. 1993;329:2029–2032. [PubMed]
(7) Ray WA. Population-based studies of adverse drug effects. N Engl J Med. 2003;349:1592–1594. [PubMed]
(8) Holbrook AM, Pereira JA, Labiris R, et al. Systematic overview of warfarin and its drug and food interactions. Arch Intern Med. 2005;165:1095–1106. [PubMed]
(9) Shorr RI, Ray WA, Daugherty JR, Griffin MR. Concurrent use of nonsteroidal anti-inflammatory drugs and oral anticoagulants places elderly persons at high risk for hemorrhagic peptic ulcer disease. Arch Intern Med. 1993;153:1665–1670. [PubMed]
(10) Ray WA, Griffin MR. Use of Medicaid data for pharmacoepidemiology. Am J Epidemiol. 1989;129:837–849. [PubMed]
(11) Piper JM, Ray WA, Griffin MR, Fought R, Daugherty JR, Mitchel E., Jr Methodological issues in evaluating expanded Medicaid coverage for pregnant women. Am J Epidemiol. 1990;132:561–571. [PubMed]
(12) Arnason T, Wells PS, van Walraven C, Forster AJ. Accuracy of coding for possible warfarin complications in hospital discharge abstracts. Thromb Res. 2006;118:253–262. [PubMed]
(13) Douketis JD, Arneklev K, Goldhaber SZ, Spandorfer J, Halperin F, Horrow J. Comparison of bleeding in patients with nonvalvular atrial fibrillation treated with ximelagatran or warfarin: assessment of incidence, case-fatality rate, time course and sites of bleeding, and risk factors for bleeding. Arch Intern Med. 2006;166:853–859. [PubMed]
(14) Olsson SB. Stroke prevention with the oral direct thrombin inhibitor ximelagatran compared with warfarin in patients with non-valvular atrial fibrillation (SPORTIF III): randomised controlled trial. Lancet. 2003;362:1691–1698. [PubMed]
(15) Albers GW, Diener HC, Frison L. Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrial fibrillation: a randomized trial. JAMA. 2005;293:690–698. [PubMed]
(16) Fiessinger JN, Huisman MV, Davidson BL. Ximelagatran vs low-molecular-weight heparin and warfarin for the treatment of deep vein thrombosis: A randomized trial. JAMA. 2005;293:681–689. [PubMed]
(17) Francis CW, Berkowitz SD, Comp PC. Comparison of Zimelagatran with warfarin for the prevention of venous thromboembolism after total knee replacement. N Engl J Med. 2003;349:1703–1712. [PubMed]
(18) Lee AYY, Levin MN, Baker RI. Low-molecular-weight heparin versus a coumarin for the prevention of recurrent venous thromboembolism in patients with cancer. N Engl J Med. 2003;349:146–153. [PubMed]
(19) Gullov AL, Koeford BG, Petersen P. Bleeding during warfarin and aspirin therapy in patients with atrial fibrillation. Arch Intern Med. 1999;159:1322–1328. [PubMed]
(20) Ray WA, Murray KT, Griffin MR, et al. Outcomes with concurrent use of clopidogrel and proton-pump inhibitors: a cohort study. Ann Intern Med. 2010;152:337–345. [PMC free article] [PubMed]
(21) Griffin MR, Ray WA, Schaffner W. Nonsteroidal anti-inflammatory drug use and death from peptic ulcer in elderly persons. Ann Intern Med. 1988;109:359–363. [PubMed]
(22) Ray WA, Chung CP, Stein CM, Smalley WE, Arbogast PG, Griffin MR. Risk of peptic ulcer hospitalizations in users of NSAIDs with gastroprotective cotherapy versus coxibs. Gastroenterol. 2007;133:790–798. [PubMed]
(23) Roumie CL, Mitchel E, Gideon PS, Varas-Lorenzo C, Castellsague J, Griffin MR. Validation of ICD-9 codes with a high positive predictive value for incident strokes resulting in hospitalization using Medicaid health data. Pharmaco Drug Safety. 2008;17:20–26. [PubMed]
(24) Ray WA, Stein M, Daugherty JR, Hall K, Arbogast P, Griffin MR. COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease. Lancet. 2002;360:1071–1073. [PubMed]