We identified six published, validated risk stratification schemes developed to predict warfarin-associated hemorrhage (Table 1) and searched for those specific risk factors in the ATRIA cohort (5-10). Variables unavailable in ATRIA (e.g., patient genotype) or not directly applicable to atrial fibrillation (e.g., acute pulmonary embolism) were not included as potential variables. Prior bleeding history was defined as any prior outpatient or inpatient ICD-9 diagnosis code of hemorrhage, including by specific organ system (e.g., prior intracranial or gastrointestinal bleeding), in the aggregate (e.g., all-cause prior bleeding), and by timing (within 90 days or > 90 days). High fall risk was defined as any prior hospitalization with a discharge diagnosis code indicating mechanical fall that occurred in either the inpatient or outpatient setting.

Clinical laboratory databases were used to identify anemia (hemoglobin < 13g/dL in men and < 12 g/dL in women), thrombocytopenia (platelet count < 90,000), and renal insufficiency (measured by serum creatinine and estimated glomerular filtration rate (eGFR))(11). Abnormal laboratory values were considered abnormal from 3 months prior to 1 year after the date of the measurement, censored by a preceding or subsequent normal test value. If results were unavailable within the time window, the test was assumed normal based on the assumption that tests would be ordered if there were clinical suspicions.

Clopidogrel and ticlopidine exposure was determined from pharmacy databases and duration defined from prescription start date to 2 months after the end of the medication supply. Accurate assessment of aspirin and non-steroidal anti-inflammatory drugs (NSAIDs) exposure was not possible since these medications were predominantly obtained without prescription.

The cohort was randomly divided into a split-sample “derivation” and “validation” cohort using a 2:1 ratio; models using time-varying covariates were developed in the derivation cohort and performance tested in the validation cohort. Covariates associated with major hemorrhage with a hazard ratio ≥ 1.5 were considered for potential inclusion in the final multivariable model. Since variable selection procedures may produce unstable results, we applied backward elimination selection on 1000 bootstrap samples from the derivation set, with ≥ 0.05 the significance level set for removing a variable. Final model variables were those selected in >50% of bootstrap samples(12). Model discrimination was evaluated using the c-index(13) and calibration by the goodness-of-fit test. Variables from the final multivariable Cox regression model were converted to a risk score, with points assigned to each predictor approximately proportional to the magnitude of the regression coefficients rounded to the nearest integer.

When collapsed into a 3-category risk score, the ATRIA risk scheme was able to identify sizable proportions of patients who fell into the most clinically meaningful categories, i.e., low or high risk for hemorrhage. The low-risk category, accounting for 83% of follow-up, had an observed major hemorrhage rate of < 1% per year. The high-risk category represented only 10.2% of patient follow-up yet accounted for 42% of the major bleeding events. The ATRIA scheme led to improvements in accurate net reclassification when compared to alternative schemes. The c-index of 0.74, while not representing perfect discrimination, indicates good performance for a prediction model and compares favorably to other widely used risk stratification schemes such as the CHADS₂ stroke risk index, which has a c-index of ~0.6(15). Certainly, identifying novel predictors of bleeding and improving current methods of risk stratification are important areas of further investigation.

The variables in our model have each been linked to increased hemorrhage risk in prior studies(5-10). Anemia was strongly associated with future bleeding risk. Although we were unable to determine the mechanism of association, anemia may reflect a predisposition to hemorrhage or recent subclinical hemorrhage. Severe renal disease was also a powerful predictor of hemorrhage risk. All-cause prior bleeding was associated with future bleeding, and presumably identifies patients with a potential bleeding lesion or diathesis. Finally, older age and hypertension were independently associated with hemorrhage risk. Similar to other hemorrhage risk schemes, this analysis focused on all-cause major hemorrhage, both intracranial and extracranial. Although intracranial hemorrhages are the most important outcomes, the rarity of such events makes their risk prediction challenging(16). High quality models to predict intracranial hemorrhage are vitally needed.

Our risk model is clinically applicable when counseling patients about the relative benefits and harms of anticoagulation. Particularly as newer, easier to administer anticoagulants become available, accurate estimates of hemorrhage risk will strongly influence the anticoagulation decision. Our risk score may not affect the anticoagulation decision for most patients at high risk for stroke, since they derive a large benefit from anticoagulation. However, bleeding risk is significantly more influential in patients at moderate or lower stroke risk. Our bleeding risk estimates can be incorporated into formal decision-analysis models or can be used to counsel individual patients about their estimated risks of stroke and bleeding. For such patients, providing estimates of the risk of bleeding on anticoagulation can be a very informative addition to individualized patient decision-making.

There are several limitations to our analysis. Our assessment of clinical risk factors was based on computerized databases that did not have information on several covariates such as measurements of blood pressure and genotype. We lacked information about non-prescription use of aspirin or NSAIDs. Although the hemorrhage rate in ATRIA was generally lower than described by the other risk schemes, the rates are similar to some recent randomized trials(17). Finally, it will be important to test the ATRIA risk scheme in a separate population. Although internal validation reduces the likelihood of chance playing a major role in development of our model, external validity needs to be tested empirically.