|Home | About | Journals | Submit | Contact Us | Français|
Participation in clinical registries is nonrandom, so participants may differ in important ways from nonparticipants. The extent to which findings from clinical registries can be generalized to broader populations is unclear.
We linked data from the Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients with Heart Failure (OPTIMIZE-HF) registry with 100% inpatient Medicare fee-for-service claims to identify matched and unmatched patients with heart failure. We evaluated differences in baseline characteristics and mortality, all-cause readmission, and cardiovascular readmission rates. We used Cox proportional hazards models to examine relationships between registry enrollment and outcomes, controlling for baseline characteristics. There were 25 245 OPTIMIZE-HF patients in the Medicare claims data and 929 161 Medicare beneficiaries with heart failure who were not enrolled in OPTIMIZE-HF. Although hospital characteristics differed, patient demographic characteristics and comorbid conditions were similar. In-hospital mortality for OPTIMIZE-HF and non-OPTIMIZE-HF patients was not significantly different (4.7% vs 4.5%; P=.37); however, OPTIMIZE-HF patients had slightly higher 30-day (11.9% vs 11.2%; P<.001) and 1-year unadjusted mortality (37.2% vs 35.7%; P<.001). Controlling for other variables, OPTIMIZE-HF patients were similar to non-OPTIMIZE-HF patients for the hazard of mortality (hazard ratio, 1.02; 95% confidence interval, 0.98–1.06). There were small but significant decreases in all-cause (hazard ratio, 0.94; 95% confidence interval, 0.92–0.97) and cardiovascular readmission (hazard ratio, 0.94; 95% confidence interval, 0.91–0.98).
Characteristics and outcomes of Medicare beneficiaries enrolled in OPTIMIZE-HF are similar to the broader Medicare population with heart failure, suggesting findings from this clinical registry may be generalized.
The use of clinical registries to analyze patient characteristics, treatments, and clinical outcomes has become increasingly common, because registries contain detailed clinical information and are more applicable to real-world settings than clinical trials.1–3 In addition, clinical registries are typically accessible and convenient to researchers and may be used continuously to examine trends in medical care.3 However, because registries are typically motivated by specific research questions or quality-improvement goals and often include hospitals and clinicians whose participation is nonrandom, the characteristics of patients enrolled in clinical registries may differ in important ways from the characteristics of nonenrolled patients. For example, lower-risk patients may be either over- or underrepresented as compared with higher-risk patients, or the geographic locations of participating hospitals may be different than the locations of nonparticipating hospitals and may affect the racial distribution of enrolled patients.1,4 Thus, the extent to which registry-based findings can be generalized to a broader population is unclear.
The Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients with Heart Failure (OPTIMIZE-HF) is a large national registry designed to evaluate and improve adherence to guidelines among hospitalized patients with heart failure.5 Since it began in 2003, OPTIMIZE-HF has led to improvements in the quality of care among patients with heart failure and identified important factors associated with clinical outcomes of heart failure.6–9 Although OPTIMIZE-HF captured select outcome data at 60 and 90 days after discharge for a small subset of patients, follow-up data beyond this point are unavailable. However, by linking registry data to inpatient Medicare claims, long-term outcomes can be observed. In this study, we compared the patient characteristics and health outcomes of Medicare beneficiaries enrolled in OPTIMIZE-HF to those of patients hospitalized with heart failure who were not enrolled to evaluate the degree to which registry participants were representative of the general Medicare population.
Patients were identified using the OPTIMIZE-HF patient registry and the 100% inpatient Medicare claims files from the Centers for Medicare & Medicaid Services (CMS). The design and rationale for OPTIMIZE-HF have been described previously.5,10,11 In summary, the registry contains data on 48 612 patients with heart failure who were admitted to one of 259 participating hospitals. Of these patients, 36 165 were admitted between January 1, 2003, and December 31, 2004, and were aged 65 years or older. Patients were eligible for inclusion in the registry if they were admitted for an episode of worsening heart failure or if they developed significant heart failure symptoms during a hospitalization for which heart failure was the primary discharge diagnosis. Hospital teams used heart failure case-ascertainment methods similar to those of the Joint Commission.12 Patient information was reported using a Web-based tool and included variables such as demographic characteristics, comorbid conditions, vital signs, and medications, as well as outcome data at 60 and 90 days after enrollment for a subset of patients.
From CMS, we obtained the 100% inpatient claims files and the corresponding denominator files for 2003 through 2005. The inpatient files contain institutional claims for facility costs covered under Medicare Part A and provide dates of service, hospital identifiers, and diagnosis and procedure codes as found on the discharge record. The denominator files contain beneficiary demographic data, such as age, sex, birth and death dates, and information about program eligibility and enrollment. These files are research-identifiable, meaning that unique patient identifiers have been assigned to each beneficiary so that they may be tracked longitudinally.
To identify OPTIMIZE-HF enrollees in the Medicare claims data, we searched each database for a series of keys that would be strongly suggestive of a match. We matched patients on sex, date of admission, date of discharge, and hospital (from the American Hospital Association identifier). In addition, we required a match on at least 2 of the 3 components of the birth date (ie, month, day, and year). To qualify as a match, records had to link on a minimum of 4 specific variables: sex, birth date, hospital, and discharge date. In the event that multiple hospitalizations were observed, we retained the first of these for the analysis. After linking the records, we excluded 142 patients (0.6%) with a primary diagnosis of rheumatic heart failure (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] code 398.91). We included only persons living in the United States who were aged 65 years or older and were enrolled in fee-for-service Medicare at the time of the index hospitalization.
We then identified all unmatched patients in the Medicare claims files who were discharged with a primary diagnosis of heart failure (ICD-9-CM codes 402.×1, 404. ×1, 428. ×, 428.2×, 428.3×, and 428.4×.) in 2003 or 2004, applying the same inclusion/exclusion criteria. As with the matched OPTIMIZE-HF patients, in the event of multiple heart failure hospitalizations, we retained only the first for the analysis. Within the set of unmatched Medicare patients, we identified the subset of patients whose index heart failure hospitalization occurred in an OPTIMIZE-HF hospital during the period in which that hospital participated in registry data collection. We defined that participation period as the discharge date for the first matched patient through the discharge date for the last matched patient. For all patients, we calculated time to death as the number of survival days between the index discharge date and the date of death.
We defined comorbid conditions using Hierarchical Condition Categories (HCCs).13 Specifically, we searched all inpatient claims generated in the 365 days before and including the heart failure discharge date for evidence of percutaneous coronary intervention (ICD-9 CM codes 36.01, 36.02, and 36.05); coronary artery bypass graft surgery (ICD-9-CM code 36.1×); use of an implantable cardioverter-defibrillator (ICD-9-CM codes 37.94, 37.95, 37.96, 37.97, and 37.98); acute myocardial infarction (HCC 81); unstable angina and other acute ischemic heart disease (HCC 82); chronic atherosclerosis (HCCs 83 and 84); cardiorespiratory failure and shock (HCC 79); valvular and rheumatic heart disease (HCC 86); hypertension (HCCs 89 and 91); stroke (HCCs 95 and 96); renal failure (HCC 131); chronic obstructive pulmonary disease (HCC 108); pneumonia (HCCs 111, 112, and 113); diabetes (HCCs 15–20 and 120); protein-calorie malnutrition (HCC 21); dementia (HCCs 49–50); hemiplegia, paraplegia, paralysis, functional disability (HCCs 100–102, 68–69, and 177–178); peripheral vascular disease (HCCs 104 and 105); metastatic cancer (HCCs 7 and 8); trauma in the past year (HCCs 154–156 and 158–162); major psychiatric disorders (HCCs 54–56); chronic liver disease (HCCs 25–27); specified heart arrhythmias (HCC 92); and other heart rhythm and conduction disorder (HCC 93). We also used the Medicare inpatient files to compare time to all-cause readmission and cardiovascular readmission after the index hospital stay. We did not count transfers or admissions for rehabilitation (Diagnostic Related Group [DRG] 462 or ICD-9-CM diagnosis code V57.xx on admission) as readmissions. Similarly, we defined cardiovascular readmissions as hospitalizations identified by DRGs 104–112, 115–118, 121–145, 479, 514–518, 525–527, 535, 536, and 547–558, excluding transfers and rehabilitation.
For hospital-level variables, we used the Medicare 100% inpatient files to calculate the average yearly volume of heart failure discharges (DRG 127). Variables regarding availability of a cardiac intensive care unit, open heart surgery and transplant services, and membership in the Council of Teaching Hospitals were extracted from American Hospital Association files.
We used standard descriptive statistics to evaluate differences in baseline characteristics between the OPTIMIZE-HF and non-OPTIMIZE-HF cohorts. We present categorical variables as percentages and continuous variables as means with SDs. To test for differences between patients in the OPTIMIZE-HF and non-OPTIMIZE HF cohorts, we used linear models for age and logistic models for categorical variables, both with robust standard errors to account for hospital clustering. We also compared hospital characteristics in OPTIMIZE-HF and non-OPTIMIZE-HF hospitals, using the Cochran-Mantel-Haenszel test for general association (categorical variables) and the Kruskal-Wallis test (heart failure discharge volume). We used Kaplan-Meier methods to calculate unadjusted in-hospital, 30-day, and 1-year mortality rates for each cohort. We limited the cohorts to patients discharged alive from the index hospitalization for all analyses of readmission. To account for the competing risk of death, we used the cumulative incidence function to calculate unadjusted all-cause and cardiovascular readmission rates at 30 days and 1 year.14 We used Cox proportional hazards models with robust standard errors to account for site clustering to examine the unadjusted relationship between OPTIMIZE-HF enrollment and outcomes and the adjusted relationship, controlling for baseline characteristics.15 In multivariable analysis, we modeled 1-year mortality as a function of OPTIMIZE-HF registry enrollment, age, sex, race, prior coronary procedures (ie, percutaneous coronary intervention, coronary artery bypass graft surgery, and use of implantable cardioverter-defibrillator), comorbid conditions, year of index hospitalization. In addition, the multivariable analysis of all-cause and cardiovascular readmission at 1 year included an indicator of length of stay longer than 7 days for the index hospitalization.16 With the exception of the variable denoting OPTIMIZE-HF registry enrollment, all covariates were derived from Medicare claims data so that risk-adjusted outcomes of OPTIMIZE-HF and non-OPTIMIZE-HF patients could be compared directly.
We used SAS software version 9.1.3 (SAS Institute Inc, Cary, North Carolina) for all analyses. The institutional review board of the Duke University Health System approved the study.
Of 36 165 OPTIMIZE-HF hospitalizations for patients aged 65 and older, 29 301 (81%) were matched to Medicare fee-for-service claims records, representing 25 901 OPTIMIZE-HF patients. After exclusions and removal of additional nonunique patients, 25 245 distinct OPTIMIZE-HF patients remained. We identified another 929 161 Medicare beneficiaries with heart failure during this time period who were not enrolled in OPTIMIZE-HF. Table 1 shows the differences in baseline characteristics between the groups. Mean age was 80 years in both groups. Less than half of all patients were men (44% in the OPTIMIZE-HF group, 43% in the non-OPTIMIZE-HF group) and both groups were predominantly white (86% in the OPTIMIZE-HF group, 85% in the non-OPTIMIZE-HF group). The geographic distribution of patients was slightly different between the cohorts. Although both groups were composed of national samples, fewer OPTIMIZE-HF patients resided in the southern and northeastern US Census regions than did non-OPTIMIZE-HF beneficiaries, reflecting the geographic distribution of OPTIMIZE-HF participating hospitals.6 Rates of comorbidity were qualitatively similar across groups. Due to large sample sizes, many differences were statistically significant, though these are unlikely to represent clinically meaningful differences between groups. Rates of chronic atherosclerosis, valvular and rheumatic heart disease, renal failure, and specified heart arrhythmias were at least 4 percentage points higher in OPTIMIZE-HF patients than in non-OPTIMIZE-HF patients. Chronic atherosclerosis, hypertension, and cardiac arrhythmia were the most commonly observed comorbid conditions in both cohorts, affecting at least half of all patients.
There were differences between the hospitals that participated in OPTIMIZE-HF and those that did not (Table 2). The average annual volume of heart failure discharges in participating hospitals was more than double the volume in nonparticipating hospitals, and participating hospitals were significantly more likely to offer advanced cardiovascular services.
Table 3 shows unadjusted mortality rates and all-cause and cardiovascular readmission rates for each cohort. Across all years, rates of in-hospital and 30-day mortality for OPTIMIZE-HF and non-OPTIMIZE-HF patients were similar. At 1 year, unadjusted mortality was significantly higher among OPTIMIZE-HF patients (37.2% vs 35.7%; P < .001) (Figure 1). In univariate analyses, the hazard of mortality at 1 year was slightly higher among OPTIMIZE-HF patients as compared with non-OPTIMIZE-HF patients (unadjusted hazard ratio [HR], 1.06; 95% confidence interval [CI], 1.02–1.09). At 30 days, both all-cause and cardiovascular readmission rates were similar for OPTIMIZE-HF and non-OPTIMIZE-HF patients. Within 30 days, approximately one fifth of all patients were readmitted to the hospital for any reason and 1 in 10 had a cardiovascular readmission. At 1 year, unadjusted all-cause readmissions were significantly lower among OPTIMIZE-HF patients (64.2% vs 65.8%; P = .04), but unadjusted cardiovascular readmissions were similar (39.4% for OPTIMIZE-HF patients vs 40.6% for non-OPTIMIZE-HF patients; P = .15) (Figure 1). In univariate analyses, OPTIMIZE-HF patients were similar to non-OPTIMIZE-HF patients with regard to readmission (HR for all-cause readmission, 0.97; 95% CI, 0.93–1.00; HR for cardiovascular readmission, 0.97; 95% CI, 0.93–1.01).
Controlling for other variables, OPTIMIZE-HF patients were similar to non-OPTIMIZE-HF patients with regard to the hazard of mortality (adjusted hazard ratio [HR], 1.02; 95% confidence interval [CI], 0.98–1.06) (Table 4). The hazard of death increased by 25% with each 5-year increase in age (95% CI, 1.25–1.25), was 19% higher in men than in women (95% CI, 1.18–1.20), and was significantly lower for black patients compared with nonblack patients (HR, 0.85; 95% CI, 0.84–0.87). The hazard of death was also significantly lower in patients with a history of previous cardiac procedures. All comorbid conditions were independently associated with mortality. Conditions associated with an increased hazard of more than 50% included cardiorespiratory failure and shock, renal failure, protein-calorie malnutrition, metastatic cancer, and chronic liver disease.
In multivariable analysis, patients in OPTIMIZE-HF were less likely than non-OPTIMIZE-HF patients to be readmitted to the hospital. We observed small but significant decreases in the hazards of all-cause (HR, 0.94; 95% CI, 0.92–0.97) and cardiovascular readmission (HR, 0.94; 95% CI, 0.91–0.98). Age and sex were less strongly associated with the hazard of all-cause readmission than with mortality (HR for age per 5 years, 1.03; 95% CI, 1.03-1.03; HR for male sex, 0.97; 95% CI, 0.97–0.98). Sex was not associated with cardiovascular readmission, although age was (HR, 1.01; 95% CI, 1.00–1.01). In contrast to the mortality model, the hazards of all-cause readmission (HR, 1.09; 95% CI, 1.08–1.10) and cardiovascular readmission (HR, 1.18; 95% CI, 1.16–1.19) were greater for black patients than for others. Most comorbid conditions were independently associated with readmission, but less strongly so than with mortality. An index length of stay greater than 7 days was associated with a 13% greater hazard of all-cause readmission (95% CI, 1.12–1.14) and a 3% greater hazard of cardiovascular readmission (95% CI, 1.02–1.04).
The characteristics and outcomes of Medicare patients hospitalized with heart failure during the registry period but not enrolled in OPTIMIZE-HF were also compared with those who were enrolled. Within OPTIMIZE-HF hospitals, 15 311 Medicare patients (38%) were discharged during the hospital’s registry enrollment period but not enrolled in the OPTIMIZE-HF registry. Patients not enrolled were qualitatively similar to enrolled patients (data not shown).Within OPTIMIZE-HF hospitals, the outcomes were similar between enrolled patients and non-enrolled patients. In-hospital mortality was slightly lower among enrolled patients (4.7% vs 5.7% in non-enrolled patients), but 30-day mortality was similar between the 2 groups (11.9% in enrolled patients vs 12.4% in non-enrolled patients). At 1 year, unadjusted mortality was slightly higher among enrolled patients (37.2% vs 35.3% in non-enrolled patients; P = .005). Readmission rates were similar in both groups at 30 days (all-cause readmission: 20.8% in enrolled patients vs 20.5% in non-enrolled patients; cardiovascular readmission: 10.5% in enrolled patients vs 10.1% in non-enrolled patients). At 1 year, readmission rates were slightly higher in enrolled patients (all-cause readmission: 64.2% in enrolled patients vs 62.6% in non-enrolled patients; P = .007; cardiovascular readmission: 39.4% in enrolled patients vs 36.1% in non-enrolled patients, P < .001).
We used data from the OPTIMIZE-HF registry linked with 100% inpatient Medicare fee-for-service claims to evaluate the representativeness of enrolled beneficiaries as compared with the general Medicare population with heart failure. Overall, we observed little difference between OPTIMIZE-HF and non-OPTIMIZE-HF patients. Although OPTIMIZE-HF patients had slightly higher rates of unadjusted mortality than non-OPTIMIZE-HF patients, these differences were not significant in multivariable analysis. Unadjusted and adjusted readmission rates were lower among OPTIMIZE-HF patients; however, controlling for other factors, the hazards of both all-cause and cardiovascular readmission were 6% less among OPTIMIZE-HF patients. The results suggest that patients with heart failure enrolled in OPTIMIZE-HF were representative of Medicare patients with heart failure during the study period.
Patient characteristics at baseline were qualitatively similar between groups. Nonetheless, OPTIMIZE-HF enrollees had slightly higher rates of comorbid conditions than nonenrolled patients. This difference may reflect a greater proportion of tertiary referral and teaching hospitals participating in OPTIMIZE-HF as compared to the nation as a whole, and it may account for at least some of the increase in in-hospital and long-term mortality observed in this population. Alternatively, this finding may simply reflect increased attention to documentation at participating hospitals, particularly with regard to comorbid cardiovascular diagnoses and risk factors. Although there is little evidence of differential enrollment of patients to the OPTIMIZE-HF registry within participating hospitals, those hospitals that participate differ in certain characteristics from hospitals that do not.
These findings have important implications. First, they suggest that the voluntary, nonrandom participation associated with clinical registries does not necessarily limit generalizability. The representativeness of registries should be established on a case-by-case basis, but there is growing evidence that the selection of participating hospitals does not necessarily bias results and that well-designed clinical registries can produce findings similar to those from entire community cohorts or national data sets.1 Nevertheless, the differences we observed between participating and nonparticipating hospitals should be considered when assessing the generalizability of hospital-level analyses. Second, the findings highlight the opportunity to assess the generalizability of clinical registries using retrospectively collected administrative data.
There are important strengths and weakness associated with both clinical registries and administrative claims data. Because registries are observational in nature, for example, they are convenient and less expensive to complete. Similarly, because registries capture data in real time, they allow for timely feedback about quality and safety. As compared with randomized clinical trials, in which enrollment is selective and care is driven by protocol, registries may better reflect real-world clinical practice. Registries can also more easily accommodate broad population-based samples across large geographic areas, thereby increasing their representativeness, and typically collect more clinical detail than can be found in administrative claims. Nonetheless, because registry participation is voluntary and participants may differ systematically from nonparticipants, data from registry patients and/or hospitals may not be generalizable to the population at large. In addition, because the unit of analysis is often a single acute care hospitalization, registries may capture limited follow-up data and generally do not provide a vehicle for analyses of long-term clinical effectiveness or safety.
Medicare administrative claims data, on the other hand, are longitudinal and capture information on eligibility, utilization, and costs for most beneficiaries through the date of death. In the case of the 100% inpatient files used in this study, information is available for all fee-for-service Medicare beneficiaries and is, therefore, free of selection bias. However, claims data lack the clinical detail that can be found in patient registries. At present, Medicare claims do not include information about prescription medications, nor do they include laboratory or clinical test results. Claims are not captured during periods of managed care enrollment because Medicare is not billed directly for these services, and data are primarily available for elderly patients only.
By combining these two sources of information, however, researchers can capitalize on the strengths of each while overcoming some of the limitations. This idea is probably best exemplified by the SEER-Medicare linked database, a collaboration between the National Cancer Institute and CMS in which population-based tumor registry data are linked with Medicare claims data.17,18 The merging of registry and claims data is an inexpensive alternative to primary data collection and can yield a data set that is rich in both clinical detail and long-term follow-up data on costs, comorbid conditions, and outcomes such as readmission and death. The opportunity is particularly keen for inpatient registries for diseases that are prevalent in the elderly, such as heart failure.
Nonetheless, our study has several limitations. First, only patients aged 65 years and older with fee-for-service Medicare are represented; the results may not be generalizable to younger OPTIMIZE-HF patients or to those with Medicare managed care. Second, the study is subject to limitations common to all secondary data sources. Where diagnostic or procedural coding is inaccurate or incomplete, the study may have failed to capture patients or conditions of interest. Multivariable risk adjustment used only variables that were available in the Medicare data set so that risk-adjusted outcomes of OPTIMIZE-HF and non-OPTIMIZE-HF patients could be compared. In this study, we relied on inpatient claims alone and, as a consequence, may have underestimated the prevalence of comorbid conditions in both the OPTIMIZE-HF and non-OPTIMZE-HF patients. A number of clinical, laboratory, and diagnostic testing variables of known prognostic importance could not be accounted for in the models. In addition, neither the Medicare claims data nor the OPTIMIZE-HF registry data included any measure of patient satisfaction, functional status, or health-related quality of life. Although we observed little difference between elderly OPTIMIZE-HF and non-OPTIMIZE-HF patients, individuals may vary on these factors or other unmeasured confounders. Both all-cause and cardiovascular rehospitalization rates were significantly lower among OPTIMIZE-HF patients compared with non-OPTIMIZE-HF patients, even after risk adjustment. This finding suggests that the enhanced processes of care employed in OPTIMIZE-HF may be associated with reductions in rehospitalizations6; however, residual measured and unmeasured confounding may also account for these observations.
In summary, although the characteristics of participating and nonparticipating hospitals differed, we observed little difference between the patient characteristics and health outcomes of Medicare beneficiaries enrolled in OPTIMIZE-HF and those who were not enrolled, suggesting that the experiences of these patients are generalizable to the broader Medicare population with heart failure. Although the validity of patient registries should be established on a case-by-case basis, this study also demonstrates the utility of matched Medicare claims data for evaluating the similarity of patient outcomes at points beyond what the registry alone allows.
Clinical registries provide a wealth of information about patient characteristics, inpatient treatments, and clinical outcomes, but the participating hospitals and clinicians are not randomly selected. Questions often arise about whether registry-based findings can be generalized to a broader population.
We thank Damon Seils of Duke University for editorial assistance and manuscript preparation. Mr Seils did not receive compensation for his assistance apart from his employment at the institution where the study was conducted.
Funding/Support Supported by GlaxoSmithKline. Dr Hernandez is a recipient of an American Heart Association Pharmaceutical Roundtable grant (0675060N). Dr Curtis was supported in part by grant 1R01AG026038-01A1 from the National Institute on Aging and grant 5U01HL66461-05 from the National Heart, Lung, and Blood Institute. Dr Fonarow holds the Eliot Corday Chair in Cardiovascular Medicine and Science at the David Geffen School of Medicine, University of California, Los Angeles, and is also supported by the Ahmanson Foundation.
Trial Registration: clinicaltrials.gov Identifier: NCT00344513
Financial Disclosures: Dr Curtis reported receiving research support from GlaxoSmithKline, Medtronic, Merck & Co, and Novartis. Dr Curtis has made available online a detailed listing of financial disclosures (http://www.dcri.duke.edu/research/coi.jsp). Dr Hernandez reported receiving research support from GlaxoSmithKline, Johnson & Johnson (Scios), Medtronic, and Novartis; and honoraria from AstraZeneca and Novartis. Dr Hernandez has made available online a detailed listing of financial disclosures (http://www.dcri.duke.edu/research/coi.jsp). Dr Fonarow reported receiving research support, personal income for consulting, and honoraria from GlaxoSmithKline, Medtronic, and Novartis. No other disclosures were reported.