|Home | About | Journals | Submit | Contact Us | Français|
Studies of acute kidney injury (AKI) commonly lack data on pre-admission renal function, often substituting an inpatient or imputed serum creatinine (SCr) as an estimate for “baseline” renal function. We examined the error introduced when applying methods to estimate “baseline” on AKI classification and mortality. Within a cohort of 4863 adults with a known outpatient baseline admitted to Vanderbilt University Hospital between 10/07 and 10/08, the following surrogates were studied: (1) an eGFR of 75 ml/min/1.73m2 as suggested by the Acute Dialysis Quality Initiative (ADQI), (2) a minimum inpatient SCr, and (3) the first admission SCr. We calculated AKI incidence and mortality rates using each surrogate, and assessed their ability to correctly classify AKI incidence and mortality compared to the most recent outpatient SCr between 7-365 days before admission. Using both imputed and minimum baseline SCr values inflated AKI incidence (38.3% and 35.9% vs. 25.5%; p<0.001), reflecting low specificities of 77% and 80%, respectively. In contrast, using an admission SCr baseline underestimated AKI incidence (13.7% vs. 25.5%, p<0.001), demonstrating a low sensitivity of 39%. Using any surrogate led to frequent misclassification of patient deaths as following AKI and differences for both in-hospital and 60-day mortality rates. In summary, commonly used surrogates for baseline SCr result in bi-directional misclassification of AKI incidence and prognosis in a hospitalized setting.
Acute kidney injury (AKI) is a common cause of morbidity and mortality in the acutely ill, and is an important risk factor for progression to end-stage renal disease (ESRD).(1-5) Recent studies indicate that even modest degrees of renal injury are associated with poor outcomes in hospitalized patients.(6, 7) With the introduction of the RIFLE and Acute Kidney Injury Network (AKIN) consensus criteria,(8, 9) studies have attempted to standardize the diagnosis and disease severity of AKI based on absolute or fractional increases in serum creatinine or progressive degrees of oliguria.
While such standardization has helped to provide meaningful comparisons of disease incidence and prognosis across different hospital settings, current consensus criteria do not uniformly define AKI that begins prior to hospital admission. As these patients are at high risk for further AKI development following admission, classification of AKI remains challenging because serum creatinine measurements, preferably reflecting stable kidney function just prior to the inciting illness, are often missing in both clinical and research settings. In the absence of a standard method to accommodate missing values, investigators performing population-based AKI studies have used a variety of surrogate measures, either substituting the admission creatinine (10-12), a minimum creatinine during the hospital stay (7, 13, 14) or calculating a serum creatinine value based on an imputed eGFR of 75 ml/min/1.73 m2 as initially recommended by the ADQI (3, 4, 15, 16). The potential error introduced by using these surrogates is unknown. We hypothesized that commonly used surrogates for baseline renal function would substantially misclassify AKI incidence, severity and prognosis when compared to a known outpatient baseline. We examined the impact of using each surrogate within a cohort of patients admitted to a tertiary care academic medical center during a 12-month period.
A total of 4863 qualifying adult patients with at least one outpatient non-emergency room serum creatinine measurement between 7 and 365 days before admission and two or more inpatient serum creatinine measurements during the first 7 days of hospitalization were admitted to Vanderbilt University Hospital (VUH) between 10/1/07 and 10/1/08. Excluded were 10477 patients with no outpatient serum creatinine values within the Vanderbilt University Medical Center (VUMC) database within a year prior to admission and 2996 patients with only one inpatient serum creatinine. The median number of outpatient creatinine measurements available per patient was 2 [IQR: 1-4] with a median measurement time of 41 [IQR: 17-120] days prior to hospital admission. The mean age of the study population was 59.6 ± 16.9 years. Overall, there was a slight female preponderance (51.1%), and the majority of subjects were identified as Caucasian (81.3%). Subjects were assigned to internal medicine-affiliated services in 71.8% of the admissions, surgical-based services in 26.6% of admissions, and to other services for the remaining 1.6% of subjects. Admission or transfer to a critical care unit occurred in 19.0% of the cohort during the first 7 days of hospitalization.
Table 1 compares the baseline demographic characteristics and co-morbidities of patients with and without AKI in the first 7 days of hospitalization using the pre-admission outpatient serum as baseline renal function. Subjects developing AKI were more likely to be older, male, non-white, and carry a diagnosis of chronic kidney disease (CKD), diabetes mellitus (DM), hypertension (HTN), coronary artery disease (CAD), congestive heart failure (CHF), and peripheral vascular disease (PVD)(p<0.05). Patients with AKI also tended to have a higher median outpatient baseline serum creatinine 1.13 [IQR: 0.9-1.5] mg/dl and lower corresponding baseline eGFR 63.6 [IQR: 45.7-85.5] than those without AKI (median SCr 1.00 [IQR: 0.83-1.23] and eGFR 72.5 [IQR: 56.1-90.0])(p<0.05).
Table 2 compares the three methods of estimating baseline serum creatinine and corresponding eGFR values with a known outpatient baseline creatinine. Median known outpatient baseline serum creatinine and eGFR were 1.03 [IQR: 0.85-1.30] mg/dl and 70.6 [IQR: 53.2-88.8] ml/min/1.73 m2, respectively. By comparison, imputing an eGFR of 75 ml/min/1.73 m2 for the cohort resulted in a similar median creatinine value, but a significantly lower range of distribution (median SCr 1.01 [IQR: 0.83-1.07] mg/dl, p<0.001). Similarly, the minimum serum creatinine observed during the first 7 days of hospitalization was lower than the known outpatient baseline creatinine (median 0.89 [IQR: 0.72-1.15] mg/dl, p<0.001) and gave a correspondingly higher estimated eGFR (median 82.7 [60.6-107.5] ml/min/1.73 m2). Finally, using the admission serum creatinine led to a higher estimate of baseline creatinine (median 1.04 [IQR: 0.83-1.37] mg/dl) (p<0.001) and lower eGFR (69.8 [IQR: 49.4-91.1] ml/min/1.73 m2)(p=0.014) despite similar median serum creatinine values.
Overall, 1241 subjects (25.5%) experienced AKI using known outpatient baseline serum creatinine values with 18.9% (N=920) classified as AKIN Stage I, 3.5% (N=169) as AKIN Stage II, and 3.1% (N=152) as AKIN Stage III within 7 days of hospitalization. Tables 3 (a-c) display the frequency and direction of AKI misclassification relative to the known outpatient baseline by using the study’s alternative approaches to determining baseline kidney function.
Compared with using a known outpatient baseline serum creatinine, applying an imputed serum creatinine increased overall AKI incidence to 38.3% (N=1863) (p<0.001)(Table 3a). With respect to AKIN staging, 29.5% (N=1432) of subjects were misclassified with 24.5% being staged into a higher degree of injury and 5.0% classified into a lower stage of injury. Application of an imputed baseline yielded a sensitivity of 84.2% and a specificity of 77.4% for the diagnosis of AKI.
Use of the minimum inpatient creatinine as a baseline also led to an overestimation of AKI incidence compared with a known outpatient value (35.9% versus 25.5%, p<0.001). Twenty-four percent (N=1178) of patients were misclassified, with 17.9% re-staged into higher AKIN severity while 6.3% were re-staged to a lower AKIN severity (Table 3b). Overall, using a minimum inpatient creatinine as a baseline yielded a sensitivity of 81.7% and specificity of 79.8% for the diagnosis of AKI.
In contrast, using the first admission creatinine as a baseline led to a reduced estimate of AKI incidence relative to a known outpatient baseline (13.7% versus 25.5%, p<0.001)(Table 3c). Overall, 4.3% of patients were reclassified as having more severe AKIN stage, and 17.0% of patients were reclassified into a less severe AKIN stage. Substituting a first admission creatinine yielded a sensitivity of 38.9% and a specificity of 94.9%.
Among all patients, 94 deaths occurred while in the hospital and an additional 269 deaths occurred following discharge up to 60 days from admission. Figures 1(a-b) display the mortality rates following AKI as diagnosed by surrogate baseline methods compared to a known outpatient baseline. In the 1241 patients developing AKI using our study reference standard, in-hospital and 60-day mortality rates were 4.8% (95% CI: 3.6-6.1%) and 13.1% (95% CI: 11.3-15.1%), respectively. Using any surrogate led to incremental increases in mortality rates with higher AKIN stage. Differences between mortality rates were apparent between methods, although many of the rate estimates had overlapping confidence intervals. In general, use of an imputed or minimum serum creatinine value as a baseline estimated a lower mortality rate than the study reference standard. In contrast, using an admission creatinine as a baseline estimated a mortality rate higher than the study reference standard, particularly for AKIN stage III.
Among deaths occurring within 60 days, use of surrogate baseline serum creatinine values frequently misclassified whether the death followed AKI. For example, using an imputed eGFR of 75 ml/min/1.73 m2 or minimum inpatient serum creatinine led to a 21.8% and 24.8% misclassification rate, respectively. The majority of these errors were generated when deaths were misclassified as following AKI when no AKI was diagnosed using the study reference standard. In contrast, use of an admission serum creatinine as a baseline misclassified 28.1% of deaths with the majority of this error generated when deaths following AKI diagnosed using the study reference standard were missed.
In order to study whether AKI misclassification occurs in other populations excluded by our selection criteria, we performed two sensitivity analyses. First, we examined subjects with a known outpatient baseline but only one inpatient serum creatinine measurement available. This group, consisting of 2996 patients, was excluded from the main analysis because two of the three baseline creatinine estimation methods studied require more than one measurement. As expected, these subjects were low-risk patients with a median outpatient baseline serum creatinine of 0.94 [0.80,1.13] mg/dl, an overall 60-day mortality of 1.0% (31/2996), and an AKI rate of 3.7% (N=110). When using an imputed estimate of baseline creatinine, we observed an overestimation of AKI incidence (15.3% versus 5.0% with known baseline, p<0.001). Secondly, we examined the differences in AKI rates among the 10,477 patients without a recorded outpatient serum creatinine using the three estimation methods; the study reference standard was not computable in this population. Both imputed and minimum methods still resulted in higher AKI rates (27.1% and 35.4%, respectively) than when using the admission serum creatinine (12.1%).
Missing pre-admission serum creatinine is a recognized problem in AKI research. While the use of surrogate measures for baseline renal function avoids selection bias, it introduces a non-discoverable measurement error leading to misclassification of both disease occurrence and prognosis. The results of this study indicate that commonly used measures to estimate baseline serum creatinine can lead to significant misclassification in AKI studies. Depending on the surrogate selected, this misclassification can occur in both directions – either underestimating or overestimating disease incidence, which in turn, impacts the staging and associated mortality of presumed AKI. If the proportion of analyzed patients using surrogates is significant, the practice could adversely affect the quality of study outcomes and conclusions.
All three surrogates used in our study introduced significant error. Since the distribution of eGFRs in our population was skewed toward lower values, imputing an eGFR value of 75 ml/min/1.73 m2 frequently overestimated the actual eGFR. The net effect of imputation was to falsely inflate AKI disease occurrence and staging. In populations enriched with CKD, including other hospitalized populations, the elderly, and those with diabetes or hypertension, applying this method would be expected to consistently yield low specificity for detecting AKI and inflate the probability of a Type I error. The misclassification of AKI led to a significant proportion of all deaths at 60 days being falsely associated with AKI in our cohort (18.5%).
The use of inpatient creatinine measurements as surrogates for baseline function also resulted in misclassification. Using a minimum serum creatinine value as a baseline also inflated disease incidence indicating that minimum creatinine was usually lower than the most recent outpatient creatinine. While the reasons for this are not entirely clear, potential explanations include the effects of aggressive volume expansion, discontinuance of renin-angiotensin-aldosterone system (RAAS) blockade, or decreased creatinine generation in the midst of acute illness(17, 18). Similar to imputation, the underestimation of baseline serum creatinine also led to low specificity for diagnosing AKI and misclassifying subsequent deaths.
In contrast to an imputed or minimum serum creatinine, use of the admission serum creatinine as a baseline resulted in nearly a 50% reduction in the reported incidence of AKI compared with using a known outpatient baseline. This decrease is perhaps best explained by the missed diagnosis of community-acquired AKI that improves or stabilizes during hospitalization. The higher mortality rates observed when using this baseline reflect the bias of using this method, which is only sensitive to AKI that continues to worsen during hospitalization.
The use of surrogates for baseline renal function across AKI studies has important implications for epidemiologic studies, clinical trial enrollment, and resource allocation. Use of different surrogate baselines makes it difficult to compare or summarize studies of AKI incidence and natural history,(19) determine accurate power estimates for AKI treatment trials, and further refine risk-prediction models for AKI-related clinical outcomes. The specific impact of baseline choice on the latter is particularly significant. For example, despite similar 60-day mortality rates using various surrogate estimates, we observed many deaths were misclassified as following AKI. The misclassification of deaths could adversely effect the development of valid prediction models for important AKI-associated outcomes as variations in the baseline applied may capture a significantly different group of patients. AKI studies using different surrogate baselines may also not be directly comparable or readily combined for meta-analysis. For example, baseline choice may explain, in part, the observed difference in AKI rates in one study comparing RIFLE and AKIN using the same cohort,(12) while a separate study did not.(4) Finally, while restricting the diagnostic window of AKI to 48-hours as recommended by current consensus criteria may “reduce the need for baseline” renal function,(9) this approach significantly reduces the sensitivity to AKI starting before admission. These observations suggest the need for additional criteria to better define the contribution of pre-admission renal injury to that currently captured by AKIN criteria.
Retrospective determination of a change in renal function using serum creatinine is the current metric for diagnosing and staging AKI. This feature distinguishes it from other acute diseases including sepsis or the Acute Respiratory Distress Syndrome (ARDS), where both injury (abnormal chest X-ray, tissue cultures) and functional (WBC count, oxygenation) markers are required to satisfy diagnostic criteria.(20, 21) Further, as the relation between serum creatinine and actual GFR is non-linear, quantification of injury severity will continue to rely on the reference point as much as the observed magnitude of the difference. This limitation continues to confront the study of AKI though current efforts to identify novel and promising candidate markers able to reflect the nature of injury such as neutrophil gelatinase-associated lipocalin (NGAL)(22) or kidney injury molecule-1 (KIM-1)(23) may introduce a much needed segregation between evolving injury and ongoing changes in function.
The study has some limitations. By design, the study selected patients with at least one pre-admission serum creatinine measurement in the prior year and two serum creatinine measurements during hospitalization, and excluded patients with fewer measured creatinine values or ones measured at a laboratory outside of VUMC. The selection criteria may have skewed the population to one that is more chronically ill. However, in the sensitivity analyses examining patients without a baseline serum creatinine and patients with one inpatient creatinine, the three methods still disagreed on AKI classification, suggesting that the use of surrogates would introduce significant error in populations not included in the main analysis. Secondly, we did not use timed urine output criteria for diagnosing AKI because the hourly urine output data required to determine oliguria within any 6, 12, or 24 hour window was not reliably captured in a non-ICU setting. The lack of accurate urine output data replicates the situation at most US hospitals, and recent data suggest that urine output data may not add materially to AKIN risk prediction when compared with serum creatinine concentration alone.(3) Whether urine output adds discrimination to certain subpopulations including those with subtle injury or those with extremely low baseline creatinine values (<0.5 mg/dl), for example, remains to be determined.
In conclusion, frequently used methods to estimate baseline kidney function do not closely agree and result in bi-directional misclassification of AKI incidence, severity, and prognosis in a large, broadly-selected hospital population. The use of different estimates may introduce a significant measurement error adversely impacting the quality of individual AKI studies as well as continue to hinder comparison of studies between settings. These findings underscore the need for additional consensus criteria to better characterize pre-admission AKI specifying a standard method to incorporate previously known baseline data. It also suggests the need to develop better methods for estimate baseline kidney function when a pre-admission serum creatinine is truly missing. The development and validation of improved surrogates would significantly improve the fidelity of future studies of AKI diagnosis and prognosis.
Adult patients (≥ 18 years old) were eligible for this study if admitted to Vanderbilt University Hospital (VUH) for a minimum of 24 continuous hours between 10/1/07 and 10/1/08. As the main adult inpatient facility within Vanderbilt University Medical Center (VUMC) Hospital, VUH is a large tertiary referral center serving middle Tennessee and the surrounding areas. Inclusion criteria required that a patient have at least two inpatient SCr measurements within the first 7 days of hospitalization and at least 1 outpatient non-emergency room SCr measured between 7 and 365 days prior to admission available within the VUMC laboratory database. The study excluded patients with previous evidence of stage 5 CKD or end stage renal disease, based on prior International Statistical Classification of Diseases and Related Health Problems Ninth Edition (ICD–9) code assignment of 585.6, 996.73,996.68,996.56, 792.5 or 458.21, or an earlier outpatient baseline calculated eGFR of < 15ml/min/1.73 m2. The study included data from the first hospitalization for patients with multiple admissions during the study interval.
General demographic information (e.g. age, gender, race, admitting service and location) and laboratory data were collected from the institutional electronic medical record (EMR), which serves inpatients and outpatients at VUMC. Data on SCr measurements were retrieved for each eligible patient for a period of 1 year prior to admission if available. The baseline outpatient serum creatinine was defined as the most recent outpatient non-emergency room measurement between 7 and 365 days prior to hospitalization. This was chosen as our reference standard as it was felt to approximate the approach most often applied by clinicians when evaluating a patient in the acute setting with potential AKI. Information on co-morbid conditions was identified using the corresponding ICD-9 codes for Diabetes Mellitus (DM), hypertension (HTN), coronary artery disease (CAD), congestive heart failure (CHF), and peripheral vascular disease (PVD) assigned prior to admission (Appendix 1). Chronic Kidney Disease was defined as using ICD-9 codes (see appendix) or a calculated eGFR < 60 ml/min/1.73 m2 from the baseline outpatient SCr value. Differences in demographic, co-morbidity, and admission data between subjects with outpatient Scr data and those with only prior inpatient or no baseline SCr data were analyzed to address potential information bias. The VUMC Institutional Review Board approved the study protocol.
To diagnose and stage AKI, we compared changes in SCr values between estimated or established baseline values and the peak inpatient value within the first 7 days of hospitalization using the Acute Kidney Injury Network (AKIN) criteria.(9) Stage I injury was defined by an increase of ≥0.3 mg/dl or 50% increase from baseline, Stage II as a > 200% to 300% increase from baseline, and Stage III by either an increase in SCr to > 300% from baseline, a SCr of ≥ 4.0 mg/dl with an increase of at least 0.5 mg/dl, or the receipt of renal replacement therapy. SCr measurements within 24 hours prior to the administrative admission time were included to capture early creatinine values drawn from the emergency room or from acute visits prompting subsequent admission. We required a minimum 50% increase in SCr when defining AKIN stage III for patients receiving acute dialysis in order to favor inclusion of subjects receiving dialysis for AKI rather for acute intoxication without AKI.
Four variables were derived from the cohort data in order to evaluate the methods of detecting AKI occurrence: 1) an outpatient baseline serum creatinine which served as a reference standard for all comparisons, 2) an imputed serum creatinine based on an assumed baseline eGFR of 75 ml/min/1.73m2 which is recommended by the ADQI, 3) the minimum serum creatinine available during the first 7 days of hospitalization, and 4) the first serum creatinine available during hospital admission. Estimated Glomerular Filtration Rate (eGFR) for all outpatient and surrogate SCr measures was derived using the abbreviated Modification of Diet and Renal Disease (MDRD) equation [GFR (mL/min/1.73 m2) = 186 × (SCr)-1.154 × (Age)-0.203 × (0.742 if female) × (1.210 if African-American) (conventional units)].
Acute dialysis was captured using hospital discharge Current Procedural Terminology (CPT) codes 90935, 90937, 90945, 90947, 90989, 90993, 90921, 90925 assigned within the first 7 days of hospitalization. A board-certified nephrologist (EDS) validated the AKIN staging algorithm by auditing 20 randomly selected patients who met criteria for each AKIN stage and 20 patients who were not defined as having AKI using each of the four baseline definitions (320 evaluations total). In-hospital death was measured by retrieving the “discharge status” field of the administrative database for the hospital. Sixty-day mortality rates were calculated by linking the Social Security Death Index (SSDI) to the study data using social security number, date of birth, and patient name. SSDI data was updated to the latest version published by the Social Security Administration as of June, 2009.
For this descriptive study, frequencies were reported as percentages and groups were compared using the Pearson χ2 test or the Fisher-exact test. Continuous variables were compared using Student’s t-test or the Wilcoxon Rank Sum test, as determined by the underlying data distribution. Percent misclassification was calculated by comparing AKIN staging based on known baseline SCr compared to AKIN staging using each of the methods used for estimating baseline kidney function. Sensitivities and specificities for the diagnosis of AKI were calculated for each surrogate method using the outpatient serum creatinine as a study reference standard. Misclassification of 60-day mortality was defined as the sum of deaths following a false-positive diagnosis of AKI and deaths following a false-negative diagnosis of AKI divided by total deaths. Two-tailed p-values of <0.05 were considered to be statistically significant. All analyses were performed using SAS.
This work was funded by an R01 LM009965-01 grant from the National Library of Medicine, K24 DK62849 and U01DK82192 from the National Institute of Diabetes, Digestive and Kidney Diseases, and Clinical Translational Science Award 1UL-1RR024975 from the National Center for Research Resources. Edward Siew is supported by the Vanderbilt Mentored Clinical Research Scholar Program 5KL2 RR024977. Michael Matheny is supported by the Veterans Administration HSR&D Career Development Award CDA-08-020. This study is also partially supported by the Assessment and Serial Evaluation of the Subsequent Sequelae of Acute Kidney Injury (ASSESS-AKI) Study (5U01DK082192-02, 5U01DK082185-02, 5U01DK082223-02). Finally, the authors would also like to thank Andrew J. Vincz and Ioana Danciu for their technical assistance.
572.*,571.* except 571.1, 571.0
90935, 90937, 90945, 90947, 90989, 90993, 90921, 90925