The results of this multicenter study support the use of administrative data to identify children hospitalized with a provider-confirmed UTI; however, they also demonstrate the need to assess the accuracy of individual codes and identification algorithms before using them to compare hospital performance. Overall, the PPV for provider-confirmed UTIs was best when UTI was listed as the principal discharge code; accuracy was reduced with the inclusion of secondary discharge codes. Despite a high NPV, the PPV for laboratory-confirmed UTIs was poor, regardless of whether discharge diagnosis codes were listed as a primary or secondary diagnosis. Finally, incorporating additional data elements from UTI identification algorithms did not substantially alter the PPV for provider or laboratory-confirmed UTIs.
This study has several implications. First, we demonstrated that accurate identification of patients is a critical step to developing a quality measure. Even with few false negatives, the inclusion of patients without a condition under study (ie, false positives) can unpredictably affect hospital-level outcomes (eg, LOS). This level of misclassification can lead to erroneous conclusions about performance. Researchers and quality improvement specialists can use the predictive values from this study to best define their patient population, and thus improve the reliability and applicability of their outcomes data. For example, a community hospital wishing to improve care for “first-time UTI” should consider limiting the patient population to those patients with select principal UTI diagnosis codes (eg, pyelonephritis with PPV 100%), whereas hospitals conducting a project focused on decreasing nosocomial UTIs, for which UTI is unlikely to be the principal diagnosis, will require additional refinement of the population definition to improve the predictive value and subsequent applicability of the outcome measure.
ICD-9 code–based identification algorithms that include additional elements, such as age restrictions, are often used to increase the predictive value to best define patient populations involved in a care process. Yet, application of such identification algorithms did not substantially alter the PPV. In addition, applying previously published identification strategies, including the AHRQ pediatric quality indicator for UTI admission rate in children as well as that described by Conway et al yielded similar results.3,7
Limiting the population to children with a principal diagnosis of UTI, however, greatly improved the validity of the case definition, regardless of the additional criteria included. Thus, other than considering only principal diagnosis in the identification of UTI hospitalizations, the use of increasingly complex algorithms does not necessarily result in improved case validity, and may actually reduce generalizability. The observed effect of identification strategies on LOS illustrates the importance of accurately definition the target population before the establishment of a benchmark for best practice.
This study had several limitations. First, this study did not capture patients who may have had a UTI but were not assigned a charge code for a UA or urine culture (ie, false negative). However, inclusion of every patient with a UTI was not critical to the quality of the measurement and monitoring of UTIs because we have no reason to suspect that such exclusions occurred systematically. Accordingly, in this study we used the PPV (the proportion of patients with a UTI that is correctly diagnosed), rather than sensitivity, to compare accuracy between different identification strategies.15
Also, for study inclusion children were required to have a UA or urine culture performed at the study hospital. This criterion led to exclusion of patients transferred from an outside institution and those whose urine testing was performed in an ambulatory-care setting, factors that should be considered when these results are applied to databases that do not include procedure or laboratory testing codes. A second limitation was that our findings are based on administrative data from 5 children's hospitals that contribute data to the PHIS database. As a result, it is not clear whether these results can be applied to other administrative data sets or to non–children's hospitals. Third, the reliability of each site's data abstractors was not assessed; however, abstraction procedures were reviewed and tested at each site and discussed in a collaborative fashion before study implementation. Finally, there was substantial discrepancy between the laboratory and provider-confirmed gold standards, with provider-confirmed UTIs resulting in significantly improved PPV. Although it is logical that provider documentation would enhance detection of UTIs, especially in cases in which UTI laboratory studies were performed at another facility, some providers still may overdiagnose UTIs by not adhering to the national standards for laboratory diagnosis. The degree to which underreporting of urine studies lowered the PPV of the laboratory-confirmed gold standard is unknown, although it is likely small because it would be limited to patients with a repeat UA or urine culture after transfer.