|Home | About | Journals | Submit | Contact Us | Français|
Substantial care variation occurs in a number of pediatric diseases.
We evaluated the variability in health care resource utilization and its association with clinical outcomes among children, aged 1–18 years, hospitalized with community acquired pneumonia (CAP). Each of 29 children’s hospitals contributing data to the Pediatric Hospital Information System was ranked based on the proportion of CAP patients receiving each of 8 diagnostic tests. Primary outcome variable was length of stay (LOS), re-visit to the ED or readmission within 14 days of discharge.
Of 21,213 children hospitalized with non-severe CAP, median age was 3 years (interquartile range [IQR], 1–6 years). Laboratory testing and antibiotic usage varied widely across hospitals; cephalosporins were the most commonly prescribed antibiotic. There were large differences in the processes of care by age categories. The median LOS was 2 days (IQR, 1–3 days) and differed across hospitals; 25% of hospitals had median LOS >= 3 days. Hospital-level variation occurred in 14-day ED visits and 14-day readmission, ranging from 0.9 to 4.9% and 1.5% to 4.4%, respectively. Increased utilization of diagnostic testing was associated with longer hospital LOS (p=0.036) but not with probability of 14-day readmission (Spearman’s ρ = 0.234; p=0.225). There was an inverse correlation between LOS and 14-day revisit to the ED (ρ=−0.48, p=0.013).
Wide variability occurred in diagnostic testing for children hospitalized with CAP. Increased diagnostic testing was associated with a longer LOS. Earlier hospital discharge did not correlate with increased 14-day readmission. The precise interaction of increased utilization with longer LOS remains unclear.
The national emphasis on value-driven healthcare (i.e., how can we achieve better quality outcomes per dollar spent)1 has important implications for the care of hospitalized children. Studies have documented substantial care variation among children hospitalized with a number of acute illnesses.2–7 Although patient- and disease-level characteristics influence management and outcomes, hospital-level factors may also contribute to variation in care. For example, institutional variation contributed to increased costs as well as increased length of stay (LOS) for children hospitalized with lower respiratory tract infections, even after controlling for sociodemographic variables and illness severity.2 This finding suggests that institutional culture both influences and contributes to unnecessary care variation, and thus represents a key potential target for improving the efficiency and quality of care delivered to children hospitalized for management of common illnesses. Reducing unwanted variation may also lead to improved outcomes and reduce costs.2,4,8
Community-acquired pneumonia (CAP), a common and potentially serious infection in childhood, affects approximately 3 million children and accounts for >200,000 hospitalizations each year in the United States.9–13 Despite decades of research and the enormous burden of CAP, there is a lack of consensus regarding optimal management strategies for pediatric CAP. Due in large part to the lack of sufficiently sensitive and specific diagnostic tests, a range of studies—including bacterial cultures, viral testing, laboratory inflammatory markers and other indices, and radiographic investigations—are variably employed in an effort to inform decision-making, and ultimately, improve outcomes. Few studies, however, have detailed the influence of variations in care utilization on outcomes for children hospitalized with CAP. Precise knowledge of care processes which improve outcomes for pediatric CAP, and conversely those which serve only to increase costs without clear benefits, is essential for developing evidence-based practice guidelines, establishing quality benchmarks, and improving children’s care.
Using a national, multi-institutional retrospective cohort of children hospitalized with CAP, we sought to describe the variability in health care resource utilization and determine its association with outcomes among children hospitalized with CAP.
Data for this multicenter retrospective cohort study were obtained from the Pediatric Health Information System (PHIS), which contains resource utilization data from 40 freestanding tertiary children’s hospitals with emergency departments (ED). Participating hospitals are located in non-competing markets of 27 states plus the District of Columbia and account for 15% of all pediatric hospitalizations in the United States in 2009 (677,291 of 4,508,323 admissions).14, 15 These hospitals provide discharge data including patient demographics, diagnoses, and procedures. Billing data detail all drugs, radiologic imaging studies, laboratory tests, and supplies charged to each patient. Data are de-identified prior to inclusion in the database, however encrypted medical record numbers allow for tracking individual patients across admissions. The Child Health Corporation of America (Shawnee Mission, KS) and participating hospitals jointly assure data quality as described previously.6,16 In accordance with the Common Rule (45 CFR 46.102(f)) and the policies of The Children’s Hospital of Philadelphia Institutional Review Board, this research, using a de-identified dataset, was not considered human subjects research.
Children aged 1–18 years hospitalized for CAP were eligible for this study if they were discharged from participating hospitals between July 1, 2005 and June 30, 2010. Subjects were included if they satisfied one of the following International Classification of Diseases, 9th Revision (ICD-9) discharge diagnosis code criteria: 1) Primary diagnosis of pneumonia (481–483.8, 485–486); 2) Primary diagnosis of a pneumonia-related symptom (780.6 or 786.00–786.52 [except 786.1] anda secondary diagnosis of pneumonia, empyema (510.0, 510.9), or pleurisy (511.0, 511.1, 511.9); or 3) Primary diagnosis of empyema or pleurisy and a secondary diagnosis of pneumonia.
For consistency, only the 29 (73%) hospitals that had complete data for the entire study period were included. Children were excluded for the following reasons: 1) inter-hospital transfer, 2) chronic comorbid condition predisposing to severe, recurrent, or healthcare-associated pneumonia (e.g., cystic fibrosis, malignancy, sickle cell disease) as defined by a previously reported classification scheme, 17 and 3) no antibiotic therapy on the first calendar day of hospitalization, suggesting that pneumonia was not present or diagnosed at the time of admission. Additional exclusion criteria were applied to create a cohort of children presenting with non-severe CAP. Thus, further exclusion criteria aimed at limiting the degree of illness included any of the following if they occurred on the first calendar day of hospitalization, suggesting more severe illness at initial presentation: intensive care unit admission, pleural drainage, or death. We did not exclude children who worsened during the hospitalization as this decline may reflect differences in treatment rather than in baseline disease severity.
Patients with chronic comorbid conditions were identified using a previously reported classification scheme.17 Pleural drainage procedures were identified using ICD-9 procedure codes for thoracentesis (34.91), chest tube placement (34.04), video-assisted thoracoscopic surgery (VATS; 34.21), and thoracotomy (34.02 or 34.09). Billing data were used to classify receipt of medications, including antibiotics.
The primary exposure of interest was hospital-level initial resource utilization. Initial resource utilization was determined by ranking each hospital from 1 (lowest) to 29 (highest) based on the proportion of patients with CAP receiving each of the following on the first day of hospitalization, (including testing performed in the emergency department): arterial blood gas measurement, blood culture, C-reactive protein, erythrocyte sedimentation rate, complete blood count, serum electrolytes, chest radiographs, and virus detection studies. These 8 ranks were then summed by hospital to provide a hospital-level utilization score with a minimum of 8 and a maximum of 232.
The primary outcome was hospital length of stay (LOS) in days. Readmission within 14 days of index discharge was a secondary outcome measure.
Hospital level covariates were summarized using median, inter-quartile range (IQR), and range values for continuous variables, and frequencies and percentages for categorical variables. We stratified processes of care, treatment, and outcomes by age groups (1–5, 6–12 and 13–18 years) and compared categorical and continuous variables across these groups using chi-square and Kruskal-Wallis tests, respectively.
To assess the relationship between utilization and outcomes, we fit linear regression models treating the study outcome as the response and the hospital-level utilization score as the predictor. Since this was a hospital-level analysis, we only had 29 data points, and scrutinized each data point as an outlier based on the following four diagnostics: residuals, the Studentized residuals, the leverage plot, and Cook’s D.18 We removed a data point only if all 4 methods agreed that the point was an outlier. After removing outliers, we fit the linear regression and plotted the relationship with the 95% confidence limits. To further minimize the impact of potential outliers, we repeated the analysis while excluding hospitals identified as outliers by any one of the 4 methods.
Because early discharge (i.e., a shorter LOS) could lead to readmission, we examined the association between LOS and 14-day readmission and between LOS and 14-day revisit to the ED: Spearman’s rank correlation coefficient was used. All statistical analyses were performed using the statistical software SAS (version 9.1, SAS Institute, Inc, Cary, NC), and p-values < 0.05 were considered statistically significant.
During the study period, 43,819 children were hospitalized with a diagnosis of CAP. After selection criteria were applied excluding children transferred from other institutions, children with chronic comorbid conditions, children who received antibiotics, ICU admission or pleural drainage on the first day and those who died on the first day (Fig., Supplemental Digital Content 1, http://links.lww.com/INF/B259), the final cohort included 21,213 children classified as having non-severe CAP.
The median age was 3 years (IQR, 1–6 years). Most children (72%) were between 1–5 five years of age; children aged 6–12 and 13–18 years accounted for 22% and 5% of CAP, respectively. Approximately 53% of patients were male. Patient race included non-Hispanic Whites (40%), non-Hispanic Blacks (22%), Hispanics (29%), Asian (3%), and other races (6%). Hospitalizations were most common in the winter (38%).
Overall and age-stratified processes of care are shown in Table 1. Most of the differences in the performance of these measures across age categories, though statistically significant, were relatively minor with the exception of serum electrolytes and non-specific markers of inflammation (i.e., C-reactive protein, erythrocyte sedimentation rate). Each of these tests was more commonly performed in older children. The rate of blood cultures remained fairly stable across age groups. Antibiotic prescribing also differed by age. Cephalosporins used as single-agent therapy were the most common antibiotic regimen, although the use of this treatment strategy decreased with increasing age. The most common cephalosporins used were ceftriaxone (56%), cefuroxime (22%) and cefotaxime (19). Combination antibiotic therapy was also common; co-administration of a cephalosporin with a macrolide antibiotic was the most frequent combination (21% overall), and this treatment strategy increased with increasing age. Antimicrobial agents with expected activity against methicillin-resistant Staphylococcus aureus (vancomycin/clindamycin) were administered in combination with cephalosporins or macrolides to approximately 20% of children.
There was marked variability in laboratory testing across hospitals (Table 2). The large variability occurred in almost every laboratory test evaluated (Figure 1). For example, complete blood counts (CBC) and blood cultures were virtually always obtained at some hospitals and almost never at other hospitals. Although specific antibiotic use varied across institutions, cephalosporins were the most common antibiotic prescribed, either as a single agent or in combination, accounting for over 80% of all antibiotic use (Fig., Supplemental Digital Content 2, http://links.lww.com/INF/B260). Penicillins or aminopenicillins were rarely used.
The overall median LOS was 2 days (IQR, 1–3 days). Twenty-five percent of children aged < 12 years were hospitalized >= 3 days, and 25% of children aged 13–18 years were hospitalized >= 4 days. Overall, 491 (2.3%) patients required readmission within 14 days of index discharge. This proportion was higher for children aged 13–18 years (4.9%) compared with children aged 1–5 (2.2%) and 6–12 (2.1%) years (p=0.004).
LOS differed across hospitals. Though the median of the median hospital LOS was 2 days, 25% of hospitals had a median LOS ≥3 days. There was also hospital-level variation in the probability of 14-day readmission, ranging from 1.5% to 4.4% with the probability of a 14-day ED return visit ranging from 0.9% to 4.9%. There was no correlation between LOS and probability of readmission across hospitals (ρ=0.12, p=0.549) but there was an inverse correlation between LOS and 14-day revisit to the ED (ρ=−0.48, p=0.013, Figure 2).
After identification and removal of outliers based on our four regression diagnostic strategies, we observed an association between increased resource utilization and duration of hospitalization; institutions that performed more diagnostic testing had a longer LOS (Figure 3 and Fig., Supplemental Digital Content 3, http://links.lww.com/INF/B261). This association persisted when outlier removal was based on at least one diagnostic strategy (p=0.028; six hospitals excluded, not shown). There was no association between the utilization of diagnostic testing and probability of readmission (Spearman’s ρ= 0.234; p=0.225).
We found substantial variation in hospital-level processes of care, empiric antibiotic selection, LOS, and 14-day readmission in this multicenter study of children hospitalized with CAP. Increased utilization of diagnostic testing was associated with a longer hospital LOS. Hospital-level differences in LOS correlated inversely with 14-day post-discharge ED visit but despite this there was no association with readmissions. Our findings highlight the need for strategies to identify the most efficient processes of care for children with CAP and to determine when children with CAP are sufficiently stable for hospital discharge.
The processes of care evaluated in this study varied widely by institution, raising questions about the utility of some of these tests. Although we did not apply a measure of severity for pneumonia in these children such as the pneumonia severity index used in adults,19 we did attempt to study a relatively uniform population by including only children admitted directly, rather than by inter-hospital transfer, to an inpatient ward (rather than an intensive care unit), who received antibiotics but no pleural drainage procedures and who lacked underlying chronic comorbid conditions. While the inverse correlation between LOS and 14-day re-visit to the ED is concerning for incompletely resolved illness it certainly does not support a lesser degree of illness for patients with shorter LOS. Additionally, the absence of a difference in the rate of 14-day readmission would favor small rather than large differences in illness severity. Despite this relative homogeneity, certain processes of care (e.g, CRP) were never or almost never used at some institutions while at other hospitals they were employed in nearly all patients, even when the median use of the test by institutions was almost zero (i.e., fewer than 10% of patients). Thus, large differences in patient population do not appear to account for variation in testing.
Substantial variability also occurred in antibiotic preference in this cohort both by institution and by age group. Furthermore, the majority of children received broad spectrum antibiotic therapy although this cohort consisted of patients with uncomplicated, non-severe CAP. This wide variability in antibiotic preference highlights the lack of clear national standards for the management of CAP in children. For adults admitted with CAP, the American Thoracic Society and the Infectious Diseases Society of America guidelines provide principles for empiric antibiotic therapy.20,21 In a study of adults with CAP, adherence to the American Thoracic Society/Infectious Diseases Society of America antibiotic therapy recommendations was associated with improved outcomes compared with non-adherence.22 The recently published guidelines for children with CAP will hopefully improve standardization of diagnosis and treatment.23
Increased use of some processes of care was associated with a longer LOS. The effect on LOS may have been magnified by several hospitals at the two extremes of care utilization that also had large differences in LOS. While evident at the institutional level which were ranked according to the total number of laboratory tests, the effect of increased utilization may not hold true for any individual child with CAP or even any individual test. The American Thoracic Society/Infectious Diseases Society of America guidelines for hospitalized adults with CAP recommend chest radiographs, complete blood count, blood cultures, electrolytes and oxygenation assessment. Some diagnostic processes such as complete blood counts and acute phase reactants (ESR, CRP) have poor sensitivity and specificity in the diagnosis of bacterial pneumonia in children.24–27 Furthermore, children who present to EDs have a low rate of bacteremia, and falsely positive blood cultures may contribute to unnecessary prolonged stay and antibiotic use.28
Although the correlation of increased utilization of processes of care with longer LOS occurred in this population, the precise interaction of processes of care with study outcomes remains unclear. The observed association between use of processes of care and LOS has two plausible explanations. First, patients at institutions with greater use of processes of care were indeed sicker and thus the diagnostic testing helped to establish the severity of their illness. We cannot exclude this possibility but the wide variation in each individual test by institution in a fairly homogenous population makes differences in illness severity an unlikely explanation. Second, increased testing may have occurred as a condition of variation in institutional practice patterns. In this case, certain processes of care may beget further testing, additional interventions, increased costs and a longer LOS. Previous studies in children have established diverse practices in other acute illnesses in children which have resulted in varied costs and outcomes.2–7
This study had several limitations. First, we applied only one approach to measuring and ranking utilization of processes of care. In so doing, we may have forced rankings on hospitals though there may not have been substantial variation in the performance of specific tests. Second, we considered that children with more severe disease would likely have a longer LOS, rendering the comparison of processes of care among children with less severe disease complicated. Thus, by design, our study was restricted to a relatively homogenous group of patients without evidence of severe or complicated disease such as the presence of an underlying chronic comorbid condition or admission to the ICU, receipt of antibiotics or a pleural drainage procedure on the first hospital day. Despite these efforts, we cannot completely rule out residual confounding by disease severity which is an inherent limitation of our data source and the lack of standardized severity scale for children with CAP. We were unable to evaluate the rate of post-discharge physician visit in this population. However, the fact that children from institutions with shorter length of stay also had higher rates of 14-day ED re-visit may be explained in many ways such as incomplete resolution of symptoms or inadequate education of family members on discharge. Yet the higher rate of 14-day ED visits does not support less severe illness in children with shorter LOS and lower rates of diagnostic testing. Third, our assessment of care processes was limited to diagnostic testing. Other hospital-level factors may influence hospital LOS. For example, it is possible that physicians across hospitals used comparable criteria to make discharge decisions but that hospitals varied in the efficiency with which they could discharge patients, leading us to falsely identify an association between higher diagnostic test utilization and a longer LOS. Fourth, variability in outpatient care which we could not measure may have affected readmission rates. Fifth, if children were readmitted to a hospital that did not contribute to PHIS we could not track those readmissions. Finally, while we identified an association between increased utilization of diagnostic testing and longer hospital LOS, we could not determine which specific tests or whether any specific tests ought to be performed less frequently.
In conclusion, we demonstrated wide variation in hospital-level processes of care, antibiotic therapy and outcomes in this study of children hospitalized with non-severe CAP. The absence of established guidelines during the study years may have contributed to this wide variability in testing, suggesting an important target for influencing both the efficiency and quality of pediatric health care. Guidelines of care that promote certain laboratory studies may need to be evaluated in light of these findings. Reducing practice variability is an important step in improving quality of care while reducing costs, and the recent publication of diagnostic and management guidelines for childhood CAP, jointly sponsored by the Pediatric Infectious Diseases Society and the Infectious Diseases Society of America,23 may contribute to reduce variability in the care of children with pneumonia.
SDC 1. Flow chart of study cohort. aRefers to events occurring on the first day of hospitalization. (figure)
SDC 2. Hospital-level variation in empiric antibiotic therapy among children with community-acquired pneumonia. Each column on the x-axis represents data from one hospital. (figure)
SDC 3. Association of process measures and length of hospital stay. Performance of process measures was associated with a longer hospital length of stay while excluding outliers detected by at least one method (p=0.028; 6 hospitals excluded). Each circle represents data from one hospital; excluded hospital data are not shown. The solid line represents the linear regression line and shaded areas are the 95% confidence limits.
Dr. Matthew Hall had full access to the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Sources of support: Dr. Shah received support from the National Institute of Allergy and Infectious Diseases (K01 AI73729) and the Robert Wood Johnson Foundation under its Physician Faculty Scholar program. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflicts of interest: None