The study was conducted at two institutions, both of which participate in the Chicago Antimicrobial Resistance Project: Cook County Hospital, a 600-bed public teaching hospital and Provident Hospital, a 120-bed community hospital. As part of the project, we created a data warehouse by using data from the admission and discharge, pharmacy, microbiology, clinical laboratory, and radiology department databases (9
). The data warehouse is a relational database that allows us to link data for individual patients from these separate departments. Data are downloaded from the various departmental databases to our warehouse once every 24 hours; therefore, the algorithms can be applied to clinical data from the previous day.
Facility-specific procedures exist for acquiring and processing blood specimens. At both hospitals, the decision to obtain blood cultures was determined solely by medical providers, without input from infection control professionals or study investigators. After CVC removal, the decision to send a distal segment of the CVC for culture was at the discretion of the medical care provider; both microbiology laboratories accepted these specimens for culture. Since considerable interfacility variability likely exists in CVC culture practices beyond Cook County and Provident Hospitals, we decided not to incorporate these culture results into our computer algorithms.
Blood cultures were obtained and processed at Cook County and Provident Hospitals by using similar methods. At Cook County Hospital, blood cultures were obtained by resident physicians or medical students. At Provident Hospital, blood cultures were obtained by phlebotomists outside of the intensive-care units and by a nurse or physician in the intensive-care unit. At each hospital, blood cultures were injected into Bactec (Becton Dickinson, Inc., Sparks, MD) bottles and incubated for up to 5 days in an automated blood culture detection system. When microbial growth was detected, samples were spread onto solid media and incubated overnight.
Using data from several sources, we compiled a list of all patients who had a positive blood culture hospitalized on inpatient units other than the pediatric or neonatal units from September 1, 2001, through February 28, 2002 (study period). Positive blood cultures obtained <2 days after hospital admission and not evaluated by an infection control professional were excluded. Positive blood cultures obtained within 5 days of the initial positive blood culture were considered as part of the same episode; i.e., these blood cultures were considered polymicrobial infections. At Cook County Hospital, we studied a random sample of positive blood cultures. At Provident Hospital, since a relatively small number of cultures were obtained during the study period, we evaluated all positive blood cultures. Approval was obtained by the local and CDC human participant review boards.
We used the CDC definition for primary, CVC-associated, laboratory-confirmed bloodstream infection (10
). Four study investigators, all of whom had previous experience applying these definitions, performed retrospective medical record reviews. Two investigators independently reviewed each medical record. If there was a judgment disagreement between the two investigators, a third reviewer categorized the blood culture. Investigators were blinded to other investigators' reviews and to determinations made by review and by computer algorithms. To minimize the likelihood of investigator interpretation approximating the computer algorithm, i.e., systematic bias in definition interpretation, the details of the computer algorithms were not disclosed to three of the four reviewers. The reviewer who participated in the construction of the computer algorithms functioned in the same capacity as the other three reviewers (i.e., all four reviewers could participate in the initial or final reviews).
Infection Control Professional Review
During the study period, infection control professionals at Cook County and Provident Hospitals performed prospective hospitalwide bloodstream infection surveillance using the CDC definitions (10
). Six infection control professionals submitted data, four at Cook County Hospital and two at Provident Hospital; all were registered nurses and had 10–30 years of infection control experience. All six had attended a 1-day surveillance seminar conducted by CDC personnel and had access to an infection control professional who had attended a CDC-sponsored infection surveillance training course; four were certified in infection control.
At Cook County Hospital, a list of all positive blood culture results was generated by a single person in the microbiology laboratory. Duplicates (i.e., the same species identified within the previous 30 days) were excluded, and the list was distributed to the infection control professionals. For those patients who had not been discharged, the infection control professionals reviewed the medical chart, and if their assessment differed from the medical record documentation, they could discuss the case with the medical team. For patients who had been discharged, only the medical record was reviewed. For polymicrobial cultures (i.e., >1 organism isolated from a blood culture), infection control professionals categorized each isolate. The infection control professionals did not participate in the medical team's ward rounds. At Provident Hospital, the procedures were similar except that the laboratory printed out all positive culture results, and the infection control professional manually excluded duplicate results.
Determinations were recorded on a standardized, scannable form, and the forms were sent to a central location, where they were evaluated for completeness and then scanned into a database. In cases where the infection control professional did not record whether the infection was hospital- or community-acquired, we categorized the infection as hospital-acquired if it was detected >2 days after hospital admission.
We evaluated several methods to categorize blood cultures. First, we evaluated a simple method that required only a computer report of a positive blood culture recovered >2 days after admission plus manual determination of whether a CVC was present.
Second, after consultation with infectious disease clinicians, we developed rules that were combined into more sophisticated computer algorithms (, ). Two rules were developed for two of the determinations that were required. For determining infection versus contamination, rule B1 used only microbiology data, while rule B2 used microbiologic and pharmacy data. For determining primary versus secondary (i.e., the organism cultured from the blood is related to an infection at another site) bloodstream infection, rule C1 was limited to a 10-day window, while rule C2 extended throughout the hospitalization. Since two options existed for two separate rules, these rules were combined into four separate algorithms. We report the results of the algorithms that had the best (rules A, B2, C2, and D) and worst (rules A, B1, C1, and D) performance. Consistent with the manual methods, polymicrobial cultures were considered a single event. Polymicrobial blood cultures were considered an infection if any isolate recovered from the same culture met infection criteria, and, in contrast to the manual methods, were considered a secondary bloodstream infection if any isolate that met infection criteria also met criteria to be classified as secondary. Third, since we could not automate CVC detection, we also evaluated augmentation of automated bloodstream infection detection with manual determination of a CVC.
Computer algorithms and corresponding NNIS system definitions to categorize blood culture isolates, September 2001–February 2002, Cook County Hospital, Chicago, Illinoisa
Figure 1 Flowchart displaying the determinations necessary for categorizing positive blood culture by computer algorithm. The rules described in are in bold. Blood cultures were obtained from patients at Cook County and Provident Hospitals, September 1, (more ...)
For Provident Hospital, since the number of positive blood cultures evaluated by each rule was relatively small, we do not report the performance characteristics. We do report the results for the best and worst computer algorithms at each hospital and at both hospitals combined.
For polymicrobial cultures, we analyzed the results at the level of the blood culture. We were primarily interested in evaluating the detection of hospital-acquired, primary, CVC-associated bloodstream infections; therefore, by investigator or infection control professional review, if any isolate from a polymicrobial culture met the necessary criteria, the blood culture was classified as a hospital-acquired, primary, CVC-associated bloodstream infection.
We present the results of comparisons for the blood cultures that were evaluated by all methods. For calculation of sensitivity, specificity, and predictive values, we considered the investigator review to be the reference standard. Next, we calculated the agreement between investigator review and the other methods using the kappa statistic (κ) (11
). Since all organisms that were not common skin commensals were considered an infection, we included only common skin commensals to evaluate the rule distinguishing infection versus contaminant.
We report bloodstream infection rates per 1,000 patient-days for certain units in the hospital. Hospital units were aggregated according to the type of patient-care delivered, as identified by hospital personnel. For example, data from all nonintensive care medical wards were aggregated. Also, because of the relatively low number of patient-days in the burn, trauma, and neurosurgical intensive-care units (ICU), we aggregated the bloodstream infection rates for these units and report them as specialty ICUs. We calculated the Pearson correlation coefficient for bloodstream infection rates determined by investigator review versus other methods, stratified by hospital unit. Since only a sample of blood cultures was evaluated at Cook County Hospital, the rates were adjusted to account for the unit-specific sampling fraction. We also calculated the Pearson correlation coefficient, comparing the number of bloodstream infections per month identified by investigator review versus the other methods. All analyses were performed by using SAS statistical package version 8.02 (SAS Institute Inc., Cary, NC).