|Home | About | Journals | Submit | Contact Us | Français|
Laboratory practice in the preanalytical phase of antimicrobial susceptibility testing (AST) was evaluated in 102 hospital, reference, physician office-clinic, and public health laboratories in Washington state. Surveys were sent to evaluate (i) use of NCCLS/CLSI (formerly NCCLS) AST performance standards, (ii) technical competence in AST case studies, challenging knowledge of contemporary testing issues, and (iii) choice of antimicrobial agents to test for Streptococcus pneumoniae. Numerous deficiencies were identified in the survey: (i) initially only 40% of the laboratories surveyed used current NCCLS/CLSI AST performance standards, (ii) the rate of accurate responses for three different case studies ranged from 29% to 69%, and (iii) variation was noted in the choice of antimicrobials tested against invasive isolates of S. pneumoniae. These deficiencies could affect therapy and detection of antimicrobial resistance. Several educational programs were implemented to improve AST policies and practices, and a follow-up survey indicated that four intervention strategies were most effective: (i) regional technical workshops, (ii) National Laboratory Training Network teleconferences, (iii) use of the Centers for Disease Control and Prevention (CDC) CD-ROM on AST, and (iv) the CDC Multilevel Antimicrobial Susceptibility Testing Resource website. The interventions could be implemented more widely in the United States to improve AST knowledge and practices.
Isolation, identification, and antimicrobial susceptibility testing of bacterial isolates from diagnostic specimens are critical functions of the clinical microbiology laboratory (15). A recent article documented several potential barriers to achieving the goal of rapidly detecting and reporting resistant bacteria; these included failure of laboratory and clinical staff to accept shared responsibility for prompt antimicrobial susceptibility testing (AST) and reporting, inadequate training of laboratory staff in AST, erroneous interpretation of susceptibility test results by clinical staff, and lack of systems to collect, analyze, and report results of process monitors (9). In 1998, a report from the CDC Active Bacterial Core Surveillance/Emerging Infections Program noted that some laboratories did not provide sufficiently accurate susceptibility testing results for effective antimicrobial resistance surveillance (2). Of 329 laboratories, 52 (16%) could not identify isolates with reduced susceptibility to vancomycin and approximately 40% of the laboratories did not confirm results for vancomycin-intermediate or -resistant Staphylococcus aureus. Rates of adherence to recommended NCCLS/CLSI (formerly NCCLS) AST performance standards (M7-A4, M100-S8, and M100-S9) were lower in laboratories of hospitals with fewer than 200 beds than in those of larger institutions, showing that greater compliance and variability in AST practices was associated with laboratory demographics. For example, managed-care-based laboratories were significantly less likely to confirm detection of S. aureus with reduced susceptibility to vancomycin than other types of laboratories. A study by the CDC published in 2000 (18) indicated that some laboratories participating in Project ICARE could not consistently detect resistance to carbapenems and extended-spectrum cephalosporins. Tenover and colleagues (19) have also documented similar problems with respect to the detection of emerging antimicrobial resistance in international laboratories. More recently, a study of microbiology laboratories in small rural hospitals in the northwestern United States noted that only 37 (50%) of 74 hospital laboratories published annual summaries of antimicrobial resistance patterns (i.e., annual cumulative antibiograms) (17). Although primary screening methods for most organisms were deemed adequate, only 30 (55%) of 55 hospital laboratories reported testing capable of adequately confirming suspected high-level penicillin resistance in pneumococcal isolates. Only 4 (20%) of 20 hospital laboratories employing disk diffusion as their primary testing method for pneumococci performed a MIC test for confirmation of penicillin-resistant isolates. The authors also advocated the expanded use of reference laboratories to confirm suspected resistance as one approach to overcoming the potential for misreporting antimicrobial susceptibility in smaller communities.
Inappropriate AST reporting can affect antimicrobial resistance estimates in a community, which in turn can impact empirical therapy recommendations (7, 10, 13). Studies conducted in Washington state (11, 12) noted that while most laboratories in hospitals of more than 200 beds used appropriate NCCLS/CLSI AST standards, most small-community laboratories did not use these AST standards and many performed AST infrequently. In addition, only 20% to 40% of S. pneumoniae isolates submitted by community hospital laboratories to commercial reference laboratories were tested by a MIC method as required to conform to NCCLS/CLSI AST guidelines.
The goal of the National Laboratory System (NLS) is to strengthen critical laboratory testing to (i) improve patient outcomes, including patient safety, (ii) improve public health surveillance and investigations through systematic improvement in the delivery of laboratory services, (iii) establish and enhance communication and interaction between laboratories and health care providers, and (iv) improve collaboration and coordination between public health and private laboratories. One critical area of laboratory testing is the detection of antimicrobial resistance in microorganisms in conformance with current NCCLS/CLSI AST performance standards (4); another is the need to develop a system for the rapid detection and reporting of resistant bacteria (1, 9). This investigation evaluated the effectiveness of novel approaches to improve AST practices through better access to, and knowledge of, practice guidelines. If effective, these approaches could be used more widely to strengthen the NLS.
A comprehensive assessment of laboratory practice in AST was undertaken in 2001 via a questionnaire sent to all 102 Washington state clinical microbiology laboratories. The questionnaire included case studies designed to assess knowledge of AST procedures and adherence to current NCCLS AST performance standards. Three case studies were used. Case study 1 involved steps taken to test an Enterococcus sp., case study 2 involved testing for methicillin-resistant S. aureus (MRSA), and case study 3 involved testing for S. pneumoniae. The questionnaire also assessed the utilization of the 2001 NCCLS performance standards for antimicrobial susceptibility testing (M100-S11) (16) and antimicrobials used to test invasive S. pneumoniae isolates.
A second survey was distributed in 2002 to determine whether any improvements in AST laboratory practice had been initiated by laboratories since the distribution of data from the 2001 survey. The survey addressed how the laboratories became aware of the 2001 results, changes that had been made in their AST practices, how AST was performed for invasive S. pneumoniae isolates, and other quality improvement steps that had taken place since the first survey.
A series of quality improvement intervention strategies were implemented in 2003 and 2004 which were broadly characterized as either national or local in scope. First, at the local level, five technical workshops on AST, utilizing local faculty and national NCCLS AST guidelines, were implemented to address problems in antimicrobial susceptibility testing that were noted in previous questionnaire surveys. Second, a “Train the Trainer” workshop was conducted to train local workshop faculty on the use of the CDC CD-ROM entitled “Antimicrobial Susceptibility Testing—A Self Study Program.” Third, the NCCLS AST documents were distributed free of charge to laboratories performing AST. A website was launched, and a quarterly newsletter (the University of Washington Clinical Laboratory Initiative [CLI] News) was initiated to disseminate information on AST. At a national level, a regional teleconference entitled “Quality Assurance in Clinical Microbiology and Antimicrobial Susceptibility Testing” was presented.
In 2005, a third comprehensive assessment of AST practice was undertaken to evaluate the effectiveness of the interventions. A survey was developed to evaluate whether laboratories were using current NCCLS/CLSI AST guidelines, the performance of the laboratories in case studies, whether laboratories had changed their policies or practices for identifying resistant organisms, whether laboratories had changed their policies or practices on when to send organisms to a reference laboratories, choice of AST panels by laboratories, and whether a cascading reporting policy had been implemented for AST results. The questionnaire was distributed to 115 laboratories licensed by the state of Washington that provided some level of microbiology services, including those that participated in the 2001 survey. Data were analyzed with an EXCEL database. A two-sided, two-sample test for equality of proportions was used for statistical analysis. Results (with 95% confidence intervals and P values) are included in Table Table11 and Table Table2.2. Statistical calculations were conducted only for data that had sufficient sample size to be meaningful.
In 2001, 61 (60%) surveys were returned from 102 laboratories. Of these, 47 (64%) responses were submitted by hospital laboratories, 5 (7%) by reference laboratories, 15 (21%) by physician office laboratories, and 1 (1%) by a public health laboratory that performed AST. Of the 73 laboratories, 68 (93%) indicated they performed AST, and only 27 (40%) of 68 laboratories indicated that they were using current NCCLS AST performance standards. Of these, 14 (93%) of 15 laboratories in moderate-to-large hospitals (>100 beds) were using current CLSI AST performance standards, as were 9 (28%) of 32 laboratories in small hospitals (<100 beds), 1 (20%) of 5 reference laboratories, and 3 (20%) of 15 physician office laboratories; the single public health laboratory surveyed was not using the current CLSI AST performance standards (Table (Table1).1). The numbers and percentages of laboratories providing accurate responses for the three different case studies were 15 (29%) of 52 laboratories for case study 1 (enterococci), 36 (69%) of 54 laboratories for case study 2 (MRSA), and 28 (54%) of 52 laboratories for case study 3 (pneumococci) (Table (Table2).2). Laboratories of physician's offices did not participate in case studies, because the clinical presentations would not occur in a physician's office.
Of 41 laboratories, 30 (73%), including 10 (91%) laboratories of 11 moderate-to-large hospitals, 17 (68%) of 25 small hospitals, and 3 (60%) of 5 reference laboratories, complied with 2001 NCCLS AST standards and tested both penicillin (or oxacillin) and a third-generation cephalosporin. The distribution of other antimicrobial agents tested included erythromycin (77%), vancomycin (70%), clindamycin (49%), tetracycline (42%), trimethoprim-sulfamethoxazole (40%), levofloxacin (26%), ofloxacin (4.0%), chloramphenicol (15%), doxycycline (8%), cephalothin (4%), ceftizoxime (4.%), ampicillin (4%), amoxicillin-clavulanic acid (4%), rifampin (4%), cefonicid (2%), ciprofloxacin (2%), and gentamicin (2%). The number of antimicrobial agents tested per panel ranged from 1 to 15.
Of 102 questionnaires, 61 (60%) were returned in 2002, 33 (46%) from hospital laboratories, 4 (6%) from reference laboratories, 22 from (31%) physician office laboratories, and 2 (3%) from public health laboratories. Sixteen (49%) of the questionnaires from 33 hospital laboratories indicated that personnel were not aware of the 2001 survey results even though they were published in the state laboratory's monthly newsletter, available on the project website, and presented at local laboratory professional meetings. Five (15%) of 33 hospital laboratories reported using the CDC AST MASTER website (http://www.phppo.cdc.gov/dls/master/default.aspx), and 2 (6%) of 33 hospital laboratories contacted the state laboratory for the NCCLS AST educational materials. Eighteen (55%) of 33 hospital laboratories indicated that they had purchased or planned to purchase the 2002 NCCLS AST performance standards guidelines (M100-S12).
In 2005, 57 (50%) of 115 questionnaires were returned. Of these, 35 (61%) were returned from hospital laboratories, 3 (5%) from reference laboratories, 18 (32%) from physician office-clinic laboratories, and 1 (2%) from a public health laboratory. The percentage of laboratories using current NCCLS/CLSI AST standards increased from 40% to 60% (P = 0.001) (Table (Table1).1). Of particular significance was the increase (P = 0.021) in usage from 28% in 2001 to 63% in 2005 among laboratories of small hospitals. Forty-five (79%) of 57 laboratories reported that they used NCCLS/CLSI AST performance standards, and 34 (60%) of 57 laboratories attributed their motivation for acquiring current NCCLS/CLSI AST standards to one or more of the following sources: attendance at CLI technical workshops (15 laboratories), participation in an NLTN teleconference in 2004 (10 laboratories), reviewing the CDC CD-ROM on AST (5 laboratories), reviewing the CDC AST MASTER website (5 laboratories), a consultation with CLI faculty (2 laboratories), or the CLI newsletter (1 laboratories). Laboratories of both small and moderate-to-large hospitals preferred attendance at CLI technical workshops as the motivator to acquire current NCCLS/CLSI AST performance standards.
Performance with respect to case studies in 2001 was compared to that in 2005; the case studies were identical in the two surveys. Overall percentages of correct responses for 2001 versus 2005 were as follows: for case study 1 (enterococci), 29% versus 54%; for case study 2 (MRSA), 69% versus 71%; and for case study 3 (pneumococci), 54% versus 75% (Table (Table2).2). Taking the data collectively, laboratories of small hospitals improved their performance in two of three case studies, namely, case study 1 (enterococci) (P = 0.021) and case study 3 (pneumococci) (improvement approached statistical significance [P = 0.082]). There were no statistically significant improvements for all laboratories combined; however, there was marginal improvement for case study 1 (enterococci) (P = 0.053). In 2005, 17 (49%) of 35 hospital laboratories reported that they would use referral laboratories to process clinical specimens similar to those of the case studies, whereas in 2002, only 6 (18%) of 33 hospital laboratories had reported that they would use referral laboratories for their AST.
Forty-one (72%) of 57 laboratories reported the configuration of their AST panels for investigations of invasive S. pneumonia. Thirty (73%) of 41 laboratories, including 10 (91%) of 11 laboratories of moderate-to-large hospitals, 17 (71%) of 24 laboratories of small hospitals, and 3 (100%) of 3 reference laboratories, tested both penicillin (or oxacillin) and a third-generation cephalosporin in accordance with CLSI performance standards. In addition, of those providing reports of panels used, 10 (91%) of 11 laboratories of moderate-to-large hospitals, 9 (38%) of 24 laboratories of small community hospitals, and 1 (33%) of 3 reference laboratories reported the testing of at least three antimicrobial agents (including penicillin, either cefotaxime or ceftriaxone, vancomycin, and/or meropenem) as recommended by CLSI for the testing and reporting of isolates of S. pneumoniae (4); none tested meropenem. Eight (47%) of 17 laboratories of small hospital tested seven or more antimicrobial agents. The distribution of other antimicrobial agents tested and routinely reported by 32 (56%) of 57 laboratories reporting results included penicillin (81%), ceftriaxone (50%), erythromycin (50%), vancomycin (44%), levofloxacin (41%), oxacillin (31%), trimethoprim-sulfamethoxazole (31%), cefotaxime (28%), tetracycline (22%), clindamycin (19%), moxifloxacin (9%), chloramphenicol (2%), imipenem (2%), rifampin (2%), and cefuroxime (2%). The response to the 2005 survey was comparable to that to the 2001 survey in the testing of both penicillin (or oxacillin) and a third-generation cephalosporin in accordance with CLSI performance standards. As in the first survey, several agents inappropriate for S. pneumoniae were included in the test panels of two or more laboratories.
In addition to information provided to laboratories as part of this NLS project, some indicated that several other sources of information influenced their decision to improve their AST and reporting; these included local resources (e.g., hospital formulary, infectious disease clinicians, or pharmacy personnel) and national resources (College of American Pathologist requirements, NCCLS/CLSI AST recommendations, NLTN teleconference, CDC AST MASTER website, or CDC CD-ROM). Eighteen (32%) of 57 laboratories indicated that they changed AST and reporting based on one or more of the resources associated with this NLS project, such as CLI technical workshops, NLTN teleconferences, the CDC AST MASTER website, the CDC CD-ROM on AST, or CLI faculty consultation. Thirty-four (60%) of 57 laboratories indicated a change in antimicrobial agents tested, and 19 (33%) of 57 laboratories developed a cascading or selective reporting policy for AST. Twelve (21%) of 57 laboratories indicated that they modified their policy for utilization of reference laboratories for AST and reporting.
This report demonstrates that systematic educational interventions can improve the quality of antimicrobial susceptibility testing and reporting in clinical laboratories, bringing laboratory practice in line with established national standards, regardless of laboratory setting or size. The smaller community settings, however, have the most to gain from these interventions. These institutions tend to have fewer doctoral scientists as directors or even microbiologists or technologists with advanced training in microbiology and typically have more limited access to educational materials and training opportunities than larger laboratories. The motivating factors were broadly characterized as either national or local in scope, emphasizing the importance of “local champions” in implementing laboratory guidelines. Regional technical workshops, NLTN teleconferences, and CDC materials, including the CD-ROM on AST and the CDC AST MASTER website, were especially effective in improving AST testing and reporting. Hospital formularies, infectious-disease physicians, and pharmacists also influenced AST and reporting in local hospital laboratories.
The following motivation resources had limited effectiveness: (i) CLI newsletter and website, (ii) on-site consultation, and (iii) state guidelines for AST. Personnel who inspect clinical laboratories are typically generalists who have limited background in clinical microbiology. Thus, the technical assistance that they can provide is minimal. The state's guidelines for AST were taken from CLSI AST documents and were viewed as redundant by laboratories (14).
Between 2001 and 2005, laboratory managers in smaller laboratories decided to discontinue on-site AST and increased the utilization of referral laboratories for AST. Some of the decisions to discontinue AST may have been due to efforts for cost containment, increased laboratory workload, or loss of trained personnel rather than due to educational motivators. However, the results indicate that laboratory managers went through a thoughtful decision process to determine the best option for AST.
The case studies are an effective strategy to evaluate technical competence, although it may be prudent to design additional cases to assess competence for different levels of laboratories, i.e., for laboratories of small hospitals that limit the scope of on-site AST versus the use of referral laboratories that perform testing that is more difficult. The expense of buying the updated CLSI documents every year is prohibitive for laboratories with limited resources. The CSLI AST recommendations are also very complex and detailed, requiring rigorous review by laboratory staff and ongoing educational interventions to maintain their technical competence (8).
Variations in the choice of drugs used in antimicrobial testing panels for S. pneumoniae may be explained by the fact that CLSI performance standards state that “selection of the most appropriate antimicrobial agents to test and report is a decision best made by each clinical laboratory in consultation with its infectious disease practitioners, pharmacy, the pharmacy and therapeutics, and the infection control committees of the medical staff” as well as the reporting of only first line drugs. Laboratories reported that local policy and medical resources such as the hospital formulary, infectious disease physicians, and pharmacists did influence their selection of antimicrobials to be tested as well as CLSI performance standards. Also, laboratories use predetermined antimicrobial panels marketed by manufacturers for disk diffusion and commercial MIC systems, and this influences the number of antimicrobial agents tested. Laboratories periodically change their panels when manufacturers inform them that new antimicrobials are available (19). Each laboratory should decide which agents in the CLSI tables to report routinely (group A) and which must be reported only selectively (from group B). Results for group B agents not reported routinely should be available on request, or they may be reported for selected specimens (4).
Managers and supervisors may not adopt accepted laboratory practice guidelines for a variety of reasons. These may include the perception that the changes are too expensive to implement, are unnecessary, or will not improve patient outcome. Although the factors that affect the adoption of guidelines have been well studied among clinicians (3), much more work is needed to understand the important decision-making factors among clinical laboratories.
The results of this study suggest that state health departments and other public health agencies must promote appropriate AST testing using a variety of different media to ensure continuous quality improvement. Clinical laboratory improvement is a significant “core function” (6) of state public health laboratories. As such, state public health laboratories have a major role and responsibility in establishing and communicating AST policies to laboratories. State public health laboratories must coordinate and promote quality assurance programs that help to ensure that all clinical laboratories are using appropriate methods for testing organisms of public health significance, that compare susceptibility testing results from clinical laboratories to those provided by reference laboratories, and that identify those antimicrobials that clinical laboratories are using for testing isolates of microorganisms of public health importance (8). In summary, this report has shown that national and local motivation strategies were effective in improving AST practices among microbiology laboratories and suggests that these strategies should be more widely employed to bolster those improvements.
We are grateful to the staff of the Northwest Center for Public Health Practice for providing support and assistance in maintaining the CLI website and newsletter, distribution of questionnaire surveys, and data entry. In addition, we acknowledge the assistance that Harvey B. Lipman, CDC, provided in the review of draft questionnaire surveys and Ian Painter, Foundation for Health Care Quality, for performing statistical analysis of data.
This work was supported by a Centers for Disease Control and Prevention cooperative agreement.
The authors do not have any commercial affiliations, consultancies, stock or equity interests, or patent-licensing arrangements that could be considered to pose a conflict of interest regarding the submitted article.
Published ahead of print on 23 May 2007.