PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of infectdisrepsLink to Publisher's site
 
Infect Dis Rep. 2017 March 30; 9(1): 6821.
Published online 2017 March 30. doi:  10.4081/idr.2017.6821
PMCID: PMC5391534

A Systematic Review of Quality Indicators for Appropriate Antibiotic Use in Hospitalized Adult Patients

Abstract

Many quality indicators for appropriate antibiotic use have been developed. We aimed to make a systematic inventory, including the development methodology and validation procedures, of currently available quality indicators (QIs) for appropriate antibiotic use in hospitalized adult patients. We performed a literature search in the Pubmed interface. From the included articles we abstracted i) the indicators developed ii) the type of infection the QIs applied to iii) study design used for the development of the QIs iv) relation of the QIs to outcome measures v) whether the QIs were validated and vi) the characteristics of the validation cohort. Fourteen studies were included, in which 200 QIs were developed. The most frequently mentioned indicators concerned empirical antibiotic therapy according to the guideline (71% of studies), followed by switch from IV to oral therapy (64% of studies), followed by drawing at least two sets of blood cultures and change to pathogen-directed therapy based on culture results (57% of studies). Most QIs were specifically developed for lower respiratory tract infection, urinary tract infection or sepsis. A RAND-modified Delphi procedure was used in the majority of studies (57%). Six studies took outcome measures into consideration during the procedure. Five out of fourteen studies (36%) tested the clinimetric properties of the QIs and 65% of the tested QIs were considered valid. Many studies report the development of quality indicators for appropriate antibiotic use in hospitalized adult patients. However, only a small number of studies validated the developed QIs. Future validation of QIs is needed if we want to implement them in daily practice.

Key words: Quality indicator, Quality improvement, Appropriate antibiotic use, Antibiotic Stewardship

Competing interest statement

Conflict of interest: the authors declare no potential conflict of interest.

Background

Today antibiotics are indispensable in practically all health care systems.1 However, the extensive use of antibiotics is also the main driving force in the emergence of resistant microorganisms.2 Worldwide, antibiotic consumption and antibiotic resistance are still on the rise, which, together with the decline in the discovery of new antibiotics, creates one of the greatest current threats to human health.2-5

To curb the rise of antibiotic resistance of medically important bacteria, better use of current agents is warranted and a decrease of inappropriate antibiotic use is imperative.3 Antibiotic stewardship programs are developed to optimize the appropriateness of antibiotic use, in order to maximize the chance of clinical cure or prevention of infection.6 At the same time, they aim to limit the unintended consequences of antibiotic use, such as the emergence of resistance, adverse drug events, and costs.6 Antibiotic stewardship programs (APSs) have shown to be effective and financially self-supporting.7-9 Multidisciplinary local stewardship teams are now established across the world, with the task to design programs in their own hospitals.

A requirement for an effective stewardship program is the ability to measure the appropriateness of antibiotic use in individual patients. Quality indicators (QIs) are measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of antibiotic care provided.10 A well-known classification to categorize QIs is: structure-, process- and outcome indicators.11

For an optimal and reliable use of the developed QIs, their clinimetric properties have first to be tested in clinical practice. Registration of data is different in every country and varies over time, which has an effect on validity and reliability of data collection. It is mandatory to locally test the clinimetric properties of the QIs, in order to discriminate between indicators that are feasible, valid and reliable in a specific setting and those that are not. There are several criteria to consider when assessing the QIs, including measurability,12-14 applicability (the indicator should be applicable to a substantial proportion of the reviewed patient records),14,15 inter-observer reliability,12-14,16-18 room for improvement,10,12-14 and case mix stability.12-14,17

During the past decade many quality indicators for appropriate antibiotic use have been developed.19 Providing information on the development and validation processes of available QIs can support healthcare professionals to select QIs that are considered reliable in their healthcare setting. In this systematic review we aimed to make an inventory, including the development methodology and validation procedures, of currently available quality indicators for appropriate antibiotic use in hospitalized adult patients.

Materials and Methods

To create an overview of the existing QIs for appropriate antibiotic use in hospitalized adult patients, two authors (MCK and JMP) performed a systematic literature search in the Pubmed interface. The search strategy is provided in Figure 1. Antibiotics were defined as antibacterial agents. Quality indicators were defined as quality measures, metrics or criteria. Limitations of the search included humans and English language. We screened title and abstract in order to identify studies reporting QIs for antibiotic use. Articles were excluded if they did not concern antibiotic use, did not concern quality measures, did not apply to adults, or concerned the outpatient setting. Duplicate studies were removed. We reviewed potentially relevant articles in full-text format. Articles were excluded if no full text was available or if the minority (<33%) of developed QIs in an article were related to antibiotic use, i.e. QIs as part of a general quality of care set. Finally, we selected the articles describing the consensus or evidence-based development of QIs for appropriate antibiotic use in hospitalized adult patients. The Drive-AB report, a recently performed but not yet published modified Delphi study to identify QIs for antibiotic use, was added to the final list with selected articles.

Figure 1.
Search strategy in Pubmed. Limits: humans, English.

From the included articles we abstracted i) the indicators developed, ii) type of infection the QIs applied to iii) the study design used for the development of the QIs iv) relation of the QIs to outcome measures v) whether the QIs were validated and vi) the characteristics of the validation cohort. See Table 1 for an overview.19-30

Table 1.
Overview of studies reporting quality indicators for appropriate antibiotic use.

Results

The systematic literature search resulted in 606 potentially relevant articles. After screening of titles and abstracts, 58 potentially relevant articles were selected for full-text screening. After the full text review another 47 articles were excluded, based on the previously mentioned criteria. Two additional articles were selected from literature references and added to the final list, together with the Drive AB report. Finally, fourteen articles describing the development of QIs regarding appropriate antibiotic use in hospitalized adult patients were included (Figure 2). Details of these studies are given in Table 1.19-30

Figure 2.
Procedure of article selection

The fourteen included articles described 200 QIs: 17 structure and 183 process indicators. Most QIs were specifically developed for lower respiratory tract infection, urinary tract infection or sepsis. See Appendix 1 for the complete list of indicators. The most frequently mentioned indicators concerned empirical antibiotic therapy according to the guideline (71% of studies), followed by switch from IV to oral therapy (64% of studies), followed by drawing at least two sets of blood cultures and change to pathogen-directed therapy based on culture results (57% of studies) (Table 2).

Table 2.
Description of the top 10 retrieved quality indicators.

A (RAND)-modified Delphi consensus procedure was used in the majority of studies (57%). All Delphi studies were performed with a multidisciplinary team of experts, working in different hospitals. Two studies, van den Bosch et al.14,29 and Drive AB,30 performed the consensus procedure with an international team of experts. The panel size varied from 11 to 51 experts.

Six studies took outcome measures, like mortality, morbidity, length of hospitalization, or cost-effectiveness, into consideration during the procedure. For two sets of indicators the relation between adherence to the QIs, and length of hospital stay was investigated in a subsequent observational multicenter trial.14,31

Five out of fourteen studies (36%) tested the clinimetric properties of the QIs. 41 of 63 tested QIs (65%) were considered valid for use in the clinical setting. The most common reasons why QIs were not considered valid were low feasibility of data abstraction from the patient files and lack of room for improvement.

Discussion and Conclusions

In this systematic review we provided an overview, including the development methodology and validation procedures, of all reported quality indicators for appropriate antibiotic use in hospitalized adult patients. Fourteen studies described 200 QIs: 17 structure and 183 process indicators. Five studies (36%) tested the clinimetric properties of the QIs. 41 of 63 tested QIs (65%) were considered valid for use in the clinical setting.

We performed a literature search in order to include all available studies describing the development of quality indicators in adults. Furthermore we extracted from the included studies the methods used to develop the quality indicators, the relation of the QIs with outcome measures, and the validation process of the QIs. To our knowledge such a detailed inventory of QIs for inpatient antibiotic use, including the development methodology and validation procedures, has not been done before. Recently, a systematic review on QIs for diagnosis and antibiotic treatment in primary care was reported.32 The majority (72%) of the 130 QIs focused on choice of antibiotics, 22% concerned the decision to prescribe antibiotics, and few (6%) concerned the diagnostic process. Most QIs were either related to respiratory tract infections or not related to any type of infection. Therefore, the QIs developed for this setting clearly differ from those developed for the inpatient setting.

Our study has several limitations. First, for pragmatic reasons the search was performed only in the Pubmed interface. There is a possibility that articles were overlooked for this reason. However we used a wide range of search terms regarding quality measurements and antibiotics and therefore we assume that the terms should identify those studies reporting the development of quality indicators. Second, we included only the articles that developed QIs exclusively related to antibiotic use. If the minority of QIs (<33%) in an article referred to antibiotic use, we did not select the article. Therefore, we might have missed QIs on antibiotic use. Finally, we excluded articles concerning QIs for pediatric care.

The question remains what the implications of these data are. To develop QIs, a systematic or non-systemic method can be used. Systematic methods rely on available scientific evidence complemented when necessary with expert opinion.33 A Delphi consensus procedure is a systematic process where QIs are developed based on scientific evidence combined with expert opinions.10,12-14,26,27 and at this moment the most rigorous way to develop QIs. In our study, the most frequently applied method indeed was a modified consensus Delphi procedure (57%). All Delphi panels consisted of a multidisciplinary team of (international) experts, usually working in different hospitals. Non-systematic methods, i.e. a point prevalence survey, were also used to develop the QIs.23,34

In a minority of studies (43%) the relation of quality indicators with an outcome measure, like mortality, length of stay or costs, was taken into account during the development process, and for only two sets of indicators the relation between adherence to the QIs, and length of hospital stay was investigated in a subsequent observational multicenter trial. Most of the developed QIs applied only to a specific patient group, for example patients with UTI, CAP or sepsis. Van den Bosch et al. is the only study so far to have developed a generic set of QIs of which the relation to outcome measures was assessed during a Delphi study and in an additional trial.14 As was stated before, a requirement for an effective stewardship program is the ability to measure the appropriateness of antibiotic use using QIs. For an optimal use of the developed QIs it is mandatory to test their clinimetric properties in daily practice, in order to discriminate between indicators that are feasible, valid and reliable in a specific setting and those that are not. In this study we showed that only 65% of the tested QIs were considered valid, implicating that one third of all developed QIs is possibly not eligible for use in clinical practice. Therefore, we recommend to locally test the applicability of these QIs in a pilot or controlled trial before implementing them. Only few study groups have tested the clinimetric properties of their QIs in daily clinical practice.

In our opinion, the set of QIs developed by the Drive AB group,30 is the most comprehensive set of QIs, as it is developed in a high-quality consensus procedure and is based on the most recent literature covering antibiotic use in the inpatient and outpatient setting. However, validation of these QIs is necessary in order to implement them in daily practice. So far, only van den Bosch et al.14,29 managed to develop a high quality, generic and valid set of QIs, and we recommend in comparable settings to apply this set of QIs in stewardship programs.

References

1. Heddini A, Nugent R. The path of least resistance. 2014. Available from: http://www.reactgroup.org/uploads/publications/react-publications/the-path-of-least-resistance.pdf.
2. Centers for Disease Control and Prevention. Report 2011 (revision): A public health action plan to combat antimicrobial resistance. Centers for Disease Control and Prevention 2011. Available from: http://www.cdc.gov/drugresistance/pdf/public-health-action-plan-combat-antimicrobial-resistance.pdf.
3. WHO. Antimicrobial resistance: global report on surveillance 2014. Available from: http://www.who.int/drugresistance/documents/surveillancereport/en
4. NethMap. SWAB. Consumption of antimicrobial agents and antimicrobial resistance among medically important bacteria in the Netherlands. 2015. Available from: http://www.swab.nl.
5. Spellberg B, Blaser M, Guidos RJ, et al. Combating antimicrobial resistance: policy recommendations to save lives. Clin Infect Dis 2011;52:397-428. [PMC free article] [PubMed]
6. Dellit TH, Owens RC, McGowan JE, et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis 2007;44:159-77. [PubMed]
7. Carling P, Fung T, Killion A, et al. Favorable impact of a multidisciplinary antibiotic management program conducted during 7 years. Infect Control Hosp Epidemiol 2003;24:699-706. [PubMed]
8. Lutters M, Harbarth S, Janssens JP, et al. Effect of a comprehensive, multidisciplinary, educational program on the use of antibiotics in a geriatric university hospital. J Am Geriatr Soc 2004;52:112-6. [PubMed]
9. Ruttimann S, Keck B, Hartmeier C, et al. Long-term antibiotic cost savings from a comprehensive intervention program in a medical department of a university-affiliated teaching hospital. Clin Infect Dis 2004;38:348-56. [PubMed]
10. Campbell SM, Braspenning J, Hutchinson A, et al. Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816-9. [PMC free article] [PubMed]
11. Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743-8. [PubMed]
12. Hermanides HS, Hulscher ME, Schouten JA, et al. Development of quality indicators for the antibiotic treatment of complicated urinary tract infections: a first step to measure and improve care. Clin Infect Dis 2008;46:703-11. [PubMed]
13. Schouten JA, Hulscher ME, Wollersheim H, et al. Quality of antibiotic use for lower respiratory tract infections at hospitals: (how) can we measure it? Clin Infect Dis 2005;41:450-60. [PubMed]
14. van den Bosch CM, Geerlings SE, Natsch S, et al. Quality indicators to measure appropriate antibiotic use in hospitalized adults. Clin Infect Dis 2015;60:281-91. [PubMed]
15. Grol R, Baker R, Moss F. Quality improvement research: understanding the science of change in health care. Qual Saf Health Care 2002;11:110-1. [PMC free article] [PubMed]
16. Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977;33:363-74. [PubMed]
17. Rubin HR, Pronovost P, Diette GB. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care 2001; 13:489-96. [PubMed]
18. Pulcini C, Defres S, Aggarwal I, et al. Design of a day 3 bundle to improve the reassessment of inpatient empirical antibiotic prescriptions. J Antimicrob Chemother 2008;61:1384-8. [PubMed]
19. Kotter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci 2012;7:21. [PMC free article] [PubMed]
20. Nathwani D, Williams F, Winter J, et al. Use of indicators to evaluate the quality of community-acquired pneumonia management. Clin Infect Dis 2002;34:318-23. [PubMed]
21. Berenholtz SM, Pronovost PJ, Ngo K, et al. Developing quality measures for sepsis care in the ICU. Jt Comm J Qual Patient Saf 2007;33:559-68. [PubMed]
22. Fry DE. Surgical site infections and the surgical care improvement project (SCIP): evolution of national quality measures. Surg Infect 2008;9:579-84. [PubMed]
23. Zarb P, Amadeo B, Muller A, et al. Identification of targets for quality improvement in antimicrobial prescribing: the web-based ESAC Point Prevalence Survey 2009. J Antimicrob Chemother 2011;66:443-9. [PubMed]
24. Zarb P, Ansari F, Muller A, et al. Drug utilization 75% (DU75%) in 17 European hospitals (2000-2005): results from the ESAC-2 Hospital Care Sub Project. Curr Clin Pharmacol 2011;6:62-70. [PubMed]
25. Coll A, Kinnear M, Kinnear A. Design of antimicrobial stewardship care bundles on the high dependency unit. Int J Clin Pharm 2012;34:845-54. [PubMed]
26. Thern J, de With K, Strauss R, et al. Selection of hospital antimicrobial prescribing quality indicators: a consensus among German antibiotic stewardship (ABS) networkers. Infection 2014;42:351-62. [PubMed]
27. van den Bosch CM, Hulscher ME, Natsch S, et al. Development of quality indicators for antimicrobial treatment in adults with sepsis. BMC Infect Dis 2014;14:345. [PMC free article] [PubMed]
28. Farida H, Rondags A, Gasem MH, et al. Development of quality indicators to evaluate antibiotic treatment of patients with community-acquired pneumonia in Indonesia. Trop Med Int Health 2015;20:501-9. [PubMed]
29. van den Bosch CM, Hulscher ME, Natsch S, et al. Applicability of generic quality indicators for appropriate antibiotic use in daily hospital practice: a cross-sectional point-prevalence multicenter study. Clin Microbiol Infect Dis 2016;22:888–e9.e9. [PubMed]
30. DRIVE-AB. WP1A quality indicators and quantity metrics of antibiotic use. 2016. Available from: http://driveab.eu/wp-content/uploads/2014/09/WP1A_Final-QMs-QIs_final.pdf.
31. Spoorenberg V, Hulscher ME, Akkermans RP, et al. Appropriate antibiotic use for patients with urinary tract infections reduces length of hospital stay. Clin Infect Dis 2014;58:164-9. [PubMed]
32. Saust LT, Monrad RN, Hansen MP, et al. Quality assessment of diagnosis and antibiotic treatment of infectious diseases in primary care: a systematic review of quality indicators. Scand J Prim Health Care 2016:1-9. [PMC free article] [PubMed]
33. Boulkedid R, Abdoul H, Loustau M, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PloS One 2011;6:20476. [PMC free article] [PubMed]
34. Ansari F, Erntell M, Goossens H, et al. The European surveillance of antimicrobial consumption (ESAC) point-prevalence survey of antibacterial use in 20 European hospitals in 2006. Clin Infect Dis 2009;49:1496-504. [PubMed]

Articles from Infectious Disease Reports are provided here courtesy of PAGEPress