|Home | About | Journals | Submit | Contact Us | Français|
The diffusion of national evidence-based practice guidelines and their impact on patient outcomes often go unmeasured.
Our objectives were to (1) evaluate implementation and compliance with clinical practices recommended in the new Centers for Disease Control and Prevention (CDC) Hand Hygiene Guideline, (2) compare rates of health care-associated infections (HAI) before and after implementation of the Guideline recommendations, and (3) examine the patterns and correlates of changes in rates of HAI. We used pre- and post-Guideline implementation site visits and surveys in the setting of 40 US hospitals—members of the National Nosocomial Infections Surveillance System—and measured HAI rates 1 year before and after publication of the CDC Guideline and used direct observation of hand hygiene compliance and Guideline implementation scores.
All study hospitals had changed their policies and procedures and provided products in compliance with Guideline recommendations; 89.8% of 1359 staff members surveyed anonymously reported that they were familiar with the Guideline. However, in 44.2% of the hospitals (19/40), there was no evidence of a multidisciplinary program to improve compliance. Hand hygiene rates remained low (mean, 56.6%). Rates of central line-associated bloodstream infections were significantly lower in hospitals with higher rates of hand hygiene (P < .001). No impact of Guideline implementation or hand hygiene compliance on other HAI rates was identified. Other factors occurring over time could affect rates of HAI. Observed hand hygiene compliance rates were likely to overestimate rates in actual practice. The study may have been of too short duration to detect the impact of a practice guideline.
Wide dissemination of this Guideline was not sufficient to change practice. Only some hospitals had initiated multidisciplinary programs; practice change is unlikely without such multidisciplinary efforts and explicit administrative support.
Clinical practice guidelines are produced by a number of federal and professional organizations for the purpose of improving the equity, efficiency, and quality of patient care and minimizing inappropriate variations in practice.1-3 Despite the burgeoning of these guidelines, few studies have assessed their impact on practice.3-5 The Centers for Disease Control and Prevention (CDC) has taken the lead in developing evidence-based guidelines for the prevention and control of health care-associated infections (HAI); one of their earliest guidelines addressed hand hygiene in hospitals. The first hand hygiene guideline was published by the CDC in 1981.6 By the 1990s, the guideline had become outdated, but the CDC, citing fiscal constraints, ceased producing guidelines in the mid-1980s. Therefore, The Association for Professionals in Infection Control and Applied Epidemiology (APIC), which has a membership of approximately 10,000 professionals in infection prevention and control, took on the task of guideline development. Their first guideline, published in 1988 and revised in 1995, was on hand hygiene.7,8 In 1992, the CDC reactivated its guideline development process with the formation of The Healthcare Infection Control Practices Advisory Committee (HICPAC), and, because of new research in the field, elected to produce a new guideline for hand hygiene in 2002. The recommendations in this Guideline required major departures from traditional clinical practice.
The 3 iterations of the Guideline are compared in Table 1, and major changes in the current Guideline are noted. These major changes include use of a waterless alcohol-based antiseptic rather than soap and water for hand hygiene except when hands are visibly soiled (ie, handwashing replaced by alcohol handrub in most patient care encounters, prohibition against artificial fingernails, and an institutional mandate to provide staff education and develop a multidisciplinary program to monitor compliance). Because these recommendations represented significant changes in staff hand hygiene practices, this Guideline provided a propitious opportunity to assess the diffusion, implementation, and impact of a single practice guideline within a few years after it was published. Hence, the purposes of this study were to (1) evaluate the implementation and compliance with clinical practices recommended in the new CDC Hand Hygiene Guideline, (2) compare rates of HAIs before and after implementation of the Guideline recommendations, and (3) examine the patterns and correlates of changes in rates of HAI.
The sample for this study was obtained from among hospitals that were members of The National Nosocomial Infections Surveillance (NNIS) System, a national reporting network of >300 acute care hospitals representing nearly every state and coordinated by the CDC. The NNIS is the largest and most comprehensive reporting system for HAI in the world9 and has recently been incorporated into the National Healthcare Safety Network (http://www.cdc.gov/ncidod/hip/NNIS/members/members.htm, accessed July 10, 2006). Because it is a voluntary system, it is not representative of all hospitals in the United States. However, NNIS hospitals are geographically representative, and their data collection methods, surveillance plans, and definitions of infections conform to rigorous national standards. Furthermore, the validity of NNIS surveillance data has been reported.10
Eligibility criteria for study participation included the following: being a NNIS hospital or using NNIS methods and definitions for at least 3 years prior to the study, providing HAI data from 1 or more intensive care units (ICUs), and not using alcohol products for hand hygiene prior to publication of the Hand Hygiene Guideline. Our sample size was based on the following calculations. We used the 50th percentile for bloodstream infection (BSI) reported by NNIS hospitals in medical-surgical ICUs for 2001.11 To find a difference of 10% in this infection rate (eg, 0.049 to 0.044/1000 central-line days) with an α of 0.05, 25,000 device days would provide a power of .81 (http://home.clara.net/sisa/power.htm). Each year, there are approximately 206,000 central-line days among the 300 NNIS hospitals. Hence, to attain a sample size of 25,000 central-line days/year would require participation of approximately 36 hospitals.
To recruit hospitals, a letter was sent by mail and by e-mail from the CDC to NNIS hospitals describing the study. Interested individuals contacted the investigators, and their eligibility was then determined. More than 70 hospitals contacted the investigators, and each hospital that met eligibility criteria was invited to join the study. We recruited 40 hospitals to account for potential dropout or smaller than predicted sample sizes.
The study was reviewed and approved by the institutional review boards of each participating hospital. Site visits were made to each hospital, beginning 1 year following release of the Guideline. This timing allowed a “breaking in” period during which each hospital initiated their educational and other efforts to implement the recommendations of the Guideline but also avoided any potential influence of the site visit on implementation (ie, if a site visit were scheduled earlier, hospitals might have increased their Guideline implementation efforts in anticipation of the visit).
Arrangements for site visits were made through the infection control/hospital epidemiology department in each hospital. During the 2-day visit, the study project director collected information from the director of the infection control department regarding changes in hand hygiene policies and procedures before and after publication of the Guideline; obtained documentation regarding staff education, infection control policies and procedures, product usage, and multidisciplinary meetings regarding hand hygiene; and collected data regarding rates of HAI within the ICUs studied.
During the site visit, the project director also made rounds in one or more ICUs in each hospital, the same units from which infection rates were available, to record the proportion of ICU rooms and common areas in which alcohol hand hygiene products were available, to observe directly staff hand hygiene in the institution's ICUs, and to administer an anonymous survey to ICU staff (ie, physicians, nurses, and any ancillary direct patient care staff such as respiratory therapists) regarding their awareness of the Hand Hygiene Guideline. Psychometric testing of the staff survey instrument is described in Larson.12
The Implementation Assessment Survey was used to interview the director of infection control. It measured implementation and diffusion of the Hand Hygiene Guideline and consisted of 3 parts: (1) the introduction of the Guideline within the hospital (eg, where copies were kept, the extent to which it had been discussed with staff, and whether or not there were special sessions conducted to educate staff about the Guideline); (2) the presence of the recommended products on clinical units; and (3) institutional policies and procedures regarding hand hygiene, including the presence of a formalized plan to monitor compliance. Possible scores on the Implementation Assessment Survey ranged from 0 to 12 with higher scores indicating better implementation of the Guideline. In addition, information regarding infection control staff and the hospital was recorded (eg, education and years of experience of staff, hospital bed size, geographic location). Prior to final use in this study, the survey form was piloted among the infection control staff in 5 hospitals in New York City for content validity and clarity, and interrater reliability was confirmed to be >95%.
Staff awareness of the Guideline was measured by a single question in a larger survey, “I am familiar with this Guideline and its recommendation,” which was reported on a 6-point Likert scale from strongly disagree to strongly agree. We also collected data in each hospital regarding factors that could potentially confound the relationship between hand hygiene and infection rates, including changes in patient populations or acuity in the ICUs studied, changes in antibiotic use practices and policies or other infection control practices, changes in infection control staffing, and occurrence of outbreaks.
The Hand Hygiene Observation Instrument was used for direct observation of hand hygiene behavior in the same ICUs for which HAI rates were also available. On the tool, indications for hand hygiene from the CDC Guideline were listed. While directly observing a patient care provider, the observer noted when one of the indications occurred and then whether hand hygiene also occurred, either with soap and water or with an alcohol-based product. From these observations, it was possible to calculate an overall hand hygiene rate (number of hand hygiene episodes/number of indications) as well as a proportion of hand hygiene episodes that occurred with either soap and water or alcohol. The instrument, used in several previous studies,13,14 was also pilot tested by 4 research assistants, who independently conducted observations in 3 different ICUs and obtained an interrater agreement of 0.98.
We collected data regarding HAI rates in the ICUs of study hospitals for 12 months before and 12 months following publication of the Hand Hygiene Guideline. Because the Hand Hygiene Guideline had been discussed months before its publication at many national and regional infectious diseases and infection control conferences, some hospitals may have begun to make practice and/or policy changes prior to its formal publication. Therefore, we defined the pre-Guideline data collection period for HAI as ending 6 months before publication of the Guideline (approximately June 2001-June 2002). This would minimize any potential bias introduced by hospitals that were “early adopters.” HAI rates for 12 months, beginning approximately 6 months after publication of the Guideline (June 2003-June 2004) were designated as postimplementation rates. The designated pre- and postimplementation time periods varied somewhat among the hospitals, depending on when they actually adopted the Guideline.
HAI data were collected using standard NNIS protocols and definitions by the infection control practitioners in each hospital prior to their recruitment into this study. Data on 7 types of HAI were provided by study hospitals: central line-associated BSI (CL-BSI), non-CL-BSI, catheter-associated urinary tract infection (CA-UTI), non-CA-UTI, ventilator-associated pneumonia (VAP), surgical site infection (SSI), and “other,” which included, for example, skin and soft tissue infections and nonventilator associated pneumonia. Numbers of HAI and device or patient days were provided on a monthly or quarterly basis by ICU, but not all units reported all 7 types of HAI.
Frequency distributions of the Hand Hygiene Guideline implementation scores and rates of observed compliance with hand hygiene were examined first. Next, using the median as the cut-off point, a new variable was created to designate hospitals with implementation scores above the median as having “high” implementation scores and hospitals with scores below the median as having “low” implementation scores. Similarly, hand hygiene compliance rates were dichotomized as “high” or “low” if they were above or below the median.
Infection rates were calculated using the number of each infection type as the numerator and number of patient days (for SSI and “other”) or device days (for CL-BSI, CA-UTI, and VAP) as the denominator. Each type of infection was examined separately. HAI rates per 1000 patient- or device-days were calculated, and the rates during pre- and post-Guideline implementation periods for each hospital were compared by calculating Z statistics. A 2-sided P value <.05 was considered to be statistically significant. Next, the pre- and post-HAI rates were examined, stratified by high or low implementation scores and hand hygiene compliance rates. In these analyses, 4 types of infections were examined: CL-BSI, CA-UTI, VAP, and SSI; non-CL-BSI, non-CA-UTI, and “other” were not included because of the small number of these infections reported.
To describe further the patterns of change in HAI rates for each study hospital (ie, whether HAI rates were higher or lower after Guideline implementation), we subtracted post-Guideline infection rates from pre-Guideline rates for each type of infection. Logistic regression and χ2 tests were used to examine the correlates of HAI change patterns.
Among the 40 study hospitals, 75% were in the eastern region of the United States, and 60% were affiliated with an academic health center. The mean number of active beds of participating hospitals was 417, with 10% having 100 to 199 beds, 40% having 200 to 399 beds, and 50% having ≥400 beds. There were 71 ICUs in the study hospitals, including 30 medical-surgical (42%), 18 medical (25%), 13 surgical (18%), and 10 pediatrics (14%). Most infection control directors (85%) had ≥9 years of experience.
The Guideline Implementation scores ranged from 7 to 12 (median, 10.5). In hospitals in which the infection control director had <9 years of infection control experience, only 16.7% had implementation scores above the median (P = .18). Information about the Guideline was widely disseminated: in all hospitals surveyed, the Guideline had been discussed at staff meetings and infection control meetings (as confirmed by meeting minutes), included in staff educational programs (as confirmed by continuing education records), and 89.7% of 1158 staff members who were anonymously surveyed confirmed that they were familiar with the Guideline. Appropriate supplies and products were readily available on all patient care units observed, and every hospital had a written policy prohibiting artificial nails. However, in 44.2% of the hospitals (19/40), there was no evidence that the Guideline recommendation to “implement multidisciplinary program to improve compliance” was in place.
Generally, educational efforts were targeted to nursing and ancillary personnel, and physician staff members did not participate. Furthermore, the majority of hospitals had no effective way to monitor hand hygiene compliance. The observed hand hygiene compliance rates ranged from 24% to 89% (mean, 56.6%) per ICU. None of the hospital characteristics examined were associated with Guideline implementation scores or observed compliance rates (Table 2).
CL-BSI and VAP rates were significantly lower after implementation of the Guideline; the VAP rate decreased from 6.2 per 1000 device-days prior to the release of the Guideline to 4.8 per 1000 device-days (P < .001), and the CL-BSI rate decreased from 5.5 to 4.8 per 1000 device-days (P < .001), Table 3. There were no significant differences in rates of CA-UTI or SSI following Guideline implementation. When stratified by implementation status (high vs low), however, the significant reductions in CL-BSI and VAP were observed among hospitals with both high and low implementation scores. SSI rates were significantly lower after Guideline implementation only in hospitals with low implementation scores (P = .04) but not in hospitals with high scores, Table 4.
When stratified by hand hygiene compliance status, a significant reduction in VAP was observed among hospitals with both high and low compliance rates. However, rates of CL-BSI were decreased significantly among hospitals with high compliance rates (from 6.3 to 4.8 per 1000 device-days, P < .001) but not among hospitals with low compliance rates (from 4.9 to 4.7 per 1000 device-days, P = .74). Again, SSI rates were reduced significantly from 1.1 to 0.9 per 1000 patient-days in the low hand hygiene compliance hospitals (P = .02) but not in the high compliance hospitals, Table 5.
There were no changes reported by the directors of infection control in patient acuity in any hospital, but 17 of 40 (42.5%) hospitals reported some type of policy change (eg, written antibiotic policy) during the course of the study. There were no significant correlations between hospitals reporting policy changes or not and any HAI rate (all P > .29). Therefore, this variable was not included in the regression models.
Overall, the majority of hospitals had lower rates of infection control following Guideline implementation: 55.6% for CL-BSI, 55.6% for CA-UTI, 63.1% for VAP, and 64.5% for SSI, but none of the pre- to post-rates of change were associated with hospital characteristics except for CA-UTI: 76.2% of hospitals affiliated with an academic health center had reduced rates as compared with 26.7% of hospitals not affiliated with an academic health center (P = .006), Table 6. That association remained significant after controlling in the logistic regression model for geographic location, bed size, and rates of hand hygiene.
The Hand Hygiene Guideline was widely disseminated in all 40 study hospitals, the majority of staff members were familiar with it, and alcohol hand hygiene products were readily available. Implementation scores were high, with a median of 10.5 out of a total possible of 12, but these scores primarily measured “structural” and “process” factors such as the presence of appropriate products on clinical units and the conduct of staff educational programs. Fewer than half of hospitals had multidisciplinary programs to improve compliance, and, despite often widespread staff educational efforts, physicians were usually not included because the hospital requirements for such education did not include medical staff.
Despite the widespread dissemination of the Guideline, hand hygiene compliance as observed in this study averaged 56.6%, a rate similar to what has been reported in the literature for the past few decades.14-21 That is, while hospitals rapidly responded at the administrative level to disseminate the Guideline, change their policies and procedures to be consistent with its recommendations, and provide appropriate products at the unit level, there were no concomitant changes in actual practice.
Other studies have demonstrated that awareness and dissemination of a guideline is not necessarily followed by adherence.4,22 For example, Bishop and Wing23 reported in a randomized controlled trial that efforts to increase the use of practice guidelines in the management of acute low back pain using patient-specific, physician-to-physician communication were largely unsuccessful. In a survey of 276 primary care physicians, 90.5% reported having read a 2004 practice guideline to treat acute otitis media, but many did not follow the recommendations.24
Multiple barriers to adherence to clinical practice guidelines or implementation of research findings have been described.25-32 These have been classified by Cabana et al25-27 as lack of knowledge and awareness, lack of agreement or outcome expectancy, lack of motivation or self-efficacy, and external barriers (eg, patient expectations for treatment or insufficient resources). In our study, lack of awareness and external barriers were unlikely to be important because most staff members knew about the Guideline and its recommendations and adequate supplies were readily available. Hence, it would appear that staff either did not agree that hand hygiene was likely to reduce risk of HAI or were not motivated to improve their practice.
This study confirms the observation that institutional adoption of a guideline does not guarantee practice changes. The fact that there were so few effective multidisciplinary programs to improve hand hygiene compliance in our study hospitals may indicate that, although hospitals make structural and administrative changes in response to the publication of guidelines, they have not empowered staff or made the institutional commitment to mandate and monitor process changes.
Efforts to bridge the gap between the dissemination of guidelines and their actual use are clearly needed but will be challenging. Although education is not sufficient in itself to influence practice, guideline knowledge has been shown to improve medical practice.31 Yealy et al33 recently demonstrated in a randomized trial in 32 emergency departments that low-intensity interventions to improve implementation of evidence-based pneumonia guidelines were relatively ineffective. Their low-intensity intervention included voluntary development of a quality improvement plan to address the guideline recommendations and mailing the guideline and supporting literature to medical providers. A high-intensity strategy, however, was effective in increasing the performance of recommended practices. The high-intensity intervention included real-time reminders, provider audits and feedback, and continuous quality improvement activities within each emergency department. This level of intervention is consistent with the Hand Hygiene Guideline recommendation for a multidisciplinary program that was lacking in most of our study hospitals but has been shown in other studies to be effective.16,34-39
We found no specific hospital characteristics (eg, affiliation with an academic health center, geographic location, level of experience of infection control staff) that were significantly associated with high or low Guideline implementation or with rates of hand hygiene compliance. Generally, it appeared that the implementation of the Hand Hygiene Guideline in our study hospitals was designated as a task to be accomplished by the infection control department but with little administrative support or action to do anything further. The directors of infection control departments, for example, noted that physicians were not required to receive education regarding the Hand Hygiene Guideline nor did they have the authority to provide such education. In this study, strategies to implement this CDC Guideline were insufficient to have a measurable impact on practice.40
There were significant reductions in rates of CL-BSI and VAP between the pre- and postimplementation time periods, but, when stratified among hospitals with high and low levels of Guideline implementation, the reduction in these rates occurred in both groups, indicating that factors other than Guideline implementation accounted for these reductions. In fact, SSI rates were actually significantly lower only among hospitals that also had lower levels of Guideline implementation.
With regard to the association between hand hygiene compliance and infection rates, rates of VAP were significantly lower, regardless of whether hospitals had high or low hand hygiene compliance, and SSI rates were lower in the low compliance hospitals. Clearly, hand hygiene compliance as measured in this study had little impact on VAP or SSI. On the other hand, hospitals with higher rates of hand hygiene also had significantly decreased rates of CL-BSI after Guideline implementation. It is possible that CL-BSI is more closely linked to hand hygiene because of staff manipulation of the catheters.
During the study period, there were a number of concurrent national efforts to reduce rates of HAI, including the “bundling” of practices designed to reduce HAI recommended by the Institute for Healthcare Improvement (http://www.ihi.org/ihi, accessed July 10, 2006), the “search and destroy” approach to reduce transmission of multiple-resistant organisms adopted by several Scandinavian countries and The Netherlands41-43 and recommended by the Society for Healthcare Epidemiologists of America,44 and the rapidly growing movement to mandate public reporting of HAI.45,46 These national efforts are likely to have had an influence on all hospitals, and the potential effect of the single Guideline studied here may have been obscured by these widespread trends, particularly because compliance with hand hygiene remained rather low.
In this study, there was only one significant association between hospital characteristics and rates of various HAI: rates of CA-UTI were significantly lower in hospitals affiliated with academic health centers. To our knowledge, this correlation has not been previously reported. Perhaps there is more urinary catheter usage in academic health centers, and staff members then acquire more skill with their management.
A randomized clinical trial would have been the ideal design to examine the impact of the new practice Guideline on HAI, with hospitals randomly assigned to adopt the new recommendations of the Guideline or to continue with current practice. Unfortunately, this was impossible because, once a guideline is published, it is considered to be the “state-of-the-science,” and nonadhering hospitals might risk difficulties with accreditation or credibility. Furthermore, if the evidence-based guideline recommendations are efficacious, a randomized trial would mean withholding best practices from patients, raising ethical concerns. The alternative design that we selected was to examine infection rates pre- and post-Guideline implementation in a sample of US hospitals, and we acknowledge the limitations inherent in this design as well as the limitation inherent in studying only hospitals that volunteered for study participation.
There are a number of potential confounding factors and changes over time that could affect the relationship between infection rates and implementation of the Guideline, including how infections are defined, how surveillance is conducted and rates calculated, other concurrent infection prevention activities, and changes in levels of patient acuity over time. We did not collect information regarding the use of antibiotic-impregnated catheters. We attempted to minimize these problems in several ways. First, we studied hospitals in the NNIS system because they use standardized definitions and surveillance procedures and the reliability and validity of their HAI data have been verified.10 Second, no changes in patient acuity were reported over the 2-year study period within individual hospitals. Finally, we surveyed each hospital for any other practice changes that occurred between the pre- and postimplementation guideline phases, and there was no relationship between reporting a policy change and infection rates in these hospitals.
Another limitation is that our assessment of hand hygiene compliance was based on just 2 days of observation and is unlikely to be an accurate reflection of ongoing practice. If anything, however, compliance rates reported in this study might be higher than usual because staff were aware that hand hygiene was being observed. It is possible that the study may have been of too short duration to detect the impact of a practice guideline.
Importantly, it is not biologically plausible that all types of HAI examined in this study would be significantly affected by hand hygiene alone. It is more likely, for example, that SSI is associated with practices in the surgical suite and during the operative procedure itself. Rates of BSI and UTI, because they are usually associated with invasive devices, are more likely to be influenced by hand hygiene because staff members generally insert and/or manipulate the devices. In fact, an association between nurse staffing and rates of BSI has been demonstrated in other studies.47-49 Furthermore, we have found that clones of Staphylococcus epidermidis causing BSI in neonates are often indistinguishable from those from the hand flora of nurses providing their care.50 These 2 observations lend credence to the hypothesis that hand hygiene may be one mechanism to reduce transmission of HAI that are associated with direct contact (BSI, UTI). Clearly, however, it is extremely difficult to sort out from among a myriad of potential causal factors the independent contribution of a single variable such as hand hygiene.
Although the CDC Hand Hygiene Guideline was widely disseminated and hospitals responded by changing written policies and procedures as well as providing recommended products, this was insufficient to effect a change in clinician practice; compliance with hand hygiene recommendations remained low. Many hospitals had no multidisciplinary program or staff feedback mechanisms to improve compliance despite the Guideline recommendation and the fact that such programs have proven effective in other studies. It is clear that effective implementation of guidelines requires a comprehensive approach involving various levels within the organization.51 Infection control departments are unlikely to be able to implement successfully such multidisciplinary efforts without additional, explicit administrative support, and this will probably be necessary before guideline recommendations are reflected in practice.
We gratefully acknowledge the support and collaboration of Teresa C. Horan, MPH, Division of Healthcare Quality Promotion, Centers for Disease Control and Prevention.
Supported by The National Institutes of Health, National Institute of Nursing Research, Impact of Hand Hygiene Guideline on Infections and Costs, 1 RO1 NR008242.