|Home | About | Journals | Submit | Contact Us | Français|
International guidelines for the management of nonvariceal upper gastrointestinal bleeding have not been widely adopted in clinical practice. We sought to determine whether a national, multifaceted intervention could improve adherence to guidelines, especially for patients at high risk of nonvariceal upper gastrointestinal bleeding.
In this randomized trial, we stratified hospitals by region and size and allocated sites to either the control or experimental group. Health care workers in the experimental group were given published guidelines, generic algorithms, stratification scoring systems and written reminders and attended multidisciplinary guideline education groups and case-based workshops. These interventions were implemented over a 12-month period after randomization, with performance feedback and benchmarking. The primary outcome of adherence rates to key guidelines in endoscopic and pharmacologic management, determined by chart review, was adjusted according to site characteristics and possible within-site dependencies. We also report the rates of adherence to other recommendations.
Forty-three sites were randomized to the experimental (n = 21) or control (n = 22) groups. In our primary analysis, we compared patients before (experimental group: n = 402 patients; control group: n = 424 patients) and after (experimental group: n = 361 patients; control group: n = 389 patients) intervention. Patient-level analysis revealed no significant difference in adherence rates to the guidelines after the intervention (experimental group: 9.8%; control group: 4.8%; p = 0.99) after adjustment for the rate of adherence before the intervention (experimental group: 13.2%; control group: 7.1%). The adherence rates to other guidelines were similar and decreased over time, varying between 5% and 93%.
This national knowledge translation–based trial suggests poor adherence to guidelines on nonvariceal upper gastrointestinal bleeding. Adherence was not improved by an educational intervention, which highlights both the complexity and poor predictability of attempting to alter the behaviour of health care providers (Trial registration: ClinicalTrials.gov, no. MCT-88113).
The care of patients with nonvariceal upper gastrointestinal bleeding has evolved dramatically over the past 10 years, with international consensus recommendations being issued in 2003.1 National benchmarking initiatives2,3 suggest that widespread variations in practice persist,4,5 with poor adherence to published recommendations.6 In this randomized knowledge-translation trial, we aimed to assess the effectiveness of a nation-wide active strategy to improve adoption of evidence-based consensus recommendations on nonvariceal upper gastrointestinal bleeding.
We performed a cluster randomized trial between August 2008 and December 2009 involving 43 hospitals across Canada; the hospitals were stratified by region and hospital size. The experimental group (n = 21 hospitals) received an intervention to facilitate the uptake of consensus recommendations; the other 22 hospitals comprised the control group. We selected the hospital as the unit of randomization to minimize contamination and because it was best suited to the trial objectives.
Physicians within each hospital received the experimental or control intervention assigned to the institution. Centralized allocation of hospitals to control or experimental groups was done by personnel independent of study coordination by use of computer-generated random numbers. The experimental group did not know what form of intervention the control group received, and vice versa. The hospitals were also chosen geographically to minimize contamination. We tracked any possible contaminating initiative at the participating institutions during the study period that could affect the outcomes.
The research ethics board at each hospital approved the trial. The trial was registered with ClinicalTrials.gov (no. MCT-88113).
We selected hospitals for inclusion based on the following criteria: a recognized level of prior patient accrual into one of the registries of nonvariceal upper gastrointestinal bleeding that we have carried out nationally (RUGBE,7 REASON,4 DURABLE5); a minimum size of 75 beds, with weekly admission of at least 4–5 patients with nonvariceal upper gastrointestinal bleeding; the availability of a trained digestive endoscopist who could provide urgent upper endoscopy within 24 hours on weekdays and 48 hours on weekends of presentation; access to an in-house intensive care unit and surgical support; and the existence of an institutional electronic pharmacy database.
We reviewed the charts of patients who fulfilled the following criteria: aged 18 or more years; received care during the study period or the pre-intervention evaluation period; and a primary or secondary discharge diagnosis of nonvariceal upper gastrointestinal bleeding according to the admitting International Classification of Disease (ICD-10) coded diagnosis7 or, alternatively, the diagnosis following endoscopy. We excluded patients whose initial assessment of the present episode was performed at another institution and who were subsequently transferred to a participating site, unless full initial management data were available. We also excluded patients whose endoscopy noted no gastroduodenal ulcer bleeding; these patients were excluded to ensure patient homogeneity.1
The study was performed over a 12-month period (Figure 1). At the time of randomization, lead investigators at both control and experimental sites received the guidelines1 and the corresponding algorithm8 on the care of patients with nonvariceal upper gastrointestinal bleeding, adapted to Canadian practice. The experimental group also received a multifaceted intervention; its specific components were determined following a national analysis of needs and barriers. The intervention was tailored to each institution’s needs based on a questionnaire administered before implementation (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.120095/-/DC1). A process evaluation was also embedded in the intervention to estimate the utility of its individual components.9 The investigators at the experimental sites were asked to share these components with the entire health care team (i.e., all medical, nursing and pharmacist professionals who cared for patients with nonvariceal upper gastrointestinal bleeding). The experimental sites also received a 1-page report containing benchmarked profiles of adherence to the guidelines before and after intervention. The details of this intervention have been published.10
During months 1–3 after intervention, the experimental sites held multidisciplinary guideline education sessions (two 45-minute standardized case-based interactive workshops with small groups, facilitated by local experts); they also received the Rockall stratification scoring system.11 During months 4–5, a 2-hour collaborative care workshop was held, with the aim of producing an institution-specific management algorithm using a published template.8
The primary outcome was adherence to the management guidelines1 that apply to patients at high risk of nonvariceal upper gastrointestinal bleeding. Therefore, we compared the proportion of patients with bleeding ulcers who exhibited high-risk stigmata (active bleeding, visible vessel and adherent clots) treated endoscopically with injection followed by thermal therapy1,12 and who thereafter received intravenous proton pump inhibitors (PPI) for a correct indication at a correct dosage (high-dose pantoprazole [80 mg bolus] followed within 6 h by 8 mg/h for a total of 72 h [both ± 12 h]) following successful endoscopic therapy.1 The pharmacotherapy criterion for the definition of appropriate indication was that used in the nation-wide DURABLE audit of in-hospital PPI prescription.13
As part of a preplanned sensitivity analysis, we broadened the endoscopic therapy criterion to include hemostasis using either thermal coagulation or the application of clips alone, in keeping with persistent controversy and evolving data14,15 and the postendoscopy timing of intravenous PPI administration and accuracy of the dose.13
As an additional a posteriori analysis, we included patients who received a 72-hour intravenous PPI infusion without receiving a bolus within the preceding 6 hours if infusion had been initiated during the pre-endoscopic period. We also included in the sensitivity analysis alternate methods of hemostasis (clips alone, thermal coagulation alone) or timing of intravenous PPI (within 12 h).
As secondary outcomes, we assessed adherence to additional individual guidelines for management of ulcer bleeding, as listed in Table 1.
Additional outcomes included continued bleeding or rebleeding, need for surgery, mortality, the proportion of patients with rebleeding following initial successful treatment, need for surgery or radiological embolization, and duration of hospital stay.
A process evaluation was embedded in the intervention to estimate the utility of individual interventional components. We completed a process analysis to assess whether individual components of the intervention had an effect on outcomes. We asked each site’s principal investigator in the intervention arm at the end of the trial to complete a 15-item questionnaire assessing different process aspects of the completed intervention.
We collected clinical data by reviewing charts of the most recent 20 consecutive patients who received care before the date of randomization. We collected data from both hospital charts and Data were collected before randomization and following the study intervention (weeks 2–15, 16–31 and 32–48; Figure 2). We used these data to provide individual feedback to the sites receiving the intervention and for benchmarking using the aggregate data from all sites. The data were collected by trained research assistants using specially designed, dedicated Web-based electronic case-report forms4 that included standardized protocols and a common glossary of definitions for all variables.4 Ten percent of all data were independently entered twice to ensure validity.7,13
We compared the characteristics of the experimental and control groups before intervention for cluster-level and patient-level characteristics. Intention to treat was the primary approach for the statistical analysis. We also performed a per-protocol analysis, in which we excluded the intervention sites that were not compliant with the intervention. All significance tests were 2-sided. The comparison of adherence rates and other binary outcome variables was performed using the adjusted χ2 procedure discussed by Donner and Klar.16
We used generalized estimating equations to perform multivariable analyses adjusting for stratification factors, preimplementation values and other potential predictors of outcomes that were present before randomization (e.g., appropriate adherence rate to combination therapy [endoscopic hemostasis with intravenous PPI infusion]) and the availability of a nurse on call. We analyzed continuous outcomes using the adjusted 2 sample t test procedure discussed by Donner and Klar16 and mixed models.
In addition, rates were averaged over sites, and comparisons at the site level were made using unpaired t tests and analysis of covariance.
The trial was powered to assess an improvement in adherence at the patient level attributable to the combination of endoscopic hemostasis with intravenous PPI infusion reaching 58% in the experimental group and 38% in the control group, based on a 2-sided type 1 error rate of 5% and a power of 80%. The value of the intracluster correlation coefficient was taken as 0.2, based on data obtained from a previous registry with many of the same participating centres.4 Twenty patient charts were used to compile data from before the intervention, and 23 charts from after the intervention were used for each subsequent data collection interval at each site.
A total of 48 sites were recruited: 25 were allocated to the experimental group and 23 to the control group. Three sites in the experimental group and 1 in the control group withdrew before starting the trial. One site in the experimental group merged with another institution. The allocation criteria of the excluded sites were not different from those included. Thus, 21 sites were included in the experimental and 22 in the control group. The site-level characteristics are presented in Table 2. There were no between-group differences in the percentage of hospitals with more than 400 beds, but the geographic distribution of hospitals was somewhat imbalanced. University hospitals were equally represented, and there was similar availability of an on-call nurse in the experimental and control groups.
Patients registered between December 2007 and February 2009 at the 43 study sites were included. A total of 826 patients registered before intervention, with 402 patients in the experimental group and 424 in the control group. In the post-intervention period, 750 patients registered; 361 in the experimental group and 389 in the control group. Before implementation, the mean age (± standard deviation) in the experimental group was 68.6 (± 15.3) years (59.7% men); the mean age in the control group was 66.3 (± 15.7) years (64.6%). After implementation, the mean age in the experimental group was 68.0 (± 14.8) years (60.4% men) and 67.8 (± 16.4) years (56.6% men) in the control group. There were no substantive between-group differences (Table 3).
Patient-level adherence to the guidelines before and after intervention is summarized in Table 1.
The primary outcome was applicable to high-risk lesions that were treated endoscopically. Before intervention, 152 of the 177 high-risk patients in the experimental group and 155 of the 186 high-risk patients in the control group met this criterion. After intervention, 143 of the 151 high-risk patients in the experimental group (2 patients were missing adherence data) and 124 of the 144 high-risk patients in the control group met this criterion. The proportion of patients who received care according to the combined 2-statement guideline (endoscopic hemostasis with intravenous PPI infusion) was higher in the experimental (9.8%) than in the control (4.8%) group after intervention. However, the difference was not significant (p = 0.99) after adjustment for pre-intervention adherence rates (13.2% in the experimental group; 7.1% in the control group), number of beds and hospital location.
No significant differences in results were noted according to preplanned sensitivity analyses that varied the method of endoscopic hemostasis, timing and dose of intravenous PPI administration, and intermediate follow-up period of data collection.
The adherence rates before and after intervention were similar when the site was used as the unit of analysis (Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.120095/-/DC1).
There were no between-group differences for any of the clinical outcomes (Table 4).
We performed a per-protocol analysis comparing the 12 hospitals that delivered at least 4 of the 6 facets of the planned intervention, to the control hospitals (Appendix 3, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.120095/-/DC1). There was no evidence of a between-group difference for the primary outcome (p = 0.9, adjusted for baseline adherence, site bed number and location).
No single aspect of the 15 questions addressed to the principal investigators at the experimental sites characterizing the applied intervention was predictive of increased adherence to guidelines in a multivariable analysis in which individual process items were independent and an overall adherence to the main outcome was the dependent variable.
Efforts to educate health care workers about following guidelines for the care of patients with nonvariceal upper gastrointestinal bleeding did not improve adherence at the patient or institutional level. The absence of significant differences in adherence rates among experimental and control groups underscores the difficulty in modifying the behaviour of practising professionals. Although elaborate in its design, the components of the experimental intervention were simple, enhancing feasibility and favouring generalizability in a real-world setting. However, 2 recent cluster-randomized trials in Canada have shown that the implementation of such simple, albeit multifaceted, measures does not necessarily translate into improved outcomes.17,18
Certain barriers may have prevented improvement, including lack of a champion at a given site to promote guideline uptake, variable uptake of the 6-facet intervention and staggered implementation in the timing of these interventions at different sites. It is possible that simultaneous implementation of these interventions could have had a greater impact on behaviour, while some effective facets may have lost benefit over time. Endoscopic hemostasis and intravenous PPI infusion were each associated with acceptable adherence rates (> 80%), but the combination of the 2 as recommended by guidelines was poorly followed. For example, physicians may have chosen to modify the duration of PPI infusion depending on local practicalities (for example, substantially shortening the infusion as some data now suggest is occurring;4 in our trial, between 29% and 54% received an infusion for 66 h or less). We found that a more relaxed interpretation of these guidelines (as part of preplanned sensitivity analyses, including the use of clips for endoscopic hemostasis and varying levels of flexibility with regards to the timing, dose and route of administration of PPI) improved the absolute values of adherence but did not result in statistically significant improvement following the educational intervention.
The influence of local practices on adherence, even if contradictory to national or international guidelines, is suggested by certain factors in our study having a higher intracluster correlation coefficient. Evolution in practice since the publication of the guidelines may have had an effect on our results; however, interventions have changed minimally over time (e.g., the use of clips has increased, but in keeping with evidence-based technological evolution, we also included the use of clips in our analysis).6 In recent years, since publication of the initial guidelines, a few non–North American placebo-controlled studies of oral or low-dose intravenous PPI have suggested benefit for gastrointestinal bleeding.19 However, the evidence base for low-dose and oral PPIs for upper gastrointestinal bleeding is less than that for high-dose PPI versus placebo. There have been few head-to-head studies of different PPI regimens.19 Nonetheless, awareness of these recent low-dose studies may have brought into question the optimal route and dose of PPI administration, and this might partly explain the observed lack of adherence.
The relatively lower cost of oral versus intravenous PPI in Canada may have also affected adherence to guidelines, with physicians altering practice to reduce treatment costs. In addition, drug shortages led to some reductions in intravenous pantoprazole use in Canada immediately before and during the study period, causing some institutions to forego intravenous PPI use or to switch to oral or disintegrating forms. There is a lack of evidence supporting shorter durations of intravenous PPI infusion, and perhaps this specific part of the guideline is less critical to the management of nonvariceal upper gastrointestinal bleeding and may deserve less weight (e.g., infusion for 60 h instead of 72 h).
Successful guideline implementation with professionals collaborating in an environment similar to that for patients with nonvariceal upper gastrointestinal bleeding has involved the use of many different tools targeting clinical decision support,20,21 education20,21 and regular performance feedback.20 There have also been national efforts to improve the prevalent inappropriate prescribing of acid suppressants.22,23 The methods of interventions have, here too, been disparate with assessment of possible needs and barriers prior to their implementation performed only in very rare instances24–27 and have resulted in varying success in prescribing behaviour or cost savings.
When we compare the data from the present study to that from past pilot work originating from national registries,4,13 we noted significantly lower adherence rates than previously reported for the most important guidelines, presumably because of some of the reasons discussed above.
The needs and barriers analysis performed before the start of the trial identified relevant obstacles. This early analysis permitted a subsequent tailoring of the interventions to each site, even though all components of intervention were delivered to all experimental sites. Nonetheless, it is likely that the observed lack of behavioural modification was caused by poorly identified barriers. Although our study focused on 6 interventions, a recent systematic review did not find a relation between the number of interventions and treatment effect.28 The complexity of the interventions, even though it was designed to be broadly applicable, may have limited its adequate functional uptake. Others have found simple and cheap reminders to have the same effect as more elaborate tailored interventions.28,29
The findings suggest poor adherence to the most critical aspects of the management of nonvariceal upper gastrointestinal bleeding following adoption of a tailored and multifaceted educational intervention. Our findings also highlight the need for measuring outcomes when attempting to alter the behaviour of health care providers.
Additional analyses may identify specific explanatory phenomena such as the barriers to implementation. Additional trials are required to identify more inventive, better adapted strategies to enhance knowledge transfer in this clinical area.
The authors thank the following physicians for participation in this trial: Dr. Alan Cockeram, Saint John Regional Hospital, Saint John; Dr. Des Leddin, Queen Elizabeth II Health Sciences Centre, Halifax; Dr. Carlo Fallone, McGill University Health Centre-Royal Victoria Hospital, Montréal; Dr. Marc Bradette, Hotel Dieu du Québec, Québec; Dr. Pierre Pare, CHAUQ-Hopital Saint Sacrement, Québec; Dr. John Marshall, Hamilton Health Sciences, Hamilton; Dr. David Morgan, St. Joseph’s Hospital, Hamilton; Dr. Flavio Habal, UHN-Toronto General Hospital, Toronto; Dr. Fred Saibil, Sunnybrook Health Sciences Centre, Toronto; Dr. William Paterson, Kingston General Hospital, Kingston; Dr. Henryk Pluta, Abbotsford Generl Hospital, Abbotsford, BC; Dr. Robert Hilsden, Calgary General Hospital, Montréal, Calgary; Dr. Ford Bursey, Health Science Centre, St. John’s; Dr. Gad Friedman, Jewish General Hospital, Montréal; Dr. Serge Mayrand, McGill University Health Centre-Montréal General Hospital; Dr. Gilles Jobin, Hopital Maisonneuve Rosemont, Montréal; Dr. Gilbert Doummar, Centre hospitalier Pierre Boucher, Longueuil; Dr. Marc Tourigny, Centre hospitalier Anna Laberge, Châteauguay; Dr. Raymond Bourdage, Centre hospitalier Hotel Dieu de Levis, Levis; Dr. Norman Marcon, St. Michael’s Hospital, Toronto; Dr. Brian Stotland, Southlake Regional Health Centre, Newmarket; Dr. Pardeep Nijhawan, York Central hospital, Richmond Hill; Dr. Naoki Chiba, Guelph General Hospital, Guelph. The authors also thank Dr. Peter Tugwell, member of the steering committee; Myriam Martel, study nurse coordinator; and Ian Hawes, trial delivery manager.
Competing interests: Alan Barkun has received travel support from AstraZeneca. He has served as a consultant to Takeda and Cook Inc., holds grants from Boston Scientific and has received payment for lectures from AstraZeneca, Takeda and Sanofi. He has received payment for the development of educational presentations from AstraZeneca. Robert Enns has served as a technology consultant for Cook Inc. and Olympus and has received payment for lectures from Cook Inc. He has received travel expenses from the Canadian Association of Gastroenterology. David Armstrong has served as an advisory board member of AstraZeneca and Takeda and as a consultant for AstraZeneca. He has received payment for lectures and manuscript preparation from AstraZeneca and Takeda. Martin Dawes has served as a consultant for Expert-24. Joseph Romagnuolo has served as a consultant for Olympus and BioMedical Insights. He holds grants from Merck, Sanofi, Pfizer, Lantheus Medical Imaging and Epigenomics. He has received payment for lectures from Cook Medical and Peer Direct. Paul Moayyedi has served as a consultant for and holds grants from AstraZeneca. He has received payment for lectures from AstraZeneca and Shire. None declared by Mamatha Bhat, Allan Donner, Janet Martin and Larry Stitt.
This article has been peer reviewed.
Funding: This study was funded by the Canadian Institutes of Health Research (CIHR; grant no. IR2–90381) and by an at-arms-length educational grant from AstraZeneca Canada as part of the CIHR/Rx&D program.
Contributors: Alan Barkun, David Armstrong, Martin Dawes, Allan Donner, Robert Enns, Janet Martin, Paul Moayyedi, Joseph Romagnuolo and Larry Stitt contributed to the study conception and design. Alan Barkun and Mamatha Bhat drafted the manuscript, which was critically revised by all authors. Allan Donner and Larry Stitt, Alan Barkun and Mamatha Bhat contributed to the analysis and interpretation of the data. All of the authors approved the final version of the manuscript submitted for publication.