PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjLink to Publisher's site
 
BMJ. 2002 August 17; 325(7360): 367.
PMCID: PMC117890

Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat

Signe Flottorp, researcher,a Andrew D Oxman, director,a Kari Håvelsrud, research associate,a Shaun Treweek, researcher,a and Jeph Herrin, research analystb

Abstract

Objective

To assess the effectiveness of tailored interventions to implement guidelines for urinary tract infections in women and sore throat.

Design

Unblinded, cluster randomised pretest-post-test trial.

Setting

142 general practices in Norway.

Participants

72 practices received interventions to implement guidelines for urinary tract infection and 70 practices received interventions to implement guidelines for sore throat, serving as controls for each other. 59 practices in the urinary tract infection group and 61 practices in the sore throat group completed the study. Outcomes were measured in 16 939 consultations for sore throat and 9887 consultations for urinary tract infection.

Interventions

Interventions were developed to overcome identified barriers to implementing the guidelines. The main components of the tailored interventions were patient educational material, computer based decision support and reminders, an increase in the fee for telephone consultations, and interactive courses for general practitioners and practice assistants.

Main outcome measures

Changes in rates of use of antibiotics, laboratory tests, and telephone consultations.

Results

Patients in the sore throat group were 3% less likely to receive antibiotics after the intervention. Women with symptoms of urinary tract infection in the intervention group were 5.1% less likely to have a laboratory test ordered. No significant differences were found between the groups for the other outcomes. Large variation was found across the included practices in the rates of antibiotic prescription, use of laboratory tests and telephone consultations, and in the extent of change for all three outcome measures.

Conclusions

Passively delivered, complex interventions targeted at identified barriers to change had little effect in changing practice.

What is already known on this topic

Interventions to change professional practice have small to moderate effects at best

Multifaceted interventions targeted at identified barriers to change are more likely to be effective for implementing guidelines than a simple intervention selected by chance

What this study adds

Large variation exists in the extent of change before and after the delivery of tailored interventions to support implementing guidelines

Rigorous methods are needed to evaluate interventions to change practice

With passively delivered, complex interventions targeted at identified barriers there was only a 3% decrease in antibiotic prescribing for sore throat and a 5% decrease in test use for urinary tract infection in women

Introduction

Many theories but limited evidence exist on the effectiveness of interventions to change professional practice.1,2 Although more complex interventions tend to be most effective, their effectiveness varies, they require more resources, and it is difficult to know which interventions to use. Identifying barriers to change and tailoring interventions to address these is a logical approach to selecting appropriate interventions.3 The effectiveness of tailored interventions remains uncertain.410 We aimed to assess the effectiveness of tailored interventions to support the implementation of guidelines for the management of urinary tract infections in women and sore throat.

We developed evidence based guidelines for urinary tract infections in women and sore throat.11,12 The main recommendation for sore throat is that most patients do not need antibiotics. Clinical examination and laboratory tests are therefore generally not necessary, and patients can be given advice by telephone. The main recommendation for urinary tract infections is that non-pregnant women aged 16-55 years with typical symptoms can be treated with antibiotics without any testing. Women who have had a previous urinary tract infection can be offered treatment by telephone.

Methods

Design

We tested the main hypotheses with a cluster randomised pretest-post-test controlled trial. General practices were randomised to receive tailored interventions to support implementing guidelines for either urinary tract infections in women or sore throat (fig (fig1).1). Practices receiving one set of guidelines served as controls for practices receiving the other. Data were collected 18 weeks before and after the intervention.

Figure 1
Flow of practices through trial

Participants

We randomly selected 292 of the 323 general practices in Norway with the WinMed electronic medical record system, and we sent a letter inviting them to participate. Overall, 142 practices were randomised by computer (fig (fig11).13

We included consultations for sore throat if they concerned patients over 3 years old with diagnosis codes relevant for sore throat from the international classification for primary care.14 We included consultations for urinary tract infection analysis if they concerned non-pregnant women aged 16 to 55 years with diagnosis codes relevant for urinary tract infections from the international classification for primary care.

Interventions

We tailored the interventions to overcome identified barriers. Both guidelines were published in the June 2000 issue of the Journal of the Norwegian Medical Association, which all Norwegian general practitioners receive.11,12 Box BoxB1B1 shows the main interventions for urinary tract infections and sore throat.

Main interventions for urinary tract infections and sore throat

The interventions were initiated in May 2000 and continued until January 2001. Because of the nature of the interventions, participating practices knew the group to which they were assigned. One member of each practice signed an informed consent form. The study protocol was approved by the Data Inspectorate, the Norwegian Board of Health, and the regional committee for medical research ethics.

Data

We extracted outcome data with two separate methods—Mediata and QTools, software that we developed in collaboration with Mediata.13 The software was sent to the practices on disk and installed by their staff. The staff ran the data extraction routines that transferred relevant data to disks, which were then posted back to us.

The Mediata software also included an interactive decision support application and a tool to collect additional data from pop-up screens that were triggered when a diagnosis code for sore throat or urinary tract infection was entered into a patient's record.

The Qtools software extracted relevant data, which is routinely collected, from the electronic medical record. We used these data for the main analysis because we only obtained data from Mediata for a subset of the consultations. We compared data from the baseline period, 1 January 2000 to 15 May 2000, with data from the follow up period, 15 September 2000 to 31 January 2001.

Outcomes

The primary outcomes for both conditions were changes in the rates of use of antibiotics, laboratory tests, and telephone consultations. We expected that implementation of the guidelines would reduce the proportion of patients who were prescribed antibiotics for sore throat but not change the proportion prescribed antibiotics for urinary tract infections, reduce the use of laboratory tests for both, and increase the proportion of telephone consultations for both.

We identified Anatomical Therapeutic Chemical Classification System codes15 for prescriptions and information from the prescription name field in WinMed. We classified the drugs as relevant or not relevant for urinary tract infections and sore throat. A standardised coding system for laboratory tests is not used in Norway. We classified all tests extracted from the field for laboratory tests in WinMed as relevant or not relevant for urinary tract infections and sore throat. We categorised a consultation as a telephone consultation if it was coded as such in WinMed or if a fee code for a telephone consultation was registered.

Sample size

We calculated sample size with a method that takes into account the intracluster correlation coefficient, the number of events, the expected effect, and the power of the study.16 We assumed an intracluster correlation coefficient of ρ=0.2, a minimum of 25 patients for each practice, and a worst case control group rate of 50%. Under these assumptions we anticipated a power of 87% to detect a difference of 15% in rates between the two groups with α=0.05 with 60 practices for each intervention group. We anticipated a loss to follow up and therefore planned to randomise 70 practices in each group.

Statistical methods

We used cluster specific methods because practices rather than patients were randomised, and we expected that variance in how patients were managed would be partly explained by the practice.17 Because some patients had more than one consultation, we added a third level to the analysis to account for the likelihood that variance in what was done at each consultation would be partly explained by the patient. For each of the outcome measures we tested the null hypothesis that the likelihood of the outcome was the same in both study groups after accounting for baseline rates, multiple patients for each practice, and repeated consultations for each patient.

We used hierarchical logistic regression to test for an effect of each intervention on the outcome measures (box (boxB2B2).1719 This technique allowed us to test for a positive interaction between time and intervention, which would indicate an effect of the intervention on outcomes. All hierarchical models were estimated with HLM5.20 All other analyses were done with Stata 7.

Three level logistical hierarchical model

Results

Thirteen practices in the urinary tract infection arm and nine practices in the sore throat arm dropped out after randomisation (fig (fig1).1). All the practices installed the Mediata and Qtools software. We did not get data from the QTools program for two urinary tract infection practices and five sore throat practices. This left 57 of 72 practices in the urinary tract infection arm and 56 of 70 practices in the sore throat arm for the main analyses.

The sore throat practices had more practitioners and registered a greater number of consultations than did the urinary tract infection practices (tables (tables113). The arms were similar for patient characteristics and baseline measurements (tables (tables224).

Table 1
 Characteristics of included practices
Table 2
 Characteristics of consultations for sore throat
Table 3
 Characteristics of consultations for urinary tract infection
Table 4
 Rate of use and percentage difference in change of use of antibiotics, laboratory tests, and telephone consultations

Outcome data were collected for 16 939 consultations for sore throat and 9887 consultations for urinary tract infection (tables (tables22 and and3).3). The intracluster correlation coefficients for the primary outcomes varied from 0.05 to 0.21 (table (table4),4), indicating that cluster specific analytical methods were appropriate.

Use of antibiotics

After adjusting for baseline rates, patients with sore throat in the intervention group were less likely to receive antibiotics than patients with sore throat in the control group (table (table4).4). The absolute reduction in the proportion of consultations where antibiotics were prescribed for sore throat was 3.0% greater in the intervention group. For patients with urinary tract infection there was little change in the proportion of consultations where antibiotics were prescribed in both the intervention group (–0.2%) and the control group (0.2%).

Use of laboratory tests

After adjusting for baseline rates, women with symptoms of urinary tract infection in the intervention group were less likely to have a laboratory test ordered than were women in the control group. The absolute reduction in the proportion of consultations for urinary tract infection where a laboratory test was ordered for urinary tract infections was 5.1% greater in the intervention group. No significant differences were found between the groups for use of laboratory tests for sore throat (table (table44).

Use of telephone consultations

After adjusting for baseline rates, the absolute increase in the proportion of telephone consultations for sore throat was 1.2% greater in the control group than it was in the intervention group (table (table4).4). The proportion of telephone consultations for urinary tract infections was decreased in both the intervention group and the control group. The reduction was 0.9% greater in the control group. These differences were not significant.

Other analyses

Large variations occurred across the included practices in the use of antibiotics, laboratory tests, and telephone consultations and in the extent of change for all three outcome measures (fig (fig2).2).

Figure 2
Rates of antibiotic use in consultations for sore throat before and after the tailored interventions from all practices with more than 10 consultations in each period

Discussion

Interventions that were tailored to address identified barriers to implementing guidelines for urinary tract infections in women and sore throat had little impact on changes in the use of antibiotics, laboratory tests, and telephone consultations.

Although only practices using WinMed—one of three electronic medical record systems used in Norway—were included in our study, there is no reason why these practices should differ from other practices.

We might have underestimated the rates for antibiotic prescriptions, ordering of laboratory tests, and telephone consultations because the practitioners may not have registered this information correctly in some instances. This is unlikely to have differed between the groups and is therefore unlikely to have affected the results.

It is possible that we underestimated the reduction in the prescription of antibiotics for sore throat. Some of the general practitioners in the study reported that they now inform patients that antibiotics normally are not necessary but prescribe them for throat infections with Streptococcus for use “if needed.” We do not know how often patients were told this or how often prescriptions that were given with this message were not used.

There are several possible explanations for the small effect that we found. We might have failed to identify important barriers to change, although neither the participants nor we identified additional barriers during the course of the study or in discussing the results. We did not tailor the interventions to the needs of individual practices or of individual general practitioners. Tailoring the intervention at this level might have had a greater effect, although we do not have any data to support this hypothesis.

Two more likely explanations are the passive character of the interventions and the lack of time. We did not have the resources to actively support the practices with outreach visits. Many practices did not have routines to support internal discussion and implementation of the guidelines. We identified some barriers—for example, problems with accessibility by telephone—that we were not able to adequately address with our interventions.

It is possible that our follow up period was not long enough to detect changes in practice. Women are used to taking a urine sample to their doctor when they have symptoms of a urinary tract infection, and patients may have expected to have laboratory tests taken and receive antibiotics for sore throats. In addition, the demands of daily practice prevented many participants from devoting much time to the project.

Few trials have been done on implementing guidelines for urinary tract infections or sore throat. A non-randomised controlled before and after study of the implementation of a telephone based guideline for managing presumed cystitis in women aged 18 to 55 years found significant decreases in the proportions of patients who received urinalysis, had a urine culture, or had an initial visit to their doctor.21 The design of this study was not robust, but the results are promising and the interventions used were intensive. A randomised controlled trial with groups of general practitioners using an educational intervention to influence prescribing for urinary tract infections found that prescription of first choice drugs increased from 52% to 70% in the intervention group and remained constant in the control group, but the intervention had no effect on the duration of treatment.22 We have not identified trials of the implementation of a guideline for sore throat similar to ours. Studies of scoring rules for sore throat have failed to show that they lower the rates for antibiotic prescription.23 It has been shown that it is possible to improve doctors' judgments of the probability of streptococcal pharyngitis, but this knowledge did not affect decisions to prescribe antibiotics.24

Figure Figure22 shows the importance of using adequately sized cluster randomised controlled trials to evaluate interventions to support the implementation of guidelines. Large variation exists in practice and in the extent of change among practices. Uncontrolled or inadequately controlled before and after evaluations in selected practices are likely to have spurious results that are, at best, difficult to interpret.

Conclusions

Despite the small effect that we found, it still seems logical to select strategies to support the implementation of guidelines that are tailored to address identified barriers. However, it is difficult to change practice, and large changes over short periods are not typical. Therefore it is important to use rigorous methods to measure the effects of the interventions that are used and to ensure that they are cost effective before widely employing them. It is also important to invest sufficient resources to ensure that active interventions can be used to engage clinicians and to ensure that they find the time that is needed to change their routines.

Acknowledgments

We thank Arild Bjørndal, Cheryl Carling, Jan Arve Dyrnes, Tor Arne Bertheussen, and Atle Fretheim for their help and support, the advisory committee for the project and the reference group for the development of the guidelines for support and advice, Mediata, which helped develop the software for data collection and decision support, and the participating practices.

Footnotes

Funding: Quality Assurance Fund of the Norwegian Medical Association and the National Institute of Public Health.

Competing interests: None declared.

References

1. Grol R. Beliefs and evidence in changing clinical practice. BMJ. 1997;315:418–421. [PMC free article] [PubMed]
2. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39:II2–45. [PubMed]
3. Oxman AD, Flottorp S. An overview of strategies to promote implementation of evidence-based health care. In: Silagy C, Haines A, editors. Evidence-based practice in primary care. London: BMJ Publishing Group; 2001. pp. 101–119.
4. Cheater F, Baker R, Hearnshaw H, Robertson N, Hicks N, Oxman A, et al. Cochrane Library. Issue 4. Oxford: Update Software; 2001. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes (protocol for a Cochrane Review)
5. Cranney M, Barton S, Walley T. Addressing barriers to change: an RCT of practice-based education to improve the management of hypertension in the elderly. Br J Gen Pract. 1999;49:522–526. [PMC free article] [PubMed]
6. Hux JE, Melady MP, DeBoer D. Confidential prescriber feedback and education to improve antibiotic use in primary care: a controlled trial. CMAJ. 1999;161:388–392. [PMC free article] [PubMed]
7. Soumerai SB, McLaughlin TJ, Gurwitz JH, Guadagnoli E, Hauptman PJ, Borbas C, et al. Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial. JAMA. 1998;279:1358–1363. [PubMed]
8. Santoso B, Suryawati S, Prawaitasari JE. Small group intervention vs formal seminar for improving appropriate drug use. Soc Sci Med. 1996;42:1163–1168. [PubMed]
9. Van der Weijden T, Grol RP, Knottnerus JA. Feasibility of a national cholesterol guideline in daily practice. A randomized controlled trial in 20 general practices. Int J Qual Health Care. 1999;11:131–137. [PubMed]
10. Baker R, Reddish S, Robertson N, Hearnshaw H, Jones B. Randomised controlled trial of tailored strategies to implement guidelines for the management of patients with depression in general practice. Br J Gen Pract. 2001;51:737–741. [PMC free article] [PubMed]
11. Flottorp S, Oxman AD, Cooper JG, Hjortdahl P, Sandberg S, Vorland LH. Retningslinjer for diagnostikk og behandling av sår hals. Tidsskr Nor Laegeforen. 2000;120:1754–1760. [PubMed]
12. Flottorp S, Oxman AD, Cooper JG, Hjortdahl P, Sandberg S, Vorland LH. Retningslinjer for diagnostikk og behandling av akutte vannlatingsplager hos kvinner. Tidsskr Nor Laegeforen. 2000;120:1748–1753. [PubMed]
13. Treweek S, Flottorp S. Using electronic medical records to evaluate healthcare interventions. Health Inf J. 2001;7:96–102.
14. Lamberts H, Wood M, Hofmans-Okkes I, editors. The international classification of primary care in the European Community, with a multi-language layer. Oxford: Oxford Medical; 1993.
15. WHO Collaborating Centre for Drug Statistics Methodology. ATC index with DDDs. Oslo: WHO Collaborating Centre for Drug Statistics Methodology; 1999.
16. Donner A, Klar N. Statistical considerations in the design and analysis of community intervention trials. J Clin Epidemiol. 1996;49:435–439. [PubMed]
17. Murray DM. Design and analysis of group-randomized trials. New York: Oxford University Press; 1998.
18. Bryk AS, Raudenbush SW. Hierarchical linear models. Newbury Park: Sage; 1992.
19. Omar RZ, Thompson SG. Analysis of a cluster randomized trial with binary outcome data using a multi-level model. Stat Med. 2000;19:2675–2688. [PubMed]
20. HLM5 2000. Hierarchical linear and nonlinear modelling. Lincolnwood IL: Scientific Software International; 2000.
21. Saint S, Scholes D, Fihn SD, Farrell RG, Stamm WE. The effectiveness of a clinical practice guideline for the management of presumed uncomplicated urinary tract infection in women. Am J Med. 1999;106:636–641. [PubMed]
22. Lundborg CS, Wahlstrom R, Oke T, Tomson G, Diwan VK. Influencing prescribing for urinary tract infection and asthma in primary care in Sweden: a randomized controlled trial of an interactive educational intervention. J Clin Epidemiol. 1999;52:801–812. [PubMed]
23. McIsaac WJ, Goel V. Effect of an explicit decision-support tool on decisions to prescribe antibiotics for sore throat. Med Decis Making. 1998;18:220–228. [PubMed]
24. Poses RM, Cebul RD, Wigton RS. You can lead a horse to water—improving physicians' knowledge of probabilities may not affect their decisions. Med Decis Making. 1995;15:65–75. [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Group