|Home | About | Journals | Submit | Contact Us | Français|
Contributors: SF and ADO developed the study protocol and interventions. SF conducted the study. KH assisted in recruitment of practices and was responsible for the communication with and follow up of the participating practices. ST helped with development and implementation of the software in the practices. JH randomised the practices and was responsible for the statistical analyses. All contributors discussed core ideas and obstacles during the study. SF drafted the manuscript with support from the other authors. SF and ADO will act as guarantors for the paper.
To assess the effectiveness of tailored interventions to implement guidelines for urinary tract infections in women and sore throat.
Unblinded, cluster randomised pretest-post-test trial.
142 general practices in Norway.
72 practices received interventions to implement guidelines for urinary tract infection and 70 practices received interventions to implement guidelines for sore throat, serving as controls for each other. 59 practices in the urinary tract infection group and 61 practices in the sore throat group completed the study. Outcomes were measured in 16939 consultations for sore throat and 9887 consultations for urinary tract infection.
Interventions were developed to overcome identified barriers to implementing the guidelines. The main components of the tailored interventions were patient educational material, computer based decision support and reminders, an increase in the fee for telephone consultations, and interactive courses for general practitioners and practice assistants.
Changes in rates of use of antibiotics, laboratory tests, and telephone consultations.
Patients in the sore throat group were 3% less likely to receive antibiotics after the intervention. Women with symptoms of urinary tract infection in the intervention group were 5.1% less likely to have a laboratory test ordered. No significant differences were found between the groups for the other outcomes. Large variation was found across the included practices in the rates of antibiotic prescription, use of laboratory tests and telephone consultations, and in the extent of change for all three outcome measures.
Passively delivered, complex interventions targeted at identified barriers to change had little effect in changing practice.
Interventions to change professional practice have small to moderate effects at best
Multifaceted interventions targeted at identified barriers to change are more likely to be effective for implementing guidelines than a simple intervention selected by chance
Large variation exists in the extent of change before and after the delivery of tailored interventions to support implementing guidelines
Rigorous methods are needed to evaluate interventions to change practice
With passively delivered, complex interventions targeted at identified barriers there was only a 3% decrease in antibiotic prescribing for sore throat and a 5% decrease in test use for urinary tract infection in women
Many theories but limited evidence exist on the effectiveness of interventions to change professional practice.1,2 Although more complex interventions tend to be most effective, their effectiveness varies, they require more resources, and it is difficult to know which interventions to use. Identifying barriers to change and tailoring interventions to address these is a logical approach to selecting appropriate interventions.3 The effectiveness of tailored interventions remains uncertain.4–10 We aimed to assess the effectiveness of tailored interventions to support the implementation of guidelines for the management of urinary tract infections in women and sore throat.
We developed evidence based guidelines for urinary tract infections in women and sore throat.11,12 The main recommendation for sore throat is that most patients do not need antibiotics. Clinical examination and laboratory tests are therefore generally not necessary, and patients can be given advice by telephone. The main recommendation for urinary tract infections is that non-pregnant women aged 16-55 years with typical symptoms can be treated with antibiotics without any testing. Women who have had a previous urinary tract infection can be offered treatment by telephone.
We tested the main hypotheses with a cluster randomised pretest-post-test controlled trial. General practices were randomised to receive tailored interventions to support implementing guidelines for either urinary tract infections in women or sore throat (fig (fig1).1). Practices receiving one set of guidelines served as controls for practices receiving the other. Data were collected 18 weeks before and after the intervention.
We randomly selected 292 of the 323 general practices in Norway with the WinMed electronic medical record system, and we sent a letter inviting them to participate. Overall, 142 practices were randomised by computer (fig (fig11).13
We included consultations for sore throat if they concerned patients over 3 years old with diagnosis codes relevant for sore throat from the international classification for primary care.14 We included consultations for urinary tract infection analysis if they concerned non-pregnant women aged 16 to 55 years with diagnosis codes relevant for urinary tract infections from the international classification for primary care.
We tailored the interventions to overcome identified barriers. Both guidelines were published in the June 2000 issue of the Journal of the Norwegian Medical Association, which all Norwegian general practitioners receive.11,12 Box BoxB1B1 shows the main interventions for urinary tract infections and sore throat.
The interventions were initiated in May 2000 and continued until January 2001. Because of the nature of the interventions, participating practices knew the group to which they were assigned. One member of each practice signed an informed consent form. The study protocol was approved by the Data Inspectorate, the Norwegian Board of Health, and the regional committee for medical research ethics.
We extracted outcome data with two separate methods—Mediata and QTools, software that we developed in collaboration with Mediata.13 The software was sent to the practices on disk and installed by their staff. The staff ran the data extraction routines that transferred relevant data to disks, which were then posted back to us.
The Mediata software also included an interactive decision support application and a tool to collect additional data from pop-up screens that were triggered when a diagnosis code for sore throat or urinary tract infection was entered into a patient's record.
The Qtools software extracted relevant data, which is routinely collected, from the electronic medical record. We used these data for the main analysis because we only obtained data from Mediata for a subset of the consultations. We compared data from the baseline period, 1 January 2000 to 15 May 2000, with data from the follow up period, 15 September 2000 to 31 January 2001.
The primary outcomes for both conditions were changes in the rates of use of antibiotics, laboratory tests, and telephone consultations. We expected that implementation of the guidelines would reduce the proportion of patients who were prescribed antibiotics for sore throat but not change the proportion prescribed antibiotics for urinary tract infections, reduce the use of laboratory tests for both, and increase the proportion of telephone consultations for both.
We identified Anatomical Therapeutic Chemical Classification System codes15 for prescriptions and information from the prescription name field in WinMed. We classified the drugs as relevant or not relevant for urinary tract infections and sore throat. A standardised coding system for laboratory tests is not used in Norway. We classified all tests extracted from the field for laboratory tests in WinMed as relevant or not relevant for urinary tract infections and sore throat. We categorised a consultation as a telephone consultation if it was coded as such in WinMed or if a fee code for a telephone consultation was registered.
We calculated sample size with a method that takes into account the intracluster correlation coefficient, the number of events, the expected effect, and the power of the study.16 We assumed an intracluster correlation coefficient of ρ=0.2, a minimum of 25 patients for each practice, and a worst case control group rate of 50%. Under these assumptions we anticipated a power of 87% to detect a difference of 15% in rates between the two groups with α=0.05 with 60 practices for each intervention group. We anticipated a loss to follow up and therefore planned to randomise 70 practices in each group.
We used cluster specific methods because practices rather than patients were randomised, and we expected that variance in how patients were managed would be partly explained by the practice.17 Because some patients had more than one consultation, we added a third level to the analysis to account for the likelihood that variance in what was done at each consultation would be partly explained by the patient. For each of the outcome measures we tested the null hypothesis that the likelihood of the outcome was the same in both study groups after accounting for baseline rates, multiple patients for each practice, and repeated consultations for each patient.
We used hierarchical logistic regression to test for an effect of each intervention on the outcome measures (box (boxB2B2).17–19 This technique allowed us to test for a positive interaction between time and intervention, which would indicate an effect of the intervention on outcomes. All hierarchical models were estimated with HLM5.20 All other analyses were done with Stata 7.
Thirteen practices in the urinary tract infection arm and nine practices in the sore throat arm dropped out after randomisation (fig (fig1).1). All the practices installed the Mediata and Qtools software. We did not get data from the QTools program for two urinary tract infection practices and five sore throat practices. This left 57 of 72 practices in the urinary tract infection arm and 56 of 70 practices in the sore throat arm for the main analyses.
The sore throat practices had more practitioners and registered a greater number of consultations than did the urinary tract infection practices (tables (tables11–3). The arms were similar for patient characteristics and baseline measurements (tables (tables22–4).
Outcome data were collected for 16939 consultations for sore throat and 9887 consultations for urinary tract infection (tables (tables22 and and3).3). The intracluster correlation coefficients for the primary outcomes varied from 0.05 to 0.21 (table (table4),4), indicating that cluster specific analytical methods were appropriate.
After adjusting for baseline rates, patients with sore throat in the intervention group were less likely to receive antibiotics than patients with sore throat in the control group (table (table4).4). The absolute reduction in the proportion of consultations where antibiotics were prescribed for sore throat was 3.0% greater in the intervention group. For patients with urinary tract infection there was little change in the proportion of consultations where antibiotics were prescribed in both the intervention group (–0.2%) and the control group (0.2%).
After adjusting for baseline rates, women with symptoms of urinary tract infection in the intervention group were less likely to have a laboratory test ordered than were women in the control group. The absolute reduction in the proportion of consultations for urinary tract infection where a laboratory test was ordered for urinary tract infections was 5.1% greater in the intervention group. No significant differences were found between the groups for use of laboratory tests for sore throat (table (table44).
After adjusting for baseline rates, the absolute increase in the proportion of telephone consultations for sore throat was 1.2% greater in the control group than it was in the intervention group (table (table4).4). The proportion of telephone consultations for urinary tract infections was decreased in both the intervention group and the control group. The reduction was 0.9% greater in the control group. These differences were not significant.
Large variations occurred across the included practices in the use of antibiotics, laboratory tests, and telephone consultations and in the extent of change for all three outcome measures (fig (fig2).2).
Interventions that were tailored to address identified barriers to implementing guidelines for urinary tract infections in women and sore throat had little impact on changes in the use of antibiotics, laboratory tests, and telephone consultations.
Although only practices using WinMed—one of three electronic medical record systems used in Norway—were included in our study, there is no reason why these practices should differ from other practices.
We might have underestimated the rates for antibiotic prescriptions, ordering of laboratory tests, and telephone consultations because the practitioners may not have registered this information correctly in some instances. This is unlikely to have differed between the groups and is therefore unlikely to have affected the results.
It is possible that we underestimated the reduction in the prescription of antibiotics for sore throat. Some of the general practitioners in the study reported that they now inform patients that antibiotics normally are not necessary but prescribe them for throat infections with Streptococcus for use “if needed.” We do not know how often patients were told this or how often prescriptions that were given with this message were not used.
There are several possible explanations for the small effect that we found. We might have failed to identify important barriers to change, although neither the participants nor we identified additional barriers during the course of the study or in discussing the results. We did not tailor the interventions to the needs of individual practices or of individual general practitioners. Tailoring the intervention at this level might have had a greater effect, although we do not have any data to support this hypothesis.
Two more likely explanations are the passive character of the interventions and the lack of time. We did not have the resources to actively support the practices with outreach visits. Many practices did not have routines to support internal discussion and implementation of the guidelines. We identified some barriers—for example, problems with accessibility by telephone—that we were not able to adequately address with our interventions.
It is possible that our follow up period was not long enough to detect changes in practice. Women are used to taking a urine sample to their doctor when they have symptoms of a urinary tract infection, and patients may have expected to have laboratory tests taken and receive antibiotics for sore throats. In addition, the demands of daily practice prevented many participants from devoting much time to the project.
Few trials have been done on implementing guidelines for urinary tract infections or sore throat. A non-randomised controlled before and after study of the implementation of a telephone based guideline for managing presumed cystitis in women aged 18 to 55 years found significant decreases in the proportions of patients who received urinalysis, had a urine culture, or had an initial visit to their doctor.21 The design of this study was not robust, but the results are promising and the interventions used were intensive. A randomised controlled trial with groups of general practitioners using an educational intervention to influence prescribing for urinary tract infections found that prescription of first choice drugs increased from 52% to 70% in the intervention group and remained constant in the control group, but the intervention had no effect on the duration of treatment.22 We have not identified trials of the implementation of a guideline for sore throat similar to ours. Studies of scoring rules for sore throat have failed to show that they lower the rates for antibiotic prescription.23 It has been shown that it is possible to improve doctors' judgments of the probability of streptococcal pharyngitis, but this knowledge did not affect decisions to prescribe antibiotics.24
Figure Figure22 shows the importance of using adequately sized cluster randomised controlled trials to evaluate interventions to support the implementation of guidelines. Large variation exists in practice and in the extent of change among practices. Uncontrolled or inadequately controlled before and after evaluations in selected practices are likely to have spurious results that are, at best, difficult to interpret.
Despite the small effect that we found, it still seems logical to select strategies to support the implementation of guidelines that are tailored to address identified barriers. However, it is difficult to change practice, and large changes over short periods are not typical. Therefore it is important to use rigorous methods to measure the effects of the interventions that are used and to ensure that they are cost effective before widely employing them. It is also important to invest sufficient resources to ensure that active interventions can be used to engage clinicians and to ensure that they find the time that is needed to change their routines.
We thank Arild Bjørndal, Cheryl Carling, Jan Arve Dyrnes, Tor Arne Bertheussen, and Atle Fretheim for their help and support, the advisory committee for the project and the reference group for the development of the guidelines for support and advice, Mediata, which helped develop the software for data collection and decision support, and the participating practices.
Funding: Quality Assurance Fund of the Norwegian Medical Association and the National Institute of Public Health.
Competing interests: None declared.