PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of cmajCMAJ Information for AuthorsCMAJ Home Page
 
CMAJ. Oct 5, 2010; 182(14): 1527–1532.
PMCID: PMC2950184
A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments
Ian G. Stiell, MD MSc, Catherine M. Clement, RN, Jeremy M. Grimshaw, MBChB PhD, Robert J. Brison, MD MPH, Brian H. Rowe, MD MSc, Jacques S. Lee, MD MSc, Amit Shah, MD, Jamie Brehaut, PhD, Brian R. Holroyd, MD, Michael J. Schull, MD MSc, R. Douglas McKnight, MD, Mary A. Eisenhauer, MD, Jonathan Dreyer, MD, Eric Letovsky, MD, Tim Rutledge, MD, Iain MacPhail, MD, Scott Ross, MD, Jeffrey J. Perry, MD, Urbain Ip, MD, Howard Lesiuk, MD, Carol Bennett, MSc, and George A. Wells, PhD
From the Departments of Emergency Medicine (Stiell, Perry) and Medicine (Wells), the Division of Neurosurgery (Lesiuk), and the Clinical Epidemiology Program, Ottawa Hospital Research Institute (Stiell, Grimshaw, Wells, Clement, Perry, Brehaut, Bennett), University of Ottawa, Ottawa, Ont.; the Department of Emergency Medicine (Rowe, Holroyd, Ross), University of Alberta, Edmonton, Alta.; the Department of Emergency Medicine (Brison), Queen’s University, Kingston, Ont.; the Division of Emergency Medicine (Schull, Lee, Rutledge, Letovsky), University of Toronto, Toronto, Ont.; the Division of Emergency Medicine (Shah, Dreyer, Eisenhauer), University of Western Ontario, London, Ont.; and the Department of Emergency Medicine (McKnight, MacPhail, Ip), University of British Columbia, Vancouver, BC
Correspondence to: Dr. Ian Stiell, Ottawa Hospital Research Institute, Clinical Epidemiology Unit, F6, 1053 Carling Ave., Ottawa ON K1Y 4E9; istiell/at/ohri.ca
Background
The Canadian CT Head Rule was developed to allow physicians to be more selective when ordering computed tomography (CT) imaging for patients with minor head injury. We sought to evaluate the effectiveness of implementing this validated decision rule at multiple emergency departments.
Methods
We conducted a matched-pair cluster-randomized trial that compared the outcomes of 4531 patients with minor head injury during two 12-month periods (before and after) at hospital emergency departments in Canada, six of which were randomly allocated as intervention sites and six as control sites. At the intervention sites, active strategies, including education, changes to policy and real-time reminders on radiologic requisitions were used to implement the Canadian CT Head Rule. The main outcome measure was referral for CT scan of the head.
Results
Baseline characteristics of patients were similar when comparing control to intervention sites. At the intervention sites, the proportion of patients referred for CT imaging increased from the “before” period (62.8%) to the “after” period (76.2%) (difference +13.3%, 95% CI 9.7%–17.0%). At the control sites, the proportion of CT imaging usage also increased, from 67.5% to 74.1% (difference +6.7%, 95% CI 2.6%–10.8%). The change in mean imaging rates from the “before” period to the “after” period for intervention versus control hospitals was not significant (p = 0.16). There were no missed brain injuries or adverse outcomes.
Interpretation
Our knowledge–translation-based trial of the Canadian CT Head Rule did not reduce rates of CT imaging in Canadian emergency departments. Future studies should identify strategies to deal with barriers to implementation of this decision rule and explore more effective approaches to knowledge translation. (ClinicalTrials.gov trial register no. NCT00993252)
More than six million instances of head and neck trauma are seen annually in emergency departments in Canada and the United States.1 Most are classified as minimal or minor head injury, but in a very small proportion, deterioration occurs and neurosurgical intervention is needed for intracranial hematoma.2,3 In recent years, North American use of computed tomography (CT) for many conditions in the emergency department, including minor head injury, has increased five-fold.1,4 Our own Canadian data showed marked variation in the use of CT for similar patients.5 Over 90% of CT scans are negative for clinically important brain injury.68 Owing to its high volume of usage, such imaging adds to health care costs. There have also been increasing concerns about radiation-related risk from unnecessary CT scans.9,10 Additionally, unnecessary use of CT scanning compounds the Canadian problems of overcrowding of emergency departments and inadequate access to advanced imaging for nonemergency outpatients.
Clinical decision rules are derived from original research and may be defined as tools for clinical decision-making that incorporate three or more variables from a patient’s history, physical examination or simple tests.1113 The Canadian CT Head Rule comprises five high-risk and two medium-risk criteria and was derived by prospectively evaluating 3121 adults with minor head injury (Figure 1) (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.091974/DC1).6 The resultant decision rule was then prospectively validated in a group of 2707 patients and showed high sensitivity (100%; 95% confidence interval [CI ] 91–100) and reliability.14 The results of its validation suggested that, in patients presenting to emergency departments with minor head trauma, a rate of usage of CT imaging as low as 62.4% was possible and safe.
Figure 1
Figure 1
The Canadian CT Head Rule, as used in the study. Note: CSF = cerebrospinal fluid, CT = computed tomography, GCS = Glasgow Coma Scale.
Unfortunately, most decision rules are never used after derivation because they are not adequately tested in validation or implementation studies.1519 We recently successfully implemented a similar rule, the Canadian C-Spine Rule, at multiple Canadian sites.20 Hence, the goal of the current study was to evaluate the effectiveness and safety of an active strategy to implement the Canadian CT Head Rule at multiple emergency departments. We wanted to test both the impact of the rule on rates of CT imaging and the effectiveness of an inexpensive and easily adopted implementation strategy. In addition, we wanted to further evaluate the accuracy of the rule.
Design
We conducted a matched-pair cluster-design trial at 12 hospital emergency department sites in the Canadian provinces of Ontario, Alberta and British Columbia. These sites were stratified by hospital status (six teaching sites, six community sites) and then matched according to estimated baseline rates of ordering of CT scans of the head. Within each pair, sites were allocated randomly to either intervention or control groups. Outcomes were recorded during two consecutive 12-month periods (“before” and “after”). During the “after” period, strategies to actively implement the Canadian CT Head Rule were carried out at the intervention sites. The study took place from Nov. 1, 2002, to Oct. 31, 2005, with different sites starting the “before” and “after” periods at slightly different times.
Study population
We enrolled all alert and stable adults who presented to the study emergency departments after sustaining an acute minor head injury, which was defined as the presence of all of the following conditions: blunt trauma to the head resulting in witnessed loss of consciousness, amnesia or witnessed disorientation, an initial Glasgow Coma Scale score of 13 or greater, and occurrence of the injury within the previous 24 hours. Daily patient census logs were reviewed to identify potential study patients and patient eligibility was determined from emergency department records. Patients were excluded from the study if they were younger than 16 years of age, had penetrating skull injury, had focal neurologic deficit, had suffered a seizure before arrival at the emergency department, had a bleeding disorder or used warfarin, or returned for reassessment of the same head injury.
For each study site, approval was obtained from the research ethics board of the hospital involved, without the need for informed consent.
Study interventions
The design of the intervention was based on theoretical considerations, current evidence and consultation with knowledge transfer experts.2123 The Canadian CT Head Rule was implemented at the six intervention sites using strategies that were simple and inexpensive rather than complex and resource-intensive. Physician groups at the emergency departments discussed the goals of implementation and agreed to order CT scans according to the decision rule. Educational initiatives included distribution of articles that outlined the CT Head Rule and reported results of the validation study, pocket cards and posters, as well as a one-hour teaching session to review the supporting evidence and clinical application of the rule. Lastly, we implemented a mandatory, real-time reminder of the rule at the point of requisition for CT scans. All CT scans ordered in the emergency department required the requesting physician to check off the decision rule criteria or indicate the reason for overriding the rule before the scan would be performed by the diagnostic imaging department.
The Canadian CT Head Rule was familiar to many clinicians because of previous passive dissemination (i.e., through journal articles and conference lectures) and because seven of the emergency departments had participated in the derivation and validation studies. At the control sites, no specific intervention was introduced to alter the CT–scan-ordering behaviour of emergency department physicians, who had not been told about the study.
Outcome measures
The primary outcome was the proportion of eligible patients referred for CT scan of the head. Radiology reports and census lists were reviewed to determine if a CT scan of the head was performed.
We also measured the clinical impact of the intervention by assessing the number of clinically important cases of brain injury (defined as any acute brain-related finding on CT scan that would normally require admission to hospital and neurosurgical follow-up) that were not identified during the initial visit to the emergency department24 and the number of deaths from head injury that occurred after the emergency department visit. Logs of patient visits to the emergency department were monitored to identify return visits within 30 days by patients who had not undergone a CT scan. In addition, we reviewed the logs for neurosurgery patients at all neurosurgical centres that were traditional referral sites for the study hospitals.
Performance of the Canadian CT Head Rule was a secondary study outcome. For cases in which physicians completed the study requisition and checked off the rule criteria during the “after” period at intervention sites, we evaluated the sensitivity of the decision rule and physicians’ accuracy in interpretation of the rule. The latter was assessed by comparing each physician’s notation on the CT scan requisition to the “gold standard” interpretation made by the investigators’ steering committee.
We conducted post-study interviews with investigators at intervention hospital sites to gather their perceptions of barriers to and facilitators of use of the decision rule.
Statistical analysis
The final analysis included all patients who satisfied the inclusion and exclusion criteria. No patient was excluded during the “after” period because of noncompliance with the rule by a physician. Subgroup analyses were used to evaluate teaching hospitals and community hospitals separately.
For analysis of dichotomous data, a standard paired t test was used to assess differences in event rates in intervention site and control site pairs (5 degrees of freedom).25 For analysis of continuous data, a standard paired t test (5 degrees of freedom) was used to examine differences in mean response between intervention site and control site pairs. We calculated 95% confidence intervals (CIs) for relative reductions in event rates. For these relative reductions, the stratum-specific differences (percent change from baseline: (before-after) × 100/before) were calculated and pooled across strata using a random-effects meta-analysis.26 The weighted paired t test was applied at the cluster level using the cluster-specific means, and the 95% CIs were calculated using the weights (nj1nj2)/nj where njk is the size of the jth stratum in the kth group and nj is the size of the jth stratum. All reported p values are two-tailed. For the proportions of CT scans of the head that were ordered, the change in proportion from the “before” to “after” periods for each cluster was determined and used in the calculation of the differences in the event rates for each intervention and control site pair.
Sample size
Sample size calculation took into account the complexity of the study design (i.e., stratified, matched-pair and cluster-randomized [unit of randomization was the hospital and unit of analysis was the patient]).27 A 15% relative reduction in the event rate (12.0% absolute reduction from the expected baseline rate of 80%) was considered a minimal clinically important change. Based on 80% power and a two-sided level of significance of p = 0.05, we estimated that six matched-pair clusters were required and a total sample of 4800 patients.
Overall, 4531 eligible patients with head injury were seen at the 12 hospitals in the study. The characteristics of study patients that were similar when comparing the “before” period to the “after” period and the intervention group to the control group are listed in Tables 1 and and22.
Table 1
Table 1
Characteristics of 4531 patients with head injury
Table 2
Table 2
Clinical outcomes of 4531 patients with head injury
Rates of CT imaging increased at all six intervention sites from the “before” period to the “after” period (Table 3), as well as at four control sites. At the intervention sites, the proportion of patients who received a CT scan increased from 62.8% in the “before” period to 76.2% in the “after” period (absolute difference +13.3%, 95% CI 9.7%–17.0%). At the control sites, use of CT scans also increased, from 67.5% to 74.1% (absolute difference +6.7%, 95% CI 2.6%–10.8%). When we assessed the relative change in mean imaging rates from the “before” period to the “after” period for intervention hospitals versus control hospitals, we found no significant difference (p = 0.16).
Table 3
Table 3
Rates of computed tomography use for 4531 patients with head injury during the 12-month “before” and “after” study periods
The subgroup analysis, which was stratified by community or teaching hospital, failed to show a positive impact of the intervention on CT imaging rates (Appendix 2, available at www.cmaj.ca/cgi/content/full/cmaj.091974/DC1). Rates of CT scanning increased more at community hospitals than at teaching hospitals, regardless of whether intervention or control. For the relative change in mean imaging rates from before to after, we found a significant increase in only the community-intervention subgroup (p = 0.001).
At the intervention hospitals, 150 physicians were involved in implementing the Canadian CT Head Rule during the “after” period. They completed the rule section of the radiology requisitions for 909 (78.0%) of the 1166 eligible patients referred for CT scan (Appendix 3, available at www.cmaj.ca/cgi/content/full/cmaj.091974/DC1). Physicians accurately interpreted the rule in 82.5% of the cases. Common errors were overcalling instances of a “dangerous mechanism” and missing scores of less than 15 on the Glasgow Coma Scale two hours post-injury. Physicians ordered CT imaging despite a negative rule in 141 cases and did not order CT imaging in six cases in which the rule was positive.
The decision rule could be interpreted for 897 patients for whom the CT scan requisition was sufficiently completed. The rule correctly identified all 88 clinically important brain injuries, achieving a sensitivity of 100% (95% CI 96%–100%). There were no missed clinically important brain injuries at any of the sites during the study period.
Our trial confirms that the Canadian CT Head Rule is highly sensitive (100%) for identifying clinically important brain injuries in patients with minor head injury.6 However, it did not show that implementation of the rule led to a reduction in emergency department use of CT imaging. In fact, there was an increase over time in use of CT scans at 10 of 12 sites, most notably at community hospitals. Many factors likely account for these findings, including our use of simple, inexpensive interventions, suboptimal compliance, crowding in Canadian emergency departments, increased ease of access to CT scans in emergency departments, and an overall secular trend toward increased use of CT imaging.
This implementation study, designed and conducted according to strict methodologic standards, represents the final stage in the development of a clinical decision rule.11,13,15 Our previous studies to derive, validate and implement the Ottawa Ankle Rules, the Ottawa Knee Rule and the Canadian C-Spine Rule all resulted in large reductions in rates of imaging.20,2831 In these studies, we also achieved higher levels of physician compliance with the protocol (86%–95%).20,2830
It is unclear why this study, whose methods closely mirrored those used in the Canadian C-Spine Rule implementation study conducted contemporaneously in the same sites, failed to achieve the desired reduction in use of CT scans. Physicians responded differently to the same intervention for different clinical conditions, suggesting that the condition itself had a modifying effect on the implementation of rules in emergency departments. Our study directly replicated interventions in the same settings, which highlights the potential importance of the interaction of attributes of the targeted behaviour with the effects of interventions.32 Perhaps more intensive interventions were required to alter use of CT imaging compared with radiography for injuries of the cervical spine, knee and ankle.
Given that baseline rates of CT imaging were lower than the potential rate predicted in the validation study, the increase in rates of CT imaging at intervention sites may simply reflect correction of unwarranted variation in practice. The increase may also signal a secular trend in increasingly congested emergency departments, where ordering a CT scan and discharging the patient is perceived as being more expedient than conducting a thorough clinical evaluation. Access to CT scanning has increased substantially in recent years, and it can now be ordered quickly and easily at most medium and large hospital emergency departments. Finally, our findings may also be a symptom of increased use of emerging technologies in medicine, where CT imaging of the head is considered the standard of care.33 We observed frequent use of CT imaging for patients with minimal head injury when there is no evidence of neurologic disturbance.
We evaluated potential barriers to successful implementation before the start of the study and found that the most common concerns expressed by clinicians were inability to remember the rule, belief that trauma services would order a CT scan even if the physician did not, and expectations by patients and families that a CT scan would be ordered. In our post-study interviews, physicians identified other potential barriers. Some were not comfortable with the definition of clinically unimportant injury and believed that even trivial findings on CT imaging should be documented. Others stated that CT imaging has become the local standard of care for patients with minor head injury. A common comment was that emergency department overcrowding led to expedient over-testing in the belief that this approach would lead to earlier discharge of patients. Finally, many physicians felt that they could easily circumvent the rule without consequences from study or hospital staff, suggesting that mandatory use of decision rules at the point of requisition for imaging may not be an effective strategy.
Limitations
Our study had limitations. The baseline imaging rates were not as balanced between intervention and control sites as planned. In addition, the baseline rates were far lower than expected, making the prospect of further reductions unlikely. Perhaps these low rates at baseline were a reflection of prior passive dissemination of the rules or of participation by seven of the sites in previous studies of the Canadian CT Head Rule.6,14 Two sites had unusual results. At one intervention site, almost twice as many patients were seen and a large increase (to 94%) in CT imaging was evident. One control site tripled its rate of imaging from an exceptionally low baseline rate of 22%. Physician compliance with the requisition form was lower than anticipated, and it is difficult to ascertain if this finding reflects a weakness of the implementation strategy or other barriers to use of the decision rule.
Our findings indicate a need for further research to understand the specific barriers to implementing the Canadian CT Head Rule and to evaluate interventions to overcome them. The findings of this and the sister study of the Canadian C-Spine Rule showed that apparently identical interventions targeting the same emergency departments for different conditions had dramatically different results. This difference highlights the importance of replication using knowledge translation trials that would allow exploration of the factors involved.34
The Canadian CT Head Rule again proved to be accurate and reliable, and can be used with confidence by clinicians to determine which patients with minor head injury would benefit from use of CT imaging. Its usefulness is particularly applicable to sites where getting access to CT imaging is difficult or transfer of patients to a large centre is required.
Conclusion
Our knowledge translation trial of the previously validated and highly sensitive Canadian CT Head Rule did not reduce rates of usage of CT imaging in Canadian emergency departments. Future studies should identify strategies to deal with barriers to implementation of the rule and explore more effective approaches to knowledge translation.
Supplementary Material
[Online Appendices]
Acknowledgements
The authors thank the physicians, nurses and radiology technicians at the six intervention hospitals for their cooperation with the study, and the following research personnel at the study hospitals: Jan Buchanan (Vancouver General Hospital), Evelyn Gilkinson (London Health Sciences Centre), Kim Bradbury, (Ottawa Hospital – Civic Campus), Michelle Pachal (Sturgeon Community Hospital), Jodie Hale and Andrea Ennis (North York General Hospital), Linda Gormley (Surrey Memorial Hospital), and Virginia Willis (University of Alberta Hospital), Sharon Ramagnano and Deborah Wright (Sunnybrook and Women’s College Health Sciences Centre), Kathy Bowes (Kingston General Hospital), Percy MacKerricher (Royal Columbian Hospital), Jeannie Brewster and Jennifer Yantha (Credit Valley Hospital), Lorraine Rollins (St. Thomas Elgin General Hospital). The authors also thank their colleagues at the Ottawa Hospital Research Institute (Christine Leclair, Sheryl Domingo, My-Linh Tran and Angela Marcantonio) for their assistance with this project.
George Wells is a biostatistical consultant for CMAJ and was not involved in the editorial decision-making process for this article.
Footnotes
Funding: Funding for this study was received from the Canadian Institutes of Health Research (grant no. MOP-62795). Dr. Rowe’s research is supported by a 21st Century Canada Research Chair from the Government of Canada.
Previously published at www.cmaj.ca
Competing interests: None declared.
Contributors: Ian Stiell, Jeremy Grimshaw, Jamie Brehaut, Robert Brison, Brian Rowe, Jeffrey Perry and George Wells developed and contributed substantially to the conception and design of the study. Ian Stiell, Catherine Clement, Robert Brison, Brian Holroyd, Jacques Lee, Amit Shah, Michael Schull, Douglas McKnight, Mary Eisenhauer, Jonathan Dreyer, Eric Letovsky, Tim Rutledge, Iain MacPhail, Scott Ross, Jeffrey Perry and Urbain Ip acquired the data. Ian Stiell, Catherine Clement, Jamie Brehaut and George Wells collated and assisted with analysis of the data. All of the authors drafted the manuscript or contributed to its revision, and all of them approved the final version submitted for publication.
This article has been peer reviewed.
1. Pitts SR, Niska RW, Xu J, et al. National Hospital Ambulatory Medical Care Survey: 2006 emergency department summary. Natl Health Stat Report. 2008 Aug 6;:1–38. [PubMed]
2. Seelig JM, Becker DP, Miller JD, et al. Traumatic acute subdural hematoma: major mortality reduction in comatose patients treated within four hours. N Engl J Med. 1981;304:1511–8. [PubMed]
3. Marshall LF, Toole BM, Bowers SA. The National Traumatic Coma Data Bank. Part 2: Patients who talk and deteriorate: implications for treatment. J Neurosurg. 1983;59:285–8. [PubMed]
4. McCaig LF. National Hospital Ambulatory Medical Care Survey: 1992 emergency department summary. Adv Data. 1994;245:1–12. [PubMed]
5. Stiell IG, Wells GA, Vandemheen K, et al. Variation in ED use of computed tomography for patients with minor head injury. Ann Emerg Med. 1997;30:14–22. [PubMed]
6. Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT Head Rule for patients with minor head injury. Lancet. 2001;357:1391–6. [PubMed]
7. Taheri PA, Karamanoukian H, Gibbons K, et al. Can patients with minor head injuries be safely discharged home? Arch Surg. 1993;128:289–92. [PubMed]
8. Servadei F, Vergoni G, Nasi MT, et al. Management of low-risk head injuries in an entire area: results of an 18-month survey. Surg Neurol. 1993;39:269–75. [PubMed]
9. Martin DR, Semelka RC. Health effects of ionising radiation from diagnostic CT. Lancet. 2006;367:1712–4. [PubMed]
10. Brenner DJ, Hall EJ. Computed tomography — an increasing source of radiation exposure. N Engl J Med. 2007;357:2277–84. [PubMed]
11. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA. 1997;277:488–94. [PubMed]
12. McGinn TG, Guyatt GH, Wyer PC, et al. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. JAMA. 2000;284:79–84. [PubMed]
13. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med. 1999;33:437–47. [PubMed]
14. Stiell IG, Clement CM, Rowe BH, et al. Comparison of the Canadian CT Head Rule and the New Orleans Criteria in patients with minor head injury. JAMA. 2005;294:1151–8. [PubMed]
15. Lee TH. Evaluating decision aids: the next painful step. J Gen Intern Med. 1990;5:528–9. [PubMed]
16. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317–22. [PubMed]
17. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? JAMA. 1999;282:1458–65. [PubMed]
18. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144:201–9. [PubMed]
19. Brehaut JC, Stiell IG, Graham ID. Will a new clinical decision rule be widely used? The case of the Canadian C-Spine Rule. Acad Emerg Med. 2006;13:413–20. [PubMed]
20. Stiell IG, Clement CM, Grimshaw J, et al. Implementation of the Canadian C-Spine Rule: prospective 12 centre cluster randomised trial. BMJ. 2009;339:B4146. doi: 10.1136/bmj.b4146. [PMC free article] [PubMed] [Cross Ref]
21. Grimshaw JM, Shirran L, Thomas RE, et al. Changing provider behaviour: an overview of systematic reviews of interventions. Med Care. 2001;39(Suppl 2):II2–45. [PubMed]
22. Oxman AD, Thomson MA, Davis DA, et al. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423–31. [PMC free article] [PubMed]
23. Bero LA, Grilli R, Grimshaw JM, et al. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 1998;317:465–8. [PMC free article] [PubMed]
24. Stiell IG, Lesiuk H, Vandemheen K, et al. Obtaining consensus for the definition of “clinically important” brain injury in the CCC Study. Acad Emerg Med. 2000;7:572.
25. Donner A, Donald A. Analysis of data arising from a stratified design with cluster as unit of randomization. Stat Med. 1987;6:43–52. [PubMed]
26. Thompson SG, Pyke SD, Hardy RJ. The design and analysis of paired cluster randomized trials: an application of meta-analysis techniques. Stat Med. 1997;16:2063–79. [PubMed]
27. Campbell MK, Mollison J, Grimshaw JM. Cluster trials in implementation research: estimation of intracluster correlations coefficients and sample size. Stat Med. 2001;20:391–9. [PubMed]
28. Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa Ankle Rules. JAMA. 1994;271:827–32. [PubMed]
29. Stiell IG, Wells GA, Laupacis A, et al. Multicentre trial to introduce the Ottawa ankle rules for use of radiography in acute ankle injuries. Multicentre Ankle Rule Study Group. BMJ. 1995;311:594–7. [PMC free article] [PubMed]
30. Stiell IG, Wells GA, Hoag RA, et al. Implementation of the Ottawa Knee Rule for the use of radiography in acute knee injuries. JAMA. 1997;278:2075–8. [PubMed]
31. Stiell IG, Clement C, McKnight RD, et al. The Canadian C-spine Rule versus the NEXUS low-risk criteria in patients with trauma. N Engl J Med. 2003;349:2510–8. [PubMed]
32. Foy R, Maclennan G, Grimshaw J, et al. Attributes of clinical recommendations that influence change in practice following audit and feedback. J Clin Epidemiol. 2002;55:717–22. [PubMed]
33. Heskestad B, Baardsen R, Helseth E, et al. Guideline compliance in management of minimal, mild, and moderate head injury: high frequency of noncompliance among individual physicians despite strong guideline support from clinical leaders. J Trauma. 2008;65:1309–13. [PubMed]
34. Holroyd BR, Wilson D, Rowe BH, et al. Uptake of validated clinical practice guidelines: experience with implementing the Ottawa Ankle Rules. Am J Emerg Med. 2004;22:149–55. [PubMed]
Articles from CMAJ : Canadian Medical Association Journal are provided here courtesy of
Canadian Medical Association