Search tips
Search criteria 


Logo of cmajCMAJ Information for AuthorsCMAJ Home Page
CMAJ. 2010 September 21; 182(13): 1449–1450.
PMCID: PMC2942925

Quality evidence important for quality guidelines

AGREE1,2 focuses on methodologic issues about guideline development and reporting. Although important, these issues are insufficient to ensure that recommendations are appropriate and valid, because methodologic rigour and validity are not necessarily correlated.3

In an assessment of pharyngitis guidelines by a panel familiar with AGREE,4 guidelines not citing European trials or the Cochrane review had higher AGREE scores than most other guidelines. Only 3 of 23 items in the AGREE instrument assesses the description of the evidence, although each item has the same weight. We therefore decided not to use AGREE in our comparison; we evaluated the evidence with an analysis of the references of the 10 existing guidelines. We concluded that high AGREE scores do not guarantee that the selection of the evidence has been adequately performed — even with the new item #9 about strengths and limitations of the evidence — and that different items may have different relevance to the validity of the recommendations.

In a study on mass colorectal screening, guidelines were found to be equally valid regarding their (different) recommendations related to fecal occult blood tests, but no relation could be found between their methodologic AGREE quality and their content validity.5 Although AGREE is important in developing guidelines, the World Health Organization has also recognized the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence.6



1. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ. 2010;182:1045–52. [PMC free article] [PubMed]
2. Brouwers MC, Kho ME, Browman G, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010;182:E472–8. [PMC free article] [PubMed]
3. Bruls A. AGREE II — Improving the quality of clinical care. Lancet. 2010 Jul 2; [Epub ahead of print]
4. Matthys J, De Meyere M, van Driel ML, et al. Differences among international pharyngitis guidelines : not just academic. Ann Fam Med. 2007;5:436–43. [PubMed]
5. Watine JC, Bunting PS. Mass colorectal cancer screening: methodological quality of practice guidelines is not related to their content validity. Clin Biochem. 2008;41:459–66. [PubMed]
6. Schünemann HJ, Fretheim A, Oxman AD. WHO Advisory Committee on Health Research. Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Res Policy Syst. 2006;4:13. [PMC free article] [PubMed]

Articles from CMAJ : Canadian Medical Association Journal are provided here courtesy of Canadian Medical Association