|Home | About | Journals | Submit | Contact Us | Français|
A key mandate of the Canadian Thoracic Society (CTS) is to promote evidence-based respiratory care through clinical practice guidelines (CPGs). To improve the quality and validity of the production, dissemination and implementation of its CPGs, the CTS has revised its guideline process and has created the Canadian Respiratory Guidelines Committee to oversee this process. The present document outlines the basic methodological tools and principles of the new CTS guideline production process. Important features include standard methods for choosing and formulating optimal questions and for finding, appraising, and summarizing the evidence; use of the Grading of Recommendations Assessment, Development and Evaluation system for rating the quality of evidence and strength of recommendations; use of the Appraisal of Guidelines for Research and Evaluation instrument for quality control during and after guideline development and for appraisal of other guidelines; use of the ADAPTE process for adaptation of existing guidelines to the local context; and use of the GuideLine Implementability Appraisal tool to augment implementability of guidelines. The CTS has also committed to develop guidelines in new areas, an annual guideline review cycle, and a new formal process for dissemination and implementation. Ultimately, it is anticipated that these changes will have a significant impact on the quality of care and clinical outcomes of individuals suffering from respiratory diseases across Canada.
L’un des mandats clés de la Société canadienne de thoracologie (SCT) est de promouvoir des soins respiratoires fondés sur des preuves par le biais de directives de pratique clinique (DPC). Afin d’améliorer la qualité et la validité de la production, de la diffusion et de l’application de ces DPC, la SCT a révisé sa marche à suivre pour leur rédaction et mis sur pied un comité canadien pour les directives en pneumologie chargé de superviser le processus. Le présent article résume les outils et les principes méthodologiques de base de cette nouvelle marche à suivre de la SCT. Parmi les éléments importants, mentionnons la standardisation des méthodes pour la sélection et la rédaction des questions les plus pertinentes et pour le recensement, l’évaluation et la synthèse des preuves; l’utilisation d’un système de catégorisation pour l’évaluation, la rédaction et la vérification des recommandations afin de mesurer la qualité des preuves et la solidité des recommandations; l’utilisation d’un instrument d’appréciation des directives pour la recherche et l’évaluation appliqué au contrôle de la qualité durant et après l’élaboration des directives et d’évaluation des autres directives; l’utilisation du procédé ADAPTE pour l’adaptation des directives existantes à un contexte local et l’utilisation d’un outil d’évaluation de l’applicabilité des directives pour faciliter leur mise en œuvre. La SCT s’est également engagée à développer des lignes directrices dans de nouveaux domaines, un cycle annuel de révisions des directives et un nouveau procédé officiel de diffusion et d’application. En terminant, on prévoit que ces changements exerceront un impact significatif sur la qualité des soins et sur l’issue clinique chez les patients qui souffrent de maladies respiratoires au Canada.
Respiratory diseases affect at least 20% of the Canadian population, and the prevalence of many of these conditions has increased in recent decades (1). Although treatments and management strategies are continually improving, inappropriate and/or incomplete care remains prevalent (2). Because of rapid scientific advances and an exponentially growing number of medical publications, it has become increasingly difficult for practitioners to effectively update knowledge and change practice. Accordingly, clinical practice guidelines (CPGs) have emerged as a tool for dissemination of new knowledge across the continuum of care.
In its mandate to reduce the burden of respiratory disease among Canadians by promoting the best respiratory care, the Canadian Thoracic Society (CTS) has produced and disseminated respiratory guidelines since the 1980s, providing the first-ever evidence-based guidelines in some areas. In addition, many CTS representatives have contributed to international guidelines. However, shortcomings in the Canadian guidelines process include the prolonged time required to produce and disseminate new guidelines, and the lack of 1) an integrated plan for guideline production, dissemination and implementation; 2) a central national body to foster collaboration between regional implementation initiatives; and 3) a dedicated process to assess the needs of target users. Furthermore, disparities have existed between individual disease subcommittees with respect to committee structures, methods and financial support.
To address these issues, the CTS has revised its entire guideline production, dissemination, implementation and evaluation process and, with the aid of interdisciplinary teams and experts, has created the Canadian Respiratory Guidelines Committee (CRGC) to oversee this process (Figure 1). New initiatives include establishing a uniform and validated guideline production process across diseases, planning implementation activities concurrently with production, ensuring that recommendations are formulated such that measurable outcomes for monitoring guidelines adherence are identifiable and measurement is undertaken, and formalizing a resource allocation process. Among the first steps toward this new strategy, in September 2008, the CTS hosted an inaugural workshop which was led by guideline experts who introduced current and valid guideline production and assessment methods and instruments to members of the various CTS guideline-writing committees from across the country. The present document summarizes the concepts presented at this meeting. A summary of the main elements of the guideline development methodology is presented in Table 1.
Designed as an instrument to summarize and incorporate vast bodies of evidence into practical recommendations to guide care by front-line clinicians, CPGs have emerged as an important knowledge translation tool.
CPGs rely on the principles of systematic literature search, appraisal, synthesis and presentation, weighing various outcomes to subsequently guide the composition of recommendations. Over the past decade, there has been an exponential growth in the number of guidelines produced, many of which address the same topics. Furthermore, in many cases, guidelines have presented conflicting recommendations (3). This inconsistency was likely attributable to variability in guideline development methodology and to the lack of a ‘gold standard’ for the production and presentation of guidelines. In a review of 279 published guidelines, Shaynefelt et al (4) found that only 17% of guidelines stated the method for identifying primary evidence, 8% reported the use of formal methods to combine scientific data (eg, meta-analysis) or, when data were lacking, to determine expert opinion (eg, the Delphi method), and 13% graded recommendations according to the strength of the evidence (4). In a separate review, guidelines produced by specialty societies in particular performed no better (5). Finally, over the past decade, the emergence of literature detailing ‘care gaps’ – disparities between evidence-based CPG recommendations and actual clinical practice – has forced researchers to reconsider not only CPG implementation but also CPGs themselves in light of this failure.
Given the vast number of existing guidelines, their heterogeneity and the prevalent methodological threats to their validity, and driven by a desire to enhance the role of CPGs as a vehicle for implementation of best evidence, researchers from around the world have since developed a suite of tools and instruments to guide valid CPG production and evaluation.
The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) (6) instrument has emerged as a widely adopted standard by which to rate and present the quality of evidence underlying guideline recommendations. The Appraisal of Guidelines for Research and Evaluation (AGREE) (7) instrument is a validated and generic instrument designed to appraise the quality of clinical guidelines. Constituting a gold standard for quality, it acts as a ‘blueprint’ for guideline developers. The ADAPTE process seeks to facilitate adaptation of existing guidelines to a new context to avert duplicate guideline production by different societies (8). Finally, the GuideLine Implementability Appraisal (GLIA) tool targets the ability of CPGs to bridge care gaps by providing a framework for measuring and improving guideline ‘implementability’ (9). These tools provide uniformity and new benchmarks for guideline process and content validity.
Incorporation of these tools into the CTS guideline production process may improve the quality and validity of our guidelines, facilitate subsequent implementation efforts and concurrent measurements of effects, and enable resource savings through guideline adaptation rather than de novo guideline production whenever possible. Both the American College of Chest Physicians and the American Thoracic Society have committed to similar guideline processes, and both organizations have also designated the GRADE system for grading strength of recommendations and quality of evidence in all of their future guidelines (10,11). Furthermore, the establishment of the CRGC will enable previously independent disease subcommittees to share resources, methodological expertise and guideline development experience, and enable uniform methodology, rigour, structure and appearance for all CTS guidelines (a detailed guideline methodology document is posted online at www.respiratoryguidelines.ca) (12).
Bringing the guideline panel together to choose and formulate specific clinical questions is the first step in the development of effective guidelines. The priority of question development can be informed by clinical care gaps such as underuse of written asthma action plans (13), in which case messages may be targeted to bridge these gaps; controversial areas such as treatment with oral corticosteroids and immunosuppressive agents in patients with idiopathic pulmonary fibrosis, in which case they can provide a summary of the literature supplemented with clinical expertise and interpretation; new literature, in which case they may be used as a vehicle for dissemination to practitioners who may not have time to keep up with subspecialty literature.
Careful attention to question formulation (‘How should it be asked?’) can minimize the time and resources required to search and review the literature. We recommend using the “PICO” method, where “P” refers to the patient group, “I” to the intervention, “C” to the comparison group and “O” to the outcome of interest (14). On occasion, it may be appropriate not to include a comparison group in the clinical question.
The steps of evidence synthesis that must precede the development of recommendations are a literature search, review of abstracts and selection of studies for full review, quality appraisal of individual studies, and combination of results from different studies that address the same question. Unlike informal search and evidence review techniques, a systematic search and review aims to decrease bias in these key steps of evidence consideration.
The importance of using systematic versus informal literature search techniques is often underappreciated by guideline developers faced with extensive process and resource requirements. Gilbert et al (15) systematically reviewed studies of the effect of sleeping position on the rate of sudden infant death syndrome and noted that by 1970, the literature demonstrated a statistically significantly increased risk of sudden infant death for front relative to back sleeping. However, guidelines in this area historically applied informal search techniques, and as a result, did not consistently recommend the back-sleeping position until 1992. Authors estimated that the earlier application of systematic review techniques could have saved 60,000 lives.
The key components of performing a systematic search include defining relevant search phrases and inclusion and exclusion criteria, searching multiple databases, seeking experts to identify missed studies, and using manual search techniques (such as bibliography review). These tasks are best supported by an information specialist (ie, a librarian). Once each search is complete, two guideline panel members or externally contracted clinical epidemiologists review abstracts and choose papers for full review based on predetermined inclusion and exclusion criteria, while maintaining a record of the reason for each exclusion. Studies extracted for full review are then rated for quality using existing published instruments. Formal data extraction tools and tables may then be used to summarize the entire body of evidence for each question, including the strength and consistency of results and the quality of evidence. Guideline writers then assess these evidence summaries in a panel discussion before making recommendations.
When recommendations are made, both the quality of the supporting evidence and the strength of the recommendation are presented. The quality of the evidence supporting a recommendation and the strength of the actual recommendation are interdependent but distinct. The quality of the evidence reflects the methodological rigour of the studies collected for that particular question. Given the importance of the evidence in determining a recommendation, the strength of a recommendation must also reflect the quality of the underlying evidence. However, many other factors influence the strength of a recommendation, including patient values, feasibility and developers’ individual perspectives. The GRADE group has developed a rating system to make these multiple components involved in writing a recommendation explicit (16). The GRADE system distinguishes four levels of overall evidence for rating the quality of evidence (‘high,’ ‘moderate’, ‘low’ and ‘very low’ quality), and two levels for rating the strength of recommendations (‘strong’ and ‘weak’). The strength of a recommendation reflects the overall certainty with which the recommended action should be taken. Strong recommendations are made when the desirable effects of an intervention clearly outweigh the undesirable effects or clearly do not, and weak recommendations are made when these trade-offs are more closely balanced, either because of uncertainty due to low-quality evidence, or because evidence suggests that desirable and undesirable effects are truly closely balanced, in which case patient preferences must be considered in clinical decision making.
Expert input is required in many steps of the guideline development process, including in considering of key questions, weighing the evidence, and making recommendations. The understanding and use of consensus development techniques enhances the ability of a guideline panel to harness expert opinion, while avoiding pitfalls common to group dynamics. Measures include using an impartial facilitator, a skilled committee chair and anonymous vote touchpads for large groups, and establishing preliminary consensus through neutral communication media such as e-mail. In addition, panels must establish an explicit and effective mechanism to deal with conflicts of interest to minimize actual and perceived bias in evidence selection and review.
Clinical practice guidelines have the potential to influence the care of a large number of patients, and there is a need to gauge their quality to mitigate the potential dangers of implementing flawed guideline recommendations. Accordingly, the AGREE Collaboration, an international consortium of CPG developers and researchers, created an instrument for guideline quality appraisal that helps users discriminate between CPGs on the basis of their quality and simultaneously acts as a quality-control tool in the development of new CPGs.
The AGREE Collaboration defines CPG quality as a function of the assessor’s degree of confidence that any biases that could affect the rigour of the development, presentation and applicability of a CPG are minimized, and that each step of the development process is reported (17). The choice of instrument items was based on the scientific literature and existing resources, with a beta version developed, tested, refined and retested following the principles of health measurement tool design. The final instrument – the AGREE instrument (version 1.0) – is composed of 23 items rated on a four-point Likert scale (strongly disagree to strongly agree), one global endorsement item rated on a four-point categorical scale, and a user guide (6,7). Items are divided into six domains: scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, and editorial independence. The instrument does not rate the quality or validity of the clinical content of the recommendations.
The AGREE instrument can be used by a range of stakeholders (eg, guideline developers, clinicians, educators) to assess CPGs that address any health question (eg, diagnosis, treatment) for any clinical condition. It has been endorsed by the World Health Organization and several other guideline enterprises, and translated into 20 languages (17).
Substantial resources are required to develop and update high-quality CPGs. In an effort to reduce duplication and enhance efficiency, adaptation of existing high-quality guidelines may serve as an alternative to de novo guideline development (8).
Guideline adaptation should involve a systematic approach for considering existing guideline(s) produced in one cultural and organizational setting for use in another. Because cultural and organizational differences between and within countries can lead to legitimate variations in recommendations, even when the evidence base is the same (18,19), guidelines produced in one setting may not necessarily be appropriate for another without contextualization. Thus, careful consideration of the specific health questions relevant to the context of use and the local needs, priorities, legislation, policies, and resources is necessary.
The ADAPTE group is an international collaboration of researchers, guideline developers and implementers who aim to enhance the efficiency of guideline development and implementation through adaptation of existing guidelines. The group’s main endeavour has been to develop and validate a generic adaptation process that fosters valid and high-quality adapted guidelines while maintaining a sense of guideline ownership among users.
The ADAPTE methodology involves 24 steps organized in three main phases, each with a set of modules (Figure 2). The set-up phase outlines the tasks to be completed before beginning the adaptation process, such as identifying necessary skills and resources. The adaptation phase assists users in moving from selection of a topic, to identification of specific clinical questions, to searching for, retrieving and assessing guidelines, to deciding whether adaptation is possible and selecting the appropriate guidelines, to preparing the draft adapted guideline. Assessment of the retrieved guidelines involves the evaluation of their quality (using the AGREE instrument), currency (ie, how up-to-date they are), and consistency, as well as the applicability and acceptability of their recommendations in the context of use. This evaluation process provides a framework for informed and transparent selection and modification of source guidelines. This process can result in one of several distinct outcomes including adaptation of a guideline produced elsewhere as a whole, adaptation of format only, modification of single recommendations and update, or production of a customized guideline based on different source guidelines. The finalization phase involves an external review of the guideline, with solicitation of feedback from stakeholders and consultation with developers of the source guidelines used in the adaptation process. Also, a process for regular review and update of the adapted guideline is established, and the final guideline document is produced.
The ADAPTE process is supported by resources including a Web site (www.adapte.org), a detailed manual, and a tool-kit to support application of the process. It is designed to be flexible and may be applied to guidelines for health promotion, screening, diagnosis, treatment and follow-up.
Approaches to improving guideline uptake include extrinsic implementation strategies, which modify the environment or change behaviour to increase guideline adherence, and intrinsic implementation strategies, which modify the guideline itself to facilitate adherence – the latter are reviewed below.
The literature suggests that health care providers seldom consult guidelines in practice; instead, they rapidly glean pieces of information from documents or colleagues with whom they construct ‘mindlines’ that inform clinical decisions (20). To emulate this practical knowledge exchange medium, guidelines should include information that can be easily understood and transmitted. Guidelines should also be specific, because guidelines requiring multiple steps are often poorly described and difficult to follow consistently (21). Intrinsic implementation strategies such as careful specification can be applied to the highest impact elements of the guideline to increase adherence, which may be a very cost-effective way to improve care (21).
Implementability is defined as ‘a set of characteristics that predict the relative ease of implementation of guideline recommendations’ (22). The GLIA tool was designed to provide information about the implementability of a guideline by considering intrinsic factors such as decidability, executability, effect on process of care, presentation/formatting, measurable outcomes, apparent validity, novelty/innovation, flexibility, and computability (22). To demonstrate some of these concepts, we have applied the GLIA tool to a recommendation from the 2003 Canadian Asthma Consensus Guidelines (CACG) (23) (Box 1). The modifications made to this recommendation address the lack of clarity regarding the efficacy of the different treatment choices and the appropriate order for step-up therapy as presented in the original recommendation. The importance of such clarity is highlighted in the GLIA tool, which provides explicit instructions to clearly define the conditions that describe and connect multiple steps in a single recommendation (9). Three other sections of the GLIA tool have been found to be particularly useful for assessing guideline implementability: the presence of measurable outcomes, which guides future evaluation of implementation; presentation and formatting, which is crucial to audience engagement; and effects on process of care delivery, which places a focus on real-world feasibility.
The increasing complexity of the guideline production enterprise has required an extensive revision of guideline processes and methods by the CTS. The present article has summarized the basic methodological tools and principles that will be applied in the future production of all CPGs by the CTS.
In addition to establishing an improved guideline production process, the CTS has prioritized the development of guidelines in new areas, a process to regularly review and update existing guidelines, and a formal process for guideline dissemination and implementation with concurrent evaluation of effectiveness. Accordingly, the CRGC has proposed a new Annual CTS guideline review cycle (Figure 3), to proceed as follows:
The CTS has committed to a new annual guideline review cycle that incorporates a uniform guideline production process that will help to produce valid guidelines while optimizing the use of limited resources. Most importantly, we hope that our renewed focus on effective dissemination of information and integration of recommendations into clinical care will lead to concrete changes in practice. Ultimately, we hope that these changes will not only reduce the overall burden of respiratory diseases in Canada but also improve the clinical outcomes and quality of life of individuals with these diseases.
The authors thank Elena Goubanova and Anne Van Dam from the Canadian Thoracic Society for their help in preparing this manuscript.
FUNDING: Dr Samir Gupta is funded by The Li Ka Shing Knowledge Institute of St Michael’s Hospital, Toronto, Ontario.
CONFLICTS OF INTEREST: Drs Samir Gupta, Onil Bhattacharyya, Melissa C Brouwers, Margaret B Harrison, Paul Hernandez and Valerie A Palda have no conflicts of interest to declare. Ms Elizabeth A Estey has no conflicts of interest to declare. Dr Louis-Philippe Boulet serves on the Advisory Boards of AstraZeneca, GlaxoSmithKline, Merck Frosst, Novartis and Schering-Plough. Dr Louis-Philippe Boulet and has received lecture fees from AstraZeneca Inc, GlaxoSmithKline, Merck Frosst, Novartis and Nycomed. Dr Louis-Philippe Boulet has received sponsorship for investigator-generated research from AstraZeneca, GlaxoSmithKline, Merck Frosst, Schering-Plough and Novartis. Dr Louis-Philippe Boulet has received research funding for participating in multicentre studies from: Alexion, AsthmaTx, AstraZeneca, Boehringer-Ingleheim, Ception, GlaxoSmithKline, Genetech, IVAX, MedImmune, Novartis, Topigen and Wyeth.