Given the uncertainties regarding thyroid nodule assessment and management, physicians require systematically and transparently developed recommendations. This systematic review assesses the quality and consistency of the recommendations of international clinical practice guidelines (CPGs) for the diagnosis and management of thyroid nodules and cancer to assist physicians in making appropriate recommendations.
The CPGs on the management of thyroid nodules and cancer published before June 2013 were retrieved. All the reviewed guidelines were in English. Four reviewers independently assessed the rigor of guideline development by using the Appraisal of Guidelines Research and Evaluation II (AGREE-II) instrument, and their reported evidence was evaluated.
Ten eligible guidelines were included: nine had been developed by professional organizations, and the remaining guideline was endorsed by an independent regional body. Three guidelines achieved a score of greater than 50% in all six AGREE-II domains. Guidelines scored highest on the measurement of ‘scope and purpose’ (≥61.1% for eight CPGs) and lowest on the measurement of ‘applicability’ (≤38.5% for five CPGs). The overall quality ranged from 3.0 to 6.25 on a seven-point scale on the AGREE-II tool. Most CPG recommendations on the management of thyroid cancer were relatively consistent. Guidelines varied regarding the indication of fine-needle aspiration for thyroid nodules, as well as in their suggestions for postoperative radioiodine ablation.
Our analysis showed that the current CPGs varied in methodological quality. More effort is needed to improve the quality of recommendations on the diagnosis and management of thyroid nodules and cancer.
Clinical practice guidelines; Thyroid cancer; Thyroid nodule; Systematic review
To assess the impact of individualised, reconciled evidence-based recommendations (IRERs) and multidisciplinary care in patients with chronic heart failure (CHF) on clinical guideline compliance for CHF and common comorbid conditions.
Design and setting
A retrospective hospital clinical audit conducted between 1st July 2006 and February 2011.
A total of 255 patients with a diagnosis of CHF who attended the Multidisciplinary Ambulatory Consulting Services (MACS) clinics, at the Royal Adelaide Hospital, were included.
Main outcome measures
Compliance with Australian clinical guideline recommendations for CHF, atrial fibrillation, diabetes mellitus and ischaemic heart disease.
Study participants had a median of eight medical conditions (IQR 6–10) and were on an average of 10 (±4) unique medications. Compliance with clinical guideline recommendations for pharmacological therapy for CHF, comorbid atrial fibrillation, diabetes or ischaemic heart disease was high, ranging from 86% for lipid lowering therapy to 98% anti-platelet agents. For all conditions, compliance with lifestyle recommendations was lower than pharmacological therapy, ranging from no podiatry reviews for CHF patients with comorbid diabetes to 75% for heart failure education. Concordance with many guideline recommendations was significantly associated if the patient had IRERs determined, a greater number of recommendations, more clinic visits or if patients participated in a heart failure program.
Despite the high number of comorbid conditions and resulting complexity of the management, high compliance to clinical guideline recommendations was associated with IRER determination in older patients with CHF. Importantly these recommendations need to be communicated to the patient’s general practitioner, regularly monitored and adjusted at clinic visits.
We examined the methodological quality of guidelines on syndromes conferring genetic susceptibility to breast cancer.
PubMed, EMBASE, and Google were searched for guidelines published up to October 2010. All guidelines in English were included. The Appraisal of Guidelines, Research and Evaluation (AGREE) instrument was used to assess the quality of the guidelines, and their reported evidence base was evaluated.
Thirteen guidelines were deemed eligible: seven had been developed by independent associations, and the other six had national/state endorsements. Four guidelines performed satisfactorily, achieving a score of greater than 50% in all six AGREE domains. Mean ± SD standardized scores for the six AGREE domains were: 90 ± 9% for 'scope and purpose', 51 ± 18% for 'stakeholder involvement', 55 ± 27% for 'rigour of development', 80 ± 11% for 'clarity and presentation', 37 ± 32% for 'applicability', and 47 ± 38% for 'editorial independence'. Ten of the thirteen guidelines were found to be based on research evidence.
Given the ethical implications and the high costs of genetic testing for hereditary breast cancer, guidelines on this topic should provide clear and evidence-based recommendations. Our analysis shows that there is scope for improving many aspects of the methodological quality of current guidelines. The AGREE instrument is a useful tool, and could be used profitably by guidelines developers to improve the quality of recommendations.
Breast cancer; BRCA1/2; Familial breast/ovarian cancer; Cancer screening; Cancer surveillance
It is important that healthcare provided in crisis settings is based on the best available research evidence. We reviewed guidelines for child and perinatal health care in crisis situations to determine whether they were based on research evidence, whether Cochrane systematic reviews were available in the clinical areas addressed by these guidelines and whether summaries of these reviews were provided in Evidence Aid.
Broad internet searches were undertaken to identify relevant guidelines. Guidelines were appraised using AGREE and the clinical areas that were relevant to perinatal or child health were extracted. We searched The Cochrane Database of Systematic Reviews to identify potentially relevant reviews. For each review we determined how many trials were included, and how many were conducted in resource-limited settings.
Six guidelines met selection criteria. None of the included guidelines were clearly based on research evidence. 198 Cochrane reviews were potentially relevant to the guidelines. These reviews predominantly addressed nutrient supplementation, breastfeeding, malaria, maternal hypertension, premature labour and prevention of HIV transmission. Most reviews included studies from developing settings. However for large portions of the guidelines, particularly health services delivery, there were no relevant reviews. Only 18 (9.1%) reviews have summaries in Evidence Aid.
We did not identify any evidence-based guidelines for perinatal and child health care in disaster settings. We found many Cochrane reviews that could contribute to the evidence-base supporting future guidelines. However there are important issues to be addressed in terms of the relevance of the available reviews and increasing the number of reviews addressing health care delivery.
This article describes the evidence review and guideline development method developed for the Clinical Preventive Guidelines for Immigrants and Refugees in Canada by the Canadian Collaboration for Immigrant and Refugee Health Guideline Committee.
The Appraisal of Guidelines for Research and Evaluation (AGREE) best-practice framework was combined with the recently developed Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to produce evidence-based clinical guidelines for immigrants and refugees in Canada.
A systematic approach was designed to produce the evidence reviews and apply the GRADE approach, including building on evidence from previous systematic reviews, searching for and comparing evidence between general and specific immigrant populations, and applying the GRADE criteria for making recommendations. This method was used for priority health conditions that had been selected by practitioners caring for immigrants and refugees in Canada.
This article outlines the 14-step method that was defined to standardize the guideline development process for each priority health condition.
Clinical practice and public health guidelines are important tools for translating research findings into practice with the aim of assisting health practitioners as well as patients and consumers in health behavior and healthcare decision-making. Numerous programs for guideline development exist around the world, with growing international collaboration to improve their quality. One of the key features in developing trustworthy guidelines is that recommendations should be based on high-quality systematic reviews of the best available evidence. The review process used by guideline developers to identify and grade relevant evidence for developing recommendations should be systematic, transparent and unbiased. In this paper, we provide an overview of current international developments in the field of practice guidelines and methods to develop guidelines, with a specific focus on the role of systematic reviews. The Guidelines International Network (G-I-N) aims to stimulate collaboration between guideline developers and systematic reviewers to optimize the use of available evidence in guideline development and to increase efficiency in the guideline development process. Considering the significant benefit of systematic reviews for the guideline community, the G-I-N Board of Trustees supports the international prospective register of systematic reviews (PROSPERO) initiative. G-I-N also recently launched a Data Extraction Resource (GINDER) to present and share data extracted from individual studies in a standardized template. PROSPERO and GINDER are complementary tools to enhance collaboration between guideline developers and systematic reviewers to allow for alignment of activities and a reduction in duplication of effort.
guidelines; reviews; systematic; methods
Research in 2007 showed that World Health Organization (WHO) recommendations were largely based on expert opinion, rarely used systematic evidence-based methods, and did not follow the organization's own “Guidelines for Guidelines”. In response, the WHO established a “Guidelines Review Committee” (GRC) to implement and oversee internationally recognized standards. We examined the impact of these changes on WHO guideline documents and explored senior staff's perceptions of the new procedures.
Methods and Findings
We used the AGREE II guideline appraisal tool to appraise ten GRC-approved guidelines from nine WHO departments, and ten pre-GRC guidelines matched by department and topic. We interviewed 20 senior staff across 16 departments and analyzed the transcripts using the framework approach. Average AGREE II scores for GRC-approved guidelines were higher across all six AGREE domains compared with pre-GRC guidelines. The biggest changes were noted for “Rigour of Development” (up 37.6%, from 30.7% to 68.3%) and “Editorial Independence” (up 52.7%, from 20.9% to 73.6%). Four main themes emerged from the interviews: (1) high standards were widely recognized as essential for WHO credibility, particularly with regard to conflicts of interest; (2) views were mixed on whether WHO needed a single quality assurance mechanism, with some departments purposefully bypassing the procedures; (3) staff expressed some uncertainties in applying the GRADE approach, with departmental staff concentrating on technicalities while the GRC remained concerned the underlying principles were not fully institutionalized; (4) the capacity to implement the new standards varied widely, with many departments looking to an overstretched GRC for technical support.
Since 2007, WHO guideline development methods have become more systematic and transparent. However, some departments are bypassing the procedures, and as yet neither the GRC, nor the quality assurance standards they have set, are fully embedded within the organization.
Main problem: Little is known about the methodological quality of guidelines for low back pain treatment. We evaluated the methods used by the developers according to established standards. Methods: PubMed, guideline databases, and the World Wide Web were used to identify guidelines. Seventeen guidelines met the inclusion criteria: interventions for low back pain stated, recommendations based on or explicitly linked to evidence, and English version available. Guidelines were evaluated independently by two appraisers using a practical tool for this purpose, Users’ Guides to the Medical Literature, and the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument. Results: Thirteen guidelines (76%) specified the most important therapies applied, but only nine (53%) included a complete description of the target population. Explicit processes to identify, select, and combine evidence were described in only six guidelines (35%). Few guidelines (3; 18%) explicitly considered all main outcomes when formulating therapeutic recommendations, and none contained a process to determine the relative value of different outcomes. Methodological criteria for grading the strength of the recommendations varied, and were often insufficiently specified. None of the guidelines assessed the impact of uncertainty associated with the evidence and values used. According to AGREE the quality score was highest for the scope and purpose, and clarity and presentation domains, and lowest for editorial independence and applicability. With regard to the recommendations, there was consensus for some of the interventions for acute pain (analgesics and NSAIDs, maintaining physical activity, and avoiding excessive bed rest), but explicit recommendations were lacking or ambiguous for 41% of the interventions. Most of the guidelines did not contemplate specific recommendations for chronic pain. Conclusions: A small number of the available guidelines for low back pain treatment achieved acceptable results for specific quality criteria. In general, the methods to develop the guidelines’ therapeutic recommendations need to be more rigorous, more explicit and better explained. In addition, greater importance should be placed on the recommendations for chronic pain.
Low back pain; Guidelines; Practice guidelines; Quality assurance; Health care
To assess the methodological rigour and transparency of reporting in clinical practice guidelines for the management of allergic rhinitis (AR).
We systematically searched MEDLINE, TRIP database (including the National Guidelines Clearinghouse) and professional society websites for guidelines about the management of AR published after the year 2000. We assumed that older guidelines would no longer influence current clinical practice. If the guideline was updated after 2000 we assessed the most recent version. We included all guidelines published in English and endorsed by an international or national government agency or professional group, irrespective of country of origin or publication status. Two reviewers independently screened search results using predefined eligibility criteria and assessed the rigour of development and reporting of included guidelines using the AGREE II instrument (www.agreetrust.org).
Our search revealed 432 records of which 34 full text articles were assessed for eligibility. Nine documents fulfilled our criteria–3 international and 6 national guidelines from Japan, Singapore, South Africa, UK and the USA. Overall methodological rigour and reporting of guidelines about the management of AR was variable—from fulfilling most AGREE II criteria to almost none. There was no association between the methodological rigour and time of publication or the target scope of the guideline (national versus international). Across all guidelines the most rigorously reported domain was “clarity of presentation” (median score 53%), mainly due to fair presentation of different management options (item 16), followed by “scope and purpose” (median score 42%). The least rigorously addressed was “applicability” domain with median score of 2% across all guidelines. Median scores for domains “stakeholder involvement”, “rigour of development” and editorial independence” were 17%, 15% and 25%, respectively. The ARIA guidelines (2010 update) achieved the highest scores in 5 out of 6 domains and the lowest score on any domain was 60%.
Guideline users should be aware of the variability in quality of development and reporting of guidelines for the management of AR. They should choose higher quality guidelines to inform their practice. For many guidelines there is much room for improvement, in particular in the domains of applicability and implementation.
Many guidelines have been developed in the area of depression but there has been no systematic assessment of their relevance to general practice.
To assess national guidelines on general practice management of depression using two complementary approaches to identify specific ways in which guidance could be made more relevant and applicable to the nature of general practice and the patients who seek help in this context.
Design of study
Review of national guidelines.
Seven English speaking countries: UK, US, Australia, New Zealand, Ireland, Canada, and Singapore.
Seven guidelines were independently reviewed quantitatively using the Appraisal of Guidelines for Research and Evaluation (AGREE) scores and qualitatively using thematic coding.
The quantitative assessment highlights that most of the guidelines fail to meet the criteria on rigour of development, applicability, and editorial independence. The qualitative assessment shows that the majority of guidelines do not address associated risk factors sufficiently and the dilemma of diagnostic uncertainty flows over into management recommendations. Management strategies for depression (antidepressants and psychological strategies) are supported by all of the guidelines, with several listing drugs before psychological therapies; there is limited attention paid to the different types of psychological therapies. Moreover, the guidelines in the main fail to acknowledge individual patient circumstances, in particular the influence on response to treatment of social issues such as adverse life events or social support.
Assessments of current national guidelines on depression management in general practice suggest significant limitations in their relevance to general practice.
appraisal; depression; guidelines
Introduction: The practice-guideline process of collecting, critically appraising, and synthesizing available evidence, then developing expert panel recommendations based on appraised evidence, makes it possible to provide high-quality care for patients. Unwanted variability in the quality and rigor of evidence summaries and Clinical Practice Guidelines has been a long-standing challenge for clinicians seeking evidence-based guidance to support patient care decisions.
Methods: A multidisciplinary group of stakeholders, with representation from all eight Kaiser Permanente Regions, is responsible for creating National Guidelines. Conducting high-quality systematic reviews and creating clinical guidelines are time-, labor-, and resource-intensive processes, which raises challenges for an organization striving to balance rigor with efficiency. For these reasons, the National Guideline Program elected to allow for the identification, assessment, and possible adoption of existing evidence-based guidelines and systematic reviews using the ADAPTE; Appraisal of Guidelines Research and Evaluation; Assessment of Multiple Systematic Reviews (AMSTAR); and Grading of Recommendations Assessment, Development, and Evaluation (GRADE) frameworks. If no acceptable external guidelines are identified, the Guideline Development Team then systematically searches for relevant high-quality systematic reviews, meta-analyses, and original studies. Existing systematic reviews are assessed for quality using a measurement tool to assess systematic reviews (the AMSTAR systematic review checklist).
Study Appraisal: Following the screening and selection process, the included studies (the “body of evidence”) are critically appraised for quality, using the GRADE methodology, which focuses on four key factors that must be considered when assigning strength to a recommendation: balance between desirable and undesirable effects, quality of evidence, values and preferences, and cost. The evidence is then used to create preliminary clinical recommendations. The strength of these recommendations is graded to reflect the extent to which a guideline panel is confident that the desirable effects of an intervention outweigh undesirable effects (or vice versa) across the range of patients for whom the recommendation is intended.
Dissemination: The Care Management Institute disseminates all KP national guidelines to its eight Regions via postings on its Clinical Library Intranet site, a Web-based internal information resource.
This paper describes the process used to arrive at recommended physical activity guidelines for Canadian school-aged children and youth (5-17 years), adults (18-64 years) and older adults (≥65 years).
The Canadian Society for Exercise Physiology (CSEP) Physical Activity Measurement and Guidelines (PAMG) Steering Committee used the Appraisal of Guidelines for Research Evaluation (AGREE II) Instrument to inform the guideline development process. Fourteen background papers and five systematic reviews were completed. Systematic review authors appraised and synthesized the data, and proposed specific recommendations at an international consensus conference of invited experts and key stakeholders. Independently, an international panel of experts interpreted the evidence from the systematic reviews and developed recommendations following attendance at the Consensus Conference.
Using the AGREE II instrument as a guide, specific foci for each of the guidelines were defined and systematic review methodology was used to synthesize the evidence base. The expert panel, CSEP PAMG Steering Committee and methodological consultants reviewed the systematic reviews and Consensus Statement. The expert panel achieved consensus on the level of evidence informing the physical activity guidelines and developed a separate document outlining key recommendations, interpretation of the evidence and justification of each recommendation.
The CSEP and Public Health Agency of Canada followed a rigorous process to examine the evidence informing potential revisions to existing physical activity guidelines for Canadians. It is believed that this is the first physical activity guideline development process in the world to be guided and assessed by AGREE II and AMSTAR instruments.
The study objective was to develop and evaluate a template for evidence-informed symptom protocols for use by nurses over the telephone for the assessment, triage, and management of patients experiencing cancer treatment-related symptoms. Guided by the CAN-IMPLEMENT© methodology, symptom protocols were developed by, conducting a systematic review of the literature to identify clinical practice guidelines and systematic reviews, appraising their quality, reaching consensus on the protocol template, and evaluating the two symptom protocols for acceptability and usability. After excluding one guideline due to poor overall quality, the symptom protocols were developed using 12 clinical practice guidelines (8 for diarrhea and 4 for fever). AGREE Instrument (Appraisal of Guidelines for Research and Evaluation) rigour domain subscale ratings ranged from 8% to 86% (median 60.1 diarrhea; 40.5 fever). Included guidelines were used to inform the protocols along with the Edmonton Symptom Assessment System questionnaire to assess symptom severity. Acceptability and usability testing of the symptom populated template with 12 practicing oncology nurses revealed high readability (n = 12), just the right amount of information (n = 10), appropriate terms (n = 10), fit with clinical work flow (n = 8), and being self-evident for how to complete (n = 5). Five nurses made suggestions and 11 rated patient self-management strategies the highest for usefulness. This new template for symptom protocols can be populated with symptom-specific evidence that nurses can use when assessing, triaging, documenting, and guiding patients to manage their-cancer treatment-related symptoms.
The Canadian Cardiovascular Outcomes Research Team was established in 2001 to improve the quality of cardiovascular care for Canadians. Initially, quality indicators (QIs) for hospital-based care for those with acute myocardial infarctions and congestive heart failure were developed and measured. Qualitative research on the acceptability of those indicators concluded that indicators were needed for ambulatory primary care practice, where the bulk of cardiovascular disease care occurs.
To systematically develop QIs for primary care practice for the primary prevention and chronic disease management of ischemic heart disease, hypertension, hyperlipidemia and heart failure.
A four-stage modified Delphi approach was used and included a literature review of evidence-based practice guidelines and previously developed QIs; the development and circulation of a survey tool with proposed QIs, asking respondents to rate each indicator for validity, necessity to record and feasibility to collect; an in-person meeting of respondents to resolve rating and content discrepancies, and suggest additional QIs; and recirculation of the survey tool for rating of additional QIs. Participants from across Canada included family physicians, primary care nurses, an emergency room family physician and cardiologists.
31 QIs were agreed on, nine of which were for primary prevention and 22 of which were for chronic disease management.
A core set of QIs for ambulatory primary care practice has been developed as a tool for practitioners to evaluate the quality of cardiovascular disease care. While the participants rated the indicators as feasible to collect, the next step will be to conduct field validation.
Congestive heart failure; Coronary disease; Health care delivery; Hypercholesterolemia; Hypertension; Primary care
The management of minimal, mild and moderate head injuries is still controversial. In 2000, the Scandinavian Neurotrauma Committee (SNC) presented evidence-based guidelines for initial management of these injuries. Since then, considerable new evidence has emerged.
General methodology according to the Appraisal of Guidelines for Research and Evaluation (AGREE) II framework and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Systematic evidence-based review according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, based upon relevant clinical questions with respect to patient-important outcomes, including Quality Assessment of Diagnostic Accuracy Studies (QUADAS) and Centre of Evidence Based Medicine (CEBM) quality ratings. Based upon the results, GRADE recommendations, guidelines and discharge instructions were drafted. A modified Delphi approach was used for consensus and relevant clinical stakeholders were consulted.
We present the updated SNC guidelines for initial management of minimal, mild and moderate head injury in adults including criteria for computed tomography (CT) scan selection, admission and discharge with suggestions for monitoring routines and discharge advice for patients. The guidelines are designed to primarily detect neurosurgical intervention with traumatic CT findings as a secondary goal. For elements lacking good evidence, such as in-hospital monitoring, routines were largely based on consensus. We suggest external validation of the guidelines before widespread clinical use is recommended.
computed tomography; GRADE; guidelines; head/brain injury/trauma; management; prediction rule; routines; S100/S100B/S100BB
Carcinoma of the esophagus is an aggressive malignancy with an increasing incidence. Its virulence, in terms of symptoms and mortality, justifies a continued search for optimal therapy. A clinical practice guideline was developed based on a systematic review investigating neoadjuvant or adjuvant therapy on resectable thoracic esophageal cancer.
A systematic review with meta-analysis was developed and clinical recommendations were drafted. External review of the practice guideline report by practitioners in Ontario, Canada was obtained through a mailed survey, and incorporated. Final approval of the practice guideline was obtained from the Practice Guidelines Coordinating Committee.
The systematic review was developed and recommendations were drafted, and the report was mailed to Ontario practitioners for external review. Ninety percent of respondents agreed with both the evidence summary and the draft recommendations, while only 69% approved of the draft recommendations as a practice guideline. Based on the external review, a revised document was created. The revised practice guideline was submitted to the Practice Guidelines Coordinating Committee for review. All 11 members of the PGCC returned ballots. Eight PGCC members approved the practice guideline report as written and three members approved the guideline conditional on specific concerns being addressed. After these recommended changes were made, the final practice guideline report was approved.
In consideration of the systematic review, external review, and subsequent Practice Guidelines Coordinating Committee revision suggestions, and final approval, the Gastrointestinal Cancer Disease Site Group recommends the following:
For adult patients with resectable thoracic esophageal cancer for whom surgery is considered appropriate, surgery alone (i.e., without neoadjuvant or adjuvant therapy) is recommended as the standard practice.
Clinical guidelines have been the subject of much criticism in primary care literature partly due to potential conflicts in their implementation among patients with multiple chronic conditions. We assessed the relevance of selected Canadian clinical guidelines on chronic diseases for patients with comorbidity and examined their quality.
We selected 16 chronic medical conditions according to their frequency of occurrence, complexity of treatment, and pertinence to primary care. Recent Canadian clinical guidelines (2004 - 2009) on these conditions, published in English or French, were retrieved. We assessed guideline relevance to the care of patients with comorbidity with a tool developed by Boyd and colleagues. Quality was assessed using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument.
Regarding relevance, 56.2% of guidelines addressed treatment for patients with multiple chronic conditions and 18.8% addressed the issue for older patients. Fifteen guidelines (93.8%) included specific recommendations for patients with one concurrent condition; only three guidelines (18.8%) addressed specific recommendations for patients with two comorbid conditions and one for more than two concurrent comorbid conditions. Quality of the evaluated guidelines was good to very good in four out of the six domains measured using the AGREE instrument. The domains with lower mean scores were Stakeholder Involvement and Applicability.
The quality of the Canadian guidelines examined is generally good, yet their relevance for patients with two or more chronic conditions is very limited and there is room for improvement in this respect.
Clinical guidelines are considered important instruments to improve quality in health care. Since 1998 the Royal Dutch Society for Physical Therapy (KNGF) produced evidence-based clinical guidelines, based on a standardized program. New developments in the field of guideline research raised the need to evaluate and update the KNGF guideline program.
Purpose of this study is to compare different guideline development programs and review the KNGF guideline program for physical therapy in the Netherlands, in order to update the program.
Six international guideline development programs were selected, and the 23 criteria of the AGREE Instrument were used to evaluate the guideline programs. Information about the programs was retrieved from published handbooks of the organizations. Also, the Dutch program for guideline development in physical therapy was evaluated using the AGREE criteria. Further comparison the six guideline programs was carried out using the following elements of the guideline development processes: Structure and organization; Preparation and initiation; Development; Validation; Dissemination and implementation; Evaluation and update.
Compliance with the AGREE criteria of the guideline programs was high. Four programs addressed 22 AGREE criteria, and two programs addressed 20 AGREE criteria. The previous Dutch program for guideline development in physical therapy lacked in compliance with the AGREE criteria, meeting only 13 criteria.
Further comparison showed that all guideline programs perform systematic literature searches to identify the available evidence. Recommendations are formulated and graded, based on evidence and other relevant factors. It is not clear how decisions in the development process are made. In particular, the process of translating evidence into practice recommendations can be improved.
As a result of international developments and consensus, the described processes for developing clinical practice guidelines have much in common. The AGREE criteria are common basis for the development of guidelines, although it is not clear how final decisions are made. Detailed comparison of the different guideline programs was used for updating the Dutch program. As a result the updated KNGF program complied with 22 AGREE criteria. International discussion is continuing and will be used for further improvement of the program.
Quality indicators (QIs) are used in many healthcare settings to measure, compare, and improve quality of care. For the efficient development of high-quality QIs, rigorous, approved, and evidence-based development methods are needed. Clinical practice guidelines are a suitable source to derive QIs from, but no gold standard for guideline-based QI development exists. This review aims to identify, describe, and compare methodological approaches to guideline-based QI development.
We systematically searched medical literature databases (Medline, EMBASE, and CINAHL) and grey literature. Two researchers selected publications reporting methodological approaches to guideline-based QI development. In order to describe and compare methodological approaches used in these publications, we extracted detailed information on common steps of guideline-based QI development (topic selection, guideline selection, extraction of recommendations, QI selection, practice test, and implementation) to predesigned extraction tables.
From 8,697 hits in the database search and several grey literature documents, we selected 48 relevant references. The studies were of heterogeneous type and quality. We found no randomized controlled trial or other studies comparing the ability of different methodological approaches to guideline-based development to generate high-quality QIs. The relevant publications featured a wide variety of methodological approaches to guideline-based QI development, especially regarding guideline selection and extraction of recommendations. Only a few studies reported patient involvement.
Further research is needed to determine which elements of the methodological approaches identified, described, and compared in this review are best suited to constitute a gold standard for guideline-based QI development. For this research, we provide a comprehensive groundwork.
As newer methods of management are made available, and accessible, survival rates with human immunodeficiency virus (HIV) are increasing. This means that chronic, metabolic complications of HIV are becoming more frequent in clinical practice, as acute morbidity is controlled. Management of HIV/acquired immunodeficiency syndrome (AIDS) is gradually expanding to include these chronic and metabolic complications of the disease, and the adverse effects associated with its treatments, including diabetes. Unfortunately, no guidelines are available to help the medical practitioners choose appropriate therapy for patients with these conditions. The aim of the South Asian Consensus Guidelines is to provide evidence-based recommendations to assist healthcare providers in the rational management of type 2 diabetes mellitus in patients with HIV. The development of these guidelines used systematic reviews of available evidence to form its key recommendations. These guidelines and associated review of literature represent a compilation of available knowledge regarding rational management of diabetes in HIV. Patients of diabetes with concomitant HIV infection are managed optimally with insulin therapy and judicious use of highly active antiretroviral therapy with suitable alternatives is also recommended. These guidelines should prove helpful to physicians, not only in South Asia, but also across the globe, while managing patients with coexistent HIV and diabetes.
Diabetes; human immunodeficiency virus; South Asian Guidelines
In social insurance, the evaluation of work disability is becoming stricter as priority is given to the resumption of work, which calls for a guarantee of quality for these evaluations. Evidence-based guidelines have become a major instrument in the quality control of health care, and the quality of these guidelines' development can be assessed using the AGREE instrument. In social insurance medicine, such guidelines are relatively new. We were interested to know what guidelines have been developed to support the medical evaluation of work disability and the quality of these guidelines.
Five European countries that were reported to use guidelines were approached, using a recent inventory of evaluations of work disability in Europe. We focused on guidelines that are disease-oriented and formally prescribed in social insurance medicine. Using the AGREE instrument, these guidelines were appraised by two researchers. We asked two experts involved in guideline development to indicate if they agreed with our results and to provide explanations for insufficient scores.
We found six German and sixteen Dutch sets of disease-oriented guidelines in official use. The AGREE instrument was applicable, requiring minor adaptations. The appraisers reached consensus on all items. Each guideline scored well on 'scope and purpose' and 'clarity and presentation'. The guidelines scored moderately on 'stakeholder involvement' in the Netherlands, but insufficiently in Germany, due mainly to the limited involvement of patients' representatives in this country. All guidelines had low scores on 'rigour of development', which was due partly to a lack of documentation and of existing evidence. 'Editorial independence' and 'applicability' had low scores in both countries as a result of how the production was organised.
Disease-oriented guidelines in social insurance medicine for the evaluation of work disability are a recent phenomenon, so far restricted to Germany and the Netherlands. The AGREE instrument is suitably applicable to assess the quality of guideline development in social insurance medicine, but some of the scoring rules need to be adapted to the context of social insurance. Existing guidelines do not meet the AGREE criteria to a sufficient level. The way patients' representatives can be involved needs further discussion. The guidelines would profit from more specific recommendations and, for providing evidence, more research is needed on the functional capacity of people with disabilities.
Objective: To determine whether North American guidelines published subsequent to and in the same topic areas as those developed by the US Agency for Health Care Policy and Research (AHCPR) meet the same methodological criteria.
Study design: A guideline appraisal instrument containing 30 criteria was used to evaluate the methodological quality of the AHCPR guidelines, "updates" of the AHCPR guidelines authored by others, and guidelines that referenced or were adapted from the AHCPR guidelines. The frequency with which the criteria appeared in each guideline was compared and an analysis was performed to determine guidelines with two key features of the ACHPR guidelines—multidisciplinary guideline development panels and systematic reviews of the literature. Data were extracted from the guidelines by one investigator and then checked for accuracy by the other.
Results: Fifty two guidelines identified by broad based searches were evaluated. 50% of the criteria were present in every AHCPR guideline. The AHCPR guidelines scored 80% or more on 24 of the 30 criteria compared with 14 for the "updates" and 11 for those that referenced/adapted the AHCPR guidelines. All of the 17 AHCPR guidelines had both multidisciplinary development panels and systematic reviews of the literature compared with five from the other two categories (p<0.05).
Conclusions: North American guidelines developed subsequent to and in the same topic areas as the AHCPR guidelines are of substantially worse methodological quality and ignore key features important to guideline development. This finding contrasts with previously published conclusions that guideline methodological quality is improving over time.
The development of evidence-based clinical practice guidelines has gained wide acceptance in high-income countries and reputable international organizations. Whereas this approach may be a desirable standard, challenges remain in low-income settings with limited capacity and resources for evidence synthesis and guideline development. We present our experience using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for the recent revision of the Kenyan pediatric clinical guidelines focusing on antibiotic treatment of pneumonia.
A team of health professionals, many with minimal prior experience conducting systematic reviews, carried out evidence synthesis for structured clinical questions. Summaries were compiled and distributed to a panel of clinicians, academicians and policy-makers to generate recommendations based on best available research evidence and locally-relevant contextual factors.
We reviewed six eligible articles on non-severe and 13 on severe/very severe pneumonia. Moderate quality evidence suggesting similar clinical outcomes comparing amoxicillin and cotrimoxazole for non-severe pneumonia received a strong recommendation against adopting amoxicillin. The panel voted strongly against amoxicillin for severe pneumonia over benzyl penicillin despite moderate quality evidence suggesting clinical equivalence between the two and additional factors favoring amoxicillin. Very low quality evidence suggesting ceftriaxone was as effective as the standard benzyl penicillin plus gentamicin for very severe pneumonia received a strong recommendation supporting the standard treatment.
Although this exercise may have fallen short of the rigorous requirements recommended by the developers of GRADE, it was arguably an improvement on previous attempts at guideline development in low-income countries and offers valuable lessons for future similar exercises where resources and locally-generated evidence are scarce.
Clinical practice guidelines have been elaborated to summarize evidence related to the management of knee osteoarthritis and to facilitate uptake of evidence-based knowledge by clinicians. The objectives of the present review were summarizing the recommendations of existing guidelines on knee osteoarthritis, and assessing the quality of the guidelines using a standardized and validated instrument – the Appraisal of Guidelines Research and Evaluation (AGREE) tool. Internet medical literature databases from 2001 to 2006 were searched for guidelines, with six guidelines being identified. Thirteen clinician researchers participated in the review. Each reviewer was trained in the AGREE instrument. The guidelines were distributed to four groups of three or four reviewers, each group reviewing one guideline with the exception of one group that reviewed two guidelines. One independent evaluator reviewed all guidelines. All guidelines effectively addressed only a minority of AGREE domains. Clarity/presentation was effectively addressed in three out of six guidelines, scope/purpose and rigour of development in two guidelines, editorial independence in one guideline, and stakeholder involvement and applicability in none. The clinical management recommendation tended to be similar among guidelines, although interventions addressed varied. Acetaminophen was recommended for initial pain treatment, combined with exercise and education. Nonsteroidal anti-inflammatory drugs were recommended if acetaminophen failed to control pain, but cautiously because of gastrointestinal risks. Surgery was recommended in the presence of persistent pain and disability. Education and activity management interventions were superficially addressed in most guidelines. Guideline creators should use the AGREE criteria when developing guidelines. Innovative and effective methods of knowledge translation to health professionals are needed.
A variety of systems have been developed to grade evidence and develop recommendations based on the available evidence. However, development of guidelines for medical tests is especially challenging given the typical indirectness of the evidence; direct evidence of the effects of testing on patient important outcomes is usually absent. We compared grading systems for medical tests on how they use evidence in guideline development.
We used a systematic strategy to look for grading systems specific to medical tests in PubMed, professional guideline websites, via personal correspondence, and handsearching back references of key articles. Using the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument as a starting point, we defined two sets of characteristics to describe these systems: methodological and process ones. Methodological characteristics are features relating to how evidence is gathered, appraised, and used in recommendations. Process characteristics are those relating to the guideline development process. Data were extracted in duplicate and differences resolved through discussion.
Twelve grading systems could be included. All varied in the degree to which methodological and process characteristics were addressed. Having a clinical scenario, identifying the care pathway and/or developing an analytical framework, having explicit criteria for appraising and linking indirect evidence, and having explicit methodologies for translating evidence into recommendations were least frequently addressed. Five systems at most addressed these, to varying degrees of explicitness and completeness. Process wise, features most frequently addressed included involvement of relevant professional groups (8/12), external peer review of completed guidelines (9/12), and recommendations on methods for dissemination (8/12). Characteristics least often addressed were whether the system was piloted (3/12) and funder information (3/12).
Five systems for grading evidence about medical tests in guideline development addressed to differing degrees of explicitness the need for and appraisal of different bodies of evidence, the linking of such evidence, and its translation into recommendations. At present, no one system addressed the full complexity of gathering, assessing and linking different bodies of evidence.
Grading systems; Quality of evidence; Diagnostic; Medical tests; Grade; Guideline development