PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1407210)

Clipboard (0)
None

Related Articles

1.  How Evidence-Based Are the Recommendations in Evidence-Based Guidelines? 
PLoS Medicine  2007;4(8):e250.
Background
Treatment recommendations for the same condition from different guideline bodies often disagree, even when the same randomized controlled trial (RCT) evidence is cited. Guideline appraisal tools focus on methodology and quality of reporting, but not on the nature of the supporting evidence. This study was done to evaluate the quality of the evidence (based on consideration of its internal validity, clinical relevance, and applicability) underlying therapy recommendations in evidence-based clinical practice guidelines.
Methods and Findings
A cross-sectional analysis of cardiovascular risk management recommendations was performed for three different conditions (diabetes mellitus, dyslipidemia, and hypertension) from three pan-national guideline panels (from the United States, Canada, and Europe). Of the 338 treatment recommendations in these nine guidelines, 231 (68%) cited RCT evidence but only 105 (45%) of these RCT-based recommendations were based on high-quality evidence. RCT-based evidence was downgraded most often because of reservations about the applicability of the RCT to the populations specified in the guideline recommendation (64/126 cases, 51%) or because the RCT reported surrogate outcomes (59/126 cases, 47%).
Conclusions
The results of internally valid RCTs may not be applicable to the populations, interventions, or outcomes specified in a guideline recommendation and therefore should not always be assumed to provide high-quality evidence for therapy recommendations.
From an analysis of cardiovascular risk-management recommendations in guidelines produced by pan-national panels, McAlister and colleagues concluded that fewer than half were based on high-quality evidence.
Editors' Summary
Background.
Until recently, doctors largely relied on their own experience to choose the best treatment for their patients. Faced with a patient with high blood pressure (hypertension), for example, the doctor had to decide whether to recommend lifestyle changes or to prescribe drugs to reduce the blood pressure. If he or she chose the latter, he or she then had to decide which drug to prescribe, set a target blood pressure, and decide how long to wait before changing the prescription if this target was not reached. But, over the past decade, numerous clinical practice guidelines have been produced by governmental bodies and medical associations to help doctors make treatment decisions like these. For each guideline, experts have searched the medical literature for the current evidence about the diagnosis and treatment of a disease, evaluated the quality of that evidence, and then made recommendations based on the best evidence available.
Why Was This Study Done?
The recommendations made in different clinical practice guidelines vary, in part because they are based on evidence of varying quality. To help clinicians decide which recommendations to follow, some guidelines indicate the strength of their recommendations by grading them, based on the methods used to collect the underlying evidence. Thus, a randomized clinical trial (RCT)—one in which patients are randomly allocated to different treatments without the patient or clinician knowing the allocation—provides higher-quality evidence than a nonrandomized trial. Similarly, internally valid trials—in which the differences between patient groups are solely due to their different treatments and not to other aspects of the trial—provide high-quality evidence. However, grading schemes rarely consider the size of studies and whether they have focused on clinical or so-called “surrogate” measures. (For example, an RCT of a treatment to reduce heart or circulation [“cardiovascular”] problems caused by high blood pressure might have death rate as a clinical measure; a surrogate endpoint would be blood pressure reduction.) Most guidelines also do not consider how generalizable (applicable) the results of a trial are to the populations, interventions, and outcomes specified in the guideline recommendation. In this study, the researchers have investigated the quality of the evidence underlying recommendations for cardiovascular risk management in nine evidence-based clinical practice guides using these additional criteria.
What Did the Researchers Do and Find?
The researchers extracted the recommendations for managing cardiovascular risk from the current US, Canadian, and European guidelines for the management of diabetes, abnormal blood lipid levels (dyslipidemia), and hypertension. They graded the quality of evidence for each recommendation using the Canadian Hypertension Education Program (CHEP) grading scheme, which considers the type of study, its internal validity, its clinical relevance, and how generally applicable the evidence is considered to be. Of 338 evidence-based recommendations, two-thirds were based on evidence collected in internally valid RCTs, but only half of these RCT-based recommendations were based on high-quality evidence. The evidence underlying 64 of the guideline recommendations failed to achieve a high CHEP grade because the RCT data were collected in a population of people with different characteristics to those covered by the guideline. For example, a recommendation to use spironolactone to reduce blood pressure in people with hypertension was based on an RCT in which the participants initially had congestive heart failure with normal blood pressure. Another 59 recommendations were downgraded because they were based on evidence from RCTs that had not focused on clinical measures of effectiveness.
What Do These Findings Mean?
These findings indicate that although most of the recommendations for cardiovascular risk management therapies in the selected guidelines were based on evidence collected in internally valid RCTs, less than one-third were based on high-quality evidence applicable to the populations, treatments, and outcomes specified in guideline recommendations. A limitation of this study is that it analyzed a subset of recommendations in only a few guidelines. Nevertheless, the findings serve to warn clinicians that evidence-based guidelines are not necessarily based on high-quality evidence. In addition, they emphasize the need to make the evidence base underlying guideline recommendations more transparent by using an extended grading system like the CHEP scheme. If this were done, the researchers suggest, it would help clinicians apply guideline recommendations appropriately to their individual patients.
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0040250.
• Wikipedia contains pages on evidence-based medicine and on clinical practice guidelines (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
• The National Guideline Clearinghouse provides information on US national guidelines
• The Guidelines International Network promotes the systematic development and application of clinical practice guidelines
• Information is available on the Canadian Hypertension Education Program (CHEP) (in French and English)
• See information on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group, an organization that has developed an grading scheme similar to the CHEP scheme (in English, Spanish, French, German, and Italian)
doi:10.1371/journal.pmed.0040250
PMCID: PMC1939859  PMID: 17683197
2.  Improving the use of research evidence in guideline development: 1. Guidelines for guidelines 
Background
The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the first of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.
Objectives
We reviewed the literature on guidelines for the development of guidelines.
Methods
We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.
Key questions and answers
We found no experimental research that compared different formats of guidelines for guidelines or studies that compared different components of guidelines for guidelines. However, there are many examples, surveys and other observational studies that compared the impact of different guideline development documents on guideline quality.
What have other organizations done to develop guidelines for guidelines from which WHO can learn?
• Establish a credible, independent committee that evaluates existing methods for developing guidelines or that updates existing ones.
• Obtain feedback and approval from various stakeholders during the development process of guidelines for guidelines.
• Develop a detailed source document (manual) that guideline developers can use as reference material.
What should be the key components of WHO guidelines for guidelines?
• Guidelines for guidelines should include information and instructions about the following components: 1) Priority setting; 2) Group composition and consultations; 3) Declaration and avoidance of conflicts of interest; 4) Group processes; 5) Identification of important outcomes; 6) Explicit definition of the questions and eligibility criteria ; 7) Type of study designs for different questions; 8) Identification of evidence; 9) Synthesis and presentation of evidence; 10) Specification and integration of values; 11) Making judgments about desirable and undesirable effects; 12) Taking account of equity; 13) Grading evidence and recommendations; 14) Taking account of costs; 15) Adaptation, applicability, transferability of guidelines; 16) Structure of reports; 17) Methods of peer review; 18) Planned methods of dissemination & implementation; 19) Evaluation of the guidelines.
What have other organizations done to implement guidelines for guidelines from which WHO can learn?
• Obtain buy-in from regions and country level representatives for guidelines for guidelines before dissemination of a revised version.
• Disseminate the guidelines for guidelines widely and make them available (e.g. on the Internet).
• Develop examples of guidelines that guideline developers can use as models when applying the guidelines for guidelines.
• Ensure training sessions for those responsible for developing guidelines.
• Continue to monitor the methodological literature on guideline development.
doi:10.1186/1478-4505-4-13
PMCID: PMC1665445  PMID: 17118181
3.  Hypertension guidelines and their effects on the health system 
Introduction
Hypertension guidelines, which have existed for many years and primarily used in the USA, Canada and Great Britain, are now becoming an issue in Germany. Strong efforts are presently underway for a German version comparable to the guidelines developed for the mentioned countries. The development of guidelines is a part of the implementation system of guidelines in Germany. It covers the mode of operation of the AWMF (work community of the scientific medical subject companies) with the clearinghouse for guidelines (CLA) and the cooperation with the centre for medical quality (ÄZQ).
In the HTA report the real use of the hypertension guidelines shall be investigated for Germany from the development trends and further possibilities of use according to a medical applicability. Economic issues and an optimisation of use are also discussed.
Question
The following questions shall be answered in particular:
How much are the guidelines used concerning hypertension? Can effects (or their influence) be established on the medical procedures? Are there statements available about costs and cost effectiveness? Are there recommendations for further use?
Methodology
To answer these questions, a comprehensive literature search was done. No empirical investigation was carried out. From this enquiry 206 articles were checked in detail but not all of them were available in full text.
Only those publications which directly dealt with high blood pressure guidelines or articles with a direct reference to the topic have been considered in the HTA report.
Publications concerning screening or methods of prevention, medical studies of the hypertension syndrome without a direct reference to guidelines and publications concerned with putting guidelines into action were excluded.
Results
After an analysis of the selected literature addressing the topic of hypertension guidelines, it was evident that the use of these guidelines cannot be gathered from existing literature at the present time. One can assume from international studies with analogical reasoning that these are confessed and have a high level of acceptance in the medical community. Unfortunately the actual usage is not represented satisfactorily in the scientific literature.
The effects of the guidelines on the medical procedures seem to be very strongly individual and the analyses to the compliance show at least an observable effect within the last few years. No publications could be found for the cost effectiveness of the guidelines.
The actual compliance with guidelines seems to be in relation with the duration of the professional practice. It seems the shorter the professional practice takes place, the stronger the guidelines are adhered.
Discussion
At present, there are only a few notes for the German health service regarding the actual effect of the hypertonus guidelines. However, the reason is not that the effect would not be possibly strong but at the methodical challenge to evaluate the sustaining effects of the application of the hypertonus guidelines. For this reason the literature is very rare regarding this topic.
For Germany it can be derived by analogical reasoning from foreign studies that guidelines will facilitate a more and more essential contribution to the design of the health system. Considering that primarily younger physicians accepted guidelines mode, the further construction, update and implementation of guidelines are essential, particularly with regard to the quality assurance. Straight guidelines can express a standard of the quality of a health system as a benchmark. The existence of guidelines or the lack thereof is considered also as a quality indicator of a health system at the organisation for economic cooperation and development (OECD).
Conclusion
Guidelines should be evaluated - especially the hypertonus guideline. Also further development and implementation should be emphasised. Methodically oriented work to the approach is pretty recent.
It is undeniable that guidelines represent a very essential and important contribution for the successful dealing with significant morbidity problems in a health system.
The fact that primarily younger doctors more frequently adopt, employ and adhere to guidelines leads to the assumption that expected sustainability for practical use will increase. Furthermore intensified use of guidelines can be considered in the "mainstream" of the development of the public health system also in an international perspective.
Not one single publication contradicts that a further acquirement, update and distribution of guidelines for the use of practices is necessary. The importance of the guideline is also not questioned in any article.
PMCID: PMC3011314  PMID: 21289932
4.  Inclusion of Ethical Issues in Dementia Guidelines: A Thematic Text Analysis 
PLoS Medicine  2013;10(8):e1001498.
Background
Clinical practice guidelines (CPGs) aim to improve professionalism in health care. However, current CPG development manuals fail to address how to include ethical issues in a systematic and transparent manner. The objective of this study was to assess the representation of ethical issues in general CPGs on dementia care.
Methods and Findings
To identify national CPGs on dementia care, five databases of guidelines were searched and national psychiatric associations were contacted in August 2011 and in June 2013. A framework for the assessment of the identified CPGs' ethical content was developed on the basis of a prior systematic review of ethical issues in dementia care. Thematic text analysis and a 4-point rating score were employed to assess how ethical issues were addressed in the identified CPGs. Twelve national CPGs were included. Thirty-one ethical issues in dementia care were identified by the prior systematic review. The proportion of these 31 ethical issues that were explicitly addressed by each CPG ranged from 22% to 77%, with a median of 49.5%. National guidelines differed substantially with respect to (a) which ethical issues were represented, (b) whether ethical recommendations were included, (c) whether justifications or citations were provided to support recommendations, and (d) to what extent the ethical issues were explained.
Conclusions
Ethical issues were inconsistently addressed in national dementia guidelines, with some guidelines including most and some including few ethical issues. Guidelines should address ethical issues and how to deal with them to help the medical profession understand how to approach care of patients with dementia, and for patients, their relatives, and the general public, all of whom might seek information and advice in national guidelines. There is a need for further research to specify how detailed ethical issues and their respective recommendations can and should be addressed in dementia guidelines.
Please see later in the article for the Editors' Summary
Editors’ Summary
Background
In the past, doctors tended to rely on their own experience to choose the best treatment for their patients. Faced with a patient with dementia (a brain disorder that affects short-term memory and the ability tocarry out normal daily activities), for example, a doctor would use his/her own experience to help decide whether the patient should remain at home or would be better cared for in a nursing home. Similarly, the doctor might have to decide whether antipsychotic drugs might be necessary to reduce behavioral or psychological symptoms such as restlessness or shouting. However, over the past two decades, numerous evidence-based clinical practice guidelines (CPGs) have been produced by governmental bodies and medical associations that aim to improve standards of clinical competence and professionalism in health care. During the development of each guideline, experts search the medical literature for the current evidence about the diagnosis and treatment of a disease, evaluate the quality of that evidence, and then make recommendations based on the best evidence available.
Why Was This Study Done?
Currently, CPG development manuals do not address how to include ethical issues in CPGs. A health-care professional is ethical if he/she behaves in accordance with the accepted principles of right and wrong that govern the medical profession. More specifically, medical professionalism is based on a set of binding ethical principles—respect for patient autonomy, beneficence, non-malfeasance (the “do no harm” principle), and justice. In particular, CPG development manuals do not address disease-specific ethical issues (DSEIs), clinical ethical situations that are relevant to the management of a specific disease. So, for example, a DSEI that arises in dementia care is the conflict between the ethical principles of non-malfeasance and patient autonomy (freedom-to-move-at-will). Thus, healthcare professionals may have to decide to physically restrain a patient with dementia to prevent the patient doing harm to him- or herself or to someone else. Given the lack of guidance on how to address ethical issues in CPG development manuals, in this thematic text analysis, the researchers assess the representation of ethical issues in CPGs on general dementia care. Thematic text analysis uses a framework for the assessment of qualitative data (information that is word-based rather than number-based) that involves pinpointing, examining, and recording patterns (themes) among the available data.
What Did the Researchers Do and Find?
The researchers identified 12 national CPGs on dementia care by searching guideline databases and by contacting national psychiatric associations. They developed a framework for the assessment of the ethical content in these CPGs based on a previous systematic review of ethical issues in dementia care. Of the 31 DSEIs included by the researchers in their analysis, the proportion that were explicitly addressed by each CPG ranged from 22% (Switzerland) to 77% (USA); on average the CPGs explicitly addressed half of the DSEIs. Four DSEIs—adequate consideration of advanced directives in decision making, usage of GPS and other monitoring techniques, covert medication, and dealing with suicidal thinking—were not addressed in at least 11 of the CPGs. The inclusion of recommendations on how to deal with DSEIs ranged from 10% of DSEIs covered in the Swiss CPG to 71% covered in the US CPG. Overall, national guidelines differed substantially with respect to which ethical issues were included, whether ethical recommendations were included, whether justifications or citations were provided to support recommendations, and to what extent the ethical issues were clearly explained.
What Do These Findings Mean?
These findings show that national CPGs on dementia care already address clinical ethical issues but that the extent to which the spectrum of DSEIs is considered varies widely within and between CPGs. They also indicate that recommendations on how to deal with DSEIs often lack the evidence that health-care professionals use to justify their clinical decisions. The researchers suggest that this situation can and should be improved, although more research is needed to determine how ethical issues and recommendations should be addressed in dementia guidelines. A more systematic and transparent inclusion of DSEIs in CPGs for dementia (and for other conditions) would further support the concept of medical professionalism as a core element of CPGs, note the researchers, but is also important for patients and their relatives who might turn to national CPGs for information and guidance at a stressful time of life.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001498.
Wikipedia contains a page on clinical practice guidelines (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The US National Guideline Clearinghouse provides information on national guidelines, including CPGs for dementia
The Guidelines International Network promotes the systematic development and application of clinical practice guidelines
The American Medical Association provides information about medical ethics; the British Medical Association provides information on all aspects of ethics and includes an essential tool kit that introduces common ethical problems and practical ways to deal with them
The UK National Health Service Choices website provides information about dementia, including a personal story about dealing with dementia
MedlinePlus provides links to additional resources about dementia and about Alzheimers disease, a specific type of dementia (in English and Spanish)
The UK Nuffield Council on Bioethics provides the report Dementia: ethical issues and additional information on the public consultation on ethical issues in dementia care
doi:10.1371/journal.pmed.1001498
PMCID: PMC3742442  PMID: 23966839
5.  Effect of an Educational Toolkit on Quality of Care: A Pragmatic Cluster Randomized Trial 
PLoS Medicine  2014;11(2):e1001588.
In a pragmatic cluster-randomized trial, Baiju Shah and colleagues evaluated the effectiveness of printed educational materials for clinician education focusing on cardiovascular disease screening and risk reduction in people with diabetes.
Please see later in the article for the Editors' Summary
Background
Printed educational materials for clinician education are one of the most commonly used approaches for quality improvement. The objective of this pragmatic cluster randomized trial was to evaluate the effectiveness of an educational toolkit focusing on cardiovascular disease screening and risk reduction in people with diabetes.
Methods and Findings
All 933,789 people aged ≥40 years with diagnosed diabetes in Ontario, Canada were studied using population-level administrative databases, with additional clinical outcome data collected from a random sample of 1,592 high risk patients. Family practices were randomly assigned to receive the educational toolkit in June 2009 (intervention group) or May 2010 (control group). The primary outcome in the administrative data study, death or non-fatal myocardial infarction, occurred in 11,736 (2.5%) patients in the intervention group and 11,536 (2.5%) in the control group (p = 0.77). The primary outcome in the clinical data study, use of a statin, occurred in 700 (88.1%) patients in the intervention group and 725 (90.1%) in the control group (p = 0.26). Pre-specified secondary outcomes, including other clinical events, processes of care, and measures of risk factor control, were also not improved by the intervention. A limitation is the high baseline rate of statin prescribing in this population.
Conclusions
The educational toolkit did not improve quality of care or cardiovascular outcomes in a population with diabetes. Despite being relatively easy and inexpensive to implement, printed educational materials were not effective. The study highlights the need for a rigorous and scientifically based approach to the development, dissemination, and evaluation of quality improvement interventions.
Trial Registration
http://www.ClinicalTrials.gov NCT01411865 and NCT01026688
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Clinical practice guidelines help health care providers deliver the best care to patients by combining all the evidence on disease management into specific recommendations for care. However, the implementation of evidence-based guidelines is often far from perfect. Take the example of diabetes. This common chronic disease, which is characterized by high levels of sugar (glucose) in the blood, impairs the quality of life of patients and shortens life expectancy by increasing the risk of cardiovascular diseases (conditions that affect the heart and circulation) and other life-threatening conditions. Patients need complex care to manage the multiple risk factors (high blood sugar, high blood pressure, high levels of fat in the blood) that are associated with the long-term complications of diabetes, and they need to be regularly screened and treated for these complications. Clinical practice guidelines for diabetes provide recommendations on screening and diagnosis, drug treatment, and cardiovascular disease risk reduction, and on helping patients self-manage their disease. Unfortunately, the care delivered to patients with diabetes frequently fails to meet the standards laid down in these guidelines.
Why Was This Study Done?
How can guideline adherence and the quality of care provided to patients be improved? A common approach is to send printed educational materials to clinicians. For example, when the Canadian Diabetes Association (CDA) updated its clinical practice guidelines in 2008, it mailed educational toolkits that contained brochures and other printed materials targeting key themes from the guidelines to family physicians. In this pragmatic cluster randomized trial, the researchers investigate the effect of the CDA educational toolkit that targeted cardiovascular disease screening and treatment on the quality of care of people with diabetes. A pragmatic trial asks whether an intervention works under real-life conditions and whether it works in terms that matter to the patient; a cluster randomized trial randomly assigns groups of people to receive alternative interventions and compares outcomes in the differently treated “clusters.”
What Did the Researchers Do and Find?
The researchers randomly assigned family practices in Ontario, Canada to receive the educational toolkit in June 2009 (intervention group) or in May 2010 (control group). They examined outcomes between July 2009 and April 2010 in all patients with diabetes in Ontario aged over 40 years (933,789 people) using population-level administrative data. In Canada, administrative databases record the personal details of people registered with provincial health plans, information on hospital visits and prescriptions, and physician service claims for consultations, assessments, and diagnostic and therapeutic procedures. They also examined clinical outcome data from a random sample of 1,592 patients at high risk of cardiovascular complications. In the administrative data study, death or non-fatal heart attack (the primary outcome) occurred in about 11,500 patients in both the intervention and control group. In the clinical data study, the primary outcome―use of a statin to lower blood fat levels―occurred in about 700 patients in both study groups. Secondary outcomes, including other clinical events, processes of care, and measures of risk factor control were also not improved by the intervention. Indeed, in the administrative data study, some processes of care outcomes related to screening for heart disease were statistically significantly worse in the intervention group than in the control group, and in the clinical data study, fewer patients in the intervention group reached blood pressure targets than in the control group.
What Do These Findings Mean?
These findings suggest that the CDA cardiovascular diseases educational toolkit did not improve quality of care or cardiovascular outcomes in a population with diabetes. Indeed, the toolkit may have led to worsening in some secondary outcomes although, because numerous secondary outcomes were examined, this may be a chance finding. Limitations of the study include its length, which may have been too short to see an effect of the intervention on clinical outcomes, and the possibility of a ceiling effect—the control group in the clinical data study generally had good care, which left little room for improvement of the quality of care in the intervention group. Overall, however, these findings suggest that printed educational materials may not be an effective way to improve the quality of care for patients with diabetes and other complex conditions and highlight the need for a rigorous, scientific approach to the development, dissemination, and evaluation of quality improvement interventions.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001588.
The US National Diabetes Information Clearinghouse provides information about diabetes for patients, health care professionals, and the general public (in English and Spanish)
The UK National Health Service Choices website provides information (including some personal stories) for patients and carers about type 2 diabetes, the commonest form of diabetes
The Canadian Diabetes Association also provides information about diabetes for patients (including some personal stories about living with diabetes) and health care professionals; its latest clinical practice guidelines are available on its website
The UK National Institute for Health and Care Excellence provides general information about clinical guidelines and about health care quality standards in the UK
The US Agency for Healthcare Research and Quality aims to improve the quality, safety, efficiency, and effectiveness of health care for all Americans (information in English and Spanish); the US National Guideline Clearinghouse is a searchable database of clinical practice guidelines
The International Diabetes Federation provides information about diabetes for patients and health care professionals, along with international statistics on the burden of diabetes
doi:10.1371/journal.pmed.1001588
PMCID: PMC3913553  PMID: 24505216
6.  A novel method of guideline development for the diagnosis and management of mild to moderate hypertension. 
BACKGROUND: There are large numbers of clinical guidelines available covering many clinical areas. However, the variable quality of their content has meant that doctors may have been offered advice that has been poorly researched or is of a conflicting nature. It has been shown that local involvement in guideline development increases the likelihood of their use. AIM: To develop a guideline to be used by general practitioners in six practices in Birmingham from existing evidence-based guidelines. METHOD: Recommendations from the four most cited international hypertension guidelines, and the more recently published New Zealand guidelines, were divided into subject areas and tabulated to facilitate direct comparison. Where there was complete or majority (> or = 3/5) agreement, the recommendation was taken as acceptable for inclusion in the new guideline. Where there was disagreement (< or = 2/5), recommendations were based on the best available evidence following a further MEDLINE literature search and critical appraisal of the relevant literature. Each recommendation was accompanied by a grade of evidence (A-D), as defined by the Canadian Hypertension Society, and an 'action required' statement of either 'must', 'should', or 'could', based on the Eli-Lilly National Clinical Audit Centre Hypertension Audit criteria. The recommendations were summarized into a guideline algorithm and a supporting document. The final format of both parts of the guideline was decided after consultation with the practice teams. The practices individually decided on methods of data collection. RESULTS: The guideline was presented as a double-sided, A4 laminated sheet and an A4 bound supporting document containing a synthesis of the original guidelines in tabular form, a table of the resulting recommendations, and appendices of current literature reviews on areas of disagreement. The content of the final Birmingham Clinical Effectiveness Group (BCEG) guideline differed minimally from any of the original guidelines. CONCLUSION: The main strength of this method of guideline development may lie, not in the actual content of the resulting guideline, but in the strength of ownership felt by the BCEG and the practices following its development. While the full process is unlikely to be possible for general practitioners working outside an academic environment, the techniques used could provide a framework for practitioners to adapt national and international guidelines for use at a local level.
PMCID: PMC1313367  PMID: 10343418
7.  Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments 
PLoS Medicine  2013;10(7):e1001489.
Background
The vast majority of medical interventions introduced into clinical development prove unsafe or ineffective. One prominent explanation for the dismal success rate is flawed preclinical research. We conducted a systematic review of preclinical research guidelines and organized recommendations according to the type of validity threat (internal, construct, or external) or programmatic research activity they primarily address.
Methods and Findings
We searched MEDLINE, Google Scholar, Google, and the EQUATOR Network website for all preclinical guideline documents published up to April 9, 2013 that addressed the design and conduct of in vivo animal experiments aimed at supporting clinical translation. To be eligible, documents had to provide guidance on the design or execution of preclinical animal experiments and represent the aggregated consensus of four or more investigators. Data from included guidelines were independently extracted by two individuals for discrete recommendations on the design and implementation of preclinical efficacy studies. These recommendations were then organized according to the type of validity threat they addressed. A total of 2,029 citations were identified through our search strategy. From these, we identified 26 guidelines that met our eligibility criteria—most of which were directed at neurological or cerebrovascular drug development. Together, these guidelines offered 55 different recommendations. Some of the most common recommendations included performance of a power calculation to determine sample size, randomized treatment allocation, and characterization of disease phenotype in the animal model prior to experimentation.
Conclusions
By identifying the most recurrent recommendations among preclinical guidelines, we provide a starting point for developing preclinical guidelines in other disease domains. We also provide a basis for the study and evaluation of preclinical research practice.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
The development process for new drugs is lengthy and complex. It begins in the laboratory, where scientists investigate the causes of diseases and identify potential new treatments. Next, promising interventions undergo preclinical research in cells and in animals (in vivo animal experiments) to test whether the intervention has the expected effect and to support the generalization (extension) of this treatment–effect relationship to patients. Drugs that pass these tests then enter clinical trials, where their safety and efficacy is tested in selected groups of patients under strictly controlled conditions. Finally, the government bodies responsible for drug approval review the results of the clinical trials, and successful drugs receive a marketing license, usually a decade or more after the initial laboratory work. Notably, only 11% of agents that enter clinical testing (investigational drugs) are ultimately licensed.
Why Was This Study Done?
The frequent failure of investigational drugs during clinical translation is potentially harmful to trial participants. Moreover, the costs of these failures are passed onto healthcare systems in the form of higher drug prices. It would be good, therefore, to reduce the attrition rate of investigational drugs. One possible explanation for the dismal success rate of clinical translation is that preclinical research, the key resource for justifying clinical development, is flawed. To address this possibility, several groups of preclinical researchers have issued guidelines intended to improve the design and execution of in vivo animal studies. In this systematic review (a study that uses predefined criteria to identify all the research on a given topic), the authors identify the experimental practices that are commonly recommended in these guidelines and organize these recommendations according to the type of threat to validity (internal, construct, or external) that they address. Internal threats to validity are factors that confound reliable inferences about treatment–effect relationships in preclinical research. For example, experimenter expectation may bias outcome assessment. Construct threats to validity arise when researchers mischaracterize the relationship between an experimental system and the clinical disease it is intended to represent. For example, researchers may use an animal model for a complex multifaceted clinical disease that only includes one characteristic of the disease. External threats to validity are unseen factors that frustrate the transfer of treatment–effect relationships from animal models to patients.
What Did the Researchers Do and Find?
The researchers identified 26 preclinical guidelines that met their predefined eligibility criteria. Twelve guidelines addressed preclinical research for neurological and cerebrovascular drug development; other disorders covered by guidelines included cardiac and circulatory disorders, sepsis, pain, and arthritis. Together, the guidelines offered 55 different recommendations for the design and execution of preclinical in vivo animal studies. Nineteen recommendations addressed threats to internal validity. The most commonly included recommendations of this type called for the use of power calculations to ensure that sample sizes are large enough to yield statistically meaningful results, random allocation of animals to treatment groups, and “blinding” of researchers who assess outcomes to treatment allocation. Among the 25 recommendations that addressed threats to construct validity, the most commonly included recommendations called for characterization of the properties of the animal model before experimentation and matching of the animal model to the human manifestation of the disease. Finally, six recommendations addressed threats to external validity. The most commonly included of these recommendations suggested that preclinical research should be replicated in different models of the same disease and in different species, and should also be replicated independently.
What Do These Findings Mean?
This systematic review identifies a range of investigational recommendations that preclinical researchers believe address threats to the validity of preclinical efficacy studies. Many of these recommendations are not widely implemented in preclinical research at present. Whether the failure to implement them explains the frequent discordance between the results on drug safety and efficacy obtained in preclinical research and in clinical trials is currently unclear. These findings provide a starting point, however, for the improvement of existing preclinical research guidelines for specific diseases, and for the development of similar guidelines for other diseases. They also provide an evidence-based platform for the analysis of preclinical evidence and for the study and evaluation of preclinical research practice. These findings should, therefore, be considered by investigators, institutional review bodies, journals, and funding agents when designing, evaluating, and sponsoring translational research.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001489.
The US Food and Drug Administration provides information about drug approval in the US for consumers and for health professionals; its Patient Network provides a step-by-step description of the drug development process that includes information on preclinical research
The UK Medicines and Healthcare Products Regulatory Agency (MHRA) provides information about all aspects of the scientific evaluation and approval of new medicines in the UK; its My Medicine: From Laboratory to Pharmacy Shelf web pages describe the drug development process from scientific discovery, through preclinical and clinical research, to licensing and ongoing monitoring
The STREAM website provides ongoing information about policy, ethics, and practices used in clinical translation of new drugs
The CAMARADES collaboration offers a “supporting framework for groups involved in the systematic review of animal studies” in stroke and other neurological diseases
doi:10.1371/journal.pmed.1001489
PMCID: PMC3720257  PMID: 23935460
8.  Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest 
Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this first paper we discuss: the target audience(s) for guidelines and their use of guidelines; identifying topics for guidelines; guideline group composition (including consumer involvement) and the processes by which guideline groups function and the important procedural issue of managing conflicts of interest in guideline development.
doi:10.1186/1748-5908-7-60
PMCID: PMC3523009  PMID: 22762776
9.  Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development 
Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this third paper we discuss the issues of: reviewing, reporting, and publishing guidelines; updating guidelines; and the two emerging issues of enhancing guideline implementability and how guideline developers should approach dealing with the issue of patients who will be the subject of guidelines having co-morbid conditions.
doi:10.1186/1748-5908-7-62
PMCID: PMC3503794  PMID: 22762242
10.  Developing evidence-based clinical practice guidelines in hospitals in Australia, Indonesia, Malaysia, the Philippines and Thailand: values, requirements and barriers 
Background
Evidence-based clinical practice guidelines support clinical decision-making by making recommendations to guide clinical practice. These recommendations are developed by integrating the expertise of a multidisciplinary group of clinicians with the perspectives of consumers and the best available research evidence. However studies have raised concerns about the quality of guideline development, and particularly the link between research and recommendations. The reasons why guideline developers are not following the established development methods are not clear.
We aimed to explore the barriers to developing evidence-based guidelines in eleven hospitals in Australia, Indonesia, Malaysia, the Philippines and Thailand, so as to better understand how evidence-based guideline development could be facilitated in these settings. The research aimed to identify the value clinicians place on guidelines, what clinicians want in guidelines developed in hospital settings and what factors limit rigorous evidence-based guideline development in these settings.
Methods
Semi-structured, face-to-face interviews were undertaken with senior and junior healthcare providers (nurses, midwives, doctors, allied health) from the maternal and neonatal services of the eleven participating hospitals. Interviews were audio-recorded, transcribed and a thematic analysis undertaken.
Results
Ninety-three individual, 25 pair and eleven group interviews were conducted. Participants were clear that they want guidelines that are based on evidence and updated regularly. They were also clear that there are major barriers to this. Most of the barriers were shared across countries, and included lack of time, lack of skills in finding, appraising and interpreting evidence, lack of access to relevant evidence and difficulty arranging meetings and achieving consensus.
Barriers that were primarily identified in Australian hospitals include cumbersome organisational processes and a feeling that guidelines are being developed for bureaucratic ends. Barriers that were primarily identified in South East Asian hospitals include difficulty accessing evidence due to limited resources available for computers, internet and journal subscriptions and limited skills in computing and English.
Conclusions
The clinicians in these eleven very different hospitals want evidence-based guidelines. However they are frustrated by guideline development processes that are enormously time, skill and resource intensive. They feel strongly that "there's got to be a better way".
The fact that the great majority of the identified barriers were shared across settings may provide an opportunity to develop a more pragmatic way of developing guidelines that can be applied in many contexts.
doi:10.1186/1472-6963-9-235
PMCID: PMC2800111  PMID: 20003536
11.  Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005) 
Background
Despite the growing availability of clinical practice guidelines since the early 1990's, little is known about how guideline development and dissemination may have changed over time in Canada. This study compares Canadian guideline development, dissemination, and evaluation in two six year periods from 1994–1999 and 2000–2005.
Methods
Survey of guideline developers who submitted their clinical practice guidelines to the Canadian Medical Association Infobase (a Canadian guideline repository) between 1994 and 2005. Survey items included information about the developers, aspects of guideline development, and dissemination and evaluation activities.
Results
Surveys were sent to the developers of 2341 guidelines in the CMA Infobase over the 12 year period, 1664 surveys were returned (response rate 71%). Of these, 730 unique guidelines were released from 1994–1999, and 630 were released from 2000–2005. Compared to the earlier period, more recent guidelines were being produced in English only. There has been little change in the type of organizations developing guidelines with most developed by provincial and national organizations. In the recent period, developers were more likely to report using computerized search strategies (94% versus 88%), publishing the search strategy (42% versus 34%), reaching consensus using open discussion (95% versus 78%), and evaluating effectiveness of the dissemination strategies (12% versus 6%) and the impact of the CPGs on health outcomes (24% versus 5%). Recent guidelines were less likely to be based on literature reviews (94% versus 99.6%) and were disseminated using fewer strategies (mean 4.78 versus 4.12).
Conclusion
Given that guideline development processes have improved in some areas over the past 12 years yet not in others, ongoing monitoring of guideline quality is required. Guidelines produced more recently in Canada are less likely to be based on a review of the evidence and only about half discuss levels of evidence underlying recommendations. Guideline dissemination and implementation activities have actually decreased. Unfortunately, the potential positive impact on patient health outcomes will not be realized until the recommendations are adopted and acted upon.
doi:10.1186/1748-5908-4-49
PMCID: PMC2731072  PMID: 19656384
12.  Rational Prescribing in Primary Care (RaPP): A Cluster Randomized Trial of a Tailored Intervention 
PLoS Medicine  2006;3(6):e134.
Background
A gap exists between evidence and practice regarding the management of cardiovascular risk factors. This gap could be narrowed if systematically developed clinical practice guidelines were effectively implemented in clinical practice. We evaluated the effects of a tailored intervention to support the implementation of systematically developed guidelines for the use of antihypertensive and cholesterol-lowering drugs for the primary prevention of cardiovascular disease.
Methods and Findings
We conducted a cluster-randomized trial comparing a tailored intervention to passive dissemination of guidelines in 146 general practices in two geographical areas in Norway. Each practice was randomized to either the tailored intervention (70 practices; 257 physicians) or control group (69 practices; 244 physicians). Patients started on medication for hypertension or hypercholesterolemia during the study period and all patients already on treatment that consulted their physician during the trial were included. A multifaceted intervention was tailored to address identified barriers to change. Key components were an educational outreach visit with audit and feedback, and computerized reminders linked to the medical record system. Pharmacists conducted the visits. Outcomes were measured for all eligible patients seen in the participating practices during 1 y before and after the intervention. The main outcomes were the proportions of (1) first-time prescriptions for hypertension where thiazides were prescribed, (2) patients assessed for cardiovascular risk before prescribing antihypertensive or cholesterol-lowering drugs, and (3) patients treated for hypertension or hypercholesterolemia for 3 mo or more who had achieved recommended treatment goals.
The intervention led to an increase in adherence to guideline recommendations on choice of antihypertensive drug. Thiazides were prescribed to 17% of patients in the intervention group versus 11% in the control group (relative risk 1.94; 95% confidence interval 1.49–2.49, adjusted for baseline differences and clustering effect). Little or no differences were found for risk assessment prior to prescribing and for achievement of treatment goals.
Conclusions
Our tailored intervention had a significant impact on prescribing of antihypertensive drugs, but was ineffective in improving the quality of other aspects of managing hypertension and hypercholesterolemia in primary care.
Editors' Summary
Background.
An important issue in health care is “getting research into practice,” in other words, making sure that, when evidence from research has established the best way to treat a disease, doctors actually use that approach with their patients. In reality, there is often a gap between evidence and practice.
  An example concerns the treatment of people who have high blood pressure (hypertension) and/or high cholesterol. These are common conditions, and both increase the risk of having a heart attack or a stroke. Research has shown that the risks can be lowered if patients with these conditions are given drugs that lower blood pressure (antihypertensives) and drugs that lower cholesterol. There are many types of these drugs now available. In many countries, the health authorities want family doctors (general practitioners) to make better use of these drugs. They want doctors to prescribe them to everyone who would benefit, using the type of drugs found to be most effective. When there is a choice of drugs that are equally effective, they want doctors to use the cheapest type. (In the case of antihypertensives, an older type, known as thiazides, is very effective and also very cheap, but many doctors prefer to give their patients newer, more expensive alternatives.) Health authorities have issued guidelines to doctors that address these issues. However, it is not easy to change prescribing practices, and research in several countries has shown that issuing guidelines has only limited effects.
Why Was This Study Done?
The researchers wanted—in two parts of Norway—to compare the effects on prescribing practices of what they called the “passive dissemination of guidelines” with a more active approach, where the use of the guidelines was strongly promoted and encouraged.
What Did the Researchers Do and Find?
They worked with 146 general practices. In half of them the guidelines were actively promoted. The remaining were regarded as a control group; they were given the guidelines but no special efforts were made to encourage their use. It was decided at random which practices would be in which group; this approach is called a randomized controlled trial. The methods used to actively promote use of the guidelines included personal visits to the practices by pharmacists and use of a computerized reminder system. Information was then collected on the number of patients who, when first treated for hypertension, were prescribed a thiazide. Other information collected included whether patients had been properly assessed for their level of risk (for strokes and heart attacks) before antihypertensive or cholesterol-lowering drugs were given. In addition, the researchers recorded whether the recommended targets for improvement in blood pressure and cholesterol level had been reached.
Only 11% of those patients visiting the control group of practices who should have been prescribed thiazides, according to the guidelines, actually received them. Of those seen by doctors in the practices where the guidelines were actively promoted, 17% received thiazides. According to statistical analysis, the increase achieved by active promotion is significant. Little or no differences were found for risk assessment prior to prescribing and for achievement of treatment goals.
What Do These Findings Mean?
Even in the active promotion group, the great majority of patients (83%) were still not receiving treatment according to the guidelines. However, active promotion of guidelines is more effective than simply issuing the guidelines by themselves. The study also demonstrates that it is very hard to change prescribing practices. The efforts made here to encourage the doctors to change were considerable, and although the results were significant, they were still disappointing. Also disappointing is the fact that achievement of treatment goals was no better in the active-promotion group. These issues are discussed further in a Perspective about this study (DOI: 10.1371/journal.pmed.0030229).
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0030134.
• The Web site of the American Academy of Family Physicians has a page on heart disease
• The MedlinePlus Medical Encyclopedia's pages on heart diseases and vascular diseases
• Information from NHS Direct (UK National Health Service) about heart attack and stroke
• Another PLoS Medicine article has also addressed trends in thiazide prescribing
Passive dissemination of management guidelines for hypertension and hypercholesterolaemia was compared with active promotion. Active promotion led to significant improvement in antihypertensive prescribing but not other aspects of management.
doi:10.1371/journal.pmed.0030134
PMCID: PMC1472695  PMID: 16737346
13.  Guidelines for Guidelines: Are They Up to the Task? A Comparative Assessment of Clinical Practice Guideline Development Handbooks 
PLoS ONE  2012;7(11):e49864.
Objectives
We conducted a comparative review of clinical practice guideline development handbooks. We aimed to identify the main guideline development tasks, assign weights to the importance of each task using expert opinions and identify the handbooks that provided a comprehensive coverage of the tasks.
Methods
We systematically searched and included handbooks published (in English language) by national, international or professional bodies responsible for evidenced-based guideline development. We reviewed the handbooks to identify the main guideline development tasks and scored each handbook for each task from 0 (the handbook did not mention the task) to 2 (the task suitably addressed and explained), and calculated a weighted score for each handbook. The tasks included in over 75% of the handbooks were considered as ‘necessary’ tasks.
Result
Nineteen guideline development handbooks and twenty seven main tasks were identified. The guideline handbooks’ weighted scores ranged from 100 to 220. Four handbooks scored over 80% of the maximum possible score, developed by the National Institute for Health and Clinical Excellence, Swiss Centre for International Health, Scottish Intercollegiate Guidelines Network and World Health Organization. Necessary tasks were: selecting the guideline topic, determining the guideline scope, identifying relevant existing guidelines, involving the consumers, forming guideline development group,, developing clinical questions, systematic search for evidence, selecting relevant evidence, appraising identifies research evidence, making group decision, grading available evidence, creating recommendations, final stakeholder consultation, guideline implementation strategies, updating recommendations and correcting potential errors.
Discussion
Adequate details for evidence based development of guidelines were still lacking from many handbooks. The tasks relevant to ethical issues and piloting were missing in most handbooks. The findings help decision makers in identifying the necessary tasks for guideline development, provide an updated comparative list of guideline development handbooks, and provide a checklist to assess the comprehensiveness of guideline development processes.
doi:10.1371/journal.pone.0049864
PMCID: PMC3506587  PMID: 23189167
14.  The Limited Incorporation of Economic Analyses in Clinical Practice Guidelines 
BACKGROUND
Because there is increasing concern that economic data are not used in the clinical guideline development process, our objective was to evaluate the extent to which economic analyses are incorporated in guideline development.
METHODS
We searched medline and HealthSTAR databases to identify English-language clinical practice guidelines (1996–1999) and economic analyses (1990–1998). Additional guidelines were obtained from The National Guidelines Clearinghouse Internet site available at . Eligible guidelines met the Institute of Medicine definition and addressed a topic included in an economic analysis. Eligible economic analyses assessed interventions addressed in a guideline and predated the guideline by 1 or more years. Economic analyses were defined as incorporated in guideline development if 1) the economic analysis or the results were mentioned in the text or 2) listed as a reference. The quality of economic analyses was assessed using a structured scoring system.
RESULTS
Using guidelines as the unit of analysis, 9 of 35 (26%) incorporated at least 1 economic analysis of above-average quality in the text and 11 of 35 (31%) incorporated at least 1 in the references. Using economic analyses as the unit of analysis, 63 economic analyses of above-average quality had opportunities for incorporation in 198 instances across the 35 guidelines. Economic analyses were incorporated in the text in 13 of 198 instances (7%) and in the references in 18 of 198 instances (9%).
CONCLUSIONS
Rigorous economic analyses may be infrequently incorporated in the development of clinical practice guidelines. A systematic approach to guideline development should be used to ensure the consideration of economic analyses so that recommendations from guidelines may impact both the quality of care and the efficient allocation of resources.
doi:10.1046/j.1525-1497.2002.10522.x
PMCID: PMC1495022  PMID: 11929508
guidelines; econimic analyses; cost-effectiveness
15.  Prospective systematic review registration: perspective from the Guidelines International Network (G-I-N) 
Systematic Reviews  2012;1:3.
Clinical practice and public health guidelines are important tools for translating research findings into practice with the aim of assisting health practitioners as well as patients and consumers in health behavior and healthcare decision-making. Numerous programs for guideline development exist around the world, with growing international collaboration to improve their quality. One of the key features in developing trustworthy guidelines is that recommendations should be based on high-quality systematic reviews of the best available evidence. The review process used by guideline developers to identify and grade relevant evidence for developing recommendations should be systematic, transparent and unbiased. In this paper, we provide an overview of current international developments in the field of practice guidelines and methods to develop guidelines, with a specific focus on the role of systematic reviews. The Guidelines International Network (G-I-N) aims to stimulate collaboration between guideline developers and systematic reviewers to optimize the use of available evidence in guideline development and to increase efficiency in the guideline development process. Considering the significant benefit of systematic reviews for the guideline community, the G-I-N Board of Trustees supports the international prospective register of systematic reviews (PROSPERO) initiative. G-I-N also recently launched a Data Extraction Resource (GINDER) to present and share data extracted from individual studies in a standardized template. PROSPERO and GINDER are complementary tools to enhance collaboration between guideline developers and systematic reviewers to allow for alignment of activities and a reduction in duplication of effort.
doi:10.1186/2046-4053-1-3
PMCID: PMC3348674  PMID: 22587933
guidelines; reviews; systematic; methods
16.  The European Society of Human Reproduction and Embryology guideline for the diagnosis and treatment of endometriosis: an electronic guideline implementability appraisal 
Background
Clinical guidelines are intended to improve healthcare. However, even if guidelines are excellent, their implementation is not assured. In subfertility care, the European Society of Human Reproduction and Embryology (ESHRE) guidelines have been inventoried, and their methodological quality has been assessed. To improve the impact of the ESHRE guidelines and to improve European subfertility care, it is important to optimise the implementability of guidelines. We therefore investigated the implementation barriers of the ESHRE guideline with the best methodological quality and evaluated the used instrument for usability and feasibility.
Methods
We reviewed the ESHRE guideline for the diagnosis and treatment of endometriosis to assess its implementability. We used an electronic version of the guideline implementability appraisal (eGLIA) instrument. This eGLIA tool consists of 31 questions grouped into 10 dimensions. Seven items address the guideline as a whole, and 24 items assess the individual recommendations in the guideline. The eGLIA instrument identifies factors that influence the implementability of the guideline recommendations. These factors can be divided into facilitators that promote implementation and barriers that oppose implementation. A panel of 10 experts from three European countries appraised all 36 recommendations of the guideline. They discussed discrepancies in a teleconference and completed a questionnaire to evaluate the ease of use and overall utility of the eGLIA instrument.
Results
Two of the 36 guideline recommendations were straightforward to implement. Five recommendations were considered simply statements because they contained no actions. The remaining 29 recommendations were implementable with some adjustments. We found facilitators of the guideline implementability in the quality of decidability, presentation and formatting, apparent validity, and novelty or innovation of the recommendations. Vaguely defined actions, lack of facilities, immeasurable outcomes, and inflexibility within the recommendations formed barriers to implementation. The eGLIA instrument was generally useful and easy to use. However, assessment with the eGLIA instrument is very time-consuming.
Conclusions
The ESHRE guideline for the diagnosis and treatment of endometriosis could be improved to facilitate its implementation in daily practice. The eGLIA instrument is a helpful tool for identifying obstacles to implementation of a guideline. However, we recommend a concise version of this instrument.
doi:10.1186/1748-5908-6-7
PMCID: PMC3034686  PMID: 21247418
17.  Guidelines on Chemotherapy in Advanced Stage Gynecological Malignancies: An Evaluation of 224 Professional Societies and Organizations 
PLoS ONE  2011;6(5):e20106.
Background
Clinical practice guidelines are important for guiding practice, but it is unclear if they are commensurate with the available evidence.
Methods
We examined guidelines produced by cancer and gynecological societies and organizations and evaluated their coverage of and stance towards chemotherapy for advanced stage disease among 4 gynecological malignancies (breast, ovarian, cervical, endometrial cancer) where the evidence for the use of chemotherapy is very different (substantial and conclusive for breast and ovarian cancer, limited and suggesting no major benefit for cervical and endometrial cancer). Eligible societies and organizations were identified through systematic internet searches (last update June 2009). Pertinent websites were scrutinized for presence of clinical practice guidelines, and relative guidelines were analyzed.
Results
Among 224 identified eligible societies and organizations, 69 (31%) provided any sort of guidelines, while recommendations for chemotherapy on advanced stage gynecological malignancies were available in 20 of them. Only 14 had developed their own guideline, and only 5 had developed guidelines for all 4 malignancies. Use of levels of evidence and grades of recommendations, and aspects of the production, implementation, and timeliness of the guidelines did not differ significantly across malignancies. Guidelines on breast and ovarian cancer utilized significantly more randomized trials and meta-analyses. Guidelines differed across malignancies on their coverage of disease-free survival (p = 0.033), response rates (p = 0.024), symptoms relief (p = 0.005), quality of life (p = 0.001) and toxicity (p = 0.039), with breast and ovarian cancer guidelines typically covering more frequently these outcomes. All guidelines explicitly or implicitly endorsed the use of chemotherapy.
Conclusions
Clinical practice guidelines are provided by the minority of professional societies and organizations. Available guidelines tend to recommend chemotherapy even for diseases where the effect of chemotherapy is controversial and recommendations are based on scant evidence.
doi:10.1371/journal.pone.0020106
PMCID: PMC3096663  PMID: 21611154
18.  Comparison of international guideline programs to evaluate and update the Dutch program for clinical guideline development in physical therapy 
Background
Clinical guidelines are considered important instruments to improve quality in health care. Since 1998 the Royal Dutch Society for Physical Therapy (KNGF) produced evidence-based clinical guidelines, based on a standardized program. New developments in the field of guideline research raised the need to evaluate and update the KNGF guideline program.
Purpose of this study is to compare different guideline development programs and review the KNGF guideline program for physical therapy in the Netherlands, in order to update the program.
Method
Six international guideline development programs were selected, and the 23 criteria of the AGREE Instrument were used to evaluate the guideline programs. Information about the programs was retrieved from published handbooks of the organizations. Also, the Dutch program for guideline development in physical therapy was evaluated using the AGREE criteria. Further comparison the six guideline programs was carried out using the following elements of the guideline development processes: Structure and organization; Preparation and initiation; Development; Validation; Dissemination and implementation; Evaluation and update.
Results
Compliance with the AGREE criteria of the guideline programs was high. Four programs addressed 22 AGREE criteria, and two programs addressed 20 AGREE criteria. The previous Dutch program for guideline development in physical therapy lacked in compliance with the AGREE criteria, meeting only 13 criteria.
Further comparison showed that all guideline programs perform systematic literature searches to identify the available evidence. Recommendations are formulated and graded, based on evidence and other relevant factors. It is not clear how decisions in the development process are made. In particular, the process of translating evidence into practice recommendations can be improved.
Conclusion
As a result of international developments and consensus, the described processes for developing clinical practice guidelines have much in common. The AGREE criteria are common basis for the development of guidelines, although it is not clear how final decisions are made. Detailed comparison of the different guideline programs was used for updating the Dutch program. As a result the updated KNGF program complied with 22 AGREE criteria. International discussion is continuing and will be used for further improvement of the program.
doi:10.1186/1472-6963-7-191
PMCID: PMC2228296  PMID: 18036215
19.  Application of GRADE: Making evidence-based recommendations about diagnostic tests in clinical practice guidelines 
Background
Accurate diagnosis is a fundamental aspect of appropriate healthcare. However, clinicians need guidance when implementing diagnostic tests given the number of tests available and resource constraints in healthcare. Practitioners of health often feel compelled to implement recommendations in guidelines, including recommendations about the use of diagnostic tests. However, the understanding about diagnostic tests by guideline panels and the methodology for developing recommendations is far from completely explored. Therefore, we evaluated the factors that guideline developers and users need to consider for the development of implementable recommendations about diagnostic tests.
Methods
Using a critical analysis of the process, we present the results of a case study using the Grading of Recommendations Applicability, Development and Evaluation (GRADE) approach to develop a clinical practice guideline for the diagnosis of Cow Milk Allergy with the World Allergy Organization.
Results
To ensure that guideline panels can develop informed recommendations about diagnostic tests, it appears that more emphasis needs to be placed on group processes, including question formulation, defining patient-important outcomes for diagnostic tests, and summarizing evidence. Explicit consideration of concepts of diagnosis from evidence-based medicine, such as pre-test probability and treatment threshold, is required to facilitate the work of a guideline panel and to formulate implementable recommendations.
Discussion
This case study provides useful guidance for guideline developers and clinicians about what they ought to demand from clinical practice guidelines to facilitate implementation and strengthen confidence in recommendations about diagnostic tests. Applying a structured framework like the GRADE approach with its requirement for transparency in the description of the evidence and factors that influence recommendations facilitates laying out the process and decision factors that are required for the development, interpretation, and implementation of recommendations about diagnostic tests.
doi:10.1186/1748-5908-6-62
PMCID: PMC3126717  PMID: 21663655
20.  Heterogeneity in cancer guidelines: should we eradicate or tolerate? 
Annals of Oncology  2008;19(12):2067-2078.
Background: Heterogeneity in aspects of development, structure and context of oncology guidelines was not evaluated. We analysed and critically examined its implications.
Materials and methods: Nine cancer clinical practice guidelines were selected on the basis of popularity among oncologists. The relevant Web sites and publications on three tumours were examined and characteristics grouped in the data domains: producing organisation, methodology, guideline structure and content, implementation and evaluation and scientific agreement.
Results: ASCO, ESMO, NICE, SIGN, START, NHMRC, NCI, NCCN and CCO guidelines were examined. Development was initiated by stakeholders or authorised bodies, run by task forces with varying degrees of multidisciplinarity, with rare endorsement of external guidelines. Recommendation formulation was on the basis of evidence, shaped via interactive processes of expert review and public consultation-based modifications. Guidelines varied in comprehensiveness per tumour type, number, size, format, grading of evidence, update and legal issues. Orientation for clinic use or as reference document, end-users and binding or elective nature also varied. Standard dissemination strategies were used, though evaluation of adoption and of impact on health outcomes was implemented with considerable heterogeneity.
Conclusions: Heterogeneity in development, structure, user and end points of guidelines is evident, though necessary in order to meet divergent demands. Crucial for their effectiveness are adherence to methodological standards, a clear definition of what the guideline intends to do for whom and a systematic evaluation of their impact on health care.
doi:10.1093/annonc/mdn418
PMCID: PMC2733109  PMID: 18662954
cancer; clinical practice; guidelines; oncology
21.  Scaling Up the 2010 World Health Organization HIV Treatment Guidelines in Resource-Limited Settings: A Model-Based Analysis 
PLoS Medicine  2010;7(12):e1000382.
Rochelle Walensky and colleagues use a model-based analysis to examine which of the 2010 WHO antiretroviral therapy guidelines should be implemented first in resource-limited settings by ranking them according to survival, cost-effectiveness, and equity.
Background
The new 2010 World Health Organization (WHO) HIV treatment guidelines recommend earlier antiretroviral therapy (ART) initiation (CD4<350 cells/µl instead of CD4<200 cells/µl), multiple sequential ART regimens, and replacement of first-line stavudine with tenofovir. This paper considers what to do first in resource-limited settings where immediate implementation of all of the WHO recommendations is not feasible.
Methods and Findings
We use a mathematical model and local input data to project clinical and economic outcomes in a South African HIV-infected cohort (mean age = 32.8 y, mean CD4 = 375/µl). For the reference strategy, we assume that all patients initiate stavudine-based ART with WHO stage III/IV disease and receive one line of ART (stavudine/WHO/one-line). We rank—in survival, cost-effectiveness, and equity terms—all 12 possible combinations of the following: (1) stavudine replacement with tenofovir, (2) ART initiation (by WHO stage, CD4<200 cells/µl, or CD4<350 cells/µl), and (3) one or two regimens, or lines, of available ART. Projected life expectancy for the reference strategy is 99.0 mo. Considering each of the guideline components separately, 5-y survival is maximized with ART initiation at CD4<350 cells/µl (stavudine/<350/µl/one-line, 87% survival) compared with stavudine/WHO/two-lines (66%) and tenofovir/WHO/one-line (66%). The greatest life expectancies are achieved via the following stepwise programmatic additions: stavudine/<350/µl/one-line (124.3 mo), stavudine/<350/µl/two-lines (177.6 mo), and tenofovir/<350/µl/two-lines (193.6 mo). Three program combinations are economically efficient: stavudine/<350/µl/one-line (cost-effectiveness ratio, US$610/years of life saved [YLS]), tenofovir/<350/µl/one-line (US$1,140/YLS), and tenofovir/<350/µl/two-lines (US$2,370/YLS).
Conclusions
In settings where immediate implementation of all of the new WHO treatment guidelines is not feasible, ART initiation at CD4<350 cells/µl provides the greatest short- and long-term survival advantage and is highly cost-effective.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Since 1981, acquired immunodeficiency syndrome (AIDS) has killed more than 25 million people, and about 33 million people (30 million of them in low- and middle-income countries) are now infected with the human immunodeficiency virus (HIV), which causes AIDS. HIV destroys immune system cells (including CD4 cells, a type of lymphocyte), leaving infected individuals susceptible to other infections (so-called opportunistic infections). Early in the AIDS epidemic, most people with HIV died within 10 years of infection. Then, in 1996, highly active antiretroviral therapy (ART)—a combination of several powerful antiretroviral drugs—was developed. Now, in resource-rich countries, clinicians care for people with HIV by prescribing ART regimens tailored to each individual's needs. They also regularly measure the amount of virus in their patients' blood, test for antiretroviral-resistant viruses, and monitor the health of their patients' immune systems through regular CD4 cell counts. As a result, the life expectancy of patients with HIV in developed countries has dramatically improved.
Why Was This Study Done?
Initially, resource-limited countries could not afford to provide ART for their populations, and the life expectancy of HIV-positive people remained low. Now, through the concerted efforts of governments, the World Health Organization (WHO), and other international agencies, more than a third of the people in low- and middle-income countries who need ART are receiving it. However, many without access are still in need of ART, and ART programs in developing countries follow a public-health approach rather than an individualized approach. That is, drug regimens, clinical decision-making, and disease monitoring are all standardized and follow recommendations in the 2006 WHO ART guidelines. This year (2010), these guidelines were revised. The guidelines now recommend the following: earlier ART initiation—when the CD4 count falls below 350/µl of blood, instead of below 200/µl as in the 2006 guidelines; the provision of sequential ART regimens instead of a single regimen; and the replacement of the antiretroviral drug stavudine with tenofovir, a less toxic but more expensive drug, in first-line ART regimens. However, many resource-limited countries are still struggling to implement the 2006 guidelines, so which of these new recommendations should be prioritized? Here, the researchers use a mathematical model to address this question.
What Did the Researchers Do and Find?
The Cost Effectiveness of AIDS Complications (CEPAC)–International model simulates the natural history and treatment of HIV disease. The researchers entered South African clinical and cost data for HIV treatment into this model and then used it to project survival and costs in a hypothetical group of South African HIV-positive patients under alternative guideline prioritization scenarios. The reference strategy for the simulations (denoted as “stavudine/WHO/one-line”) assumed that patients (with a mean CD4 count of 375/µl) began a single stavudine-based ART regimen when they developed WHO stage III/IV HIV disease (i.e., when patients develop multiple opportunistic infections such as tuberculosis and pneumonia). When the new guideline recommendations were considered separately, ART initiation at CD4<350/µl (stavudine/<350/µl/one-line) maximized five-year survival. Stepwise adjustment from the reference strategy (which had a life expectancy 99.0 months) through strategies of stavudine/<350/µl/one-line (a projected life expectancy of 124.3 months), stavudine/<350/µl/two-lines (177.6 months), and tenofovir/<350/µl/two-lines (193.6 months) produced the greatest improvements in life expectancy. Finally, strategies of stavudine/<350/µl/one-line, tenofovir/<350/µl/one-line, and tenofovir/<350µl/two-lines produced incremental cost-effectiveness ratios of US$610, US$1,140, and US$2,370 per year of life saved, respectively.
What Do These Findings Mean?
As with all mathematical models, the accuracy of these findings are dependent on the assumptions included in the model and on the data populating it. Nevertheless, these findings suggest that, where resources are limited and immediate implementation of all the new WHO recommendations is impossible, ART initiation at a CD4 count of less than 350/µl would provide the greatest survival advantage and would be very cost-effective. In countries that are already initiating ART at this threshold and that have access to CD4 monitoring, a switch from stavudine to tenofovir would further increase survival and would also be cost-effective. Finally, although access to second-line ART regimen would provide more clinical benefits than access to tenofovir, the cost of this change in strategy would be substantially greater. Importantly, these findings should help to avoid the complete dismissal of the revised WHO guidelines on the basis of cost and should help policy makers adjust their ART program strategies to maximize their clinical benefits and cost effectiveness.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000382.
Information is available from the US National Institute of Allergy and Infectious Diseases on HIV infection and AIDS
HIV InSite has comprehensive information on all aspects of HIV/AIDS
Information is available from Avert, an international AIDS charity, on many aspects of HIV/AIDS, including information on HIV/AIDS in South Africa and on HIV/AIDS treatment and care (in English and Spanish)
WHO provides information about universal access to AIDS treatment (in English, French, and Spanish); its 2010 ART guidelines can be downloaded
More information on the CEPAC model is available
doi:10.1371/journal.pmed.1000382
PMCID: PMC3014084  PMID: 21209794
22.  Maternal and perinatal guideline development in hospitals in South East Asia: the experience of the SEA-ORCHID project 
Background
Clinical practice guidelines (CPGs) are commonly used to support practitioners to improve practice. However many studies have raised concerns about guideline quality. The reasons why guidelines are not developed following the established development methods are not clear.
The SEA-ORCHID project aims to increase the generation and use of locally relevant research and improve clinical practice in maternal and perinatal care in four countries in South East Asia. Baseline data highlighted that development of evidence-based CPGs according to recommended processes was very rare in the SEA-ORCHID hospitals. The project investigators suggested that there were aspects of the recommended development process that made it very difficult in the participating hospitals.
We therefore aimed to explore the experience of guideline development and particularly the enablers of and barriers to developing evidence-based guidelines in the nine hospitals in South East Asia participating in the SEA-ORCHID project, so as to better understand how evidence-based guideline development could be facilitated in these settings.
Methods
Semi-structured, face-to-face interviews were undertaken with senior and junior healthcare providers (nurses, midwives, doctors) from the maternal and neonatal services at each of the nine participating hospitals. Interviews were audio-recorded, transcribed and a thematic analysis undertaken.
Results
Seventy-five individual, 25 pair and eleven group interviews were conducted. Participants clearly valued evidence-based guidelines. However they also identified several major barriers to guideline development including time, lack of awareness of process, difficulties searching for evidence and arranging guideline development group meetings, issues with achieving multi-disciplinarity and consumer involvement. They also highlighted the central importance of keeping guidelines up-to-date.
Conclusion
Healthcare providers in the SEA-ORCHID hospitals face a series of barriers to developing evidence-based guidelines. At present, in many hospitals, several of these barriers are insurmountable, and as a result, rigorous, evidence-based guidelines are not being developed. Given the acknowledged benefits of evidence-based guidelines, perhaps a new approach to supporting their development in these contexts is needed.
doi:10.1186/1478-4505-7-10
PMCID: PMC2683833  PMID: 19422716
23.  Formal consensus: the development of a national clinical guideline 
Quality in Health Care : QHC  2001;10(4):238-244.
Background—There is currently a political enthusiasm for the development and use of clinical guidelines despite, paradoxically, there being relatively few healthcare issues that have a sound research evidence base. As decisions have to be made even where there is an undetermined evidence base and that limiting recommendations to where evidence exists may reduce the scope of guidelines, thus limiting their value to practitioners, guideline developers have to rely on various different sources of evidence and adapt their methods accordingly. This paper outlines a method for guideline development which incorporates a consensus process devised to tackle the challenges of a variable research evidence base for the development of a national clinical guideline on risk assessment and prevention of pressure ulcers.
Method—To inform the recommendations of the guideline a formal consensus process based on a nominal group technique was used to incorporate three strands of evidence: research, clinical expertise, and patient experience.
Results—The recommendations for this guideline were derived directly from the statements agreed in the formal consensus process and from key evidence-based findings from the systematic reviews. The existing format of the statements that participants had rated allowed a straightforward revision to "active" recommendations, thus reducing further risk of subjectivity entering into the process.
Conclusions—The method outlined proved to be a practical and systematic way of integrating a number of different evidence sources. The resultant guideline is a mixture of research based and consensus based recommendations. Given the lack of available guidance on how to mix research with expert opinion and patient experiences, the method used for the development of this guideline has been outlined so that other guideline developers may use, adapt, and test it further.
Key Words: guidelines; guideline development; formal consensus process; nominal group technique; pressure ulcers
doi:10.1136/qhc.0100238..
PMCID: PMC1743460  PMID: 11743153
24.  The Systematic Guideline Review: Method, rationale, and test on chronic heart failure 
Background
Evidence-based guidelines have the potential to improve healthcare. However, their de-novo-development requires substantial resources – especially for complex conditions, and adaptation may be biased by contextually influenced recommendations in source guidelines. In this paper we describe a new approach to guideline development – the systematic guideline review method (SGR), and its application in the development of an evidence-based guideline for family physicians on chronic heart failure (CHF).
Methods
A systematic search for guidelines was carried out. Evidence-based guidelines on CHF management in adults in ambulatory care published in English or German between the years 2000 and 2004 were included. Guidelines on acute or right heart failure were excluded. Eligibility was assessed by two reviewers, methodological quality of selected guidelines was appraised using the AGREE instrument, and a framework of relevant clinical questions for diagnostics and treatment was derived. Data were extracted into evidence tables, systematically compared by means of a consistency analysis and synthesized in a preliminary draft. Most relevant primary sources were re-assessed to verify the cited evidence. Evidence and recommendations were summarized in a draft guideline.
Results
Of 16 included guidelines five were of good quality. A total of 35 recommendations were systematically compared: 25/35 were consistent, 9/35 inconsistent, and 1/35 un-rateable (derived from a single guideline). Of the 25 consistencies, 14 were based on consensus, seven on evidence and four differed in grading. Major inconsistencies were found in 3/9 of the inconsistent recommendations. We re-evaluated the evidence for 17 recommendations (evidence-based, differing evidence levels and minor inconsistencies) – the majority was congruent. Incongruity was found where the stated evidence could not be verified in the cited primary sources, or where the evaluation in the source guidelines focused on treatment benefits and underestimated the risks. The draft guideline was completed in 8.5 man-months. The main limitation to this study was the lack of a second reviewer.
Conclusion
The systematic guideline review including framework development, consistency analysis and validation is an effective, valid, and resource saving-approach to the development of evidence-based guidelines.
doi:10.1186/1472-6963-9-74
PMCID: PMC2698839  PMID: 19426504
25.  A critical review of guidelines for low back pain treatment 
European Spine Journal  2005;15(5):543-553.
Main problem: Little is known about the methodological quality of guidelines for low back pain treatment. We evaluated the methods used by the developers according to established standards. Methods: PubMed, guideline databases, and the World Wide Web were used to identify guidelines. Seventeen guidelines met the inclusion criteria: interventions for low back pain stated, recommendations based on or explicitly linked to evidence, and English version available. Guidelines were evaluated independently by two appraisers using a practical tool for this purpose, Users’ Guides to the Medical Literature, and the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument. Results: Thirteen guidelines (76%) specified the most important therapies applied, but only nine (53%) included a complete description of the target population. Explicit processes to identify, select, and combine evidence were described in only six guidelines (35%). Few guidelines (3; 18%) explicitly considered all main outcomes when formulating therapeutic recommendations, and none contained a process to determine the relative value of different outcomes. Methodological criteria for grading the strength of the recommendations varied, and were often insufficiently specified. None of the guidelines assessed the impact of uncertainty associated with the evidence and values used. According to AGREE the quality score was highest for the scope and purpose, and clarity and presentation domains, and lowest for editorial independence and applicability. With regard to the recommendations, there was consensus for some of the interventions for acute pain (analgesics and NSAIDs, maintaining physical activity, and avoiding excessive bed rest), but explicit recommendations were lacking or ambiguous for 41% of the interventions. Most of the guidelines did not contemplate specific recommendations for chronic pain. Conclusions: A small number of the available guidelines for low back pain treatment achieved acceptable results for specific quality criteria. In general, the methods to develop the guidelines’ therapeutic recommendations need to be more rigorous, more explicit and better explained. In addition, greater importance should be placed on the recommendations for chronic pain.
doi:10.1007/s00586-005-1027-y
PMCID: PMC3489347  PMID: 16217664
Low back pain; Guidelines; Practice guidelines; Quality assurance; Health care

Results 1-25 (1407210)