Search tips
Search criteria

Results 1-25 (1309628)

Clipboard (0)

Related Articles

1.  Effectiveness guidance document (EGD) for Chinese medicine trials: a consensus document 
Trials  2014;15:169.
There is a need for more Comparative Effectiveness Research (CER) on Chinese medicine (CM) to inform clinical and policy decision-making. This document aims to provide consensus advice for the design of CER trials on CM for researchers. It broadly aims to ensure more adequate design and optimal use of resources in generating evidence for CM to inform stakeholder decision-making.
The Effectiveness Guidance Document (EGD) development was based on multiple consensus procedures (survey, written Delphi rounds, interactive consensus workshop, international expert review). To balance aspects of internal and external validity, multiple stakeholders, including patients, clinicians, researchers and payers were involved in creating this document.
Recommendations were developed for “using available data” and “future clinical studies”. The recommendations for future trials focus on randomized trials and cover the following areas: designing CER studies, treatments, expertise and setting, outcomes, study design and statistical analyses, economic evaluation, and publication.
The present EGD provides the first systematic methodological guidance for future CER trials on CM and can be applied to single or multi-component treatments. While CONSORT statements provide guidelines for reporting studies, EGDs provide recommendations for the design of future studies and can contribute to a more strategic use of limited research resources, as well as greater consistency in trial design.
PMCID: PMC4045891  PMID: 24885146
Comparative effectiveness research; Effectiveness guidance document; Chinese medicine research
2.  Building a Strategic Framework for Comparative Effectiveness Research in Complementary and Integrative Medicine 
The increasing burden of chronic diseases presents not only challenges to the knowledge and expertise of the professional medical community, but also highlights the need to improve the quality and relevance of clinical research in this domain. Many patients now turn to complementary and integrative medicine (CIM) to treat their chronic illnesses; however, there is very little evidence to guide their decision-making in usual care. The following research recommendations were derived from a CIM Stakeholder Symposium on Comparative Effectiveness Research (CER): (1) CER studies should be made a priority in this field; (2) stakeholders should be engaged at every stage of the research; (3) CER study designs should highlight effectiveness over efficacy; (4) research questions should be well defined to enable the selection of an appropriate CER study design; (5) the CIM community should cultivate widely shared understandings, discourse, tools, and technologies to support the use and validity of CER methods; (6) Effectiveness Guidance Documents on methodological standards should be developed to shape future CER studies. CER is an emerging field and its development and impact must be reflected in future research strategies within CIM. This stakeholder symposium was a first step in providing systematic guidance for future CER in this field.
PMCID: PMC3544532  PMID: 23346206
3.  Prioritizing comparative effectiveness research for cancer diagnostics using a regional stakeholder approach 
This paper describes our process to engage regional stakeholders for prioritizing comparative effectiveness research (CER) in cancer diagnostics. We also describe a novel methodology for incorporating stakeholder data and input to inform the objectives of selected CER studies.
Materials & methods
As an integrated component to establishing the infrastructure for community-based CER on diagnostic technologies, we have assembled a regional stakeholder group composed of local payers, clinicians and state healthcare representatives to not only identify and prioritize CER topics most important to the western Washington State region, but also to inform the study design of selected research areas. A landscape analysis process combining literature searches, expert consultations and stakeholder discussions was used to identify possible CER topics in cancer diagnostics. Stakeholders prioritized the top topics using a modified Delphi/group-nominal method and a standardized evaluation criteria framework to determine a final selected CER study area. Implementation of the selected study was immediate due to a unique American Recovery and Reinvestment Act funding structure involving the same researchers and stakeholders in both the prioritization and execution phases of the project. Stakeholder engagement was enhanced after study selection via a rapid analysis of a subset of payers’ internal claims, coordinated by the research team, to obtain summary data of imaging patterns of use. Results of this preliminary analysis, which we termed an ‘internal analysis,’ were used to determine with the stakeholders the most important and feasible study objectives.
Stakeholders identified PET and MRI in cancers including breast, lung, lymphoma and colorectal as top priorities. In an internal analysis of breast cancer imaging, summary data from three payers demonstrated utilization rates of advanced imaging increased between 2002 and 2009 in the study population, with a great deal of variability in use between different health plans. Assessing whether breast MRI affects treatment decisions was the top breast cancer study objective selected by the stakeholders. There were other high-priority research areas including whether MRI use improved survival that were not deemed feasible with the length of follow-up time following MRI adoption.
Continuous stakeholder engagement greatly enhanced their enthusiasm for the project. We believe CER implementation will be more successful when undertaken by regional stakeholders.
PMCID: PMC3480224  PMID: 23105966
breast cancer; cancer imaging; comparative effectiveness research; research prioritization; stakeholder involvement
4.  Protocol for CONSORT-SPI: an extension for social and psychological interventions 
Determining the effectiveness of social and psychological interventions is important for improving individual and population health. Such interventions are complex and, where possible, are best evaluated by randomised controlled trials (RCTs). The use of research findings in policy and practice decision making is hindered by poor reporting of RCTs. Poor reporting limits the ability to replicate interventions, synthesise evidence in systematic reviews, and utilise findings for evidence-based policy and practice. The lack of guidance for reporting the specific methodological features of complex intervention RCTs contributes to poor reporting. We aim to develop an extension of the Consolidated Standards of Reporting Trials Statement for Social and Psychological Interventions (CONSORT-SPI).
This research project will be conducted in five phases. The first phase was the project launch, which consisted of the establishment of a Project Executive and International Advisory Group, and recruitment of journal editors and the CONSORT Group. The second phase involves a Delphi process that will generate a list of possible items to include in the CONSORT Extension. Next, there will be a formal consensus meeting to select the reporting items to add to, or modify for, the CONSORT-SPI Extension. Fourth, guideline documents will be written, including an explanation and elaboration (E&E) document that will provide detailed advice for each item and examples of good reporting. The final phase will comprise guideline dissemination, with simultaneous publication and endorsement of the guideline in multiple journals, endorsement by funding agencies, presentations at conferences and other meetings, and a dedicated website that will facilitate feedback about the guideline.
As demonstrated by previous CONSORT guidelines, the development of an evidence-based reporting guideline for social and psychological intervention RCTs should improve the accuracy, comprehensiveness, and transparency of study reports. This, in turn, promises to improve the critical appraisal of research and its use in policy and practice decision making. We invite readers to participate in the project by visiting our website (
PMCID: PMC3766255  PMID: 24004579
CONSORT-SPI; Randomised controlled trial; RCT; Reporting guidelines; Complex interventions
5.  Use of Expert Panels to Define the Reference Standard in Diagnostic Research: A Systematic Review of Published Methods and Reporting 
PLoS Medicine  2013;10(10):e1001531.
Loes C. M. Bertens and colleagues survey the published diagnostic research literature for use of expert panels to define the reference standard, characterize components and missing information, and recommend elements that should be reported in diagnostic studies.
Please see later in the article for the Editors' Summary
In diagnostic studies, a single and error-free test that can be used as the reference (gold) standard often does not exist. One solution is the use of panel diagnosis, i.e., a group of experts who assess the results from multiple tests to reach a final diagnosis in each patient. Although panel diagnosis, also known as consensus or expert diagnosis, is frequently used as the reference standard, guidance on preferred methodology is lacking. The aim of this study is to provide an overview of methods used in panel diagnoses and to provide initial guidance on the use and reporting of panel diagnosis as reference standard.
Methods and Findings
PubMed was systematically searched for diagnostic studies applying a panel diagnosis as reference standard published up to May 31, 2012. We included diagnostic studies in which the final diagnosis was made by two or more persons based on results from multiple tests. General study characteristics and details of panel methodology were extracted. Eighty-one studies were included, of which most reported on psychiatry (37%) and cardiovascular (21%) diseases. Data extraction was hampered by incomplete reporting; one or more pieces of critical information about panel reference standard methodology was missing in 83% of studies. In most studies (75%), the panel consisted of three or fewer members. Panel members were blinded to the results of the index test results in 31% of studies. Reproducibility of the decision process was assessed in 17 (21%) studies. Reported details on panel constitution, information for diagnosis and methods of decision making varied considerably between studies.
Methods of panel diagnosis varied substantially across studies and many aspects of the procedure were either unclear or not reported. On the basis of our review, we identified areas for improvement and developed a checklist and flow chart for initial guidance for researchers conducting and reporting of studies involving panel diagnosis.
Please see later in the article for the Editors' Summary
Editors' Summary
Before any disease or condition can be treated, a correct diagnosis of the condition has to be made. Faced with a patient with medical problems and no diagnosis, a doctor will ask the patient about their symptoms and medical history and generally will examine the patient. On the basis of this questioning and examination, the clinician will form an initial impression of the possible conditions the patient may have, usually with a most likely diagnosis in mind. To support or reject the most likely diagnosis and to exclude the other possible diagnoses, the clinician will then order a series of tests and diagnostic procedures. These may include laboratory tests (such as the measurement of blood sugar levels), imaging procedures (such as an MRI scan), or functional tests (such as spirometry, which tests lung function). Finally, the clinician will use all the data s/he has collected to reach a firm diagnosis and will recommend a program of treatment or observation for the patient.
Why Was This Study Done?
Researchers are continually looking for new, improved diagnostic tests and multivariable diagnostic models—combinations of tests and characteristics that point to a diagnosis. Diagnostic research, which assesses the accuracy of new tests and models, requires that each patient involved in a diagnostic study has a final correct diagnosis. Unfortunately, for most conditions, there is no single, error-free test that can be used as the reference (gold) standard for diagnosis. If an imperfect reference standard is used, errors in the final disease classification may bias the results of the diagnostic study and may lead to a new test being adopted that is actually less accurate than existing tests. One widely used solution to the lack of a reference standard is “panel diagnosis” in which two or more experts assess the results from multiple tests to reach a final diagnosis for each patient in a diagnostic study. However, there is currently no formal guidance available on the conduct and reporting of panel diagnosis. Here, the researchers undertake a systematic review (a study that uses predefined criteria to identify research on a given topic) to provide an overview of the methodology and reporting of panel diagnosis.
What Did the Researchers Do and Find?
The researchers identified 81 published diagnostic studies that used panel diagnosis as a reference standard. 37% of these studies reported on psychiatric diseases, 21% reported on cardiovascular diseases, and 12% reported on respiratory diseases. Most of the studies (64%) were designed to assess the accuracy of one or more diagnostic test. Notably, one or more critical piece of information on methodology was missing in 83% of the studies. Specifically, information on the constitution of the panel was missing in a quarter of the studies and information on the decision-making process (whether, for example, a diagnosis was reached by discussion among panel members or by combining individual panel member's assessments) was incomplete in more than two-thirds of the studies. In three-quarters of the studies for which information was available, the panel consisted of only two or three members; different fields of expertise were represented in the panels in nearly two-thirds of the studies. In a third of the studies for which information was available, panel members made their diagnoses without access to the results of the test being assessed. Finally, the reproducibility of the decision-making process was assessed in a fifth of the studies.
What Do These Findings Mean?
These findings indicate that the methodology of panel diagnosis varies substantially among diagnostic studies and that reporting of this methodology is often unclear or absent. Both the methodology and reporting of panel diagnosis could, therefore, be improved substantially. Based on their findings, the researchers provide a checklist and flow chart to help guide the conduct and reporting of studies involving panel diagnosis. For example, they suggest that, when designing a study that uses panel diagnosis as the reference standard, the number and background of panel members should be considered, and they provide a list of options that should be considered when planning the decision-making process. Although more research into each of the options identified by the researchers is needed, their recommendations provide a starting point for the development of formal guidelines on the methodology and reporting of panel diagnosis for use as a reference standard in diagnostic research.
Additional Information
Please access these Web sites via the online version of this summary at
Wikipedia has a page on medical diagnosis (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The Equator Network is an international initiative that seeks to improve the reliability and value of medical research literature by promoting transparent and accurate reporting of research studies; its website includes information on a wide range of reporting guidelines, including the STAndards for the Reporting of Diagnostic accuracy studies (STARD), an initiative that aims to improve the accuracy and completeness of reporting of studies of diagnostic accuracy
PMCID: PMC3797139  PMID: 24143138
6.  The Status and Future of Acupuncture Clinical Research 
On November 8–9, 2007, the Society for Acupuncture Research (SAR) hosted an international conference to mark the tenth anniversary of the landmark National Institutes of Health Consensus Development Conference on Acupuncture. More than 300 acupuncture researchers, practitioners, students, funding agency personnel, and health policy analysts from 20 countries attended the SAR meeting held at the University of Maryland School of Medicine, Baltimore, MD. This paper summarizes important invited lectures in the area of clinical research. Specifically, included are: a review of the recently conducted German trials and observational studies on low-back pain (LBP), gonarthrosis, migraine, and tension-type headache (the Acupuncture Research Trials and the German Acupuncture Trials, plus observational studies); a systematic review of acupuncture treatment for knee osteoarthritis (OA); and an overview of acupuncture trials in neurologic conditions, LBP, women's health, psychiatric disorders, and functional bowel disorders. A summary of the use of acupuncture in cancer care is also provided. Researchers involved in the German trials concluded that acupuncture is effective for treating chronic pain, but the correct selection of acupuncture points seems to play a limited role; no conclusions could be drawn about the placebo aspect of acupuncture, due to the design of the studies. Overall, when compared to sham, acupuncture did not show a benefit in treating knee OA or LBP, but acupuncture was better than a wait-list control and standard of care, respectively. In women's health, acupuncture has been found to be beneficial for patients with premenstrual syndrome, dysmenorrhea, several pregnancy-related conditions, and nausea in females who have cancers. Evidence on moxibustion for breech presentation, induction of labor, and reduction of menopausal symptoms is still inconclusive. In mental health, evidence for acupuncture's efficacy in treating neurologic and functional bowel disorder is still inconclusive. For chronic cancer-related problems such as pain, acupuncture may work well in stand-alone clinics; however, for acute or treatment-related symptoms, integration of acupuncture care into a busy and complex clinical environment is unlikely, unless compelling evidence of a considerable patient benefit can be established.
PMCID: PMC3155101  PMID: 18803496
7.  Evidence Gaps in Advanced Cancer Care: Community-Based Clinicians' Perspectives and Priorities for Comparative Effectiveness Research 
Journal of Oncology Practice  2012;8(3 Suppl):28s-33s.
Comparative effectiveness research and pragmatic clinical trials are valued methods to address the limitations of traditional randomized trials, answer questions of cost-effectiveness or noninferiority, and inform data-driven dialogue and decision making by stakeholders.
Although much effort has focused on identifying national comparative effectiveness research (CER) priorities, little is known about the CER priorities of community-based practitioners treating patients with advanced cancer. CER priorities of managed-care–based clinicians may be valuable as reflections of both payer and provider research interests.
We conducted mixed methods interviews with 10 clinicians (five oncologists and five pharmacists) at five health plans within the Health Maintenance Organization Cancer Research Network. We asked, “What evidence do you most wish you had when treating patients with advanced cancer?” and questioned participants on their impressions and knowledge of CER and pragmatic clinical trials (PCTs). We conducted qualitative analyses to identify themes across interviews.
Ninety percent of participants had heard of CER, 20% had heard of PCTs, and all rated CER/PCTs as highly relevant to patient and health plan decision making. Each participant offered between three and 10 research priorities. Half (49%) involved head-to-head treatment comparisons; another 20% involved comparing different schedules or dosing regimens of the same treatment. The majority included alternative outcomes to survival (eg, toxicity, quality of life, noninferiority). Participants cited several limitations to existing evidence including lack of generalizability, funding biases, and rapid development of new treatments.
Head-to-head treatment comparisons remain a major evidence need among community-based oncology clinicians, and CER/PCTs are highly valued methods to address the limitations of traditional randomized trials, answer questions of cost-effectiveness or noninferiority, and inform data-driven dialogue and decision making by all stakeholders.
PMCID: PMC3348591  PMID: 22942821
8.  Building the Evidence Base for Decision-making in Cancer Genomic Medicine Using Comparative Effectiveness Research 
The clinical utility is uncertain for many cancer genomic applications. Comparative effectiveness research (CER) can provide evidence to clarify this uncertainty.
To identify approaches to help stakeholders make evidence-based decisions, and to describe potential challenges and opportunities using CER to produce evidence-based guidance.
We identified general CER approaches for genomic applications through literature review, the authors’ experiences, and lessons learned from a recent, seven-site CER initiative in cancer genomic medicine. Case studies illustrate the use of CER approaches.
Evidence generation and synthesis approaches include comparative observational and randomized trials, patient reported outcomes, decision modeling, and economic analysis. We identified significant challenges to conducting CER in cancer genomics: the rapid pace of innovation, the lack of regulation, the limited evidence for clinical utility, and the beliefs that genomic tests could have personal utility without having clinical utility. Opportunities to capitalize on CER methods in cancer genomics include improvements in the conduct of evidence synthesis, stakeholder engagement, increasing the number of comparative studies, and developing approaches to inform clinical guidelines and research prioritization.
CER offers a variety of methodological approaches to address stakeholders’ needs. Innovative approaches are needed to ensure an effective translation of genomic discoveries.
PMCID: PMC3632438  PMID: 22516979
evidence synthesis; evidence generation; stakeholder; clinical utility
9.  Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments 
PLoS Medicine  2013;10(7):e1001489.
The vast majority of medical interventions introduced into clinical development prove unsafe or ineffective. One prominent explanation for the dismal success rate is flawed preclinical research. We conducted a systematic review of preclinical research guidelines and organized recommendations according to the type of validity threat (internal, construct, or external) or programmatic research activity they primarily address.
Methods and Findings
We searched MEDLINE, Google Scholar, Google, and the EQUATOR Network website for all preclinical guideline documents published up to April 9, 2013 that addressed the design and conduct of in vivo animal experiments aimed at supporting clinical translation. To be eligible, documents had to provide guidance on the design or execution of preclinical animal experiments and represent the aggregated consensus of four or more investigators. Data from included guidelines were independently extracted by two individuals for discrete recommendations on the design and implementation of preclinical efficacy studies. These recommendations were then organized according to the type of validity threat they addressed. A total of 2,029 citations were identified through our search strategy. From these, we identified 26 guidelines that met our eligibility criteria—most of which were directed at neurological or cerebrovascular drug development. Together, these guidelines offered 55 different recommendations. Some of the most common recommendations included performance of a power calculation to determine sample size, randomized treatment allocation, and characterization of disease phenotype in the animal model prior to experimentation.
By identifying the most recurrent recommendations among preclinical guidelines, we provide a starting point for developing preclinical guidelines in other disease domains. We also provide a basis for the study and evaluation of preclinical research practice.
Please see later in the article for the Editors' Summary
Editors' Summary
The development process for new drugs is lengthy and complex. It begins in the laboratory, where scientists investigate the causes of diseases and identify potential new treatments. Next, promising interventions undergo preclinical research in cells and in animals (in vivo animal experiments) to test whether the intervention has the expected effect and to support the generalization (extension) of this treatment–effect relationship to patients. Drugs that pass these tests then enter clinical trials, where their safety and efficacy is tested in selected groups of patients under strictly controlled conditions. Finally, the government bodies responsible for drug approval review the results of the clinical trials, and successful drugs receive a marketing license, usually a decade or more after the initial laboratory work. Notably, only 11% of agents that enter clinical testing (investigational drugs) are ultimately licensed.
Why Was This Study Done?
The frequent failure of investigational drugs during clinical translation is potentially harmful to trial participants. Moreover, the costs of these failures are passed onto healthcare systems in the form of higher drug prices. It would be good, therefore, to reduce the attrition rate of investigational drugs. One possible explanation for the dismal success rate of clinical translation is that preclinical research, the key resource for justifying clinical development, is flawed. To address this possibility, several groups of preclinical researchers have issued guidelines intended to improve the design and execution of in vivo animal studies. In this systematic review (a study that uses predefined criteria to identify all the research on a given topic), the authors identify the experimental practices that are commonly recommended in these guidelines and organize these recommendations according to the type of threat to validity (internal, construct, or external) that they address. Internal threats to validity are factors that confound reliable inferences about treatment–effect relationships in preclinical research. For example, experimenter expectation may bias outcome assessment. Construct threats to validity arise when researchers mischaracterize the relationship between an experimental system and the clinical disease it is intended to represent. For example, researchers may use an animal model for a complex multifaceted clinical disease that only includes one characteristic of the disease. External threats to validity are unseen factors that frustrate the transfer of treatment–effect relationships from animal models to patients.
What Did the Researchers Do and Find?
The researchers identified 26 preclinical guidelines that met their predefined eligibility criteria. Twelve guidelines addressed preclinical research for neurological and cerebrovascular drug development; other disorders covered by guidelines included cardiac and circulatory disorders, sepsis, pain, and arthritis. Together, the guidelines offered 55 different recommendations for the design and execution of preclinical in vivo animal studies. Nineteen recommendations addressed threats to internal validity. The most commonly included recommendations of this type called for the use of power calculations to ensure that sample sizes are large enough to yield statistically meaningful results, random allocation of animals to treatment groups, and “blinding” of researchers who assess outcomes to treatment allocation. Among the 25 recommendations that addressed threats to construct validity, the most commonly included recommendations called for characterization of the properties of the animal model before experimentation and matching of the animal model to the human manifestation of the disease. Finally, six recommendations addressed threats to external validity. The most commonly included of these recommendations suggested that preclinical research should be replicated in different models of the same disease and in different species, and should also be replicated independently.
What Do These Findings Mean?
This systematic review identifies a range of investigational recommendations that preclinical researchers believe address threats to the validity of preclinical efficacy studies. Many of these recommendations are not widely implemented in preclinical research at present. Whether the failure to implement them explains the frequent discordance between the results on drug safety and efficacy obtained in preclinical research and in clinical trials is currently unclear. These findings provide a starting point, however, for the improvement of existing preclinical research guidelines for specific diseases, and for the development of similar guidelines for other diseases. They also provide an evidence-based platform for the analysis of preclinical evidence and for the study and evaluation of preclinical research practice. These findings should, therefore, be considered by investigators, institutional review bodies, journals, and funding agents when designing, evaluating, and sponsoring translational research.
Additional Information
Please access these websites via the online version of this summary at
The US Food and Drug Administration provides information about drug approval in the US for consumers and for health professionals; its Patient Network provides a step-by-step description of the drug development process that includes information on preclinical research
The UK Medicines and Healthcare Products Regulatory Agency (MHRA) provides information about all aspects of the scientific evaluation and approval of new medicines in the UK; its My Medicine: From Laboratory to Pharmacy Shelf web pages describe the drug development process from scientific discovery, through preclinical and clinical research, to licensing and ongoing monitoring
The STREAM website provides ongoing information about policy, ethics, and practices used in clinical translation of new drugs
The CAMARADES collaboration offers a “supporting framework for groups involved in the systematic review of animal studies” in stroke and other neurological diseases
PMCID: PMC3720257  PMID: 23935460
10.  Diagnosis and treatment of hyponatremia: a systematic review of clinical practice guidelines and consensus statements 
BMC Medicine  2014;12(1):1.
Hyponatremia is a common electrolyte disorder. Multiple organizations have published guidance documents to assist clinicians in managing hyponatremia. We aimed to explore the scope, content, and consistency of these documents.
We searched MEDLINE, EMBASE, and websites of guideline organizations and professional societies to September 2014 without language restriction for Clinical Practice Guidelines (defined as any document providing guidance informed by systematic literature review) and Consensus Statements (any other guidance document) developed specifically to guide differential diagnosis or treatment of hyponatremia. Four reviewers appraised guideline quality using the 23-item AGREE II instrument, which rates reporting of the guidance development process across six domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Total scores were calculated as standardized averages by domain.
We found ten guidance documents; five clinical practice guidelines and five consensus statements. Overall, quality was mixed: two clinical practice guidelines attained an average score of >50% for all of the domains, three rated the evidence in a systematic way and two graded strength of the recommendations. All five consensus statements received AGREE scores below 60% for each of the specific domains.
The guidance documents varied widely in scope. All dealt with therapy and seven included recommendations on diagnosis, using serum osmolality to confirm hypotonic hyponatremia, and volume status, urinary sodium concentration, and urinary osmolality for further classification of the hyponatremia. They differed, however, in classification thresholds, what additional tests to consider, and when to initiate diagnostic work-up. Eight guidance documents advocated hypertonic NaCl in severely symptomatic, acute onset (<48 h) hyponatremia. In chronic (>48 h) or asymptomatic cases, recommended treatments were NaCl 0.9%, fluid restriction, and cause-specific therapy for hypovolemic, euvolemic, and hypervolemic hyponatremia, respectively. Eight guidance documents recommended limits for speed of increase of sodium concentration, but these varied between 8 and 12 mmol/L per 24 h. Inconsistencies also existed in the recommended dose of NaCl, its initial infusion speed, and which second line interventions to consider.
Current guidance documents on the assessment and treatment of hyponatremia vary in methodological rigor and recommendations are not always consistent.
Electronic supplementary material
The online version of this article (doi:10.1186/s12916-014-0231-1) contains supplementary material, which is available to authorized users.
PMCID: PMC4276109  PMID: 25539784
Clinical practice guideline; Hyponatremia; Systematic review
11.  Eurocan plus report: feasibility study for coordination of national cancer research activities 
The EUROCAN+PLUS Project, called for by the European Parliament, was launched in October 2005 as a feasibility study for coordination of national cancer research activities in Europe. Over the course of the next two years, the Project process organized over 60 large meetings and countless smaller meetings that gathered in total over a thousand people, the largest Europe–wide consultation ever conducted in the field of cancer research.
Despite a strong tradition in biomedical science in Europe, fragmentation and lack of sustainability remain formidable challenges for implementing innovative cancer research and cancer care improvement. There is an enormous duplication of research effort in the Member States, which wastes time, wastes money and severely limits the total intellectual concentration on the wide cancer problem. There is a striking lack of communication between some of the biggest actors on the European scene, and there are palpable tensions between funders and those researchers seeking funds.
It is essential to include the patients’ voice in the establishment of priority areas in cancer research at the present time. The necessity to have dialogue between funders and scientists to establish the best mechanisms to meet the needs of the entire community is evident. A top priority should be the development of translational research (in its widest form), leading to the development of effective and innovative cancer treatments and preventive strategies. Translational research ranges from bench–to–bedside innovative cancer therapies and extends to include bringing about changes in population behaviours when a risk factor is established.
The EUROCAN+PLUS Project recommends the creation of a small, permanent and independent European Cancer Initiative (ECI). This should be a model structure and was widely supported at both General Assemblies of the project. The ECI should assume responsibility for stimulating innovative cancer research and facilitating processes, becoming the common voice of the cancer research community and serving as an interface between the cancer research community and European citizens, patients’ organizations, European institutions, Member States, industry and small and medium enterprises (SMEs), putting into practice solutions aimed at alleviating barriers to collaboration and coordination of cancer research activities in the European Union, and dealing with legal and regulatory issues. The development of an effective ECI will require time, but this entity should be established immediately. As an initial step, coordination efforts should be directed towards the creation of a platform on translational research that could encompass (1) coordination between basic, clinical and epidemiological research; (2) formal agreements of co–operation between comprehensive cancer centres and basic research laboratories throughout Europe and (3) networking between funding bodies at the European level.
The European Parliament and its instruments have had a major influence in cancer control in Europe, notably in tobacco control and in the implementation of effective population–based screening. To make further progress there is a need for novelty and innovation in cancer research and prevention in Europe, and having a platform such as the ECI, where those involved in all aspects of cancer research can meet, discuss and interact, is a decisive development for Europe.
Executive Summary
Cancer is one of the biggest public health crises facing Europe in the 21st century—one for which Europe is currently not prepared nor preparing itself. Cancer is a major cause of death in Europe with two million casualties and three million new cases diagnosed annually, and the situation is set to worsen as the population ages.
These facts led the European Parliament, through the Research Directorate-General of the European Commission, to call for initiatives for better coordination of cancer research efforts in the European Union. The EUROCAN+PLUS Project was launched in October 2005 as a feasibility study for coordination of national cancer research activities. Over the course of the next two years, the Project process organized over 60 large meetings and countless smaller meetings that gathered in total over a thousand people. In this respect, the Project became the largest Europe-wide consultation ever conducted in the field of cancer research, implicating researchers, cancer centres and hospitals, administrators, healthcare professionals, funding agencies, industry, patients’ organizations and patients.
The Project first identified barriers impeding research and collaboration in research in Europe. Despite a strong tradition in biomedical science in Europe, fragmentation and lack of sustainability remain the formidable challenges for implementing innovative cancer research and cancer care improvement. There is an enormous duplication of research effort in the Member States, which wastes time, wastes money and severely limits the total intellectual concentration on the wide cancer problem. There is a striking lack of communication between some of the biggest actors on the European scene, and there are palpable tensions between funders and those researchers seeking funds.
In addition, there is a shortage of leadership, a multiplicity of institutions each focusing on its own agenda, sub–optimal contact with industry, inadequate training, non–existent career paths, low personnel mobility in research especially among clinicians and inefficient funding—all conspiring against efficient collaboration in cancer care and research. European cancer research today does not have a functional translational research continuum, that is the process that exploits biomedical research innovations and converts them into prevention methods, diagnostic tools and therapies. Moreover, epidemiological research is not integrated with other types of cancer research, and the implementation of the European Directives on Clinical Trials 1 and on Personal Data Protection 2 has further slowed the innovation process in Europe. Furthermore, large inequalities in health and research exist between the EU–15 and the New Member States.
The picture is not entirely bleak, however, as the European cancer research scene presents several strengths, such as excellent basic research and clinical research and innovative etiological research that should be better exploited.
When considering recommendations, several priority dimensions had to be retained. It is essential that proposals include actions and recommendations that can benefit all Member States of the European Union and not just States with the elite centres. It is also essential to have a broader patient orientation to help provide the knowledge to establish cancer control possibilities when we exhaust what can be achieved by the implementation of current knowledge. It is vital that the actions proposed can contribute to the Lisbon Strategy to make Europe more innovative and competitive in (cancer) research.
The Project participants identified six areas for which consensus solutions should be implemented in order to obtain better coordination of cancer research activities. The required solutions are as follows. The proactive management of innovation, detection, facilitation of collaborations and maintenance of healthy competition within the European cancer research community.The establishment of an exchange portal of information for health professionals, patients and policy makers.The provision of guidance for translational and clinical research including the establishment of a translational research platform involving comprehensive cancer centres and cancer research centres.The coordination of calls and financial management of cancer research projects.The construction of a ‘one–stop shop’ as a contact interface between the industry, small and medium enterprises, scientists and other stakeholders.The support of greater involvement of healthcare professionals in translational research and multidisciplinary training.
In the course of the EUROCAN+PLUS consultative process, several key collaborative projects emerged between the various groups and institutes engaged in the consultation. There was a collaboration network established with Europe’s leading Comprehensive Cancer Centres; funding was awarded for a closer collaboration of Owners of Cancer Registries in Europe (EUROCOURSE); there was funding received from FP7 for an extensive network of leading Biological Resource Centres in Europe (BBMRI); a Working Group identified the special needs of Central, Eastern and South–eastern Europe and proposed a remedy (‘Warsaw Declaration’), and the concept of developing a one–stop shop for dealing with academia and industry including the Innovative Medicines Initiative (IMI) was discussed in detail.
Several other dimensions currently lacking were identified. There is an absolute necessity to include the patients’ voice in the establishment of priority areas in cancer research at the present time. It was a salutary lesson when it was recognized that all that is known about the quality of life of the cancer patient comes from the experience of a tiny proportion of cancer patients included in a few clinical trials. The necessity to have dialogue between funders and scientists to establish the best mechanisms to meet the needs of the entire community was evident. A top priority should be the development of translational research (in its widest form) and the development of effective and innovative cancer treatments and preventative strategies in the European Union. Translational research ranges from bench-to-bedside innovative cancer therapies and extends to include bringing about changes in population behaviours when a risk factor is established.
Having taken note of the barriers and the solutions and having examined relevant examples of existing European organizations in the field, it was agreed during the General Assembly of 19 November 2007 that the EUROCAN+PLUS Project had to recommend the creation of a small, permanent and neutral ECI. This should be a model structure and was widely supported at both General Assemblies of the project. The proposal is based on the successful model of the European Molecular Biology Organisation (EMBO), and its principal aims include providing a forum where researchers from all backgrounds and from all countries can meet with members of other specialities including patients, nurses, clinicians, funders and scientific administrators to develop priority programmes to make Europe more competitive in research and more focused on the cancer patient.
The ECI should assume responsibility for: stimulating innovative cancer research and facilitating processes;becoming the common voice of the cancer research community and serving as an interface between the cancer research community and European citizens, patients’ and organizations;European institutions, Member States, industry and small and medium enterprises;putting into practice the aforementioned solutions aimed at alleviating barriers and coordinating cancer research activities in the EU;dealing with legal and regulatory issues.
Solutions implemented through the ECI will lead to better coordination and collaboration throughout Europe, more efficient use of resources, an increase in Europe’s attractiveness to the biomedical industry and better quality of cancer research and education of health professionals.
The Project considered that European legal instruments currently available were inadequate for addressing many aspects of the barriers identified and for the implementation of effective, lasting solutions. Therefore, the legal environment that could shelter an idea like the ECI remains to be defined but should be done so as a priority. In this context, the initiative of the European Commission for a new legal entity for research infrastructure might be a step in this direction. The development of an effective ECI will require time, but this should be established immediately. As an initial step, coordination efforts should be directed towards the creation of a platform on translational research that could encompass: (1) coordination between basic, clinical and epidemiological research; (2) formal agreements of co-operation between comprehensive cancer centres and basic research laboratories throughout Europe; (3) networking between funding bodies at the European level. Another topic deserving immediate attention is the creation of a European database on cancer research projects and cancer research facilities.
Despite enormous progress in cancer control in Europe during the past two decades, there was an increase of 300,000 in the number of new cases of cancer diagnosed between 2004 and 2006. The European Parliament and its instruments have had a major influence in cancer control, notably in tobacco control and in the implementation of effective population–based screening. To make further progress there is a need for novelty and innovation in cancer research and prevention in Europe, and having a platform such as the ECI, where those involved in all aspects of cancer research can meet, discuss and interact, is a decisive development for Europe.
PMCID: PMC3234055  PMID: 22274749
12.  Comparative Effectiveness Research in Lung Diseases and Sleep Disorders 
The Division of Lung Diseases of the National Heart, Lung, and Blood Institute (NHLBI) held a workshop to develop recommendations on topics, methodologies, and resources for comparative effectiveness research (CER) that will guide clinical decision making about available treatment options for lung diseases and sleep disorders. A multidisciplinary group of experts with experience in efficacy, effectiveness, implementation, and economic research identified (a) what types of studies the domain of CER in lung diseases and sleep disorders should include, (b) the criteria and process for setting priorities, and (c) current resources for and barriers to CER in lung diseases. Key recommendations were to (1) increase efforts to engage stakeholders in developing CER questions and study designs; (2) invest in further development of databases and other infrastructure, including efficient methods for data sharing; (3) make full use of a broad range of study designs; (4) increase the appropriate use of observational designs and the support of methodologic research; (5) ensure that committees that review CER grant applications include persons with appropriate perspective and expertise; and (6) further develop the workforce for CER by supporting training opportunities that focus on the methodologic and practical skills needed.
PMCID: PMC3265273  PMID: 21965016
randomized controlled trials; observational studies; implementation; study designs; methodology
13.  Representation and Misrepresentation of Scientific Evidence in Contemporary Tobacco Regulation: A Review of Tobacco Industry Submissions to the UK Government Consultation on Standardised Packaging 
PLoS Medicine  2014;11(3):e1001629.
Selda Ulucanlar and colleagues analyze submissions by two tobacco companies to the UK government consultation on standardized packaging.
Please see later in the article for the Editors' Summary
Standardised packaging (SP) of tobacco products is an innovative tobacco control measure opposed by transnational tobacco companies (TTCs) whose responses to the UK government's public consultation on SP argued that evidence was inadequate to support implementing the measure. The government's initial decision, announced 11 months after the consultation closed, was to wait for ‘more evidence’, but four months later a second ‘independent review’ was launched. In view of the centrality of evidence to debates over SP and TTCs' history of denying harms and manufacturing uncertainty about scientific evidence, we analysed their submissions to examine how they used evidence to oppose SP.
Methods and Findings
We purposively selected and analysed two TTC submissions using a verification-oriented cross-documentary method to ascertain how published studies were used and interpretive analysis with a constructivist grounded theory approach to examine the conceptual significance of TTC critiques. The companies' overall argument was that the SP evidence base was seriously flawed and did not warrant the introduction of SP. However, this argument was underpinned by three complementary techniques that misrepresented the evidence base. First, published studies were repeatedly misquoted, distorting the main messages. Second, ‘mimicked scientific critique’ was used to undermine evidence; this form of critique insisted on methodological perfection, rejected methodological pluralism, adopted a litigation (not scientific) model, and was not rigorous. Third, TTCs engaged in ‘evidential landscaping’, promoting a parallel evidence base to deflect attention from SP and excluding company-held evidence relevant to SP. The study's sample was limited to sub-sections of two out of four submissions, but leaked industry documents suggest at least one other company used a similar approach.
The TTCs' claim that SP will not lead to public health benefits is largely without foundation. The tools of Better Regulation, particularly stakeholder consultation, provide an opportunity for highly resourced corporations to slow, weaken, or prevent public health policies.
Please see later in the article for the Editors' Summary
Editors' Summary
Every year, about 6 million people die from tobacco-related diseases and, if current trends continue, annual tobacco-related deaths will increase to more than 8 million by 2030. To reduce this loss of life, national and international bodies have drawn up various conventions and directives designed to implement tobacco control measures such as the adoption of taxation policies aimed at reducing tobacco consumption and bans on tobacco advertising, promotion, and sponsorship. One innovative but largely unused tobacco control measure is standardised packaging of tobacco products. Standardised packaging aims to prevent the use of packaging as a marketing tool by removing all brand imagery and text (other than name) and by introducing packs of a standard shape and colour that include prominent pictorial health warnings. Standardised packaging was first suggested as a tobacco control measure in 1986 but has been consistently opposed by the tobacco industry.
Why Was This Study Done?
The UK is currently considering standardised packaging of tobacco products. In the UK, Better Regulation guidance obliges officials to seek the views of stakeholders, including corporations, on the government's cost and benefit estimates of regulatory measures such as standardised packaging and on the evidence underlying these estimates. In response to a public consultation about standardised packaging in July 2013, which considered submissions from several transnational tobacco companies (TTCs), the UK government announced that it would wait for the results of the standardised packaging legislation that Australia adopted in December 2012 before making its final decision about this tobacco control measure. Parliamentary debates and media statements have suggested that doubt over the adequacy of the evidence was the main reason for this ‘wait and see’ decision. Notably, TTCs have a history of manufacturing uncertainty about the scientific evidence related to the harms of tobacco. Given the centrality of evidence to the debate about standardised packaging, in this study, the researchers analyse submissions made by two TTCs, British American Tobacco (BAT) and Japan Tobacco International (JTI), to the first UK consultation on standardised packaging (a second review is currently underway and will report shortly) to examine how TTCs used evidence to oppose standardised packaging.
What Did the Researchers Do and Find?
The researchers analysed sub-sections of two of the four TTC submissions (those submitted by BAT and JTI) made to the public consultation using verification-oriented cross-documentary analysis, which compared references made to published sources with the original sources to ascertain how these sources had been used, and interpretative analysis to examine the conceptual significance of TTC critiques of the evidence on standardised packaging. The researchers report that the companies' overall argument was that the evidence base in support of standardised packaging was seriously flawed and did not warrant the introduction of such packaging. The researchers identified three ways in which the TTC reports misrepresented the evidence base. First, the TTCs misquoted published studies, thereby distorting the main messages of these studies. For example, the TTCs sometimes omitted important qualifying information when quoting from published studies. Second, the TTCs undermined evidence by employing experts to review published studies for methodological rigor and value in ways that did not conform to normal scientific critique approaches (‘mimicked scientific critique’). So, for example, the experts considered each piece of evidence in isolation for its ability to support standardised packaging rather than considering the cumulative weight of the evidence. Finally, the TTCs engaged in ‘evidential landscaping’. That is, they promoted research that deflected attention from standardised packaging (for example, research into social explanations of smoking behaviour) and omitted internal industry research on the role of packaging in marketing.
What Do These Findings Mean?
These findings suggest that the TTC critique of the evidence in favour of standardised packaging that was presented to the UK public consultation on this tobacco control measure is highly misleading. However, because the researchers' analysis only considered subsections of the submissions from two TTCs, these findings may not be applicable to the other submissions or to other TTCs. Moreover, their analysis only considered the efforts made by TTCs to influence public health policy and not the effectiveness of these efforts. Nevertheless, these findings suggest that the claim of TTCs that standardised packaging will not lead to public health benefits is largely without foundation. More generally, these findings highlight the possibility that the tools of Better Regulation, particularly stakeholder consultation, provide an opportunity for wealthy corporations to slow, weaken, or prevent the implementation of public health policies.
Additional Information
Please access these websites via the online version of this summary at
The World Health Organization provides information about the dangers of tobacco (in several languages) and an article about first experiences with Australia's tobacco plain packaging law; for information about the tobacco industry's influence on policy, see the 2009 World Health Organization report ‘Tobacco industry interference with tobacco control’
A UK parliamentary briefing on standardised packaging of tobacco products, a press release about the consultation, and a summary report of the consultation are available; the ideas behind the UK's Better Regulation guidance are described in a leaflet produced by the Better Regulation Task Force
Cancer Research UK (CRUK) has a web page with information on standardised packaging and includes videos
Wikipedia has a page on standardised packaging of tobacco products (note: Wikipedia is a free online encyclopaedia that anyone can edit; available in several languages)
The UK Centre for Tobacco Control Studies is a network of UK universities that undertakes original research, policy development, advocacy, and teaching and training in the field of tobacco control, an online resource managed by the University of Bath, provides up-to-date information on the tobacco industry and the tactics it uses to influence tobacco regulation
SmokeFree, a website provided by the UK National Health Service, offers advice on quitting smoking and includes personal stories from people who have stopped smoking, from the US National Cancer Institute, offers online tools and resources to help people quit smoking
PMCID: PMC3965396  PMID: 24667150
14.  Paradoxes in Acupuncture Research: Strategies for Moving Forward 
In November 2007, the Society for Acupuncture Research (SAR) held an international symposium to mark the 10th anniversary of the 1997 NIH Consensus Development Conference on Acupuncture. The symposium presentations revealed the considerable maturation of the field of acupuncture research, yet two provocative paradoxes emerged. First, a number of well-designed clinical trials have reported that true acupuncture is superior to usual care, but does not significantly outperform sham acupuncture, findings apparently at odds with traditional theories regarding acupuncture point specificity. Second, although many studies using animal and human experimental models have reported physiological effects that vary as a function of needling parameters (e.g., mode of stimulation) the extent to which these parameters influence therapeutic outcomes in clinical trials is unclear. This White Paper, collaboratively written by the SAR Board of Directors, identifies gaps in knowledge underlying the paradoxes and proposes strategies for their resolution through translational research. We recommend that acupuncture treatments should be studied (1) “top down” as multi-component “whole-system” interventions and (2) “bottom up” as mechanistic studies that focus on understanding how individual treatment components interact and translate into clinical and physiological outcomes. Such a strategy, incorporating considerations of efficacy, effectiveness and qualitative measures, will strengthen the evidence base for such complex interventions as acupuncture.
PMCID: PMC2957136  PMID: 20976074
15.  Reporting Guidelines for Survey Research: An Analysis of Published Guidance and Reporting Practices 
PLoS Medicine  2011;8(8):e1001069.
Carol Bennett and colleagues review the evidence and find that there is limited guidance and no consensus on the optimal reporting of survey research.
Research needs to be reported transparently so readers can critically assess the strengths and weaknesses of the design, conduct, and analysis of studies. Reporting guidelines have been developed to inform reporting for a variety of study designs. The objective of this study was to identify whether there is a need to develop a reporting guideline for survey research.
Methods and Findings
We conducted a three-part project: (1) a systematic review of the literature (including “Instructions to Authors” from the top five journals of 33 medical specialties and top 15 general and internal medicine journals) to identify guidance for reporting survey research; (2) a systematic review of evidence on the quality of reporting of surveys; and (3) a review of reporting of key quality criteria for survey research in 117 recently published reports of self-administered surveys. Fewer than 7% of medical journals (n = 165) provided guidance to authors on survey research despite a majority having published survey-based studies in recent years. We identified four published checklists for conducting or reporting survey research, none of which were validated. We identified eight previous reviews of survey reporting quality, which focused on issues of non-response and accessibility of questionnaires. Our own review of 117 published survey studies revealed that many items were poorly reported: few studies provided the survey or core questions (35%), reported the validity or reliability of the instrument (19%), defined the response rate (25%), discussed the representativeness of the sample (11%), or identified how missing data were handled (11%).
There is limited guidance and no consensus regarding the optimal reporting of survey research. The majority of key reporting criteria are poorly reported in peer-reviewed survey research articles. Our findings highlight the need for clear and consistent reporting guidelines specific to survey research.
Please see later in the article for the Editors' Summary
Editors' Summary
Surveys, or questionnaires, are an essential component of many types of research, including health, and usually gather information by asking a sample of people questions on a specific topic and then generalizing the results to a larger population. Surveys are especially important when addressing topics that are difficult to assess using other approaches and usually rely on self reporting, for example self-reported behaviors, such as eating habits, satisfaction, beliefs, knowledge, attitudes, opinions. However, the methods used in conducting survey research can significantly affect the reliability, validity, and generalizability of study results, and without clear reporting of the methods used in surveys, it is difficult or impossible to assess these characteristics and therefore to have confidence in the findings.
Why Was This Study Done?
This uncertainty in other forms of research has given rise to Reporting Guidelines—evidence-based, validated tools that aim to improve the reporting quality of health research. The STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) Statement includes cross-sectional studies, which often involve surveys. But not all surveys are epidemiological, and STROBE does not include methods' and results' reporting characteristics that are unique to surveys. Therefore, the researchers conducted this study to help determine whether there is a need for a reporting guideline for health survey research.
What Did the Researchers Do and Find?
The researchers identified any previous relevant guidance for survey research, and any evidence on the quality of reporting of survey research, by: reviewing current guidance for reporting survey research in the “Instructions to Authors” of leading medical journals and in published literature; conducting a systematic review of evidence on the quality of reporting of surveys; identifying key quality criteria for the conduct of survey research; and finally, reviewing how these criteria are currently reported by conducting a review of recently published reports of self-administered surveys.
The researchers found that 154 of the 165 journals searched (93.3%) did not provide any guidance on survey reporting, even though the majority (81.8%) have published survey research. Only three of the 11 journals that provided some guidance gave more than one directive or statement. Five papers and one Internet site provided guidance on the reporting of survey research, but none used validated measures or explicit methods for development. The researchers identified eight papers that addressed the quality of reporting of some aspect of survey research: the reporting of response rates; the reporting of non-response analyses in survey research; and the degree to which authors make their survey instrument available to readers. In their review of 117 published survey studies, the researchers found that many items were poorly reported: few studies provided the survey or core questions (35%), reported the validity or reliability of the instrument (19%), discussed the representativeness of the sample (11%), or identified how missing data were handled (11%). Furthermore, (88 [75%]) did not include any information on consent procedures for research participants, and one-third (40 [34%]) of papers did not report whether the study had received research ethics board review.
What Do These Findings Mean?
Overall, these results show that guidance is limited and consensus lacking about the optimal reporting of survey research, and they highlight the need for a well-developed reporting guideline specifically for survey research—possibly an extension of the guideline for observational studies in epidemiology (STROBE)—that will provide the structure to ensure more complete reporting and allow clearer review and interpretation of the results from surveys.
Additional Information
Please access these web sites via the online version of this summary at
More than 100 reporting guidelines covering a broad spectrum of research types are indexed on the EQUATOR Networks web site
More information about STROBE is available on the STROBE Statement web site
PMCID: PMC3149080  PMID: 21829330
16.  Improving the use of research evidence in guideline development: 2. Priority setting 
The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the second of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.
We reviewed the literature on priority setting for health care guidelines, recommendations and technology assessments.
We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.
Key questions and answers
There is little empirical evidence to guide the choice of criteria and processes for establishing priorities, but there are broad similarities in the criteria that are used by various organisations and practical arguments for setting priorities explicitly rather than implicitly,
What criteria should be used to establish priorities?
• WHO has limited resources and capacity to develop recommendations. It should use these resources where it has the greatest chance of improving health, equity, and efficient use of healthcare resources.
• We suggest the following criteria for establishing priorities for developing recommendations based on WHO's aims and strategic advantages:
• Problems associated with a high burden of illness in low and middle-income countries, or new and emerging diseases.
• No existing recommendations of good quality.
• The feasibility of developing recommendations that will improve health outcomes, reduce inequities or reduce unnecessary costs if they are implemented.
• Implementation is feasible, will not exhaustively use available resources, and barriers to change are not likely to be so high that they cannot be overcome.
• Additional priorities for WHO include interventions that will likely require system changes and interventions where there might be a conflict in choices between individual and societal perspectives.
What processes should be used to agree on priorities?
• The allocation of resources to the development of recommendations should be part of the routine budgeting process rather than a separate exercise.
• Criteria for establishing priorities should be applied using a systematic and transparent process.
• Because data to inform judgements are often lacking, unmeasured factors should also be considered – explicitly and transparently.
• The process should include consultation with potential end users and other stakeholders, including the public, using well-constructed questions, and possibly using Delphi-like procedures.
• Groups that include stakeholders and people with relevant types of expertise should make decisions. Group processes should ensure full participation by all members of the group.
• The process used to select topics should be documented and open to inspection.
Should WHO have a centralised or decentralised process?
• Both centralised and decentralised processes should be used. Decentralised processes can be considered as separate "tracks".
• Separate tracks should be used for considering issues for specific areas, populations, conditions or concerns. The rationales for designating special tracks should be defined clearly; i.e. why they warrant special consideration.
• Updating of guidelines could also be considered as a separate "track", taking account of issues such as the need for corrections and the availability of new evidence.
PMCID: PMC1702532  PMID: 17134481
17.  Stakeholder Priorities for Comparative Effectiveness Research in Chronic Obstructive Pulmonary Disease 
Comparative effectiveness research (CER) is intended to address the expressed needs of patients, clinicians, and other stakeholders. Representatives of 54 stakeholder groups with an interest in chronic obstructive pulmonary disease (COPD) participated in workshops convened by the COPD Outcomes-based Network for Clinical Effectiveness and Research Translation (CONCERT) over a 2-year period. Year 1 focused on chronic care and care coordination. Year 2 focused on acute care and transitions in care between healthcare settings. Discussions and provisional voting were conducted via teleconferences and e-mail exchanges before the workshop. Final prioritization votes occurred after in-person discussions at the workshop. We used a modified Delphi approach to facilitate discussions and consensus building. To more easily quantify preferences and to evaluate the internal consistency of rankings, the Analytic Hierarchy Process was incorporated in Year 2. Results of preworkshop and final workshop voting often differed, suggesting that prioritization efforts relying solely on requests for topics from stakeholder groups without in-person discussion may provide different research priorities. Research priorities varied across stakeholder groups, but generally focused on studies to evaluate different approaches to healthcare delivery (e.g., spirometry for diagnosis and treatment, integrated healthcare strategies during transitions in care) rather than head-to-head comparisons of medications. This research agenda may help to inform groups intending to respond to CER funding opportunities in COPD. The methodologies used, detailed in the online supplement, may also help to inform prioritization efforts for CER in other health conditions.
PMCID: PMC3603554  PMID: 23155144
health services research; research priorities; care coordination; stakeholders
18.  Introducing patient perspective in health technology assessment at the local level 
Recognizing the importance of increased patient participation in healthcare decisions leads decision makers to consider effective ways to incorporate patient perspectives in Health Technology Assessment (HTA) processes. The implementation of local health HTA units in university hospitals in Quebec provides a unique opportunity to foster an increased participation of patients in decisions regarding health technologies and clinical interventions. This project explores strategies that could be effective in involving patients in HTA activities at the local level. To do so, three objectives are pursued: 1) To synthesise international knowledge and experiences on patient and public involvement in HTA activities; 2) To explore the perceptions of stakeholders (administrators, clinical managers, healthcare professionals, HTA producers, and patients) regarding strategies for involving patients in various HTA activities; and 3) To produce a consensual strategic framework that could guide interventions for involving patients in HTA activities at the local level.
A systematic review of the literature will be conducted to synthesise international knowledge and experiments regarding the implication of patients and public in HTA. Then, focus groups will be carried out with representatives of various stakeholder groups in order to explore their perceptions regarding patient participation in HTA. Based on findings from the systematic review and the focus groups, a framework to support patient participation in HTA activities will be proposed. It will then be validated during a deliberative meeting with the research team, composed of scientists and decision makers, and representatives from different groups involved in HTA in Quebec. This deliberative meeting will aim at identifying the type and the degree of participation as well as the adequate timing for involving patients in local HTA activities.
Given the actual state of evidence, integrating patient perspective in HTA activities has the potential to improve the quality of healthcare services. This study provides an opportunity to bridge the gap between HTA producers and its ultimate end-user: the patient. It will provide guidance to support local HTA units in Quebec and elsewhere in their decisions regarding patient participation. The framework developed could be applied to design and implement strategies for involving patients in HTA activities.
PMCID: PMC2671501  PMID: 19327160
19.  Translating comparative effectiveness of depression medications into practice by comparing the depression medication choice decision aid to usual care: study protocol for a randomized controlled trial 
Trials  2013;14:127.
Comparative effectiveness research (CER) documents important differences in antidepressants in terms of efficacy, safety, cost, and burden to the patient. Decision aids can adapt this evidence to help patients participate in making informed choices. In turn, antidepressant therapy will more likely reflect patients’ values and context, leading to improved adherence and mood outcomes.
The objective of this study is to develop the Depression Medication Choice decision aid for use during primary care encounters, and to test its efficacy by conducting a clustered practical randomized trial comparing the decision aid to usual depression care in primary care practices.
We will use a novel practice-based, patient-centered approach based on participatory action research that involves a multidisciplinary team of designers, investigators, clinicians, patient representatives, and other stakeholders for the development of the decision aid. We will then conduct a clustered practical randomized trial enrolling clinicians and their patients (n = 300) with moderate to severe depression from rural, suburban and inner city primary care practices (n = 10). The intervention will consist of the use of the depression medication choice decision aid during the clinical encounter. This trial will generate preliminary evidence of the relative impact of the decision aid on patient involvement in decision making, decision making quality, patient knowledge, and 6-month measures of medication adherence and mental health compared to usual depression care.
Upon completion of the proposed research, we will have developed and evaluated the efficacy of the decision aid depression medication choice as a novel translational tool for CER in depression treatment, engaged patients with depression in their care, and refined the process by which we conduct practice-based trials with limited research footprint.
Trial registration
Clinical NCT01502891
PMCID: PMC3663744  PMID: 23782672
Depression; Comparative effectiveness research; Shared decision making; Decision aid; Implementation; Randomized controlled trial
20.  Developing a decision aid to guide public sector health policy decisions: A study protocol 
Decision aids have been developed in a number of health disciplines to support evidence-informed decision making, including patient decision aids and clinical practice guidelines. However, policy contexts differ from clinical contexts in terms of complexity and uncertainty, requiring different approaches for identifying, interpreting, and applying many different types of evidence to support decisions. With few studies in the literature offering decision guidance specifically to health policymakers, the present study aims to facilitate the structured and systematic incorporation of research evidence and, where there is currently very little guidance, values and other non-research-based evidence, into the policy making process. The resulting decision aid is intended to help public sector health policy decision makers who are tasked with making evidence-informed decisions on behalf of populations. The intent is not to develop a decision aid that will yield uniform recommendations across jurisdictions, but rather to facilitate more transparent policy decisions that reflect a balanced consideration of all relevant factors.
The study comprises three phases: a modified meta-narrative review, the use of focus groups, and the application of a Delphi method. The modified meta-narrative review will inform the initial development of the decision aid by identifying as many policy decision factors as possible and other features of methodological guidance deemed to be desirable in the literatures of all relevant disciplines. The first of two focus groups will then seek to marry these findings with focus group members' own experience and expertise in public sector population-based health policy making and screening decisions. The second focus group will examine issues surrounding the application of the decision aid and act as a sounding board for initial feedback and refinement of the draft decision aid. Finally, the Delphi method will be used to further inform and refine the decision aid with a larger audience of potential end-users.
The product of this research will be a working version of a decision aid to support policy makers in population-based health policy decisions. The decision aid will address the need for more structured and systematic ways of incorporating various evidentiary sources where applicable.
PMCID: PMC3108337  PMID: 21569255
21.  Evaluating complex interventions in End of Life Care: the MORECare Statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews 
BMC Medicine  2013;11:111.
Despite being a core business of medicine, end of life care (EoLC) is neglected. It is hampered by research that is difficult to conduct with no common standards. We aimed to develop evidence-based guidance on the best methods for the design and conduct of research on EoLC to further knowledge in the field.
The Methods Of Researching End of life Care (MORECare) project built on the Medical Research Council guidance on the development and evaluation of complex circumstances. We conducted systematic literature reviews, transparent expert consultations (TEC) involving consensus methods of nominal group and online voting, and stakeholder workshops to identify challenges and best practice in EoLC research, including: participation recruitment, ethics, attrition, integration of mixed methods, complex outcomes and economic evaluation. We synthesised all findings to develop a guidance statement on the best methods to research EoLC.
We integrated data from three systematic reviews and five TECs with 133 online responses. We recommend research designs extending beyond randomised trials and encompassing mixed methods. Patients and families value participation in research, and consumer or patient collaboration in developing studies can resolve some ethical concerns. It is ethically desirable to offer patients and families the opportunity to participate in research. Outcome measures should be short, responsive to change and ideally used for both clinical practice and research. Attrition should be anticipated in studies and may affirm inclusion of the relevant population, but careful reporting is necessitated using a new classification. Eventual implementation requires consideration at all stages of the project.
The MORECare statement provides 36 best practice solutions for research evaluating services and treatments in EoLC to improve study quality and set the standard for future research. The statement may be used alongside existing statements and provides a first step in setting common, much needed standards for evaluative research in EoLC. These are relevant to those undertaking research, trainee researchers, research funders, ethical committees and editors.
PMCID: PMC3635872  PMID: 23618406
Palliative care; Terminal care; Research design; Methods; Evaluation studies; Review; Consensus
22.  The translation research in a dental setting (TRiaDS) programme protocol 
It is well documented that the translation of knowledge into clinical practice is a slow and haphazard process. This is no less true for dental healthcare than other types of healthcare. One common policy strategy to help promote knowledge translation is the production of clinical guidance, but it has been demonstrated that the simple publication of guidance is unlikely to optimise practice. Additional knowledge translation interventions have been shown to be effective, but effectiveness varies and much of this variation is unexplained. The need for researchers to move beyond single studies to develop a generalisable, theory based, knowledge translation framework has been identified.
For dentistry in Scotland, the production of clinical guidance is the responsibility of the Scottish Dental Clinical Effectiveness Programme (SDCEP). TRiaDS (Translation Research in a Dental Setting) is a multidisciplinary research collaboration, embedded within the SDCEP guidance development process, which aims to establish a practical evaluative framework for the translation of guidance and to conduct and evaluate a programme of integrated, multi-disciplinary research to enhance the science of knowledge translation.
Set in General Dental Practice the TRiaDS programmatic evaluation employs a standardised process using optimal methods and theory. For each SDCEP guidance document a diagnostic analysis is undertaken alongside the guidance development process. Information is gathered about current dental care activities. Key recommendations and their required behaviours are identified and prioritised. Stakeholder questionnaires and interviews are used to identify and elicit salient beliefs regarding potential barriers and enablers towards the key recommendations and behaviours. Where possible routinely collected data are used to measure compliance with the guidance and to inform decisions about whether a knowledge translation intervention is required. Interventions are theory based and informed by evidence gathered during the diagnostic phase and by prior published evidence. They are evaluated using a range of experimental and quasi-experimental study designs, and data collection continues beyond the end of the intervention to investigate the sustainability of an intervention effect.
The TRiaDS programmatic approach is a significant step forward towards the development of a practical, generalisable framework for knowledge translation research. The multidisciplinary composition of the TRiaDS team enables consideration of the individual, organisational and system determinants of professional behaviour change. In addition the embedding of TRiaDS within a national programme of guidance development offers a unique opportunity to inform and influence the guidance development process, and enables TRiaDS to inform dental services practitioners, policy makers and patients on how best to translate national recommendations into routine clinical activities.
PMCID: PMC2920875  PMID: 20646275
23.  Using a Delphi consensus process to develop an acupuncture treatment protocol by consensus for women undergoing Assisted Reproductive Technology (ART) treatment 
Assisted reproductive technologies (ART) are increasingly utilised for resolving difficulties conceiving. These technologies are expensive to both the public purse and the individual consumers. Acupuncture is widely used as an adjunct to ART with indications that it may assist reducing the time to conception and increasing live birth rates. Heterogeneity is high between treatment protocols.
The aim of this study was to examine what fertility acupuncturists consider key components of best practice acupuncture during an ART cycle, and to establish an acupuncture protocol by consensus.
Fifteen international acupuncturists with extensive experience treating women during ART interventions participated in 3 rounds of Delphi questionnaires. The first round focused on identifying the parameters of acupuncture treatment as adjunct to ART, the second round evaluated statements derived from the earlier round, and the third evaluated specific parameters for a proposed trial protocol. Consensus was defined as greater than 80% agreement.
Significant agreement was achieved on the parameters of best practice acupuncture, including an acupuncture protocol suitable for future research. Study participants confirmed the importance of needling aspects relating to the dose of acupuncture, the therapeutic relationship, tailoring treatment to the individual, and the role of co-interventions. From two rounds of the Delphi a consensus was achieved on seven treatment parameters for the design of the acupuncture treatment to be used in a clinical trial of acupuncture as an adjunct to ART. The treatment protocol includes the use of the traditional Chinese medicine acupuncture, use of manual acupuncture, a first treatment administered between day 6–8 of the stimulated ART cycle which is individualised to the participant, two treatments will be administered on the day of embryo transfer, and will include points SP8, SP10, LR3, ST29, CV4, and post transfer include: GV20, KD3, ST36, SP6, and PC6. Auricular points Shenmen and Zigong will be used. Practitioner intent or yi will be addressed in the treatment protocol.
Despite a lack of homogeneity in the research and clinical literature on ART and acupuncture, a consensus amongst experts on key components of a best practice treatment protocol was possible. Such consensus offers guidance for further research.
PMCID: PMC3416745  PMID: 22769059
24.  How Well Do Randomized Trials Inform Decision Making: Systematic Review Using Comparative Effectiveness Research Measures on Acupuncture for Back Pain 
PLoS ONE  2012;7(2):e32399.
For Comparative Effectiveness Research (CER) there is a need to develop scales for appraisal of available clinical research. Aims were to 1) test the feasibility of applying the pragmatic-explanatory continuum indicator summary tool and the six CER defining characteristics of the Institute of Medicine to RCTs of acupuncture for treatment of low back pain, and 2) evaluate the extent to which the evidence from these RCTs is relevant to clinical and health policy decision making.
We searched Medline, the AcuTrials™ Database to February 2011 and reference lists and included full-report randomized trials in English that compared needle acupuncture with a conventional treatment in adults with non-specific acute and/or chronic low back pain and restricted to those with ≥30 patients in the acupuncture group. Papers were evaluated by 5 raters.
Principal Findings
From 119 abstracts, 44 full-text publications were screened and 10 trials (4,901 patients) were evaluated. Due to missing information and initial difficulties in operationalizing the scoring items, the first scoring revealed inter-rater and inter-item variance (intraclass correlations 0.02–0.60), which improved after consensus discussions to 0.20–1.00. The 10 trials were found to cover the efficacy-effectiveness continuum; those with more flexible acupuncture and no placebo control scored closer to effectiveness.
Both instruments proved useful, but need further development. In addition, CONSORT guidelines for reporting pragmatic trials should be expanded. Most studies in this review already reflect the movement towards CER and similar approaches can be taken to evaluate comparative effectiveness relevance of RCTs for other treatments.
PMCID: PMC3289651  PMID: 22389699
25.  The making of nursing practice Law in Lebanon: a policy analysis case study 
Evidence-informed decisions can strengthen health systems, improve health, and reduce health inequities. Despite the Beijing, Montreux, and Bamako calls for action, literature shows that research evidence is underemployed in policymaking, especially in the East Mediterranean region (EMR). Selecting the draft nursing practice law as a case study, this policy analysis exercise aims at generating in-depth insights on the public policymaking process, identifying the factors that influence policymaking and assessing to what extent evidence is used in this process.
This study utilized a qualitative research design using a case study approach and was conducted in two phases: data collection and analysis, and validation. In the first phase, data was collected through key informant interviews that covered 17 stakeholders. In the second phase, a panel discussion was organized to validate the findings, identify any gaps, and gain insights and feedback of the panelists. Thematic analysis was conducted and guided by the Walt & Gilson’s “Policy Triangle Framework” as themes were categorized into content, actors, process, and context.
Findings shed light on the complex nature of health policymaking and the unstructured approach of decision making. This study uncovered the barriers that hindered the progress of the draft nursing law and the main barriers against the use of evidence in policymaking. Findings also uncovered the risk involved in the use of international recommendations without the involvement of stakeholders and without accounting for contextual factors and implementation barriers. Findings were interpreted within the context of the Lebanese political environment and the power play between stakeholders, taking into account equity considerations.
This policy analysis exercise presents findings that are helpful for policymakers and all other stakeholders and can feed into revising the draft nursing law to reach an effective alternative that is feasible in Lebanon. Our findings are relevant in local and regional context as policymakers and other stakeholders can benefit from this experience when drafting laws and at the global context, as international organizations can consider this case study when developing global guidance and recommendations.
PMCID: PMC4163164  PMID: 25193112
Evidence-informed policymaking; Lebanon; Nursing law; Policy analysis

Results 1-25 (1309628)