PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1443598)

Clipboard (0)
None

Related Articles

1.  Orthopaedic Surgeons Prefer to Participate in Expertise-based Randomized Trials 
Empiric data and theoretical arguments suggest an alternative randomized clinical trial (RCT) design, called expertise-based RCT, has enhanced validity, applicability, and ethical integrity compared with conventional RCT. Little is known, however, about whether physicians will participate in an expertise-based RCT. In a cross-sectional survey of Canadian orthopaedic surgeons, we evaluated preference for and willingness to participate in an expertise-based versus a conventional RCT if given the opportunity to participate in a trial investigating the effectiveness of high tibial osteotomy versus unicompartmental knee arthroplasty. Using an electronic survey (©2005 SurveyMonkey.com), we invited all 767 members of the Canadian Orthopaedic Association (2005) to participate; 276 surgeons completed the questionnaire (37.5% response rate). One hundred two surgeons (53.4%) were willing to participate in an expertise-based RCT compared with 35 surgeons (18.3%) willing to participate in a conventional RCT. Ninety-seven surgeons (52.4%) strongly or moderately preferred the expertise-based design compared with 25 (13.5%) who preferred the conventional design. For the clinical example we presented, the majority of Canadian orthopaedic surgeons were willing to participate in and preferred the expertise-based design. The expertise-based randomized clinical trial design may overcome some of the barriers to conducting clinical trials in orthopaedic surgery and improve the validity of their conclusions.
doi:10.1007/s11999-008-0273-9
PMCID: PMC2505251  PMID: 18446421
2.  Implantable Cardioverter Defibrillators. Prophylactic Use 
Executive Summary
Objective
The use of implantable cardiac defibrillators (ICDs) to prevent sudden cardiac death (SCD) in patients resuscitated from cardiac arrest or documented dangerous ventricular arrhythmias (secondary prevention of SCD) is an insured service. In 2003 (before the establishment of the Ontario Health Technology Advisory Committee), the Medical Advisory Secretariat conducted a health technology policy assessment on the prophylactic use (primary prevention of SCD) of ICDs for patients at high risk of SCD. The Medical Advisory Secretariat concluded that ICDs are effective for the primary prevention of SCD. Moreover, it found that a more clearly defined target population at risk for SCD that would be likely to benefit from ICDs is needed, given that the number needed to treat (NNT) from recent studies is 13 to 18, and given that the per-unit cost of ICDs is $32,000, which means that the projected cost to Ontario is $770 million (Cdn).
Accordingly, as part of an annual review and publication of more recent articles, the Medical Advisory Secretariat updated its health technology policy assessment of ICDs.
Clinical Need
Sudden cardiac death is caused by the sudden onset of fatal arrhythmias, or abnormal heart rhythms: ventricular tachycardia (VT), a rhythm abnormality in which the ventricles cause the heart to beat too fast, and ventricular fibrillation (VF), an abnormal, rapid and erratic heart rhythm. About 80% of fatal arrhythmias are associated with ischemic heart disease, which is caused by insufficient blood flow to the heart.
Management of VT and VF with antiarrhythmic drugs is not very effective; for this reason, nonpharmacological treatments have been explored. One such treatment is the ICD.
The Technology
An ICD is a battery-powered device that, once implanted, monitors heart rhythm and can deliver an electric shock to restore normal rhythm when potentially fatal arrhythmias are detected. The use of ICDs to prevent SCD in patients resuscitated from cardiac arrest or documented dangerous ventricular arrhythmias (secondary prevention) is an insured service in Ontario.
Primary prevention of SCD involves identification of and preventive therapy for patients who are at high risk for SCD. Most of the studies in the literature that have examined the prevention of fatal ventricular arrhythmias have focused on patients with ischemic heart disease, in particular, those with heart failure (HF), which has been shown to increase the risk of SCD. The risk of HF is determined by left ventricular ejection fraction (LVEF); most studies have focused on patients with an LVEF under 0.35 or 0.30. While most studies have found ICDs to reduce significantly the risk for SCD in patients with an LVEF less than 0.35, a more recent study (Sudden Cardiac Death in Heart Failure Trial [SCD-HeFT]) reported that patients with HF with nonischemic heart disease could also benefit from this technology. Based on the generalization of the SCD-HeFT study, the Centers for Medicare and Medicaid in the United States recently announced that it would allocate $10 billion (US) annually toward the primary prevention of SCD for patients with ischemic and nonischemic heart disease and an LVEF under 0.35.
Review Strategy
The aim of this literature review was to assess the effectiveness, safety, and cost effectiveness of ICDs for the primary prevention of SCD.
The standard search strategy used by the Medical Advisory Secretariat was used. This included a search of all international health technology assessments as well as a search of the medical literature from January 2003–May 2005.
A modification of the GRADE approach (1) was used to make judgments about the quality of evidence and strength of recommendations systematically and explicitly. GRADE provides a framework for structured reflection and can help to ensure that appropriate judgments are made. GRADE takes into account a study’s design, quality, consistency, and directness in judging the quality of evidence for each outcome. The balance between benefits and harms, quality of evidence, applicability, and the certainty of the baseline risks are considered in judgments about the strength of recommendations.
Summary of Findings
Overall, ICDs are effective for the primary prevention of SCD. Three studies – the Multicentre Automatic Defibrillator Implantation Trial I (MADIT I), the Multicentre Automatic Defibrillator Implantation Trial II (MADIT II), and SCD-HeFT – showed there was a statistically significant decrease in total mortality for patients who prophylactically received an ICD compared with those who received conventional therapy (Table 1).
Results of Key Studies on the Use of Implantable Cardioverter Defibrillators for the Primary Prevention of Sudden Cardiac Death – All-Cause Mortality
MADIT I: Multicentre Automatic Defibrillator Implantation Trial I; MADIT II: Multicentre Automatic Defibrillator Implantation Trial II; SCD-HeFT: Sudden Cardiac Death in Heart Failure Trial.
EP indicates electrophysiology; ICD, implantable cardioverter defibrillator; NNT, number needed to treat; NSVT, nonsustained ventricular tachycardia. The NNT will appear higher if follow-up is short. For ICDs, the absolute benefit increases over time for at least a 5-year period; the NNT declines, often substantially, in studies with a longer follow-up. When the NNT are equalized for a similar period as the SCD-HeFT duration (5 years), the NNT for MADIT-I is 2.2; for MADIT-II, it is 6.3.
GRADE Quality of the Evidence
Using the GRADE Working Group criteria, the quality of these 3 trials was examined (Table 2).
Quality refers to the criteria such as the adequacy of allocation concealment, blinding and follow-up.
Consistency refers to the similarity of estimates of effect across studies. If there is important unexplained inconsistency in the results, our confidence in the estimate of effect for that outcome decreases. Differences in the direction of effect, the size of the differences in effect, and the significance of the differences guide the decision about whether important inconsistency exists.
Directness refers to the extent to which the people interventions and outcome measures are similar to those of interest. For example, there may be uncertainty about the directness of the evidence if the people of interest are older, sicker or have more comorbidity than those in the studies.
As stated by the GRADE Working Group, the following definitions were used to grade the quality of the evidence:
High: Further research is very unlikely to change our confidence n the estimate of effect.
Moderate: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
Low: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
Very low: Any estimate of effect is very uncertain.
Quality of Evidence – MADIT I, MADIT II, and SCD-HeFT*
MADIT I: Multicentre Automatic Defibrillator Implantation Trial I; MADIT II: Multicentre Automatic Defibrillator Implantation Trial II; SCD-HeFT: Sudden Cardiac Death in Heart Failure Trial.
The 3 trials had 3 different sets of eligibility criteria for implantation of an ICD for primary prevention of SCD. Conclusions
Conclusions
Overall, there is evidence that ICDs are effective for the primary prevention of SCD. Three trials have found a statistically significant decrease in total mortality for patients who prophylactically received an ICD compared with those who received conventional therapy in their respective study populations.
As per the GRADE Working Group, recommendations consider 4 main factors:
The tradeoffs, taking into account the estimated size of the effect for the main outcome, the confidence limits around those estimates, and the relative value placed on the outcome;
The quality of the evidence (Table 2);
Translation of the evidence into practice in a specific setting, taking into consideration important factors that could be expected to modify the size of the expected effects, such as proximity to a hospital or availability of necessary expertise; and
Uncertainty about the baseline risk for the population of interest
The GRADE Working Group also recommends that incremental costs of health care alternatives should be considered explicitly with the expected health benefits and harms. Recommendations rely on judgments about the value of the incremental health benefits in relation to the incremental costs. The last column in Table 3 is the overall trade-off between benefits and harms and incorporates any risk or uncertainty.
For MADIT I, the overall GRADE and strength of the recommendation is “moderate” – the quality of the evidence is “moderate” (uncertainty due to methodological limitations in the study design), and risk/uncertainty in cost and budget impact was mitigated by the use of filters to help target the prevalent population at risk (Table 3).
For MADIT II, the overall GRADE and strength of the recommendation is “very weak” – the quality of the evidence is “weak” (uncertainty due to methodological limitations in the study design), but there is risk or uncertainty regarding the high prevalence, cost, and budget impact. It is not clear why screening for high-risk patients was dropped, given that in MADIT II the absolute reduction in mortality was small (5.6%) compared to MADIT I, which used electrophysiological screening (23%) (Table 3).
For SCD-HeFT, the overall GRADE and strength of the recommendation is “weak” – the study quality is “moderate,” but there is also risk/uncertainty due to a high NNT at 5 years (13 compared to the MADIT II NNT of 6 and MADIT I NNT of 2 at 5 years), high prevalent population (N = 23,700), and a high budget impact ($770 million). A filter (as demonstrated in MADIT 1) is required to help target the prevalent population at risk and mitigate the risk or uncertainty relating to the high NNT, prevalence, and budget impact (Table 3).
The results of the most recent ICD trial (SCD-HeFT) are not generalizable to the prevalent population in Ontario (Table 3). Given that the current funding rate of an ICD is $32,500 (Cdn), the estimated budget impact for Ontario would be as high as $770 million (Cdn). The uncertainty around the cost estimate of treating the prevalent population with LVEF < 0.30 in Ontario, the lack of human resources to implement such a strategy and the high number of patients required to prevent one SCD (NNT = 13) calls for an alternative strategy that allows the appropriate uptake and diffusion of ICDs for primary prevention for patients at maximum risk for SCD within the SCD-HeFT population.
The uptake and diffusion of ICDs for primary prevention of SCD should therefore be based on risk stratification through the use of appropriate screen(s) that would identify patients at highest risk who could derive the most benefit from this technology.
Overall GRADE and Strength of Recommendation for the Use of Implantable Cardioverter Defibrillators for the Primary Prevention of Sudden Cardiac Death
MADIT I: Multicentre Automatic Defibrillator Implantation Trial I; MADIT II: Multicentre Automatic Defibrillator Implantation Trial II; SCD-HeFT: Sudden Cardiac Death in Heart Failure Trial.
NNT indicates number needed to treat. The NNT will appear higher if follow-up is short. For ICDs, the absolute benefit increases over time for at least a 5-year period; the NNT declines, often substantially, in studies with a longer follow-up. When the NNT are equalized for a similar period as the SCD-HeFT duration (5 years), the NNT for MADIT-I is 2.2; for MADIT-II, it is 6.3.
NSVT indicates nonsustained ventricular tachycardia; VT, ventricular tachycardia.
PMCID: PMC3382404  PMID: 23074465
3.  Managing the Personal Side of Health: How Patient Expertise Differs from the Expertise of Clinicians 
Background
When patients need health information to manage their personal health, they turn to both health professionals and other patients. Yet, we know little about how the information exchanged among patients (ie, patient expertise) contrasts with the information offered by health professionals (ie, clinician expertise). Understanding how patients’ experiential expertise contrasts with the medical expertise of health professionals is necessary to inform the design of peer-support tools that meet patients’ needs, particularly with the growing prevalence of largely unguided advice sharing through Internet-based social software.
Objective
The objective of our study was to enhance our understanding of patient expertise and to inform the design of peer-support tools. We compared the characteristics of patient expertise with that of clinician expertise for breast cancer.
Methods
Through a comparative content analysis of topics discussed and recommendations offered in Internet message boards and books, we contrasted the topic, form, and style of expertise shared in sources of patient expertise with sources of clinician expertise.
Results
Patient expertise focused on strategies for coping with day-to-day personal health issues gained through trial and error of the lived experience; thus, it was predominately personal in topic. It offered a wealth of actionable advice that was frequently expressed through the narrative style of personal stories about managing responsibilities and activities associated with family, friends, work, and the home during illness. In contrast, clinician expertise was carried through a prescriptive style and focused on explicit facts and opinions that tied closely to the health care delivery system, biomedical research, and health professionals’ work. These differences were significant between sources of patient expertise and sources of clinician expertise in topic (P < .001), form (P < .001), and style (P < .001).
Conclusion
Patients offer other patients substantial expertise that differs significantly from the expertise offered by health professionals. Our findings suggest that experienced patients do not necessarily serve as “amateur doctors” who offer more accessible but less comprehensive or detailed medical information. Rather, they offer valuable personal information that clinicians cannot necessarily provide. The characteristics of patient expertise and the resulting design implications that we identified will help informaticians enhance the design of peer-support tools that will help meet patients’ diverse information needs.
doi:10.2196/jmir.1728
PMCID: PMC3222167  PMID: 21846635
Health knowledge; attitudes; practice; social support; community networks; peer group; consumer health informatics; online communities; patient expertise; personalized health
4.  All-arthroscopic versus mini-open repair of small or moderate-sized rotator cuff tears: A protocol for a randomized trial [NCT00128076] 
Background
Rotator cuff tears are the most common source of shoulder pain and disability. Only poor quality studies have compared mini-open to arthroscopic repair, leaving surgeons with inadequate evidence to support optimal, minimally-invasive repair.
Methods/Design
This randomized, multi-centre, national trial will determine whether an arthroscopic or mini-open repair provides better quality of life for patients with small or moderate-sized rotator cuff tears. A national consensus meeting of investigators in the Joint Orthopaedic Initiative for National Trials of the Shoulder (JOINTS Canada) identified this question as the top priority for shoulder surgeons across Canada. The primary outcome measure is a valid quality-of-life scale (Western Ontario Rotator Cuff (WORC)) that addresses 5 domains of health affected by rotator cuff disease. Secondary outcomes will assess rotator cuff functionality (ROM, strength, Constant score), secondary dimensions of health (general health status (SF-12) and work limitations), and repair integrity (MRI). Outcomes are measured at baseline, at 6 weeks, 3, 6, 12, and 24 months post-operatively by blinded research assistants and musculoskeletal radiologists. Patients (n = 250) with small or medium-sized cuff tears identified by clinical examination and MRI who meet eligibility criteria will be recruited. This sample size will provide 80% power to statistically detect a clinically important difference of 20% in WORC scores between procedures after controlling for baseline WORC score (α = 0.05). A central methods centre will manage randomization, data management, and monitoring under supervision of experienced epidemiologists. Surgeons will participate in either conventional or expertise-based designs according to defined criteria to avoid biases from differential surgeon expertise. Mini-open or all-arthroscopic repair procedures will be performed according to a standardized protocol. Central Adjudication (of cases), Trial Oversight and Safety Committees will monitor trial conduct. We will use an analysis of covariance (ANCOVA), where the baseline WORC score is used as a covariate, to compare the quality of life (WORC score) at 2 years post-operatively. As a secondary analysis, we will conduct the same statistical test but will include age and tear size as covariates with the baseline score. Enrollment will require 2 years and follow-up an additional 2 years. The trial will commence when funding is in place.
Discussion
These results will have immediate impact on the practice behaviors of practicing surgeons and surgical trainees at JOINTS centres across Canada. JOINTS Canada is actively engaged in knowledge exchange and will publish and present findings internationally to facilitate wider application. This trial will establish definitive evidence on this question at an international level.
doi:10.1186/1471-2474-7-25
PMCID: PMC1421402  PMID: 16529658
5.  Open versus endovascular repair of abdominal aortic aneurysm: a survey of Canadian vascular surgeons 
Canadian Journal of Surgery  2008;51(2):142-149.
Objective
The aim of this survey was to determine Canadian vascular surgeons' experience with elective endovascular aortic repair (EVAR) and traditional open repair and their interest in participating in an expertise-based randomized controlled trial (RCT) as opposed to a conventional RCT comparing these 2 procedures.
Methods
A single-page questionnaire was developed and sent by fax, email or post to all vascular surgeons in Canada. Nonresponders were recontacted on 2 additional occasions to improve the response rate. The questionnaire had 2 sections. The first inquired about current and past practice patterns, including experience in both open and endovascular techniques. The second investigated the surgeons' belief in the value of open as opposed to endovascular repair and the value of expertise-based RCT methodology; it also canvassed their interest in participating in a future trial. Definitions of expertise in open and endovascular repair were drawn from the published literature. Criteria to determine the feasibility of conducting an expertise-based RCT were established a priori.
Results
The questionnaire was sent to 259 surgeons who appeared in multiple vascular surgery databases, and the overall response rate was 56% (95% confidence interval [CI] 50%–62%). The mean career experience was 406 cases (standard deviation [SD] 359) for conventional open abdominal aortic aneurysm (AAA) repair and 24 cases (SD 48) for endovascular repair. Of the responding surgeons, 51% (95% CI 41%–60%) ranked conventional open repair as “probably superior.” Respondents were equally interested in participating in an RCT using either expertise-based methodology (54%, 95% CI 44%–63%) or conventional design (51%, 95% CI 41%–60%).
Conclusion
Uncertainty exists among vascular surgeons in Canada as to the role of endovascular surgery in the repair of AAA. A national RCT comparing open with endovascular repair in the elective setting is potentially feasible with either expertise-based or conventional design. Increases in the number of surgeons who are willing to participate and have expertise in EVAR, in addition to high recruitment rates among eligible patients, will be necessary to make such a trial feasible in Canada.
PMCID: PMC2386348  PMID: 18377756
6.  Methodological choices for the clinical development of medical devices 
Clinical evidence available for the assessment of medical devices (MDs) is frequently insufficient. New MDs should be subjected to high quality clinical studies to demonstrate their benefit to patients. The randomized controlled trial (RCT) is the study design reaching the highest level of evidence in order to demonstrate the efficacy of a new MD. However, the clinical context of some MDs makes it difficult to carry out a conventional RCT. The objectives of this review are to present problems related to conducting conventional RCTs and to identify other experimental designs, their limitations, and their applications. A systematic literature search was conducted for the period January 2000 to July 2012 by searching medical bibliographic databases. Problems related to conducting conventional RCTs of MDs were identified: timing the assessment, eligible population and recruitment, acceptability, blinding, choice of comparator group, and learning curve. Other types of experimental designs have been described. Zelen’s design trials and randomized consent design trials facilitate the recruitment of patients, but can cause ethical problems to arise. Expertise-based RCTs involve randomization to a team that specializes in a given intervention. Sometimes, the feasibility of an expertise-based randomized trial may be greater than that of a conventional trial. Cross-over trials reduce the number of patients, but are not applicable when a learning curve is required. Sequential trials have the advantage of allowing a trial to be stopped early depending on the results of first inclusions, but they require an independent committee. Bayesian methods combine existing information with information from the ongoing trial. These methods are particularly useful in situations where the number of subjects is small. The disadvantage is the risk of including erroneous prior information. Other types of experimental designs exist when conventional trials cannot always be applied to the clinical development of MDs.
doi:10.2147/MDER.S63869
PMCID: PMC4181748  PMID: 25285025
medical device; randomized controlled trials; assessment; clinical development
7.  Optical Coherence Tomography for Age-Related Macular Degeneration and Diabetic Macular Edema 
Executive Summary
Objective
The purpose of this evidence-based review was to examine the effectiveness and cost-effectiveness of spectral-domain (SD) optical coherence tomography (OCT) in the diagnosis and monitoring of patients with retinal disease, specifically age-related macular degeneration (AMD) and diabetic macular edema (DME). Specifically, the research question addressed was:
What is the sensitivity and specificity of spectral domain OCT relative to the gold standard?
Clinical Need: Target Population and Condition
The incidence of blindness has been increasing worldwide. In Canada, vision loss in those 65 years of age and older is primarily due to AMD, while loss of vision in those 18 years of age and older is mainly due to DME. Both of these conditions are diseases of the retina, which is located at the back of the eye. At the center of the retina is the macula, a 5 mm region that is responsible for what we see in front of us, our ability to detect colour, and fine detail. Damage to the macula gives rise to vision loss, but early detection of asymptomatic disease may lead to the prevention or slowing of the vision loss process.
There are two main types of AMD, ‘dry’ and ‘wet’. Dry AMD is the more prevalent of the two, accounting for approximately 85% of cases and characterized by small deposits of extracellular material called “drusen” that build up in Bruch’s membrane of the eye. Central vision loss is gradual with blurring and eventual colour fading. Wet AMD is a less prevalent condition (15% of all AMD cases) but it accounts for 90% of severe cases. It’s characterized by the appearance of retinal fluid with vision loss due to abnormal blood vessels/leakage within weeks to months of diagnosis. In 2003, the Canadian National Institute for the Blind (CNIB) prevalence estimate for AMD was 1 million Canadians, including approximately 400,000 affected Ontarians. The incidence in 2003 was estimated to be 78,000 new cases in Canada, with approximately one-third of these cases arising in Ontario (n=26,000). Over the next 25 years, the number of new cases is expected to triple.
DME is caused by complications of diabetes mellitus, both Type 1 and Type 2. It is estimated that 1-in-4 persons with diabetes has this condition, though it occurs more frequently among those with type 2 diabetes. The condition is characterized by a swelling of the retina caused by leakage of blood vessels at the back of the eye. In early stages of the disease, vision may still be normal but it can degrade rapidly in later stages. In 2003, the CNIB prevalence estimate for DME was 0.5 million Canadians, with approximately 200,000 Ontarians affected. The incidence of DME is more difficult to ascertain; however, based on an annual incidence rate of 0.8% (for those 20 years of age or older) and the assumption that 1-in-4 persons with diabetes is affected, the incidence of DME in Ontario is estimated to be 21,000 new cases per year.
Optical Coherence Tomography
Prior to the availability of OCT, the standard of care in the diagnosis and/or monitoring of retinal disease was serial testing with fluorescein angiography (FA), biomicroscopy (BM), and stereo-fundus photography (SFP). Each of these is a qualitative measure of disease based on subjective evaluations that are largely dependent on physician expertise. OCT is the first quantitative visual test available for the diagnosis of eye disease. As such, it is allows for a more objective evaluation of the presence/absence of retinal disease and it is the only test that provides a measure of retinal thickness. The technology was developed at the Michigan Institute of Technology (MIT) in 1991 as a real-time imaging modality and is considered comparable to histology. It’s a light-wave based technology producing cross-sectional images with scan rates and resolution parameters that have greatly improved over the last 10 years. It’s also a non-invasive, non-contact visual test that requires just 3 to 5 minutes to assess both eyes.
There are two main types of OCT system, both licensed by Health Canada as class II devices. The original patent was based on a time domain (TD) system (available from 1995) that had an image rate of 100 to 400 scans per second and provided information for a limited view of the retina with a resolution in the range of 10 to 20 μm. The newer system, spectral domain (SD) OCT, has been available since 2006. Improvements with this system include (i) a faster scan speed of approximately 27,000 scans per second; (ii) the ability to scan larger areas of the retina by taking six scans radially-oriented 30 degrees from each other; (iii) increased resolution at 5μm; and (iv) ‘real-time registration,’ which was not previously available with TD.
The increased scan speed of SD systems enables the collection of additional real-time information on larger regions of the retina, thus, reducing the reliance on assumptions required for retinal thickness and volume estimates based on software algorithms. The faster scan speed also eliminates image distortion arising from patient movement (not previously possible with TD), while the improvement in resolution allows for clearer and more distinguishable retinal layers with the possibility of detecting earlier signs of disease. Real-time registration is a new feature of SD that enables the identification of specific anatomical locations on the retina, against which subsequent tests can be evaluated. This is of particular importance in the monitoring of patients. In the evaluation of treatment effects, for example, this enables the same anatomic retinal location to be identified at each visit.
Methods
Literature Search
A literature search was performed on February 13, 2009 using Ovid MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, the Cumulative Index to Nursing & Allied Health Literature (CINAHL), the Cochrane Library, and the International Agency for Health Technology Assessment (INAHTA) for studies published from January 2003 to February 2009. The subject headings and keywords searched included AMD, DME, and OCT (the detailed search strategy can be viewed in Appendix 1). Excluded were case reports, comments, editorials, non-systematic reviews, and letters. Abstacts were reviewed by a single reviewer and, for those studies meeting the eligibility criteria, full-text articles were obtained. In total, 542 articles were included for review.
English-language articles and health technology assessments.
RCTs and observational studies of OCT and AMD or DME.
Studies focusing on either diagnosis or monitoring of disease.
Studies in which outcomes were not specific to those of interest in this report.
Studies of pediatric populations.
Studies on OCT as a screening tool.
Studies that did not assess comparative effectiveness of OCT with a referent, as specified below in “Comparisons of Interest”.
Outcomes of Interest
Studies of sensitivity, specificity.
Comparisons of Interest
Evidence exists for the following comparisons of interest:
OCT compared with the reference “fluorescein angiography” for AMD.
OCT compared with the reference “biomicroscopy” or “stereo or fundus photography” for DME.
Summary of Existing Evidence
No evidence for the accuracy of SD OCT compared to either FA, BM or SFP was published between January 2006 to February 2009; however, two technology assessments were found, one from Alberta and the other from Germany, both of which contain evidence for TD OCT. Although these HTAs included eight studies each, only one study from each report was specific to this review. Additionally, one systematic review was identified for OCT and DME. It is these three articles, all pertaining to time and not spectral domain OCT, as well as comments from experts in the field of OCT and retinal disease, that comprise the evidence contained in this review.
Upon further assessment and consultations with experts in the methodology of clinical test evaluation, it was concluded that these comparators could not be used as references in the evaluation of OCT. The main conclusion was that, without a third test as an arbiter, it is not possible to directly compare the sensitivity and specificity of OCT relative to either FA for AMD and stereo- or fundus – photography for DME. Therefore, in the absence of published evidence, it was deemed appropriate to consult a panel of experts for their views and opinions on the validity of OCT and its utility in clinical settings. This panel consisted of four clinicians with expertise in AMD and/or DME and OCT, as well as a medical biophysicist with scientific expertise in ocular technologies. This is considered level 5 evidence, but in the absence of an appropriate comparator for further evaluation of OCT, this may be the highest level of evidence possible.
Summary of Findings
The conclusions for SD OCT based on Level 5 evidence, or expert consultation, are as follows:
OCT is considered an essential part of the diagnosis and follow-up of patients with DME and AMD.
OCT is adjunctive to FA for both AMD and DME but should decrease utilization of FA as a monitoring modality.
OCT will result in a decline in the use of BM in the monitoring of patients with DME, given its increased accuracy and consistency.
OCT is diffusing rapidly and the technology is changing. Since FA is still considered pivotal in the diagnosis and treatment of AMD and DME, and there is no common outcome against which to compare these technologies, it is unlikely that RCT evidence of efficacy for OCT will ever be forthcoming.
In addition to the accuracy of OCT in the detection of disease, assessment of the clinical utility of this technology included a rapid review of treatment effects for AMD and DME. The treatment of choice for AMD is Lucentis®, with or without Avastin® and photodynamic therapy. For DME the treatment of choice is laser photocoagulation, which may be replaced with Lucentis® injections (Expert consultation). The evidence, as presented in systematic reviews and other health technology assessments, indicates that there are effective treatments available for both AMD and DME.
Considerations for the Ontario Health System
OCT testing is presently an uninsured service in Ontario with patients paying approximately $150 out-of-pocket per test. Several provinces do provide funding for this procedure, including British Columbia, Alberta, Saskatchewan, Newfoundland, Nova Scotia, Prince Edward Island, and the Yukon Territory. Provinces that do not provide such funding are Quebec, Manitoba and New Brunswick.
The demand for OCT is expected to increase with aging of the population.
PMCID: PMC3377511  PMID: 23074517
8.  Timing and Completeness of Trial Results Posted at ClinicalTrials.gov and Published in Journals 
PLoS Medicine  2013;10(12):e1001566.
Agnes Dechartres and colleagues searched ClinicalTrials.gov for completed drug RCTs with results reported and then searched for corresponding studies in PubMed to evaluate timeliness and completeness of reporting.
Please see later in the article for the Editors' Summary
Background
The US Food and Drug Administration Amendments Act requires results from clinical trials of Food and Drug Administration–approved drugs to be posted at ClinicalTrials.gov within 1 y after trial completion. We compared the timing and completeness of results of drug trials posted at ClinicalTrials.gov and published in journals.
Methods and Findings
We searched ClinicalTrials.gov on March 27, 2012, for randomized controlled trials of drugs with posted results. For a random sample of these trials, we searched PubMed for corresponding publications. Data were extracted independently from ClinicalTrials.gov and from the published articles for trials with results both posted and published. We assessed the time to first public posting or publishing of results and compared the completeness of results posted at ClinicalTrials.gov versus published in journal articles. Completeness was defined as the reporting of all key elements, according to three experts, for the flow of participants, efficacy results, adverse events, and serious adverse events (e.g., for adverse events, reporting of the number of adverse events per arm, without restriction to statistically significant differences between arms for all randomized patients or for those who received at least one treatment dose).
From the 600 trials with results posted at ClinicalTrials.gov, we randomly sampled 50% (n = 297) had no corresponding published article. For trials with both posted and published results (n = 202), the median time between primary completion date and first results publicly posted was 19 mo (first quartile = 14, third quartile = 30 mo), and the median time between primary completion date and journal publication was 21 mo (first quartile = 14, third quartile = 28 mo). Reporting was significantly more complete at ClinicalTrials.gov than in the published article for the flow of participants (64% versus 48% of trials, p<0.001), efficacy results (79% versus 69%, p = 0.02), adverse events (73% versus 45%, p<0.001), and serious adverse events (99% versus 63%, p<0.001).
The main study limitation was that we considered only the publication describing the results for the primary outcomes.
Conclusions
Our results highlight the need to search ClinicalTrials.gov for both unpublished and published trials. Trial results, especially serious adverse events, are more completely reported at ClinicalTrials.gov than in the published article.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
When patients consult a doctor, they expect to be recommended what their doctor believes is the most effective treatment with the fewest adverse effects. To determine which treatment to recommend, clinicians rely on sources that include research studies. Among studies, the best evidence is generally agreed to come from systematic reviews and randomized controlled clinical trials (RCTs), studies that test the efficacy and safety of medical interventions by comparing clinical outcomes in groups of patients randomly chosen to receive different interventions. Decision-making based on the best available evidence is called evidence-based medicine. However, evidence-based medicine can only guide clinicians if trial results are published in a timely and complete manner. Unfortunately, underreporting of trials is common. For example, an RCT in which a new drug performs better than existing drugs is more likely to be published than one in which the new drug performs badly or has unwanted adverse effects (publication bias). There can also be a delay in publishing the results of negative trials (time-lag bias) or a failure to publish complete results for all the prespecified outcomes of a trial (reporting bias). All three types of bias threaten informed medical decision-making and the health of patients.
Why Was This Study Done?
One initiative that aims to prevent these biases was included in the 2007 US Food and Drug Administration Amendments Act (FDAAA). The Food and Drug Administration (FDA) is responsible for approving drugs and devices that are marketed in the US. The FDAAA requires that results from clinical trials of FDA-approved drugs and devices conducted in the United States be made publicly available at ClinicalTrials.gov within one year of trial completion. ClinicalTrials.gov—a web-based registry that includes US and international clinical trials—was established in 2000 in response to the 1997 FDA Modernization Act, which required mandatory registration of trial titles and designs and of the conditions and interventions under study. The FDAAA expanded these mandatory requirements by requiring researchers studying FDA-approved drugs and devices to report additional information such as the baseline characteristics of the participants in each arm of the trial and the results of primary and secondary outcome measures (the effects of the intervention on predefined clinical measurements) and their statistical significance (an indication of whether differences in outcomes might have happened by chance). Researchers of other trials registered in ClinicalTrials.gov are welcome to post trial results as well. Here, the researchers compare the timing and completeness (i.e., whether all relevant information was fully reported) of results of drug trials posted at ClinicalTrials.gov with those published in medical journals.
What Did the Researchers Do and Find?
The researchers searched ClinicalTrials.gov for reports of completed phase III and IV (late-stage) RCTs of drugs with posted results. For a random sample of 600 eligible trials, they searched PubMed (a database of biomedical publications) for corresponding publications. Only 50% of trials with results posted at ClinicalTrials.gov had a matching published article. For 202 trials with both posted and published results, the researchers compared the timing and completeness of the results posted at ClinicalTrials.gov and of results reported in the corresponding journal publication. The median time between the study completion date and the first results being publicly posted at ClinicalTrials.gov was 19 months, whereas the time between completion and publication in a journal was 21 months. The flow of participants through trials was completely reported in 64% of the ClinicalTrials.gov postings but in only 48% of the corresponding publications. Results for the primary outcome measure were completely reported in 79% and 69% of the ClinicalTrials.gov postings and corresponding publications, respectively. Finally, adverse events were completely reported in 73% of the ClinicalTrials.gov postings but in only 45% of the corresponding publications, and serious adverse events were reported in 99% and 63% of the ClinicalTrials.gov postings and corresponding publications, respectively.
What Do These Findings Mean?
These findings suggest that the reporting of trial results is significantly more complete at ClinicalTrials.gov than in published journal articles reporting the main trial results. Certain aspects of this study may affect the accuracy of this conclusion. For example, the researchers compared the results posted at ClinicalTrials.gov only with the results in the publication that described the primary outcome of each trial, even though some trials had multiple publications. Importantly, these findings suggest that, to enable patients and physicians to make informed treatment decisions, experts undertaking assessments of drugs should consider seeking efficacy and safety data posted at ClinicalTrials.gov, both for trials whose results are not published yet and for trials whose results are published. Moreover, they suggest that the use of templates to guide standardized reporting of trial results in journals and broader mandatory posting of results may help to improve the reporting and transparency of clinical trials and, consequently, the evidence available to inform treatment of patients.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001566.
Wikipedia has pages on evidence-based medicine and on publication bias (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
The US Food and Drug Administration provides information about drug approval in the US for consumers and health-care professionals, plus detailed information on the 2007 Food and Drug Administration Amendments Act
ClinicalTrials.gov provides information about the US National Institutes of Health clinical trial registry, including background information about clinical trials, and a fact sheet detailing the requirements of the 2007 Food and Drug Administration Amendments Act
PLOS Medicine recently launched a Reporting Guidelines Collection, an open access collection of reporting guidelines, commentary, and related research on guidelines from across PLOS journals that aims to help advance the efficiency, effectiveness, and equitability of the dissemination of biomedical information; a 2008 PLOS Medicine editorial discusses the 2007 Food and Drug Administration Amendments Act
doi:10.1371/journal.pmed.1001566
PMCID: PMC3849189  PMID: 24311990
9.  The Effectiveness of Mobile-Health Technologies to Improve Health Care Service Delivery Processes: A Systematic Review and Meta-Analysis 
PLoS Medicine  2013;10(1):e1001363.
Caroline Free and colleagues systematically review controlled trials of mobile technology interventions to improve health care delivery processes and show that current interventions give only modest benefits and that high-quality trials measuring clinical outcomes are needed.
Background
Mobile health interventions could have beneficial effects on health care delivery processes. We aimed to conduct a systematic review of controlled trials of mobile technology interventions to improve health care delivery processes.
Methods and Findings
We searched for all controlled trials of mobile technology based health interventions using MEDLINE, EMBASE, PsycINFO, Global Health, Web of Science, Cochrane Library, UK NHS HTA (Jan 1990–Sept 2010). Two authors independently extracted data on allocation concealment, allocation sequence, blinding, completeness of follow-up, and measures of effect. We calculated effect estimates and we used random effects meta-analysis to give pooled estimates.
We identified 42 trials. None of the trials had low risk of bias. Seven trials of health care provider support reported 25 outcomes regarding appropriate disease management, of which 11 showed statistically significant benefits. One trial reported a statistically significant improvement in nurse/surgeon communication using mobile phones. Two trials reported statistically significant reductions in correct diagnoses using mobile technology photos compared to gold standard. The pooled effect on appointment attendance using text message (short message service or SMS) reminders versus no reminder was increased, with a relative risk (RR) of 1.06 (95% CI 1.05–1.07, I2 = 6%). The pooled effects on the number of cancelled appointments was not significantly increased RR 1.08 (95% CI 0.89–1.30). There was no difference in attendance using SMS reminders versus other reminders (RR 0.98, 95% CI 0.94–1.02, respectively). To address the limitation of the older search, we also reviewed more recent literature.
Conclusions
The results for health care provider support interventions on diagnosis and management outcomes are generally consistent with modest benefits. Trials using mobile technology-based photos reported reductions in correct diagnoses when compared to the gold standard. SMS appointment reminders have modest benefits and may be appropriate for implementation. High quality trials measuring clinical outcomes are needed.
Please see later in the article for the Editors' Summary
Editors’ Summary
Background
Over the past few decades, computing and communication technologies have changed dramatically. Bulky, slow computers have been replaced by portable devices that can complete increasingly complex tasks in less and less time. Similarly, landlines have been replaced by mobile phones and other mobile communication technologies that can connect people anytime and anywhere, and that can transmit text messages (short message service; SMS), photographs, and data at the touch of a button. These advances have led to the development of mobile-health (mHealth)—the use of mobile computing and communication technologies in health care and public health. mHealth has many applications. It can be used to facilitate data collection and to encourage health-care consumers to adopt healthy lifestyles or to self-manage chronic conditions. It can also be used to improve health-care service delivery processes by targeting health-care providers or communication between these providers and their patients. So, for example, mobile technologies can be used to provide clinical management support in settings where there are no specialist clinicians, and they can be used to send patients test results and timely reminders of appointments.
Why Was This Study Done?
Many experts believe that mHealth interventions could greatly improve health-care delivery processes, particularly in resource-poor settings. The results of several controlled trials (studies that compare the outcomes of people who do or do not receive an intervention) of mHealth interventions designed to improve health-care delivery processes have been published. However, these data have not been comprehensively reviewed, and the effectiveness of this type of mHealth intervention has not been quantified. Here, the researchers rectify this situation by undertaking a systematic review and meta-analysis of controlled trials of mobile technology-based interventions designed to improve health-care service delivery processes. A systematic review is a study that uses predefined criteria to identify all the research on a given topic; a meta-analysis is a statistical approach that is used to pool the results of several independent studies.
What Did the Researchers Do and Find?
The researchers identified 42 controlled trials that investigated mobile technology-based interventions designed to improve health-care service delivery processes. None of the trials were of high quality—many had methodological problems likely to affect the accuracy of their findings—and nearly all were undertaken in high-income countries. Thirty-two of the trials tested interventions directed at health-care providers. Of these trials, seven investigated interventions providing health-care provider education, 18 investigated interventions supporting clinical diagnosis and treatment, and seven investigated interventions to facilitate communication between health-care providers. Several of the trials reported that the tested intervention led to statistically significant improvements (improvements unlikely to have happened by chance) in outcomes related to disease management. However, two trials that used mobile phones to transmit photos to off-site clinicians for diagnosis reported significant reductions in correct diagnoses compared to diagnosis by an on-site specialist. Ten of the 42 trials investigated interventions targeting communication between health-care providers and patients. Eight of these trials investigated SMS-based appointment reminders. Meta-analyses of the results of these trials indicated that using SMS appointment reminders significantly but modestly increased patient attendance compared to no reminders. However, SMS reminders were no more effective than postal or phone call reminders, and texting reminders to patients who persistently missed appointments did not significantly change the number of cancelled appointments.
What Do These Findings Mean?
These findings indicate that some mHealth interventions designed to improve health-care service delivery processes are modestly effective, but they also highlight the need for more trials of these interventions. Specifically, these findings show that although some interventions designed to provide support for health-care providers modestly improved some aspects of clinical diagnosis and management, other interventions had deleterious effects—most notably, the use of mobile technology–based photos for diagnosis. In terms of mHealth interventions targeting communication between health-care providers and patients, the finding that SMS appointment reminders have modest benefits suggests that implementation of this intervention should be considered, at least in high-income settings. However, the researchers stress that more trials are needed to robustly establish the ability of mobile technology-based interventions to improve health-care delivery processes. These trials need to be of high quality, they should be undertaken in resource-limited settings as well as in high-income countries, and, ideally, they should consider interventions that combine mHealth and conventional approaches.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001363.
A related PLOS Medicine Research Article by Free et al. investigates the effectiveness of mHealth technology-based health behavior change and disease management interventions for health-care consumers
Wikipedia has a page on mHealth (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
mHealth: New horizons for health through mobile technologies is a global survey of mHealth prepared by the World Health Organization’s Global Observatory for eHealth (eHealth is health-care practice supported by electronic processes and communication)
The mHealth in Low-Resource Settings website, which is maintained by the Netherlands Royal Tropical Institute, provides information on the current use, potential, and limitations of mHealth in low-resource settings
The US National Institutes of Health Fogarty International Center provides links to resources and information about mHealth
doi:10.1371/journal.pmed.1001363
PMCID: PMC3566926  PMID: 23458994
10.  Towards evidence based medicine for paediatricians 
In order to give the best care to patients and families, paediatricians need to integrate the highest quality scientific evidence with clinical expertise and the opinions of the family.1Archimedes seeks to assist practising clinicians by providing “evidence‐based” answers to common questions which are not at the forefront of research but are at the core of practice. In doing this, we are adapting a format which has been successfully developed by Kevin Macaway‐Jones and the group at the Emergency Medicine Journal—“BestBets”.
A word of warning. The topic summaries are not systematic reviews, though they are as exhaustive as a practising clinician can produce. They make no attempt to statistically aggregate the data, nor search the grey, unpublished literature. What Archimedes offers are practical, best evidence‐based answers to practical, clinical questions.
The format of Archimedes may be familiar. A description of the clinical setting is followed by a structured clinical question. (These aid in focusing the mind, assisting searching2 and gaining answers.3) A brief report of the search used follows—this has been performed in a hierarchical way, to search for the best‐quality evidence to answer the question. (http://www.cebm.net). A table provides a summary of the evidence and key points of the critical appraisal. For further information on critical appraisal, and the measures of effect (such as number needed to treat), books by Sackett et al4 and Moyer et al5 may help. To pull the information together, a commentary is provided. But to make it all much more accessible, a box provides the clinical bottom lines.
Electronic‐only topics that have been published on the BestBets site (www.bestbets.org) and may be of interest to paediatricians include:
When is a second course of indomethacin effective for PDA in neonates?
Does delayed cord clamping prevent sepsis?
Readers wishing to submit their own questions—with best evidence answers—are encouraged to review those already proposed at www.bestbets.org. If your question still hasn't been answered, feel free to submit your summary according to the Instructions for Authors at www.archdischild.com. Three topics are covered in this issue of the journal:
In children aged <3 years does procalcitonin help exclude serious bacterial infection in fever without focus?
Does avoidance of breast feeding reduce mother‐to‐infant transmission of hepatitis C virus infection?
Should children under treatment for juvenile idiopathic arthritis receive flu vaccination?
CAN gambling with other people's children
When we use tests to “rule out” a condition, we generally accept that we are left with a small risk of being wrong. (I think we have all discharged a child with an “upper respiratory tract infection” on a Friday to be greeted with them on antibiotics for pneumonia the following Monday.) How much faith we place in a test result is a product of two things: our initial assumption about the likelihood of the diagnosis (pretest probability) and our opinion as to how effective the test is (accuracy), but our actions do not just reflect these factors.
For instance, a well, afebrile child with a scattering of petechiae over its wrist 8 hours before, is unlikely to have meningococcal disease. If you perform a couple of tests, you can find that it has a low C‐reactive protein and a normal full blood count. What we do with this varies widely; some people would treat this with 48 h of antibiotics, others would discharge the patient home.
It is interesting to reflect on two things: first, what chance of meningococcal disease would you put on this clinical picture (before the test), and what about with the test results? What about your colleagues? You may be surprised by how widely this varies. Second, even those who have the same estimates of risk of disease may have different preferred actions (depending on their attitude to risk).
In looking at the diagnostic test for the ruling out of a disease, we can make our arguments more useful by having some data on the assumptions we make, and then transparently discussing our attitudes to risk. It is only after doing this that we can really decide if a test is good enough for us, regardless of how accurate it might be.
References
1Moyer VA, Ellior EJ. Preface. In: Moyer VA, Elliott EJ, Davis RL, et al, eds. Evidence based pediatrics and child health, Issue 1. London: BMJ Books, 2000.
2Richardson WS, Wilson MC, Nishikawa J, et al. The well‐built clinical question: a key to evidence‐based decisions. ACP J Club 1995;123:A12–13.
3Bergus GR, Randall CS, Sinift SD, et al. Does the structure of clinical questions affect the outcome of curbside consultations with specialty colleagues? Arch Fam Med 2000;9:541–7.
4Sackett DL, Starus S, Richardson WS, et al. Evidence‐based medicine. How to practice and teach EBM. San Diego: Harcourt‐Brace, 2000.
5Moyer VA, Elliott EJ, Davis RL, et al, eds. Evidence based pediatrics and child health. Issue 1. London: BMJ Books, 2000.
PMCID: PMC2083694  PMID: 17376947
11.  Airway Clearance Devices for Cystic Fibrosis 
Executive Summary
Objective
The purpose of this evidence-based analysis is to examine the safety and efficacy of airway clearance devices (ACDs) for cystic fibrosis and attempt to differentiate between devices, where possible, on grounds of clinical efficacy, quality of life, safety and/or patient preference.
Background
Cystic fibrosis (CF) is a common, inherited, life-limiting disease that affects multiple systems of the human body. Respiratory dysfunction is the primary complication and leading cause of death due to CF. CF causes abnormal mucus secretion in the airways, leading to airway obstruction and mucus plugging, which in turn can lead to bacterial infection and further mucous production. Over time, this almost cyclical process contributes to severe airway damage and loss of respiratory function. Removal of airway secretions, termed airway clearance, is thus an integral component of the management of CF.
A variety of methods are available for airway clearance, some requiring mechanical devices, others physical manipulation of the body (e.g. physiotherapy). Conventional chest physiotherapy (CCPT), through the assistance of a caregiver, is the current standard of care for achieving airway clearance, particularly in young patients up to the ages of six or seven. CF patients are, however, living much longer now than in decades past. The median age of survival in Canada has risen to 37.0 years for the period of 1998-2002 (5-year window), up from 22.8 years for the 5-year window ending in 1977. The prevalence has also risen accordingly, last recorded as 3,453 in Canada in 2002, up from 1,630 in 1977. With individuals living longer, there is a greater need for independent methods of airway clearance.
Airway Clearance Devices
There are at least three classes of airway clearance devices: positive expiratory pressure devices (PEP), airway oscillating devices (AOD; either handheld or stationary) and high frequency chest compression (HFCC)/mechanical percussion (MP) devices. Within these classes are numerous different brands of devices from various manufacturers, each with subtle iterations. At least 10 devices are licensed by Health Canada (ranging from Class 1 to Class 3 devices).
Evidence-Based Analysis of Effectiveness
Research Questions
Does long-term use of ACDs improve outcomes of interest in comparison to CCPT in patients with CF?
Does long-term use of one class of ACD improve outcomes of interest in comparison to another class of ACD in CF patients?
Literature Search
A comprehensive literature search was performed on March 7, 2009 using OVID MEDLINE, MEDLINE In-Process and Other Non-Indexed Citations, EMBASE, the Cumulative Index to Nursing & Allied Health Literature (CINAHL), the Cochrane Library, and the International Agency for Health Technology Assessment (INAHTA) for studies published from January 1, 1950 to March 7, 2009.
Inclusion Criteria
All randomized controlled trials including those of parallel and crossover design,
Systematic reviews and/or meta-analyses. Randomized controlled trials (RCTs), systematic reviews and meta-analyses
Exclusion Criteria
Abstracts were generally excluded because their methods could not be examined; however, abstract data was included in several Cochrane meta-analyses presented in this paper;
Studies of less than seven days duration (including single treatment studies);
Studies that did not report primary outcomes;
Studies in which less than 10 patients completed the study.
Outcomes of Interest
Primary outcomes under review were percent-predicted forced expiratory volume (FEV-1), forced vital capacity (FVC), and forced expiratory flow between 25%-75% (FEF25-75). Secondary outcomes included number of hospitalizations, adherence, patient preference, quality of life and adverse events. All outcomes were decided a priori.
Summary of Findings
Literature searching and back-searching identified 13 RCTs meeting the inclusion criteria, along with three Cochrane systematic reviews. The Cochrane reviews were identified in preliminary searching and used as the basis for formulating this review. Results were subgrouped by comparison and according to the available literature. For example, results from Cochrane meta-analyses included abstract data and therefore, additional meta-analyses were also performed on trials reported as full publications only (MAS generally excludes abstracted data when full publications are available as the methodological quality of trials reported in abstract cannot be properly assessed).
Executive Summary Table 1 summarizes the results across all comparisons and subgroupings for primary outcomes of pulmonary function. Only two comparisons yielded evidence of moderate or high quality according to GRADE criteria–the comparisons of CCPT vs. PEP and handheld AOD vs. PEP–but only the comparison of CCPT vs. PEP noted a significant difference between treatment groups. In comparison to CCPT, there was a significant difference in favour of PEP for % predicted FEV-1 and FVC according to one long-term, parallel RCT. This trial was accepted as the best available evidence for the comparison. The body of evidence for the remaining comparisons was low to very low, according to GRADE criteria, being downgraded most often because of poor methodological quality and low generalizability. Specifically, trials were likely not adequately powered (low sample sizes), did not conduct intention-to-treat analyses, were conducted primarily in children and young adolescents, and outdated (conducted more than 10 years ago).
Secondary outcomes were poorly or inconsistently reported, and were generally not of value to decision-making. Of note, there were a significantly higher number of hospitalizations among participants undergoing AOD therapy in comparison to PEP therapy.
Summarization of results for primary outcomes by comparison and subgroupings
Bolding indicates significant difference
Positive summary statistics favour the former intervention
Abbreviations: AOD, airway oscillating device; CCPT, conventional chest physiotherapy; CI, confidence interval; HFCC, high frequency chest compression; MP, mechanical percussion; N/A: not applicable; PEP, positive expiratory pressure
Economic Analysis
Devices ranged in cost from around $60 for PEP and handheld AODs to upwards of $18,000 for a HFCC vest device. Although the majority of device costs are paid out-of-pocket by the patients themselves, their parents, or covered by third-party medical insurance, Ontario did provide funding assistance through the Assistive Devices Program (ADP) for postural drainage boards and MP devices. These technologies, however, are either obsolete or their clinical efficacy is not supported by evidence. ADP provided roughly $16,000 in funding for the 2008/09 fiscal year. Using device costs and prevalent and incident cases of CF in Ontario, budget impact projections were generated for Ontario. Prevalence of CF in Ontario for patients from ages 6 to 71 was cited as 1,047 cases in 2002 while incidence was estimated at 46 new cases of CF diagnosed per year in 2002. Budget impact projections indicated that PEP and handheld AODs were highly economically feasible costing around $90,000 for the entire prevalent population and less than $3,000 per year to cover new incident cases. HFCC vest devices were by far the most expensive, costing in excess of $19 million to cover the prevalent population alone.
Conclusions
There is currently a lack of sufficiently powered, long-term, parallel randomized controlled trials investigating the use of ACDs in comparison to other airway clearance techniques. While much of the current evidence suggests no significant difference between various ACDs and alternative therapies/technologies, at least according to outcomes of pulmonary function, there is a strong possibility that past trials were not sufficiently powered to identify a difference. Unfortunately, it is unlikely that there will be any future trials comparing ACDs to CCPT as withholding therapy using an ACD may be seen as unethical at present.
Conclusions of clinical effectiveness are as follows:
Moderate quality evidence suggests that PEP is at least as effective as or more effective than CCPT, according to primary outcomes of pulmonary function.
Moderate quality evidence suggests that there is no significant difference between PEP and handheld AODs, according to primary outcomes of pulmonary function; however, secondary outcomes may favour PEP.
Low quality evidence suggests that there is no significant difference between AODs or HFCC/MP and CCPT, according to both primary and secondary outcomes.
Very low quality evidence suggests that there is no significant difference between handheld AOD and CCPT, according to primary outcomes of pulmonary function.
Budget impact projections show PEP and handheld AODs to be highly economically feasible.
PMCID: PMC3377547  PMID: 23074531
12.  Factors Associated with Findings of Published Trials of Drug–Drug Comparisons: Why Some Statins Appear More Efficacious than Others 
PLoS Medicine  2007;4(6):e184.
Background
Published pharmaceutical industry–sponsored trials are more likely than non-industry-sponsored trials to report results and conclusions that favor drug over placebo. Little is known about potential biases in drug–drug comparisons. This study examined associations between research funding source, study design characteristics aimed at reducing bias, and other factors that potentially influence results and conclusions in randomized controlled trials (RCTs) of statin–drug comparisons.
Methods and Findings
This is a cross-sectional study of 192 published RCTs comparing a statin drug to another statin drug or non-statin drug. Data on concealment of allocation, selection bias, blinding, sample size, disclosed funding source, financial ties of authors, results for primary outcomes, and author conclusions were extracted by two coders (weighted kappa 0.80 to 0.97). Univariate and multivariate logistic regression identified associations between independent variables and favorable results and conclusions. Of the RCTs, 50% (95/192) were funded by industry, and 37% (70/192) did not disclose any funding source. Looking at the totality of available evidence, we found that almost all studies (98%, 189/192) used only surrogate outcome measures. Moreover, study design weaknesses common to published statin–drug comparisons included inadequate blinding, lack of concealment of allocation, poor follow-up, and lack of intention-to-treat analyses. In multivariate analysis of the full sample, trials with adequate blinding were less likely to report results favoring the test drug, and sample size was associated with favorable conclusions when controlling for other factors. In multivariate analysis of industry-funded RCTs, funding from the test drug company was associated with results (odds ratio = 20.16 [95% confidence interval 4.37–92.98], p < 0.001) and conclusions (odds ratio = 34.55 [95% confidence interval 7.09–168.4], p < 0.001) that favor the test drug when controlling for other factors. Studies with adequate blinding were less likely to report statistically significant results favoring the test drug.
Conclusions
RCTs of head-to-head comparisons of statins with other drugs are more likely to report results and conclusions favoring the sponsor's product compared to the comparator drug. This bias in drug–drug comparison trials should be considered when making decisions regarding drug choice.
Lisa Bero and colleagues found published trials comparing one statin with another were more likely to report results and conclusions favoring the sponsor's product than the comparison drug.
Editors' Summary
Background.
Randomized controlled trials are generally considered to be the most reliable type of experimental study for evaluating the effectiveness of different treatments. Randomization involves the assignment of participants in the trial to different treatment groups by the play of chance. Properly done, this procedure means that the different groups are comparable at outset, reducing the chance that outside factors could be responsible for treatment effects seen in the trial. When done properly, randomization also ensures that the clinicians recruiting participants into the trial cannot know the treatment group to which a patient will end up being assigned. However, despite these advantages, a large number of factors can still result in bias creeping in. Bias comes about when the findings of research appear to differ in some systematic way from the true result. Other research studies have suggested that funding is a source of bias; studies sponsored by drug companies seem to more often favor the sponsor's drug than trials not sponsored by drug companies
Why Was This Study Done?
The researchers wanted to more precisely understand the impact of different possible sources of bias in the findings of randomized controlled trials. In particular, they wanted to study the outcomes of “head-to-head” drug comparison studies for one particular class of drugs, the statins. Drugs in this class are commonly prescribed to reduce the levels of cholesterol in blood amongst people who are at risk of heart and other types of disease. This drug class is a good example for studying the role of bias in drug–drug comparison trials, because these trials are extensively used in decision making by health-policy makers.
What Did the Researchers Do and Find?
This research study was based on searching PubMed, a biomedical literature database, with the aim of finding all randomized controlled trials of statins carried out between January 1999 and May 2005 (reference lists also were searched). Only trials which compared one statin to another statin or one statin to another type of drug were included. The researchers extracted the following information from each article: the study's source of funding, aspects of study design, the overall results, and the authors' conclusions. The results were categorized to show whether the findings were favorable to the test drug (the newer statin), inconclusive, or not favorable to the test drug. Aspects of each study's design were also categorized in relation to various features, such as how well the randomization was done (in particular, the degree to which the processes used would have prevented physicians from knowing which treatment a patient was likely to receive on enrollment); whether all participants enrolled in the trial were eventually analyzed; and whether investigators or participants knew what treatment an individual was receiving.
One hundred and ninety-two trials were included in this study, and of these, 95 declared drug company funding; 23 declared government or other nonprofit funding while 74 did not declare funding or were not funded. Trials that were properly blinded (where participants and investigators did not know what treatment an individual received) were less likely to have conclusions favoring the test drug. However, large trials were more likely to favor the test drug than smaller trials. When looking specifically at the trials funded by drug companies, the researchers found various factors that predicted whether a result or conclusion favored the test drug. These included the impact of the journal publishing the results; the size of the trial; and whether funding came from the maker of the test drug. However, properly blinded trials were less likely to produce results favoring the test drug. Even once all other factors were accounted for, the funding source for the study was still linked with results and conclusions that favored the maker of the test drug.
What Do These Findings Mean?
This study shows that the type of sponsorship available for randomized controlled trials of statins was strongly linked to the results and conclusions of those studies, even when other factors were taken into account. However, it is not clear from this study why sponsorship has such a strong link to the overall findings. There are many possible reasons why this might be. Some people have suggested that drug companies may deliberately choose lower dosages for the comparison drug when they carry out “head-to-head” trials; this tactic is likely to result in the company's product doing better in the trial. Others have suggested that trials which produce unfavorable results are not published, or that unfavorable outcomes are suppressed. Whatever the reasons for these findings, the implications are important, and suggest that the evidence base relating to statins may be substantially biased.
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0040184.
The James Lind Library has been created to help people understand fair tests of treatments in health care by illustrating how fair tests have developed over the centuries
The International Committee of Medical Journal Editors has provided guidance regarding sponsorship, authorship, and accountability
The CONSORT statement is a research tool that provides an evidence-based approach for reporting the results of randomized controlled trials
Good Publication Practice guidelines provide standards for responsible publication of research sponsored by pharmaceutical companies
Information from Wikipedia on Statins. Wikipedia is an internet encyclopedia anyone can edit
doi:10.1371/journal.pmed.0040184
PMCID: PMC1885451  PMID: 17550302
13.  A Comparison of Cost Effectiveness Using Data from Randomized Trials or Actual Clinical Practice: Selective Cox-2 Inhibitors as an Example 
PLoS Medicine  2009;6(12):e1000194.
Tjeerd-Pieter van Staa and colleagues estimate the likely cost effectiveness of selective Cox-2 inhibitors prescribed during routine clinical practice, as compared to the cost effectiveness predicted from randomized controlled trial data.
Background
Data on absolute risks of outcomes and patterns of drug use in cost-effectiveness analyses are often based on randomised clinical trials (RCTs). The objective of this study was to evaluate the external validity of published cost-effectiveness studies by comparing the data used in these studies (typically based on RCTs) to observational data from actual clinical practice. Selective Cox-2 inhibitors (coxibs) were used as an example.
Methods and Findings
The UK General Practice Research Database (GPRD) was used to estimate the exposure characteristics and individual probabilities of upper gastrointestinal (GI) events during current exposure to nonsteroidal anti-inflammatory drugs (NSAIDs) or coxibs. A basic cost-effectiveness model was developed evaluating two alternative strategies: prescription of a conventional NSAID or coxib. Outcomes included upper GI events as recorded in GPRD and hospitalisation for upper GI events recorded in the national registry of hospitalisations (Hospital Episode Statistics) linked to GPRD. Prescription costs were based on the prescribed number of tables as recorded in GPRD and the 2006 cost data from the British National Formulary. The study population included over 1 million patients prescribed conventional NSAIDs or coxibs. Only a minority of patients used the drugs long-term and daily (34.5% of conventional NSAIDs and 44.2% of coxibs), whereas coxib RCTs required daily use for at least 6–9 months. The mean cost of preventing one upper GI event as recorded in GPRD was US$104k (ranging from US$64k with long-term daily use to US$182k with intermittent use) and US$298k for hospitalizations. The mean costs (for GPRD events) over calendar time were US$58k during 1990–1993 and US$174k during 2002–2005. Using RCT data rather than GPRD data for event probabilities, the mean cost was US$16k with the VIGOR RCT and US$20k with the CLASS RCT.
Conclusions
The published cost-effectiveness analyses of coxibs lacked external validity, did not represent patients in actual clinical practice, and should not have been used to inform prescribing policies. External validity should be an explicit requirement for cost-effectiveness analyses.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Before a new treatment for a specific disease becomes an established part of clinical practice, it goes through a long process of development and clinical testing. This process starts with extensive studies of the new treatment in the laboratory and in animals and then moves into clinical trials. The most important of these trials are randomized controlled trials (RCTs), studies in which the efficacy and safety of the new drug and an established drug are compared by giving the two drugs to randomized groups of patients with the disease. The final hurdle that a drug or any other healthcare technology often has to jump before being adopted for widespread clinical use is a health technology assessment, which aims to provide policymakers, clinicians, and patients with information about the balance between the clinical and financial costs of the drug and its benefits (its cost-effectiveness). In England and Wales, for example, the National Institute for Health and Clinical Excellence (NICE), which promotes clinical excellence and the effective use of resources within the National Health Service, routinely commissions such assessments.
Why Was This Study Done?
Data on the risks of various outcomes associated with a new treatment are needed for cost-effectiveness analyses. These data are usually obtained from RCTs, but although RCTs are the best way of determining a drug's potency in experienced hands under ideal conditions (its efficacy), they may not be a good way to determine a drug's success in an average clinical setting (its effectiveness). In this study, the researchers compare the data from RCTs that have been used in several published cost-effectiveness analyses of a class of drugs called selective cyclooxygenase-2 inhibitors (“coxibs”) with observational data from actual clinical practice. They then ask whether the published cost-effectiveness studies, which generally used RCT data, should have been used to inform coxib prescribing policies. Coxibs are nonsteroidal anti-inflammatory drugs (NSAIDs) that were developed in the 1990s to treat arthritis and other chronic inflammatory conditions. Conventional NSAIDs can cause gastric ulcers and bleeding from the gut (upper gastrointestinal events) if taken for a long time. The use of coxibs avoids this problem.
What Did the Researchers Do and Find?
The researchers extracted data on the real-life use of conventional NSAIDs and coxibs and on the incidence of upper gastrointestinal events from the UK General Practice Research Database (GPRD) and from the national registry of hospitalizations. Only a minority of the million patients who were prescribed conventional NSAIDs (average cost per prescription US$17.80) or coxibs (average cost per prescription US$47.04) for a variety of inflammatory conditions took them on a long-term daily basis, whereas in the RCTs of coxibs, patients with a few carefully defined conditions took NSAIDs daily for at least 6–9 months. The researchers then developed a cost-effectiveness model to evaluate the costs of the alternative strategies of prescribing a conventional NSAID or a coxib. The mean additional cost of preventing one gastrointestinal event recorded in the GPRD by using a coxib instead of a NSAID, they report, was US$104,000; the mean cost of preventing one hospitalization for such an event was US$298,000. By contrast, the mean cost of preventing one gastrointestinal event by using a coxib instead of a NSAID calculated from data obtained in RCTs was about US$20,000.
What Do These Findings Mean?
These findings suggest that the published cost-effectiveness analyses of coxibs greatly underestimate the cost of preventing gastrointestinal events by replacing prescriptions of conventional NSAIDs with prescriptions of coxibs. That is, if data from actual clinical practice had been used in cost-effectiveness analyses rather than data from RCTs, the conclusions of the published cost-effectiveness analyses of coxibs would have been radically different and may have led to different prescribing guidelines for this class of drug. More generally, these findings provide a good illustration of how important it is to ensure that cost-effectiveness analyses have “external” validity by using realistic estimates for event rates and costs rather than relying on data from RCTs that do not always reflect the real-world situation. The researchers suggest, therefore, that health technology assessments should move from evaluating cost-efficacy in ideal populations with ideal interventions to evaluating cost-effectiveness in real populations with real interventions.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000194.
The UK National Institute for Health Research provides information about health technology assessment
The National Institute for Health and Clinical Excellence Web site describes how this organization provides guidance on promoting good health within the England and Wales National Health Service
Information on the UK General Practice Research Database is available
Wikipedia has pages on health technology assessment and on selective cyclooxygenase-2 inhibitors (note that Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
doi:10.1371/journal.pmed.1000194
PMCID: PMC2779340  PMID: 19997499
14.  Electrostimulation: Current Status, Strength of Evidence Guidelines, and Meta-Analysis 
Advances in Wound Care  2014;3(2):118-126.
Significance: Delayed healing of skin wounds is a serious problem for the patients, clinicians, and society. The application of interventions with proven effectiveness to increase wound healing is relevant.
Recent Advances: This article summarizes the results of effect studies with the application of electrostimulation (ES) as additional treatment to standard wound care (SWC). Therefore, five published narrative reviews are discussed. In addition, 15 studies with a clear randomized controlled trial design are analyzed systematically and the results are presented in four forest plots. The healing rate is expressed in the outcome measure percentage area reduction in 4 weeks of treatment (PAR4). This leads to a continuous measure with mean differences between the percentage healing in the experimental group (SWC plus ES) and in the control group (SWC alone or SWC plus placebo ES). Adding ES to SWC in all wound types increases PAR4 by an extra 26.7% (95% confidence interval [CI] 15.6, 37.8); adding unidirectional ES to SWC increases PAR4 by 30.8% (95% CI 20.9, 40.6) and adding unidirectional ES to the treatment of pressure ulcers increases PAR4 by 42.7% (95% CI 32.0, 53.3).
Critical Issues: There is a discrepancy between the proven effectiveness of ES as additional treatment to SWC and the application of ES in real practice. Possible drawbacks are the lack of clinical expertise concerning the proper application of ES and the extra time effort and necessary equipment that are needed.
Future Directions: Clinicians concerned about the optimal treatment of patients with delayed wound healing should improve their practical competency to be able to apply ES.
doi:10.1089/wound.2013.0448
PMCID: PMC3928827  PMID: 24761352
15.  Rehabilitation for the management of knee osteoarthritis using comprehensive traditional Chinese medicine in community health centers: study protocol for a randomized controlled trial 
Trials  2013;14:367.
Background
It is becoming increasingly necessary for community health centers to make rehabilitation services available to patients with osteoarthritis of the knee. However, for a number of reasons, including a lack of expertise, the small size of community health centers and the availability of only simple medical equipment, conventional rehabilitation therapy has not been widely used in China. Consequently, most patients with knee osteoarthritis seek treatment in high-grade hospitals. However, many patients cannot manage the techniques that they were taught in the hospital. Methods such as acupuncture, tuina, Chinese medical herb fumigation-washing and t’ai chi are easy to do and have been reported to have curative effects in those with knee osteoarthritis. To date, there have been no randomized controlled trials validating comprehensive traditional Chinese medicine for the rehabilitation of knee osteoarthritis in a community health center. Furthermore, there is no standard rehabilitation protocol using traditional Chinese medicine for knee osteoarthritis. The aim of the current study is to develop a comprehensive rehabilitation protocol using traditional Chinese medicine for the management of knee osteoarthritis in a community health center.
Method/design
This will be a randomized controlled clinical trial with blinded assessment. There will be a 4-week intervention utilizing rehabilitation protocols from traditional Chinese medicine and conventional therapy. Follow-up will be conducted for a period of 12 weeks. A total of 722 participants with knee osteoarthritis will be recruited. Participants will be randomly divided into two groups: experimental and control. Primary outcomes will include range of motion, girth measurement, the visual analogue scale, and results from the manual muscle, six-minute walking and stair-climbing tests. Secondary outcomes will include average daily consumption of pain medication, ability to perform daily tasks and health-related quality-of-life assessments. Other outcomes will include rate of adverse events and economic effects. Relative cost-effectiveness will be determined from health service usage and outcome data.
Discussion
The primary aim of this trial is to develop a standard protocol for traditional Chinese medicine, which can be adopted by community health centers in China and worldwide, for the rehabilitation of patients with knee osteoarthritis.
Trial registration
Clinical Trials Registration: ChiCTR-TRC-12002538
doi:10.1186/1745-6215-14-367
PMCID: PMC4228261  PMID: 24188276
Knee osteoarthritis; Traditional Chinese medicine; Rehabilitation; Clinical trial
16.  How Can Research Keep Up With eHealth? Ten Strategies for Increasing the Timeliness and Usefulness of eHealth Research 
Background
eHealth interventions appear and change so quickly that they challenge the way we conduct research. By the time a randomized trial of a new intervention is published, technological improvements and clinical discoveries may make the intervention dated and unappealing. This and the spate of health-related apps and websites may lead consumers, patients, and caregivers to use interventions that lack evidence of efficacy.
Objective
This paper aims to offer strategies for increasing the speed and usefulness of eHealth research.
Methods
The paper describes two types of strategies based on the authors’ own research and the research literature: those that improve the efficiency of eHealth research, and those that improve its quality.
Results
Efficiency strategies include: (1) think small: conduct small studies that can target discrete but significant questions and thereby speed knowledge acquisition; (2) use efficient designs: use such methods as fractional-factorial and quasi-experimental designs and surrogate endpoints, and experimentally modify and evaluate interventions and delivery systems already in use; (3) study universals: focus on timeless behavioral, psychological, and cognitive principles and systems; (4) anticipate the next big thing: listen to voices outside normal practice and connect different perspectives for new insights; (5) improve information delivery systems: researchers should apply their communications expertise to enhance inter-researcher communication, which could synergistically accelerate progress and capitalize upon the availability of “big data”; and (6) develop models, including mediators and moderators: valid models are remarkably generative, and tests of moderation and mediation should elucidate boundary conditions of effects and treatment mechanisms. Quality strategies include: (1) continuous quality improvement: researchers need to borrow engineering practices such as the continuous enhancement of interventions to incorporate clinical and technological progress; (2) help consumers identify quality: consumers, clinicians, and others all need to easily identify quality, suggesting the need to efficiently and publicly index intervention quality; (3) reduce the costs of care: concern with health care costs can drive intervention adoption and use and lead to novel intervention effects (eg, reduced falls in the elderly); and (4) deeply understand users: a rigorous evaluation of the consumer’s needs is a key starting point for intervention development.
Conclusions
The challenges of distinguishing and distributing scientifically validated interventions are formidable. The strategies described are meant to spur discussion and further thinking, which are important, given the potential of eHealth interventions to help patients and families.
doi:10.2196/jmir.2925
PMCID: PMC3961695  PMID: 24554442
social media; Internet; randomized clinical trials; experimental designs; research techniques; patient education; patient engagement; health communication; telemedicine
17.  Effectiveness guidance document (EGD) for Chinese medicine trials: a consensus document 
Trials  2014;15:169.
Background
There is a need for more Comparative Effectiveness Research (CER) on Chinese medicine (CM) to inform clinical and policy decision-making. This document aims to provide consensus advice for the design of CER trials on CM for researchers. It broadly aims to ensure more adequate design and optimal use of resources in generating evidence for CM to inform stakeholder decision-making.
Methods
The Effectiveness Guidance Document (EGD) development was based on multiple consensus procedures (survey, written Delphi rounds, interactive consensus workshop, international expert review). To balance aspects of internal and external validity, multiple stakeholders, including patients, clinicians, researchers and payers were involved in creating this document.
Results
Recommendations were developed for “using available data” and “future clinical studies”. The recommendations for future trials focus on randomized trials and cover the following areas: designing CER studies, treatments, expertise and setting, outcomes, study design and statistical analyses, economic evaluation, and publication.
Conclusion
The present EGD provides the first systematic methodological guidance for future CER trials on CM and can be applied to single or multi-component treatments. While CONSORT statements provide guidelines for reporting studies, EGDs provide recommendations for the design of future studies and can contribute to a more strategic use of limited research resources, as well as greater consistency in trial design.
doi:10.1186/1745-6215-15-169
PMCID: PMC4045891  PMID: 24885146
Comparative effectiveness research; Effectiveness guidance document; Chinese medicine research
18.  Development and description of a decision analysis based decision support tool for stroke prevention in atrial fibrillation 
Background: There is an increasing move towards clinical decision making that engages the patient, which has led to the development and use of decision aids to support better decisions. The treatment of patients in atrial fibrillation (AF) with warfarin to prevent stroke is a decision that is sensitive to patient preferences as shown by a previous decision analysis.
Aim: To develop a computerised decision support tool, building upon a previous decision analysis, which would engage individual patient preferences in reaching a shared decision on whether to take warfarin to prevent stroke.
Methods: The development process had two main phases: (1) the development phase which employed focus groups and repeated interviews with GPs/practice nurses and patients alongside an iterative development of a computerised tool; (2) the training and testing phase in which GPs and practice nurses underwent training in the use of the tool, including the use of simulated patients. The tool was then used in a feasibility study in a small number of patients with AF to inform the design of a subsequent randomised controlled trial.
Results: The prototype tool had three components: (1) derivation of an individual patient's values for relevant health states using a standard gamble; (2) presentation/discussion of a patient's risks of stroke using the Framingham equation and the benefits/risks of warfarin from a systematic literature review; and (3) decision making component incorporating the outcome of a Markov decision analysis model. Older patients could be taken through the decision analysis based computerised tool, and patients and clinicians welcomed information on risks and benefits of treatments. The tool required time and training to use. Patients' decisions in the feasibility phase did not necessarily coincide with the output of the decision analysis model, but decision conflict appeared to be reduced and both patients and GPs were satisfied with the process.
Conclusions: It is feasible to develop a decision analysis based computer software package that is acceptable to elderly patients and clinicians, but it requires time and expertise to use. It is most likely that a tool of this type will best be used by a small number of clinicians who have developed experience of its use and can maintain their skills.
doi:10.1136/qhc.11.1.25
PMCID: PMC1743557  PMID: 12078365
19.  Methods of Blinding in Reports of Randomized Controlled Trials Assessing Pharmacologic Treatments: A Systematic Review 
PLoS Medicine  2006;3(10):e425.
Background
Blinding is a cornerstone of therapeutic evaluation because lack of blinding can bias treatment effect estimates. An inventory of the blinding methods would help trialists conduct high-quality clinical trials and readers appraise the quality of results of published trials. We aimed to systematically classify and describe methods to establish and maintain blinding of patients and health care providers and methods to obtain blinding of outcome assessors in randomized controlled trials of pharmacologic treatments.
Methods and Findings
We undertook a systematic review of all reports of randomized controlled trials assessing pharmacologic treatments with blinding published in 2004 in high impact-factor journals from Medline and the Cochrane Methodology Register. We used a standardized data collection form to extract data. The blinding methods were classified according to whether they primarily (1) established blinding of patients or health care providers, (2) maintained the blinding of patients or health care providers, and (3) obtained blinding of assessors of the main outcomes. We identified 819 articles, with 472 (58%) describing the method of blinding. Methods to establish blinding of patients and/or health care providers concerned mainly treatments provided in identical form, specific methods to mask some characteristics of the treatments (e.g., added flavor or opaque coverage), or use of double dummy procedures or simulation of an injection. Methods to avoid unblinding of patients and/or health care providers involved use of active placebo, centralized assessment of side effects, patients informed only in part about the potential side effects of each treatment, centralized adapted dosage, or provision of sham results of complementary investigations. The methods reported for blinding outcome assessors mainly relied on a centralized assessment of complementary investigations, clinical examination (i.e., use of video, audiotape, or photography), or adjudication of clinical events.
Conclusions
This review classifies blinding methods and provides a detailed description of methods that could help trialists overcome some barriers to blinding in clinical trials and readers interpret the quality of pharmalogic trials.
Following a systematic review of all reports of randomized controlled trials assessing pharmacologic treatments involving blinding, a classification of blinding methods is proposed.
Editors' Summary
Background.
In evidence-based medicine, good-quality randomized controlled trials are generally considered to be the most reliable source of information about the effects of different treatments, such as drugs. In a randomized trial, patients are assigned to receive one treatment or another by the play of chance. This technique helps makes sure that the two groups of patients receiving the different treatments are equivalent at the start of the trial. Proper randomization also prevents doctors from controlling or affecting which treatment patients get, which could distort the results. An additional tool that is also used to make trials more precise is “blinding.” Blinding involves taking steps to prevent patients, doctors, or other people involved in the trial (e.g., those people recording measurements) from finding out which patients got what treatment. Properly done, blinding should make sure the results of a trial are more accurate. This is because in an unblinded study, participants may respond better if they know they have received a promising new treatment (or worse if they only got placebo or an old drug); doctors may “want” a particular treatment to do better in the trial, and unthinking bias could creep into their measurements or actions; the same applies for practitioners and researchers who record patients' outcomes in the trial. However, blinding is not a simple, single step; the people carrying out the trial often have to set up a variety of different procedures that depend on the type of trial that is being done.
Why Was This Study Done?
The researchers here wanted to thoroughly examine different methods that have been used to achieve blinding in randomized trials of drug treatments, and to describe and classify them. They hoped that a better understanding of the different blinding methods would help people doing trials to design better trials in the future, and also help readers to interpret the quality of trials that had been done.
What Did the Researchers Do and Find?
This group of researchers conducted what is called a “systematic review.” They systematically searched the published medical literature to find all randomized, blinded drug trials published in 2004 in a number of different “high-impact” journals (journals whose articles are often mentioned in other articles). Then, the researchers classified information from the published trial reports. The researchers ended up with 819 trial reports, and nearly 60% of them described how blinding was done. Their classification of blinding was divided up into three main areas. First, they detailed methods used to hide which drugs are given to particular patients, such as preparing identically appearing treatments; using strong flavors to mask taste; matching the colors of pills; using saline injections and so on. Second, they described a number of methods that could be used to reduce the risk of unblinding (of doctors or patients), such as using an “active placebo” (a sugar pill that mimics some of the expected side effects of the drug treatment). Finally, they defined methods for blinded measurement of outcomes (such as using a central committee to collect data).
What Do These Findings Mean?
The researchers' classification will help people to work out how different techniques can be used to achieve, and keep, blinding in a trial. This will assist others to understand whether any particular trial was likely to have been blinded properly, and therefore work out whether the results are reliable. The researchers also suggest that, generally, blinding methods are not described in enough detail in published scientific papers, and recommend that guidelines for describing results of randomized trials be improved.
Additional Information.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0030425.
James Lind Library has been created to help patients and researchers understand fair tests of treatments in health care by illustrating how fair tests have developed over the centuries
ClinicalTrials.gov, a trial registry created by the US National Institutes of Health, has an introduction to understanding clinical trials
National Electronic Library for Health introduction to controlled clinical trials
doi:10.1371/journal.pmed.0030425
PMCID: PMC1626553  PMID: 17076559
20.  Intra-Articular Viscosupplementation With Hylan G-F 20 To Treat Osteoarthritis of the Knee 
Executive Summary
Objective
To assess the effectiveness and cost-effectiveness of hylan G-F 20 as a substitute for existing treatments for pain due to osteoarthritis (OA) of the knee, other viscosupplementation devices, and/or as an adjunct to conventional therapy.
Hylan G-F 20 (brand name Synvisc, which is manufactured by Genzyme) is a high molecular weight derivative of hyaluronan, a component of joint synovial fluid. It acts as a lubricant and shock absorber. It is administered by injection into the joint space to treat pain associated with OA of the knee. Although the injection procedure is an insured service in Ontario, the device, hylan G-F 20, is not.
Clinical Need
Osteoarthritis is prevalent in 10% to 12% of Ontario adults, and exceeds 40% in Ontario residents aged 65 years and older. About one-half of these people have mild, moderate, or severe OA of the knee. Conventional treatment involves a combination of nonpharmacological management (e.g., weight loss, exercise, social support, and patient education), drugs, (e.g., acetaminophen, COX-2 inhibitors, nonsteroidal anti-inflammatory drugs with/without misoprostol, intra-articular glucocorticoids, opioids, and topical analgesics) and surgical interventions, such as debridement and total knee replacement, when pharmacological management fails.
The growing burden of OA of the knee in the aging Ontario population combined with recent safety concerns about COX-2 inhibitors and long wait times for total joint replacement is placing pressure on the demand for new, effective technologies to manage the pain of OA.
The Technology
Hylan G-F 20 is derived from rooster comb hyaluronan (HA). At the time of writing, eight viscosupplement hyaluronic products are licensed in Canada. Hylan G-F 20 is distinguished from the other products by its chemical structure (i.e., cross-linked hyaluronan, hence hylan) and relatively higher molecular weight, which may bestow greater therapeutic viscoelastic properties. A complete treatment cycle of hylan G-F 20 involves an intra-articular injection of 2 ml of hylan G-F 20 once a week for 3 weeks. It is licensed for use for patients in all stages of joint pathology, but should not be used in infected or severely inflamed joints, in joints with large effusion, in patients that have skin diseases or infections in the area of the injection site, or in patients with venous stasis. It is also contraindicated in patients with hypersensitivities to avian proteins.
Review Strategy
The Medical Advisory Secretariat used its standard search protocol to review the literature for evidence on the effectiveness of intra-articular hylan G-F 20 compared with placebo, as a substitute for alternate active treatments, or as an adjunct to conventional care for treatment of the pain of OA of the knee. All English-language journal articles and reviews with clearly described designs and methods (i.e., those sufficient to assign a Jadad score to) published or released between 1966 and February 2005 were included. Two more recently published meta-analyses were also included. The databases searched were Ovid MEDLINE, EMBASE, the Cochrane database and leading international organizations for health technology assessments, including the International Network of Agencies for Health Technology Assessments. The search terms were as follows: hyaluronan, hyaluronate adj sodium, hylan, hylan G-F 20 (Synvisc), Synvisc, Hyalgan, Orthovisc, Supartz, Artz, Artzal, BioHY, NASHA, NRD101, viscosupplementation, osteoarthritis, knee, knee joint. The primary outcome of interest was a clinically significant difference, defined as greater than 10 mm on 100 mm visual analogue scale, or a change from baseline of more than 20% in the mean magnitude of pain relief experienced among patients treated with hylan G-F 20 compared with those treated with the control intervention.
One clinical epidemiologist reviewed the full-text reports and extracted data using an extraction form. Key variables included, but were not limited to, the characteristics of the patients, method of randomization, type of control intervention, outcome measures for effectiveness and safety, and length of follow-up. The quality of the studies and level of the evidence was initially scored by one clinical epidemiologist using the Jadad scale and GRADE approach. Level of quality depends on the amount of certainty about the magnitude of effect and is based on study designs, extent of methodological limitations, consistency of results and applicability (i.e. directness) to the Ontario clinical context. The GRADE approach also permits comment on the strength of recommendations resulting from the evidence, based on estimates of the magnitude of effect relative to the magnitude of risk and burden and the level of certainty around these estimates. The quality assessments were subsequently peer-reviewed.
Summary of Findings
The literature search revealed 2 previous health technology assessments, 3 meta-analyses of placebo-controlled trials, 1 Cochrane review and meta-analysis encompassing 18 randomized controlled trials (RCTs) that compared hylan G-F 20 to either placebo or active treatments, 11 RCTs of hylan G-F 20 (all included in the Cochrane review), and 10 observational studies. Given the preponderance of evidence, the Medical Advisory Secretariat’s analysis focused on studies with Level 1 evidence of effectiveness (i.e., the meta-analyses of RCTs and the RCTs). Only safety data from the observational studies were included.
The authors of the 2 health technology assessments concluded that the data were sparse and poor quality. There was some evidence that hylan G-F 20 delivered a small, clinical benefit at 3 to 6 months after treatment on a magnitude comparable to NSAIDs and intra-articular steroids. Hylan G-F 20 appeared to carry a risk of a local adverse reaction of in the range of 3% to 18% per 100 injections, but there was no apparent risk of a severe adverse event, although the data were limited.
Each of the 3 meta-analyses of placebo-controlled trials of intra-articular hyaluronans had only 3 trials involving hylan G-F 20. There results were inconsistent, with one study concluding that intra-articular hyaluronans were efficacious, whereas the 2 other analyses concluded the effect size was small (0.32) and probably not clinically significant. The risk of a minor adverse event ranged from 8% to 19% per 100 injections. Major adverse events were rare.
The authors of the Cochrane review concluded that a pooled analysis supported the efficacy of hyaluronans, including hylan G-F 20. The 5- to 13-week post-injection period showed an improvement from baseline of 11% to 54% for pain and 9% to 15% for function. Comparable efficacy was noted against NSAIDs, and longer-term benefits were noted in against steroids. Few adverse events were noted.
When the Medical Advisory Secretariat applied the criterion of clinical significance to the magnitude of pain relief reported in the RCTs on hylan G-F 20, the following was noted:
There was inconsistent evidence that hylan G-F 20 was clinically superior to placebo at 5 to 26 weeks after treatment.
There was consistent evidence that, in terms of delivering pain relief, hylan G-F 20 was no better or worse than NSAIDs or intra-articular steroids at 5 to 26 weeks after treatment.
There was consistent evidence that hylan G-F 20 was not clinically superior to other hyaluronic products.
There was consistent evidence that hylan G-F 20 delivered a small magnitude of clinical benefit at 12 to 52 weeks post-injection when administered as an adjunct to conventional care.
There were limitations to the methods in many of the RCTs involving hylan G-F 20. When only the results from the higher-quality studies were considered, there was level 2 evidence that hylan G-F 20 was not clinically superior to placebo (or another hyaluronan) at 1 to 26 weeks after treatment in older patients with advanced disease for whom total knee replacement was indicated. There was level 2 evidence that hylan G-F 2- was comparable to NSAIDs at 4 to 13 weeks after treatment, and level 2 evidence that hylan G-F 20 was superior to placebo as an adjunct to conventional care 4 to 26 weeks after treatment.
With respect to safety, overall, hylan G-F 20 carries a risk of a minor, local adverse event rate of about 8% to 19% per 100 injections. Incidents of moderate-severe post-injection inflammatory joint reactions have been reported, but the likelihood appears to be low (0.15% of patients).
Economic Analysis
Case-costing estimates suggest that the annual cost of 2 treatment cycles of hylan G-F 20 (plus analgesics for breakthrough pain) is almost equivalent to the annual cost of taking a NSAID (with a gastroprotective agent) and is more expensive that taking intra-articular corticosteroids (plus analgesics for breakthrough pain). The estimated cost of funding hylan G-F 20 as an adjunct to conventional therapy (i.e., any of analgesics, NSAIDs, intra-articular steroids, physiotherapy, and surgery) is $700 per patient per year. Given the huge burden of mild to moderate OA among adults who seek medical care for it in Ontario (about 300,000), funding hylan G-F 20 as an adjunct to existing treatment could be expensive, depending on its diffusion and uptake. If only 10% to 30% of patients choose this option, then the estimated budget impact would be $21 million to $63 million (Cdn) per year.
Conclusions
When the benefits relative to the risks and costs are considered, NSAIDs and hylan G-F 20 appear comparable, as the table shows. Consequently, there’s little evidence on which to recommend hylan G-F 20 over NSAIDs, except perhaps for patients who cannot tolerate NSAIDs, although this evidence is indirect, since no studies looked specifically at this population.
CC indicates conventional care; IA, intra-articular; NSAID, nonsteroidal anti-inflammatory drug.
Intra-articular steroids appear to deliver the same risks and clinical benefits as hylan G-F 20 at a lower cost; therefore, there’s evidence that intra-articular steroids are the preferred option. Hylan G-F 20 as an adjunct to conventional care appears to deliver some clinical benefit, although funding hylan G-F 20 as an adjunct would have considerable budget impact, so the benefits of this option do not clearly outweigh the costs. There’s some uncertainty about the effect of hylan G-F 20 relative to other hyaluronans, mostly because some of the trials of this comparison were not published.
Many of the studies of hylan G-F 20 have considerable methodological limitations that result in uncertainty about the magnitude of effect. An upcoming review of the evidence by the Osteoarthritis Advisory Panel of clinical experts will likely help to reduce some of this uncertainty.
There is moderate evidence that hylan G-F 20 is no more clinically effective than NSAIDs. The evidence that hylan G-F 20 might be an appropriate option for a person with OA of the knee who cannot tolerate NSAIDs is indirect. The possible benefit of fewer cases of NSAID-induced gastropathy in this population must be weighed against the uncertainty of a severe inflammatory adverse reaction to hylan G-F 20.
Similarly, there is moderate evidence that hylan G-F 20 is no more clinically effective than intra-articular corticosteroids. The lower cost of intra-articular corticosteroids makes them the preferred option.
There is moderate evidence that hylan G-F 20 is effective as an adjunct to conventional care, delivering a small magnitude of temporary relief at 4 to 26 weeks after treatment. The estimated additional cost to the system of providing hylan G-F 20 as an adjunct to conventional care is about $700 (Cdn) per patient annually. The magnitude and duration of clinical benefit of hylan G-F 20 must be weighed against the uncertainty and potential magnitude of the budget impact (about $35 million to $105 million (Cdn) per year) of funding this device given the high burden of OA in Ontario adults.
There is level 2 evidence that hylan G-F 20 is not effective in people with advanced OA for whom total knee replacement is indicated.
PMCID: PMC3382385  PMID: 23074461
21.  Use of drug therapy in the management of symptomatic ureteric stones in hospitalized adults (SUSPEND), a multicentre, placebo-controlled, randomized trial of a calcium-channel blocker (nifedipine) and an α-blocker (tamsulosin): study protocol for a randomized controlled trial 
Trials  2014;15:238.
Background
Urinary stone disease is common, with an estimated prevalence among the general population of 2% to 3%. Ureteric stones can cause severe pain and have a significant impact on quality of life, accounting for over 15,000 hospital admissions in England annually. Uncomplicated cases of smaller stones in the lower ureter are traditionally treated expectantly. Those who fail standard care or develop complications undergo active treatment, such as extracorporeal shock wave lithotripsy or ureteroscopy with stone retrieval. Such interventions are expensive, require urological expertise and carry a risk of complications.
Growing understanding of ureteric function and pathophysiology has led to the hypothesis that drugs causing relaxation of ureteric smooth muscle, such as the selective α-blocker tamsulosin and the calcium-channel blocker nifedipine, can enhance the spontaneous passage of ureteric stones. The use of drugs in augmenting stone passage, reducing the morbidity and costs associated with ureteric stone disease, is promising. However, the majority of clinical trials conducted to date have been small, poor to moderate quality and lacking in comprehensive economic evaluation.
This trial aims to determine the clinical and cost-effectiveness of tamsulosin and nifedipine in the management of symptomatic urinary stones.
Methods/design
The SUSPEND (Spontaneous Urinary Stone Passage ENabled by Drugs) trial is a multicentre, double-blind, randomized controlled trial evaluating two medical expulsive therapy strategies (nifedipine or tamsulosin) versus placebo.
Patients aged 18 to 65 with a ureteric stone confirmed by non-contrast computed tomography of the kidney, ureter and bladder will be randomized to receive nifedipine, tamsulosin or placebo (400 participants per arm) for a maximum of 28 days. The primary clinical outcome is spontaneous passage of ureteric stones at 4 weeks (defined as no further intervention required to facilitate stone passage). The primary economic outcome is a reduction in the incremental cost per quality-adjusted life years, determined at 12 weeks. The analysis will be based on all participants as randomized (intention to treat). The trial has 90% power with a type I error rate of 5% to detect a 10% increase in primary outcome between the tamsulosin and nifedipine treatment groups.
Trial registration
ISRCTN69423238; EudraCT number: 2010-019469-26
doi:10.1186/1745-6215-15-238
PMCID: PMC4090633  PMID: 24947817
medical expulsive therapy; nifedipine; tamsulosin; ureteric stone
22.  A point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen 
Background Clinical trials are widely considered the gold standard in comparative effectiveness research (CER) but the high cost and complexity of traditional trials and concerns about generalizability to broad patient populations and general clinical practice limit their appeal. Unsuccessful implementation of CER results limits the value of even the highest quality trials. Planning for a trial comparing two standard strategies of insulin administration for hospitalized patients led us to develop a new method for a clinical trial designed to be embedded directly into the clinical care setting thereby lowering the cost, increasing the pragmatic nature of the overall trial, strengthening implementation, and creating an integrated environment of research-based care.
Purpose We describe a novel randomized clinical trial that uses the informatics and statistics infrastructure of the Veterans Affairs Healthcare System (VA) to illustrate one key component (called the point-of-care clinical trial – POC-CT) of a ‘learning healthcare system,’ and settles a clinical question of interest to the VA.
Methods This study is an open-label, randomized trial comparing sliding scale regular insulin to a weight-based regimen for control of hyperglycemia, using the primary outcome length of stay, in non-ICU inpatients within the northeast region of the VA. All non-ICU patients who require in-hospital insulin therapy are eligible for the trial, and the VA’s automated systems will be used to assess eligibility and present the possibility of randomization to the clinician at the point of care. Clinicians will indicate their approval for informed consent to be obtained by study staff. Adaptive randomization will assign up to 3000 patients, preferentially to the currently ‘winning’ strategy, and all care will proceed according to usual practices. Based on a Bayesian stopping rule, the study has acceptable frequentist operating characteristics (Type I error 6%, power 86%) against a 12% reduction of median length of stay from 5 to 4.4 days. The adaptive stopping rule promotes implementation of a successful treatment strategy.
Limitations Despite clinical equipoise, individual healthcare providers may have strong treatment preferences that jeopardize the success and implementation of the trial design, leading to low rates of randomization. Unblinded treatment assignment may bias results. In addition, generalization of clinical results to other healthcare systems may be limited by differences in patient population. Generalizability of the POC-CT method depends on the level of informatics and statistics infrastructure available to a healthcare system.
Conclusions The methods proposed will demonstrate outcome-based evaluation of control of hyperglycemia in hospitalized veterans. By institutionalizing a process of statistically sound and efficient learning, and by integrating that learning with automatic implementation of best practice, the participating VA Healthcare Systems will accelerate improvements in the effectiveness of care.
doi:10.1177/1740774511398368
PMCID: PMC3195898  PMID: 21478329
23.  Comparison of conventional medicine, TCM treatment, and combination of both conventional medicine and TCM treatment for patients with chronic obstructive pulmonary disease: study protocol of a randomized comparative effectiveness research trial 
Trials  2014;15:153.
Background
Chronic obstructive pulmonary disease (COPD) affects millions worldwide. Although many therapies exist and are being developed to relieve symptoms and reduce mortality, few data are available to understand which of the therapeutic alternatives is the most cost-effective for COPD patients in everyday clinical practice, especially for traditional Chinese medicine (TCM). Comparative effectiveness research can help patients, clinicians, and decision-makers make best informed treatment decisions where such evidence was previously lacking. This study aims to compare the effectiveness and economic evaluation of three treatments: (1) conventional Western medicine; (2) TCM treatments, which have been evaluated and have certain effect; and (3) a combination of both conventional Western medicine and TCM treatments, and then determine which treatment is the most suitable for COPD patients.
Methods/design
A multicenter, pragmatic, randomized, controlled trial is adopted. A total of 360 patients will be recruited and randomly assigned to one of the three treatments group, with 120 in each group. Patients in the conventional Western medicine group will be given Salbutamol, Formoterol, Salmeterol/fluticasone, respectively, according to the guidelines. For the TCM group, patients will be given Bufei granule, Bu-Fei Jian-Pi granule, Bu-Fei Yi-Shen granule, and Yi-Qi Zi-Shen granule based on their corresponding TCM syndrome patterns, respectively. For the combination of conventional medicine and TCM treatments group, patients will be given a combination of conventional Western medicine and TCM granules. Treatments in each group are recognized as a whole comprehensive intervention. After the 26-week treatment, another 26 weeks will be followed up. The outcome measures including the frequency and duration of acute exacerbations, lung function, dyspnea, exercise capacity, quality of life, and economic evaluation will be assessed.
Discussion
It is hypothesized that each of the three treatments will have beneficial effects in reducing the frequency and duration of acute exacerbations, improving exercise capacity and psychosocial function of COPD patients. In addition, the combination of conventional medicine and TCM treatments may be most suitable for COPD patients with better effectiveness and economic evaluation.
Trial registration
ClinicalTrials.gov NCT01836016.
doi:10.1186/1745-6215-15-153
PMCID: PMC4017822  PMID: 24885672
Chronic obstructive pulmonary disease; Comparative effectiveness research; Traditional Chinese medicine
24.  Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview 
PLoS Medicine  2011;8(5):e1001026.
Su Golder and colleagues carry out an overview of meta-analyses to assess whether estimates of the risk of harm outcomes differ between randomized trials and observational studies. They find that, on average, there is no difference in the estimates of risk between overviews of observational studies and overviews of randomized trials.
Background
There is considerable debate as to the relative merits of using randomised controlled trial (RCT) data as opposed to observational data in systematic reviews of adverse effects. This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of RCTs as compared to meta-analysis of observational studies.
Methods and Findings
Searches were carried out in ten databases in addition to reference checking, contacting experts, citation searches, and hand-searching key journals, conference proceedings, and Web sites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from RCTs could be directly compared, using the ratio of odds ratios, with the pooled estimate for the same adverse effect arising from observational studies. Nineteen studies, yielding 58 meta-analyses, were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% confidence interval 0.93–1.15). There was less discrepancy with larger studies. The symmetric funnel plot suggests that there is no consistent difference between risk estimates from meta-analysis of RCT data and those from meta-analysis of observational studies. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped (54/58, 93%). In terms of statistical significance, in nearly two-thirds (37/58, 64%), the results agreed (both studies showing a significant increase or significant decrease or both showing no significant difference). In only one meta-analysis about one adverse effect was there opposing statistical significance.
Conclusions
Empirical evidence from this overview indicates that there is no difference on average in the risk estimate of adverse effects of an intervention derived from meta-analyses of RCTs and meta-analyses of observational studies. This suggests that systematic reviews of adverse effects should not be restricted to specific study types.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Whenever patients consult a doctor, they expect the treatments they receive to be effective and to have minimal adverse effects (side effects). To ensure that this is the case, all treatments now undergo exhaustive clinical research—carefully designed investigations that test new treatments and therapies in people. Clinical investigations fall into two main groups—randomized controlled trials (RCTs) and observational, or non-randomized, studies. In RCTs, groups of patients with a specific disease or condition are randomly assigned to receive the new treatment or a control treatment, and the outcomes (for example, improvements in health and the occurrence of specific adverse effects) of the two groups of patients are compared. Because the patients are randomly chosen, differences in outcomes between the two groups are likely to be treatment-related. In observational studies, patients who are receiving a specific treatment are enrolled and outcomes in this group are compared to those in a similar group of untreated patients. Because the patient groups are not randomly chosen, differences in outcomes between cases and controls may be the result of a hidden shared characteristic among the cases rather than treatment-related (so-called confounding variables).
Why Was This Study Done?
Although data from individual trials and studies are valuable, much more information about a potential new treatment can be obtained by systematically reviewing all the evidence and then doing a meta-analysis (so-called evidence-based medicine). A systematic review uses predefined criteria to identify all the research on a treatment; meta-analysis is a statistical method for combining the results of several studies to yield “pooled estimates” of the treatment effect (the efficacy of a treatment) and the risk of harm. Treatment effect estimates can differ between RCTs and observational studies, but what about adverse effect estimates? Can different study designs provide a consistent picture of the risk of harm, or are the results from different study designs so disparate that it would be meaningless to combine them in a single review? In this methodological overview, which comprises a systematic review and meta-analyses, the researchers assess the level of agreement in the estimates of harm derived from meta-analysis of RCTs with estimates derived from meta-analysis of observational studies.
What Did the Researchers Do and Find?
The researchers searched literature databases and reference lists, consulted experts, and hand-searched various other sources for studies in which the pooled estimate of an adverse effect from RCTs could be directly compared to the pooled estimate for the same adverse effect from observational studies. They identified 19 studies that together covered 58 separate adverse effects. In almost all instances, the estimates of harm obtained from meta-analyses of RCTs and observational studies had overlapping 95% confidence intervals. That is, in statistical terms, the estimates of harm were similar. Moreover, in nearly two-thirds of cases, there was agreement between RCTs and observational studies about whether a treatment caused a significant increase in adverse effects, a significant decrease, or no significant change (a significant change is one unlikely to have occurred by chance). Finally, the researchers used meta-analysis to calculate that the pooled ratio of the odds ratios (a statistical measurement of risk) of RCTs compared to observational studies was 1.03. This figure suggests that there was no consistent difference between risk estimates obtained from meta-analysis of RCT data and those obtained from meta-analysis of observational study data.
What Do These Findings Mean?
The findings of this methodological overview suggest that there is no difference on average in the risk estimate of an intervention's adverse effects obtained from meta-analyses of RCTs and from meta-analyses of observational studies. Although limited by some aspects of its design, this overview has several important implications for the conduct of systematic reviews of adverse effects. In particular, it suggests that, rather than limiting systematic reviews to certain study designs, it might be better to evaluate a broad range of studies. In this way, it might be possible to build a more complete, more generalizable picture of potential harms associated with an intervention, without any loss of validity, than by evaluating a single type of study. Such a picture, in combination with estimates of treatment effects also obtained from systematic reviews and meta-analyses, would help clinicians decide the best treatment for their patients.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001026.
The US National Institutes of Health provide information on clinical research; the UK National Health Service Choices Web site also has a page on clinical trials and medical research
The Cochrane Collaboration produces and disseminates systematic reviews of health-care interventions
Medline Plus provides links to further information about clinical trials (in English and Spanish)
doi:10.1371/journal.pmed.1001026
PMCID: PMC3086872  PMID: 21559325
25.  Design of a trial to evaluate the impact of clinical pharmacists and community health promoters working with African-Americans and Latinos with Diabetes 
BMC Public Health  2012;12:891.
Background
Given the increasing prevalence of diabetes and the lack of patients reaching recommended therapeutic goals, novel models of team-based care are emerging. These teams typically include a combination of physicians, nurses, case managers, pharmacists, and community-based peer health promoters (HPs). Recent evidence supports the role of pharmacists in diabetes management to improve glycemic control, as they offer expertise in medication management with the ability to collaboratively intensify therapy. However, few studies of pharmacy-based models of care have focused on low income, minority populations that are most in need of intervention. Alternatively, HP interventions have focused largely upon low income minority groups, addressing their unique psychosocial and environmental challenges in diabetes self-care. This study will evaluate the impact of HPs as a complement to pharmacist management in a randomized controlled trial.
Methods/Design
The primary aim of this randomized trial is to evaluate the effectiveness of clinical pharmacists and HPs on diabetes behaviors (including healthy eating, physical activity, and medication adherence), hemoglobin A1c, blood pressure, and LDL-cholesterol levels. A total of 300 minority patients with uncontrolled diabetes from the University of Illinois Medical Center ambulatory network in Chicago will be randomized to either pharmacist management alone, or pharmacist management plus HP support. After one year, the pharmacist-only group will be intensified by the addition of HP support and maintenance will be assessed by phasing out HP support from the pharmacist plus HP group (crossover design). Outcomes will be evaluated at baseline, 6, 12, and 24 months. In addition, program and healthcare utilization data will be incorporated into cost and cost-effectiveness evaluations of pharmacist management with and without HP support.
Discussion
The study will evaluate an innovative, integrated approach to chronic disease management in minorities with poorly controlled diabetes. The approach is comprised of clinic-based pharmacists and community-based health promoters collaborating together. They will target patient-level factors (e.g., lack of adherence to lifestyle modification and medications) and provider-level factors (e.g., clinical inertia) that contribute to poor clinical outcomes in diabetes. Importantly, the study design and analytic approach will help determine the differential and combined impact of adherence to lifestyle changes, medication, and intensification on clinical outcomes.
Trial registration
ClinicalTrials.gov identifier: NCT01498159
doi:10.1186/1471-2458-12-891
PMCID: PMC3571948  PMID: 23088168
(3–10): Diabetes mellitus/drug therapy; Patient compliance; Patient education; Pharmacists; Community health workers

Results 1-25 (1443598)