Systematic reviews and meta-analyses of randomized trials that include patient-reported outcomes (PROs) often provide crucial information for patients and clinicians facing challenging health care decisions. Based on emerging methods, guidance on combining PROs in meta-analysis is likely to enhance their usefulness.
The objectives of this paper are: i) to describe PROs and why they are important for health care decision-making, ii) illustrate the key risk of bias issues that systematic reviewers should consider and, iii) address outcome characteristics of PROs and provide guidance for combining outcomes.
We suggest a step-by-step approach to addressing issues of PROs in meta-analyses. Systematic reviewers should begin by asking themselves if trials have addressed all the important effects of treatment on patients’ quality of life. If the trials have addressed PROs, have investigators chosen the appropriate instruments? In particular, does evidence suggest the PROs used are valid and responsive, and is the review free of outcome reporting bias? Systematic reviewers must then decide how to categorize PROs and when to pool results.
Patient-reported outcomes; Health-related quality of life; Meta-analysis; Systematic review; Health care decision-making
Depression is the most frequent reason for receiving disability benefits in North America, and treatment with psychotherapy is often funded by private insurers. No studies have explored the association between the provision of psychotherapy for depression and time to claim closure.
Using administrative data from a Canadian disability insurer, we evaluated the association between the provision of psychotherapy and short-term disability (STD) and long-term disability (LTD) claim closure by performing Cox proportional hazards regression.
We analyzed 10,508 STD and 10,338 LTD claims for depression. In our adjusted analyses, receipt of psychotherapy was associated with longer time to STD closure (HR [99% CI] = 0.81 [0.68 to 0.97]) and faster LTD claim closure (1.42 [1.33 to 1.52]). In both STD and LTD, older age (0.90 [0.88 to 0.92] and 0.83 [0.80 to 0.85]), per decade), a primary diagnosis of recurrent depression versus non-recurrent major depression (0.78 [0.69 to 0.87] and 0.80 [0.72 to 0.89]), a psychological secondary diagnosis (0.90 [0.84 to 0.97] and 0.66 [0.61 to 0.71]), or a non-psychological secondary diagnosis (0.81 [0.73 to 0.90] and 0.77 [0.71 to 0.83]) versus no secondary diagnosis, and an administrative services only policy ([0.94 [0.88 to 1.00] and 0.87 [0.75 to 0.996]) or refund policy (0.86 [0.80 to 0.92] and 0.73 [0.68 to 0.78]) compared to non-refund policy claims were independently associated with longer time to STD claim closure.
We found, paradoxically, that receipt of psychotherapy was independently associated with longer time to STD claim closure and faster LTD claim closure in patients with depression. We also found multiple factors that were predictive of time to both STD and LTD claim closure. Our study has limitations, and well-designed prospective studies are needed to establish the effect of psychotherapy on disabling depression.
To explore the role of patients’ beliefs in their likelihood of recovery from severe physical trauma.
We developed and validated an instrument designed to capture the impact of patients’ beliefs on functional recovery from injury; the Somatic Pre-occupation and Coping (SPOC) questionnaire. At 6-weeks post-surgical fixation, we administered the SPOC questionnaire to 359 consecutive patients with operatively managed tibial shaft fractures. We constructed multivariable regression models to explore the association between SPOC scores and functional outcome at 1-year, as measured by return to work and short form-36 (SF-36) physical component summary (PCS) and mental component summary (MCS) scores.
In our adjusted multivariable regression models that included pre-injury SF-36 scores, SPOC scores at 6-weeks post-surgery accounted for 18% of the variation in SF-36 PCS scores and 18% of SF-36 MCS scores at 1-year. In both models, 6-week SPOC scores were a far more powerful predictor of functional recovery than age, gender, fracture type, smoking status, or the presence of multi-trauma. Our adjusted analysis found that for each 14 point increment in SPOC score at 6-weeks (14 chosen on the basis of half a standard deviation of the mean SPOC score) the odds of returning to work at 1-year decreased by 40% (odds ratio = 0.60; 95% CI = 0.50 to 0.73).
The SPOC questionnaire is a valid measurement of illness beliefs in tibial fracture patients and is highly predictive of their long-term functional recovery. Future research should explore if these results extend to other trauma populations and if modification of unhelpful illness beliefs is feasible and would result in improved functional outcomes.
Investigators often face challenges when recruiting participants into randomized controlled trials (RCTs). Some data suggest that telephone reminders may lead to greater participant enrollment.
Patients aged 50 to 70 years from family practice rosters were initially mailed invitations to participate in an RCT of colorectal cancer screening. Patients who did not respond were randomly allocated to follow-up invitations by either telephone or mail four weeks after the initial invitation. The primary outcome was attendance for eligibility screening with the study nurse.
After mailing invitations to 1,348 patients, 104 patients were initially enrolled in the RCT of colon cancer screening. Of 952 patients who did not respond to the initial mailed invitation, we randomly allocated 480 to follow-up invitation by telephone and 472 to follow-up invitation by mail. Attendance for eligibility screening with the study nurse was more frequent when non-responders were followed-up by telephone (84/480, 17.5%) than by mail (43/472, 9.1%) (relative risk (RR) 1.92, 95% confidence interval (CI) 1.36 to 2.71, P < 0.001). Enrollment into the RCT was also greater among patients followed-up by telephone (59/480, 12.3%) compared to those followed-up by mail (35/472, 7.4%) (RR 1.66, 95% CI 1.11 to 2.47, P=0.01).
Telephone-based follow-up results in greater enrollment compared to a mail-based method. Our findings should be of interest to investigators conducting RCTs, particularly trials of screening interventions involving asymptomatic participants for which volunteer participation may be challenging.
Randomized controlled trial; Recruitment; Telephone; Mail; Colorectal cancer screening
To perform a systematic review and individual participant data meta-analysis to identify preoperative factors associated with a good seizure outcome in children with Tuberous Sclerosis Complex undergoing resective epilepsy surgery.
Electronic databases (MEDLINE, EMBASE, CINAHL and Web of Science), archives of major epilepsy and neurosurgery meetings, and bibliographies of relevant articles, with no language or date restrictions.
We included case-control or cohort studies of consecutive participants undergoing resective epilepsy surgery that reported seizure outcomes. We performed title and abstract and full text screening independently and in duplicate. We resolved disagreements through discussion.
One author performed data extraction which was verified by a second author using predefined data fields including study quality assessment using a risk of bias instrument we developed. We recorded all preoperative factors that may plausibly predict seizure outcomes.
To identify predictors of a good seizure outcome (i.e. Engel Class I or II) we used logistic regression adjusting for length of follow-up for each preoperative variable.
Of 9863 citations, 20 articles reporting on 181 participants were eligible. Good seizure outcomes were observed in 126 (69%) participants (Engel Class I: 102(56%); Engel class II: 24(13%)). In univariable analyses, absence of generalized seizure semiology (OR = 3.1, 95%CI = 1.2–8.2, p = 0.022), no or mild developmental delay (OR = 7.3, 95%CI = 2.1–24.7, p = 0.001), unifocal ictal scalp electroencephalographic (EEG) abnormality (OR = 3.2, 95%CI = 1.4–7.6, p = 0.008) and EEG/Magnetic resonance imaging concordance (OR = 4.9, 95%CI = 1.8–13.5, p = 0.002) were associated with a good postoperative seizure outcome.
Small retrospective cohort studies are inherently prone to bias, some of which are overcome using individual participant data. The best available evidence suggests four preoperative factors predictive of good seizure outcomes following resective epilepsy surgery. Large long-term prospective multicenter observational studies are required to further evaluate the risk factors identified in this review.
Evidence-based practice (EBP) may help improve healthcare quality. However, not all healthcare professionals and managers use EBP in their daily practice. We systematically reviewed the literature to summarise self-reported appreciation of EBP and organisational infrastructure solutions proposed to promote EBP.
Systematic review. Two investigators independently performed the systematic reviewing process.
MEDLINE, EMBASE and Cochrane Library were searched for publications between 2000 and 2011.
Eligibility criteria for included studies
Reviews and surveys of EBP attitude, knowledge, awareness, skills, barriers and facilitators among managers, doctors and nurses in clinical settings.
We found 31 surveys of fairly good quality. General attitude towards EBP was welcoming. Respondents perceived several barriers, but also many facilitators for EBP implementation. Solutions were proposed at various organisational levels, including (inter)national associations and hospital management promoting EBP, pregraduate and postgraduate education, as well as individual support by EBP mentors on the wards to move EBP from the classroom to the bedside.
More than 20 years after its introduction, the EBP paradigm has been embraced by healthcare professionals as an important means to improve quality of patient care, but its implementation is still deficient. Policy exerted at microlevel , middlelevel and macrolevel, and supported by professional, educational and managerial role models, may further facilitate EBP.
Medical Education & Training
To assess the perception of diseases and the willingness to use public-tax revenue for their treatment among relevant stakeholders.
A population-based, cross-sectional mailed survey.
3000 laypeople, 1500 doctors, 1500 nurses (randomly identified from the databases of the Finnish Population Register, the Finnish Medical Association and the Finnish Nurses Association) and all 200 parliament members.
Main outcome measures
Respondents’ perspectives on a five-point Likert scale on two claims on 60 states of being: ‘(This state of being) is a disease’; and ‘(This state of being) should be treated with public tax revenue’.
Of the 6200 individuals approached, 3280 (53%) responded. Of the 60 states of being, ≥80% of respondents considered 12 to be diseases (Likert scale responses of ‘4’ and ‘5’) and five not to be diseases (Likert scale responses of ‘1’ and ‘2’). There was considerable variability in most states, and great variability in 10 (≥20% of respondents of all groups considered it a disease and ≥20% rejected as a disease). Doctors were more inclined to consider states of being as diseases than laypeople; nurses and members were intermediate (p<0.001), but all groups showed large variability. Responses to the two claims were very strongly correlated (r=0.96 (95% CI 0.94 to 0.98); p<0.001).
There is large disagreement among the public, health professionals and legislators regarding the classification of states of being as diseases and whether their management should be publicly funded. Understanding attitudinal differences can help to enlighten social discourse on a number of contentious public policy issues.
Epidemiology; Medical Ethics; General Medicine (see Internal Medicine); Health Economics; Public Health
Differences in medical care in the United States compared with Canada, including greater reliance on private funding and for-profit delivery, as well as markedly higher expenditures, may result in different health outcomes.
To systematically review studies comparing health outcomes in the United States and Canada among patients treated for similar underlying medical conditions.
We identified studies comparing health outcomes of patients in Canada and the United States by searching multiple bibliographic databases and resources. We masked study results before determining study eligibility. We abstracted study characteristics, including methodological quality and generalizability.
We identified 38 studies comparing populations of patients in Canada and the United States. Studies addressed diverse problems, including cancer, coronary artery disease, chronic medical illnesses and surgical procedures. Of 10 studies that included extensive statistical adjustment and enrolled broad populations, 5 favoured Canada, 2 favoured the United States, and 3 showed equivalent or mixed results. Of 28 studies that failed one of these criteria, 9 favoured Canada, 3 favoured the United States, and 16 showed equivalent or mixed results. Overall, results for mortality favoured Canada (relative risk 0.95, 95% confidence interval 0.92-0.98, p= 0.002) but were very heterogeneous, and we failed to find convincing explanations for this heterogeneity. The only condition in which results consistently favoured one country was end-stage renal disease, in which Canadian patients fared better.
Available studies suggest that health outcomes may be superior in patients cared for in Canada versus the United States, but differences are not consistent.
Pregnant women with prior venous thromboembolism (VTE) are at risk of recurrence. Low molecular weight heparin (LWMH) reduces the risk of pregnancy-related VTE. LMWH prophylaxis is, however, inconvenient, uncomfortable, costly, medicalizes pregnancy, and may be associated with increased risks of obstetrical bleeding. Further, there is uncertainty in the estimates of both the baseline risk of pregnancy-related recurrent VTE and the effects of antepartum LMWH prophylaxis. The values and treatment preferences of pregnant women, crucial when making recommendations for prophylaxis, are currently unknown. The objective of this study is to address this gap in knowledge.
We will perform a multi-center cross-sectional interview study in Canada, USA, Norway and Finland. The study population will consist of 100 women with a history of lower extremity deep vein thrombosis (DVT) or pulmonary embolism (PE), and who are either pregnant, planning pregnancy, or may in the future consider pregnancy (women between 18 and 45 years). We will exclude individuals who are on full dose anticoagulation or thromboprophylaxis, who have undergone surgical sterilization, or whose partners have undergone vasectomy. We will determine each participant's willingness to receive LMWH prophylaxis during pregnancy through direct choice exercises based on real life and hypothetical scenarios, preference-elicitation using a visual analog scale (“feeling thermometer”), and a probability trade-off exercise. The primary outcome will be the minimum reduction (threshold) in VTE risk at which women change from declining to accepting LMWH prophylaxis. We will explore possible determinants of this choice, including educational attainment, the characteristics of the women’s prior VTE, and prior experience with LMWH. We will determine the utilities that women place on the burden of LMWH prophylaxis, pregnancy-related DVT, pregnancy-related PE and pregnancy-related hemorrhage. We will generate a “personalized decision analysis” using participants’ utilities and their personalized risk of recurrent VTE as inputs to a decision analytic model. We will compare the personalized decision analysis to the participant’s stated choice.
The preferences of pregnant women at risk of VTE with respect to the use of antithrombotic therapy remain unexplored. This research will provide explicit, quantitative expressions of women's valuations of health states related to recurrent VTE and its prevention with LMWH. This information will be crucial for both guideline developers and for clinicians.
Guideline panellists have differing opinions on whether resource use should influence decisions on individual patients. As medical care costs rise, resource use considerations become more compelling, but panellists may find dealing with such considerations challenging
The GRADE system can be used to grade the quality of evidence and strength of recommendations for diagnostic tests or strategies. This article explains how patient-important outcomes are taken into account in this process
The GRADE system classifies recommendations made in guidelines as either strong or weak. This article explores the meaning of these descriptions and their implications for patients, clinicians, and policy makers
Guideline developers use a bewildering variety of systems to rate the quality of the evidence underlying their recommendations. Some are facile, some confused, and others sophisticated but complex
Guidelines are inconsistent in how they rate the quality of evidence and the strength of recommendations. This article explores the advantages of the GRADE system, which is increasingly being adopted by organisations worldwide
Acute lung injury (ALI) and acute respiratory distress syndrome (ARDS) are life threatening clinical conditions seen in critically ill patients with diverse underlying illnesses. Lung injury may be perpetuated by ventilation strategies that do not limit lung volumes and airway pressures. We conducted a systematic review and meta-analysis of randomized controlled trials (RCTs) comparing pressure and volume-limited (PVL) ventilation strategies with more traditional mechanical ventilation in adults with ALI and ARDS.
Methods and Findings
We searched Medline, EMBASE, HEALTHSTAR and CENTRAL, related articles on PubMed™, conference proceedings and bibliographies of identified articles for randomized trials comparing PVL ventilation with traditional approaches to ventilation in critically ill adults with ALI and ARDS. Two reviewers independently selected trials, assessed trial quality, and abstracted data. We identified ten trials (n = 1,749) meeting study inclusion criteria. Tidal volumes achieved in control groups were at the lower end of the traditional range of 10–15 mL/kg. We found a clinically important but borderline statistically significant reduction in hospital mortality with PVL [relative risk (RR) 0.84; 95% CI 0.70, 1.00; p = 0.05]. This reduction in risk was attenuated (RR 0.90; 95% CI 0.74, 1.09, p = 0.27) in a sensitivity analysis which excluded 2 trials that combined PVL with open-lung strategies and stopped early for benefit. We found no effect of PVL on barotrauma; however, use of paralytic agents increased significantly with PVL (RR 1.37; 95% CI, 1.04, 1.82; p = 0.03).
This systematic review suggests that PVL strategies for mechanical ventilation in ALI and ARDS reduce mortality and are associated with increased use of paralytic agents.
Guidelines exist for the surgical treatment of hip fracture, but the effect of early surgery on mortality and other outcomes that are important for patients remains unclear. We conducted a systematic review and meta-analysis to determine the effect of early surgery on the risk of death and common postoperative complications among elderly patients with hip fracture.
We searched electronic databases (including MEDLINE and EMBASE), the archives of meetings of orthopedic associations and the bibliographies of relevant articles and questioned experts to identify prospective studies, published in any language, that evaluated the effects of early surgery in patients undergoing procedures for hip fracture. Two reviewers independently assessed methodologic quality and extracted relevant data. We pooled data by means of the DerSimonian and Laird random-effects model, which is based on the inverse variance method.
We identified 1939 citations, of which 16 observational studies met our inclusion criteria. These studies had a total of 13 478 patients for whom mortality data were complete (1764 total deaths). Based on the five studies that reported adjusted risk of death (4208 patients, 721 deaths), irrespective of the cut-off for delay (24, 48 or 72 hours), earlier surgery (i.e., within the cut-off time) was associated with a significant reduction in mortality (relative risk [RR] 0.81, 95% confidence interval [CI] 0.68–0.96, p = 0.01). Unadjusted data indicated that earlier surgery also reduced in-hospital pneumonia (RR 0.59, 95% CI 0.37–0.93, p = 0.02) and pressure sores (RR 0.48, 95% CI 0.34–0.69, p < 0.001).
Earlier surgery was associated with a lower risk of death and lower rates of postoperative pneumonia and pressure sores among elderly patients with hip fracture. These results suggest that reducing delays may reduce mortality and complications.
Systematic reviews of randomized trials that include measurements of health-related quality of life potentially provide critical information for patient and clinicians facing challenging health care decisions. When, as is most often the case, individual randomized trials use different measurement instruments for the same construct (such as physical or emotional function), authors typically report differences between intervention and control in standard deviation units (so-called "standardized mean difference" or "effect size"). This approach has statistical limitations (it is influenced by the heterogeneity of the population) and is non-intuitive for decision makers. We suggest an alternative approach: reporting results in minimal important difference units (the smallest difference patients experience as important). This approach provides a potential solution to both the statistical and interpretational problems of existing methods.
The current epidemic of diabetes makes a drug to prevent it attractive. But despite promotion of recent research evidence, Victor Montori, William Isley, and Gordon Guyatt argue that we are not there yet
People with known risk factors for chronic obstructive pulmonary disease (COPD) are important targets for screening and early intervention. We sought to measure the prevalence of COPD among such individuals visiting a primary care practitioner for any reason. We also evaluated the accuracy of prior diagnosis or nondiagnosis of COPD and identified associated clinical characteristics.
We recruited patients from three primary care sites who were 40 years or older and had a smoking history of at least 20 pack-years. Participants were asked about respiratory symptoms and underwent postbronchodilator spirometry. COPD was defined as a ratio of forced expiratory volume in the first second of expiration to forced vital capacity (FEV1/FVC) of less than 0.7 and an FEV1 of less than 80% predicted.
Of the 1459 patients who met the study criteria, 1003 (68.7%) completed spirometry testing. Of these, 208 were found to have COPD, for a prevalence of 20.7% (95% confidence interval 18.3%–23.4%). Of the 205 participants with COPD who completed the interview about respiratory symptoms before spirometry, only 67 (32.7%) were aware of their diagnosis before the study. Compared with patients in whom COPD had been correctly diagnosed before the study, those in whom COPD had been over-diagnosed or undiagnosed were similar in terms of age, sex, current smoking status and number of visits to a primary care practitioner because of a respiratory problem.
Among adult patients visiting a primary care practitioner, as many as one in five with known risk factors met spirometric criteria for COPD. Underdiagnosis of COPD was frequent, which suggests a need for greater screening of at-risk individuals. Knowledge of the prevalence of COPD will help plan strategies for disease management.
Comparing the effectiveness of interventions is now a requirement for regulatory approval in several countries. It also aids in clinical and public health decision-making. However, in the absence of head-to-head randomized trials (RCTs), determining the relative effectiveness of interventions is challenging. Several methodological options are now available. We aimed to determine the comparative validity of the adjusted indirect comparisons of RCTs with the mixed treatment comparison approach.
Using systematic searching, we identified all meta-analyses evaluating more than 3 interventions for a similar disease state with binary outcomes. We abstracted data on each clinical trial including population n and outcomes. We conducted fixed effects meta-analysis of each intervention versus mutual comparator and then applied the adjusted indirect comparison. We conducted a mixed treatment meta-analysis on all trials and compared the point estimates and 95% confidence/credible intervals (CIs/CrIs) to determine important differences.
We included data from 7 reviews that met our inclusion criteria, allowing a total of 51 comparisons. According to the a priori consistency rule, we found 2 examples where the analytic comparisons were statistically significant using the mixed treatment comparison over the adjusted indirect comparisons and 1 example where this was vice versa. We found 6 examples where the direction of effect differed according to the indirect comparison method chosen and we found 9 examples where the confidence intervals were importantly different between approaches.
In most analyses, the adjusted indirect comparison yields estimates of relative effectiveness equal to the mixed treatment comparison. In less complex indirect comparisons, where all studies share a mutual comparator, both approaches yield similar benefits. As comparisons become more complex, the mixed treatment comparison may be favoured.
Teaching the content of clinical practice guidelines (CPGs) is important to both clinical care and graduate medical education. The objective of this study was to determine the characteristics of curricula for teaching the content of CPGs in family medicine and internal medicine residency programs in the United States.
We surveyed the directors of family medicine and internal medicine residency programs in the United States. The questionnaire included questions about the characteristics of the teaching of CPGs: goals and objectives, educational activities, evaluation, aspects of CPGs that the program teaches, the methods of making texts of CPGs available to residents, and the major barriers to teaching CPGs.
Of 434 programs responding (out of 839, 52%), 14% percent reported having written goals and objectives related to teaching CPGs. The most frequently taught aspect was the content of specific CPGs (76%). The top two educational strategies used were didactic sessions (76%) and journal clubs (64%). Auditing for adherence by residents was the primary evaluation strategy (44%), although 36% of program directors conducted no evaluation. Programs made texts of CPGs available to residents most commonly in the form of paper copies (54%) while the most important barrier was time constraints on faculty (56%).
Residency programs teach different aspects of CPGs to varying degrees, and the majority uses educational strategies not supported by research evidence.
Although crossover trials enjoy wide use, standards for analysis and reporting have not been established. We reviewed methodological aspects and quality of reporting in a representative sample of published crossover trials.
We searched MEDLINE for December 2000 and identified all randomized crossover trials. We abstracted data independently, in duplicate, on 14 design criteria, 13 analysis criteria, and 14 criteria assessing the data presentation.
We identified 526 randomized controlled trials, of which 116 were crossover trials. Trials were drug efficacy (48%), pharmacokinetic (28%), and nonpharmacologic (30%). The median sample size was 15 (interquartile range 8–38). Most (72%) trials used 2 treatments and had 2 periods (64%). Few trials reported allocation concealment (17%) or sequence generation (7%). Only 20% of trials reported a sample size calculation and only 31% of these considered pairing of data in the calculation. Carry-over issues were addressed in 29% of trial's methods. Most trials reported and defended a washout period (70%). Almost all trials (93%) tested for treatment effects using paired data and also presented details on by-group results (95%). Only 29% presented CIs or SE so that data could be entered into a meta-analysis.
Reports of crossover trials frequently omit important methodological issues in design, analysis, and presentation. Guidelines for the conduct and reporting of crossover trials might improve the conduct and reporting of studies using this important trial design.