Systematic review to determine whether participation in a trial influences outcome

doi:10.1136/bmj.330.7501.1175

BMJ. 2005 May 21; 330(7501): 1175.

PMCID: PMC558011

Systematic review to determine whether participation in a trial influences outcome

Gunn Elisabeth Vist, senior researcher,¹ Kåre Birger Hagen, senior researcher,² P J Devereaux, senior researcher,³ Dianne Bryant, senior researcher,³ Doris Tove Kristoffersen, statistician,¹ and Andrew David Oxman, researcher¹

¹ Norwegian Health Services Research Centre, PO Box 7004, 0130 Oslo, Norway

² National Research Centre for Rehabilitation in Rheumatology, Diakonhjemmet Hospital, Oslo

³ Clinical Epidemiology and Biostatistics, McMaster University Health Sciences Centre, Hamilton, Ontario, Canada

Correspondence to: G E Vist gev/at/nhsrc.no

Accepted March 31, 2005.

This article has been cited by other articles in PMC.

Abstract

Objective To systematically compare the outcomes of participants in randomised controlled trials (RCTs) with those in comparable non-participants who received the same or similar treatment.

Data sources Bibliographic databases, reference lists from eligible articles, medical journals, and study authors.

Review methods RCTs and cohort studies that evaluated the clinical outcomes of participants in RCTs and comparable non-participants who received the same or similar treatment.

Results Five RCTs (six comparisons) and 50 cohort studies (85 comparisons) provided data on 31 140 patients treated in RCTs and 20 380 comparable patients treated outside RCTs. In the five RCTs, in which patients were given the option of participating or not, the comparisons provided limited information because of small sample sizes (a total of 412 patients) and the nature of the questions considered. 73 dichotomous outcomes were compared, of which 59 reported no statistically significant differences. For patients treated within RCTs, 10 comparisons reported significantly better outcomes and four reported significantly worse outcomes. Significantly heterogeneity was found (I² = 89%) among the comparisons of 73 dichotomous outcomes; none of our a priori explanatory factors helped explain this heterogeneity. The 18 comparisons of continuous outcomes showed no significant differences in heterogeneity (I² = 0%). The overall pooled estimate for continuous outcomes of the effect of participating in an RCT was not significant (standardised mean difference 0.01, 95% confidence interval -0.10 to 0.12).

Conclusion No strong evidence was found of a harmful or beneficial effect of participating in RCTs compared with receiving the same or similar treatment outside such trials.

Introduction

Properly conducted randomised controlled trials (RCTs) provide the strongest evidence of the effects of treatment.¹ It is, however, controversial as to whether participants of such trials benefit directly or whether these studies are solely for the benefit of future patients. In addition, there is much scepticism about the applicability of the results to usual practice.²

Four reviews that considered whether it is beneficial or harmful to participate in RCTs drew varied conclusions.³^-⁶ These reviews compared patients who were treated within trials with those treated outside the trials, regardless of differences between the clinical interventions or between the participants and non-participants. It is therefore uncertain whether the results reflect the effects of participating in an RCT (trial effects), differences in the clinical interventions (treatment effects), or differences between participants and non-participants. We determined whether the outcomes of participants in RCTs differed from those of comparable non-participants who received the same or similar treatment.

Methods

Our review was undertaken as a Cochrane methodology review (see Cochrane Library for fuller details of our methods and updated versions of the review).⁷

We included studies that compared participants in RCTs with comparable non-participants who received the same or similar treatment. We included observational studies and RCTs in which participation or the option of participation was randomly allocated.

Search strategies

We used seven strategies to identify relevant studies: consultation with experts; search of personal files; electronic searches of the Cochrane central register of controlled trials, Medline, Embase, the Cochrane methodology register, and PsycINFO; a review of reference lists from eligible articles; and a search of PubMed using the “related articles” feature and SciSearch. We also hand searched articles published in 2000 in five medical journals (BMJ, Annals of Internal Medicine, JAMA, Lancet, and New England Journal of Medicine) to identify RCTs with over 200 patients and at least 100 eligible non-participants. Studies' authors were contacted for data on the treatment and outcomes of eligible non-participants.

Assessment of study eligibility

Two reviewers independently assessed each article for eligibility. Disagreements were resolved by discussion. A third reviewer was consulted when consensus could not be reached.

Data abstraction

Two reviewers independently abstracted data from eligible studies. Each study was assessed for selection bias (differences between participants and non-participants), detection bias, and exclusion bias (losses to follow-up). On the basis of the combined risks of the three biases, we grouped each comparison into overall quality groups (randomised, controlled comparisons, partially controlled comparisons, and poorly controlled comparisons) for analysis. Missing data were sought from the investigators.

Analysis

We compared the experimental group of the RCT with their respective eligible non-participants who received the treatment and the control group and eligible non-participants who received the control treatment outside of the trial. For each comparison, we analysed the main outcome, and, when reported, we analysed mortality separately. For all of the included comparisons we used the main outcome as reported by the investigators. We analysed the dichotomous and continuous results separately. The results are reported as relative risks with 95% confidence intervals, using adjusted estimates when available. For a summary of the dichotomous results in one table, we calculated the associated standard error from the natural logarithm of the unadjusted relative risk. Heterogeneity was assessed by χ² test and the I² statistic using RevMan version 4.2.⁸ For the unadjusted relative risk analysis, we used the Mantel-Haenszel test in RevMan. A fixed effect model was used to calculate summary statistics if no statistically significant (P < 0.10) heterogeneity was found among similar comparisons. For statistically heterogeneous results we described the variation in the estimates and key explanatory factors; where possible relating the explanatory factors to observed differences in estimates of the effects of participation. We constructed a funnel plot to explore the possibility of publication bias.

Table 1

Summary of studies included in sensitivity analysis

Results

Overall, 55 studies, totalling 91 comparisons, met our inclusion criteria (fig 1). Forty one studies are still awaiting assessment, which currently cannot be included or excluded on the basis of the published information.

Fig 1

Selection process of eligible studies

We identified five RCTs (six comparisons) in which patients were randomised according to whether they had the option to participate. These studies provided limited data because of their small sample sizes and the nature of the questions considered. Two studies randomised 82 patients to “n of 1 trials” compared with standard practice—that is, randomised, double blind, multiple crossover comparisons of an active drug with a placebo in a single patient.⁹^,10 One study (60 patients) measured spontaneously self reported side effects in patients who had or had not been informed that they were in an RCT.¹¹ One study (227 patients) reported satisfaction among patients randomised to an RCT compared with patients randomised to a patient preference trial in which they had a choice of treatment.¹² Another study (43 patients) reported pain reduction among patients randomised to an RCT compared with those who were not invited to participate.¹³ None of these studies found significant differences in outcomes between patients treated in or outside RCTs.

Non-randomised cohort studies

We identified 50 cohort studies (85 comparisons) totalling 30 862 patients participating in RCTs compared with 20 246 patients treated outside RCTs.¹⁴^-⁶³ Seventy comparisons comprised dichotomous outcomes, of which 12 reported adjusted estimates, and 15 comparisons comprised continuous outcomes.

We found significant heterogeneity (I² = 89.0%) among the results of comparisons with dichotomous main outcomes (fig 2); these results were therefore not pooled. Of these 73 comparisons, 59 reported no significant differences between outcomes for patients treated in RCTs and those receiving similar treatments outside RCTs; 10 reported significantly better outcomes for patients treated in RCTs, and four reported significantly worse outcomes for patients treated in RCTs.

Fig 2

Results of dichotomous main outcomes in participants of randomised controlled trials and comparable non-participants who received the same or similar treatment

Figure 3 shows the results of the 18 comparisons with continuous main outcomes. We found no significant heterogeneity (I² = 0%). The pooled estimate found no differences in outcomes for patients treated in and outside RCTs (standardised mean difference 0.01, 95% confidence interval -0.10 to 0.12).

Fig 3

Results of continuous main outcomes in participants of randomised controlled trials and comparable non-participants who received the same or similar treatment

In 17 studies (32 comparisons) with data on mortality (fig 4), we found significant heterogeneity (I² = 88.8%); the results were therefore not pooled. In 24 of the 32 comparisons we found no significant difference in mortality. Four comparisons reported a significant lower risk of mortality for patients treated in RCTs and four comparisons reported a significantly higher risk of mortality.

Fig 4

Comparison of mortality between participants of randomised controlled trials and comparable non-participants who received the same or similar treatment

Separate subgroup analyses could not explain the observed heterogeneity by the different types of eligible non-participants, treatments, clinical specialities, or study quality (selection bias, detection bias, and exclusion bias). The table summarises the studies included in the sensitivity analysis.

The funnel plot of the dichotomous comparisons showed no asymmetry (fig 5), indicating a low risk of publication bias.

Fig 5

Funnel plot of dichotomous comparisons in participants of randomised controlled trials and comparable non-participants who received the same or similar treatment

Discussion

Our systematic review found no strong evidence of a harmful or beneficial trial effect of participating in randomised controlled trials (RCTs). The five included RCTs provided limited evidence because of their small sample sizes and the nature of the questions they considered, but they did show that it is possible to consider questions about the effects of participating in RCTs by using randomised designs. Our interpretation of the 50 non-randomised cohort studies was limited by the quality and size of the comparisons and the wide variations in participants, clinical interventions, and outcomes. Most of the 85 non-randomised cohort comparisons found no statistically significant differences, although 10 reported better outcomes for patients in RCTs and four reported better outcomes for patients outside of RCTs.

Previous reviews that considered a less precise question than the one we evaluated drew varied conclusions. For example, one identified 14 articles reporting data from 21 trials and concluded that, if anything, randomised trials tend to have beneficial rather than harmful effects.⁴ Another review included seven of these 14 articles and 17 additional articles.⁶ Only eight of the studies compared trial patients with non-trial patients who met the same eligibility criteria, and it was only possible to separate treatment effects from trial effects in three of these trials. A further review found 10 comparisons on survival or quality of life of patients treated in RCTs for life threatening illnesses (eight were cancer treatments) with those treated outside RCTs.⁵ It found evidence of longer survival in participants, but the authors were not confident of the results.

Our review differs from these reviews in several ways, including the scope and comprehensiveness of our search, our method of analysis, and the question we asked, which controlled for differences in the effects of different interventions and differences between participants and non-participants. (See bmj.com for an evidence profile of our results according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group.) Our results are based mainly on comparisons of cohorts and are subject to the usual uncertainty associated with observational studies.¹ Additionally, we could not explain the significant heterogeneity between studies, which affects our confidence in the results and reduces the overall quality of information. Other relevant studies apart from those included in this review may exist, as indicated by the number of studies awaiting assessment and the difficulty we and others had in searching for this type of study in electronic databases. As we did not find evidence of publication bias, it is unlikely that the studies that we failed to identify would provide strong evidence of either harmful or beneficial effects.

An important corollary of this finding is that it counters suggestions that the results of RCTs cannot be applied to usual clinical practice, because most of the studies found no significant difference in outcomes for participants of RCTs compared with comparable non-participants who received similar treatment.

In most cases, RCTs seem to provide estimates of treatment effects that are applicable to comparable patients who receive similar interventions in usual clinical practice. In addition to being informed about the risks and harms of an intervention when invited to participate in RCTs, patients can be told that, independently of the effects of the interventions being compared, participating in a trial is likely to result in similar outcomes to patients who receive the same or similar treatment outside of the trial.

What is already known on this topic

Some people believe that participation in a randomised controlled trial (RCT) increases a patient's risk of a bad outcome

Some people claim that the results of RCTs are not applicable to usual clinical practice

What this study adds

Participants in RCTs had similar outcomes to comparable patients who received the same or similar treatment outside the trial

The results of RCTs are therefore applicable to comparable patients in usual clinical practice

Supplementary Material

Table

Click here to view.

Notes

An external file that holds a picture, illustration, etc.
Object name is webplus.f1.gif

A table giving the evidence profile of results is on bmj.com

We thank Dave Sackett for initiating this review, for his contributions to the protocol, and for his support; Lena Nordheim for help with the search strategy; and the following for additional information about their studies: S M Albert, B J Barrett, J M Bennett, S B Black, S S Brehm, G Berglund, J F Bergmann, N Bijker, M Blichert-Toft, L Boros, J G Cairncross, K M Carroll, D W Chadwick, S P Chauhan, C Chilvers, G P Clagett, C J Cohen, K Collins, J S Cooper, K G Cooper, V Cottin, U Creutzig, S Davis, E P DiMagno, J van Eys, P J Ferrone, G M Forbes, J Frisell, B Glimelius, C Gluud, F Guillemin, J C Hauth, M Helsing, D Hellberg, J M Henderson, A M Hutter, L P Jensen, J J C Jonker, T Karrison, S B King, D Kjar, R A Kronmal, R W Licht, E Lidbrink, W F Liu, J K Madsen, J L Mahon, M A Malangoni, C Marcocci, J R McKay, M P Mourits, C Moynihan, G R Mundy, C D Naylor, M G Neuwirth, J R O'Fallon, J L Paradise, A Pinchera, M J Playforth, A V Pollock, P D Rokke, M A Rosen, M M Rovers, R G Rychtarik, B Schmidt, C Schmoor, B E Sha, J H Shelhamer, B G Stegmayr, C A Stiller, J M Stone, R Takolander, I F Tannock, P Vestergaard, B Ward, C Weijer, D J Weisdorf, D G Wyse, B Yersin, and V L Yu.

Contributors: GEV carried out the literature search. GEV and KBH selected the studies. GEV, KBH, PJD, and DB extracted the data. GEV, DTK, and ADO analysed the data. GEV, KBH, PJD, DB, and ADO drafted the manuscript. GEV, KBH, PJD, and ADO developed the protocol and review. DB hand searched the five medical journals. GEV is guarantor for the paper.

Funding: Norwegian Health Services Research Centre, McMaster University, and the Nuffield Trust.

Competing interests: None declared.

Ethical approval: Not required.

References

1. Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in healthcare trials (Cochrane methodology review). Issue 4. Oxford: Update Software, 2002.

2. Rothwell PM. External validity of randomised controlled trials: “To whom do the results of this trial apply?” Lancet 2005;365: 82-93. [PubMed]

3. Stiller CA. Centralised treatment, entry to trials and survival. Br J Cancer 1994;70: 352-62. [PMC free article] [PubMed]

4. Braunholtz DA, Edwards SJL, Lilford RJ. Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect.” J Clin Epidemiol 2001;54: 217-24. [PubMed]

5. The Emergency Care Research Institute 2002. Patients' reasons for participation in clinical trials and effect of trial participation on patient outcomes. www.ecri.org/Patient_Information/Patient_Reference_Guide/evidence.pdf (accessed Nov 2004).

6. Peppercorn JM, Weeks JC, Cook EFC, Joffe S. Comparison of outcomes in cancer patients treated within and outside clinical trials: conceptual framework and structured review. Lancet 2004;363: 263-70. [PubMed]

7. Vist GE, Hagen KB, Devereaux P, Bryant D, Kristoffersen DT, Oxman AD. Outcomes of patients who participate in randomised controlled trials compared to similar patients receiving similar interventions who do not participate (Cochrane methodology reviews). Issue 1. Oxford: Update Software, 2005. [PubMed]

8. Higgins JP, Thompson SG. Quantifying heterogeneity in meta-analysis. Stat Med 2002;21: 1539-58. [PubMed]

9. Mahon J, Laupacis A, Donner A, Wood T. Randomised study of n of 1 trials versus standard practice. BMJ 1996;312: 1069-74. [PMC free article] [PubMed]

10. Mahon JL, Laupacis A, Hodder RV, McKim DA, Paterson NAM, Wood TE, et al. Theophylline for irreversible chronic airflow limitation. A randomized study comparing n of 1 trials to standard practice. Chest 1999;115: 38-48. [PubMed]

11. Dahan R, Caulin C, Figea L, Kanis JA, Caulin F, Segrestaa JM. Does informed consent influence therapeutic outcome? A clinical trial of the hypnotic activity of placebo in patients admitted to hospital. BMJ 1986;293: 363-4. [PMC free article] [PubMed]

12. Cooper KG, Grant AM, Garratt AM. The impact of using a partially randomised patient preference design when evaluating alternative managements for heavy menstrual bleeding. Br J Obstet Gynaecol 1997;104: 1367-73. [PubMed]

13. Bergmann JF, Chassany O, Gandiol J, Deblois P, Kanis JA, Segrestaa JM, et al. A randomised clinical trial of the effect of informed consent on the analgesic activity of placebo and naproxen in cancer pain. Clin Trials Metaanal 1994;29: 41-7. [PubMed]

14. Balmukhanov SB, Beisebaev AA, Aitkoolova ZI, Mustaphin JS, Philippenko VI, Rismuhamedova RS, et al. Intramural and parametrial infusion of metronidazole in the radiotherapy of uterine cervix cancer: preliminary report. Int J Radiation Oncol Biol Phys 1989;16: 1061-3. [PubMed]

15. Baum E, Sather H, Nachman J, Seinfeld J, Kritvit W, Leikin S, et al. Relapse rates following cessation of chemotherapy during complete remission of acute lymphocytic leukemia: a report from childrens cancer study group. Med Pediatr Oncol 1979;7: 25-34. [PubMed]

16. Bedi N, Chilvers C, Churchill R, Dewey M, Duggan C, Fielding K, et al. Assessing effectiveness of treatment of depression in primary care: partially randomised preference trial. Br J Psychol 2000;177: 312-8. [PubMed]

17. Berglund G, Bolund C, Gustafsson UL, Sjoden PO. Is the wish to participate in a cancer rehabilitation program an indicator of the need? Comparisons of participants and non-participants in a randomized study. Psychooncology 1997;6: 35-46. [PubMed]

18. Bhattacharya S, Cameron IM, Mollison J, Parkin DE, Abramovich DR, Kitchener HC. Admission-discharge policies for hysteroscopic surgery: a randomised comparison of day case with in-patient admission. Eur J Obstet Gynecol Reprod Biol 1998;76: 81-4. [PubMed]

19. Bijker N, Peterse JL, Fentiman TS, Julien JP, Hart AAM, Avril A, et al. Effects of patient selection on the applicability of results from a randomised clinical trial (EORTC 10853) investigating breast-conserving treatment for DCIS. Breast conserving therapy for ductal carcinoma in situ. PhD Thesis, 2000.

20. Blichert-Toft M, Brincker H, Andersen JA, Andresen KW, Axelsson CK, Mouridsen HT, et al.. A Danish randomized trial comparing breast-conserving therapy with mastectomy in mammary carcinoma: preliminary results. Acta Oncologica 1988;27: 671-7. [PubMed]

21. Coronary artery surgery study (CASS): a randomised trial of coronary artery bypass surgery. Comparability of entry characteristics and survival in randomized patients and nonrandomized patients meeting randomization criteria. J Am Coll Cardiol 1984;3: 114-28. [PubMed]

22. Chauhan SP, Rutherford SE, Hess LW, Morrison JC. Prophylactic intrapartum amnioinfusion for patients with oligohydramnios: a prospective randomized study. J Reprod Med 1991;37: 817-20. [PubMed]

23. Chilvers C, Dewey M, Fielding K, Gretton V, Miller P, Palmer B, et al. Antidepressant drugs and generic counselling for treatment of major depression in primary care: randomised trial with patient preference arms. BMJ 2001;322: 772-5. [PMC free article] [PubMed]

24. Clagett GP, Youkey JR, Bringham RA, Orecchia PM, Salander JM, Collins GJ, et al. Asymptomatic cervical bruit and abnormal ocular pneumoplethysmography: a prospective study comparing two approaches to management. Surgery 1984;96: 823-30. [PubMed]

25. Creutzig U, Ritter J, Zimmermann M, Schellong G. Does cranial irradiation reduce the risk of bone marrow relapse in acute myelogenous leukemia? Unexpected results of the childhood acute myelogenous leukemia study BFM-87. J Clin Oncol 1993;11: 279-86. [PubMed]

26. Davis S, Wright PW, Schulman SF, Hill LD, Pinkham RD, Johnson LP, et al. Participants in prospective, randomized clinical trials for resected non-small cell lung cancer have improved survival compared with nonparticipants in such trials. Cancer 1985;56: 1710-8. [PubMed]

27. Edsmyr F, Esposti PL, Johansson B, Strindberg B. Clinical experimental randomized study of 2.6-CIS-diphenylhexamethylcyclotetrasiloxane and estramurine-17-phosphate in the treatment of prostatic carcinoma. J Urol 1978;120: 705-7. [PubMed]

28. Feit F, Brooks MM, Sopko G, Keller NM, Rosen A, Krone R, et al. Long-term clinical outcome in the bypass angioplasty revascularization investigation registry: comparison with the randomized trial. Circulation 2000;101: 2795-802. [PubMed]

29. Forbes GM, Collins BJ. Nitrous oxide for colonoscopy: a randomized controlled study. Gastrointest Endosc 2000;51: 271-7. [PubMed]

30. Forssell C, Takolander R, Bergqvist D, Johansson A, Persson NH. Local versus general anaesthesia in carotid surgery. A prospective, randomised study. Eur J Vasc Surg 1989;3: 503-9. [PubMed]

31. Helsing M, Bergman B, Thaning L, Hero U, for the Joint Lung Cancer Study Group. Quality of life and survival in patients with advanced non-small cell lung cancer receiving supportive care plus chemotherapy with carboplatin and etoposide or supportive care only. A multicentre randomised phase III trial. Eur J Cancer 1998;34: 1036-44. [PubMed]

32. Henshaw RC, Naji SA, Templeton AA. Comparison of medical abortion with surgical vacuum aspiration: women's preferences and acceptability of treatment. BMJ 1993;307: 714-7. [PMC free article] [PubMed]

33. Kieler H, Hellberg D, Nilsson S, Waldenstrøm U, Axelsson O. Pregnancy outcome among non-participants in a trial on ultrasound screening. Ultrasound Obstet Gynecol 1998;11: 104-9. [PubMed]

34. King III SB, Barnhart HX, Kosinski AS, Weintraub WS, Lembo NJ, Petersen JY, et al. Angioplasty or surgery for multivessel coronary artery disease: comparison of eligible registry and randomized patients in the EAST trial and influence of treatment selection on outcomes. Am J Cardiol 1997;79: 1453-9. [PubMed]

35. Lidbrink E, Frisell J, Rosendahl I, Rutqvist LE. Nonattendance in the Stockholm mammography screening trial: relative mortality and reasons for nonattendance. Breast Cancer Res Treat 1995;35: 267-75. [PubMed]

36. Link MP, Goorin AM, Horowitz M, Meyer WH, Belasco J, Baker A, et al. Adjuvant chemotherapy of high-grade osteosarcoma of the extremity. Updated results of the multi-institutional osteosarcoma study. Clin Orthop Relat Res 1991;270: 8-14. [PubMed]

37. Liu WF, Harrington T. The need for delivery room intubation of thin meconium in the low risk newborn: a clinical trial. Am J Perinatol 1998;15: 675-82. [PubMed]

38. Mayo Asymptomatic Carotid Endarterectomy Study Group. Results of a randomized controlled trial of carotid endarterectomy for asymptomatic carotid stenosis. Mayo Clinic Proc 1992;67: 513-8. [PubMed]

39. Martinez-Amenos A, Ferre MLF, Vidal CM, Rocasalbas JA. Evaluation of two educative models in primary care hypertension programme. J Hum Hypertens 1990;4: 362-4. [PubMed]

40. McCaughey ES, Mulligan J, Voss LD, Betts PR. Randomised trial of growth hormone in short normal girls. Lancet 1998;351: 940-4. [PubMed]

41. McKay JR, Alterman AI, McLellan AT, Snider EC, O'Brian CP. Effect of random versus nonrandom assignment in a comparison of inpatient and day hospital rehabilitation for male alcoholics. J Consult Clin Psychol 1995;63: 70-8. [PubMed]

42. McKay JR, Alterman AI, McLellan AT, Boardman CR, Mulvaney FD, O'Brian CP. Random versus nonrandom assignment in the evaluation of treatment for cocaine abusers. J Consult Clin Psychol 1998;66: 697-701. [PubMed]

43. Moertel CG, Childs DS, O'Fallon JR, Holbrook MA, Schutt AJ, Reitemeier RJ. Combined 5-fluorouracil and radiation therapy as a surgical adjuvant for poor prognosis gastric carcinoma. J Clin Oncol 1984;2: 1249-54. [PubMed]

44. Mosekilde L, Beck-Nielsen H, Sørensen OH, Nielsen SP, Charles P, Vestergaard P, et al. Hormonal replacement therapy reduces forearm fracture incidence in recent postmenopausal women—results of the Danish osteoporosis prevention study. Maturitas 2000;36: 181-93. [PubMed]

45. Nagel HTC, Vanderbussche FPHA, Keirse MJNC, Oepkes D, Oosterwijk JC, Beverstock G, et al. Amniocentesis before 14 completed weeks as an alternative to transabdominal chorionic villus sampling: a controlled trial with infant follow-up. Prenat Diagn 1998;18: 465-75. [PubMed]

46. Nicolaides K, Brizot MdL, Patel F, Snijders R. Comparison of chorionic villus sampling and amniocentesis for fetal karyotyping at 10-13 weeks' gestation. Lancet 1994;344: 435-9. [PubMed]

47. Paradise JL, Bluestone CD, Bachman RZ, Colborn DK, Bernard BS, Taylor FH, et al. Efficacy of tonsillectomy for recurrent throat infection in severely affected children. Results of parallel randomized clinical trials. N Engl J Med 1984;310: 674-83. [PubMed]

48. Paradise JL, Bluestone CD, Rogers KD, Taylor FH, Colborn DK, Bachman RZ, et al. Efficacy of adenoidectomy for recurrent otitis media in children previously treated with tympanostomy-tube replacement. Results of parallel randomized and nonrandomized trials. JAMA 1990;263: 2066-73. [PubMed]

49. Playforth MJ, Smith GMR, Evans M, Pollock AV. Antimicrobial bowel preparation. Oral, parenteral, or both? Dis Colon Rectum 1988;31: 90-3. [PubMed]

50. Rigg JRA, Jamrozik K, Myles PS, Silbert B, Peyton P, Parsons RW, et al. Design of the multicenter Australian study of epidural anesthesia and analgesia in major surgery: the MASTER trial. Control Clin Trial 2000;21: 244-56. [PubMed]

51. Rosen MA, Roizen MF, Eger II EI, Glass RH, Martin M, Dandekar PV, et al. The effect of nitrous oxide on in vitro fertilization success rate. Anesthesiol 1987;67: 42-4. [PubMed]

52. Rovers MM, Straatman H, Ingels K, van der Wilt GJ, van den Broek P, Zielhuis GA. Generalizability of trial results based on randomized versus nonrandomized allocation of OME infants to ventilation tubes or watchful waiting. J Clin Epidemiol 2001;54: 789-94. [PubMed]

53. Schmoor C, Oleschewski M, Schumacher M. Randomized and non-randomized patients in clinical trials: experiences with comprehensive cohort studies. Stat Med 1996;15: 263-71. [PubMed]

54. Strandberg TE, Salomaa VV, Vanhanen HT, Naukkarin VA, Sarna AJ, Miettinen TA. Mortality in participants and non-participants of a multifactorial prevention study of cardiovascular diseases: a 28 year follow up of the Helsinki businessmen study. Br Heart J 1995;74: 449-54. [PMC free article] [PubMed]

55. Sullivan MP, Fuller LM, Chen T, Fisher R, Fryer C, Gehan E, et al. Intergroup Hodgkin's disease in children study of stages I and II: a preliminary report. Cancer Treat Rep 1982;66: 937-47. [PubMed]

56. Urban P, Stauffer J-C, Bleed D, Khatchatrian N, Amann W, Bertel O, et al. A randomized evaluation of early revascularization to treat shock complicating acute myocardial infarction. Eur Heart J 1999;20: 1030-8. [PubMed]

57. Villamaria FJ, Baisden CE, Hillis A, Rajab MH, Rinaldi PA. Forced-air warming is no more effective than conventional methods for raising postoperative core temperature after cardiac surgery. J Cardiothorac Vasc Anesth 1997;11: 708-11. [PubMed]

58. Wetzner SM, Vincent ME, Robbins AH. Ceruletide-assisted cholecystography: a clinical assessment. Radiology 1979;131: 23-6. [PubMed]

59. Wikdahl AM, Granbom L, Stegmayr BG. CAPD bag changing with integrated disconnect system gives lower incidence of peritonitis than with UV-box system. Adv Perit Dial 1992;8: 276-80. [PubMed]

60. Williford WO, Krol WF, Buzby GP. Comparison of eligible randomized patients with two groups of ineligible patients: can the results of the VA total parenteral nutrition clinical trial be generalized? J Clin Epidemiol 1993;46: 1025-34. [PubMed]

61. Yamamoto H, Hughes RW, Schroeder KW, Viggiano TR, DiMagno EP. Treatment of benign esophageal stricture by Eder-Puestow or balloon dilators: a comparison between randomized and prospective nonrandomized trials. Mayo Clinic Proc 1992;67: 228-36. [PubMed]

62. Yersin B, Besson J, Duc-Mingot S, Burnand B. Screening and referral of alcoholic patients in a general hospital. A clinical trial. Eur Addict Res 1996;2: 94-101.

63. Young JDH, MacEwen CJ, Ogston SA. Congenital nasolacrimal duct obstruction in the second year of life: a multicentre trial of management. Eye 1996;10: 485-91. [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Publishing Group