Search tips
Search criteria 


Logo of bmjThe BMJ
BMJ. 1994 November 12; 309(6964): 1286–1291.
PMCID: PMC2541778

Identifying relevant studies for systematic reviews.


OBJECTIVE--To examine the sensitivity and precision of Medline searching for randomised clinical trials. DESIGN--Comparison of results of Medline searches to a "gold standard" of known randomised clinical trials in ophthalmology published in 1988; systematic review (meta-analysis) of results of similar, but separate, studies from many fields of medicine. POPULATIONS--Randomised clinical trials published in 1988 in journals indexed in Medline, and those not indexed in Medline and identified by hand search, comprised the gold standard. Gold standards for the other studies combined in the meta-analysis were based on: randomised clinical trials published in any journal, whether indexed in Medline or not; those published in any journal indexed in Medline; or those published in a selected group of journals indexed in Medline. MAIN OUTCOME MEASURE--Sensitivity (proportion of the total number of known randomised clinical trials identified by the search) and precision (proportion of publications retrieved by Medline that were actually randomised clinical trials) were calculated for each study and combined to obtain weighted means. Searches producing the "best" sensitivity were used for sensitivity and precision estimates when multiple searches were performed. RESULTS--The sensitivity of searching for ophthalmology randomised clinical trials published in 1988 was 82%, when the gold standard was for any journal, 87% for any journal indexed in Medline, and 88% for selected journals indexed in Medline. Weighted means for sensitivity across all studies were 51%, 77%, and 63%, respectively. The weighted mean for precision was 8% (median 32.5%). Most searchers seemed not to use freetext subject terms and truncation of those terms. CONCLUSION--Although the indexing terms available for searching Medline for randomised clinical trials have improved, sensitivity still remains unsatisfactory. A mechanism is needed to "'register" known trials, preferably by retrospective tagging of Medline entries, and incorporating trials published before 1966 and in journals not indexed by Medline into the system.

Full text

Full text is available as a scanned copy of the original print version. Get a printable copy (PDF file) of the complete article (1.3M), or click on a page image below to browse page by page. Links to PubMed are also available for Selected References.

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Dickersin K, Berlin JA. Meta-analysis: state-of-the-science. Epidemiol Rev. 1992;14:154–176. [PubMed]
  • Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith H., Jr Publication bias and clinical trials. Control Clin Trials. 1987 Dec;8(4):343–353. [PubMed]
  • Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991 Apr 13;337(8746):867–872. [PubMed]
  • Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. 1992 Jan 15;267(3):374–378. [PubMed]
  • Dickersin K, Hewitt P, Mutch L, Chalmers I, Chalmers TC. Perusing the literature: comparison of MEDLINE searching with a perinatal trials database. Control Clin Trials. 1985 Dec;6(4):306–317. [PubMed]
  • Jadad AR, McQuay HJ. A high-yield strategy to identify randomized controlled trials for systematic reviews. Online J Curr Clin Trials. 1993 Feb 27;DOC:[3973 words–paragraphs]. [PubMed]
  • Gøtzsche PC, Lange B. Comparison of search strategies for recalling double-blind trials from MEDLINE. Dan Med Bull. 1991 Dec;38(6):476–478. [PubMed]
  • Poynard T, Conn HO. The retrieval of randomized clinical trials in liver disease from the medical literature. A comparison of MEDLARS and manual methods. Control Clin Trials. 1985 Dec;6(4):271–279. [PubMed]
  • Bernstein F. The retrieval of randomized clinical trials in liver diseases from the medical literature: manual versus MEDLARS searches. Control Clin Trials. 1988 Mar;9(1):23–31. [PubMed]
  • Silagy C. Developing a register of randomised controlled trials in primary care. BMJ. 1993 Apr 3;306(6882):897–900. [PMC free article] [PubMed]
  • Ohlsson A. Treatments of preterm premature rupture of the membranes: a meta-analysis. Am J Obstet Gynecol. 1989 Apr;160(4):890–906. [PubMed]
  • Daya S. Efficacy of progesterone support for pregnancy in women with recurrent miscarriage. A meta-analysis of controlled trials. Br J Obstet Gynaecol. 1989 Mar;96(3):275–280. [PubMed]
  • Kirpalani H, Schmidt B, McKibbon KA, Haynes RB, Sinclair JC. Searching MEDLINE for randomized clinical trials involving care of the newborn. Pediatrics. 1989 Apr;83(4):543–546. [PubMed]
  • Kleijnen J, Knipschild P. The comprehensiveness of Medline and Embase computer searches. Searches for controlled trials of homoeopathy, ascorbic acid for common cold and ginkgo biloba for cerebral insufficiency and intermittent claudication. Pharm Weekbl Sci. 1992 Oct 16;14(5):316–320. [PubMed]
  • Solomon MJ, Laxamana A, Devore L, McLeod RS. Randomized controlled trials in surgery. Surgery. 1994 Jun;115(6):707–712. [PubMed]
  • Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994 Jul 13;272(2):158–162. [PubMed]
  • Dickersin K, Min YI. Publication bias: the problem that won't go away. Ann N Y Acad Sci. 1993 Dec 31;703:135–148. [PubMed]
  • Chalmers TC, Berrier J, Sacks HS, Levin H, Reitman D, Nagalingam R. Meta-analysis of clinical trials as a scientific discipline. II: Replicate variability and comparison of studies that agree and disagree. Stat Med. 1987 Oct-Nov;6(7):733–744. [PubMed]
  • Stewart LA, Parmar MK. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993 Feb 13;341(8842):418–422. [PubMed]
  • Chalmers I, Dickersin K, Chalmers TC. Getting to grips with Archie Cochrane's agenda. BMJ. 1992 Oct 3;305(6857):786–788. [PMC free article] [PubMed]
  • Easterbrook PJ. Directory of registries of clinical trials. Stat Med. 1992 Feb 15;11(3):363–423. [PubMed]
  • Freedman B. Suspended judgment. AIDS and the ethics of clinical trials: learning the right lessons. Control Clin Trials. 1992 Feb;13(1):1–5. [PubMed]

Articles from The BMJ are provided here courtesy of BMJ Publishing Group