PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jrsocmedLink to Publisher's site
 
J R Soc Med. 2007 April; 100(4): 180–181.
PMCID: PMC1847742

Bufale spotting, part two: assessing systematic reviews

QUICK INSTRUMENT (FIGURE 1)

Just because systematic reviews are popular, it does not mean that you can take their content as gospel. Reviews follow the same IMRAD structure as research papers, so the same rules apply. However, the methods section should have a clear description of the rules for identifying, including, assessing and synthesizing evidence from the studies in the review. If these are unclear or do not match the objectives of the review, bin the review. Often, long and complicated search strategies are referred to in full format on the website of the journal. That's fine, but just make sure they are there. Bin anything with humorous inclusion criteria—for instance restricting inclusion to studies with a threshold denominator. What a nonsense!

FULL INSTRUMENT

QUOROM is an instrument developed (and widely accepted) for the structuring and presentation of systematic reviews (http://www.consort-statement.org/Evidence/evidence.html#quorom). Using it takes time, especially if you are unfamiliar with systematic reviews. Alternatively, you can read any review from the Cochrane Library. The format is standardized but turgid.turgid.

RATIONALE

Beware: popular topics or methods always attract quacks. Systematic reviews are very popular, so some of them are of dubious quality. To give you an idea of the attraction reviews exert, bear in mind that my work has been cleverly plagiarized twice in three years by people who are trying to make a name for themselves in the field. Industry has also started using reviews for marketing purposes. Take a good hard look at the sponsors and the conflicts of interest of the authors.

Meta-analysis has evolved into a self-standing discipline. If there is no statistician in the author line-up or in the acknowledgments, bin the review. Reviews may be very complicated studies, sometimes exerting great influence on decision-makers. Bin anything which does not provide a clear and reproducible audit trail of methods.

Notes

This is the third in a series of articles on making evidence-based medicine work for you. The series is based on the book ‘Attenti Alle Bufale’ by Tom Jefferson (www.attentiallebufale.it).

References

1. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM Group. Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement. Br J Surg 2002;87: 1447-58 [PubMed]

Articles from Journal of the Royal Society of Medicine are provided here courtesy of Royal Society of Medicine Press