PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of ijorthoHomeCurrent issueInstructionsSubmit article
 
Indian J Orthop. 2011 Jan-Mar; 45(1): 4–5.
PMCID: PMC3004078

The 3-min appraisal of a meta-analysis

Meta-analysis can be defined as a quantitative method that mathematically combines the results from multiple studies covering the same overall topic, or the statistical pooling of the results of two or more similar studies.1,2 The results from meta-analyses can vary depending on the quality of the studies included and the methodological rigor used to select studies.2 We provide quick and simple criteria for critical appraisal, and a practical example illustrates the approach.

KEY CRITERIA FOR CRITICAL APPRAISAL

Overall, three broad questions should be asked [Table 1].

Table 1
Questions that should be considered when critically appraising a meta-analysis

Are the results of the study valid?

One of the most important steps in critically appraising a meta-analysis is determining the methodological quality of the study design and the level of bias incorporated in the analysis. One key factor affecting the quality of meta-analysis is the quality of the studies that are included in the meta-analysis itself. One should remember that quality in equals quality out. Moreover, a meta-analysis should have a focused research question and a comprehensive literature search. 1 More importantly, the literature search should be systematic and reproducible.1,2 Readers should also be aware of publication bias, which refers to the increased probability of studies with positive results to be published.1,3

What are the results?

The next step in critical appraisal is determining the main results and how they are expressed. Most times the results will be presented through a forest plot (or pictorial of the individual study findings). Overall combined results are usually presented as a relative risk or odds ratio when the outcomes are categorical (i.e. alive or dead, infection or no infection). When the results are continuous, as in a functional outcome score, the pooled results across many studies can be presented with standardized mean differences or effect sizes (difference in means divided by the standard deviation).

What is the applicability of the results?

When critically appraising a meta-analysis the last step should be determining the clinical applicability of the results. Sometimes the results of a meta-analysis may show statistical significance, but may have no importance to practice. A simple questionnaire provided in Table 1 can be used as a quick reference guide for the critical appraisal of a meta-analysis.

A PRACTICAL EXAMPLE: FIXED- VERSUS MOBILE-BEARING TOTAL KNEE REPLACEMENT

Smith and colleagues conducted a review of 33 studies assessing the outcomes of 3532 total knee replacements (TKRs).4 Analysis suggested that there was no significant difference in clinical or radiological outcomes and complication rates between fixed- and mobile-bearing TKRs.

The research question: To analyze the difference in clinical and radiological outcomes between fixed- and mobile-bearing TKRs.

Literature search, study selection, and quality analysis: The search strategy involved the use of Medline, CINAHL, AMED, and EMBASE to find included studies. Various search terms specific to the research question such as “knee AND fixed bearing OR mobile bearing” were formulated. Bias was accounted for by searching for the unpublished literature using the System for Information on Grey Literature (SIGLE). The selection criteria for all included and excluded studies are outlined and described. Furthermore, the quality of each study was methodologically and independently assessed by two reviewers using the Physiotherapy Evidence Database (PEDro) appraisal tool.

Outcome measures and combination of studies: The primary and secondary outcome measures are clearly indicated. Data were pooled using either pooled mean difference, standardised mean difference for continuous variables, or relative risk for dichotomous variables. These concepts were briefly described above.

Main results and tests of significance: Overall, it was found that there was no statistically significant difference with respect to functional, clinical, radiological outcomes or complication rates between fixed- and mobile-bearing TKR designs.

Clinical relevance of results: Since the results find no statistical significance it can be concluded that either fixed- or mobile-bearing TKR designs can be used. However, this should be done with caution since the study indicates limitations in the current evidence.

CONCLUSION

Critical appraisal is an invaluable tool used in evidence-based medicine, and it is important for a clinician to determine the best quality evidence for practice. Along with the questionnaire provided in Table 1, a checklist is provided in Table 2 of the various items that should be included in a superior meta-analysis. Gaining a proper understanding the concepts described above and utilizing the tables provided will help in the evidence-based approach!

Table 2
The Do’s and Don’ts of a meta-analysis

Footnotes

Source of Support: Nil

Conflict of Interest: None

REFERENCES

1. Zlowodzki M, Poolman RW, Kerkoffs GM, Tornetta P, Bhandari M. How to interpret a meta-analysis and judge its value as guide for clinical practice. Acta Orthopaedica. 2008;78:598–609. [PubMed]
2. Bhandari M, Joensson A. Clinical Research for Surgeons. Stuttgart: Thieme; 2009. The Meta-analysis; pp. 145–150.
3. Garg AX, Hackam D, Marcello T. Systematic review and meta-analysis: when one study is just not enough. Clin J Am Soc Nephrol. 2008;3:253–60. [PubMed]
4. Smith TO, Ejtehadi F, Nichols R, Davies L, Donell ST, Hing CB. Clinical and radiological outcomes of fixed- versus mobile-bearing total knee replacement: a meta-analysis. Knee Surg Sports Traumatol Arthrosc. 2010;18:325–40. [PubMed]

Articles from Indian Journal of Orthopaedics are provided here courtesy of Wolters Kluwer -- Medknow Publications