Meta-analysis is a method of statistically combining results of similar studies [1
]. For binary outcome variables both difference and ratio methods are commonly used. For each study, the risk difference is the difference in proportions of patients experiencing the outcome of interest between the experimental and control groups, the risk ratio is the ratio of these proportions, and the odds ratio is the ratio of the odds. Meta-analytic techniques are used to combine each study's effect measure to generate a pooled effect measure. Standard meta-analytic procedures for each of these effect measures also estimate heterogeneity, which is the variability in treatment effects of individual trials beyond that expected by chance. Each effect measure (risk difference, risk ratio, odds ratio) has advantages and disadvantages in terms of consistency, mathematical properties, and ease of interpretation, implying that none is universally optimal [2
In contrast, for continuous outcome variables, only difference methods are commonly used for group comparison studies [3
]. If the outcome of interest is measured in identical units across trials, then the effect measure for each trial is the difference in means, and the pooled effect measure is the mean difference (MD), which more accurately should be described as the weighted mean of mean differences. If the outcome of interest is measured in different units, then each trial's effect measure is the difference in mean values divided by the pooled standard deviation of the two groups, and the pooled effect measure is the standardized mean difference (SMD), which more accurately should be described as the weighted mean of standardized mean differences. Normalizing the differences using the standard deviation allows pooling of such results, in addition to allowing comparison of effect sizes across unrelated interventions. By convention [4
], SMD's of 0.2, 0.5, and 0.8 are considered "small", "medium", and "large" effect sizes, respectively. When trials in meta-analyses are weighted by the inverse of the variance of the effect measure (the weighting scheme generally used for MD and SMD), the pooled SMD has the unfavorable statistical property of negative bias (i.e. towards the null value) [5
]. Alternative methods of estimating the variance of individual trial SMDs used in the inverse variance method have been proposed to minimize this bias [5
In principle, meta-analysts could also use ratio methods to analyze continuous outcomes, by calculating a ratio of mean values instead of a difference. Since the ratio is unitless, this calculation can be carried out regardless of the specific units used in individual trials. Moreover, as with SMD, a ratio can be used to combine related but different outcomes (e.g. quality of life scales). We have recently used this Ratio of Means (RoM) method in meta-analyses [7
] in which we estimated the variance of this ratio using the delta method [10
]. For this method, each individual study RoM is converted to its natural logarithm before being pooled, and the pooled result is then back transformed, similar to odds and risk ratio calculations used for binary outcomes. Table presents the pooled results for the continuous variables from the meta-analysis of low-dose dopamine for renal dysfunction [7
], analyzed using mean difference methods and the RoM method, in addition to heterogeneity expressed using the I2
is the percentage of total variation in results across studies due to heterogeneity rather than chance [11
].) Table shows similar results among the three methods. The point estimates are similar in direction (i.e. a positive mean difference or standardized mean difference corresponds to a RoM greater than one, while a negative mean difference corresponds to a RoM less than one). The confidence intervals result in similar p-values for statistically significant increases or decreases for each of these parameters. Finally, heterogeneity is similar.
Renal Physiological Parameters from Low-Dose Dopamine Meta-Analysis 1 Day After Starting Therapy .
Given the similarity of these results, the objective of this current study was to test the hypothesis that MD, SMD, and RoM methods exhibit comparable performance characteristics in terms of bias, coverage and statistical power, using simulated data sets with a range of parameters commonly encountered in meta-analyses.