The study sample and its origin are described elsewhere.19 20
Twenty one meta-analyses described in 17 reports were eligible. Of these, 13 meta-analyses16 36 37 38 39 40 41 42 43 44 45 46
(153 trials with 41
605 patients) included both small and large trials and contributed to the current analyses. The median number of trials included per meta-analysis was 12 (range 3-24) and the median number of patients was 1849 (347-13
659). The pooled effect sizes ranged from −0.07 to −1.11 and the heterogeneity between trials from a τ2
of 0.00 to 0.47. Eight meta-analyses assessed drug interventions, and five assessed non-drug interventions. Four assessed interventions in complementary medicine, and nine assessed interventions in conventional medicine.
Table 1 describes the characteristics of the 153 component trials; 58 (38%) trials included at least 100 patients per arm and 95 (62%) trials were smaller. The number of allocated patients ranged from 201 to 2957 in large trials, and from 8 to 362 in small trials. Large trials were published more recently (P=0.001) and were more likely to report adequate concealment of allocation (P=0.01) and calculation of sample size (P<0.001).
Table 1 Comparison of characteristics between small and large trials in meta-analyses in osteoarthritis research. Figures are numbers (percentages)
The average difference in effect sizes between large and small trials across the 13 included meta-analyses was −0.21 (95% confidence interval −0.34 to −0.08, P=0.001), with more beneficial effects found in small trials (fig 1). At the level of individual meta-analyses, tests for interaction between treatment benefits and trial size were positive in four meta-analyses (31%).16 37 39 45
The variability across meta-analyses was small to moderate, with a τ2
estimate of 0.03 (P=0.005). Table 2 shows the average difference in effect sizes between large and small trials, both crude and after adjustment for the methodological quality of trials. Differences in effect sizes between small and large trials were robust after adjustment for blinding of patients (−0.21, −0.33 to −0.09, P=0.001), slightly attenuated after adjustment for concealment of allocation (−0.16, −0.27 to −0.06, P=0.002), but nearly halved after adjustment for intention to treat analysis (−0.12, −0.21 to −0.02, P=0.016). The variability across meta-analyses was similar between crude and adjusted analyses.
Fig 1 Difference in effect sizes between 95 small trials with fewer than 100 patients per arm and 58 large trials. Negative differences indicate that small trials show more beneficial treatment effects. P values are for interaction between sample size (more ...)
Estimates of small study effects in meta-analyses of osteoarthritis trials
Table 3 presents results from analyses stratified according to the magnitude of treatment effects, the heterogeneity between trials found in overall meta-analyses, and the type of experimental intervention. Differences in effect sizes between large and small trials were most pronounced in meta-analyses with large treatment benefits, meta-analyses with a high degree of heterogeneity between trials, and meta-analyses of complementary interventions (P for interaction all <0.001).
Table 3 Analyses stratified according to characteristics of meta-analyses of osteoarthritis trials
Figure 2 shows funnel plots of all 13 meta-analyses including prediction lines from meta-regression models with the SE as an explanatory variable and 5% contour areas to display areas of significance and non-significance. For six funnel plots, the scatter of effect estimates and the prediction line indicated asymmetry (A, D, G, H, L, M).16 37 39 42 44 45
For two other funnel plots, the prediction lines mainly suggested asymmetry (C, E),40 46
whereas the remaining five funnel plots seemed symmetrical and prediction lines nearly upright (B, F, I, J, K).36 38 41 43 44
The regression test was significant at P≤0.05 in four meta-analyses (D, G, H, M)16 37 39 42
and showed a statistical trend in another two (P≤0.10, A, L).44 45
In five funnel plots, the contours to distinguish between areas of significance and non-significance at P=0.05 suggested missing trials in areas of non-significance (A, C, D, H, L).16 42 44 45 46
The weighted average of asymmetry coefficients across all meta-analyses was −1.79 (−2.81 to −0.78). This indicates that, on average, the estimated treatment benefit increases by 1.79 SD units for each unit increase in the SE. It was much the same after adjustment for concealment of allocation (−1.86, −2.98 to −0.74), slightly more pronounced after adjustment for blinding (−2.22, −3.28 to −1.17), but slightly less pronounced after adjustment for intention to treat analysis (−1.41, −2.27 to −0.54). Confidence intervals of adjusted and the unadjusted estimates overlapped widely.
Fig 2 Funnel plots of 13 included meta-analyses including prediction lines from univariable meta-regression models with SE as explanatory variable (dashed red) and 5% contour areas to display areas of significance (blue) and non-significance (pale blue). (more ...)
Figure 3 presents a graphical summary of results of individual meta-analyses of all trials (blue circle), meta-analyses restricted to large trials (open circle), and predicted effect sizes for trials with a SE of 0.1 (green square). Results of all three analytical approaches were concordant in seven meta-analyses (fig 3, B, E, F, H, I, J, K).36 38 41 42 43 44
In the six remaining, both approaches, the restricted analysis, and the predicted effect were discordant to the overall analysis (A, C, D, G, L, M).16 37 39 44 45 46
In three of these, significance at the conventional level of 0.05 was lost when the analysis was restricted to large trials and when predicting the effect (D, G, M); in the other three, significance was lost when predicting the effect but not when the analysis was restricted (A, C, L).44 45 46
The median estimated treatment benefit decreased from −0.39 (range −1.11-−0.06) in meta-analyses of all trials to −0.23 (−0.59-−0.04) in meta-analyses restricted to large trials (P=0.005) and the median heterogeneity between trials decreased from a τ2
of 0.20 (0.00-0.69) to a τ2
of 0.04 (0.00-0.31, P=0.030). P values of pooled effect sizes increased from a median of <0.001 (<0.001-0.13) to 0.007 (<0.001-0.61, P=0.016) in restricted meta-analyses, whereas precisions of pooled effect sizes were much the same (median 13 (2-24) v
14 (7-21), P=0.70).
Fig 3 Results of individual random effects meta-analyses of all trials (blue circle), results of random effects meta-analyses restricted to large trials with at least 100 patients per arm (open circle), and effect sizes for trials with SE of 0.1 predicted (more ...)