In discussing their results, the study authors state they have completed ‘the first randomized control trial to examine massage therapy for enhancing development and decreasing maladaptive behaviors in young Dominican children infected with HIV,’ but this is not true, for the study is not a randomized control trial if it does not make between-groups comparisons of the dependent variables. The entire purpose of employing randomization to create treatment and control groups, as opposed to using a simpler within-group design, is so that one can make between-groups comparisons that control for the validity threats of spontaneous remission, placebo effects, and statistical regression.
Having performed analyses inconsistent with the study design, and that introduce rather than control these threats, the authors reach conclusions that are difficult to justify. To conclude that MT improved the behavior of the older sample of children, that it provided a ‘marginally significant’ increase in IQ, or that it ‘appears to be a viable therapy for promoting greater daily functioning and communication’ based on within-group analyses is misleading, because any or all of those pre–post effects could be observed as a result of the aforementioned threats even if MT was wholly ineffective. Further, those threats cannot be controlled by the authors’ occasional use of what could be called ‘side-by-side’ within-group comparisons, in which they test each group separately for its own pre–post effect, and then give the impression that MT worked if it shows a statistically significant pre-post effect and the control group does not. This approach is not at all equivalent to making the required between-groups comparison, and the result is misleading, as it is easily possible in that situation (especially with small samples) to have only one of the two within-group comparisons be statistically significant when there is actually no difference between the groups.
Possibly the authors’ decision to exclude between-groups analyses of dependent variables was motivated by concern that the tests would be underpowered and therefore not attain statistical significance. However, even if this was the case, the small size and exploratory nature of the study cannot be used to justify the decision to use within-group analyses. If problems pertaining to the statistical power of between-groups analyses were anticipated, there are defensible alternatives. These include choosing a more liberal value for alpha (e.g. using P
< 0.10 or P
< 0.20, rather than P
< 0.05), placing greater emphasis on effect sizes and their confidence intervals than on probability values, and examination of clinical significance (6
). Simply stated, between-groups study designs such as randomized control trials logically demand between-groups analyses. Until this is reflected in MT research, our knowledge of what it does and does not do will continue to be seriously hampered.