PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmcmrmBioMed Centralsearchsubmit a manuscriptregisterthis articleBMC Medical Research Methodology
 
BMC Med Res Methodol. 2012; 12: 78.
Published online Jun 14, 2012. doi:  10.1186/1471-2288-12-78
PMCID: PMC3445820
t-tests, non-parametric tests, and large studies—a paradox of statistical practice?
Morten W Fagerlandcorresponding author1
1Unit of Biostatistics and Epidemiology, Oslo University Hospital, Oslo, N-0407, Norway
corresponding authorCorresponding author.
Morten W Fagerland: morten.fagerland/at/medisin.uio.no
Received January 11, 2012; Accepted June 14, 2012.
Abstract
Background
During the last 30 years, the median sample size of research studies published in high-impact medical journals has increased manyfold, while the use of non-parametric tests has increased at the expense of t-tests. This paper explores this paradoxical practice and illustrates its consequences.
Methods
A simulation study is used to compare the rejection rates of the Wilcoxon-Mann-Whitney (WMW) test and the two-sample t-test for increasing sample size. Samples are drawn from skewed distributions with equal means and medians but with a small difference in spread. A hypothetical case study is used for illustration and motivation.
Results
The WMW test produces, on average, smaller p-values than the t-test. This discrepancy increases with increasing sample size, skewness, and difference in spread. For heavily skewed data, the proportion of p<0.05 with the WMW test can be greater than 90% if the standard deviations differ by 10% and the number of observations is 1000 in each group. The high rejection rates of the WMW test should be interpreted as the power to detect that the probability that a random sample from one of the distributions is less than a random sample from the other distribution is greater than 50%.
Conclusions
Non-parametric tests are most useful for small studies. Using non-parametric tests in large studies may provide answers to the wrong question, thus confusing readers. For studies with a large sample size, t-tests and their corresponding confidence intervals can and should be used even for heavily skewed data.
Keywords: T-test, Non-parametric test, Wilcoxon-Mann-Whitney test, Welch test, Sample size, Statistical practice
Articles from BMC Medical Research Methodology are provided here courtesy of
BioMed Central