Summary: We introduce REJECTOR, software for parameter estimation and comparison of alternate models of population history from genetic data via a rejection algorithm. Through coalescent simulation, REJECTOR generates numerous gene genealogies, and hence simulated data, under a model of population history specified by the user. Summary statistics derived from such simulated data are compared with observed statistics, leading to acceptance or rejection of a given set of parameter values. We performed tests of the software using known parameter values in order to assess the inferential power provided by each summary statistic. The tests demonstrate the precision and accuracy of estimation made possible using this approach.

The rejection algorithm method can be an efficient way to infer distributions of genetic and demographic parameters using polymorphism data (Ramakrishnan et al., 2004). In this approach values for a parameter of interest are simulated from a prior distribution and accepted with a probability proportional to their likelihood. Since likelihoods are difficult to compute directly, approximate simulation methods have been developed wherein summary statistics are used in place of the full dataset, and the candidate parameter value is accepted based on the values of the statistics (Tavaré et al., 1997). Bayesian statistics incorporate the use of prior information in the form of distributions of parameters of interest. The rejection algorithm method has been extended by simulation of parameter values from prior distributions and by coalescent simulation of gene genealogies given these parameter values (Pritchard et al., 1999). Summary statistics calculated from these simulated genealogies can be compared with summary statistics calculated from observed data, and if the simulated summary statistics fall within a specified tolerance of the observed summary statistics, the parameter values are accepted to form a sample from the posterior distribution. The combination of summary statistics in place of the full dataset and Bayesian prior distributions is termed approximate Bayesian computation. Approximate Bayesian methods using summary statistics can deal with complex models and provide estimates of parameter values of interest under any model (Excoffier and Heckel, 2006). REJECTOR implements an approximate Bayesian method termed rejection-based approximate Bayesian inference (Beaumont et al., 2002), for inferring population history.

REJECTOR is a package composed of four executables, one of which is the SIMCOAL2 software by Laval and Excoffier. We have written the other three executables—REJSTATS, PRIOR and REJECTOR—in C++. We have compiled, tested and run the software under Mac OS X, Linux and Windows.

An example of the recovery test method described above demonstrates the precision and accuracy of REJECTOR using well-chosen summary statistics and data types. We simulated 100 unlinked microsatellites under the target model described in Section 2 (two populations of 1000 chromosomes each, no growth, time of divergence of 2000 generations and a mutation rate of 1.5×10⁻³ per microsatellite per generation) and recorded the values for the summary statistics δμ², the average squared distance in microsatellite lengths (Goldstein et al., 1995) and the average variance in microsatellite repeat lengths, or microsatellite variance. We then ran REJECTOR for 100 000 iterations under a model introducing Gaussian prior distributions for time of divergence and size of Population 1 whose means were not equal to the true parameter values. We accepted iterations into the posterior distribution if both δμ² and microsatellite variance matched the target values within a tolerance of 0.05.