The most studied type of two-stage design is based on an hypothesis testing paradigm: a SNP is said to be disease-associated if its test statistic exceeds a fixed critical value c₁ at stage 1 and if its test statistic at stage 2 (usually based on the combined stage 1 and stage 2 data) exceeds a fixed critical value c₂ (Müller et al., 2007, Satagopan & Elston, 2003, Skol, 2007, Skol et al., 2006, Wang et al., 2006). This hypothesis testing approach not only identifies promising SNPs in the first stage, but also provides an overall p-value for assessing association with disease.

However, many studies are not conducted this way. Instead, a SNP is selected for further study in stage 1 if its p-value ranks among the T₁ smallest p-values. Thus, selection of promising SNPs is not based on a fixed critical value but on ranking the SNPs against each other. This strategy was used in the Cancer Genetic Markers of Susceptibility Study (CGEMS) for prostate cancer (http://cgems.cancer.gov/about/replication_strategy.asp) and in studies of various other diseases(Broderick et al., 2007, Buch et al., 2007, van Es et al., 2007). In the CGEMS prostate cancer study, about n₁ = 1200 cases and controls were studied in stage 1 (i.e. π_sample = 0.14), and roughly T₁ = 28,000 SNPs with the smallest p-values were selected for study in 7200 additional cases and controls in subsequent stages.

One approach following stage 1 is to continue ranking and selecting the SNPs based on data in stage 2. In particular, one selects a SNP as highly promising at the end of stage 2 if its p-value, which may be based on stage 2 data alone or on some combination of stage 1 and stage 2 data, ranks among the lowest T₂ p-values from the T₁ SNPs studied in stage 2. In this paper we describe the properties of this two-stage selection procedure, and in particular, we calculate the probability that a disease-associated SNP will be selected at the end of stage 2 (the “detection probability”) and the proportion of selected SNPs that are expected to be disease-associated (the “proportion positive”). If a two-stage procedure is designed to have a high detection probability and proportion positive, then one can expect that most disease-associated SNPs will be included in the T₂ selected SNPs and that independent epidemiologic studies will confirm the association with disease in a good proportion of the T₂ selected SNPs.

A second approach is to test for associations with disease among the SNPs selected in stage 1. In a study of gallstone disease(Buch et al., 2007), the authors used the independent stage 2 data alone to produce a p-value. In a study of amyotrophic lateral sclerosis(van Es et al., 2007), the authors ranked SNPs in stage 1 and culled SNPs further by requiring a nominal p-value <0.1 in stage 2 before assigning a p-value based on independent stage 3 data. To study genetic associations with colorectal cancer, investigators combined data from stages 1 and 2 to produce an overall p-value(Broderick et al., 2007). The statistical properties of these hybrid approaches have not been evaluated, although p-values that depend only on an independent final stage should have nominal significance levels.

Assuming that the T₀ tagging SNPs are in linkage equilibrium as in(Skol et al., 2006), we proved that SNP genotypes are independent not only in controls, but also in cases for a rare disease(Gail et al., 2008). We used these ideas and asymptotic theory for the Wald and score tests to develop efficient procedures for simulating the chi-square test statistics for the M₀ disease-associated SNPs (“disease SNPs”) and the T₀−M₀ non-disease SNPs in stage 1. For independent stage 2 data, we now extend these methods to simulate independent chi-square tests for T₁ SNPs, of which a random number, M₁, are disease SNPs that were selected at stage 1, and the remaining T₁−M₁ are non-disease SNPs. We study odds ratios per allele of 1.1, 1.2, 1.3, and 1.5, but we use 1.2 in the following description. We consider two models(Gail et al., 2008) for disease SNPs. In the fixed effects model, the log odds ratio per disease allele is fixed at β = log (1.2) for each disease SNP. Thus the relative odds is 1.44 for a homozygote and 1.2 for a heterozygote. In the random effects model, β is drawn independently for each disease SNP from a normal distribution with mean zero and standard deviation τ = (π/2)^1/2 log(1.2) ≈1.253log(1.2). This value of τ yields an expected absolute value of β of log(1.2). Under both models, β =0 for non-disease SNPs. One chi-square test is the squared Wald statistic, ²/ar (), for testing β =0 in a model for log odds of disease that equals an intercept plus β times the number of minor alleles(Gail et al., 2008). We also studied the corresponding squared score test(Armitage, 1955, Sasieni, 1997). Each of these chi-square tests has the same value whether the major or minor allele confers risk.

At stage 1, the T₁ SNPs with the largest chi-square tests (or smallest p-values) are selected. We study two analytical approaches, “replication analysis” and “joint analysis,” similar to (Skol et al., 2006). For a given SNP whose test statistic was in the critical region in stage 1, Skol et al. called an hypothesis test based only on the stage 2 data a “replication analysis.” They used the term “joint analysis” for a final hypothesis test based a linear combination of the test statistics for stage 1 and stage 2 data. Analogously, we use the term “replication analysis” if final selection of the T₂ SNPs from among the T₁ SNPs selected in stage 1 depends only on their rankings derived from the independent stage 2 data. In “joint analysis”, for each of the T₁ SNPs selected in stage 1, we compute λ C₁ + (1−λ)C₂ for λ = 0, 0.05, 0.10,…,1.0, where C₁ and C₂ are the chi-square statistics observed in stages 1 and 2 respectively. For each λ we estimate DP, and we present the maximal P over the 21 values ofλ, together with the corresponding λ_opt. If several values maximize DP, we define λ_opt as their average.

For the simulations, we assume that 8000 cases and 8000 controls are available to be apportioned between stages 1 and 2 with equal numbers of cases and controls in each stage. We present data for M₀ = 1 and 10 and for π_sample = 0.125, 0.25, 0.50, and 1.0, both for fixed effects and random effects models. For each odds ratio, we conducted 12 independent simulation studies, each with 10,000 replications, to estimate DP for each combination of M₀, π_sample =0.125, 0.25 and 0.50 and fixed and random effects models. We allow for realistic random variation in minor allele frequency (MAF). In each simulation and for each of the T₀ = 500,000 SNPs, we randomly select a MAF from the distribution in controls in CGEMS (https://caintegrator.nci.nih.gov/cgems/), truncated to the interval [0.05, 0.5]; the mean MAF is 0.2673, which is corrected from 0.2763 in (Gail et al., 2008). For each SNP, a chi-square value is generated that depends on the MAF, β, and numbers of cases and controls(Gail et al., 2008). Letting I (m, ISIM) =1 if disease SNP m is among the T₁ SNPs selected after stage 1 and among the T₂ SNPs selected after stage 2 and 0 otherwise, we estimate(Gail et al., 2008) DP as

We note from the exchangeability of the disease SNPs that DP can be interpreted either as the probability that a particular disease SNP will be selected at stage 2 or as the proportion of disease SNPs selected at stage 2(Gail et al., 2008). The proportion positive, PP, can be estimated(Gail et al., 2008) as

The proportion positive, PP, can also be estimated for β = log(1.2) from Table 1 and equation (2) assuming M₀ is known. For M ₀ =1, T₁ =25,000 and T₂ =10, P =0.065 for π_sample =0.125 and 0.093 for the one-stage design. If M₀ =10, the corresponding P values are 0.654 and 0.898. For M ₀ =1, T₁ =25,000 and T₂ =1, P =0.630 for π_sample =0.125 and 0.882 for the one-stage design.

For the random effects model with τ = (π/2)^1/2 log(1.2) ≈ 0.2284 and with M₀ =1(Table 2), P ranged from 0.550 to 0.646 for the one-stage design. With π_sample = 0.5, P was modestly reduced for the replication analysis, but only slightly reduced for the joint analysis. λ_opt ranged from 0.25 to 0.45. For π_sample = 0.25 or 0.125, P was considerably reduced for the replication analysis, and the joint analysis yielded little if any increase in P. For example, for π_sample =0.125, T₁ =25,000 and T₂ =10, P was 0.599 for the one-stage analysis, 0.465 for the replication analysis, and 0.465 for the joint analysis. λ_opt was smaller for π_sample =0.125 or 0.25 than for 0.50, and λ_opt was usually smaller for T₂ =100 than for T₂ = 1 or 10. For M₀ =10, results were very close to those for M₀ =1 except when T₂ =1. With T₂ =1, the M₀ =10 disease SNPs competed against each other, reducing P to near 0.100 for all designs and analyses. As for the fixed effects model, application of equation (2) shows that P increases with increasing M₀.

To examine how much the DP of the two-stage design is reduced compared to the one-stage design for fixed effects models with odds ratios per allele of 1.1, 1.2, 1.3 and 1.5, we plotted P for the two-stage design and joint analysis against the P for the corresponding one-stage design with 8000 cases and 8000 controls for π_sample =0.125 and M₀ =1 (Figure 1). The bold loci correspond to T₁=25,000 for T₂=1, 10 or 100, and the unbolded loci correspond to T₁=1,000 for T₂=1, 10 or 100. The points on each locus correspond from left to right to odds ratios 1.1, 1.2, 1.3 and 1.5. The vertical distance from the equiangular line is the decrease in estimated DP from using the two-stage design. For example, for odds ratio 1.2, and for T₁=25,000 and T₂=100, the vertical distance is 0.966−0.664=0.302; for T₁=1,000 and T₂=100, the vertical distance is 0.966−0.269=0.697, as can also be seen from Table I. The absolute decrease in P for two-stage designs is substantial for odds ratios 1.1, 1.2 and 1.3, but diminishes for odds ratio 1.5, as both designs attain P approaching 1.0. Except for T₂=1, very similar graphs are obtained for M₀ =10 (not shown).

For the random effects model with M₀ =1 and π_sample =0.125, decreases in P for the two-stage design are appreciable for standard deviations of log odds ratios of (π/2)^1/2 times log(1.1), log( 1.2), log(1.3), and log(1.5) (Figure 2). Note that compared to the corresponding fixed effects models, P is smaller for both the one-stage and two-stage designs. Except for T₂=1, very similar graphs are obtained for M₀ =10 (not shown).

For the fixed effects model with M₀ =1 and π_sample =0.25, the decreases in P for the two-stage design are smaller than for π_sample =0.125 (compare Figures 3 and ​and1).1). Nonetheless, the decreases in P remain appreciable for odds ratios 1.1 and 1.2 (Figure 3). Except for T₂=1, very similar graphs are obtained for M₀ =10 (not shown).

For the random effects model with M₀ =1 and π_sample =0.25, the decreases in P for the two-stage model are smaller than for π_sample =0.125 (compare Figures 4 and ​and2).2). The decreases in P are small for T₁ =25,000, but, for T₁ =1,000, remain appreciable for standard deviations of log odds ratios of (π/2)^1/2 times log(1.1), log( 1.2), log(1.3), and log(1.5) (Figure 4). Except for T₂=1, very similar graphs are obtained for M₀ =10 (not shown).

Our data suggest that additional stage 1 genotyping in most previous studies with π_sample ≤0.25 will yield additional promising SNPs and that future multistage designs should not use π_sample ≤0.25, unless the numbers of available cases and controls are very large. Other considerations also favor using larger values of π_sample. As the cost per genotype of chips designed for stage 1 decreases relative to that for specialized platforms for subsequent stages, cost considerations argue for larger values of π_sample (Skol, 2007, Wang et al., 2006). The costs of obtaining cases and controls also favor a larger value of π_sample (Müller et al., 2007).

The two-stage ranking and selection procedure analyzed in this paper differs from two-stage procedures that apply the same fixed critical values to data from each SNP and are designed to select promising SNPs in stage 1 and provide a final p-value for testing an association following stage 2, as in (Skol et al., 2006). In particular, the two-stage ranking and selection procedure does not attempt to control the overall p-value, but only to obtain a very promising set of T₂ SNPs at the end of stage 2. Despite these different goals and methods, Figure 2 in (Skol et al., 2006) shows that power diminishes appreciably, and cannot be retrieved by joint analysis, if π_sample ≤ 0.20 and π_{mar ker} T₁/T₀ ≤ 0.1, in agreement with our findings for DP.

In some circumstances, power calculations can be used to approximate DP. For a one-stage design with M₀ = 1, equation (2.6) in (Gail et al., 2008) shows that DP can be approximated by the power that corresponds to an hypothesis test with size α = T₁/T₀. Although power calculations performed in this way and extended to the two-stage design may approximate the DP under certain conditions, the results in (Skol et al., 2006) were not based on such significance levels and critical values. We illustrate these differences using the program provided by (Skol et al., 2006) at http://csg.sph.umich.edu. For π_sample =0.125, π_{mar ker} T₁/T₀ = 25, 000/500, 000 = 0.05, genome-wide false-positive rate 0.05, which corresponds to α = 0.05/500, 000 = 10⁻⁷ for the joint analysis, a single fixed minor allele frequency of 0.2673 for all SNPs, and an assumed disease prevalence of 0.10, this program yields power estimates of 0.83 for the replication analysis and 0.83 for the joint analysis. The corresponding critical value for a normal deviate for stage 1 is 1.96, and for stage 2, the critical values are 4.611 for the replication analysis and 5.189 for the joint analysis. For the ranking procedure with M₀ =1, T₁ = 25, 000 and T₂ = 1, 10, or 100, the DP was 0.63, 0.65, and 0.66 respectively for the replication analysis and 0.64, 0.66 and 0.66 respectively for the joint analysis (Table 1) in the realistic setting in which minor allele frequencies are drawn from the distribution in CGEMS, with mean 0.2673. If instead it was assumed that all SNPs had the same minor allele frequency, 0.2673, as was assumed for the power calculations, the corresponding estimates of DP were 0.74, 0.75 and 0.75 for the joint analysis. These calculations illustrate that there are quantitative differences between assessments based on power and those based on detection probability, even though the broad conclusion that π_sample should not be too small is supported by both analyses.

It is worthwhile to recount some differences between power and detection probability. Power is the probability that the test statistic for a given SNP will fall into the pre-determined critical region for a one- or two-stage design that is chosen to control a genome-wide significance level, as for example in (Skol et al., 2006). Power thus depends on the chosen significance level; DP depends, instead, on T₀, T₁, and T₂. The power to reject the null hypothesis for a given SNP does not depend on the test statistics for any other SNP; DP depends on the test results for all SNPs. Power does not depend on the number of disease-associated SNPs, M ₀; DP can be sharply reduced by competition among disease-associated SNPs, especially if T₂ is less than M ₀. Most power calculations assume that all disease-associated SNPs have the same minor allele frequency; DP calculations routinely allow for allele-frequencies to be drawn from a realistic distribution of allele minor frequencies. DP estimates the probability that a given disease-associated SNP will have among the smallest T₂ p-values at the end of a two-stage study; power, as routinely calculated, does not have this interpretation. If disease-associated SNPs are exchangeable, as we assume in the fixed effect and random effect models (see METHODS), DP also has an interpretation as the proportion of disease-associated SNPs that have among the smallest T₂ p-values at the end of a two-stage study. Thus, the estimate of DP can be used to estimate how many more disease-associated SNPs with similar odds ratios to those already found are likely to be discovered by conducting another study with a larger stage1 sample.

Satagopan and colleagues (Satagopan et al., 2004, Satagopan et al., 2002) also studied ranking procedures to identify disease-susceptibility SNPs in two-stage designs, but used different rank-based criteria from DP and also assumed that the disease allele was known. The two-sided versions of a Wald test or a score test that we used have the same value whether one counts major or minor alleles, and hence are particularly appropriate for GWASs(Devlin & Roeder, 1999, Pfeiffer & Gail, 2003).

The ranking and selection methods used in this paper depend on the assumption that tagging SNPs are independent (Gail et al., 2008), an assumption that is also widely used in power calculations, e.g. (Skol et al., 2006). Zaykin and Zhivotovsky (Zaykin & Zhivotovsky, 2005) analyzed different ranking criteria for one-stage designs and found that selection probabilities were little affected by correlations of p-values within linkage disequilibrium blocks or among such blocks. In unreported simulations in which non-disease associated SNPs were paired and each member of the pair assigned the same chi-square value (perfect correlation within pairs), we found almost no effect on the estimates of DP and PP, compared to the situation in which SNP gentotypes are independent. Thus it is likely that our estimates of DP are robust to local correlations among tagging SNPs.