We now summarize the main findings of the simulation study. presents the FWER and expected number of false positives corresponding to each method with M
= 100,000 for various values of pind
from 0.95 to 1.00, the proportion of markers that follow gene-environment independence. The simulation is carried out with n1
= 2,000. One can note that the 2-step and the case-control procedures always maintain FWER, whereas the FWER for the case-only method is at 0.80 even when 99.95% of the SNPs are independent of E
. The FWER control of EB-type procedures and model-averaging procedures is much superior than the case-only method, and FWER is maintained if the fraction of SNPs that actually follow the G
independence assumption is 99.0% or more. The model-averaging procedures BMA and AIC offer better control of FWER compared with EB-type procedures when pind
is lower, for example, 0.95, when more than 5% of SNPs are associated with E
. One may note that, for higher values of pind
closer to 1.00, which is likely to be realistic in practice, the EB-type as well as model-averaging procedures can maintain strict FWER and even be conservative. Web Table 1
contains FWER results with M
= 10, 000.
Family-Wise Type I Error Rate (Expected Number of False Positives) Corresponding to the 8 Testing Procedures When pind Varies From 0.95 to 1.00a,b
In terms of the expected number of false positives in , the case-only analysis is still worse among all methods but does not appear to be an unreasonable strategy with the expected number of false positives less than 1 when pind = 0.9995 and around 7 when pind = 0.9975, which rises to around 158 with pind = 0.95. This may be a more rational metric to examine in GWAS instead of FWER, which considers only the more conservative criterion of probability of at least 1 false rejection under the global null hypotheses.
and represent the power values for testing H0
= 0 at the causal locus for the 8 methods with M
= 100,000. The exact numerical values corresponding to the graphs are contained in the Web tables and figures
. The fraction of null SNPs that satisfy the independence assumption is set at 0.995 in each case.
Figure 1. Empirical power curves for the 8 approaches: case-control (CC), case-only (CO), empirical Bayes (EB) and empirical Bayes, version 2 (EB2), 2 step (TS) with different first step α1, Akaike Information Criterion (AIC) model averaging, and Bayes (more ...)
Figure 2. Empirical power curves for the 8 approaches: case-control (CC), case-only (CO), empirical Bayes (EB) and empirical Bayes, version 2 (EB2), 2 step (TS) with different first step α1, Akaike Information Criterion (AIC) model averaging, and Bayes (more ...)
We first discuss the main features in with n1 = n0 = 2,000 when the independence assumption holds at the causal locus (exp(θGE) = 1.0). As expected, case-only analysis has the maximum power compared with all other contenders. Among hybrid methods, the 2-step and EB perform similarly, and these 2 methods generally have higher power than BMA or AIC. The 2-step approach with α1 = 5 × 10−4 has slightly higher power than EB for an interaction odds ratio exceeding 1.6. For example, with n1 = n0 = 2,000, at interaction odds ratio = 1.8, the power of EB is 0.68, of EB2 is 0.58, and of BMA and AIC is 0.59, whereas the 2-step method with α1 = 5 × 10−4 has power of 0.74. In this setting, the 2-step method with α1 = 0.05 attains a power of 0.53. The case-only method has power of 0.92, whereas the case-control analysis has a low power of 0.31. For lower values of the interaction odds ratio (≤1.6), the EB and TS have very similar performance, and they outperform the model-averaging approaches. For example, at interaction odds ratio = 1.6, both EB and TS (α1 = 5 × 10−4) have a power of 0.30, whereas the power of BMA and AIC is 0.18. The EB2 power is at 0.23, whereas TS with α1 = 0.05 has power of 0.20. The case-control and case-only analyses have power values of 0.08 and 0.54, respectively. For an interaction odds ratio less than 1.3, the EB procedure appeared to have slightly greater power than TS, but both of the power values were very low under this sample size.
The bottom panel of with n1 = 2,000 and n0 = 4,000 shows a similar trend but weaker performance by the TS method with both choices of α1. With the case:control ratio being tilted toward having more controls than cases, the first-step screening test in the combined sample of cases and controls loses the power advantage of a case-only approach when compared to 1:1 case:control ratio. Thus, we note a surprising and counterintuitive finding that, under identical simulation settings, the power of the 2-step procedure actually decreases with an increasing number of controls, if the number of cases is held fixed, as the power depends on the case:control ratio. For example, under the independence assumption, with interaction odds ratio = 1.9, with n0 = 2,000, TS (α1 = 5 × 10−4) has power of 0.87 that reduces to 0.82 as n0 increases to 4,000. On the other hand, under the same setting, the power of TS (α1 = 0.05) is 0.69 with n0 = 2,000 and increases to 0.89 with n0 = 4,000. This indicates that the optimal screening threshold α1 in the 2-step procedure does heavily depend on the case:control ratio, everything else remaining the same.
Under departures from the independence assumption at the causal locus, we consider 2 situations: one with a positive and the other with a negative association between G
in the controls. With (exp(θGE
) = 1.1), again the case-only method has the highest power, and the 2-step with α1
= 5 × 10−4
has the second highest power and a clear dominance over other hybrid methods. In contrast, under negative dependence at the causal locus (exp(θGE
) = 0.8), case-control analysis is the most powerful analysis, and case-only analysis performs quite poorly (refer also to the report by Li and Conti (14
)). In this situation, where βGE
is positive and θGE
is negative, the G
log odds ratio in cases (which is simply βGE
for a 2 × 4 table) is close to null, explaining the loss of power. The 2-step approach also performs quite poorly in this setting, especially with the more stringent choice of α1
= 5 × 10−4
. The BMA, EB, EB2, and AIC perform comparably among the hybrid methods, with BMA/AIC having an edge over the EB-type methods in this scenario. The loss of power in TS under a study design with more controls than cases becomes quite drastic with negative G
association as one can notice in the leftmost panels of . TS with both choices of α1
loses power as n0
increases under such negative dependence.
In order to understand the phenomenon of the better power property of EB over TS at smaller values of interaction odds ratio under independence, we increased the sample size to n1
= 10,000 and repeated the same simulation over a more modest range of interaction odds ratio from 1.1 to 1.5. essentially captures the same features of the different methods as discussed for . Under independence, EB has power advantages over TS for smaller values of interaction, especially for an unequal case:control ratio (bottom panel, center graph, in ). Under positive dependence, TS has a clear dominance and, under negative dependence, BMA/AIC has the advantage over EB, whereas TS performs quite poorly. This larger sample size setting is more reflective of current post-GWAS consortium studies exploring G-E
effects. Results for several other simulation settings are presented in Web Figures 1–3
. Exact numbers corresponding to each figure are contained in Web Tables 2–9
presents the estimated relative MSE corresponding to the log odds-ratio parameter at the causal locus under the simulation setting of for all the methods except the 2-step method (which is more of a screening tool and not an estimation method). The MSE for each method is divided by the MSE of standard case-control analysis. One can notice the advantage of EB-type methods in terms of this metric as one tries to balance between bias and efficiency in a data-adaptive way. The case-only method is best only when the independence assumption is true (the central block) and performs worse under any departures from the independence assumption.
Figure 3. Relative mean-squared error of the 6 approaches except for the 2-step approach corresponding to the estimation of βGE at the causal locus. The 6 approaches are case-control (CC), case-only (CO), empirical Bayes (EB) and empirical Bayes, version (more ...)