In the gene discovery phase, researchers often make use of highly selected series of patients and controls. Patients are selected for severe pathology, early onset and familial clustering of disease, and controls for the absence of pathology. This procedure substantially improves the statistical power of gene discovery research without creating any bias. But hyperselection of cases and controls can be a problem for evaluating the usefulness of genetic testing, as it typically leads to an overestimation of the effect sizes and, thus, to an overestimation of the predictive value. Effect sizes are inflated because frequencies of the risk genotypes are particularly increased in enriched patient populations and particularly decreased in controls that have no pathology related to the disease of interest.
Table shows that many studies on the predictive value of genetic profiling were conducted in hyperselected case-control series, comparing, for example, type 2 diabetes patients with normoglycemic individuals [
10,
11], patients with severe hypertriglyceridemia with normolipidemic controls [
12], or patients with end-stage AMD with individuals who have no eye pathology [
13]. By excluding individuals with modestly elevated glucose or lipid levels, these case-control series largely lose their relevance for investigating predictive potential in clinical practice, where persons with such levels are part of the population. Predicting progression to disease is most difficult in individuals with early symptoms or mild pathology, but prediction in this population is clinically highly relevant. One could argue that if the predictive value of genetic profiling is low in the samples used in these studies [
10-
13], it will be even poorer in unselected cohorts. Thus, hyperselected case-control studies can be useful to demonstrate that predictive genetic testing is not informative and, given the commercial interest in genome-based applications, this is an important message to get across.
Another consideration is the use of case-control studies in general, as illustrated by the recent findings on type 2 diabetes. Lango
et al. [
11] investigated the predictive value of 18 polymorphisms in a case-control study, comparing patients with normoglycemic controls, and van Hoek
et al. [
14] looked at the same polymorphisms in a prospective cohort of individuals aged 55 years and older. In both studies [
11,
14], the AUC of the 18 polymorphisms was 0.60 and the improvement in AUC beyond prediction from age, sex and body mass index (BMI) was limited (ΔAUC = 0.02). But a more detailed analysis of the results reveals that even though the AUC was 0.60 in both studies, it was mainly contributed by different genetic variants in the two studies (Table ). Moreover, the 0.02 improvement increased the AUC to 0.80 in the case-control study but to only 0.68 in the prospective cohort study. This difference is mostly explained by the difference in BMI. Mean BMI in the case-control study was 31.5 kg/m
2 in patients and 26.9 kg/m
2 in controls compared with 28.0 kg/m
2 and 26.0 kg/m
2 in the prospective cohort study, indicating that BMI was a stronger predictor of type 2 diabetes in the case-control study.
| Table 3Effect estimates of 18 established susceptibility variants on type 2 diabetes risk in two studies |
Case-control studies tend to overestimate odds ratios and this may be related to selection bias (most likely the case in the example above) or information bias (patients may attribute a disease to a known risk factor and they over-report this exposure). An issue that is often ignored in gene discovery studies but that is extremely relevant in studies evaluating the predictive value is that of survival bias. If genes increase the risk of disease, they may also increase the risk of (early) mortality. Therefore, there are strong arguments that show the necessity that predictive testing in preventive medicine should be investigated in cohort studies consisting of individuals who do not have the disease of interest, and predictive testing for prognosis and therapy response should be evaluated prospectively in clinically relevant patient series.
There is no single golden standard by which study population and study design should be selected, other than that predictive genetic tests need to be evaluated in populations representative for their intended use. The choice of the target population is not arbitrary, but rather is a trade-off of the effectiveness, costs and harmful side effects of available interventions, among other factors. Table shows three prospective cohort studies evaluating the prediction of coronary heart disease, one in Caucasian men of European ancestry aged 50-64 years [
15], one in a general population of 45-64 years [
16] and one in patients with familial hypercholesterolemia [
17]. These different study populations assume different target populations for genetic profiling, and the predictive value will differ between these populations when disease risks, genotype frequencies and effect sizes are different.