In our study involving 5590 case subjects with breast cancer and 5998 control subjects, the addition of information on 10 genetic variants to a standard clinical breast-cancer risk model predicted the risk of breast cancer only slightly better than the clinical model alone. In the inclusive model, the ROC curve was above 60% of the possible AUC. That is, about 60% of the time, a randomly selected patient with breast cancer had a higher estimated risk than the risk for a randomly selected woman in whom breast cancer did not develop during the follow-up period. By contrast, a single dichotomous risk factor detected in 60% of case subjects and 40% of control subjects (odds ratio, 2.25; AUC, 60%) would discriminate about as well as our model by the AUC criterion. We saw similarly modest improvement using measures based on the change in estimated risk. Although our data suggest that the SNP-only model predicted risk slightly better than the Gail model, the nongenetic clinical variables are available at essentially no cost, whereas the costs of obtaining genetic information are likely to be substantial.
We have evaluated risk-prediction models as we would any other clinical test. Our results are presented in terms of absolute risks, which are easily translated into positive and negative predictive values in considering who will benefit from a clinical test, including a risk model.24-26
Our findings were based on a large number of case subjects and control subjects, drawn from four prospective cohorts and one population-based case–control study. Our inclusive model incorporated information on components of the Gail model and genotypes of newly established SNPs; this information is not often included together in the same study. Our analysis also had technical advantages over other risk-modeling efforts: we fitted the effects of age and cohort so as not to give the models credit for fit based on demographic factors; we were able to assess the effect of adding SNPs on estimated risk and on changes in risk category for case subjects and for a hypothetical population.
There were some unavoidable weaknesses of this study and its data sources, particularly in evaluating risk for clinical use. For example, we pooled data from four U.S. cohort studies and a case–control study from Poland; these studies had different designs and enrollment characteristics. We included only women of European ancestry in the empirical analysis, and we did not consider subtypes of breast cancer. As a group, the study participants are not representative of any specific population. Screening practice may vary within and among studies. Because of self-selection for participation in the studies, we expect that the average risks in the U.S. studies will be different from each other and from the U.S. average. Similarly, in a clinical context, we cannot expect the risk for an individual woman to be the same as the average risk for a population. Our estimates of the performance of the risk models may be slightly higher than can be expected in typical clinical settings because we report results with minor overfitting. Given the complexity of the model with 4 Gail model components and 10 SNPs, we chose to fit each factor without constraints but did not attempt to evaluate hundreds of interactions among the factors or with age and cohort. Our simple interaction models gave no suggestion of improved performance from interactions between genetic and Gail model components; more sophisticated modeling may improve the performance of these factors in predicting breast-cancer risk. Finally, comparisons of models are slightly unfavorable to the Gail model because we could not include the history of atypical hyperplasia and information on mammographic density. Although they are not routinely incorporated into the Gail model, these factors may improve the performance of future versions of the model.14
Our analysis indicates that the genetic variants we studied provide modest improvements in discrimination and prediction models, whether measured as the AUC, as a discrimination index, or as a change in position in broad bands of risk, such as might be used in clinical settings. We see little evidence of benefit from including genetic variants at the extremes of high and low risk, categories in which further stratification might be most valuable. This empirical demonstration of the potential benefit of adding SNP data to breast-cancer risk models based on individual data is generally consistent with theoretical predictions15,27
that use published estimates of effect.
Because statistics such as the AUC and integrated discrimination improvement do not provide a readily intuitive sense of the clinical usefulness of these models, we focused much of our discussion on the degree of incremental improvement associated with adding the genetic variants. As in diabetes28
and cardiovascular disease,29
the addition of the common SNPs added little to the predictive value of the clinical models. On the basis of theoretical models, Gail30
has shown that increases in the AUC similar to those observed here are not sufficiently large to improve meaningfully the identification of women who might benefit from tamoxifen prophylaxis or assignment of screening mammography.
Although the Gail model and SNP-inclusive models may help to identify groups of women who have an increased risk of breast cancer for trials of interventions, none of the models in our set of data accurately predicted the development of breast cancer. Our results indicate that the recent identification of common genetic variants does not herald the arrival of personalized prevention of breast cancer in most women. Even with the addition of these common variants, breast-cancer risk models are not yet able to identify women at reduced or elevated risk in a clinically useful way.