We examined several models for predicting absolute risk. We examined the effects of including the four Gail variables, with modified variable definitions, which we refer to as the Swe-Gail variables, as well as the effect of including PD and BMI. In all, 18 breast cancer susceptibility loci with common risk alleles have been examined in this study (Table ); referred to as The18 herein. We also selected out the earlier known subset of seven markers studied by Gail [2
], referred to herein as The7.
We first examined the classification abilities of models with and without PD and BMI, but including Swe-Gail risk factors age at menarche, age at first live birth, family history, benign breast disease, in 1,739 cases and 1,672 controls (Table ). Without PD and BMI we observed an AUC of 0.569 (95% confidence interval (CI) = 0.550 to 0.588), compared with an AUC of 0.602 (95% CI = 0.584 to 0.621) with PD and BMI. The difference in AUCs was statistically significant (ΔAUC = 0.033, P = 1.17 × 10-7). Based on a subset of women with complete data on Gail variables and SNPs, a statistically significant improvement in AUCs was seen when adding The7 to the Swe-Gail model. Improvement was further enhanced when the recently discovered 11 SNPs were added (ΔAUC = 0.018, P = 4.69 × 10-4). Furthermore, a gain in AUCs was observed from including these 11 SNPs when the baseline model also included PD and BMI. We finally selected a subset of women with complete data on Gail variables, SNPs, PD and BMI and compared the performances of the Swe-Gail model and a model additionally including PD, BMI and The18. The latter model, referred to as the full model herein, obtained an AUC of 0.619, improving the AUC by 0.067 (P = 3.24 × 10-9). In this subset, with PD and BMI only we observed on AUC of 0.541 (95% CI = 0.515 to 0.568), with The18 only we observed an AUC of 0.589 (95% CI = 0.563 to 0.614) and with PD, BMI and The18 we observed an AUC of 0.600 (95% CI = 0.575 to 0.626).
Areas under the receiver operating characteristic curves for different combinations of prediction models.
The values of absolute five-year risk of breast cancer for the women included in our study, calculated at time of sampling/diagnosis based on the Swe-Gail model and the model additionally containing PD, BMI and The18 are plotted in Figure . Complementing the Gail model with PD and The18 increases the spread of the predicted absolute risks. A marked difference in distributions between cases and controls was observed for the full model. The means of the absolute five-year risks were 3.69% and 2.84% for cases and controls, respectively. Of the controls and the cases, 47.9% and 64.8%, respectively, had a five-year absolute risk higher than 2.5%. The difference in distributions between cases and controls was more subtle for the Swe-Gail model.
Distributions of estimated absolute risk by case-control status using the Swe-Gail model and the full model (with displayed proportions of women with five-year absolute risks greater than (multiples of 2.5%).
Assuming three risk categories, we used reclassification tables to compare pairs of models in terms of their assignment of women to low (0,ε1
), intermediate (ε1
), and elevated risk categories (ε2
,1), based on five-year absolute risk estimates (Table ). As cut-off values we choose ε1
(= 2.41%) to correspond to the first quartile of the estimated risk based on the Swe-Gail model and ε2
(= 4.11%) to correspond to the third quartile. The NRI value for the comparison of the Swe-Gail model with the full model was 0.170 (Z = 5.750, P
= 8.93 × 10-9
). In total, 46% of women were reclassified. Reclassification based on the full model was overall in the right direction, with an upward shift in risk categories for cases and a downward shift for controls. The global IDI measure was 0.004 (Z = 5.742, P
= 9.33 × 10-9
). Using the cut-off values suggested by Mealiffe et al.
], i.e. ε1
= 2%, the NRI value for comparing the same two models was estimated to be 0.193 (Z = 8.229, P
= 2.22 × 10-16
Reclassification for the Swe-Gail model compared with the full model, based on cut-off values determined by first and third quartile of predicted risk by the Gail model.
Model calibration was assessed using the Hosmer-Lemeshow approach and by calculating Brier scores. All models showed lack of fit; however, lack of model fit does not necessarily limit classification ability based on estimated risks [32
]. Results for the Swe-Gail model and the full model are displayed in Tables A2 and A3 [see Additional file 1
] and in Figure . Both the Brier score and the Hosmer-Lemeshow test statistic values indicate an improvement in goodness of fit as a result of updating the Swe-Gail model with PD, BMI and SNP data.
Observed versus predicted proportions of cases for deciles of risk score for the Swe-Gail model and the full model.
One way to assess the predictive power of a model is to estimate the proportions of cases that are accounted for by given percentages of the population at the highest risk [30
]. Figure displays these proportions based on the risk distribution generated by the Swe-Gail model, and by the full model. For the full model the proportion of cases explained by the 20% of the population at the highest risk was equal to 40.1%, compared with 35.1% for the Swe-Gail model.
Proportion of breast cancer cases explained by the proportion of the population at highest risk of the disease, for the Swe-Gail model and the full model.
For four personalised screening models, we compared the percentage of individuals eligible for screening and the percentage of cases potentially detectable by screening, for eligible cases, against the (current) approach of screening all women aged 40 to 75 years at three cut-offs of eligibility [see Table A4 of Additional file 1
]. The full model with eligibility for screening defined by an absolute risk cut-off value of 2% has a slightly lower level of eligibility than the Swe-Gail model with a 2.5% cut-off for eligibility (74% and 76%, respectively), but has a substantially higher catchment; for the full model 90% of the cases are potentially screen detectable, while for the latter only 85% are potentially detectable. As a consequence of adding SNPs, BMI and PD to the risk-prediction model, resources are more efficiently re-allocated to women with high-risk profiles. With the aim of comparing screening strategies with equal resources, we also calculated the number of cases captured by screening based on the most efficient age-only screening program, and based on individualised screening using the full model when confined to including only 76% of women aged 40 to 75 years. The percentage of all cases aged 40 to 75 years covered by the age-only based program was 81%, that is 4% less than the program based on the Swe-Gail model and this was 10% less than the program based on the full risk-prediction model, for which 91% of cases were screened. Results are summarised in Table . Age stratified percentages of cases eligible for screening together with the percentages of cases covered by screening, for the three programs with 76% coverage, along with a selection of models presented in Table A4, are presented in Table A5 [see Additional file 1
Percentage of cases detectable by screening for the screening strategies with 76% eligibility.