We evaluated the performance of the BRCAPRO and BOADICEA BRCA mutation carrier prediction models in three racial/ethnic groups, consisting of African-American, Hispanic and non-Ashkenazi Jewish NHW breast cancer patients from the San Francisco Bay Area. To our knowledge, this is the first study to compare these BRCA mutation-prediction models across population samples of such racial/ethnic diversity. In general, the models showed similar discrimination within each racial/ethnic group, but differences in calibration: BRCAPRO under-predicted mutation carriage in Hispanics, whereas BOADICEA over-predicted mutations in African Americans and in older NHWs.
The strength of this study is its focus on population-based samples of racial/ethnic minorities, in contrast to most prior evaluations of BRCA mutation prediction models. Some prior studies have found similar accuracy of BRCAPRO in racial/ethnic minorities as in NHWs (13
), but we and others reported under-prediction by BRCAPRO and other models among clinic-based minorities including Asian Americans and Hispanics (18
). Models may perform less well in racial/ethnic minorities because the prevalence of carriers among breast cancer cases may differ by race/ethnicity. In non-Ashkenazi Jewish NHW cases, our prevalence estimates for BRCA1 (2.1%) and BRCA2 (2.3%) are similar to those used by the BRCAPRO and BOADICEA models (3
); by contrast, we found that African-American cases had lower (BRCA1 1.1%; BRCA2 1.8%), and Hispanic cases higher (BRCA1 3.2%; BRCA2 3.2%), carrier prevalence. Notably, the exception to BOADICEA’s general over-prediction occurred in Hispanics, as did a significant under-prediction by BRCAPRO, both consistent with Hispanics’ higher mutation prevalence than observed in non-Ashkenazi Jewish NHWs. Within subsets specific for family history and age, calibration worsened with increasing divergence from the mutation prevalence expected by the models; for example, both models over-predicted significantly only in African Americans with family history of breast cancer, while under-predicting in Hispanics lacking such family history. Recent publications have reported a higher prevalence of the 185delAG BRCA1 founder mutation in Hispanics than was initially appreciated (18
), leading some to suggest a common origin for this mutation in Sephardic and Ashkenazi Jewish populations (18
). The present results contrast with those from a recent single-center clinic-based study of BRCAPRO in Hispanics, which reported better model performance than we found (17
); variations in the use of BRCAPRO between studies may explain some of this difference. Our finding of lesser BRCAPRO model accuracy in Hispanics also prompts questions as to whether BRCA mutation penetrance, or associated cancer risk, might differ between Hispanics and NHW. Given the growing size of the U.S. Hispanic population, further study of this issue has important implications for health policy and resource allocation.
In contrast to prior studies of the calibration of BRCAPRO and similar models in clinic-based settings (1
), this analysis considered populations having lower BRCA mutation prevalence, with 85 (6%) carriers identified among 1,365 tested patients. This study sample reflects the reality of current clinical BRCA mutation testing, given patient preference and practice guidelines that support more inclusive testing than previously advised2
. Our finding that BRCAPRO and BOADICEA over-predicted in a substantial proportion of patients, particularly in patients with family history of breast cancer or with greater than 80% predicted probability of mutation carriage, likely results from lower mutation frequency, and perhaps higher sporadic breast cancer incidence (39
), in these groups than the model parameters assume; we anticipate that population-specific corrections may improve model calibration, as others have demonstrated (7
Prior studies of the BRCAPRO model’s discrimination have reported AUCs in the range of 60–88%. Comparisons of BRCAPRO to other BRCA mutation prediction models, including BOADICEA, Couch, Finnish, National Cancer Institute, Frank/Myriad II, the Manchester Scoring System, the Family History Assessment Tool, and Shattuck-Eidens/Myriad I, have revealed relatively few differences in terms of discrimination (1
). Exceptions include the slightly superior performance of BOADICEA and the Manchester Scoring System in the United Kingdom, of the Italian IC software modification of BRCAPRO among Italians, and of the LAMBDA model among Ashkenazi Jewish probands; some of these models use population-specific mutation prevalence estimates, which tailored them to the groups under study (7
). In the present study, we evaluated model discrimination in three separate racial/ethnic populations, which may differ in their prevalence of BRCA mutations, and in the variance of their carriage probabilities. Variation in the probability of mutation carriage within a racial/ethnic population affects a model’s AUC. For example, if all breast cancer patients in a racial/ethnic group had the same mutation carrier probability, its AUC (which is the likelihood that the carriage probability for a randomly selected carrier exceeds that of a randomly selected noncarrier) would equal its minimum of 50%, indicating that the model is no better at discriminating between carriers and noncarriers than random chance. Given such intra-group variability, it is difficult to compare a model’s discrimination across different racial/ethnic groups. Comparing BRCAPRO’s and BOADICEA’s discriminative abilities within a single population is more straightforward, and we found no difference between models in any of the three racial/ethnic groups under study. Within subsets defined by age and family history of breast cancer, we observed some trends in model performance which did not reach statistical significance (for example, both models discriminated better in younger, compared to older, African Americans). Future research should evaluate race/ethnicity-specific modifications to BRCAPRO’s and BOADICEA’s mutation prevalence assumptions, and compare each model’s discrimination to that of other prediction tools, within the racial/ethnic groups we studied. As understanding of the spectrum of BRCA mutations across race/ethnicity matures, it may prove optimal to develop models specific to each racial/ethnic population.
Although BRCA mutation testing was completed for only 67% of those invited to enroll in the NC-BCFR, the testing rate was similar for patients in Categories A (68%) and B (65%). This similarity suggests that family history was not related to patients’ willingness to participate in the registry and provide biospecimens for research. We assumed that the combination of BRCA testing methods used was 90% sensitive for detection of deleterious mutations (35
); however, if testing sensitivity was actually lower than 90%, then the models may over-predict less, and under-predict more, than we report here.
In conclusion, the BRCAPRO and BOADICEA models showed differences in performance across racial/ethnic and age groups in a large population-based series of breast cancer patients. This finding emphasizes the need for further study of BRCA mutations in specific racial/ethnic and age groups, and for development of more accurate mutation prediction methods, with customization for the populations to which they are applied.