|Home | About | Journals | Submit | Contact Us | Français|
BRCAPRO, a BRCA mutation carrier prediction model, was developed on the basis of studies in individuals of Ashkenazi Jewish and European ancestry. We evaluated the performance of the BRCAPRO model among clinic-based minority families. We also assessed the clinical utility of mutation status of probands (the first individual tested in a family) in the recommendation of BRCA mutation testing for other at-risk family members.
A total of 292 minority families with at least one member who was tested for BRCA mutations were identified through the Breast Cancer Family Registry and the University of Chicago. Using the BRCAPRO model, the predicted likelihood of carrying BRCA mutations was generated. Area under the receiver operating characteristic curves (AUCs) were calculated.
There were 104 African American, 130 Hispanic, 37 Asian-American, and 21 other minority families. The AUC was 0.748 (95% CI, 0.672 to 0.823) for all minorities combined. There was a statistically nonsignificant trend for BRCAPRO to perform better in Hispanic families than in other minority families. After taking into account the mutation status of probands, BRCAPRO performance in additional tested family members was improved: the AUC increased from 0.760 to 0.902.
The findings support the use of BRCAPRO in pretest BRCA mutation prediction among minority families in clinical settings, but there is room for improvement in ethnic groups other than Hispanics. Knowledge of the mutation status of the proband provides additional predictive value, which may guide genetic counselors in recommending BRCA testing of additional relatives when a proband has tested negative.
Germline mutations in the BRCA1 and BRCA2 genes significantly increase the risks of breast and ovarian cancer. A recent meta-analysis of 10 studies estimated that the risk of developing breast cancer by age 70 years was 57% and 49% for BRCA1 and BRCA2 mutation carriers, respectively.1 The corresponding risks for ovarian cancer were 40% and 18%, respectively. As risk-reduction strategies, including chemoprevention and prophylactic surgeries, have been demonstrated to be effective in high-risk individuals,2 genetic testing for BRCA gene mutations has been increasingly utilized in clinical settings. Because genetic testing is expensive, genetic counselors and third-party payers often use the pretest probability that a woman carries a deleterious mutation in BRCA1 or BRCA2 as a guide in the recommendation of genetic testing. It is, therefore, important to ensure that the prediction models are accurate and appropriate in the populations to which they are applied.
Several carrier prediction models have been developed by using either a Mendelian approach3,4 or empirical data.5–8 Of these, the BRCAPRO model has been widely used and extensively evaluated, mostly in non-Hispanic white populations.9–16 This model, like other carrier prediction tools, was developed on the basis of mutation rates and penetrances observed mainly in women of Ashkenazi Jewish and European ancestry. To our knowledge, no large study has evaluated the utility of the BRCAPRO model in African American, Hispanic and Asian American populations.17,18 This is not surprising, given that minority populations remain underrepresented in genetic counseling and testing programs. A recent study identified large racial disparities between African Americans and whites in the use of BRCA genetic testing.19,20 Data on BRCA testing from Myriad Genetic Laboratories showed that less than 10% of individuals tested were from minority populations,7 though minorities constitute 31% of the US population.21
In clinical counseling, it is not always clear whether to offer genetic testing to family members for BRCA mutations after the proband has tested negative, because the proband's breast cancer could be nonattributable to BRCA mutations. Recalculation of the carrier probability for other family members by incorporating the proband's test result could be helpful in decision making, but the magnitude of improvement in risk prediction is unknown, because no empirical data are available. Another issue that confronts genetic counselors is that increasing numbers of individuals from high-risk families have undergone prophylactic interventions, such as oophorectomy or mastectomy. Prophylactic interventions alter mutation penetrance, and relatives who believe themselves to be at higher risk of developing cancer or of carrying a BRCA1 or BRCA2 mutation may be more likely to opt for prophylactic intervention. Therefore, ignoring medical interventions could diminish the accuracy of carrier prediction models.22 Although BRCAPRO now accounts for medical intervention and genetic testing results of family members,22,23 no study, to our knowledge, has evaluated these options by using empirical data.
This study was conducted to validate the performance of the BRCAPRO model among ethnic minority families seen in clinical settings and to examine whether the incorporation of prophylactic oophorectomy and/or genetic test results of probands improves the accuracy of carrier status prediction.
The study sample consists of individuals from 292 families identified through two sources. The first group of families was identified through the Breast Cancer Family Registry (BCFR),24 a consortium established by the National Cancer Institute in 1995. Three (New York City, Philadelphia, and Salt Lake City) of the six sites identified families through clinic-based and/or community-based recruitment between 1996 and 2005 and were included in this study. All BCFR participants were administered a questionnaire to obtain information on demographics, ethnicity, personal history of cancer, and family history of cancer in at least first- and second-degree relatives. Pathology report verification was sought to confirm reported breast and ovarian cancer history. The second group of families was identified through the Cancer Risk Clinic at the University of Chicago between 1993 and 2006. Family history was assessed by experienced genetic counselors, and a pedigree of at least three generations was constructed. All participants were informed that their DNA samples would be used for mutation screening under protocols approved by the institutional review board at each institution.
This study included self-reported African American, Hispanic, Asian-American, and Native American families. Families with Ashkenazi Jewish ancestry were excluded from the analysis. A family was eligible if at least one member had been tested for BRCA1 and BRCA2 mutations. For families with two or more members tested, the first family member enrolled at the respective institutions was designated as the proband. Appendix Table A1 (online only) lists characteristics of families by study centers.
Genomic DNA samples of participants from the University of Chicago and from the the New York City and Salt Lake City sites of the BCFR were tested at Myriad Genetic Laboratories (Salt Lake City, UT). Full sequencing analysis was done for probands, and single-site testing for the family-specific mutation was done for relatives of mutation-positive probands. For participants from the BCFR Philadelphia site, polymerase chain reaction fragments that covered the BRCA1 and BRCA2 genes were analyzed by using the enzyme mutation detection assay or heteroduplex analysis. Candidate mutations were confirmed by using direct sequencing.
Test results were considered positive if the mutation was protein truncating (ie, nonsense, frame-shifting insertions or deletions, splice site mutations) or a known deleterious missense mutation according to the Breast Cancer Information Core (http://research.nhgri.nih.gov/projects/bic). In some instances, mutation classification was determined in consultation with Myriad Genetic Laboratories. Variants of unknown significance were considered negative.
Probability that a proband or relative carried a BRCA1 and BRCA2 mutation was calculated by using the BRCAPRO model on the basis of her personal and first- and second-degree family history of breast and ovarian cancers.3 We used the version implemented in the BayesMendel 1.3-2 package,23 a library of the R statistical software.25 The default penetrances1 and allele frequencies4 for non–Ashkenazi Jewish populations were used in this analysis. The allele frequencies of BRCA1 and BRCA2 mutations were 0.058% and 0.068%, respectively, which corresponded to a mutation carrier frequency of 0.25%.4 For each proband, two sets of carrier probabilities were calculated by ignoring or incorporating family history of prophylactic oophorectomy. We used the full set of individuals and incorporated information on prophylactic oophorectomy when available. Postintervention penetrances were calculated with hazard ratios, as detailed by Katki.22 For each tested relative, two sets of carrier probabilities were calculated by ignoring or considering the proband's mutation test result.
Performance of the BRCAPRO model in ethnic minorities was evaluated in terms of its discriminatory ability and calibration.26,27 Regarding discriminatory ability, receiver operating characteristic (ROC) curves were constructed by plotting the sensitivities against 1 minus specificities by using each value of BRCAPRO prediction probabilities as a cutoff point. The area under the ROC curve (AUC)—that is, the concordance index—is a measure of the overall ability of discriminating BRCA mutation carriers from noncarriers. The AUC is also the likelihood that a randomly selected positively tested individual will have a higher predicted probability of BRCA mutation than a randomly selected negatively tested individual. An AUC of 0.5 indicated no discriminating ability, and an AUC of 1 meant perfect discrimination. AUCs were compared between ethnic groups and between predictions with or without incorporation of prophylactic oophorectomy by using the method described by DeLong et al.28 CIs for tested relatives' mutation probabilities with or without incorporation of test results of probands were evaluated by using the bootstrap method.29 For comparison with previous studies, the sensitivity and specificity of the model that used 10% mutation carrier probability as a cutoff were also presented.
Regarding calibration, the observed and predicted proportions of mutation carriers were compared in each ethnic group. Because the sensitivity of BRCA mutation detection techniques ranged from 63% to 85%,11,30 with the full sequencing method being the most sensitive method, whereas the specificity was believed to be 100%, the carrier probability calculated by the BRCAPRO model was converted for fair evaluation. Because a majority of the probands was screened by the full sequencing method, we assumed a sensitivity of 85% and calculated the carrier probabilities as the probabilities from the BRCAPRO model multiplied by 0.85. The assumed 15% false negative rate was due to undetected large deletions or rearrangements in BRCA1/2, deleterious missense variants in BRCA1/2, and breast and ovarian cancer susceptibility genes other than BRCA1/2 (D. Berry, personal communication, September 2008). We also grouped probands into four categories according to the converted probability: less than 0.05, 0.05 to 0.09, 0.10 to 0.24, and ≥ 0.25. Within each category, we calculated the average probability of positive tests (predicted proportion) and compared it with the observed proportion of mutations and the corresponding 95% CI (calculated by using the exact binomial distribution). The mean square error of prediction (ie, Brier score) was calculated to quantify the overall accuracy.31 The smaller the Brier score, the closer the predicted probability to the observed mutation status. Statistical analysis was performed by using SAS 9.1 (SAS Institute, Cary, NC) and Stata 10.0 (Stata Corp, College Station, TX).
In total, 292 minority families were identified through the BCFR and the University of Chicago, which included 104 African Americans, 130 Hispanics, 37 Asian-Americans, 16 Native Americans, and five families of multiple ethnicity. Because of their small numbers, Asian-American, Native American, and multiple-ethnicity families were combined and were referred to as other ethnicity. Table 1 lists the characteristics of families from the three ethnic groups. Fifty-eight (19.9%) of 292 probands tested positive for deleterious mutations: 32 in BRCA1 and 26 in BRCA2. Cancer history data were collected on 5,265 family members, and the average number was 18 individuals per family.
The mean carrier probability per BRCAPRO was 22.4% overall: 47.6% in probands who tested positive and 16.2% in those who tested negative (Wilcoxon rank sum test; P < 10−8). The AUC was 0.748 (95% CI, 0.672 to 0.823) for all minority groups combined. The AUCs were 0.832 (95% CI, 0.716 to 0.947), 0.679 (95% CI, 0.560 to 0.799), and 0.710 (95% CI, 0.505 to 0.916) for Hispanics, African Americans, and other minority groups, respectively (Table 2; Fig 1A).
Table 3 shows the observed and predicted proportion of mutations, stratified by risk categories, ethnicity, and family structure. Overall, BRCAPRO predicted 56 detectable mutation carriers (19.1%), when that the sensitivity of genetic testing techniques was assumed to be 0.85. This was close to the observed number of 58 mutations (19.9%). However, the expected proportion was higher than the observed proportion in high-risk probands (≥ 0.25 prior probability), and the opposite was seen in low-risk probands (< 0.05 prior probability). As indicated by Brier scores, the prediction accuracy (or calibration) was the highest in Hispanics, followed by other minority groups, and was the lowest in African Americans.
There were 120 probands with breast cancer but with no family history of breast cancer. BRCAPRO had extremely low discriminatory value in these families (AUC, 0.555; Fig 2B). Predicted and observed mutation proportions for these individuals were quite different (Table 3). For the remaining 172 probands, BRCAPRO had good discriminatory value (AUC, 0.794).
Prophylactic oophorectomy was performed in 62 individuals from 55 families (19%). After a history of prophylactic oophorectomy was accounted for, the AUC, sensitivity, and specificity at a 10% threshold changed only slightly (Appendix Table A2, online only; Fig 2A). For all 55 probands, the predicted carrier probability was increased, but the absolute change was small (mean, 0.015) and the increase in probability of greater than 0.05 was noted in only five probands.
In total, 393 individuals from the 292 families were tested for BRCA1 and BRCA2 mutations, and 43 families (15%) had more than one person tested. The mutation test results of probands were incorporated in the mutation prediction of the 101 additional family members tested by using a specific option in BRCAPRO, which resulted in a significant increase in discriminatory ability. Figure 2B and Appendix Table A2 show that the AUC increased from 0.760 to 0.902 for prediction of BRCA1 or BRCA2 (P = .02). In the 17 families in whom probands tested negative, 22 additional family members were tested for BRCA mutations. Their mean predicted probability decreased from 0.146 to 0.086 after incorporation of probands' test results.
In this study, we evaluated the performance of the BRCAPRO model in nearly 300 ethnic minority families in the United States and demonstrated that the model had good overall discrimination between BRCA mutation carriers and noncarriers in minority families. The AUC was 0.75 for all minority groups combined, which is within the range of those reported in non-Hispanic white populations (0.71 to 0.83).9,10,13,14,16,32–34
Of the minority groups examined, BRCAPRO performed the best in Hispanics, as indicated by the highest AUC (0.83) and the smallest Brier score (0.089), in which the AUC was higher than in a previous observation in Hispanics (0.77).18 The model had the worst performance in African Americans (AUC, 0.68; Brier score, 0.208), in which the AUC was lower than our previous observation in African Americans (0.77).17 The performance of BRCARPO was intermediate for the other minority groups, mainly Asian-Americans and Native Americans (AUC, 0.71; Brier score, 0.139), and was similar to that observed in Han Chinese (0.70).35 However, the difference between ethnic groups was not statistically significant. Thus, the data suggest that BRCAPRO performs reasonably well and is applicable to all minorities. However, the sample size of this study may not be sufficient for subgroup comparisons. Real differences possibly exist in BRCAPRO performance across ethnic minority groups, because the same penetrances and non-Ashkenazi white allele frequencies were used in the calculation. These genetic parameters likely vary across ethnicity. Better BRCAPRO performance in Hispanics might occur because Hispanics are genetically closer to other white populations. Thus, mutation prediction could improve if population-specific allele frequencies and penetrance data were available. Our findings underscore the need for population-based studies in these minority populations.36,37
A single instance of breast cancer in families that have limited family structure presents a challenge for mutation prediction with the BRCAPRO model and probably with other models. Weitzel et al15 reported an AUC of 0.67 for the BRCAPRO model in women with breast cancer who did not have any first- or second-degree relatives with breast or ovarian cancers. We observed a similar finding in the current cohort. In terms of accuracy of the model, we showed that the numbers of mutations predicted and observed were virtually the same when the mutation detection method is assumed to have 85% sensitivity. Consistent with previous studies in predominantly white populations,9–11 we found that BRCAPRO overestimated mutation probability for high-risk persons and underestimated mutation probability for low-risk persons. Interestingly, this pattern was also observed for the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA) model, which allows for polygenic locus effects,10 and for the Log Odds of the Probability of Carrying an Ancestral Mutation in BRCA1 or BRCA2 for a Defined Personal and Family Cancer History in an Ashkenazi Jewish Woman (LAMBDA) model, which was developed empirically.34 Reasons for this pattern are unclear and may be related to families that have a single breast cancer diagnosis and limited family structure, familial clustering caused by environmental factors, heterogeneous penetrance caused by genetic/environmental modifiers and mutation spectrum, other low penetrance genes, and imperfect sensitivity of molecular methodology. We showed that lack of information in families that have a single breast cancer diagnosis is an important cause of this pattern. Whatever the reasons, genetic counselors should treat the calculated mutation probability from pretest prediction models with caution and should consider the amount of available information when interpreting mutation probability. Given the performance in both discrimination and accuracy, these results indicate that BRCAPRO is a useful risk assessment tool in clinical settings for minority families, but clinicians should exercise judgment in using the numbers to make recommendations.
Incorporation of the proband's test result significantly improved the ability to distinguish likely mutation carriers from noncarriers (ie, AUC increased from 0.76 to 0.90). This is the first study, to our knowledge, to estimate the magnitude of improvement in clinical settings. Thus, it is helpful to recalculate the mutation probability for other family members after the proband has been tested, especially if the proband tests negative. In the two examples shown in Figure 3, the probands tested negative. For the 36-year-old breast cancer patient in family A, her mutation probability was 0.254 before the proband was tested, and the recalculated probability from BRCAPRO incorporation of the proband's test result was 0.075. Thus, a recommendation not to test for BRCA mutations could be entertained. (Indeed, her BRCA test result was negative.) In contrast, the family member with bilateral breast cancer in family B had a mutation probability of 0.882 before the proband's test result was considered, and her recalculated probability was 0.740 after incorporation of the proband's test result. Thus, a strong recommendation to test for BRCA mutation could be made. (In fact, she had a BRCA1 mutation).
We did not find that the inclusion of information on prophylactic oophorectomy, a new feature of BRCAPRO, improved mutation prediction. Improvement may be relatively small in the current cohort, because less than 20% of families had at least one member with a history of oophorectomy. Of the 62 individuals who underwent prophylactic oophorectomy, 22 later developed breast cancer after 11 years, on average. As illustrated by Katki,22 distortion caused by ignorance of information on prophylactic oophorectomy is large if the family member has lived many years after the surgery. Therefore, we may not be able to see an influence yet, as prophylactic interventions for familial breast cancer are still relatively recent. Another explanation is that the proband may not know whether her relatives had a prophylactic oophorectomy.
In conclusion, this study demonstrates that the BRCAPRO model performs well in clinic settings and supports its routine use in pretest prediction of BRCA mutations in minority families, especially Hispanic families. Mutation test results of probands, especially if negative, provide additional discriminatory ability for counselees, which may help counselors decide whether to offer other family members testing when one member has already tested negative. The study also highlights the need for population-based studies to estimate penetrance and allele frequencies of BRCA genes in minority populations. Lastly, the inaccuracy in carrier prediction using BRCAPRO for families with a single breast cancer diagnosis is a challenge worthy of additional investigation. Genetic counselors should recognize this limitation when using BRCAPRO or other models to recommend genetic testing in single-diagnosis families.
We thank Jeniffer Iriondo-Perez and Chaehyung Ahn at RTI International for database management; Alice Whittemore, PhD, Hormuzd Katki, PhD, Giovanni Parmigiani, PhD, Edwin Iverson, PhD, Esther M. John, PhD, Rita Nanda, MD, James Fackenthal, PhD, and Betsy Bove, PhD, for advice; Masha Kocherginsky, PhD, for R programming help; and Michelle Porcellino for critical reading of the manuscript.
|Characteristic||Study Center Location|
|Philadelphia||New York||Salt Lake City||Chicago|
|Total No. of families/probands||41||166||3||82|
|Probands with unilateral breast cancer||35||85.4||123||74.1||1||33.3||55||67.1|
|Probands with bilateral breast cancer||4||9.8||20||12.0||2||66.7||13||15.9|
|Probands with ovarian cancer||1||2.4||13||7.8||0||0||8||9.8|
|Probands with BRCA1 deleterious mutation||4||9.8||12||7.2||3||100||13||15.9|
|Probands with BRCA2 deleterious mutation||5||12.2||9||5.4||0||0||12||14.6|
|Families with ≥ 2 individuals tested||0||24||14.5||3||100||16||19.5|
|Families with oophorectomy||9||22.0||38||23.9||3||100||5||6.1|
|No. of individuals per family|
|No. of individuals with breast cancer per family|
|No. of individuals with bilateral breast cancer per family|
|No. of individuals with ovarian cancer per family|
Abbreviation: SD, standard deviation.
|Analysis Type by Group||Prediction Analysis Data|
|Unadjusted||292||0.707||0.645||0.748||0.672 to 0.823|
|Incorporation of oophorectomy||292||0.707||0.641||0.747||0.672 to 0.823||.84|
|Unadjusted||101||0.800||0.607||0.760||0.612 to 0.884|
|Incorporation of mutation results of probands||101||1.000||0.607||0.902||0.830 to 0.962||.02|
NOTE. BRCAPRO is a BRCA carrier prediction model.
Abbreviation: AUC, area under the receiver operating characteristic curve.
Supported by National Cancer Institute Grant No. CA-RO1 89085-01A, by the Falk Medical Research Trust, and by the Entertainment Industry National Women's Cancer Research Alliance. O.I.O. is a McArthur Fellow.
Presented in part at the 30th Annual San Antonio Breast Cancer Symposium, December 13-16, 2007, San Antonio, TX; and at the 43rd Annual Meeting of the American Society of Clinical Oncology, June 1-5, 2007, Chicago, IL.
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
The author(s) indicated no potential conflicts of interest.
Conception and design: Dezheng Huo, Olufunmilayo I. Olopade
Financial support: Olufunmilayo I. Olopade
Administrative support: Shelly Cummings, Kisha Hope, Olufunmilayo I. Olopade
Provision of study materials or patients: Ruby T. Senie, Mary Daly, Saundra S. Buys, Shelly Cummings, Kisha Hope, Olufunmilayo I. Olopade
Collection and assembly of data: Ruby T. Senie, Mary Daly, Shelly Cummings, Jacqueline Ogutha, Kisha Hope, Olufunmilayo I. Olopade
Data analysis and interpretation: Dezheng Huo, Ruby T. Senie
Manuscript writing: Dezheng Huo, Olufunmilayo I. Olopade
Final approval of manuscript: Dezheng Huo, Ruby T. Senie, Mary Daly, Saundra S. Buys, Shelly Cummings, Jacqueline Ogutha, Kisha Hope, Olufunmilayo I. Olopade