|Home | About | Journals | Submit | Contact Us | Français|
It is often difficult to distinguish a benign pelvic mass from a malignancy and tools to help referring physician are needed. The purpose of this study was to validate the Risk of Ovarian Malignancy Algorithm (ROMA) in women presenting with a pelvic mass.
This was a prospective, multicenter, blinded clinical trial that included women who presented to a gynecologist, a family practitioner, an internist or a general surgeon with an adnexal mass. Serum HE4 and CA125 were determined preoperatively. A ROMA score was calculated and classified patients into high and low-risk groups for having a malignancy. The sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV) of ROMA was estimated.
A total of 472 patients were evaluated with 383 women diagnosed with benign disease and 89 women with a malignancy. The incidence of all cancers was 15% and 10% for ovarian cancer. In the postmenopausal group a sensitivity of 92.3% and a specificity of 76.0% and for the premenopausal group ROMA had a sensitivity of 100% and specificity of 74.2% for detecting ovarian cancer. When considering all women together, ROMA had a sensitivity of 93.8%, a specificity of 74.9% and a NPV of 99.0%.
The use of the serum biomarkers HE4 and CA125 with ROMA has a high sensitivity for the prediction of ovarian cancer in women with a pelvic mass. These findings support the use of ROMA as a tool for the triage of women with an adnexal mass to gynecologic oncologists.
Ovarian cancer is the leading cause of death from gynecologic malignancies in the United States, with an annual incidence of 22,000 cases and an annual mortality of approximately 14,000(1). The optimal treatment for ovarian cancer includes cytoreductive surgery followed by adjuvant chemotherapy with significantly better prognostic outcomes if the initial surgery and treatment are performed by surgeons and at centers experienced in the management of ovarian cancer (2–6).
At the time of the initial diagnosis, women with ovarian cancer symptoms and adnexal masses present primarily to gynecologists, primary care physicians or general surgeons. Triage guidelines put forth by the American College of Obstetricians and Gynecologist and the Society of Gynecologic Oncologists recommends referral of women with a pelvic mass at high risk for ovarian cancer to gynecologic oncologist(7;8). An important dilemma is faced by the physicians who initially see these patients as to which patients are appropriate to refer to a gynecologic oncologist. The triage of women at high risk for an ovarian malignancy is of vital importance as recent studies indicate that ovarian cancer patients managed by gynecologic oncologists and at high volume institutions are more likely to undergo complete surgical staging and optimal cytoreductive surgery with fewer complications and better survival rates than patients treated by surgeons less familiar with the management of ovarian cancer(2;3;6;9–12). To improve the triage of patients with pelvic masses, we performed a pilot study that measured 7 biomarkers in sera drawn from women with pelvic mass prior to surgery. We found that when adding an additional biomarker to CA125 only HE4 was able to increase the sensitivity compared with using CA125 alone(13). Data from this pilot study and a second pilot study were combined to develop the risk of ovarian malignancy algorithm (ROMA) which combines serum levels of the biomarkers CA125 and HE4 along with menopausal status in a logistic regression model to classify patients with a pelvic mass into high or low risk groups for having epithelial ovarian cancer (EOC). The clinical performance of ROMA was first validated in a prospective, double-blind clinical study on patients with an adnexal mass recruited at tertiary care centers in patients that presented to gynecologic oncologist and thus were considered a high risk cohort(14). At a set specificity of 75%, ROMA displayed a sensitivity of 94% for distinguishing benign status from EOC, and 85% sensitivity to identify early stage I and II disease. Additionally, ROMA outperformed the Risk of Malignancy Index (RMI),an algorithm that employs ultrasound, menopausal status and CA125 for differentiating benign form malignant disease (15).
The purpose of the current study was to validate the use of ROMA in a low risk population of women presenting to a generalist with an adnexal mass and evaluating the clinical utility of ROMA in aiding the triage of women to gynecologic oncologists.
This study was a prospective, multi-center, double blinded clinical trial conducted at 13 clinical study sites (7 general and 6 specialty hospitals) across the United States between October 2009 and August 2010. The study protocol was approved by Institutional Review Boards at each site and was registered on ClinicalTrials.gov web site (accession number NCT00987649). Written informed consent was obtained from each patient prior to entry onto the trial and the collection of blood.
The study included premenopausal and postmenopausal women 18 years of age or older presenting to a generalist (defined as a general gynecologist, internist, family practitioner gastroenterologist or general surgeon) with an ovarian cyst or an adnexal mass and subsequently scheduled to undergo surgery. An adnexal mass was defined as a simple, complex or a solid ovarian cyst/or any mass in pelvis as determined by imaging (ie, ultrasound, CT scan or MRI). Menopausal status was determined by the physicians through history and physical exam. If the menopausal status was not known or reported, the following criteria were employed: Women 48 years or younger were considered premenopausal, and women 55 years and older were considered postmenopausal. If the patient’s age was between 49 – 55 years and the last menstrual cycle was unknown, the FSH levels were analyzed (Abbott, Abbott Park, IL, USA). Women with FSH levels greater than 22 mIU/mL were considered postmenopausal and lower than 22 were considered premenopausal as recommended in the FDA package insert for the Abbott ARCHITECT i2000 platform. Women with a previous history of ovarian cancer, bilateral oophorectomy, currently known to be pregnant or unable to provide informed consent were excluded from the study.
Blood samples were collected from all subjects within 30 days prior to their surgical procedure and all samples were drawn prior to the induction of anesthesia. Blood was collected into Serum Separator Tubes (SSTs) and was allowed to clot for at least 30 minutes prior to centrifugation. The blood samples were centrifuged for 10 minutes at 1100 – 1300 g, serum and plasma were separated, transferred to cryovials, and frozen at −20°C until testing. Appropriate volumes of serum were tested to quantitate the concentrations of CA125 and HE4. Serum HE4 levels were determined using enzyme linked immunoabsorbent assay (EIA) kit (Fujirebio Diagnostics, Inc., Malvern, PA, USA) and serum CA125 levels were determined using an ARCHITECT CA125II assays on the ARCHITECT Instrument (Abbott, Abbott Park, IL, USA). All the samples were tested in duplicates. Coefficient of variation (CV) for all the tests did not exceed 15.0%.
The primary endpoint of the current clinical study was to determine the effectiveness of the Risk of Ovarian Malignancy Algorithm (ROMA) for predicting EOC in patients presenting to a generalist with a pelvic mass. A detailed description of the development of the ROMA and statistical approach has been described in a prior publication by Moore et al(14).
To calculate the ROMA score, a predictive index (PI) was calculated using the serum HE4 and serum CA125 II levels and one of the following equations, depending on the patient’s menopausal status:
The following equation used the predictive index (PI) for each patient to calculate a Risk of Ovarian Malignancy (ROMA) score:
ROMA was used to stratify women into high risk or low risk groups for having a pelvic mass that is malignant or benign respectively. For both statistical and medical reasons, cut-points were chosen that provided a set specificity of 75% for the HE4 EIA and ARCHITECT CA125 II assay panel(14). In the original pilot trial and prior validation trial, ROMA score thresholds of ≥ 13.1% and 27.7% for premenopausal and postmenopausal women, respectively, achieved a set specificity of 75% as desired(14;16). Thus, for premenopausal patients, a ROMA score of ≥13.1% is considered high risk for malignancy and for postmenopausal patients, a ROMA score of ≥27.7% is considered high risk for malignancy.
Preoperative ROMA scores were calculated and the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were determined for all groups and subgroups. The Wilcoxon rank-sum test was used for the comparison of benign versus malignant groups and between menopausal status when examining serum levels of HE4 and CA125. A p-value of <0.05 was considered as statistically significant. The p-values were not adjusted for multiple evaluations.
Thirteen diverse sites from across the United Stated enrolled 512 women with a pelvic mass of which 472 (92.2%) were evaluable and are the focus of this report. The demographics for this cohort and pathologic classification are presented in Tables 1 and and22 respectively. There were 255 premenopausal patients and 217 postmenopausal patients. Plasma FSH levels were employed to determine menopausal status in 36 women, 22 of which had at least a remaining ovary after a prior hysterectomy. The mean age of all women entered on the trial was 50.3 years (range: 18 to 89). The mean age for the subgroup of premenopausal women was 39.7 years (range: 18 to 56) and for the subgroup of postmenopausal women 62.8 years (range: 44 to 89). Women diagnosed with benign disease made up 81.1% (383) of the cohort (150 postmenopausal and 233 premenopausal), and women diagnosed with a malignancy or a LMP tumor made up 18.9% (89) of the cohort (67 postmenopausal and 22 premenopausal). The histologic classification of the benign pathology is provided in Table 1 and malignant disease along with the primary site is displayed in Table 2.
Forty eight women (39 postmenopausal and 9 premenopausal) were diagnosed with a primary ovarian, fallopian tube, or primary peritoneal cancer all categorized under EOC. The stage of these cancers was as follows: 8 stage I, 4 stage II, 32 stage III, 2 stage IV and 2 unstaged. Low malignant potential tumors (LMP) were diagnosed in 19 patients (12 postmenopausal and 7 premenopausal), 2 patients were diagnosed with non EOC, 11 patients with other gynecologic cancers and 9 patients with non-gynecologic cancers (Table 2).
Evaluation of serum levels of HE4 and CA125 was carried out for all patients and for separate menopausal groups (Table 3). Significant differences for serum HE4 and CA125 levels were detected when comparing benign cases to all malignant cases or to all EOC and LMP tumors together (all p-values <0.0001).
The ROMA assignment of women into high and low risk categories for harboring a malignancy based on the menopausal cut points are illustrated in (Table 4). The sensitivity, specificity, positive predictive value (PPV) and negative predictive values (NPV) of ROMA for discrimination of benign disease from EOC and LMP tumors are displayed in Table 5.
Examination of all women with benign disease (N=383) or with EOC and LMP tumors (N=67) revealed ROMA classified 59 of the 67 women with EOC or LMP into the high risk group and 287 of the 383 women with benign tumors into the low risk group providing a sensitivity of 88.1% (95% CI: 77.8 – 94.7%), a specificity of 74.9% (95% CI: 70.3 – 79.2%), and a NPV of 97.3% (95% CI: 94.7 – 98.8%) as illustrated in Table 5. The algorithm incorrectly classified 5 out of 19 patients with LMP tumors and only 3 out of 48 with EOC (all 3 stage I-II) to the low risk group, providing a sensitivity for EOC of 93.8% (95% CI: 82.8% – 98.7%) at a specificity of 74.9% (95% CI: 70.3% – 79.2%) and a NPV of 99.0% (95% CI: 97.0 – 99.8%) (Table 6).
Examination of postmenopausal women with benign disease (N=150) or with EOC and LMP tumors (N=51) revealed ROMA classified 46 of the 51 women with EOC or LMP into the high risk group and 114 of the 150 women with benign tumors into the low risk group providing a sensitivity of 90.2% (95% CI: 78.6 – 96.7%), a specificity of 76.0% (95% CI: 68.4 – 82.6%), and a NPV of 95.8% (95% CI: 90.5 – 98.6%) as illustrated in Table 5. The algorithm incorrectly classified 2 out of 12 patients with LMP tumors and only 3 out of 39 with EOC (all 3 stage I-II) to the low risk group, providing a sensitivity for EOC of 92.3% (95% CI: 79.1 – 98.4%) at a specificity of 76.0% (95% CI: 68.4 – 82.6%) and a NPV of 97.4% (95% CI: 92.7 – 99.5%) (Table 6).
Examination of premenopausal women with benign disease (N=233) or with EOC and LMP tumors (N=16) revealed ROMA classified 13 of the 16 women with EOC or LMP into the high risk group and 173 of the 233 women with benign tumors into the low risk group providing a sensitivity of 81.3% (95% CI: 54.4 – 96.0%), a specificity of 74.2% (95% CI: 68.1 – 79.7%), and an NPV of 98.3% (95% CI: 95.1 – 99.6%) as illustrated in Table 5. The algorithm incorrectly classified 3 out of 7 patients with LMP tumors. All patients with EOC were correctly classified into the high risk group. Thus, out of the 9 premenopausal patients with EOC, all were correctly classified providing a sensitivity for EOC of 100% (95% CI: 66.4 – 100%) at a specificity of 74.2% (95% CI: 68.1 – 79.7%) and a NPV of 100% (95% CI: 97.9 – 100%) (Table 6).
On consideration of women with early stage disease alone (stage I and II) ROMA classified 9 of 12 patients with EOC correctly into the high risk group thus achieving a sensitivity for early stage EOC of 75.0% (95% CI: 42.8 – 94.5%), a specificity of 74.9% (95% CI: 70.3 – 79.2%) and a NPV of 99.0% (95% CI: 97.0 – 99.8%). When including early stage EOC and LMP tumors ROMA classified 23 of 31 patients with EOC or LMP tumors correctly to the high risk group thus achieving a sensitivity of 74.2% (95% CI: 55.4 – 88.1%) a specificity of 74.9% (95% CI: 70.3 – 79.2%) and a NPV of 97.3% (95% CI: 94.7 – 98.8%).
Examination of all women with benign neoplasms (N=383) or with any malignancy including LMP tumors (N=89) ROMA correctly classified 72 of 89 malignant or LMP tumors achieving a sensitivity of 80.9% (95% CI: 71.2 – 88.5%) at a specificity of 74.9% (95% CI: 70.3 – 79.2%). In postmenopausal women only, ROMA correctly classified 56 of 67 malignant or LMP tumors achieving a sensitivity of 83.6% (95% CI: 72.5 – 91.5%) at a specificity of 76.0% (95% CI: 68.4 – 82.6%) and in premenopausal women only, ROMA correctly classified 16 of 22 malignant or LMP tumors achieving a sensitivity of 72.7% (95% CI: 49.8 – 89.3%) at a specificity of 74.2% (95% CI: 68.1 – 79.7%).
The serum biomarker HE4 is a novel marker recently cleared by the United States Food and Drug Administration for ovarian cancer monitoring. HE4 is a member of a family of protease inhibitors that function in protective immunity and consists of two whey acidic protein domains and a 4 disulfide core. The HE4 gene ise expressed by epithelial ovarian tumors and can be detected through immunohistochemical staining of malignant ovarian tissue(17–20). The HE4 protein expressed by epithelial ovarian malignancies can also be detected in the serum of ovarian cancer patients(21). Equally important, HE4 has been shown to be a sensitive marker for the differentiation of benign ovarian tumors from those that are malignant and a marker for adenocarcinomas of the endometrium(13;14;16;21). Using area under the receiver operator characteristics curves (AUC ROC) as a measure of test performance, a pilot study examining nine potential serum biomarkers for epithelial ovarian cancers found HE4 to be the most sensitive marker for ovarian cancer. More importantly, it was demonstrated that combination of the serum biomarkers HE4 and CA125 achieved a higher AUC ROC and therefore had increased sensitivities than either marker alone. These findings have since been independently validated. Huhtinen et al demonstrated the combination of serum HE4 and CA125 can differentiate endometriosis from an ovarian malignancy with a sensitivity of 79% compared to CA125 and HE4 alone with sensitivities of 64% and 71% respectively. As well, when studying ovarian cancers versus controls the dual marker combination achieved a sensitivity of 93% compared with CA125 or HE4 which each achieved a sensitivity of 79% alone (22). More recently, Nolen et al evaluated 65 ovarian cancer related biomarkers in the circulation of women diagnosed with an adnexal mass and found that as individual markers, HE4 and CA125 provided the greatest level of discrimination between benign and malignant cases. In addition, consistent with our prior trial, these researchers also demonstrated that the combination of the biomarkers HE4 and CA125 provided a higher level of discriminatory power than either marker considered alone(23).
The stratification of women into menopausal groups is critical as biomarker expression, both HE4 and CA125, can vary depending on age and the presence of benign or malignant tumors in the two groups. Therefore these variables must be taken into account when developing algorithms employing biomarkers. The Risk of Ovarian Malignancy Algorithm (ROMA) was developed from the combination of two pilot studies and employs serum levels of HE4 and CA125 along with menopausal status to stratify patients into high and low risk groups for epithelial ovarian cancer(13;14). ROMA was then subsequently validated in a multicenter trial assessing women that presented with a pelvic mass. The cohort of women in this trial was considered to be at increased risk for having a malignancy as all patients were initially evaluated and enrolled by gynecologic oncologists. The initial validation trial had an incidence of all cancers of 33% and 24% for EOC alone for the women enrolled onto the trial. With this in mind, the application of ROMA to the higher risk cohort of women separated patients into high and low risk groups effectively. When assessing premenopausal and postmenopausal women together, ROMA stratified women with benign versus EOC and low malignant potential tumors (LMP) into high and low risk groups with a sensitivity of 89%, a specificity of 75% and a 94% negative predictive value (NPV). When assessing postmenopausal women alone, ROMA achieved a sensitivity of 92% and a specificity of 75%, with a negative predictive value of 93%. For premenopausal women alone, ROMA achieved a sensitivity of 77% and a specificity of 75%, with a negative predictive value of 95%. More importantly, ROMA detected 94% of the women with an invasive EOC and 85% of the women with early stage EOC in the initial validation trial. The findings of a decreased sensitivity for the premenopausal group can be accounted for by a higher incidence of LMP tumors that do not over express CA125 or HE4 and the increased incidence of benign tumors that over express CA125 in the premenopausal giving rise to false positive tests.
In the current trial reported here, all patients were initially evaluated by gynecologists, family practitioners, internists, gastroenterologists or general surgeons. For this cohort, the incidence of all cancers was 15% and for EOC alone it was 10%. These rates are significantly lower than in the previously reported validation trial and more in line with the population of women being assessed by generalists and primary care physicians. Despite the lower incidence of cancer in this study, ROMA stratified women into high and low risk groups with similar sensitivities and specificities as seen in the prior validation trial. The high sensitivity achieved through the use of ROMA allows for 94% of women with an epithelial ovarian cancer to be identified and triaged to a gynecologic oncologist while 75% of the patients with a pelvic mass will be treated by their gynecologist in their community. Equally important is the high NPV (97%) achieved by ROMA for women with EOC and LMP tumors. This high NPV provides a strong reassurance that a pelvic mass is benign. Accurate detection of women at low risk for malignancy would reduce unnecessary referrals to gynecologic oncologist allowing the patient to stay in their community with their primary gynecologist and support network with a low risk of ultimately being diagnosed with a malignancy.
Traditionally, the biomarker CA125 has been demonstrated to be elevated in only half of early stage ovarian cancer patients and elevated in up to 80 to 90% of all patients with ovarian cancer(24;25). Therefore the use of biomarkers for the detection of early stage disease has been limited. Examining patients with stage I and II disease, ROMA stratified the majority (75%) patients with early stage ovarian cancer correctly to the high risk group therefore identifying preoperatively patients that will benefit from full surgical staging which will dictate subsequent management and the need for chemotherapy.
Recently, ROMA has been evaluated and validated in populations outside of the United States. Montagnana et al reported on their findings on the performance of HE4 and CA125 for detecting ovarian cancer. Using an AUC ROC analysis, these researchers found that HE4 achieved higher AUC ROC compared with CA125 alone (AUC: 0.77 vs 0.64) for premenopausal women and (AUC ROC: 0.94 and 0.84) for postmenopausal women, similar to the finding described in our pilot trial(26). Likewise, when combining HE4 and CA125 and employing the ROMA in premenopausal and postmenopausal women separately, the authors report that ROMA had an AUC ROC of 0.77 compared with an AUC ROC of 0.64 for CA125 alone in the premenopausal group and a AUC ROC of 0.92 for ROMA and 0.84 for CA125. Interestingly the AUC ROC of HE4 versus that of ROMA were near equal in both the premenopausal and postmenopausal groups(26). The size of this study was small compared with previously reported trials and may not have had the power to detect a difference between the use of CA125 and HE4 over that of HE4 alone. Kim et al studied the use of ROMA in a Korean population and found that ROMA classified women into high and low risk groups with a sensitivity of 88% and a specificity of 94%, findings that are consistent with our two multicenter trials (27).
The current trial reported in this issue validates the previously reported trial for the use of ROMA to stratify patients into risk groups. Despite a lower incidence of epithelial ovarian cancer in this study population, ROMA performed equally as well, attaining high sensitivities and specificities for the prediction of a women having ovarian cancer. There is an urgent need for tools to help physicians identify which women with an ovarian cyst or pelvic mass are at high risk for a malignancy in order to accurately triage patients to surgeons and institutions specializing in the care and management of ovarian cancer while retaining most of the women with benign disease for management by their gynecologist. The accumulating data supports the use of HE4 and CA125 along with ROMA as an accurate tool to assist in the triage of women with a pelvic mass.
Funded by Fujirebio Diagnostics Inc. Richard G. Moore is partially supported by NCI grant CA136491-01 and Steven J. Skates partially supported by NCI grant CA152990.
Financial Disclosure: Richard G. Moore receives research funding from Fujirebio Diagnostics Inc. and Abbott Diagnostics Inc. M. Craig Miller receives consulting fees from Fujirebio Diagnostics Inc. John J. Ball received research funding from Fujirebio Diagnostics Inc.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
The other authors did not report any potential conflicts of interest.