Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Gynecol Oncol. Author manuscript; available in PMC 2010 March 1.
Published in final edited form as:
PMCID: PMC2664404

Improved Detection of Ovarian Cancer Metastases by Intraoperative Quantitative Fluorescence Protease Imaging in a Pre-Clinical Model



Cytoreductive surgery is a cornerstone of therapy in metastatic ovarian cancer. While conventional white light (WL) inspection detects many obvious tumor foci, careful histologic comparison has shown considerable miss rates for smaller foci. The goal of this study was to compare tumor detection using WL versus near infrared (NIR) imaging with a protease activatable probe, as well as to evaluate the ability to quantify NIR fluorescence using a novel quantitative optical imaging system.


A murine model for peritoneal carcinomatosis was generated and metastatic foci were imaged using WL and NIR imaging following the i.v. administration of the protease activatable probe ProSense750. The presence of tumor was confirmed by histology. Additionally, the ability to account for variations in fluorescence signal intensity due to changes in distance between the catheter and target lesion during laparoscopic procedures was evaluated.


NIR imaging with a ProSense750 significantly improved upon the target-to-background ratios (TBRs) of tumor foci in comparison to WL imaging (minimum improvement was approximately 3.5 fold). Based on 52 histologically validated samples, the sensitivity for WL imaging was 69%, while the sensitivity for NIR imaging was 100%. The effects of intraoperative distance changes upon fluorescence intensity were corrected in realtime, resulting in a decrease from 89% to 5% in signal variance during fluorescence laparoscopy.


With its molecular specificity, low background autofluorescence, high TBRs, and quantitative signal, optical imaging with NIR protease activatable probes greatly improves upon the intraoperative detection of ovarian cancer metastases.


Cytoreductive surgery in metastatic ovarian cancer has been shown to have a significant beneficial effect on patient outcome and represents an important cornerstone in the treatment of this disease [1-3]. Current guidelines define an “optimal” debulking procedure as one in which the surgeon leaves the abdomen with less than 1 cubic cm of residual tumor; additional evidence suggests that greater improvements in clinical outcome can be achieved by setting even lower thresholds for residual tumor [4]. There is an unmet clinical need for an intraoperative imaging modality to assist the surgeon in identifying tumor foci. Improved intraoperative tumor detection has the potential to yield both more accurate staging as well as decreased residual metastatic disease.

A variety of imaging modalities have been applied towards more sensitive detection of diffuse peritoneal carcinomatosis, with near infrared (NIR) optical imaging showing particular promise. The advent of fluorescent molecular probes with a propensity for differentially illuminating tumor foci against normal background tissue has allowed for the application of optical imaging towards a number of cancer models [5, 6], including ovarian cancer [7-9]. Our laboratory has developed a novel class of “smart” probes that are administered in an optically silent state, but in the presence of specific proteases, are cleaved and thus activated, resulting in up to a hundred-fold signal amplification [10, 11]. Protease, particularly cathepsin B, up-regulation has been shown to occur in a variety of disease states, including ovarian cancer[12], and thus these probes represent highly sensitive molecular beacons that may be applied for the detection of metastatic disease [13].

One commonly cited drawback to intravital optical imaging is the inability to quantify fluorescent signal. The collected signal varies dramatically as the distance between the tissue under investigation and the imaging apparatus changes during an operative procedure; this variability precludes quantitative analysis of the molecular probe's signal intensity, thus severely limiting the applicability of optical imaging to reliable minimally invasive tumor detection. To account for this distance dependence, we have recently developed methods that allow for the intravital and real-time quantification of NIR fluorescence [6, 14]. In this study, we demonstrate the ability of NIR “smart” probes to detect ovarian cancer metastases with high TBRs, as well as to improve upon the sensitivity for the visualization of cancer foci adherent to and within abdominal structures, versus standard white light (WL) evaluation. Finally, we apply a distance dependence correction algorithm to allow fluorescence imaging in a quantitative manner during in vivo laparoscopic procedures.

Materials and Methods

NIR activatable probe

The imaging probe used for this study is a commercially available protease-activatable NIR fluorescent probe, Prosense750 (VisEn Medical, Woburn, MA). The probe's structure consists of a synthetic graft polymer composed of poly-L-lysine that is sterically protected by multiple methoxypolyethylene glycol side chains. Conjugated onto this lysine backbone are multiple NIR fluorochromes whose fluorescence is quenched by their proximity to one another in a phenomenon similar to fluorescence resonance energy transfer. Following a cleavage event by proteases that degrades the lysine backbone, the fluorochromes are released and thus regain their fluorescent properties, increasing the NIR signal intensity.

Cell culture

The human metastatic epithelial ovarian cancer cell line OVCAR-3 (ATCC, Manassas, VA) was grown in RPMI media with 1% L-glutamine, 1% HEPES buffer, 1% sodium pyruvate, 2% sodium bicarbonate, 20% fetal bovine serum, and 0.1 mg/mL bovine insulin. The cells were incubated in a standard 37° incubator under 5% CO2 and 95% air. The cells were harvested when they reached 90% confluence by trypsinization and were suspended in HBSS prior to implantation in the animal models described below.

Ex vivo white-light and fluorescence imaging of metastatic disease

All animal experiments were approved by the institutional animal care committee. A model for metastatic ovarian cancer was generated by i.p. injection of 1 × 107 OVCAR-3 cells suspended in 100μL HBSS in n = 10 female nude mice (Taconic, Germantown, NY). A control arm was established by i.p. injection of 100μL HBSS in n = 4 female nude mice. Four weeks after i.p. injection, all mice were given 2 nmol of ProSense750 via tail vein injection. Twenty-four hours later, the animals were administered general anesthesia (2% isofluorane in 1 L/min O2) and then sacrificed by cervical dislocation. The following abdominal and pelvic tissues were excised and placed on a non-fluorescent tray: diaphragm, omentum, peritoneum, liver, spleen, stomach, small bowel, large bowel, bladder, uterus, ovaries, and para-aortic and mesenteric lymph nodes. Surface reflectance imaging was then performed on the samples using an epifluorescence system (Olympus Small Animal Imaging System OV100; Olympus Corp., Tokyo, Japan). The samples collected from the control mice were imaged first to establish anatomic and fluorescent baselines. The explanted tissues were first visualized under standard white light illumination; subsequently, the imaging system's filter sets were adjusted for epifluorescent imaging in the NIR spectrum, and the samples were again imaged, this time to measure ProSense750 signal. Regions of interest (ROIs) were drawn within each sample and a mean fluorescence value for each abdominal and pelvic organ was calculated from the 4 control mice using standard image analysis software (ImageJ). After this, the tissues collected from the peritoneal cancer model mice were imaged under WL and then NIR illumination. A positive focus of ProSense750 signal was defined as an area of fluorescence two standard deviations above the mean fluorescence for normal tissue, as previously determined by the imaging results from the control study. Any tissue that was considered positive for tumor by either WL or NIR imaging (n = 29) was preserved in 70% ethanol for subsequent histologic validation; moreover, a random sampling of tissue that was considered negative for tumor by both WL and NIR imaging (n = 23) was preserved and prepared for histologic validation in an identical fashion. Samples with positive tumor foci by imaging and with the presence of malignancy subsequently confirmed by histology were considered as true positives (TP); similarly, samples with no tumor foci by imaging and with the absence of malignancy subsequently confirmed by histology were considered as true negatives (TN). Samples with positive tumor foci by imaging that showed no sign of malignancy by histology were considered as false positives (FP); samples with no tumor foci by imaging that however were found to be malignant by histology were considered as false negatives (FN). Sensitivity was then calculated as TP/(TP + FN), and specificity was calculated as TN/(TN + FP).

Histologic assessment for presence of tumor in explanted tissue

Explanted tissues were embedded in freezing media and frozen, and 10 μm thick slices were obtained using a cryostat (Leica, Bannockburn, IL). Samples were fixed in 4% paraformaldehyde and stained with hematoxylin-eosin. Histologic assessment for the presence or absence of tumor in each sample was performed by an experienced pathologist blinded to the imaging results.

In vivo WL and quantitative NIR imaging of peritoneal cancer by laparoscopy

A focal peritoneal metastasis model for ovarian cancer was generated by implantation of 1 × 107 OVCAR-3 cells suspended in 100μL HBSS into the peritoneal surface immediately deep to the anterior abdominal wall in n = 3 female nude mice. After four weeks, the animals received 2 nmol ProSense750 by tail vein injection. Twenty-four hours after probe administration, the animals were imaged laparoscopically by a previously described quantitative fiberoptic catheter-based imaging system that allows for the simultaneous imaging of WL and quantitative NIR fluorescence [14, 15]. For these imaging experiments, the animals were placed in a supine position under general anesthesia with 2% isofluorane in 1 L/min O2. The fiberoptic catheter was introduced through a small puncture wound, and the abdomen was inflated with CO2. The focal metastatic lesions were imaged at both near and far distances from the tip of the catheter to evaluate the ability of the correction algorithm to account for the variations in collected NIR photons based on positional changes in the catheter.


Ex vivo imaging of metastatic ovarian cancer with conventional WL and ProSense750

To evaluate the ability of ProSense750 to detect ovarian cancer metastases, explanted abdominal and pelvic organs from a peritoneal carcinomatosis animal model were imaged under both conventional WL and NIR fluorescence; the results of this study are summarized in Figure 1 by images of representative tissue samples with tumor foci. Figure 1A demonstrates a section of omentum with a large, fixed, and irregular mass visible in WL that fluoresces vividly under NIR interrogation; histologic analysis subsequently confirms the malignant nature of the lesion. Similarly, Figure 1B illustrates a mass adherent to a section of peritoneum that was found to have high levels of ProSense750 activation and was subsequently confirmed to be metastatic disease by histology: within the peritoneal fat there is a cap of irregular epithelial cells representing malignant cells sitting atop a reactive lymphoid aggregate. Finally, Figure 1C shows a large para-aortic lymph node with strong ProSense750, with histology revealing infiltrating, large, abnormal cells effacing the sinusoidal pattern that would otherwise be seen if they represented reactive histiocytes within the lymph node.

Fig. 1
Representative WL and NIR images with corresponding histology of tumor foci in explanted tissues from the peritoneal carcinomatosis animal model 24 hours following administration of 2 nmol ProSense750. A, WL, NIR, and histology of a cancer focus in omental ...

Having demonstrated the ability of ProSense750 to detect metastatic foci, we next evaluated the improvement in TBR appreciated by NIR imaging over the current standard of care WL method; this improvement is depicted graphically in Figure 2 for a number of representative tissues. The dashed line represents a TBR value equal to 1, and the error bars shown represent two standard deviations from the mean. For all tissues evaluated in this experiment, NIR imaging with ProSense750 demonstrated a significant enhancement in tumor TBR over conventional WL imaging.

Fig. 2
Target-to-background ratios in WL versus NIR imaging for tumor foci in explanted tissue from the peritoneal cancer animal model, 24 hours after the administration of 2 nmol ProSense750. The dashed line represents a TBR value equal to 1. NIR imaging of ...

We then sought to investigate whether in addition to highlighting areas of tumor that can also be detected by WL imaging, NIR imaging could potentially identify metastatic foci that were otherwise invisible under solely WL interrogation. Figure 3 illustrates three examples of lesions with strong ProSense750 activation signal that were confirmed to be cancerous by histology but were not seen in WL imaging. Figure 3A is a segment of large bowel with no gross abnormality in the WL image but with a positive finding in the NIR; histology was significant for diffuse, patchy, hypercellular sheets of irregular cells lacking structural organization and with distorted cellular morphology. Figure 3B shows an explanted diaphragm that appears innocuous under WL investigation; however, a small tumor focus is readily apparent with NIR imaging. Finally, Figure 3C is a segment of omentum with a large cancerous lesion with strong NIR signal that was not detected under standard WL visualization, as the lesion was deep to healthy tissue. Histology reveals a lymphoid aggregate within the omental fat with infiltrating, irregular epithelial cells representing malignancy, distorting the lymphoid structure.

Fig. 3
Improved imaging of occult metastatic disease with ProSense750 over conventional WL in the peritoneal cancer animal model 24 hours following administration of 2 nmol ProSense750, with histologic validation of the presence of tumor. Red boxes in the WL ...

To quantify this improvement in tumor foci visualization, sensitivities and specificities for both conventional WL and NIR imaging with ProSense750 were calculated; results of this experiment are summarized in Table 1. Under standard WL imaging, a total of 18 true positives, 25 true negatives, 1 false positive, and 8 false negatives were found; from these data, the sensitivity was calculated to be 69%, and the specificity was calculated to be 96%. Under NIR imaging 24 hours after ProSense750 administration, a total of 26 true positives, 23 true negatives, 3 false positives, and 0 false negatives were found; the sensitivity was calculated to be 100%, and the specificity was calculated to be 88%. Overall, a total of 8 tumor foci were identified in the NIR channel that were not detected under WL. All 3 of the false positives for NIR imaging represented enlarged, reactive lymph nodes with no evidence by histology of tumor infiltration but with signs of follicular activation.

Table 1
Statistical analysis of ex vivo WL and NIR imaging results from the peritoneal cancer animal model experiment. Data represents a summary of the results collected from WL and NIR imaging, as well as histologic validation, performed on 52 explanted tissue ...

In vivo laparoscopic imaging of focal peritoneal cancer with quantitative fluorescence endoscopy

Animals with focally implanted OVCAR-3 tumors were imaged using a custom built imaging system; all animals were found to have focal lesions 4 weeks after implantation. Figure 4A shows WL, raw NIR, and corrected NIR frames from an in vivo imaging video of a peritoneal tumor at two distances for one representative animal. While the raw NIR signal intensity increases greatly as the catheter tip approaches the lesion, the corrected NIR stays relatively invariant across this distance. This effect is quantified in Figure 4B, where it is shown that the percent change in mean fluorescence collected from a tumor focus in a representative animal between the two distances is 89% for the raw NIR but only 5.3% for the corrected image. A supplemental video of a representative in vivo laparoscopy demonstrates this correction in real time.

Fig. 4
In vivo laparoscopic imaging of focal peritoneal cancer in a representative animal using quantitative fluorescence endoscopy, 24 hours following administration of 2 nmol ProSense750. A, WL, raw NIR fluorescence, and quantitative NIR fluorescence at two ...


Optical imaging with molecularly targeted probes has numerous advantages compared to other detection methods for evaluation of potential residual disease. The approach focally highlights disease, lacks ionizing radiation, and has a low barrier to incorporation in the operating room. Near infrared protease imaging in particular offers a number of distinct advantages in the detection of ovarian cancer peritoneal metastases, by targeting an enzyme class widely present across most ovarian cancers[12]. In this study, we have conducted a direct comparison between conventional white-light imaging and NIR imaging with a protease sensitive imaging probe for ovarian tumor foci identification, confirmed with histological validation. We have demonstrated via a comprehensive ex vivo survey of abdominal and pelvic organs excised from a peritoneal carcinomatosis animal model that areas of metastatic disease are vividly fluorescent in the NIR following administration of ProSense750. We have also evaluated the tumor to background signal intensity ratios for WL and NIR imaging for a number of explanted tissues and have shown a marked improvement with NIR imaging over the near 1:1 ratio for conventional WL methods for all tissues examined.

Beyond improving the conspicuity of metastatic lesions that are otherwise visible by WL imaging, NIR imaging with the use of a protease sensitive probe has the ability to identify cancer foci that were not apparent under WL investigation. The majority of these foci were missed by WL imaging because of their small size and because of their visual and textural similarity to the surrounding parenchyma; a number were missed as well because they were deep to normal tissue. The discernment of such lesions is of major clinical relevance because though they are small, they contribute to the overall residual tumor burden; moreover, they can be readily removed by techniques such as argon laser coagulation, as well as other approaches currently practiced during cytoreductive surgery to remove tumor studding [16].

The results of our ex vivo imaging survey of abdominal and pelvic organs for metastases using WL and NIR imaging revealed sensitivity and specificity values of 69% and 96%, respectively, for WL and 100% and 88%, respectively, for NIR. NIR imaging is acquired simultaneously with WL imaging and the two provide complementary information. In addition to the protease map created by the NIR imaging, one can use standard anatomic cues for additional guidance. When an area of abnormal anatomy is detected by WL imaging, the area most likely represents cancer. However, WL imaging alone missed a significant number of tumor foci, leading to a less than ideal sensitivity; on the other hand, the sensitivity of NIR imaging for tumor foci was 100%. Sensitivity is the pertinent statistic during cytoreductive surgery: it is far more important and beneficial to the patient to be able to remove all tumor along with a few benign nodules than to conservatively remove only those lesions that are certainly malignant and risk leaving tumor within the abdomen. Moreover, the false positives by NIR imaging were only in reactive lymph nodes, and not other foci within the peritoneum.

This study was designed in a manner to directly evaluate the clinical relevancy of optical molecular imaging in ovarian cancer by comparing in a side-by-side fashion the current standard of care WL imaging with NIR imaging using ProSense750. The results of the ex vivo imaging experiments indicate that this imaging modality is readily applicable to and a significant improvement upon the conventional clinical approach. However, a barrier to entry for optical imaging into the operating room is the dependence of fluorescence signal intensity upon the distance between the target tissue and the endoscope or detection camera. During a surgical procedure, as the surgeon manipulates instruments and the organs under investigation move, this distance is in constant flux. It is therefore difficult to determine in such a dynamic environment what the “true” fluorescence value of a particular area of tissue is, a drawback that constrains the utility of optical imaging in an intraoperative setting. We have recently designed and built a real-time optical imaging system that corrects for NIR fluorescence distance dependence [14, 15]. We have shown in this study that the system, when applied to the imaging of focal peritoneal cancer following the administration of ProSense750, is able to quantitate NIR signal regardless of positional changes in the imaging catheter. These data suggest that the correction algorithm implemented by this system overcomes a major hurdle impeding the translatability of optical imaging for intraoperative applications, especially with respect to minimally invasive approaches.

We envision the role of optical molecular imaging in ovarian cancer to be in the improved detection of metastatic disease in an intraoperative setting. We see the concept of “gross residual disease” to be a moving target, and that with better detection techniques, tumor will be better visualized and thus more thoroughly eradicated from the abdomen. An analogy can be drawn to the realm of colorectal cancer, where miss rates for polyp detection have been estimated to be as high as 20%[17], based on tandem colonoscopy studies; an improved method to detect these tumors, such as with molecular optical imaging, would significantly improve upon our ability to visualize and remove the lesions.

Another possible application for this technology is in the augmentation of second-look laparoscopy. The addition of optical molecular imaging has the potential to significantly increase the utility of surgical re-exploration in ovarian cancer. Fluorescence-based protease imaging would enhance the ability to detect residual or recurrent disease, and the associated improvement in tumor debulking may positively impact long-term outcomes. Moreover, the quantification of tumor protease activity will strengthen the evaluation of response to chemotherapy: a decrease in protease activity in the second-look procedure compared to the initial cytoreduction may help in assessing the efficacy of the chemotherapy regimen and in the planning of subsequent treatment cycles. Finally, quantification of tumor protease activity may additionally provide further prognostic information by informing upon the malignant potential of the visualized tumor.

The methodology presented here advances the possibility of incorporation of additional imaging practices to improve the detection and management of ovarian cancer. Optical imaging with protease-activatable NIR probes, combined with the development of tools that allow for the intravital quantitation of fluorescent signal may yield improved outcomes through decreased post operative tumor burden as a result of enhanced visualization of residual disease. The approach represents an important step towards the integration of molecular optical imaging in the operating room. Furthermore, the molecular beacon effect of the protease activatable probes has the potential to improve upon the diagnostic and interventional utility of second-look laparoscopies when monitoring disease regression or recurrence.

Supplementary Material


This research was supported in part by NIH grant R01-EB001872 and U24-CA92782.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest Statement

RW is a shareholder and member of the Board of Directors for VisEn Medical. All other authors have no conflicts of interest to declare.


Quantitative optical molecular imaging with protease activatable probes significantly improves tumor detection in metastatic ovarian cancer.

Supplementary data In vivo video of laparoscopic imaging of a peritoneal cancer focus, 24 hours post-administration of 2nmol of ProSense750, demonstrating the correction of the distance dependence of NIR signal in real-time using the in-house built catheter-based imaging system.


1. Tingulstad S, Skjeldestad FE, Halvorsen TB, Hagen B. Survival and prognostic factors in patients with ovarian cancer. Obstet Gynecol. 2003;101:885–91. [PubMed]
2. Jemal A, Siegel R, Ward E, Murray T, Xu J, Thun MJ. Cancer statistics, 2007. CA: a Cancer Journal for Clinicians. 2007;57:43–66. [PubMed]
3. Olson SH, Mignone L, Nakraseive C, Caputo TA, Barakat RR, Harlap S. Symptoms of ovarian cancer. Obstet Gynecol. 2001;98:212–7. [PubMed]
4. Chi DS, Eisenhauer EL, Lang J, Huh J, Haddad L, Abu-Rustum NR, Sonoda Y, Levine DA, Hensley M, Barakat RR. What is the optimal goal of primary cytoreductive surgery for bulky stage IIIC epithelial ovarian carcinoma (EOC)? Gynecol Oncol. 2006;103:559–64. [PubMed]
5. Figueiredo JL, Alencar H, Weissleder R, Mahmood U. Near infrared thoracoscopy of tumoral protease activity for improved detection of peripheral lung cancer. Int J Cancer. 2006;118:2672–7. [PubMed]
6. Funovics MA, Alencar H, Su HS, Khazaie K, Weissleder R, Mahmood U. Miniaturized multichannel near infrared endoscope for mouse imaging. Mol Imaging. 2003;2:350–7. [PubMed]
7. Funovics MA, Weissleder R, Mahmood U. Catheter-based in vivo imaging of enzyme activity and gene expression: feasibility study in mice. Radiology. 2004;231:659–66. [PubMed]
8. Hama Y, Urano Y, Koyama Y, Choyke PL, Kobayashi H. Activatable fluorescent molecular imaging of peritoneal metastases following pretargeting with a biotinylated monoclonal antibody. Cancer Res. 2007;67:3809–17. [PubMed]
9. Hama Y, Urano Y, Koyama Y, Gunn AJ, Choyke PL, Kobayashi H. A self-quenched galactosamine-serum albumin-rhodamineX conjugate: a “smart” fluorescent molecular imaging probe synthesized with clinically applicable material for detecting peritoneal ovarian cancer metastases. Clin Cancer Res. 2007;13:6335–43. [PubMed]
10. Tung CH, Mahmood U, Bredow S, Weissleder R. In vivo imaging of proteolytic enzyme activity using a novel molecular reporter. Cancer Res. 2000;60:4953–8. [PubMed]
11. Weissleder R, Tung CH, Mahmood U, Bogdanov A., Jr. In vivo imaging of tumors with protease-activated near-infrared fluorescent probes. Nat Biotechnol. 1999;17:375–8. [PubMed]
12. Nishikawa H, Ozaki Y, Nakanishi T, Blomgren K, Tada T, Arakawa A, Suzumori K. The role of cathepsin B and cystatin C in the mechanisms of invasion by ovarian cancer. Gynecol Oncol. 2004;92:881–6. [PubMed]
13. Mahmood U, Weissleder R. Near-infrared optical imaging of proteases in cancer. Mol Cancer Ther. 2003;2:489–96. [PubMed]
14. Upadhyay R, Sheth RA, Weissleder R, Mahmood U. Quantitative real-time catheter-based fluorescence molecular imaging in mice. Radiology. 2007;245:523–31. [PubMed]
15. Sheth RA, Upadhyay R, Weissleder R, Mahmood U. Real-time multichannel imaging framework for endoscopy, catheters, and fixed geometry intraoperative systems. Mol Imaging. 2007;6:147–55. [PubMed]
16. Fader AN, Rose PG. Role of surgery in ovarian carcinoma. J Clin Oncol. 2007;25:2873–83. [PubMed]
17. van Rijn JC, Reitsma JB, Stoker J, Bossuyt PM, van Deventer SJ, Dekker E. Polyp miss rate determined by tandem colonoscopy: a systematic review. Am J Gastroenterol. 2006;101:343–50. [PubMed]