PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of springeropenLink to Publisher's site
Annals of Nuclear Medicine
 
Ann Nucl Med. 2017; 31(7): 521–528.
Published online 2017 April 29. doi:  10.1007/s12149-017-1175-2
PMCID: PMC5517559

Diagnostic performance of a computer-assisted diagnosis system for bone scintigraphy of newly developed skeletal metastasis in prostate cancer patients: search for low-sensitivity subgroups

Abstract

Purpose

The computer-assisted diagnostic system for bone scintigraphy (BS) BONENAVI is used to evaluate skeletal metastasis. We investigated its diagnostic performance in prostate cancer patients with and without skeletal metastasis and searched for the problems.

Methods

An artificial neural network (ANN) value was calculated in 226 prostate cancer patients (124 with skeletal metastasis and 101 without) using BS. Receiver operating characteristic curve analysis was performed and the sensitivity and specificity determined (cutoff ANN = 0.5). Patient’s situation at the time of diagnosis of skeletal metastasis, computed tomography (CT) type, extent of disease (EOD), and BS uptake grade were analyzed. False-negative and false-positive results were recorded.

Results

BONENAVI showed 82% (102/124) of sensitivity and 83% (84/101) specificity for metastasis detection. There were no significant differences among CT types, although low EOD and faint BS uptake were associated with low ANN values and low sensitivity. Patients showed lower sensitivity during the follow-up period than staging work-up. False-negative lesions were often located in the pelvis or adjacent to it. They comprised not only solitary, faint BS lesions but also overlaying to urinary excretion.

Conclusions

BONENAVI with BS has good sensitivity and specificity for detecting prostate cancer’s osseous metastasis. Low EOD and faint BS uptake are associated with low sensitivity but not the CT type. Prostate cancer patients likely to have false-negative results during the follow-up period had a solitary lesion in the pelvis with faint BS uptake or lesions overlaying to urinary excretion.

Keywords: Computer-assisted diagnosis, Bone scintigraphy, Bone metastasis, Prostate cancer

Introduction

Prostate cancer is the most common solid cancer in men in the US [1] and the third most common in Japan, and its incidence is increasing [2]. Bone is the most common site of distant metastases of prostate cancer [3]. These skeletal metastases are usually osteoblastic in contrast to those from other cancers, which are mostly osteolytic [4]. Therefore, bone scintigraphy (BS) is used to diagnose skeletal metastasis from prostate cancer. To date, BS has been interpreted visually. Therefore, there was a need for an appropriate quantitative approach for BS, and computer-assisted diagnostic (CAD) software for BS was developed and evaluated [48]. A Japanese version of this software is BONENAVI. We formally published the development and clinical evaluation of a revised version, called BONENAVI II [9]. As above mentioned, skeletal metastases of prostate cancer are mostly osteoblastic, other types of skeletal metastasis exist. Our clinical questions concerned the performance of BONENAVI with the various types of skeletal metastasis and what factors, if any, caused the failures (false-negative and false-positive results) to diagnose skeletal metastases from prostate cancer.

We conducted this study to clarify the diagnostic performance of BONENAVI for detecting newly developed skeletal metastases from prostate cancer as an adjunct to the clinical information, BS results, and computed tomography (CT) appearance and then analyzed the factors as to how they related to the diagnostic failures. Identifying the factors and the reasons they produce a low artificial neural network (ANN) value and low sensitivity may help improve the performance of CAD software.

Materials and methods

Patients

Between February 2013 and January 2017, consecutive patients with histologically proven prostate cancer and who showed typical or suspected skeletal metastases on BS were enrolled. These patients were reviewed regarding the presence of skeletal metastasis, and the both BS and CT scan of the osseous metastatic sites were performed at the time of the first diagnosis of skeletal metastasis, which was established by agreement of at least two different modalities, follow-up studies, and/or biopsy.

Eligible patients had a diagnosis of skeletal metastasis and the interval between BS and CT scan was within a month. These patients were then divided into the following diagnostic groups depending on the time of staging and the follow-up period after initial therapy. The follow-up period was further divided into non-castration-resistant prostate cancer (non-CRPC) or castration-resistant prostate cancer (CRPC). The criterion for CRPC was a testosterone level of below 50 ng/dl.

Consecutive prostate cancer patients who showed negative for skeletal metastasis on BS were enrolled between January and April 2013. These patients were followed up for at least 3 years and were confirmed negative for skeletal metastasis.

This retrospective study was approved by the local institutional review board.

BS and BONENAVI analysis

BS was performed about 3 h after the injection of 99mTc-MDP. After obtaining anterior and posterior whole-body scans, local images including the pelvic axial view were acquired or, in some patients, bone SPECT/CT was performed as follow-up to the whole-body images. Patients in whom BS revealed metastasis were further analyzed. The bone metastatic burden of these patients, as shown on BS, was scored using Soloway’s extent of disease (EOD) system [9]. Briefly, EOD scores were decided on the basis of number and extent of metastases. The bone scans were divided into 5 EOD grades, 0 normal or abnormal due to benign bone disease, I number of bony metastases less than 6, each of which is less than 50% of vertebral body (one lesion about the size of a vertebral body would be counted as two lesions), II number of bone metastases between 6 and 20, size of lesions described above. III number of metastases more than 20 but less than a ‘super scan’, and IV ‘super scan’ or its equivalent, i.e., more than 75% of ribs, vertebrae, and pelvic bones. The intensity of skeletal metastatic uptake was visually graded as faint, usual, or intense on whole-body images. Figure 1 shows BS uptake patterns; faint uptake was visualized as a weak hot spot, whereas intense uptake was very strong uptake, which could be seen in both anterior and posterior views. Usual uptake was visualized somewhere between faint and intense uptake. This visual classification was independently performed by two radiologists-nuclear physicians, and 15 discordant cases were decided through discussion.

Fig. 1
Bone scintigraphy uptake patterns are shown. Faint lesion uptake is shown on the right iliac wing (a), usual lesion uptake is shown on the left iliosacral site (b), and intense lesion uptake is shown on the iliosacral site (c). Faint uptake a is equal ...

BONENAVI analysis was performed as previously reported [10]. Briefly, the CAD system “BONENAVI version 2.1.7” (FUJIFILM RI Pharma Co., Ltd., Tokyo, Japan) was used to analyze BS scans using anterior and posterior whole-body images. This BONENAVI system shows two imaging markers: ANN and a bone scan index (BSI). The ANN value indicates the probability of having skeletal metastasis. The range of ANN is 0–1, where “0” means no possibility of skeletal metastasis, and “1” means there is a high suspicion of having skeletal metastasis. The BSI indicates the tumor metastatic burden (proportion of bone metastatic area to whole body skeleton). In the present study, the ANN values were used.

CT scans

CT scans were performed with a 2-mm thickness at 5-mm intervals.

The CT morphological classification of skeletal metastasis included the categories osteoblastic, osteolytic, mixed osteoblastic and osteolytic, and intertrabecular (invisible on CT). Figure 2 shows the morphologic lesion patterns on CT scans. This classification was independently performed by two radiologists-nuclear physicians, and discordant cases were decided through discussion. The confirmation of skeletal metastasis of intertrabecular or invisible on CT was performed by follow-up CT studies. All invisible lesions turned to osteoblastic or mixed CT appearance on later CT studies.

Fig. 2
Representative examples of the various computed tomography (CT) morphologies: a osteoblastic pattern (left ilium); b osteolytic pattern (left ilium at the iliosacral site); c mixed osteolysis and osteoblastic pattern (lumbar vertebra); and d invisible ...

Data analysis and statistical methods

The patient-base was evaluated as follows:

  1. Receiver operating characteristics (ROC) curve analysis was performed using the patient’s ANN value in all patients.

Then, sensitivity and specificity were calculated with the cutoff level set at ANN = 0.5 for both skeletal metastasis positive and negative patients. The cutoff level was set at ANN = 0.5 throughout the study according to other Japanese studies [7, 8].

  • 2.
    The sensitivity was also calculated as a function of the diagnostic setting (staging period, follow-up). Patients during the follow-up period were further divided into non-CRPC and CRPC groups. For the staging and follow-up (non-CRPC and CRPC) periods, we applied one-way analysis of variance (ANOVA) test followed by post hoc analysis using ANN values. The contingency table analysis was carried out using cutoff at ANN = 0.5.

Similar analyses were performed in regard to CT appearances, that is, osteoblastic, osteolytic, mixed, and intertrabecular (invisible) types, EOD scores (EOD = I, II, III, and IV), and BS uptake intensities (faint, usual, and intense).

  • 3.
    False-negative and false-positive lesions were analyzed. Analysis of false-negative lesions was performed in patients with skeletal metastasis and ANN ≤ 0.5 value. False-negative patients were further divided into two groups of ANN 0–0.25, and 0.26–0.5 [7, 8]. Analysis of false-positive lesions was performed in patients with non-skeletal metastasis and ANN > 0.5. False-positive patients were also divided into two groups of ANN 0.51–0.75 and 0.76–1.00.

Statistical analysis software (SPSS version 24; IBM, Armonk, NY, USA) was used. A value of p < 0.05 was considered to indicate statistically significance. For ANOVA, Levene’s test for the homogeneity of variance was performed first. When the homogeneity of variance was assumed, a t test was followed by Turkey’s test. When the homogeneity was not assumed, Welch’s test was followed by the Games-Howell test. For contingency table analysis, if the more than 20% of cell boxes was expected numbered <5, Fisher’s exact test was used. Pearson’s Chi-square test was applied to the others.

Results

This study included 226 patients with prostate cancer: 124 with skeletal metastasis and 102 without.

ROC analysis with sensitivity and specificity

The ROC curve is shown in Fig. 3. The area under the curve was 0.888 (95% confidence interval 0.843–0.932). The mean ANN values were 0.78 (SD = 0.29) for the skeletal metastasis patients and 0.22 (SD = 0.30) for the non-skeletal metastasis patients. Sensitivity was 82% (102/124), and specificity was 83% (84/101).

Fig. 3
Receiver operating characteristics curve

Subgroups analysis

Tables 1, ,2,2, ,33 and and44 summarized the ANN values with ANOVA analyses and the contingency tables analyses using ANN cutoff value = 0.5 regarding diagnostic situation, CT appearance, EOD and BS uptake grade.

Table 1
ANN in skeletal metastasis among diagnostic situation
Table 2
ANN among CT appearance of skeletal metastasis
Table 3
ANN among EOD score of skeletal metastasis
Table 4
ANN among BS uptake grade of skeletal metastasis

As shown in Table 1, skeletal metastasis at follow-up period showed lower ANN values and lower sensitivity than those at staging with statistically significance.

Mixed type of CT appearance osseous metastasis showed high mean ANN value (mean ± SD, 0.90 ± 0.16) and high sensitivity (95%). Other types of CT appearance showed similar results (both ANN values and sensitivity; Table 2).

There was no statistical difference among the types of CT appearance.

Analysis of EOD differences showed that EOD = I patients had lower mean ANN value and lower sensitivity compared with EOD = II, III, IV patients. The ANN values and the sensitivity between EOD = I and EOD II-IV differed statistical significantly (Table 3).

The ANN value of for those with faint BS uptake was significantly lower than those for patients with usual or intense BS uptake. The sensitivity was also significantly lower in the patients with faint BS uptake group compared with that in those with usual or intense uptake (Table 4).

Analysis of false-negative cases

Table 5 shows the 22 patients with false-negative results (ANN ≤ 0.5). Sub-groups of false-negative results were no suspicion (ANN 0–0.25) and less suspicion (ANN 0.26–0.5) according to Horikoshi [7]. Ten skeletal metastasis patients with no suspicion group (ANN 0–0.25) showed ANN = 0. All patients had a small bone tumor burden (EOD = I); most (9/10) had a solitary skeletal metastasis; most (9/10) were diagnosed during the follow-up period; and the BS uptake was generally faint (8/10). The 8 of 10 patients had lesions in pelvic bone: 2 in the ilium, 3 in the ischium, 1 in the sacrum, and 2 in the pubis. In 12 patients with less suspicion of skeletal metastasis (ANN 0.26–0.50), most patients (11/12) had small tumor burden (EOD = I). The BS uptake was higher (faint = 8, usual = 2, and intense = 2) than those of no suspicion group. Seven patients (7/12) had lesions in pelvic bone.

Table 5
False negative patients list

Analysis of false-positive cases

Table 6 shows the 17 patients with ANN > 0.5. These patients were divided into seven patients of highly suspicion group (ANN 0.76–1.0) and 10 of suspicion group (ANN 0.5–0.74). These patients did not have skeletal metastasis, although BONENAVI analysis had indicated there was a possibility of skeletal metastasis. Most of the patients showed false-positive hot (red) spots on the vertebral column (16/17). Even though the BS hot spots existed at those sites, none of these patients were judged to have skeletal metastasis based on an experienced nuclear physician diagnosing those areas of uptake as degenerative changes. At their follow-up evaluations at least 3 years later, none of the patients had developed skeletal metastasis.

Table 6
False positive patients list

Discussion

BS has a high sensitivity for diagnosing skeletal metastasis and is an effective technique for the whole-body skeletal examination. However, interpretation of BS is performed visually. To address this problem, a quantitative or automatic approach was developed in the form of software using artificial intelligence to compute an ANN. This software offers the possibility of detecting skeletal metastasis. In addition, a BSI, using the bone tumor burden/whole skeleton ratio, was developed and used to diagnose bone metastasis and its follow-up [47]. This CAD software, commercially available as EXINI bone (EXINI Diagnostics, Lund, Sweden), is capable of learning to detect metastatic lesions on whole-body scans. The technology is able to identify hotspots, adjust whole-body intensity and quantify hotspot intensity, and classify hot spot lesions.

The Japanese version of this software, BONENAVI (FUJIFILM RI Pharma, Tokyo, Japan), was modified using a cohort of Japanese patients [8, 9], and revised [11]. In the present study, we used ANN obtained using BONENAVI version 2.1.7. to analyze diagnostic accuracy. Another index, BSI, shows the proportion of bone tumor in regard to the whole skeleton. The BSI has been reported to evaluate the therapeutic effect of various therapies on bone metastasis [1214].

Skeletal metastasis from prostate cancer is usually osteoblastic, unlike other cancers, which are mostly osteolytic [15]. This study showed that 67% (83/124) of the lesions were osteoblastic in nature at the initial diagnosis: osteolytic type 6% (7/124); mixed type 15% (19/124); and invisible type 12% (15/124). That is, 82% (102/124) of patients (blastic + mixed) have an osteoblastic component on CT at the initial diagnosis of skeletal metastasis. From another viewpoint, 21% (26/124) of skeletal metastases have a lytic component (lytic + mixed). The importance of the lytic component is obvious in that anti-resorption medicines, bisphosphonate and anti-RANKL monoclonal antibody, are used to treat skeletal metastases of prostate cancer [16, 17]. We formerly reported that both formation and resorption markers increased in prostate cancer patients with skeletal metastasis [18].

The radiopharmaceutical used for BS accumulates at the bone-forming portion of the tumor, not at the tumor itself, and BS is weak to detect lytic metastases. Therefore, we thought that there might be some difference in the BONENAVI sensitivity among CT types. However, this study showed that the sensitivity of the lytic type did not have different ANN values (no statistically significant difference), and the sensitivity was only slightly low 71% (5/7), without statistical significance (Table 2). These findings mean that the ANN value and sensitivity of BONENAVI are robust for the various CT types. We formerly reported that the ANN and sensitivity of BONENAVI were not different among the CT types in breast cancer or lung cancer patients with skeletal metastasis [9].

Patients with EOD = I patients and those with BS weak uptake showed low ANN values, and the rate of identifying false-negative lesions was significantly high (Tables 3, ,4).4). These findings indicated that limited spread and weak uptake were associated with low ANN values and a high rate of false-negative results. That is, it is difficult to detect the early phase of skeletal metastasis using BONENAVI.

Patients showed lower ANN values and a higher false-negative rate during the follow-up period than staging period (Table 1). Because the patients who completed the initial therapy were followed up carefully, especially with PSA measurement, early stage of skeletal metastasis was supposed to be diagnosed by BS–hence the high false-negative rate at follow-up.

Moreover, analysis of false-negative patients with ANN = 0 revealed that patients who had a solitary bone lesion in the pelvis (or adjacent to it) often had ANN = 0. Putative reasons were not only being solitary and/or weak uptake but also an overlay of urinary (ureteral and bladder) activity even though the uptake is usual.

Therefore, we routinely add a pelvic axial view or pelvic SPECT/CT to avoid this overlay problem. Because BONENAVI analysis employs the anterior and posterior whole-body images, urinary activity overlay seems to be difficult to overcome using the present methods. Our study also tried to determine the reason for the false-positive results. Patients with degenerative changes of the vertebra (mostly multiple) showed false-positive results.

In conclusion, prostate cancer patients with and without skeletal metastasis were assessed by BONENAVI (version 2.1.7) for BS. The sensitivity and specificity were 82% (102/124) and 83% (84/101), respectively. The sensitivities were not different among CT types, although they were statistically significantly different among EOD scores, BS uptake, and diagnostic situations. Patients with EOD = I, faint BS uptake, and low ANN values also had low sensitivity during the follow-up period. Patients with a solitary pelvic osseous metastasis with faint BS uptake or adjacent to the urinary system may show false-negative BONENAVI results.

Acknowledgements

We thank to Ms. Hiromi Ito for editing this manuscript.

Compliance with ethical standards

Compliance with ethical standards

Conflict of interest

The authors declare that we have no conflict of interest.

References

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2015. CA Cancer J Clin. 2015;65:5–29. doi: 10.3322/caac.21254. [PubMed] [Cross Ref]
2. Cancer statistics in Japan. Cancer Information Service. 2013. http://ganjoho.jp.
3. Hess KR, Varadhachary GR, Taylor SH, Wei W, Raber MN, Lenzi R, et al. Metastatic patterns in adenocarcinoma. Cancer. 2006;106:1624–1633. doi: 10.1002/cncr.21778. [PubMed] [Cross Ref]
4. Imbriaco M, Larson SM, Yeung HW, Mawlawi OR, Edi Y, Venkatraman ES, et al. A new parameter for measuring metastatic bone involvement by prostate cancer: the bone scan index. Clin Cancer Res. 1998;4:1765–1772. [PubMed]
5. Sadik M, Jakobsson D, Olofsson F, Ohlsson M, Suurkula M, Edenbrandt L. A new computer-based decision-support system for the interpretation of bone scans. Nucl Med Commun. 2006;27:417–423. doi: 10.1097/00006231-200605000-00002. [PubMed] [Cross Ref]
6. Sadik M, Hamadeh I, Nordblom P, Suurkula M, Hoglund P, Ohlsson M, et al. Computer-assisted interpretation of planar whole-body bone scans. J Nucl Med. 2008;49:1958–1965. doi: 10.2967/jnumed.108.055061. [PubMed] [Cross Ref]
7. Horikoshi H, Kikuchi A, Onoguchi M, Sjostrand K, Edenbrandt L. Computer-aided diagnosis system for bone scintigrams from Japanese patients: importance of training database. Ann Nucl Med. 2012;26:622–626. doi: 10.1007/s12149-012-0620-5. [PMC free article] [PubMed] [Cross Ref]
8. Nakajima K, Nakajima Y, Horikoshi H, Ueno M, Wakabayashi H, Shiga T, et al. Enhanced diagnostic accuracy for quantitative bone scan using an artificial neural network system: a Japanese multi-center database project. Eur J Nucl Med Mol Imaging Res. 2013;26:83. [PMC free article] [PubMed]
9. Koizumi M, Wagatsuma K, Miyaji N, Murata T, Miwa K, Takiguchi T, et al. Evaluation of computer-assisted diagnosis system, BONENAVI version 2, for bone scintigraphy in cancer patients in routine clinical setting. Ann Nucl Med. 2015;29:138–142. doi: 10.1007/s12149-014-0921-y. [PubMed] [Cross Ref]
10. Soloway MS, Hardeman SW, Hickey D, Raymond J, Todd B, Soloway S, et al. Stratification of patients with metastatic prostate cancer based on extent of disease on initial bone scan. Cancer. 1988;61:195–202. doi: 10.1002/1097-0142(19880101)61:1<195::AID-CNCR2820610133>3.0.CO;2-Y. [PubMed] [Cross Ref]
11. Koizumi M, Miyaji N, Murata T, Motegi K, Miwa K, Koyama M, et al. Evaluation of a revised version of computer-assisted diagnosis system, BONENAVI version 2.1.7, for bone scintigraphy in cancer patients. Ann Nucl Med. 2015;29:659–665. doi: 10.1007/s12149-015-0988-0. [PubMed] [Cross Ref]
12. Dennis ER, Jia X, Mezheritskiy IS, Stephenson RD, Schoder H, Fox JJ, et al. Bone scan index: a quantitative treatment response biomarker for castration-resistant metastatic prostate cancer. J Clin Oncol. 2012;30:519–524. doi: 10.1200/JCO.2011.36.5791. [PMC free article] [PubMed] [Cross Ref]
13. Mitsui Y, Shiina H, Yamamoto Y, Haramoto M, Arichi N, Yasumoto H, et al. Prediction of survival benefit using an automated bone scan index in patients with castration-resistant prostate cancer. BJU int. 2012;110:628–634. doi: 10.1111/j.1464-410X.2012.11355.x. [PubMed] [Cross Ref]
14. Alva A, Nordquist L, Daignault S, George S, Ramos J, Albany C, et al. Clinical correlates of benefit from radium-223 therapy in metastatic castration resistant prostate cancer. Prostate. 2017;77:479–488. doi: 10.1002/pros.23286. [PubMed] [Cross Ref]
15. Resnick D, Kyriakos M, Greenway G (2005) Skeletal metastasis, bone and joint imaging, 3rd edn. In: Resnick D, Kransdorf MJ (eds) Chapter 72. Philadelphia: Elsevier Saunders; 2005. pp. 1245–64.
16. Saad F, Gleason DM, Murray R, Tchekmedyian S, Venner P, Lacombe L, et al. A randomized, placebo-controlled trial of zoledronic acid in patients with hormone-refractory metastatic prostate carcinoma. J Nat Cancer Inst. 2002;94:1458–1468. doi: 10.1093/jnci/94.19.1458. [PubMed] [Cross Ref]
17. Fizazi K, Carducci M, Smith M, Damiao R, Brown J, Karsh L, et al. Denosumab versus zoledronic acid for treatment of bone metastases in men with castration-resistant prostate cancer: a randomized, double-blind study. Lancet. 2011;377:813–822. doi: 10.1016/S0140-6736(10)62344-6. [PMC free article] [PubMed] [Cross Ref]
18. Maeda H, Koizumi M, Yoshimura K, Yamauchi T, Kawai T, Ogata E. Correlation between bone metabolic markers and bone scan in prostatic cancer. J Urol. 1997;157:539–543. doi: 10.1016/S0022-5347(01)65196-6. [PubMed] [Cross Ref]

Articles from Springer Open Choice are provided here courtesy of Springer