|Home | About | Journals | Submit | Contact Us | Français|
The aim of this study was to evaluate the effectiveness of computer-aided detection (CAD) to mark the cancer on digital mammograms at the time of breast cancer diagnosis and also review retrospectively whether CAD marked the cancer if visible on any available prior mammograms, thus potentially identifying breast cancer at an earlier stage. We sought to determine why breast lesions may or may not be marked by CAD. In particular, we analyzed factors such as breast density, mammographic views, and lesion characteristics.
Retrospective review from 2004 to 2008 revealed 3445 diagnosed breast cancers in both symptomatic and asymptomatic patients; 1293 of these were imaged with full field digital mammography (FFDM). After cancer diagnosis, in a retrospective review held by the radiologist staff, 43 of these cancers were found to be visible on prior-year mammograms (false-negative cases); these breast cancer cases are the basis of this analysis. All cases had CAD evaluation available at the time of cancer diagnosis and on prior mammography studies. Data collected included patient demographics, breast density, palpability, lesion type, mammographic size, CAD marks on current- and prior-year mammograms, needle biopsy method, pathology results (core needle and/or surgical), surgery type, and lesion size.
On retrospective review of the mammograms by the staff radiologists, 43 cancers were discovered to be visible on prior-year mammograms. All 43 cancers were masses (mass classification included mass, mass with calcification, and mass with architectural distortion); no pure microcalcifications were identified in this cohort. Mammograms with CAD applied at the time of breast cancer diagnosis were able to detect 79% (34/43) of the cases and 56% (24/43) from mammograms with CAD applied during prior year(s). In heterogeneously dense/extremely dense tissue, CAD marked 79% (27/34) on mammograms taken at the time of diagnosis and 56% (19/34) on mammograms with CAD applied during the prior year(s). At time of diagnosis, CAD marked lesions in 32% (11/34) on the craniocaudal (CC) view, 21% (7/34) on the mediolateral oblique (MLO) view. Lesion size of those marked by CAD or not marked were similar, the average being 15 and 12 mm, respectively.
CAD marked cancers on mammograms at the time of diagnosis in 79% of the cases and in 56% of the cases from the mammograms with CAD applied in the prior year(s). Our review demonstrated that CAD can mark invasive breast carcinomas in even dense breast tissue. CAD marked a significant portion on the CC view only, which may be an indicator to radiologists to be especially vigilant when a lesion is marked on this view.
Screening mammography reduces breast cancer mortality by approximately 30%. Mammography has a sensitivity of 65-90% in detecting disease,[2–4] but continues to miss cancers. Double-reading mammograms, meaning two radiologists reading all the films, improve radiologist's detection of lesions, increase sensitivity, and reduce false negatives, but increase recalls, and also increase costs. Approximately 5-10% of patients are recalled from screening, with a cancer diagnosis occurring in only 3-10 out of 1000 patients. To minimize false-negative mammograms, computer-aided detection (CAD) was developed. The CAD computer algorithm is designed to work with mammography (film screen and digital) to mark worrisome areas in the breast. CAD can be effective in alerting radiologists to an early stage cancer,[6,7] as incorporating CAD can increase cancer detection by 23%.
Research has demonstrated CAD's ability to detect breast cancer.[1,5–20] Our 2009 review evaluated CAD and found it had the potential to detect cancers in standard projections for all lesion types across a range of breast densities and demonstrated image sensitivity for the mediolateral oblique (MLO) and craniocaudal (CC) views to be 69% and 78%, respectively. Gromet compared CAD and one radiologist to double reading by two radiologists and found that both double reading and single reading with CAD are effective at increasing sensitivity for experienced radiologists. Gilbert et al., demonstrated that single reading with CAD enabled 49.1% cancer detection, compared to 42.6% with double reading.
A benefit of using CAD is the reduction in false negative rates. Our 2004 review found that CAD decreased the false negative rate at double reading from 31% to 19%.
The purpose of our study was to evaluate the effectiveness of CAD in marking cancerous lesions on current digital mammograms (meaning at the time of breast cancer diagnosis) and prior year's mammograms to determine why breast lesions may or may not be marked by CAD. In particular, we analyzed factors such as breast density, mammographic views, and lesion characteristics.
Under Institutional Review Board (IRB) approval, a retrospective review of all diagnosed cancers imaged with full field digital mammography (FFDM) between 2004 and 2008 was conducted at a community-based private breast imaging center. A total of 344,243 patients were seen during this time period; 287,295 were routine for screening (asymptomatic) and 56,948 underwent diagnostic workup (symptomatic). A total of 3445 breast cancers were diagnosed on symptomatic and asymptomatic patients, 1293 of which were imaged with FFDM. To be eligible for inclusion in the study, cancers needed to be visible in retrospect on at least one prior mammogram (false negative), had CAD evaluation available for review at the time of diagnosis and prior year(s), and were imaged with FFDM at the time of diagnosis. Forty-three cases (30 symptomatic/diagnostic and 13 asymptomatic/screening) fulfilled this study criterion. We found while evaluating these 43 cancers that all were mass lesions (mass classification included mass, mass with calcification, and mass with architectural distortion). Patients were imaged on the following FFDM units; 30 Selenia®, Hologic, Bedford, MA; 8 Senographe®, GE, Milwaukee, WI; 5 Fuji CRm, Stamford, CT. All screening mammograms were double read at the time of interpretation. Questionable areas were worked up with additional mammographic views and ultrasound. Biopsies were performed using ultrasound or stereotactic guidance based on how the lesion was best visualized.
At the time a breast cancer diagnosis is made, the radiologist reviews the current digital mammogram along with all available prior mammograms. If the radiologist determines that the cancer was visible on any prior mammograms (false negative), the case is flagged for further review. During a formal review session, all radiologists together review the flagged cancers to confirm if the cancer was truly visible on prior mammograms. For the purposes of this study, all eligible cases were re-reviewed by the study radiologist, along with CAD results, to confirm if the lesion was marked in each respective mammographic view in both the year of diagnosis and any prior years that were available. Prior-year availability consisted of FFDM with CAD or film screen mammography with CAD. When prior years were available, all were reviewed. In most cases, the two prior-year mammograms were reviewed. If the patient did not have a mammogram in the year prior, then the two most recent prior-year mammograms were reviewed. Twenty-two patients had 2 years of prior mammograms for review, 17 patients had one prior-year mammogram for review, and 4 patients had 3 years of prior mammograms for review. When reviewing prior film screen mammograms, the films were digitized and viewed on our Picture Archiving Communication System (PACS) workstation, and the CAD marks were also digitized and visible for viewing.
Data were recorded on patient demographics, lesion demographics (lesion type, mammographic lesion size), presence of CAD markings (for the year of diagnosis and prior year(s), when available), and biopsy data (type of biopsy, pathology results).
False negative was determined by the American College of Radiology (ACR) to mean “diagnosis of cancer within one year of a mammographic examination with normal or probably benign findings” (BI-RADS® category 1, 2, and 3). In addition we also utilized a variant method such as mammographically not evident, below threshold for concern by the radiologist, or oversight for our review. We considered the case to be false negative even if the interval was longer than a year but the lesion was noted to be visible on any prior-year mammogram on retrospective review with a CAD marking over the appropriate area.
In our center, CAD is applied on all mammograms. During the study period, several versions of CAD were used (R2 Technology Image Checker; R2 Technology, Sunnyvale, CA): 5.2.10, 5.3, 5.4, and 8.3. CADX versions 5.2, 8.2, and 8.3 and iCAD (Nashua, New Hampshire) version 7.2 were also used from 2004 to 2006. Images are automatically sent from acquisition to the CAD server, and then the CAD Structured Report (SR) is sent to the PACS where it is displayed on the Soft Copy Review (SCR) workstation. A button push displays and views the images with or without CAD when interpreting the images. For study purposes, a cancer was considered “marked” if the CAD marked the lesion on either view of the current mammogram; the same method was employed with prior mammograms. In regard to prior mammograms, some of which were film screen, R2 Technology versions 5.3, 8.1.19, 8.1, and 8.5 were used. For film screen, the films were scanned through the Image Checker and digitized images were generated and sent to PACS for viewing.
A total of 43 patients with a diagnosis of cancer were discovered to be visible in retrospect on prior screening or diagnostic mammograms. The average patient age was 63 years (range 42 to 89). BI-RADS® breast density distribution consisted of 21% (n = 9) scattered, 35% (n = 15) heterogeneously dense, and 44% (n = 19) extremely dense. There was no breast density entirely composed of fat in this cohort. All were mass lesions, specifically 39 masses, 3 masses with calcium, and 1 mass with architectural distortion. Overall there was an average of 4.8 CAD marks per case. True positive marks averaged at 1.5 CAD marks.
The cancer was marked by CAD in 79% of the cases (n = 34/43) at the time they presented for evaluation: 22 diagnostic and 12 screening [Figure 1]. There was an average of 5.4 CAD marks per case, 3.9 false positive marks, and 1.5 true positive marks. Of the 34 cancers that were marked, 32% (n = 11) were marked on the CC view only, 21% (n = 7) were marked on the MLO view only, and 47% (n = 16) were marked on both views. Average lesion size was 15 mm. A breakdown of CAD marks by lesion type and mammographic view is displayed in Table 1. A majority of the lesions marked by CAD were in dense breasts (27/34, 79%, Table 2). Needle core biopsy was performed for all 34 lesions: 32% (n = 11) stereotactically and 67% (n = 23) under ultrasound guidance. Biopsy revealed 27 invasive ductal carcinoma (IDC), 4 ductal carcinoma in situ (DCIS), 1 invasive lobular carcinoma (ILC), 1 papillary carcinoma, and 1 adenocarcinoma (metastatic colon carcinoma), 88% invasive at surgery. Thirty three (97%) of these patients had surgical excision: 17 lumpectomy, and 16 mastectomy (2 bilateral mastectomy). One reported no surgery. Of this group, 15 lesions were marked by CAD in the current year only. Of these, six had mastectomy, eight lumpectomy, and one reported no surgery.
There were nine lesions (21%) that were not marked by CAD in any mammographic views [Figure 2]. Of these, eight were masses, and one mass with calcifications.
The average size of the lesion was 12 mm. Of these nine lesions, 22% were in scattered breast tissue; 33% in heterogeneously dense tissue, and 44% in extremely dense tissue. Biopsy methods consisted of 11% (n = 1) open surgical biopsy, 11% (n = 1) stereotactic core biopsy, and 77% (n = 7) ultrasound guided core biopsy. Biopsy pathology consisted of 5 IDC, 2 ILC, 1 infiltrating mucinous carcinoma with DCIS, and 1 metastatic malignant melanoma; all were invasive at surgery.
CAD evaluation in the prior year(s) was available for all cases. CAD marked the area of the cancer 56% (n = 24/43) of the time on the mammogram from the prior year(s) [Figure 3]. CAD marked the area of the cancer on the prior study on the CC view (33%, n = 8), on the MLO view (29%, n = 7), or on both views (38%, n = 9). Of the lesions marked in the prior year(s), a majority (80%) were in dense breast tissue, 38% were in heterogeneously dense tissue, and 42% in extremely dense tissue. Average lesion size at the time of diagnosis that CAD marked in the prior-year mammogram was 16 mm. These lesions were not always marked by CAD in the current-year mammogram, the year the radiologists made the diagnosis (year of diagnosis), as the technology is not 100% accurate.
A total of 19 lesions were marked by CAD in both the prior year and the year of diagnosis. Sixteen (16) presented as a mass, 2 were mass with calcifications, and 1 was mass with architectural distortion. Breast density distribution was as follows: 3 in scattered tissue, 8 in heterogeneously dense tissue, and 8 in extremely dense tissue. Average lesion size at the time of diagnosis was 8 mm. At the time of diagnosis, seven lesions were marked on only the CC view, four lesions on only the MLO view, and eight lesions were marked on both views. In the prior-year mammograms, 8 lesions were marked on only the CC view, 5 on only the MLO view, and 6 lesions were marked on both views. Eighteen of the 19 lesions were pathologically proven invasive carcinomas. The one lesion that was not invasive was diagnosed as papillary carcinoma. Of these 19 lesions, 10 (53%) had mastectomy and 9 (47%) lumpectomy.
Studies in recent years have provided much information regarding sensitivity of CAD, as well as their utility. However, other studies such as the studies published by Fenton et al., have provided an opposing argument stating increased work-up rates, increased costs, and increase in radiologists’ time.[22,23] In 2007, Fenton reported that sensitivity increased from 80.4% to 84.0% with new users as well as increased biopsy rate (19.7%). The study found that diagnostic specificity decreased from 90.2% to 87.2% after implementation of CAD, as did the positive predictive value, from 4.1% to 3.2%.This study reported that the change in cancer-detection rate was not significant and concluded that the use of CAD is associated with reduced accuracy of interpretation of screening mammograms. Fenton et al., in 2011, published a follow-up CAD study, again evaluating screen-film mammography. This study concluded that the use of CAD with film screen is associated with decreased specificity and no improvement in the detection rate of invasive cancer, although they did find increased sensitivity for ductal carcinoma in situ. It is important to note that in both of the Fenton analyses, the facilities included were sites that had low volume, as well as radiologists who were inexperienced with using CAD. Both studies concluded that CAD use hinders accuracy of interpretation of screening mammograms. Our findings demonstrate that CAD does have value as it marked the carcinoma at the time of diagnosis 79% of the time, and when reviewing images from the year(s) prior, CAD marked the carcinoma 56% of the time. Burhenne and colleagues reviewed cancers and prior mammograms and found that CAD marked 77% of cases where the cancer was visible in retrospect. Our 2004 review found that CAD correctly marked 71% of actionable findings read as negative in previous years. As Burhenne's study points out, if ample attention is directed to the visible and actionable lesions, these lesions can be addressed at the time of presentation; thus, more cancers may be diagnosed earlier.
Lack of CAD interpretation training or education can often be a reason for overlooking true positive marks. Roehrig wrote that training radiologists in the use of CAD is more important than was originally thought. Traditionally a training session of 1 day has been performed; however, the author states there is evidence supporting the value of longer training. Astley and Gilbert, in a preliminary study, had a 7-week training program in the UK with promising results. Guerriero et al., in 2011, wrote that 4 days of CAD training was required for radiologists and additionally, those using CAD should be retrained every 3 years. Luo et al. showed a statistically significant difference in observer performance in utilizing CAD with mammography interpretation before and after training. The training consisted of three participants who read a pretest set of 80 (mixed) benign and malignant cases and after 4 weeks of training read the posttest set of cases. The authors concluded that CAD training influences perception, recognition, and interpretation of early breast cancer and CAD performance studies. We believe inadequate training may be a factor in some of the negative results reported in the literature that reveal reduction in sensitivity, increase recall rates, and increase biopsy rate. Training, as has been shown, may be valuable to the radiologist to understand the technology and thus the marks. The radiologist may acknowledge CAD marks but ultimately ignore them due to concentration in another area of the breast. The problem with this scenario may be a lack of trust of the marks and/or multiple false-positive marks. Research on interactive CAD technology is ongoing. One interactive method would allow the radiologist to click on an area of concern and only this area would display potential marks. This may provide a way to minimize false–positive marks and reduce the distraction of multiple marks, which can potentially lead to cancers being overlooked.
An interesting observation in our review was that there were no lesions with pure calcifications included in our study cohort. This leads us to believe that we are detecting and diagnosing calcifications. FFDM has helped identify tiny clusters of microcalcifications that may otherwise not be identified on routine screening mammography. Improvement in detecting calcifications has been reported previously.[29–31] A high level of trust exists with CAD marking calcifications, so radiologists are acting on the CAD prompts when it does mark these lesions. This is supported by the literature, which has demonstrated the sensitivity of CAD for detecting malignant microcalcifications to be as high as 99%. Our review of evaluating CAD marking invasive lobular carcinoma found that CAD marked 100% of calcifications. This leads to the question, why are masses more frequently missed? The sensitivity of malignant masses has been reported at 75-89%.[1,33,34] In this review, 59% (n = 27/46) of lesions were marked on just one view. Radiologists may not trust the marks when just in one view; however, this study demonstrates that attention needs to be paid even in this scenario, as we found that CAD marked a more significant portion on the CC view (32%, n = 11/34) compared with 21% (n = 7/24), the MLO view. This is in agreement with our prior publication which found that CAD marked a majority of lesions in the CC view.
This review showed that CAD does have the ability to mark lesions in dense breast tissue. In the year of diagnosis, CAD marked 79% in heterogeneously dense or extremely dense tissue. CAD in prior year(s) marked 80% in heterogeneously dense or extremely dense breasts. Lesion size of those detected and not detected by CAD were very similar, with the average being 14 and 13 mm, respectively. A study by Brem and colleagues also found no significant difference in performance based on cancer size.
A study limitation is that several different versions of CAD were used throughout the study period. This could have led to variations in what CAD did and did not mark. Additionally, some of the prior studies were film screen and having all digital priors would have minimized technical variables, although the priors were all digitized for comparison at the time of diagnosis and for the retrospective review. Additionally, we had a mixture of diagnostic and screening patients at the time of presentation and cancer diagnosis. However, we view our diagnostic and screening patient images and CAD similarly.
In conclusion, this review revealed that CAD marked the breast carcinoma on the mammogram at the time of diagnosis 79% of the time (the radiologists detected 100% of the cancers) and CAD marked the visible cancer 56% of the time, in the prior-year(s) mammograms (the radiologists did not detect). Even though CAD marked the lesion of interest on the prior-year studies, radiologists continue to disregard the CAD marks. As no calcifications were missed in this cohort, we were able to presume that CAD is marking calcifications, and the radiologist is accurately working up these cases. Our review demonstrated that masses are being marked by CAD, even in dense breast tissue, and are invasive carcinomas of significant size. We found that CAD is marking a significant portion of lesions on only the CC view; potentially this is an indicator to radiologists to be especially vigilant when a lesion is marked on this view.
Source of Support: Nil
Conflict of Interest: None declared.
Available FREE in open access from: http://www.clinicalimagingscience.org/text.asp?2012/2/1/45/99179