The purpose of this study was to evaluate the performance of a direct computer-aided detection (d-CAD) system integrated with full-field digital mammography (FFDM) in assessment of amorphous calcifications.
From 1438 consecutive stereotactic-guided biopsies, FFDM images with amorphous calcifications were selected for retrospective evaluation by d-CAD in 122 females (mean age, 56 years; range, 35–84 years). The sensitivity, specificity, accuracy and false-positive rate of the d-CAD system were calculated in the total group of 124 lesions and in the subgroups based on breast density, mammographic lesion distribution and extension. Logistic regression analysis was used to stratify the risk of malignancy by patient risk factors and age.
The d-CAD marked all (36/36) breast cancers, 85% (11/13) of the high-risk lesions and 80% (60/75) of benign amorphous calcifications (p<0.01) correctly. The sensitivity, specificity and diagnostic accuracy for the combined malignant and “high-risk” lesions was 96, 80 and 86%, respectively. The likelihood of malignancy was 29%. There was no significant difference between the marking of fatty or dense breasts (p>0.05); however, d-CAD marks showed differences for small (<7 mm) lesions (p=0.02) and clustered calcifications (p=0.03). The false-positive rate of d-CAD was 1.76 marks per full examination.
The d-CAD system correctly marked all biopsy-proven breast cancers and a large number of biopsy-proven high-risk lesions that presented as amorphous calcifications. Given our 29% likelihood of malignancy, imaging-guided biopsy appears to be a reasonable recommendation in cases of amorphous calcifications marked by d-CAD.
The aim of this study was to evaluate the effectiveness of computer-aided detection (CAD) to mark the cancer on digital mammograms at the time of breast cancer diagnosis and also review retrospectively whether CAD marked the cancer if visible on any available prior mammograms, thus potentially identifying breast cancer at an earlier stage. We sought to determine why breast lesions may or may not be marked by CAD. In particular, we analyzed factors such as breast density, mammographic views, and lesion characteristics.
Materials and Methods:
Retrospective review from 2004 to 2008 revealed 3445 diagnosed breast cancers in both symptomatic and asymptomatic patients; 1293 of these were imaged with full field digital mammography (FFDM). After cancer diagnosis, in a retrospective review held by the radiologist staff, 43 of these cancers were found to be visible on prior-year mammograms (false-negative cases); these breast cancer cases are the basis of this analysis. All cases had CAD evaluation available at the time of cancer diagnosis and on prior mammography studies. Data collected included patient demographics, breast density, palpability, lesion type, mammographic size, CAD marks on current- and prior-year mammograms, needle biopsy method, pathology results (core needle and/or surgical), surgery type, and lesion size.
On retrospective review of the mammograms by the staff radiologists, 43 cancers were discovered to be visible on prior-year mammograms. All 43 cancers were masses (mass classification included mass, mass with calcification, and mass with architectural distortion); no pure microcalcifications were identified in this cohort. Mammograms with CAD applied at the time of breast cancer diagnosis were able to detect 79% (34/43) of the cases and 56% (24/43) from mammograms with CAD applied during prior year(s). In heterogeneously dense/extremely dense tissue, CAD marked 79% (27/34) on mammograms taken at the time of diagnosis and 56% (19/34) on mammograms with CAD applied during the prior year(s). At time of diagnosis, CAD marked lesions in 32% (11/34) on the craniocaudal (CC) view, 21% (7/34) on the mediolateral oblique (MLO) view. Lesion size of those marked by CAD or not marked were similar, the average being 15 and 12 mm, respectively.
CAD marked cancers on mammograms at the time of diagnosis in 79% of the cases and in 56% of the cases from the mammograms with CAD applied in the prior year(s). Our review demonstrated that CAD can mark invasive breast carcinomas in even dense breast tissue. CAD marked a significant portion on the CC view only, which may be an indicator to radiologists to be especially vigilant when a lesion is marked on this view.
Breast carcinoma; breast imaging; calcifications; computer-aided detection; digital mammography
To evaluate a noncommercial computer-aided detection (CAD) program for breast cancer detection with screening mammography.
MATERIALS AND METHODS
A CAD program was developed for mammographic breast cancer detection. The program was applied to 2,389 patients’ screening mammograms at two geographically remote academic institutions (institutions A and B). Thirteen radiologists who specialized in breast imaging participated in this pilot study. For each case, the individual radiologist performed a prospective Breast Imaging Reporting and Data System (BI-RADS) assessment after viewing of the screening mammogram. Subsequently, the radiologist was shown CAD results and rendered a second BI-RADS assessment by using knowledge of both mammographic appearance and CAD results. Outcome analysis of results of examination in patients recalled for a repeat examination, of biopsy, and of 1-year follow-up examination was recorded. Correct detection with CAD included a computer-generated mark indicating a possible malignancy on craniocaudal or mediolateral oblique views or both.
Eleven (0.46%) of 2,389 patients had mammographically detected nonpalpable breast cancers. Ten (91%) of 11 (95% CI: 74%, 100%) cancers were correctly identified with CAD. Radiologist sensitivity without CAD was 91% (10 of 11; 95% CI: 74%, 100%). In 1,077 patients, follow-up findings were documented at 1 year. Five (0.46%) patients developed cancers, which were found on subsequent screening mammograms. The area where the cancers developed in two (40%) of these five patients was marked (true-positive finding) by the computer in the preceding year. Because of CAD results, a 9.7% increase in recall rate from 14.4% (344 of 2,389) to 15.8% (378 of 2,389) occurred. Radiologists’ recall rate of study patients prior to use of CAD was 31% higher than the average rate for nonstudy cases (10.3%) during the same time period at institution A.
Performance of the CAD program had a very high sensitivity of 91% (95% CI: 74%, 100%).
Breast neoplasms, diagnosis, 00.30; Cancer screening; Computers, diagnostic aid; Diagnostic radiology, observer performance
To evaluate the number of actual detections versus “accidental” detections by a computer-aided detection (CAD) system for small nodular lung cancers (≤30 mm) on chest radiographs, using two different criteria for measuring performance. A Food-and-Drug-Administration-approved CAD program (version 1.0; Riverain Medical) was applied to 34 chest radiographs with a “radiologist-missed” nodular cancer and 36 radiographs with a radiologist-mentioned nodule (a newer version 3.0 was also applied to the 36-case database). The marks applied by this CAD system consisted of 5-cm-diameter circles. A strict “nodule-in-center” criterion and a generous “nodule-in-circle” criterion were compared as methods for the calculation of CAD sensitivity. The increased sensitivities by the nodule-in-circle criterion were considered as nodules detected by chance. The number of false-positive (FP) marks was also analyzed. For the 34 radiologist-missed cancers, the nodule-in-circle criterion caused eight more cancers (24%) to be detected by chance, as compared to the nodule-in-center criterion, when using the version 1.0 results. For the 36 radiologist-mentioned nodules, the nodule-in-circle criterion caused seven more lesions (19%) to be detected by chance, as compared to the nodule-in-center criterion, when using the version 1.0 results, and three more lesions (8%) to be detected by chance when using the version 3.0 results. Version 1.0 yielded a mean of six FP marks per image, while version 3.0 yielded only three FP marks per image. The specific criteria used to define true- and false-positive CAD detections can substantially influence the apparent accuracy of a CAD system.
Lung; neoplasms; computer-aided detection; chest radiography
To evaluate an interactive computer-aided detection (CAD) system for reading mammograms to improve decision making.
A dedicated mammographic workstation has been developed in which readers can probe image locations for the presence of CAD information. If present, CAD findings are displayed with the computed malignancy rating. A reader study was conducted in which four screening radiologists and five non-radiologists participated to study the effect of this system on detection performance. The participants read 120 cases of which 40 cases had a malignant mass that was missed at the original screening. The readers read each mammogram both with and without CAD in separate sessions. Each reader reported localized findings and assigned a malignancy score per finding. Mean sensitivity was computed in an interval of false-positive fractions less than 10%.
Mean sensitivity was 25.1% in the sessions without CAD and 34.8% in the CAD-assisted sessions. The increase in detection performance was significant (p = 0.012). Average reading time was 84.7 ± 61.5 s/case in the unaided sessions and was not significantly higher when interactive CAD was used (85.9 ± 57.8 s/case).
Interactive use of CAD in mammography may be more effective than traditional CAD for improving mass detection without affecting reading time.
Mammography; Breast; Early detection of cancer; Decision making; Computer-assisted; Radiographic image interpretation
Current computer-aided detection (CAD) schemes for detecting mammographic masses have several limitations including high correlation with radiologists’ detection and cueing most subtle masses only on one view. To increase CAD sensitivity in cueing more subtle masses that are likely missed and/or overlooked by radiologists without increasing false-positive rates, we investigated a new case-dependent cueing method by combining the original CAD-generated detection scores with a computed bilateral mammographic density asymmetry index. Using the new method, we adaptively raise CAD-generated scores of regions detected on “high-risk” cases to cue more subtle mass regions and reduce CAD scores of regions detected on “low-risk” cases to discard more false-positive regions. A testing dataset involving 78 positive and 338 negative cases was used to test this adaptive cueing method. Each positive case involves two sequential examinations in which the mass was detected in “current” examination and missed in “prior” examination but detected in a retrospective review by radiologists. Applying to this dataset, a pre-optimized CAD scheme yielded 75% case-based and 55% region-based sensitivity on “current” examinations at a false-positive rate of 0.25 per image. CAD sensitivity was reduced to 42% (case-based) and 27% (region-based) on “prior” examinations. Using the new cueing method, case-based and region-based sensitivity could maximally increase 9% and 33% on the “prior” examinations, respectively. The percentages of the masses cued on two views also increased from 27% to 65%. The study demonstrated that using this adaptive cueing method enabled to help CAD cue more subtle cancers without increasing false-positive cueing rate.
Computer-aided detection (CAD); Digital mammography; Bilateral mammographic density asymmetry; Mass detection
Computer-aided detection (CAD) is applied during screening mammography for millions of US women annually, although it is uncertain whether CAD improves breast cancer detection when used by community radiologists.
We investigated the association between CAD use during film-screen screening mammography and specificity, sensitivity, positive predictive value, cancer detection rates, and prognostic characteristics of breast cancers (stage, size, and node involvement). Records from 684 956 women who received more than 1.6 million film-screen mammograms at Breast Cancer Surveillance Consortium facilities in seven states in the United States from 1998 to 2006 were analyzed. We used random-effects logistic regression to estimate associations between CAD and specificity (true-negative examinations among women without breast cancer), sensitivity (true-positive examinations among women with breast cancer diagnosed within 1 year of mammography), and positive predictive value (breast cancer diagnosed after positive mammograms) while adjusting for mammography registry, patient age, time since previous mammography, breast density, use of hormone replacement therapy, and year of examination (1998–2002 vs 2003–2006). All statistical tests were two-sided.
Of 90 total facilities, 25 (27.8%) adopted CAD and used it for an average of 27.5 study months. In adjusted analyses, CAD use was associated with statistically significantly lower specificity (OR = 0.87, 95% confidence interval [CI] = 0.85 to 0.89, P < .001) and positive predictive value (OR = 0.89, 95% CI = 0.80 to 0.99, P = .03). A non-statistically significant increase in overall sensitivity with CAD (OR = 1.06, 95% CI = 0.84 to 1.33, P = .62) was attributed to increased sensitivity for ductal carcinoma in situ (OR = 1.55, 95% CI = 0.83 to 2.91; P = .17), although sensitivity for invasive cancer was similar with or without CAD (OR = 0.96, 95% CI = 0.75 to 1.24; P = .77). CAD was not associated with higher breast cancer detection rates or more favorable stage, size, or lymph node status of invasive breast cancer.
CAD use during film-screen screening mammography in the United States is associated with decreased specificity but not with improvement in the detection rate or prognostic characteristics of invasive breast cancer.
To evaluate performance of computer-aided detection (CAD) beyond double reading for pulmonary nodules on low-dose computed tomography (CT) by nodule volume.
A total of 400 low-dose chest CT examinations were randomly selected from the NELSON lung cancer screening trial. CTs were evaluated by two independent readers and processed by CAD. A total of 1,667 findings marked by readers and/or CAD were evaluated by a consensus panel of expert chest radiologists. Performance was evaluated by calculating sensitivity of pulmonary nodule detection and number of false positives, by nodule characteristics and volume.
According to the screening protocol, 90.9 % of the findings could be excluded from further evaluation, 49.2 % being small nodules (less than 50 mm3). Excluding small nodules reduced false-positive detections by CAD from 3.7 to 1.9 per examination. Of 151 findings that needed further evaluation, 33 (21.9 %) were detected by CAD only, one of them being diagnosed as lung cancer the following year. The sensitivity of nodule detection was 78.1 % for double reading and 96.7 % for CAD. A total of 69.7 % of nodules undetected by readers were attached nodules of which 78.3 % were vessel-attached.
CAD is valuable in lung cancer screening to improve sensitivity of pulmonary nodule detection beyond double reading, at a low false-positive rate when excluding small nodules.
• Computer-aided detection (CAD) has known advantages for computed tomography (CT).
• Combined CAD/nodule size cut-off parameters assist CT lung cancer screening.
• This combination improves the sensitivity of pulmonary nodule detection by CT.
• It increases the positive predictive value for cancer detection.
Computer-aided detection; Multi-detector computed tomography; Pulmonary nodules; Low dose; Volumetry
We have developed a computer-aided detection (CAD) system to detect clustered microcalcification automatically on full-field digital mammograms (FFDMs) and a CAD system for screen-film mammograms (SFMs). The two systems used the same computer vision algorithms but their false positive (FP) classifiers were trained separately with sample images of each modality. In this study, we compared the performance of the CAD systems for detection of clustered microcalcifications on pairs of FFDM and SFM obtained from the same patient. For case-based performance evaluation, the FFDM CAD system achieved detection sensitivities of 70%, 80%, and 90% at an average FP cluster rate of 0.07, 0.16, and 0.63 per image, compared with an average FP cluster rate of 0.15, 0.38, and 2.02 per image for the SFM CAD system. The difference was statistically significant with the alternative free-response receiver operating characteristic (AFROC) analysis. When evaluated on data sets negative for microcalcification clusters, the average FP cluster rates of the FFDM CAD system were 0.04, 0.11, and 0.33 per image at detection sensitivity level of 70%, 80%, and 90%, compared with an average FP cluster rate of 0.08, 0.14, and 0.50 per image for the SFM CAD system. When evaluated for malignant cases only, the difference of the performance of the two CAD systems was not statistically significant with AFROC analysis.
To determine whether the display of computer-aided detection (CAD) marks on individual polyps on both the supine and prone scans leads to improved polyp detection by radiologists compared to the display of CAD marks on individual polyps on either the supine or the prone scan, but not both.
METHOD AND MATERIALS
The acquisition of patient data for this study was approved by the Institutional Review Board and was HIPAA-compliant. Subsequently, the use of the data was declared exempt from further IRB review. Four radiologists interpreted 33 CT colonography cases, 21 of which had one adenoma 6 to 9 mm in size, with the assistance of a CAD system in the first reader mode, i.e., the radiologists reviewed only the CAD marks. The radiologists were shown each case twice, with different sets of CAD marks for each of the two readings. In one reading a true positive CAD mark for the same polyp was displayed on both the supine and prone scans (a double-mark reading). In the other reading a true positive CAD mark was displayed either on the supine or prone scan but not both (a single-mark reading). True positive marks were randomized between readings and there was at least a one-month delay between readings to minimize recall bias. Sensitivity and specificity were determined and receiver operating characteristic (ROC) and multiple-reader multiple-case analyses were performed.
The average per polyp sensitivities were 60% [38%, 81%] vs. 71% [52%, 91%] (p =.03) for single-mark and double mark readings, respectively. The areas [95% confidence intervals] under the ROC curves were 0.76 [0.62, 0.88] and 0.79 [0.58, 0.96], respectively (p=NS). Specificities were similar for the single-mark compared to the double-mark readings.
The display of CAD marks on a polyp on both the supine and prone scans led to more frequent detection of polyps by radiologists without adversely affecting specificity for detecting 6–9 mm adenomas.
CT, colon; CT, virtual imaging; Colon cancer; image processing; automated detection; observer performance
The objective of this study is to assess the impact on nodule detection and efficiency using a computer-aided detection (CAD) device seamlessly integrated into a commercially available picture archiving and communication system (PACS). Forty-eight consecutive low-dose thoracic computed tomography studies were retrospectively included from an ongoing multi-institutional screening study. CAD results were sent to PACS as a separate image series for each study. Five fellowship-trained thoracic radiologists interpreted each case first on contiguous 5 mm sections, then evaluated the CAD output series (with CAD marks on corresponding axial sections). The standard of reference was based on three-reader agreement with expert adjudication. The time to interpret CAD marking was automatically recorded. A total of 134 true-positive nodules, measuring 3 mm and larger were included in our study; with 85 ≥ 4 and 50 ≥ 5 mm in size. Readers detection improved significantly in each size category when using CAD, respectively, from 44 to 57 % for ≥3 mm, 48 to 61 % for ≥4 mm, and 44 to 60 % for ≥5 mm. CAD stand-alone sensitivity was 65, 68, and 66 % for nodules ≥3, ≥4, and ≥5 mm, respectively, with CAD significantly increasing the false positives for two readers only. The average time to interpret and annotate a CAD mark was 15.1 s, after localizing it in the original image series. The integration of CAD into PACS increases reader sensitivity with minimal impact on interpretation time and supports such implementation into daily clinical practice.
Radiographic image interpretation; Computer-assisted; Radiography; Thoracic; PACS reading; Clinical workflow; Lung; Efficiency; Computed tomography; Computer-assisted detection; Chest CT
Rationale and Objective
Lesion conspicuity is typically highly correlated with visual difficulty for lesion detection and computer-aided detection (CAD) has been widely used as a “second reader” in mammography. Hence, increasing CAD sensitivity in detecting subtle cancers without increasing false-positive rates is important. This study investigates the effect of training database case selection on CAD performance in detecting low conspicuity breast masses.
Materials and Methods
A full-field digital mammography image database that includes 525 cases depicting malignant masses was randomly partitioned into three subsets. A CAD scheme was applied to detect all initially suspected mass regions and compute region conspicuity. We iteratively selected training samples from two of the subsets. Four types of training datasets, namely; (1) one including all available true-positive mass regions in the two subsets (termed here “All”); (2) one including 350 randomly selected mass regions (“diverse”); (3) one including 350 high conspicuity mass regions (“easy”); and (4) one including 350 low conspicuity mass regions (“difficult”), were assembled. In each training dataset the same number of randomly selected false-positive regions as the true-positives was also included. Two classifiers, an artificial neural network (ANN) and a k-nearest neighbor algorithm (KNN), were trained using each of the four training datasets and tested on all suspected regions in the remaining dataset. Using a 3-fold cross-validation method, we computed and compared the performance changes of the CAD schemes trained using one of the four training datasets.
CAD initially detected 1025 true-positive mass regions depicted on 507 cases (97% case-based sensitivity) and 9569 false-positive regions (3.5 per image) in the entire database. Using the “All” training dataset, CAD achieved the highest overall performance on the entire testing database. However, CAD detected the highest number of low conspicuity masses when the “difficult” training dataset was used for training. Results did concord for both ANN and KNN based classifiers in all tests. Compared with the use of the “All” training dataset, sensitivity of the schemes trained using the “difficult” dataset decreased by 8.6% and 8.4% for ANN and KNN on the entire database, respectively, but the detection of low conspicuity masses increased by 7.1% and 15.1% for ANN and KNN at a false-positive rate of 0.3 per image.
CAD performance depends on the size, diversity, and “difficulty” level of the training database. To increase CAD sensitivity in detecting subtle cancer, one should increase the fraction of “difficult” cases in the training database rather than simply increase the training dataset size.
Computer-aided detection (CAD); Full-field digital mammography (FFDM); Image databases; Performance assessment
To evaluate the effect of computer-aided detection (CAD) system on observer performance in the detection of malignant lung nodules on chest radiograph.
Materials and Methods
Two hundred chest radiographs (100 normal and 100 abnormal with malignant solitary lung nodules) were evaluated. With CT and histological confirmation serving as a reference, the mean nodule size was 15.4 mm (range, 7-20 mm). Five chest radiologists and five radiology residents independently interpreted both the original radiographs and CAD output images using the sequential testing method. The performances of the observers for the detection of malignant nodules with and without CAD were compared using the jackknife free-response receiver operating characteristic analysis.
Fifty-nine nodules were detected by the CAD system with a false positive rate of 1.9 nodules per case. The detection of malignant lung nodules significantly increased from 0.90 to 0.92 for a group of observers, excluding one first-year resident (p = 0.04). When lowering the confidence score was not allowed, the average figure of merit also increased from 0.90 to 0.91 (p = 0.04) for all observers after a CAD review. On average, the sensitivities with and without CAD were 87% and 84%, respectively; the false positive rates per case with and without CAD were 0.19 and 0.17, respectively. The number of additional malignancies detected following true positive CAD marks ranged from zero to seven for the various observers.
The CAD system may help improve observer performance in detecting malignant lung nodules on chest radiographs and contribute to a decrease in missed lung cancer.
Computer-aided detection; Lung nodules; Lung cancer; Chest radiograph
Rationale and Objectives
To examine radiologists’ use and perceptions of computer-aided detection (CAD) and double reading for screening mammography interpretation.
Materials and Methods
A mailed survey of 257 community radiologists participating in the national Breast Cancer Surveillance Consortium assessed perceptions and practices related to CAD and double reading. We used latent class analysis to classify radiologists’ overall perceptions of CAD and double reading based on their agreement or disagreement with specific statements about CAD and double reading.
Most radiologists (64%) reported using CAD for more than half the screening mammograms they interpreted, but only <5% reported double reading that much. More radiologists perceived that double reading improved cancer detection rates compared with CAD (74% vs. 55% reported), while fewer radiologists thought that double reading decreased recall rates compared with CAD (50% vs. 65% reported). Radiologists with the most favorable perceptions of CAD were more likely to think that CAD improved cancer detection rate without taking too much time compared with radiologists with the most unfavorable overall perceptions. In latent class analysis an overall favorable perception of CAD was significantly associated with use of CAD (81%), higher percent of workload in screening mammography (80%), academic affiliation (71%), and fellowship training (58%). Perceptions of double reading that were most favorable were associated with academic affiliation (98%).
Radiologists’ perceptions were more favorable toward double reading by a second clinician than by a computer, although fewer used double reading in their own practice. The majority of radiologists perceived both CAD and double reading at least somewhat favorably, although for largely different reasons.
Breast imaging; mammography; computer-aided detection (CAD); double reading
To assess the sensitivities and false detection rates of two CADe systems when applied to digital or screen-film mammograms in detecting the known breast cancer cases from the DMIST breast cancer screening population.
Materials and Methods
Available screen-film and digital mammograms of 161 breast cancer cases from DMIST were analyzed by two CADe systems, iCAD SecondLook (iCAD) and R2 ImageChecker (R2). Three experienced breast imaging radiologists reviewed the CADe marks generated for each available cancer case, recording the number and locations of CADe marks and whether each CADe mark location corresponded with the known location of the cancer.
For the 161 cancer cases included in this study, the sensitivities of the DMIST reader without CAD were 0.43 (69/161, 95% CI 0.35 to 0.51) for digital and 0.41 (66/161, 95% CI 0.33 to 0.49) for film-screen mammography. The sensitivities of iCAD were 0.74 (119/161, 95% CI 0.66 to 0.81) for digital and 0.69 (111/161, 95% CI 0.61 to 0.76) for screen-film mammogram, both significantly higher than the DMIST study sensitivities (p< 0.0001 for both). The average number of false CADe marks per case of iCAD was 2.57 (SD 1.92) for digital and 3.06(SD 1.72) for screen-film mammography. The sensitivity of R2 was 0.74 (119/161, 95% CI 0.66 to 0.81) for digital, and 0.60 (97/161, 95% CI 0.52 to 0.68) for screen-film mammography, both significantly higher than the DMIST study sensitivities (p< 0.0001 for both). The average number of false CADe marks per case of R2 was 2.07 (SD 1.57) for digital and 1.52(SD 1.45) for screen-film mammogram.
Our results suggest the use of CADe in interpretation of digital and screen-film mammograms could lead to improvements in cancer detection.
Computer-aided detection (CAD) systems have been shown to improve the diagnostic performance of CT colonography (CTC) in the detection of premalignant colorectal polyps. Despite the improvement, the overall system is not optimal. CAD annotations on true lesions are incorrectly dismissed, and false positives are misinterpreted as true polyps. Here, we conduct an observer performance study utilizing distributed human intelligence in the form of anonymous knowledge workers (KWs) to investigate human performance in classifying polyp candidates under different presentation strategies. We evaluated 600 polyp candidates from 50 patients, each case having at least one polyp • 6 mm, from a large database of CTC studies. Each polyp candidate was labeled independently as a true or false polyp by 20 KWs and an expert radiologist. We asked each labeler to determine whether the candidate was a true polyp after looking at a single 3D-rendered image of the candidate and after watching a video fly-around of the candidate. We found that distributed human intelligence improved significantly when presented with the additional information in the video fly-around. We noted that performance degraded with increasing interpretation time and increasing difficulty, but distributed human intelligence performed better than our CAD classifier for “easy” and “moderate” polyp candidates. Further, we observed numerous parallels between the expert radiologist and the KWs. Both showed similar improvement in classification moving from single-image to video interpretation. Additionally, difficulty estimates obtained from the KWs using an expectation maximization algorithm correlated well with the difficulty rating assigned by the expert radiologist. Our results suggest that distributed human intelligence is a powerful tool that will aid in the development of CAD for CTC.
Computed tomography colonography; observer performance study; crowdsourcing; distributed human intelligence; video analysis; labeling modeling
Purpose of Review
Computer-Aided Diagnosis (CAD) is a technology used for detection and characterization of cancer. Although CAD is not limited to a single type of cancer, a large number of CAD systems to date have been designed and used for breast cancer. The aim of this review is to discuss the current state of the CAD systems for breast cancer diagnosis, their application as a second reader in clinical practice, and the studies that evaluated the effect of CAD on radiologists’ performance.
A large number of CAD applications are being developed for different imaging modalities. The main clinical use of CAD to date is for screen-film mammography due to the commercially available FDA approved systems. Many studies showed that CAD improves radiologists’ performance. A large number of academic institutions have devoted substantial research effort to develop CAD methods.
CAD systems will play an increasingly important role in the clinic as a second reader. Clinical trials have shown that CAD can improve the accuracy of breast cancer detection. Preclinical studies have demonstrated the potential of CAD to improve classification of malignant and benign lesions. An increased number of CAD systems are being developed for different breast imaging modalities.
Computer aided diagnosis; breast cancer; mammography; detection; characterization
Computer-aided diagnosis (CAD) for characterization of mammographic masses as malignant or benign has the potential to assist radiologists in reducing the biopsy rate without increasing false negatives. The purpose of this study was to develop an automated method for mammographic mass segmentation and explore new image based features in combination with patient information in order to improve the performance of mass characterization. Our previous CAD system, which used the active contour segmentation, and morphological, textural, and spiculation features, has achieved promising results in mass characterization. The new CAD system is based on the level set method, and includes two new types of image features related to the presence of microcalcifications with the mass and abruptness of the mass margin, and patient age. A linear discriminant analysis (LDA) classifier with stepwise feature selection was used to merge the extracted features into a classification score. The classification accuracy was evaluated using the area under the receiver operating characteristic curve. Our primary data set consisted of 427 biopsy-proven masses (200 malignant and 227 benign) in 909 regions of interest (ROIs) (451 malignant and 458 benign) from multiple mammographic views. Leave-one-case-out resampling was used for training and testing. The new CAD system based on the level set segmentation and the new mammographic feature space achieved a view-based Az value of 0.83±0.01. The improvement compared to the previous CAD system was statistically significant (p=0.02). When patient age was included in the new CAD system, view-based and case-based Az values were 0.85±0.01 and 0.87±0.02, respectively. The performance of the new CAD system was also compared to an experienced radiologist’s likelihood of malignancy rating. When patient age was used in classification, the accuracy of the new CAD system was comparable to that of the radiologist (p=0.34). The study also demonstrated the consistency of the newly developed CAD system by evaluating the statistics of the weights of the LDA classifiers in leave-one-case-out classification. Finally, an independent test on the publicly available digital database for screening mammography (DDSM) with 132 benign and 197 malignant ROIs containing masses achieved a view-based Az value of 0.84±0.02.
computer-aided diagnosis; mammography; breast masses; level set; segmentation; classification
To evaluate the additional value of computer-aided detection (CAD) in breast MRI by assessing radiologists’ accuracy in discriminating benign from malignant breast lesions.
A literature search was performed with inclusion of relevant studies using a commercially available CAD system with automatic colour mapping. Two independent researchers assessed the quality of the studies. The accuracy of the radiologists’ performance with and without CAD was presented as pooled sensitivity and specificity.
Of 587 articles, 10 met the inclusion criteria, all of good methodological quality. Experienced radiologists reached comparable pooled sensitivity and specificity before and after using CAD (sensitivity: without CAD: 89%; 95% CI: 78–94%, with CAD: 89%; 95%CI: 81–94%) (specificity: without CAD: 86%; 95% CI: 79–91%, with CAD: 82%; 95% CI: 76–87%). For residents the pooled sensitivity increased from 72% (95% CI: 62–81%) without CAD to 89% (95% CI: 80–94%) with CAD, however, not significantly. Concerning specificity, the results were similar (without CAD: 79%; 95% CI: 69–86%, with CAD: 78%; 95% CI: 69–84%).
CAD in breast MRI has little influence on the sensitivity and specificity of experienced radiologists and therefore their interpretation remains essential. However, residents or inexperienced radiologists seem to benefit from CAD concerning breast MRI evaluation.
Magnetic resonance imaging; Breast; Computer aided detection; CAD; Meta-analysis
Computer-aided detection (CAD) has been attracting extensive research interest during the last two decades. It is recognized that the full potential of CAD can only be realized by improving the performance and robustness of CAD algorithms and this requires good evaluation methodology that would permit CAD designers to optimize their algorithms. Free-response receiver operating characteristic (FROC) curves are widely used to assess CAD performance, however, evaluation rarely proceeds beyond determination of lesion localization fraction (sensitivity) at an arbitrarily selected value of non-lesion localizations (false marks) per image. This work describes an FROC curve fitting procedure that uses a recent model of visual search that serves as a framework for the free-response task. A maximum likelihood procedure for estimating the parameters of the model from free-response data and fitting CAD generated FROC curves was implemented. Procedures were implemented to estimate two figures of merit and associated statistics such as 95% confidence intervals and goodness of fit. One of the figures of merit does not require the arbitrary specification of an operating point at which to evaluate CAD performance. For comparison a related method termed initial detection and candidate analysis (IDCA) was also implemented that is applicable when all suspicious regions are known, no matter how low the degree of suspicion (or confidence level). The two methods were tested on seven mammography CAD data sets and both yielded good-excellent fits. The search model approach has the advantage that it can potentially be applied to radiologist generated free-response data where not all suspicious regions are reported, only the ones that are deemed sufficiently suspicious to warrant clinical follow-up. This work represents the first practical application of the search model to an important evaluation problem in diagnostic radiology. Software based on this work is expected to benefit CAD developers working in diverse areas of medical imaging.
CAD evaluation; free-response; FROC curves; lesion localization; search model; maximum likelihood; figure of merit; imaging system optimization
In accordance with European guidelines, mammography screening comprises independent readings by two breast radiologists (double reading). CAD (computer-aided detection) has been suggested to complement or replace one of the two readers (single reading + CAD).
The aim of this systematic review is to address the following question: Is the reading of mammographic x-ray images by a single breast radiologist together with CAD at least as accurate as double reading?
The electronic literature search included the databases Pub Med, EMBASE and The Cochrane Library. Two independent reviewers assessed abstracts and full-text articles.
1049 abstracts were identified, of which 996 were excluded with reference to inclusion and exclusion criteria; 53 full-text articles were assessed for eligibility. Finally, four articles were included in the qualitative analysis, and one in a GRADE synthesis.
The scientific evidence is insufficient to determine whether the accuracy of single reading + CAD is at least equivalent to that obtained in standard practice, i.e. double reading where two breast radiologists independently read the mammographic images.
CAD; Mammography; Screening; Breast; Cancer; Single reading; Double reading
Although mammography is the only clinically accepted imaging modality for screening the general population to detect breast cancer, interpreting mammograms is difficult with lower sensitivity and specificity. To provide radiologists “a visual aid” in interpreting mammograms, we developed and tested an interactive system for computer-aided detection and diagnosis (CAD) of mass-like cancers. Using this system, an observer can view CAD-cued mass regions depicted on one image and then query any suspicious regions (either cued or not cued by CAD). CAD scheme automatically segments the suspicious region or accepts manually defined region and computes a set of image features. Using content-based image retrieval (CBIR) algorithm, CAD searches for a set of reference images depicting “abnormalities” similar to the queried region. Based on image retrieval results and a decision algorithm, a classification score is assigned to the queried region. In this study, a reference database with 1,800 malignant mass regions and 1,800 benign and CAD-generated false-positive regions was used. A modified CBIR algorithm with a new function of stretching the attributes in the multi-dimensional space and decision scheme was optimized using a genetic algorithm. Using a leave-one-out testing method to classify suspicious mass regions, we compared the classification performance using two CBIR algorithms with either equally weighted or optimally stretched attributes. Using the modified CBIR algorithm, the area under receiver operating characteristic curve was significantly increased from 0.865 ± 0.006 to 0.897 ± 0.005 (p < 0.001). This study demonstrated the feasibility of developing an interactive CAD system with a large reference database and achieving improved performance.
Computer-aided detection and diagnosis (CAD); Content-based image retrieval (CBIR); Breast cancer; Mammograms
Computer-assisted mammography imaging comprises computer-based analysis of digitized images resulting in prompts aiding mammographic interpretation and computerized stereotactic localization devices which improve location accuracy. The commercial prompting systems available are designed to draw attention to mammographic abnormalities detected by algorithms based on symptomatic practise in North America. High sensitivity rates are important commercially but result in increased false prompt rates, which are known to distract radiologists. A national shortage of breast radiologists in the UK necessitates evaluation of such systems in a population breast screening programme to determine effectiveness in increasing cancer detection and feasibility of implementation.
algorithm; computer-assisted mammography; digital; digitally acquired; digitised; effectiveness; prompt
Rationale and Objectives
In our earlier studies we reported an evidence-based Computer Assisted Decision (CAD) system for location-specific interrogation of mammograms. A content-based image retrieval framework with information theoretic (IT) similarity measures serves as the foundation for this system. Specifically, the normalized mutual information (NMI) was shown to be the most effective similarity measure for reduction of false positive marks generated by other, prescreening mass detection schemes. The objective of this work was to investigate the importance of image filtering as a possible preprocessing step in our IT-CAD system.
Materials and Methods
Different filters were applied, each one aiming to compensate for known limitations of the NMI similarity measure. The study was based on a region-of-interest database that included true masses and false positive regions from digitized mammograms.
Receiver Operating Characteristics (ROC) analysis showed that IT-CAD is affected slightly by image filtering. Modest, yet statistically significant performance gain was observed with median filtering (overall ROC area index Az improved from 0.78 to 0.82). However, Gabor filtering improved performance for the high sensitivity portion of the ROC curve where a typical false positive reduction scheme should operate (partial ROC area index 0.90Az improved from 0.33 to 0.37). Fusion of IT-CAD decisions from different filtering schemes markedly improved performance (Az=0.90 and 0.90Az=0.55). At 95% sensitivity, the system’s specificity improved by 36.6%.
Additional improvement in false positive reduction can be achieved by incorporating image filtering as a preprocessing step in our information-theoretic CAD system.
CAD; mammography; image processing; information theory
Building an optimal image reference library is a critical step in developing the interactive computer-aided detection and diagnosis (I-CAD) systems of medical images using content-based image retrieval (CBIR) schemes. In this study, the authors conducted two experiments to investigate (1) the relationship between I-CAD performance and size of reference library and (2) a new reference selection strategy to optimize the library and improve I-CAD performance. The authors assembled a reference library that includes 3153 regions of interest (ROI) depicting either malignant masses (1592) or CAD-cued false-positive regions (1561) and an independent testing data set including 200 masses and 200 false-positive regions. A CBIR scheme using a distance-weighted K-nearest neighbor algorithm is applied to retrieve references that are considered similar to the testing sample from the library. The area under receiver operating characteristic curve (Az) is used as an index to evaluate the I-CAD performance. In the first experiment, the authors systematically increased reference library size and tested I-CAD performance. The result indicates that scheme performance improves initially from Az=0.715 to 0.874 and then plateaus when the library size reaches approximately half of its maximum capacity. In the second experiment, based on the hypothesis that a ROI should be removed if it performs poorly compared to a group of similar ROIs in a large and diverse reference library, the authors applied a new strategy to identify “poorly effective” references. By removing 174 identified ROIs from the reference library, I-CAD performance significantly increases to Az=0.914 (p < 0.01). The study demonstrates that increasing reference library size and removing poorly effective references can significantly improve I-CAD performance.
content-based image retrieval (CBIR); interactive computer-aided diagnosis (I-CAD); K-nearest neighbor algorithm; mammographic masses; performance evaluation