Search tips
Search criteria

Results 1-25 (1386219)

Clipboard (0)

Related Articles

1.  Evaluation of breast amorphous calcifications by a computer-aided detection system in full-field digital mammography 
The British Journal of Radiology  2012;85(1013):517-522.
The purpose of this study was to evaluate the performance of a direct computer-aided detection (d-CAD) system integrated with full-field digital mammography (FFDM) in assessment of amorphous calcifications.
From 1438 consecutive stereotactic-guided biopsies, FFDM images with amorphous calcifications were selected for retrospective evaluation by d-CAD in 122 females (mean age, 56 years; range, 35–84 years). The sensitivity, specificity, accuracy and false-positive rate of the d-CAD system were calculated in the total group of 124 lesions and in the subgroups based on breast density, mammographic lesion distribution and extension. Logistic regression analysis was used to stratify the risk of malignancy by patient risk factors and age.
The d-CAD marked all (36/36) breast cancers, 85% (11/13) of the high-risk lesions and 80% (60/75) of benign amorphous calcifications (p<0.01) correctly. The sensitivity, specificity and diagnostic accuracy for the combined malignant and “high-risk” lesions was 96, 80 and 86%, respectively. The likelihood of malignancy was 29%. There was no significant difference between the marking of fatty or dense breasts (p>0.05); however, d-CAD marks showed differences for small (<7 mm) lesions (p=0.02) and clustered calcifications (p=0.03). The false-positive rate of d-CAD was 1.76 marks per full examination.
The d-CAD system correctly marked all biopsy-proven breast cancers and a large number of biopsy-proven high-risk lesions that presented as amorphous calcifications. Given our 29% likelihood of malignancy, imaging-guided biopsy appears to be a reasonable recommendation in cases of amorphous calcifications marked by d-CAD.
PMCID: PMC3479871  PMID: 22556404
2.  CAD May Not be Necessary for Microcalcifications in the Digital era, CAD May Benefit Radiologists for Masses 
The aim of this study was to evaluate the effectiveness of computer-aided detection (CAD) to mark the cancer on digital mammograms at the time of breast cancer diagnosis and also review retrospectively whether CAD marked the cancer if visible on any available prior mammograms, thus potentially identifying breast cancer at an earlier stage. We sought to determine why breast lesions may or may not be marked by CAD. In particular, we analyzed factors such as breast density, mammographic views, and lesion characteristics.
Materials and Methods:
Retrospective review from 2004 to 2008 revealed 3445 diagnosed breast cancers in both symptomatic and asymptomatic patients; 1293 of these were imaged with full field digital mammography (FFDM). After cancer diagnosis, in a retrospective review held by the radiologist staff, 43 of these cancers were found to be visible on prior-year mammograms (false-negative cases); these breast cancer cases are the basis of this analysis. All cases had CAD evaluation available at the time of cancer diagnosis and on prior mammography studies. Data collected included patient demographics, breast density, palpability, lesion type, mammographic size, CAD marks on current- and prior-year mammograms, needle biopsy method, pathology results (core needle and/or surgical), surgery type, and lesion size.
On retrospective review of the mammograms by the staff radiologists, 43 cancers were discovered to be visible on prior-year mammograms. All 43 cancers were masses (mass classification included mass, mass with calcification, and mass with architectural distortion); no pure microcalcifications were identified in this cohort. Mammograms with CAD applied at the time of breast cancer diagnosis were able to detect 79% (34/43) of the cases and 56% (24/43) from mammograms with CAD applied during prior year(s). In heterogeneously dense/extremely dense tissue, CAD marked 79% (27/34) on mammograms taken at the time of diagnosis and 56% (19/34) on mammograms with CAD applied during the prior year(s). At time of diagnosis, CAD marked lesions in 32% (11/34) on the craniocaudal (CC) view, 21% (7/34) on the mediolateral oblique (MLO) view. Lesion size of those marked by CAD or not marked were similar, the average being 15 and 12 mm, respectively.
CAD marked cancers on mammograms at the time of diagnosis in 79% of the cases and in 56% of the cases from the mammograms with CAD applied in the prior year(s). Our review demonstrated that CAD can mark invasive breast carcinomas in even dense breast tissue. CAD marked a significant portion on the CC view only, which may be an indicator to radiologists to be especially vigilant when a lesion is marked on this view.
PMCID: PMC3424776  PMID: 22919559
Breast carcinoma; breast imaging; calcifications; computer-aided detection; digital mammography
3.  Sensitivity of Noncommercial Computer-aided Detection System for Mammographic Breast Cancer Detection: Pilot Clinical Trial 
Radiology  2004;231(1):208-214.
To evaluate a noncommercial computer-aided detection (CAD) program for breast cancer detection with screening mammography.
A CAD program was developed for mammographic breast cancer detection. The program was applied to 2,389 patients’ screening mammograms at two geographically remote academic institutions (institutions A and B). Thirteen radiologists who specialized in breast imaging participated in this pilot study. For each case, the individual radiologist performed a prospective Breast Imaging Reporting and Data System (BI-RADS) assessment after viewing of the screening mammogram. Subsequently, the radiologist was shown CAD results and rendered a second BI-RADS assessment by using knowledge of both mammographic appearance and CAD results. Outcome analysis of results of examination in patients recalled for a repeat examination, of biopsy, and of 1-year follow-up examination was recorded. Correct detection with CAD included a computer-generated mark indicating a possible malignancy on craniocaudal or mediolateral oblique views or both.
Eleven (0.46%) of 2,389 patients had mammographically detected nonpalpable breast cancers. Ten (91%) of 11 (95% CI: 74%, 100%) cancers were correctly identified with CAD. Radiologist sensitivity without CAD was 91% (10 of 11; 95% CI: 74%, 100%). In 1,077 patients, follow-up findings were documented at 1 year. Five (0.46%) patients developed cancers, which were found on subsequent screening mammograms. The area where the cancers developed in two (40%) of these five patients was marked (true-positive finding) by the computer in the preceding year. Because of CAD results, a 9.7% increase in recall rate from 14.4% (344 of 2,389) to 15.8% (378 of 2,389) occurred. Radiologists’ recall rate of study patients prior to use of CAD was 31% higher than the average rate for nonstudy cases (10.3%) during the same time period at institution A.
Performance of the CAD program had a very high sensitivity of 91% (95% CI: 74%, 100%).
PMCID: PMC2742201  PMID: 14990808
Breast neoplasms, diagnosis, 00.30; Cancer screening; Computers, diagnostic aid; Diagnostic radiology, observer performance
4.  Role of Computer-Aided Detection in Very Small Screening Detected Invasive Breast Cancers 
Journal of Digital Imaging  2012;26(3):572-577.
This study aims to assess computer-aided detection (CAD) performance with full-field digital mammography (FFDM) in very small (equal to or less than 1 cm) invasive breast cancers. Sixty-eight invasive breast cancers less than or equal to 1 cm were retrospectively studied. All cases were detected with FFDM in women aged 49–69 years from our breast cancer screening program. Radiological characteristics of lesions following BI-RADS descriptors were recorded and compared with CAD sensitivity. Age, size, BI-RADS classification, breast density type, histological type of the neoplasm, and role of the CAD were also assessed. Per-study specificity and mass false-positive rate were determined by using 100 normal consecutive studies. Thirty-seven (54.4 %) masses, 17 (25 %) calcifications, 6 (8.8 %) masses with calcifications, 7 (10.3 %) architectural distortions, and 1 asymmetry (1.5 %) were found. CAD showed an overall sensitivity of 86.7 % (masses, 86.5 %; calcifications, 100 %; masses with calcifications, 100 %; and architectural distortion, 57.14 %), CAD failed to detect 9 out of 68 cases: 5 of 37 masses, 3 of 7 architectural distortions, and 1 of 1 asymmetry. Fifteen out of 37 masses were hyperdense, and all of them were detected by CAD. No association was seen among mass morphology or margins and detectability. Per-study specificity and CAD false-positive rate was 26 % and 1.76 false marks per study. In conclusion, CAD shows a high sensitivity and a low specificity. Lesion size, histology, and breast density do not influence sensitivity. Mammographic features, mass density, and thickness of the spicules in architectural distortions do influence.
PMCID: PMC3649063  PMID: 23131867
Breast neoplasm; Cancer detection; Computer-assisted detection
5.  Effect of Breast Density on Computer Aided Detection 
Journal of Digital Imaging  2005;18(3):227-233.
Purpose: This study was conducted to assess the clinical impact of breast density and density of the lesion’s background on the performance of a computer-aided detection (CAD) system in the detection of breast masses (MA) and microcalcifications (MC). Materials and Methods: A total of 200 screening mammograms interpreted as BI-RADS 1 and suspicious mammograms of 150 patients having a histologically verified malignancy from 1992 to 2000 were selected by using a sampler of tumor cases. Excluding those cases having more than one lesion or a contralateral malignancy attributable to statistical reasons, 127 cases with 127 malignant findings were analyzed with a CAD system (Second Look 5.0, CADx Systems, Inc., Beavercreek, OH). Of the 127 malignant lesions, 56 presented as MC and 101 presented as MA, including 30 cases with both malignant signs. Overall breast density of the mammogram and density of the lesion’s background were determined by two observers in congruence (density a: entirely fatty, density b: scattered fibroglandular tissue, density c: heterogeneously dense, density d: extremely dense). Results: Within the unsuspicious group, 100/200 cases did not have any CAD MA marks and were therefore truly negative (specificity 50%), and 151/200 cases did not have any CAD MC marks (specificity 75.5%). For these 200 cases, the numbers of marks per image were 0.41 and 0.37 (density a), 0.38 and 0.97 (density b), 0.44 and 0.91 (density c), and 0.58 and 0.68 (density d) for MC and MA marks, respectively (Fisher’s t-test: n.s. for MC, p < 0.05 for MA). Malignant lesions were correctly detected in at least one view by the CAD system for 52/56 (92.8%) MC and 91/101 (90.1%) MA. Detection rate versus breast density was: 4/6 (66.7%) and 18/19 (94.7%) (density a), 32/33 (97.0%) and 49/51 (96.1%) (density b), 14/15 (93.3%) and 23/28 (82.1%) (density c), and 2/2 (100%) and 1/3 (33.3%) (density d) for MC and MA, respectively. Detection rate versus the lesion’s background was: 19/21 (90.5%) and 36/38 (94.7%) (density a), 34/36 (94.4%) and 59/62 (95.2%) (density b), 8/9 (88.9%) and 20/24 (83.3%) (density c), and 9/10 (90%) and 4/8 (50%) (density d) for groups 2 and 3, respectively. Detection rates differed significantly for masses in heterogeneously dense and extremely dense tissue (overall or lesion’s background) versus all other densities (Fisher’s t-test: p < 0.05). A significantly lowered FP rate for masses was found on mammograms of entirely fatty tissue. Conclusion: Overall breast density and density at a lesion’s background do not appear to have a significant effect on CAD sensitivity or specificity for MC. CAD sensitivity for MA may be lowered in cases with heterogeneously and extremely dense breasts, and CAD specificity for MA is highest in cases with extremely fatty breasts. The effects of overall breast density and density of a lesion’s background appear to be similar.
PMCID: PMC3046715  PMID: 15827823
CAD; breast density; cancer detection
6.  Effectiveness of Computer-Aided Detection in Community Mammography Practice 
Computer-aided detection (CAD) is applied during screening mammography for millions of US women annually, although it is uncertain whether CAD improves breast cancer detection when used by community radiologists.
We investigated the association between CAD use during film-screen screening mammography and specificity, sensitivity, positive predictive value, cancer detection rates, and prognostic characteristics of breast cancers (stage, size, and node involvement). Records from 684 956 women who received more than 1.6 million film-screen mammograms at Breast Cancer Surveillance Consortium facilities in seven states in the United States from 1998 to 2006 were analyzed. We used random-effects logistic regression to estimate associations between CAD and specificity (true-negative examinations among women without breast cancer), sensitivity (true-positive examinations among women with breast cancer diagnosed within 1 year of mammography), and positive predictive value (breast cancer diagnosed after positive mammograms) while adjusting for mammography registry, patient age, time since previous mammography, breast density, use of hormone replacement therapy, and year of examination (1998–2002 vs 2003–2006). All statistical tests were two-sided.
Of 90 total facilities, 25 (27.8%) adopted CAD and used it for an average of 27.5 study months. In adjusted analyses, CAD use was associated with statistically significantly lower specificity (OR = 0.87, 95% confidence interval [CI] = 0.85 to 0.89, P < .001) and positive predictive value (OR = 0.89, 95% CI = 0.80 to 0.99, P = .03). A non-statistically significant increase in overall sensitivity with CAD (OR = 1.06, 95% CI = 0.84 to 1.33, P = .62) was attributed to increased sensitivity for ductal carcinoma in situ (OR = 1.55, 95% CI = 0.83 to 2.91; P = .17), although sensitivity for invasive cancer was similar with or without CAD (OR = 0.96, 95% CI = 0.75 to 1.24; P = .77). CAD was not associated with higher breast cancer detection rates or more favorable stage, size, or lymph node status of invasive breast cancer.
CAD use during film-screen screening mammography in the United States is associated with decreased specificity but not with improvement in the detection rate or prognostic characteristics of invasive breast cancer.
PMCID: PMC3149041  PMID: 21795668
7.  True Detection Versus “Accidental” Detection of Small Lung Cancer by a Computer-Aided Detection (CAD) Program on Chest Radiographs 
To evaluate the number of actual detections versus “accidental” detections by a computer-aided detection (CAD) system for small nodular lung cancers (≤30 mm) on chest radiographs, using two different criteria for measuring performance. A Food-and-Drug-Administration-approved CAD program (version 1.0; Riverain Medical) was applied to 34 chest radiographs with a “radiologist-missed” nodular cancer and 36 radiographs with a radiologist-mentioned nodule (a newer version 3.0 was also applied to the 36-case database). The marks applied by this CAD system consisted of 5-cm-diameter circles. A strict “nodule-in-center” criterion and a generous “nodule-in-circle” criterion were compared as methods for the calculation of CAD sensitivity. The increased sensitivities by the nodule-in-circle criterion were considered as nodules detected by chance. The number of false-positive (FP) marks was also analyzed. For the 34 radiologist-missed cancers, the nodule-in-circle criterion caused eight more cancers (24%) to be detected by chance, as compared to the nodule-in-center criterion, when using the version 1.0 results. For the 36 radiologist-mentioned nodules, the nodule-in-circle criterion caused seven more lesions (19%) to be detected by chance, as compared to the nodule-in-center criterion, when using the version 1.0 results, and three more lesions (8%) to be detected by chance when using the version 3.0 results. Version 1.0 yielded a mean of six FP marks per image, while version 3.0 yielded only three FP marks per image. The specific criteria used to define true- and false-positive CAD detections can substantially influence the apparent accuracy of a CAD system.
PMCID: PMC3043747  PMID: 19421813
Lung; neoplasms; computer-aided detection; chest radiography
8.  Characterization of Mammographic Masses Based on Level Set Segmentation with New Image Features and Patient Information 
Medical physics  2008;35(1):280-290.
Computer-aided diagnosis (CAD) for characterization of mammographic masses as malignant or benign has the potential to assist radiologists in reducing the biopsy rate without increasing false negatives. The purpose of this study was to develop an automated method for mammographic mass segmentation and explore new image based features in combination with patient information in order to improve the performance of mass characterization. Our previous CAD system, which used the active contour segmentation, and morphological, textural, and spiculation features, has achieved promising results in mass characterization. The new CAD system is based on the level set method, and includes two new types of image features related to the presence of microcalcifications with the mass and abruptness of the mass margin, and patient age. A linear discriminant analysis (LDA) classifier with stepwise feature selection was used to merge the extracted features into a classification score. The classification accuracy was evaluated using the area under the receiver operating characteristic curve. Our primary data set consisted of 427 biopsy-proven masses (200 malignant and 227 benign) in 909 regions of interest (ROIs) (451 malignant and 458 benign) from multiple mammographic views. Leave-one-case-out resampling was used for training and testing. The new CAD system based on the level set segmentation and the new mammographic feature space achieved a view-based Az value of 0.83±0.01. The improvement compared to the previous CAD system was statistically significant (p=0.02). When patient age was included in the new CAD system, view-based and case-based Az values were 0.85±0.01 and 0.87±0.02, respectively. The performance of the new CAD system was also compared to an experienced radiologist’s likelihood of malignancy rating. When patient age was used in classification, the accuracy of the new CAD system was comparable to that of the radiologist (p=0.34). The study also demonstrated the consistency of the newly developed CAD system by evaluating the statistics of the weights of the LDA classifiers in leave-one-case-out classification. Finally, an independent test on the publicly available digital database for screening mammography (DDSM) with 132 benign and 197 malignant ROIs containing masses achieved a view-based Az value of 0.84±0.02.
PMCID: PMC2728555  PMID: 18293583
computer-aided diagnosis; mammography; breast masses; level set; segmentation; classification
9.  Assessing the Standalone Sensitivity of Computer-aided Detection (CADe) with Cancer Cases from the Digital Mammographic Imaging Screening Trial (DMIST) 
AJR. American journal of roentgenology  2012;199(3):W392-W401.
To assess the sensitivities and false detection rates of two CADe systems when applied to digital or screen-film mammograms in detecting the known breast cancer cases from the DMIST breast cancer screening population.
Materials and Methods
Available screen-film and digital mammograms of 161 breast cancer cases from DMIST were analyzed by two CADe systems, iCAD SecondLook (iCAD) and R2 ImageChecker (R2). Three experienced breast imaging radiologists reviewed the CADe marks generated for each available cancer case, recording the number and locations of CADe marks and whether each CADe mark location corresponded with the known location of the cancer.
For the 161 cancer cases included in this study, the sensitivities of the DMIST reader without CAD were 0.43 (69/161, 95% CI 0.35 to 0.51) for digital and 0.41 (66/161, 95% CI 0.33 to 0.49) for film-screen mammography. The sensitivities of iCAD were 0.74 (119/161, 95% CI 0.66 to 0.81) for digital and 0.69 (111/161, 95% CI 0.61 to 0.76) for screen-film mammogram, both significantly higher than the DMIST study sensitivities (p< 0.0001 for both). The average number of false CADe marks per case of iCAD was 2.57 (SD 1.92) for digital and 3.06(SD 1.72) for screen-film mammography. The sensitivity of R2 was 0.74 (119/161, 95% CI 0.66 to 0.81) for digital, and 0.60 (97/161, 95% CI 0.52 to 0.68) for screen-film mammography, both significantly higher than the DMIST study sensitivities (p< 0.0001 for both). The average number of false CADe marks per case of R2 was 2.07 (SD 1.57) for digital and 1.52(SD 1.45) for screen-film mammogram.
Our results suggest the use of CADe in interpretation of digital and screen-film mammograms could lead to improvements in cancer detection.
PMCID: PMC3649852  PMID: 22915432
10.  Who could benefit the most from using a computer-aided detection system in full-field digital mammography? 
The computer-aided detection (CAD) system on mammography has the potential to assist radiologists in breast cancer screening. The purpose of this study is to evaluate the diagnostic performance of the CAD system in full-field digital mammography for detecting breast cancer when used by dedicated breast radiologist (BR) and radiology resident (RR), and to reveal who could benefit the most from a CAD application.
We retrospectively chose 100 image sets from mammographies performed with CAD between June 2008 and June 2010. Thirty masses (15 benign and 15 malignant), 30 microcalcifications (15 benign and 15 malignant), and 40 normal mammography images were included. The participating radiologists consisted of 7 BRs and 13 RRs. We calculated the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for total, normal plus microcalcification and normal plus mass both with and without CAD use for each reader. We compared the diagnostic performance values obtained with and without CAD use for the BR and RR groups, respectively. The reading time reviewing one set of 100 images and time reduction with CAD use for the BR and RR groups were also evaluated.
The diagnostic performance was generally higher in the BR group than in the RR group. Sensitivity improved with CAD use in the BR and RR groups (from 81.10 to 84.29% for BR; 75.38 to 77.95% for RR). A tendency for improvement in all diagnostic performance values was observed in the BR group, whereas in the RR group, sensitivity improved but specificity, PPV, and NPV did not. None of the diagnostic performance parameters were significantly different. The mean reading time was shortened with CAD use in both the BR and RR groups (111.6 minutes to 94.3 minutes for BR; 135.5 minutes to 109.8 minutes for RR). The mean time reduction was higher for the RR than that in the BR group.
CAD was helpful for dedicated BRs to improve their diagnostic performance and for RRs to improve the sensitivity in a screening setting. CAD could be essential for radiologists by decreasing reading time without decreasing diagnostic performance.
PMCID: PMC4046038  PMID: 24885214
Computer-aided detection (CAD); Mammography; Screening
11.  CT Colonography Computer-Aided Polyp Detection: Effect on Radiologist Observers of Polyp Identification by CAD on Both the Supine and Prone Scans 
Academic radiology  2010;17(8):948-959.
To determine whether the display of computer-aided detection (CAD) marks on individual polyps on both the supine and prone scans leads to improved polyp detection by radiologists compared to the display of CAD marks on individual polyps on either the supine or the prone scan, but not both.
The acquisition of patient data for this study was approved by the Institutional Review Board and was HIPAA-compliant. Subsequently, the use of the data was declared exempt from further IRB review. Four radiologists interpreted 33 CT colonography cases, 21 of which had one adenoma 6 to 9 mm in size, with the assistance of a CAD system in the first reader mode, i.e., the radiologists reviewed only the CAD marks. The radiologists were shown each case twice, with different sets of CAD marks for each of the two readings. In one reading a true positive CAD mark for the same polyp was displayed on both the supine and prone scans (a double-mark reading). In the other reading a true positive CAD mark was displayed either on the supine or prone scan but not both (a single-mark reading). True positive marks were randomized between readings and there was at least a one-month delay between readings to minimize recall bias. Sensitivity and specificity were determined and receiver operating characteristic (ROC) and multiple-reader multiple-case analyses were performed.
The average per polyp sensitivities were 60% [38%, 81%] vs. 71% [52%, 91%] (p =.03) for single-mark and double mark readings, respectively. The areas [95% confidence intervals] under the ROC curves were 0.76 [0.62, 0.88] and 0.79 [0.58, 0.96], respectively (p=NS). Specificities were similar for the single-mark compared to the double-mark readings.
The display of CAD marks on a polyp on both the supine and prone scans led to more frequent detection of polyps by radiologists without adversely affecting specificity for detecting 6–9 mm adenomas.
PMCID: PMC2898513  PMID: 20542452
CT, colon; CT, virtual imaging; Colon cancer; image processing; automated detection; observer performance
12.  Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume 
European Radiology  2012;22(10):2076-2084.
To evaluate performance of computer-aided detection (CAD) beyond double reading for pulmonary nodules on low-dose computed tomography (CT) by nodule volume.
A total of 400 low-dose chest CT examinations were randomly selected from the NELSON lung cancer screening trial. CTs were evaluated by two independent readers and processed by CAD. A total of 1,667 findings marked by readers and/or CAD were evaluated by a consensus panel of expert chest radiologists. Performance was evaluated by calculating sensitivity of pulmonary nodule detection and number of false positives, by nodule characteristics and volume.
According to the screening protocol, 90.9 % of the findings could be excluded from further evaluation, 49.2 % being small nodules (less than 50 mm3). Excluding small nodules reduced false-positive detections by CAD from 3.7 to 1.9 per examination. Of 151 findings that needed further evaluation, 33 (21.9 %) were detected by CAD only, one of them being diagnosed as lung cancer the following year. The sensitivity of nodule detection was 78.1 % for double reading and 96.7 % for CAD. A total of 69.7 % of nodules undetected by readers were attached nodules of which 78.3 % were vessel-attached.
CAD is valuable in lung cancer screening to improve sensitivity of pulmonary nodule detection beyond double reading, at a low false-positive rate when excluding small nodules.
Key Points
• Computer-aided detection (CAD) has known advantages for computed tomography (CT).
• Combined CAD/nodule size cut-off parameters assist CT lung cancer screening.
• This combination improves the sensitivity of pulmonary nodule detection by CT.
• It increases the positive predictive value for cancer detection.
PMCID: PMC3431468  PMID: 22814824
Computer-aided detection; Multi-detector computed tomography; Pulmonary nodules; Low dose; Volumetry
13.  Lung Nodule Detection on Chest CT: Evaluation of a Computer-Aided Detection (CAD) System 
Korean Journal of Radiology  2005;6(2):89-93.
To evaluate the capacity of a computer-aided detection (CAD) system to detect lung nodules in clinical chest CT.
Materials and Methods
A total of 210 consecutive clinical chest CT scans and their reports were reviewed by two chest radiologists and 70 were selected (33 without nodules and 37 with 1-6 nodules, 4-15.4 mm in diameter). The CAD system (ImageChecker® CT LN-1000) developed by R2 Technology, Inc. (Sunnyvale, CA) was used. Its algorithm was designed to detect nodules with a diameter of 4-20 mm. The two chest radiologists working with the CAD system detected a total of 78 nodules. These 78 nodules form the database for this study. Four independent observers interpreted the studies with and without the CAD system.
The detection rates of the four independent observers without CAD were 81% (63/78), 85% (66/78), 83% (65/78), and 83% (65/78), respectively. With CAD their rates were 87% (68/78), 85% (66/78), 86% (67/78), and 85% (66/78), respectively. The differences between these two sets of detection rates did not reach statistical significance. In addition, CAD detected eight nodules that were not mentioned in the original clinical radiology reports. The CAD system produced 1.56 false-positive nodules per CT study. The four test observers had 0, 0.1, 0.17, and 0.26 false-positive results per study without CAD and 0.07, 0.2, 0.23, and 0.39 with CAD, respectively.
The CAD system can assist radiologists in detecting pulmonary nodules in chest CT, but with a potential increase in their false positive rates. Technological improvements to the system could increase the sensitivity and specificity for the detection of pulmonary nodules and reduce these false-positive results.
PMCID: PMC2686425  PMID: 15968147
Lung nodule detection; Computed tomography (CT); Computer-aided detection
14.  Improving Performance of Computer-Aided Detection of Masses by Incorporating Bilateral Mammographic Density Asymmetry: An Assessment 
Academic Radiology  2011;19(3):303-310.
Rationale and Objectives
Bilateral mammographic density asymmetry is a promising indicator in assessing risk of having or developing breast cancer. This study aims to assess the performance improvement of a computer-aided detection (CAD) scheme in detecting masses by incorporating bilateral mammographic density asymmetrical information.
Materials and Methods
A testing dataset containing 2400 full-field digital mammograms (FFDM) acquired from 600 examination cases was established. Among them, 300 are positive cases with verified cancer associated with malignant masses and 300 are negative cases. Two computerized schemes were applied to process images of each case. The first single-image based CAD scheme detected suspicious mass regions and the second scheme computed average and difference of mammographic tissue density depicted between the left and right breast. A fusion method based on rotation of the CAD scoring projection reference axis was then applied to combine CAD-generated mass detection scores and either the computed average or difference (asymmetry) of bilateral mammographic density scores. The CAD performance levels with and without incorporating mammographic density information were evaluated and compared using a free-response receiver operating characteristic (FROC) type data analysis method.
CAD achieved a case-based mass detection sensitivity of 0.74 and a region-based sensitivity of 0.56 at a false-positive rate of 0.25 per image. By fusing the CAD and bilateral mammographic density asymmetry scores, the case-based and region-based sensitivity levels of the CAD scheme were increased to 0.84 and 0.69, respectively, at the same false-positive rate. Fusion with average mammographic density only slightly increased CAD sensitivity to 0.75 (case-based) and 0.57 (region-based).
This study indicated that (1) bilateral mammographic density asymmetry was a stronger indicator of the case depicting suspicious masses than the average density computed from two breasts and (2) fusion between the conventional CAD scores and bilateral mammographic density asymmetry information could substantially increase CAD performance in mass detection.
PMCID: PMC3274572  PMID: 22173323
Breast cancer; computer-aided detection of mammograms; mammographic density
15.  Computer-aided detection of breast masses on full field digital mammograms 
Medical physics  2005;32(9):2827-2838.
We are developing a computer-aided detection (CAD) system for breast masses on full field digital mammographic (FFDM) images. To develop a CAD system that is independent of the FFDM manufacturer’s proprietary preprocessing methods, we used the raw FFDM image as input and developed a multiresolution preprocessing scheme for image enhancement. A two-stage prescreening method that combines gradient field analysis with gray level information was developed to identify mass candidates on the processed images. The suspicious structure in each identified region was extracted by clustering-based region growing. Morphological and spatial gray-level dependence texture features were extracted for each suspicious object. Stepwise linear discriminant analysis (LDA) with simplex optimization was used to select the most useful features. Finally, rule-based and LDA classifiers were designed to differentiate masses from normal tissues. Two data sets were collected: a mass data set containing 110 cases of two-view mammograms with a total of 220 images, and a no-mass data set containing 90 cases of two-view mammograms with a total of 180 images. All cases were acquired with a GE Senographe 2000D FFDM system. The true locations of the masses were identified by an experienced radiologist. Free-response receiver operating characteristic analysis was used to evaluate the performance of the CAD system. It was found that our CAD system achieved a case-based sensitivity of 70%, 80%, and 90% at 0.72, 1.08, and 1.82 false positive (FP) marks/image on the mass data set. The FP rates on the no-mass data set were 0.85, 1.31, and 2.14 FP marks/image, respectively, at the corresponding sensitivities. This study demonstrated the usefulness of our CAD techniques for automated detection of masses on FFDM images.
PMCID: PMC2742215  PMID: 16266097
computer-aided detection; full field digital mammogram (FFDM); multiresolution image enhancement; gradient field analysis; stepwise linear discriminant analysis
16.  Impact of Computer-Aided Detection Systems on Radiologist Accuracy With Digital Mammography 
The purpose of this study was to assess the impact of computer-aided detection (CAD) systems on the performance of radiologists with digital mammograms acquired during the Digital Mammographic Imaging Screening Trial (DMIST).
Only those DMIST cases with proven cancer status by biopsy or 1-year follow-up that had available digital images were included in this multireader, multicase ROC study. Two commercially available CAD systems for digital mammography were used: iCAD SecondLook, version 1.4; and R2 ImageChecker Cenova, version 1.0. Fourteen radiologists interpreted, without and with CAD, a set of 300 cases (150 cancer, 150 benign or normal) on the iCAD SecondLook system, and 15 radiologists interpreted a different set of 300 cases (150 cancer, 150 benign or normal) on the R2 ImageChecker Cenova system.
The average AUC was 0.71 (95% CI, 0.66–0.76) without and 0.72 (95% CI, 0.67–0.77) with the iCAD system (p = 0.07). Similarly, the average AUC was 0.71 (95% CI, 0.66–0.76) without and 0.72 (95% CI 0.67–0.77) with the R2 system (p = 0.08). Sensitivity and specificity differences without and with CAD for both systems also were not significant.
Radiologists in our studies rarely changed their diagnostic decisions after the addition of CAD. The application of CAD had no statistically significant effect on radiologist AUC, sensitivity, or specificity performance with digital mammograms from DMIST.
PMCID: PMC4286296  PMID: 25247960
AUC; computer-aided detection; mammography; sensitivity; specificity
17.  An Interactive System for Computer-Aided Diagnosis of Breast Masses 
Journal of Digital Imaging  2012;25(5):570-579.
Although mammography is the only clinically accepted imaging modality for screening the general population to detect breast cancer, interpreting mammograms is difficult with lower sensitivity and specificity. To provide radiologists “a visual aid” in interpreting mammograms, we developed and tested an interactive system for computer-aided detection and diagnosis (CAD) of mass-like cancers. Using this system, an observer can view CAD-cued mass regions depicted on one image and then query any suspicious regions (either cued or not cued by CAD). CAD scheme automatically segments the suspicious region or accepts manually defined region and computes a set of image features. Using content-based image retrieval (CBIR) algorithm, CAD searches for a set of reference images depicting “abnormalities” similar to the queried region. Based on image retrieval results and a decision algorithm, a classification score is assigned to the queried region. In this study, a reference database with 1,800 malignant mass regions and 1,800 benign and CAD-generated false-positive regions was used. A modified CBIR algorithm with a new function of stretching the attributes in the multi-dimensional space and decision scheme was optimized using a genetic algorithm. Using a leave-one-out testing method to classify suspicious mass regions, we compared the classification performance using two CBIR algorithms with either equally weighted or optimally stretched attributes. Using the modified CBIR algorithm, the area under receiver operating characteristic curve was significantly increased from 0.865 ± 0.006 to 0.897 ± 0.005 (p < 0.001). This study demonstrated the feasibility of developing an interactive CAD system with a large reference database and achieving improved performance.
PMCID: PMC3447094  PMID: 22234836
Computer-aided detection and diagnosis (CAD); Content-based image retrieval (CBIR); Breast cancer; Mammograms
18.  Multi-Detector Computed Tomography Angiography for Coronary Artery Disease 
Executive Summary
Computed tomography (CT) scanning continues to be an important modality for the diagnosis of injury and disease, most notably for indications of the head and abdomen. (1) According to a recent report published by the Canadian Institutes of Health Information, (1) there were about 10.3 scanners per million people in Canada as of January 2004. Ontario had the fewest number of CT scanners per million compared to the other provinces (8 CT scanners per million). The wait time for CT in Ontario of 5 weeks approaches the Canadian median of 6 weeks.
This health technology and policy appraisal systematically reviews the published literature on multidetector CT (MDCT) angiography as a diagnostic tool for the newest indication for CT, coronary artery disease (CAD), and will apply the results of the review to current health care practices in Ontario. This review does not evaluate MDCT to detect coronary calcification without contrast medium for CAD screening purposes.
The Technology
Compared with conventional CT scanning, MDCT can provide smaller pieces of information and can cover a larger area faster. (2) Advancing MDCT technology (8, 16, 32, 64 slice systems) is capable of producing more images in less time. For general CT scanning, this faster capability can reduce the time that patients must stay still during the procedure, thereby reducing potential movement artefact. However, the additional clinical utility of images obtained from faster scanners compared to the images obtained from conventional CT scanners for current CT indications (i.e., non-moving body parts) is not known.
There are suggestions that the new fast scanners can reduce wait times for general CT. MDCT angiography that utilizes a contrast medium, has been proposed as a minimally invasive replacement to coronary angiography to detect coronary artery disease. MDCT may take between 15 to 45 minutes; coronary angiography may take up to 1 hour.
Although 16-slice and 32-slice CT scanners have been available for a few years, 64-slice CT scanners were released only at the end of 2004.
Review Strategy
There are many proven, evidence-based indications for conventional CT. It is not clear how MDCT will add to the clinical utility and management of patients for established CT indications. Therefore, because cardiac imaging, specifically MDCT angiography, is a new indication for CT, this literature review focused on the safety, effectiveness, and cost-effectiveness of MDCT angiography compared with coronary angiography in the diagnosis and management of people with CAD.
This review asked the following questions:
Is the most recent MDCT angiography effective in the imaging of the coronary arteries compared with conventional angiography to correctly diagnose of significant (> 50% lumen reduction) CAD?
What is the utility of MDCT angiography in the management and treatment of patients with CAD?
How does MDCT angiography in the management and treatment of patients with CAD affect longterm outcomes?
The published literature from January 2003 to January 31, 2005 was searched for articles that focused on the detection of coronary artery disease using 16-slice CT or faster, compared with coronary angiography. The search yielded 138 articles; however, 125 were excluded because they did not meet the inclusion criteria (comparison with coronary angiography, diagnostic accuracy measures calculated, and a sample size of 20 or more). As screening for CAD is not advised, studies that utilized MDCT for this purpose or studies that utilized MDCT without contrast media were also excluded. Overall, 13 studies were included in this review.
Summary of Findings
The published literature focused on 16-slice CT angiography for the detection of CAD. Two abstracts that were presented at the 2005 European Congress of Radiology meeting in Vienna compared 64-slice CT angiography with coronary angiography.
The 13 studies focussing on 16-slice CT angiography were stratified into 2 groups: Group 1 included 9 studies that focused on the detection of CAD in symptomatic patients, and Group 2 included 4 studies that examined the use of 16-slice CT angiography to detect disease progression after cardiac interventions. The 2 abstracts on 64-slice CT angiography were presented separately, but were not critically appraised due to the lack of information provided in the abstracts.
16-Slice Computed Tomography Angiography
The STARD initiative to evaluate the reporting quality of studies that focus on diagnostic tests was used. Overall the studies were relatively small (fewer than 100 people), and only about one-half recruited consecutive patients. Most studies reported inclusion criteria, but 5 did not report exclusion criteria. In these 5, the patients were highly selected; therefore, how representative they are of the general population of people with suspicion if CAD or those with disease progression after cardiac intervention is questionable. In most studies, patients were either already taking, or were given, β-blockers to reduce their heart rates to improve image quality sufficiently. Only 6 of the 13 studies reported interobserver reliability quantitatively. The studies typically assessed the quality of the images obtained from 16-slice CT angiography, excluded those of poor quality, and compared the rest with the gold standard, coronary angiography. This practice necessarily inflated the diagnostic accuracy measures. Only 3 studies reported confidence intervals around their measures.
Evaluation of the studies in Group 1 reported variable sensitivity, from just over 60% to 96%, but a more stable specificity, at more than 95%. The false positive rate ranged from 5% to 8%, but the false negative rate was at best under 10% and at worst about 30%. This means that up to one-third of patients who have disease may be missed. These patients may therefore progress to a more severe level of disease and require more invasive procedures. The calculated positive and negative likelihood ratios across the studies suggested that 16-slice CT angiography may be useful to detect disease, but it is not useful to rule out disease. The prevalence of disease, measured by conventional coronoary angiography, was from 50% to 80% across the studies in this review. Overall, 16-slice CT angiography may be useful, but there is no conclusive evidence to suggest that it is equivalent to or better than coronary angiography to detect CAD in symptomatic patients.
In the 4 studies in Group 2, sensitivity and specificity were both reported at more than 95% (except for 1 that reported sensitivity of about 80%). The positive and negative likelihood ratios suggested that the test might be useful to detect disease progression in patients who had cardiac interventions. However, 2 of the 4 studies recruited patients who had been asymptomatic since their intervention. As many of the patients studied were not symptomatic, the relevance of performing MDCT angiography in the patient population may be in question.
64-Slice Computed Tomography Angiography
An analysis from the interim results based on 2 abstracts revealed that 64-slice CT angiography was insufficient compared to coronary angiography and may not be better than 16-slice CT angiography to detect CAD.
Cardiac imaging is a relatively new indication for CT. A systematic review of the literature was performed from 2003 to January 2005 to determine the effectiveness of MDCT angiography (16-slice and 64-slice) compared to coronary angiography to detect CAD. At the time of this report, there was no published literature on 64-slice CT for any indications.
Based on this review, the Medical Advisory Secretariat concluded that there is insufficient evidence to suggest that 16-slice or 64-slice CT angiography is equal to or better than coronary angiography to diagnose CAD in people with symptoms or to detect disease progression in patients who had previous cardiac interventions. An analysis of the evidence suggested that in investigating suspicion of CAD, a substantial number of patients would be missed. This means that these people would not be appropriately treated. These patients might progress to more severe disease and possibly more adverse events. Overall, the clinical utility of MDCT in patient management and long-term outcomes is unknown.
Based on the current evidence, it is unlikely that CT angiography will replace coronary angiography completely, but will probably be used adjunctively with other cardiac diagnostic tests until more definitive evidence is published.
If multi-slice CT scanners are used for coronary angiography in Ontario, access to the current compliment of CT scanners will necessarily increase wait times for general CT scanning. It is unlikely that these newer-generation scanners will improve patient throughput, despite the claim that they are faster.
Screening for CAD in asymptomatic patients and who have no history of ischemic heart disease using any modality is not advised, based on the World Health Organization criteria for screening. Therefore, this review did not examine the use of multi-slice CT for this purpose.
PMCID: PMC3382628  PMID: 23074474
19.  Using computer-aided detection in mammography as a decision support 
European Radiology  2010;20(10):2323-2330.
To evaluate an interactive computer-aided detection (CAD) system for reading mammograms to improve decision making.
A dedicated mammographic workstation has been developed in which readers can probe image locations for the presence of CAD information. If present, CAD findings are displayed with the computed malignancy rating. A reader study was conducted in which four screening radiologists and five non-radiologists participated to study the effect of this system on detection performance. The participants read 120 cases of which 40 cases had a malignant mass that was missed at the original screening. The readers read each mammogram both with and without CAD in separate sessions. Each reader reported localized findings and assigned a malignancy score per finding. Mean sensitivity was computed in an interval of false-positive fractions less than 10%.
Mean sensitivity was 25.1% in the sessions without CAD and 34.8% in the CAD-assisted sessions. The increase in detection performance was significant (p = 0.012). Average reading time was 84.7 ± 61.5 s/case in the unaided sessions and was not significantly higher when interactive CAD was used (85.9 ± 57.8 s/case).
Interactive use of CAD in mammography may be more effective than traditional CAD for improving mass detection without affecting reading time.
PMCID: PMC2940044  PMID: 20532890
Mammography; Breast; Early detection of cancer; Decision making; Computer-assisted; Radiographic image interpretation
20.  Improving Performance of Computer-Aided Detection Scheme by Combining Results from Two Machine Learning Classifiers 
Academic radiology  2009;16(3):266-274.
Rationale and Objectives
Global data and local instance based machine learning methods and classifiers have been widely used to optimize computer-aided detection (CAD) schemes to classify between true-positive and false-positive detections. In this study the authors investigated the correlation between these two types of classifiers using a new independent testing dataset and assessed the potential improvement of a CAD scheme performance by combining the results of the two classifiers in detecting breast masses.
Materials and Methods
The CAD scheme first used image filtering and a multi-layer topographic region growth algorithm to detect and segment suspicious mass regions. The scheme then used an image feature based classifier to classify these regions into true-positive and false-positive regions. Two classifiers were used in this study. One was a global data based machine learning classifier, an artificial neural network (ANN), and the other one was a local instance based machine learning classifier, a k-nearest neighbor (KNN) algorithm. An independent image database involving 400 mammography examinations was used in this study. Among them, 200 were cancer cases and 200 were negative cases. The pre-optimized CAD scheme was applied twice to the database using the two different classifiers. The correlation between the two sets of classification results was analyzed. Three sets of CAD performances using the ANN, KNN, and average detection scores from both classifiers were assessed and compared using the free-response receiver operating characteristics (FROC) method.
The results showed that the ANN achieved higher performance than the KNN with a normalized area under the performance curve (AUC) of 0.891 versus 0.845. The correlation coefficients between the detection scores generated by the two classifiers were 0.436 and 0.161 for the true-positive and false-positive detections, respectively. The average detection scores of the two classifiers improved CAD performance and reliability by increasing AUC to 0.912 and reducing the standard error of the estimated AUC by 14.4%. The detection sensitivity was also increased from 75.8% (ANN) and 65.9% (KNN) to 80.3% at the false-positive detection rate of 0.3 per image.
The study demonstrates that the global and local data based machine learning classifiers (ANN and KNN) generate low correlated detection results and combining the detection scores of these two classifiers can significantly improve overall CAD performance (p < 0.01) and reduce standard error in CAD performance assessment.
PMCID: PMC2675918  PMID: 19201355
Computer-aided diagnosis; Performance assessment; Machine learning; Mass detection
21.  Impact of a Computer-Aided Detection (CAD) System Integrated into a Picture Archiving and Communication System (PACS) on Reader Sensitivity and Efficiency for the Detection of Lung Nodules in Thoracic CT Exams 
Journal of Digital Imaging  2012;25(6):771-781.
The objective of this study is to assess the impact on nodule detection and efficiency using a computer-aided detection (CAD) device seamlessly integrated into a commercially available picture archiving and communication system (PACS). Forty-eight consecutive low-dose thoracic computed tomography studies were retrospectively included from an ongoing multi-institutional screening study. CAD results were sent to PACS as a separate image series for each study. Five fellowship-trained thoracic radiologists interpreted each case first on contiguous 5 mm sections, then evaluated the CAD output series (with CAD marks on corresponding axial sections). The standard of reference was based on three-reader agreement with expert adjudication. The time to interpret CAD marking was automatically recorded. A total of 134 true-positive nodules, measuring 3 mm and larger were included in our study; with 85 ≥ 4 and 50 ≥ 5 mm in size. Readers detection improved significantly in each size category when using CAD, respectively, from 44 to 57 % for ≥3 mm, 48 to 61 % for ≥4 mm, and 44 to 60 % for ≥5 mm. CAD stand-alone sensitivity was 65, 68, and 66 % for nodules ≥3, ≥4, and ≥5 mm, respectively, with CAD significantly increasing the false positives for two readers only. The average time to interpret and annotate a CAD mark was 15.1 s, after localizing it in the original image series. The integration of CAD into PACS increases reader sensitivity with minimal impact on interpretation time and supports such implementation into daily clinical practice.
PMCID: PMC3491162  PMID: 22710985
Radiographic image interpretation; Computer-assisted; Radiography; Thoracic; PACS reading; Clinical workflow; Lung; Efficiency; Computed tomography; Computer-assisted detection; Chest CT
22.  Computer-Aided Detection – The Effect of Training Databases on Detection of Subtle Breast Masses 
Academic radiology  2010;17(11):1401-1408.
Rationale and Objective
Lesion conspicuity is typically highly correlated with visual difficulty for lesion detection and computer-aided detection (CAD) has been widely used as a “second reader” in mammography. Hence, increasing CAD sensitivity in detecting subtle cancers without increasing false-positive rates is important. This study investigates the effect of training database case selection on CAD performance in detecting low conspicuity breast masses.
Materials and Methods
A full-field digital mammography image database that includes 525 cases depicting malignant masses was randomly partitioned into three subsets. A CAD scheme was applied to detect all initially suspected mass regions and compute region conspicuity. We iteratively selected training samples from two of the subsets. Four types of training datasets, namely; (1) one including all available true-positive mass regions in the two subsets (termed here “All”); (2) one including 350 randomly selected mass regions (“diverse”); (3) one including 350 high conspicuity mass regions (“easy”); and (4) one including 350 low conspicuity mass regions (“difficult”), were assembled. In each training dataset the same number of randomly selected false-positive regions as the true-positives was also included. Two classifiers, an artificial neural network (ANN) and a k-nearest neighbor algorithm (KNN), were trained using each of the four training datasets and tested on all suspected regions in the remaining dataset. Using a 3-fold cross-validation method, we computed and compared the performance changes of the CAD schemes trained using one of the four training datasets.
CAD initially detected 1025 true-positive mass regions depicted on 507 cases (97% case-based sensitivity) and 9569 false-positive regions (3.5 per image) in the entire database. Using the “All” training dataset, CAD achieved the highest overall performance on the entire testing database. However, CAD detected the highest number of low conspicuity masses when the “difficult” training dataset was used for training. Results did concord for both ANN and KNN based classifiers in all tests. Compared with the use of the “All” training dataset, sensitivity of the schemes trained using the “difficult” dataset decreased by 8.6% and 8.4% for ANN and KNN on the entire database, respectively, but the detection of low conspicuity masses increased by 7.1% and 15.1% for ANN and KNN at a false-positive rate of 0.3 per image.
CAD performance depends on the size, diversity, and “difficulty” level of the training database. To increase CAD sensitivity in detecting subtle cancer, one should increase the fraction of “difficult” cases in the training database rather than simply increase the training dataset size.
PMCID: PMC2952663  PMID: 20650667
Computer-aided detection (CAD); Full-field digital mammography (FFDM); Image databases; Performance assessment
23.  Computer-aided Detection of Breast Masses Depicted on Full-Field Digital Mammograms: A Performance Assessment 
The British Journal of Radiology  2011;85(1014):e153-e161.
We investigated the feasibility of converting a computer-aided detection (CAD) scheme for digitized screen-film mammograms to full-field digital mammograms (FFDM) and assessing CAD performance on a large database that included 6478 FFDM images acquired on 1120 women with 525 cancer and 595 negative cases. The database was divided into five case groups: (1) cancer detected during screening, (2) interval cancers, (3) “high-risk” recommended for surgical excision, (4) recalled but negative, and (5) screening negative (not-recalled). A previously developed CAD scheme for masses depicted on digitized images was converted and re-optimized for FFDM images while keeping the same image processing structure. CAD performance was analyzed on the entire database. The case-based CAD sensitivity was 75.6% (397/525) for the “current” mammograms and 40.8% (42/103) for the “prior” mammograms deemed negative during clinical interpretation but “visible” during retrospective review. The region-based CAD sensitivity was 58.1% (618/1064) for the “current” mammograms and 28.4% (57/201) for the “prior” mammograms. The CAD scheme marked 55.7% (221/397) and 35.7% (15/42) of the masses on both views of the “current” and the “prior” examinations, respectively. The overall CAD-cued false-positive rate was 0.32 per image ranging from 0.29 to 0.51 for the five case groups. This study suggests that (1) digitized image based CAD can be converted for FFDM while performing at a comparable, or better, level, (2) CAD detects a substantial fraction of cancers depicted on “prior” examinations, albeit most were marked only on one view, and (3) CAD tends to mark more false-positives on “difficult” negative cases that are more visually difficult for radiologists to interpret.
PMCID: PMC3120913  PMID: 21343322
24.  Strategies for Improved Interpretation of Computer-Aided Detections for CT Colonography Utilizing Distributed Human Intelligence 
Medical image analysis  2012;16(6):1280-1292.
Computer-aided detection (CAD) systems have been shown to improve the diagnostic performance of CT colonography (CTC) in the detection of premalignant colorectal polyps. Despite the improvement, the overall system is not optimal. CAD annotations on true lesions are incorrectly dismissed, and false positives are misinterpreted as true polyps. Here, we conduct an observer performance study utilizing distributed human intelligence in the form of anonymous knowledge workers (KWs) to investigate human performance in classifying polyp candidates under different presentation strategies. We evaluated 600 polyp candidates from 50 patients, each case having at least one polyp • 6 mm, from a large database of CTC studies. Each polyp candidate was labeled independently as a true or false polyp by 20 KWs and an expert radiologist. We asked each labeler to determine whether the candidate was a true polyp after looking at a single 3D-rendered image of the candidate and after watching a video fly-around of the candidate. We found that distributed human intelligence improved significantly when presented with the additional information in the video fly-around. We noted that performance degraded with increasing interpretation time and increasing difficulty, but distributed human intelligence performed better than our CAD classifier for “easy” and “moderate” polyp candidates. Further, we observed numerous parallels between the expert radiologist and the KWs. Both showed similar improvement in classification moving from single-image to video interpretation. Additionally, difficulty estimates obtained from the KWs using an expectation maximization algorithm correlated well with the difficulty rating assigned by the expert radiologist. Our results suggest that distributed human intelligence is a powerful tool that will aid in the development of CAD for CTC.
PMCID: PMC3443285  PMID: 22705287
Computed tomography colonography; observer performance study; crowdsourcing; distributed human intelligence; video analysis; labeling modeling
25.  Computer-Aided Detection of Malignant Lung Nodules on Chest Radiographs: Effect on Observers' Performance 
Korean Journal of Radiology  2012;13(5):564-571.
To evaluate the effect of computer-aided detection (CAD) system on observer performance in the detection of malignant lung nodules on chest radiograph.
Materials and Methods
Two hundred chest radiographs (100 normal and 100 abnormal with malignant solitary lung nodules) were evaluated. With CT and histological confirmation serving as a reference, the mean nodule size was 15.4 mm (range, 7-20 mm). Five chest radiologists and five radiology residents independently interpreted both the original radiographs and CAD output images using the sequential testing method. The performances of the observers for the detection of malignant nodules with and without CAD were compared using the jackknife free-response receiver operating characteristic analysis.
Fifty-nine nodules were detected by the CAD system with a false positive rate of 1.9 nodules per case. The detection of malignant lung nodules significantly increased from 0.90 to 0.92 for a group of observers, excluding one first-year resident (p = 0.04). When lowering the confidence score was not allowed, the average figure of merit also increased from 0.90 to 0.91 (p = 0.04) for all observers after a CAD review. On average, the sensitivities with and without CAD were 87% and 84%, respectively; the false positive rates per case with and without CAD were 0.19 and 0.17, respectively. The number of additional malignancies detected following true positive CAD marks ranged from zero to seven for the various observers.
The CAD system may help improve observer performance in detecting malignant lung nodules on chest radiographs and contribute to a decrease in missed lung cancer.
PMCID: PMC3435853  PMID: 22977323
Computer-aided detection; Lung nodules; Lung cancer; Chest radiograph

Results 1-25 (1386219)