|Home | About | Journals | Submit | Contact Us | Français|
Computerized determination of optimal search areas on mammograms for matching breast mass regions depicted on two ispilateral views remains a challenge for developing multi-view based computer-aided detection (CAD) schemes. The purpose of this study is to compare three methods aimed to match CAD-cued mass regions depicted on two views and the associated impact on CAD performance.
The three search methods use (1) an annular (fan-shaped) band, (2) a straight strip perpendicular to the estimated centerline, and (3) a mixed search area bound on the chest wall side by a straight line and an annular arc on the nipple side, respectively. An image database of 200 positive examinations depicting the masses on two views and 200 negative examinations was used for testing. Two performance assessment experiments were conducted. The first one investigated the maximum matching sensitivity as the function of the search area size and the second one assessed the change of CAD performance using these three search methods.
In order to include all 200 paired mass regions within the search areas, maximum widths were 28mm and 68mm for the use of straight strip and the annular band search methods, respectively. When applying a single-image based CAD scheme to this image database, 172 masses (86% sensitivity) and 523 false-positive (FP) regions (0.33 per image) were detected and cued. Among the positive findings 92 were cued by the CAD on both views and 80 were cued only on one view. In an attempt to match as many of the 172 CAD-cued masses (TP) on two views by incrementally reducing the CAD threshold inside the different search areas, CAD scheme generated 158 TP-TP paired matches with 14 TP-FP paired matches, 142 TP-TP paired matches with 30 TP-FP paired matches, and 146 TP-TP paired matches with 26 TP-FP paired matches, using the methods involving the straight strips, the annular bands, and the mixed search areas, respectively. Using the straight strip search method, the CAD also eliminated 25% of FP regions initially cued by the single-image based CAD scheme and generated the lowest case-based FP detection rate, namely 15% less than that generated by the annular band method.
The study showed that among these three search methods, the straight strip method required a smaller search area while achieving the highest level of CAD performance.
Single-image based computer-aided detection (CAD) systems for mammograms in which the most of subtle cancers are cued only on one view  and therefore are more likely to be discarded by radiologists as false-positive cues [2-4] may be limited in their ability to improve radiologists’ performance [5, 6]. Hence, a number of research groups (including ours) have been attempting to develop multi-view based CAD schemes [7-14] that enable to incorporate the related information regarding abnormalities depicted on two ipsilateral (i.e., cranio-caudal (CC) and medio-lateral oblique (MLO)) views into the decision making process of the schemes. Similarly, this information can also be used to improve CAD performance in terms of maximizing the number of paired depicted and cued masses. The first step needed in these multi-view based CAD schemes is the matching (pairing) of detected suspicious regions depicting the same abnormality (i.e., mass) on two views. For this purpose, the scheme has to detect either one or two fiducial landmarks (e.g., the nipple and/or the chest wall) to enable a “global” registration of the paired images in question by establishing a reference matching coordinates for both views. Once a suspicious mass region is detected on one view (“a queried region”), the scheme identifies a search area on the corresponding ipsilateral view and searches for a CAD-detected suspicious region within this area that is considered “the most similar” to the queried region. These two “matched” regions are defined as a “pair” of mass regions. Under the assumption that true-positive mass regions have a higher probability than false-positive detections to be correctly matched on two views , the resulting overall performance of multi-view based CAD schemes may improve by keeping (cuing) “correctly matched” suspicious mass regions depicted on two views while discarding all un-matched regions as false-positive identifications . However, some true subtle masses are only cued on one view by the single-image based CAD schemes for several reasons including but not limited to the different surrounding background depicted on the two projection views in question. As a result, the correlation between two subtle mass regions depicted on two views is often low, which may generate higher rate of mismatching (i.e., true-positive and false-positive pairs). Previous studies have shown that in order to achieve a similar case-based detection sensitivity to the single image based CAD scheme while cueing the majority of detected masses on both views, current multi-view based CAD schemes typically generate a substantially larger false-positive detection rates [10-13].
The goal of most investigations on multi-view based CAD schemes is to increase the number of subtle masses being cued on both views while maintaining a comparable or preferably a reduced case-based false-positive detection rate as a single image based CAD scheme. It is well known that when soft tissue as breast is compressed, it is not likely to be compressed the same manner under the two views; hence registration methods based on either rigid or non-rigid geometrical relationship between specific breast tissues are not exact and frequently do not yield high accuracy. Because of the non-linear deformation that occurs during breast compression which is also largely patient specific, developing optimal methods to define a minimum search area that can adequately cover possible matched mass regions depicted on two views without generating a high false-positive rate, remains a challenge. Similar to other groups we have been basing our work  on the facts that areas close to the chest wall are generally more bound (confined) and the nipple area is “pushed” or displaced forward during compression. Therefore, among the many possible approaches to defining the search areas, including non-linear deformations, there are primarily three relatively simple to implement approaches that should be considered for this purpose. The first one assumes that the nipple remains the pivot point under compression resulting in an annular (arc based fan-shaped) band type search area. The second uses the chest wall as a “fixed” reference resulting in a straight line based strip type search area that is parallel to the chest wall but at a given distance to the chest wall or the nipple on the centerline. The third approach is a mixed method that combines the previous two by using both the “nipple” based approach and the “chest wall” based reference, resulting in a search area bound by a straight line on the posterior side (chest wall side) and a concave (arc) shape in the front (nipple side). The first two search methods had been previously applied in developing multi-view based CAD schemes [10-14]. However, there are no data comparing the impact, if any, of the selected approach on CAD performance. We compared these three methods for computing the search area for matched mass regions in terms of their impact on performance of a multi-view based CAD scheme using a common image database. The approach taken was to assess the maximum matching sensitivity or the number of possible matches in search areas generated by the three methods as a function of the width of the region of interest (search area), hence the total area defined as possibly having matched regions depicted on ipsilateral views, without a decrease in the number of true-positive masses depicted within the search areas. Hence, we were attempting to reduce the number of possible matching candidates inside the search area that could result in mismatches and at the same time increase the number of false-positive regions that are initially cued by the single-image based CAD schemes and discarded during the matching procedure.
An image dataset of 1600 digitized mammograms acquired on 400 women who underwent screening mammography examinations was selected. Each examination (case) includes four images, CC and MLO view of the left and right breasts. Among these examinations, 200 depict masses that were pathology-verified as cancer (one in each case), and the remaining 200 were diagnosed as negative cases during the current screening examination and verified later by follow-up examinations (at least one and in the majority of cases more than one). Each positive examination depicted one mass that was considered to be visible on both the CC and the MLO views by experienced radiologists. The detailed image characteristics of these masses (including size and subjective subtleness rating) have been reported elsewhere . In brief, the average measured mass size is 1.5 cm2 (median 1.1 cm2) with the range of 0.1 to 9.5 cm2. Approximately one-half of the masses were rated subjectively as “somewhat subtle” to “very subtle” by the radiologists. The film-based mammograms were digitized using a Howtek digitizer (iCAD Inc., Nashua, NH) with original pixel sizes of 43μm × 43μm. For the purpose of mass detection, these images were then sub-sampled using a pixel-averaging method to reduce image size by 8-fold in two dimensions (with final pixel size of 0.344mm × 0.344mm) . A truth file that records (marks) the radiologists’ verified coordinate locations of all 400 mass regions depicting cancer was also established.
When applying a previously developed single-image based CAD scheme  to automatically detect and segment the 400 mass regions depicted in this dataset, the distribution of three representative image features including mass region size, contrast, and shape factor ratio are shown in Figures Figures11 and and2,2, respectively. Mass size was computed by counting the number of pixels inside the CAD-segmented region multiplied by the unit size of a pixel in mm 2. Mass region contrast value was computed by difference between the average pixel value inside the segmented mass region and the average pixel value of the surrounding area. Boundary frame of the surrounding area was defined as 10mm away from the mass boundary in all four directions. Shape factor ratio was computed as the square of the segmented mass region perimeter divided by mass region size. It reflects the “smoothness” (or associated spiculation level) of the mass region. In general, the smaller the shape factor ratio, the smoother of the mass region boundary is. A detailed definition and computational method of the three image features had been reported in a previous study . The figures show the diversity of this dataset as well as the feature differences when computed for the same masses depicted on two views. Due to these image feature differences computed from two projected images, the CAD scheme may detect the two suspicious regions associated with the same mass depicted on CC and MLO view with very different detection scores (the likelihood of a suspected mass region being true-positive finding). As a result, CAD scheme cues a large number of subtle masses only on one view by discarding the corresponding mass region on the ipsilateral view due to a lower detection score. In our CAD scheme, the detection score was computed using a pre-trained multi-feature based artificial neural network .
Figure 3 shows the three ways of defining search areas for matching (“pairing”) mass regions depicted on both the CC (Figure 3(a)) and the MLO views (Figure 3(b) - (d)) of the same breast. The first method defines a search area with a shape of an annular (fan-shaped) band (Figure 3(b)). It is generally based on the idea that radial distances from the nipple to the mass, or any point within the breast for that matter, remain relatively unchanged during breast compression under different views. Specifically, to define such a search area, CAD scheme detects the nipple locations on two views and computes the Euclidian distance between the nipple and the center of a CAD-cued mass region depicted on one view (i.e., Figure 3(a)). The scheme maps this distance to the other view and then uses the nipple location as an origin to define an annular search band on the other view. Because one can assume that the distance between the nipple and the center of the mass region to be matched on the other view is and the CAD scheme does not know in advance whether or , the search area is a defined as a band whose center arc has the mapped radial distance of (as shown in Figure 3(a) and (b)). Thus, to match two mass regions depicted on two views, the width of the search area should be or the search band should be bound by two arcs (curves) with the radial length of (on the front or nipple side) and (on the interior or chest wall side), respectively. Technically this approach is the simplest one compared to other methods including the other two methods discussed and compared in this study because only one point on each view, namely, the estimated location of the nipple, is required for computing the search area of interest on the ipsilateral view. This approach had been used for matching masses [10, 12] and micro-calcification clusters  in multi-view based CAD scheme development.
The second method defines a search area as a straight line based strip  as shown in Figure 3(c). It is based on the general concept that the chest wall constrains the deformation during breast compression in a manner that points in the breast move forward with displacement of similar distances under the two views. In this approach, CAD scheme detects both nipple and chest wall on the two views. Since chest wall is frequently not depicted on the CC view, it was assumed to be parallel to the image edge . The chest wall depicted on MLO view was automatically detected by CAD scheme (as shown in Figure 3(b) - (d)). Once a suspected mass region is detected on one view, the scheme computes the distance between the nipple and the mass center projected onto the centerline (a reference line passing through the nipple and being perpendicular to the chest wall). The scheme then maps the same projected distance to the centerline of the other view and defines a straight line based strip of interest as the search area. Since the search strip is always parallel to the chest wall, all suspicious mass regions located on the centerline of the strip have the same distance to the chest wall as well as the same projected distance to the nipple along the centerline perpendicular to the chest wall. At the same time the Euclidian distance of a suspected region to the nipple varies (unless it is actually on the centerline).
The third search method is a mixed method that combines the characteristics of the previous two methods. It aims to simultaneously account for the possibility that either of the two deformations may be applicable. This method expands the search area toward the nipple (forward) direction by utilizing a concave (arc) boundary and toward the chest wall (interior) using a straight line. Hence, this method allows for variation of the two distances (between the mass center and either the nipple or the chest wall) during compression under the two views as shown in Figure 3(d).
In our previous study, we assumed that the chest wall depicted on CC view was always parallel to the film (image) edge and the centerline was a horizontal line (as shown in Figure 3(a)) . However, our recent observation and image analysis found that this was not the case in a small fraction of examinations (i.e., < 5% in the current test dataset), primarily due to the variations in patient positioning and breast compression during the examinations. For example, the previously estimated centerline on the CC view shown in Figure 4(a) is not a good representation of the actual centerline of this imaged breast. To automatically detect and correct for this problem, if applicable, we added a new image processing step to the scheme. The scheme generates a new “perpendicular” line that crosses the centerline in each CC view (Figure 4(a)) and detects three crossing pixels between this line and (1) the initial centerline, (2) the top skin line (boundary), and (3) the bottom skin line. If the difference of two distances , where d1 is the distance between centerline crossing pixel and the top skin line crossing pixel and d2 is the distance between centerline crossing pixel and the bottom skin line crossing pixel, is greater than a pre-determined threshold (e.g., 0.1), then the initially generated centerline is classified as unacceptable and an adjustment is made.
In this study, we tested two methods to adjust the initially estimated centerline if Δd is larger than the threshold. The first one detects and uses the center pixel of the breast area along the edge of image. The modified centerline is then defined as the line that connects the nipple to the center pixel (Figure 4(b)). The second method applies iteratively small rotations of the center line (e.g., 3°) until the recomputed Δd reaches the minimum (Figure 4(c)). We compared the impact of the two methods using our dataset and concluded that the difference between them in generating the final centerline was quite small as shown in Figure 4(d). Due to the computational simplicity associated with the first method we implemented it in our CAD scheme to define the modified centerlines (as needed). As an example, application of this modified method to define the centerline of the case shown in Figure 4 resulted in the width of the search strip being reduced from 38.6mm to 15.8mm (Figure 5).
During the development of a typical multi-view based CAD scheme one usually plotted the sensitivity of pairing or matching two corresponding mass regions depicted on ipsilateral (CC and MLO) views against the search area width that represents the maximum allowed difference in matching distances (i.e., either Euclidian or projected distance between the nipple and the detected mass region center) of two corresponding mass regions [12, 14]. The sensitivity level ranges from 0 at a search area width or size of 0 to 100% when the search area covers all paired depiction of masses within the dataset with a known truth but it definitely reaches 100% when the search area size is equal to or exceeds the size of the image. Since increasing the width or size of the search area of interest also increases the number of possible false-positive identifications (regions) inside the search area, it results in the increase of the likelihood of mismatching (e.g., the true-positive mass region matching with a false-positive region). Thus, in the development of a multi-view based CAD scheme, one needs to determine the “optimal” width or size of the search areas to be used in this specific CAD scheme. This task was typically performed either empirically or by some related formulation to allow a certain upper level of matching sensitivity to be achieved.
We conducted two performance assessment experiments in this study. First, we investigated the maximum matching sensitivity as the function of the search area size (or width) when applying a search method to all positive images that include 200 verified masses depicted on two views in our dataset. Specifically, in this experiment we measured and compared the fraction of masses included in the search area regardless of the CAD itself as a function of the width of the search area for the use of annular band and straight strip based search methods. Second, we applied our in-house developed single-image based CAD scheme  to detect suspicious mass regions depicted on all 1600 images in this testing dataset. Using a pre-optimized operating threshold on the CAD-generated detection scores, all initially detected suspicious mass regions with detection scores ≥ 0.55 (operating threshold) were cued on the images and those with scores below this threshold were discarded (not cued). Then, the three search methods were separately applied to define search areas for matching CAD-detected suspicious regions depicted on ipsilateral views. For each suspicious mass regions (either true-positive or false-positive) initially cued by the single-image based CAD scheme (with detection score lager than the operating threshold), the multi-view based CAD scheme continues to search for all other initially detected suspicious mass regions (including those with the detection scores lower than the operating threshold) within the defined search area. The identified regions were sorted based on the CAD-generated detection scores (from 1 to 0). The suspicious region that had the highest detection score was defined as the “matched” region to the region depicted on the other view. The multi-view CAD scheme cued the two matched regions even if the selected matched region had a detection score lower than the original operating threshold. When no CAD-detected suspicious mass region was found inside the search area, the un-matched (isolated) region was considered a false-positive finding and was discarded. We then compared the four CAD performance levels generated by the original single-image based CAD scheme and the multi-view based CAD schemes when using each of the three search methods described in this study. Both a case-based (in which a suspicious mass is considered detected if it is cued on either or both views) and a region-based (in which a CAD-cued region is independently counted) performance levels, namely, sensitivity and false-positive rate at the same operating threshold were tabulated and compared. In addition, similar to other matching approaches using different similarity measures, a mismatch (a TP-FP pair) occurs when multiple suspicious regions are detected inside the search area and a false-positive (FP) region has a higher detection score than the relevant true-positive (TP) region. Thus, we also compared the matching accuracy in terms of TP-TP regions among the three image search methods.
The number of incrementally matched pairs as a function of the width of the search area is provided in Figure 6 for the first two methods that use the annular bands and straight strips, respectively. As can be seen in this figure, on average, for the same matching sensitivity the search area width for the straight strip based method is substantially smaller than that using the annular bands. To enable matching all 200 pairs the corresponding widths of the search areas were 28mm and 68mm for the use of straight strips and annular bands, respectively. The matching performance curves based on these data are shown in Figure 7. Of note is that at 28mm width of the search area when the straight strip based search method achieves 100% sensitivity, while the annular band search method reaches a sensitivity of only 82%.
Figure 8 shows the case-based and region-based free-response receiver operating characteristics (FROC) type performance curves as well as the operating threshold line for the single-image based CAD scheme. The CAD scheme initially detected 4851 suspicious mass regions in this test set of 1600 images (no detection threshold). Among these, 400 are all true-positive (TP) mass regions and 4451 false-positive (FP) regions result in a maximum FP detection rate of 2.78 per image at 100% sensitivity. Applying the operating threshold for detection scores at ≥ 0.55, the scheme cued 172 masses (or 264 mass regions) and 523 FP regions. Thus, the single image based CAD scheme achieved a detection sensitivity level of 86% (case-based) and 66% (region-based) at a FP detection rate of 0.33 per image as indicated by the operating threshold line plotted on Figure 8.
Table 1 summaries the detection results for the single image based CAD scheme and the multi-view based CAD schemes when using each of the three search methods. The single image based CAD scheme cued 92 masses on both views and 80 masses only on one view. Applying the straight line based strip search method with strip width wS = 28 mm the scheme cued the same 172 masses. Among these, 158 were TP-TP matches and 14 were TP-FP matches. At the same time the scheme cued 331 pairs of FP “masses” in the whole dataset (0.21 per image). Among these paired “matched” regions, 61 FP pairs had both detection scores higher than the CAD operating threshold. Based on the matching result, the single-image based CAD scheme initially generated 462 case-based FP detections (including 61 “paired” cues on two views and 401 cues only on one view). Among the total set of 523 FP regions initially cued by the single image based CAD scheme, 392 (331 + 61) were also cued by the multi-view based CAD scheme and the remaining 131 (25%) were un-matched and discarded. Hence, the multi-view based scheme reduced the case-based FP detection rate by 28.4% from 0.29 (462/1600) to 0.21 (331/1600) per image; while maintaining the same case-based detection sensitivity (86%). The fraction of true-positive masses being cued on the two views increased from 53.5% (92/172) to 91.9% (158/172).
When applying the annular band based search method with area width wS = 28 mm, the scheme cued 148 masses and discarded 24 un-matched masses. The case-based detection sensitivity was reduced from 86% to 74%. To match all 172 initially cued TP masses, the width of the annular band based search areas needed to be increased to wS = 68 mm. At this width, the average number of candidate regions for matching within the search area increased from 1.62 (when wS = 28 mm) to 2.73. Among the 172 cued masses, 142 were TP-TP matches and 30 were TP-FP matches. At the same time the scheme cued 389 pairs of FP regions. Among these, 72 had detection scores higher than the operating threshold for both cued regions. Thus, the multi-view based CAD scheme cued 461 of 523 FP regions that were initially cued by the single image based scheme and discarded the remaining 62 FP regions (11.9%).
When applying the mixed search method, the experimental results showed that in order to maintain the original case-based CAD-cueing sensitivity (86%), the minimum width along the centerline of the search areas could be further reduced to wS = 20 mm (Table 1). However, as the shape of the search areas gradually increase away from the centerline, the actual search area in this mixed approach was actually substantially larger than the area used in the straight strip based approach. As a result, using this search method, the scheme cued 146 paired masses on the two views with 26 TP-FP mismatches. The scheme also cued 383 FP-FP pairs of mass regions and discarded 76 of the originally cued FP regions (14.5%). In summary, as shown in Table 1, the CAD performance level using the mixed approach to defining the search area is slightly better than when using the annular band based search method, but it performed substantially poorer than when using the straight strip based search method.
In the clinical environment radiologists rely heavily on comparisons of multiple images including, but not limiting to the identifying matched suspicious mass regions depicted on two ipsilateral views. At the same time, current commercially available CAD systems use primarily a single-image based CAD schemes with the exception perhaps of limiting the total number of cued regions on an examination . Hence, these CAD schemes are somewhat “disadvantaged” as compared with the users (the radiologists) in the type of information being used during the decision making process. With the exception of relatively “easy” masses, single-image based CAD schemes tend to detect and cue the majority of the more subtle masses as well as the majority of the false-positive identifications only on one view . As a result, if a radiologist either misses or discards a subtle mass during the initial image viewing, he/she is also likely to discard most single-view based CAD-cues as false-positive detections and this had been shown in both laboratory and clinical reading environments (studies) [2-4]. Therefore, the development of multi-image or multi-view based CAD schemes is of great interest if we wish to realize the full potential of using CAD technology in the clinical practice.
Defining optimal search areas on the corresponding ispilateral view constitutes an important step for developing multi-view based CAD schemes. In this study, we presented and compared three different approaches to address this difficult problem. Our experimental results showed that in this testing image dataset the straight (line) strip based search method was more accurate and required narrower (smaller) search areas than the other two methods. The approach proposed and tested in this study to correct the orientation of the estimated centerline in a small fraction of examinations (CC view) further improved results substantially in these cases (as shown in Figure 5). The experimental results also suggest that Euclidian distances between nipple and the center of mass regions as measured on two views of a compressed breast vary quite a bit in particular when the mass is located closer to the nipple and farther away from the centerline (inner and outer). In this case, the distance between the nipple and mass region center depicted on CC view is substantially shorter than the same distance depicted on MLO view as shown in Figure 3(a) and (b). This large difference results in the need for a wider search area for pairing (matching) two depictions of the same mass on two views. The maximum width of annular band search area needed to be applied in our dataset was quite comparable to that previously reported by other groups when using different image datasets (i.e., ≥72mm  and 48mm ). We recognize that it is difficult to directly compare the size differences in the required search areas when using the three methods we described. However, based on the average number of matching candidates (Table 1) depicted on these three types of search areas, we can estimated that the straight strip search method generated the smallest searching areas followed by the mixed search method, while the annual band search method generated the largest searching areas.
Our experimental results confirm the advantages associated with the use of smaller search areas. First, due to the low correlation (or “similarity” measures) between the subtle masses depicted on two views of projected images, the use of smaller search areas increases matching accuracy by reducing the likelihood of incorrect matching between the true-positive and false-positive regions. In our study, in order to maintain the same case-based CAD scheme sensitivity of 86%, the narrow straight strip based search method which had the smallest search areas substantially increased the number of TP-TP matches (158) as compared with the other two approaches (142 and 146). Second, the smaller the search areas the more effective the scheme in reducing the number of CAD-generated false-positive cues through a reduction of matching candidates. Our experimental results showed that by reducing the size of search areas, CAD scheme eliminated 11.9% (annular bands), 14.5% (the mixed search areas), and 25.0% (straight strips) of false-positives initially detected by the single-image based CAD scheme, respectively. The straight strip search method also generated the lowest case-based false-positive detection rate that was 15% lower than using annular band based search method (331 vs. 389 in 1600 images).
We also recognized that these three searching methods could cover quite different searching area depicted on images. For example, some CAD-detected suspicious mass regions can be included inside the straight strip but outside the annular band or vice versa. Thus, the identified matched regions using different methods can be different in many cases in particular for the match between two false-positive regions. Despite of such difference, the trend of their impact on overall CAD performance is clear. It is not the searching method but the size of the search areas that is the key fact to determine the matching accuracy and the potential impact on overall CAD performance. This study shows that among these three compared search methods, the straight strip approach required a smaller search area while achieving the highest level of CAD performance.
This work is supported in part by Grants CA77850 and CA101733 to the University of Pittsburgh from the National Cancer Institute, National Institutes of Health.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.