|Home | About | Journals | Submit | Contact Us | Français|
To demonstrate the efficacy of an automated 3-D template matching-based algorithm in detecting brain metastases on conventional magnetic resonance (MR) scans and the potential of our algorithm to be developed into a computer-aided detection tool that will allow radiologists to maintain a high level of detection sensitivity while reducing image reading time.
Spherical tumor appearance models were created to match the expected geometry of brain metastases while accounting for partial volume effects and offsets due to the cut of MRI sampling planes. A 3-D normalized cross-correlation coefficient was calculated between the brain volume and spherical templates of varying radii using a fast frequency domain algorithm to identify likely positions of brain metastases.
Algorithm parameters were optimized on training datasets and then data were collected on 22 patient datasets containing 79 total brain metastases producing a sensitivity of 89.9% with a false positive rate of 0.22 per image slice when restricted to the brain mass.
Study results demonstrate that the 3-D template matching-based method can be an effective, fast, and accurate approach that could serve as a useful tool for assisting radiologists in providing earlier and more definitive diagnoses of metastases within the brain.
At least one brain metastasis will develop in an estimated 100,000 to 200,000 patients out of the more than 1 million people who are diagnosed with cancer each year in the United States of America (1, 2). Considerable mortality and morbidity can arise from the presence of brain metastases with patients most commonly presenting with headaches, seizures, or other neurological symptoms. However, a significant number of brain metastases are asymptomatic, at least initially, leading to belated detection and treatment and consequently, poorer prognoses (3, 4). The accurate characterization of brain involvement is a critical factor in both the aggressive therapy regimens for good prognosis patients and the palliative control efforts directed at the brain metastases patients with terminal systemic disease (5). Numerous studies have demonstrated that enhanced quality of life and prolonged survival are readily attainable goals in many patients when given a proper and timely diagnosis (6 – 11). As a result, regular brain metastasis screenings for patients with extracranial primary tumors must be performed in order to ensure appropriate treatment for all patients.
The attractiveness of magnetic resonance imaging (MRI) as a screening method stems heavily from its non-invasive nature and lack of known toxicity to patients, regardless of frequency of exposure, except for the rare occurrence of gadolinium-associated nephrogenic systemic fibrosis (NSF) seen only in patients with renal failure (12, 13). In addition, MRI has been shown to be more sensitive in the detection of multiple brain metastases when compared directly to CT scanning (14, 15). This ability of MR screening to identify asymptomatic brain metastases has made it an important component of the staging process for newly diagnosed cancer patients (16, 17). Yet, completely manual image reading predisposes to an increased tendency for reading errors, especially for small nodules, due to the limitations of 2-D image viewing, user subjectivity, and the demanding workload caused by the large number of images produced by high-resolution volumetric MR imaging (18). A brain metastasis computer detection algorithm designed to assist radiologists can counter these issues through its consistency, reduction of user bias, and capacity to consider simultaneously the full 3-D tumor information.
The majority of existing medical imaging automated detection algorithms can be classified as either being primarily intensity-based or model-based. Intensity-based approaches are centered around thresholding operations and intensity histogram analysis relying upon the target lesion being consistently brighter (or possibly darker) than the encompassing normal tissue. The chief limitation of strictly intensity-based algorithms is that they lack the means to discard any non-lesion areas of the normal anatomy that also share the same general intensity profile, which is especially problematic in MR images where intensity inhomogeneities and artifacts affect the absolute voxel intensity values. In general, model-based approaches focus on using shape or appearance models that are designed to represent the expected morphology of the target object to match potentially suspicious regions within images. While 2-D models have been implemented for object matching (19 – 20), their lack of capacity to consider 3-D imaging information makes them a less specific and less desirable option for automated detection. In the process of constructing a model, feature-based analysis may be employed where up to dozens of features are extracted from identified target lesions during an extensive training phase and then prioritized for use based on the quantitative correlation between their presence and the likelihood of the corresponding image area being a region of interest. One notable issue with expanded feature-based analysis methods is that there is a considerable risk of dependence on too many parameters and over-training leading to a loss of generalizability to lesions beyond the utilized lesion training dataset. The ideal strategy is to use the minimum number of features that describe adequately your target without also applying to other non-target areas. For this reason, we have selected an approach dependent only upon the three most direct descriptors of brain metastases in contrast-enhanced MR brain scans: spherical or pseudospherical 3-D shape, a brighter core than periphery, and distinct borders from the surrounding normal brain tissue.
While computer-aided detection (CAD) approaches have been applied to tumor evaluation in breast and prostate MR images (21 – 25), the literature lacks any brain MR CAD for metastatic tumor detection. In this paper, we present the application of our 3-D template matching-based nodule detection algorithm to clinical MR brain scans with the goal of identifying a high percentage of all metastatic brain tumors while introducing a reasonably small number of false positives into the final results. The purpose of this study is to illustrate that through the optimization of our algorithm for MR brain screening, we will be able to offer a CAD tool for metastatic brain tumors that will address issues of accuracy and efficiency by providing radiologists with locations of high suspicion for nodule presence.
The tumor detection operations for this research were conducted entirely upon patient MR scans that had been accumulated previously at our institution. This patient medical image-centered study was approved by our institution’s review board and was compliant with the Health Insurance Portability and Accountability Act (HIPAA). All anatomical MR scans that were used for metastasis detection were collected with a 1.5T GE Genesis Signa scanner and have an in-plane resolution of 0.43 mm and a slice thickness and separation of 2.5 mm. These T1-weighted post-contrast images were obtained using a spoiled gradient echo pulse sequence (SPGR) with TR/TE = 14/3 ms, flip angle = 30, 512×512 matrix, and field of view (FOV) = 220 mm. In this study, 9 patient datasets consisting of 540 coronal MR slices designated as training datasets and 22 patient datasets consisting of 1320 coronal MR slices assigned as testing datasets were processed using our detection algorithm. A total of 124 brain metastases were present within the training datasets ranging in diameter from 2 to 26 mm (median = 7 mm) where the median number of metastatic tumors per dataset was 12 with one dataset containing zero metastases. A total of 79 brain metastases were present within the testing datasets ranging in diameter from 3 to 45 mm (median = 7 mm) where the median number of metastatic tumors per dataset was 3 with one dataset containing zero metastases. The presence and location of brain metastases were confirmed by follow-up examination, listing in the radiologist clinical report, and when necessary, independent radiologist reading at the authors’ institution.
Semi-automated extraction was performed on each patient dataset prior to it being subjected to the detection component of the algorithm so as to increase computation speed. An interactive method for skull-stripping was selected rather than a fully-automated approach in order to ensure consistency in anatomical voxels present among processed datasets and to guarantee that study results were dependent only upon the performance of our detection algorithm and not on the quality of the brain extraction. An active contour (snake) modified by our lab to incorporate 3-D image slice data for control point and curve initialization was used as our means of separating the brain mass from the skull and other non-brain soft tissue (26). The utilized snake implementation employed cubic Hermite spline interpolation where user interaction involved the ability to adjust control point location and slope as necessary. Snake placement was setup to achieve the simplest contouring of the brain mass where the ventricles, the dural venous sinuses at the brain’s surface, and the branches of the internal carotid arteries adjacent to the brain were not excluded. On average, semi-automated brain extraction required less than 20 minutes per MR dataset (60 coronal slices). Pre-processing of datasets to correct for MR intensity artifacts and inhomogeneities was not performed for this study.
Brain metastases have been described as spherical or spheroid structures that are well-circumscribed in appearance on contrast-enhancing MR brain scans as a result of clear margins with surrounding anatomical tissue (16, 27). Our algorithm employs these typical features of brain metastasis morphology through the use of a 3-D spherical template that incorporates a zero padding parameter to account for the well-demarcated nature of metastasis borders (28). Zero padding refers to the inclusion of a uniform number of zero intensity values around the border of the high-intensity spherical template. Both in-plane and slice thickness direction partial volume effects were included in the generation of these models so as to match more accurately the voxel representation of a 3-D nodule in MR imaging. Specifically, three different spatial models were used for each size 3-D template to compensate for the partial volume effects seen in the slice thickness direction. One spatial model placed the central MR cut-plane exactly through the center of the spherical template while the other two models offset the center of the sphere from the central cut-plane in equal and opposite directions by one-third the slice thickness (Fig. 1). A complete template included a zero padding slice both before and after spherical template slices.
For the optimal balance between the algorithm’s sensitivity and the false positive rate, the threshold value for our similarity metric, the normalized cross-correlation coefficient (NCCC), and the appropriate size for the zero padding can be selected on the basis of the results from a set of patient images. Formal training datasets were utilized for this objective and then the optimal NCCC threshold value and zero padding size were applied to a separate group of testing datasets so as to ensure the generalizability of the algorithm results. Twelve 3-D spherical templates were applied to each MR dataset with radii ranging at evenly spaced intervals from four times to twenty times the in-plane voxel size. Preliminary empirical results had demonstrated that this range of template sizes would provide a sufficient capture range for clinically relevant tumors where CAD would be most beneficial. In our nodule detection approach, an individual NCCC value was computed for each specific spherical template, defined by its unique combination of radius and spatial orientation, at every voxel within each processed MR dataset. This was accomplished by aligning iteratively the center voxel of each spherical template with each voxel of the MR dataset until an NCCC value had been calculated throughout the entire 3-D image volume. The processing time for an average 60 slice extracted dataset was approximately 30 minutes running in The MathWorks, Inc.’s MATLAB 7.2 on a Power Mac G5 with quad 2.5 GHz processors and 4 gigabytes of RAM. Only the eleven largest templates were utilized when the algorithm was directed to target nodules with a manually measured diameter greater than twice the slice thickness (diameter greater than 5 mm).
A normalized cross-correlation coefficient (NCCC) was utilized to identify the degree of similarity between the 3-D regions of the image and spherical templates of varying radii. The NCCC was selected as the similarity metric as it allows for 3-D feature matching that is not dependent on the absolute voxel intensity values, alleviating concerns over brightness variations that can occur among MR datasets due to different scanning parameters and intensity inhomogeneities. In order to reduce computational cost, the calculation of the NCCC was performed in the Fourier domain of the MR image region and the spherical template using a 3-D extension of Lewis’ approach (29):
where t is the template, f is the image, is the mean of the template, u,v,w is the mean of the image region under the template, and summations are performed over x,y,z under the window containing the template positioned at u,v,w. The NCCC can produce a range of values from −1 to +1 depending upon the similarity of the analyzed image regions where a value of +1 indicates an exact matching of the image regions, a value of −1 indicates an exact matching of the inverse of one image region with the other image region, and a value of zero indicates no correlation. Only image coordinates that generated an NCCC value above an experimentally determined threshold were identified as locations of potential brain metastases. Redundant detections were removed by grouping all image points connected in 3-D that provided an above-threshold NCCC value. A detection was considered a true positive if the voxel with the highest NCCC value within a group of connected above-threshold image points overlapped with the contrast-enhancing voxels of an identified brain metastasis. All other detections were labeled as false positives.
Nodules in the testing datasets were stratified with regard to diameter and MR appearance characteristics in order to establish patterns in false negatives produced by the algorithm results. Ring-enhancing nodules were classified as ‘Hollow’, nodules compressed at any exterior or interior anatomical surface of the brain were classified as ‘Edge’, nodules that have their full 3-D shape cut-off by either the first or last slice of the MR dataset were classified as ‘Partial’, and nodules within patient brains that were treated with radiation therapy prior to MR scan acquisition were classified as ‘Radiated’. Similarly, false positives found within the results for the testing datasets were analyzed for patterns in image location. Specifically, the number of false positives found within the dural venous sinuses at the exterior surface of the brain mass and false positives found within the internal carotid artery branches adjacent to the brain mass were tabulated to determine the percentage of false positives provided by the MR representation of these anatomical structures.
The major underlying hypothesis for our tumor detection algorithm is that a 3-D spherical template surrounded with a uniform layer of zero padding will correlate highly with the typical patterns of brain metastases on MR datasets. Normal anatomical structures such as cerebral blood vessels should be avoided as false positives in our final detection results due to poor matching of their 3-D branching with the zero padding parameter. Figure 2 demonstrates the marked 3-D correlation that can be achieved between brain metastases and a spherical template of the appropriate diameter without also matching the surrounding extraneous tissue.
An important component of our tumor detection efforts is determining the most effective zero padding size and NCCC threshold value to achieve consistently a high level of specificity for metastasis detection. Figure 3a illustrates the effects of changing the amount of zero padding that uniformly surrounds each spherical template on the overall accuracy of the detection process for one patient’s dataset (total of 10 lesions). The same patient’s dataset was then used to produce Figure 3b which demonstrates the relationship between the efficacy of the detection method and the NCCC threshold that must be exceeded between a pixel and the spherical template for that spatial position to be considered a metastasis candidate. These two figures exhibit the need to find the optimal settings for both zero padding size and the NCCC threshold value due to the fact that increasing the value of either parameter reduces substantially the number of false positives while at the same time increasing the risk that a true positive will be missed.
In order to establish the optimal settings for our template matching approach, a total of 9 patient datasets (540 total coronal MR slices) were processed by our algorithm using 25 different combinations of zero padding size and NCCC threshold value. Figure 4 visualizes this effect of varying the zero padding size and NCCC threshold value on the efficacy of our detection algorithm by providing the relationship between the sensitivity and the number of false positives per MR slice as the two parameters are adjusted. This figure indicates that the optimal balance between sensitivity and false positive rate for these 9 datasets was achieved at a zero padding size of 0.86 mm and a NCCC threshold equal to 0.525 where the overall sensitivity was 87.1% (108 out of 124 nodules detected) and the false positive rate was 0.62 per image slice.
The determined optimal algorithm parameters were then applied to testing datasets for the purpose of ascertaining our algorithm’s current level of performance. Sensitivity equal to 89.9% (71 out of 79 nodules detected) with a false positive rate of 0.58 per image slice was attained using the zero padding size of 0.86 mm and a NCCC threshold of 0.525. For these same zero padding size and NCCC threshold values, the sensitivity remained relatively constant at 90.5% (57 out of 63 nodules detected) while the false positive rate was reduced to 0.36 per image slice when omitting from consideration nodules less than two slices (5 mm) in diameter.
Figure 5 shows MR image examples of the dural venous sinuses and branches of the internal carotid arteries where artifacts of brain extraction or typical variations in MR image quality can lead to these blood vessels having a roughly spherical MR image appearance. An examination of our twenty-two processed testing datasets revealed that on average 43.7% (± 8.9%) of the produced false positives were found in some region of the dural venous sinuses and 18.0% (± 8.3%) of the false positives were located within the branches of the internal carotid artery (Table 1). Consequently, this leads to our algorithm having a false positive rate of 0.22 per image slice (sensitivity = 89.9%) for the full range of nodule diameters when analysis is restricted to the brain mass only, excluding the areas of the dural venous sinuses and internal carotid artery branches along the brain’s border. A false positive rate of 0.12 per image slice (sensitivity = 90.5%) is achieved when both analysis is restricted to the brain mass and nodules less than two slices (5 mm) in diameter are omitted from consideration.
Another noteworthy point of evaluation for our algorithm is its applicability to brain metastases that deviate from the classic patterns of contrast-enhancement or 3-D morphology represented by our templates. The MR intensity profile of certain nodules may appear to be hollow (ring-enhancing) as a result of their necrotic cores. Brain metastases may deviate from strictly spherical geometry due to compression from their growth in locations adjacent to either the brain’s exterior surface or medial anatomical structures. The nodule being located within the brain such that its full 3-D structure is clipped off in either the first or last slice of the image set can cause the same effect on a metastasis’ appearance in an MR dataset. In addition, the ability to detect accurately both metastases that have been irradiated and untreated nodules is critical for the chief clinical applications that may be served by our algorithm. Table 2 shows the effects of these different metastasis characteristics on our algorithm’s detection sensitivity where each column presents the sensitivity results for only nodules within the given category and then all nodules except those that are within the given category. Furthermore, Table 3 demonstrates the accuracy of our approach when applied to varying ranges of metastasis diameter where each column presents the sensitivity results for only nodules below the given diameter and then all nodules except those below the given diameter. The data presented in these tables demonstrate the general robustness of our standard 3-D template matching-based detection algorithm with regard to variations in metastasis morphology, intensity profile, and size.
In this paper, we have presented our automated approach applied specifically for the detection of metastases on brain MR scans. The basis for our algorithm is 3-D template matching between MR image features and 3-D spherical tumor appearance models constructed to represent typical geometrical patterns of brain metastases. At its current optimal settings for zero padding size and NCCC threshold, our algorithm achieved a sensitivity of 89.9% with 0.58 false positives per slice when applied to coronal MR datasets containing metastases ranging in diameter from 3 to 45 mm. These values are extremely similar to the sensitivity (87.1%) and false positive rate (0.62 per image slice) found for the training datasets at the same algorithm parameters and thus, indicate a presumed representative nature of the results. Due to a lack of data in the literature regarding automatic detection of MR brain lesions, we are unable to compare our results to a pre-existing standard. Nevertheless, these results are extremely encouraging as they represent a high detection rate and a manageable number of false positives without any restrictions having been placed on target nodules with regard to size, heterogeneity, or whether a metastasis’ morphology has been altered by treatment with radiation prior to the patient’s scanning. When restricting our algorithm to detecting only nodules with a measured diameter greater than or equal to twice the slice thickness (diameter greater than or equal to 5 mm), our sensitivity can be maintained relatively constant at 90.5% while our false positive rate drops to 0.36 per image slice.
Another positive aspect of these results is that there are clearly identifiable patterns for our approach with regard to the sources of false positive readings. The fact that greater than 60% of our false positives stem from either the dural venous sinuses or the branches of the internal carotid arteries indicates that the incorporation of anatomical model-based information into our extraction methods will lead to the elimination of the majority of our false positives. Recently, an automatic brain extraction approach has been presented that has shown the ability to remove the dural venous sinuses and branches of the internal carotid arteries while preserving brain metastasis morphology (30). The substantial reduction in false positive rate that would be provided by the automatic extraction of the dural venous sinuses and branches of the internal carotid arteries would enable the utilization of less stringent algorithm parameters (zero padding size and NCCC threshold), leading to an additional rise in detection sensitivity (data not shown). Similarly, the relatively predictable 3-D morphology and intensity profiles of the dural venous sinuses and branches of the internal carotid arteries suggest that direct detection of these false positive sources through template matching, filtering, or another similar pre-processing technique may prove to be an alternative and equally efficacious means of achieving improved tumor detection algorithm performance (31). Furthermore, human readers would dismiss easily false positives found within normal anatomical structures outside the brain and this notion reinforces our algorithm’s potential as a radiologist CAD tool in its current form. As it stands now, the restriction of analysis only to areas within the brain mass reduces the false positive rate to 0.22 per image slice (sensitivity = 89.9%) for the full range of nodule sizes while the false positive rate is 0.12 per image slice (sensitivity = 90.5%) when the algorithm is also restricted to searching for only nodules with a diameter greater than twice the MR slice thickness. These results are comparable to the better results from CAD algorithms applied to other organs such as lung (28 and references therein).
The results showing the number of false positives found within the dural venous sinuses and the branches of the internal carotid arteries also elucidate the advantage of a 3-D template matching-based approach over simple intensity thresholding. An MR brain metastasis detection algorithm that relies heavily on intensity thresholding without 3-D morphology consideration will generate false positives throughout all regions of the dural venous sinuses and internal carotid artery branches. Our 3-D template matching-based approach provides the general exclusion of the greater parts of cylindrical structures such as the brain’s major blood vessels both at the surface of and enclosed within the brain mass. Under the same principle, spherical template matching is also avoiding many other cerebral sources of false positives that would be found by algorithms relying strictly on intensity thresholding. The dural venous sinuses, internal carotid artery branches, and other normal cerebral structures only appear as false positives with spherical template matching when inhomogeneities in MR scan intensities cause regions of these structures to appear spherical.
The major potential limitation for our algorithm is the possibility that the brain metastases to be detected may differ considerably from the hypothesized spherical morphology and intensity profile represented by our 3-D tumor appearance models. We utilize MR imaging because it has been shown to be more sensitive in detecting multiple brain metastases, however the presence of scan inhomogeneities and patchy enhancement with gadolinium contrast may exacerbate metastasis heterogeneity. Although our results indicate that our 3-D template matching-based system can often be effective when applied in these circumstances, the goal of achieving a detection sensitivity more closely approaching 100% necessitates accounting for the varied presentations of metastases that may arise on conventional brain MR scans. For this purpose, we have begun implementing template modifications that will enable improved matching for metastases that are problematic to detect due to features such as hollow, necrotic cores and irregular growth patterns at the brain’s borders. In particular, techniques such as template matching with hollow templates, the use of a negative NCCC to detect the dark, spherical cores of ring-enhancing nodules, and the incorporation of 3-D brain morphology data to introduce anatomical border effects into the tumor appearance model construction all have the potential to allow a further increase in sensitivity while decreasing the false positive rate at a small additional computational cost.
In conclusion, the primary objective of this study was to demonstrate that brain metastases follow consistent and predictable patterns of general 3-D morphology and intensity profiles on MRI scans that can be matched successfully by 3-D spherical tumor appearance models through the use of the NCCC as an accurate and low computational cost similarity measure. The most significant practical consequence of this work was illustrating the feasibility of a 3-D template matching-based approach in forming the foundation behind a computer-aided detection tool that could be employed to assist radiologists in brain metastasis screening. While our algorithm may not reach or exceed the capabilities of a trained radiologist, it does achieve the critical CAD algorithm goal of being complementary to human reader performance (32). For example, although our CAD algorithm did produce a false negative rate equal to 10.1% (8 out of 79 nodules not detected), all 8 nodules not detected by our CAD algorithm had been detected successfully by prior first-pass manual radiologist reading. In its current state, our system can reduce the number of spatial positions that need to be examined from potentially thousands to less than one per slice in most datasets. This CAD tool would serve to alleviate greatly the workload burden on radiologists by reducing image reading time as well as meet a previously described need for an automated system of metastasis target labeling (5). Moreover, due to the high volume of scans to be read, reader fatigue, and the complexity of images, it is inevitable that some small brain metastases are currently being missed without the use of a computer assist tool, as observed in 2 of the 22 testing datasets (Fig. 6). As a result, the utilization of a CAD tool would serve to improve not only the efficiency, but also the accuracy of brain metastasis screening performed on at-risk cancer patients.
The authors would like to acknowledge Dr. Balasubramanya Kolar, M.D. for his valuable assistance in confirming brain metastasis locations, Dr. Paul Okunieff, M.D. and Dr. Delphine Davis, Ph.D. for their helpful input and advice, and David Fuller for the development of the image viewer used to confirm our detection results.
Grant Support: Medical Scientist Training Program, NIH T32 GM-07356