|Home | About | Journals | Submit | Contact Us | Français|
To evaluate the feasibility of fusion of morphologic and functional imaging modalities to facilitate treatment planning, probe placement, probe re-positioning, and early detection of residual disease following radiofrequency ablation (RFA) of cancer.
Multi-modality datasets were separately acquired that included functional (FDG-PET and DCE-MRI) and standard morphologic studies (CT and MRI). Different combinations of imaging modalities were registered and fused prior to, during, and following percutaneous image-guided tumor ablation with radiofrequency. Different algorithms and visualization tools were evaluated for both intra-modality and inter-modality image registration using the software MIPAV (Medical Image Processing, Analysis and Visualization). Semi-automated and automated registration algorithms were used on a standard PC workstation: 1) landmark-based least-squares rigid registration, 2) landmark-based thin-plate spline elastic registration, and 3) automatic voxel-similarity, affine registration.
Intra- and inter-modality image fusion were successfully performed prior to, during and after RFA procedures. Fusion of morphologic and functional images provided a useful view of the spatial relationship of lesion structure and functional significance. Fused axial images and segmented three-dimensional surface models were used for treatment planning and post-RFA evaluation, to assess potential for optimizing needle placement during procedures.
Fusion of morphologic and functional images is feasible before, during and after radiofrequency ablation of tumors in abdominal organs. For routine use, the semi-automated registration algorithms may be most practical. Image fusion may facilitate interventional procedures like RFA and should be further evaluated.
The value of image processing and fusion has been investigated for diagnostic and prognostic purposes; still it has been a less studied tool for interventional radiological procedures. Registration and fusion of radiological images is by no means a new post processing technique [1–5]. There are numerous technical approaches described to coalesce imaging data from different modalities [6–8] and use them to provide better health care for the patient. Registration is defined as aligning the two imaging data sets spatially to each other. While fusion is defined as overlaying them and visualizing them as one image. Algorithms for registration of anatomical and functional data sets have been mostly studied in fixed or rigid organs such as the spine or brain [2, 4, 7, 9, 10]. But registration is somewhat more difficult in region with physiologic movements like the neck  or in moving organs such as lungs [12, 13]. Major challenges due to physiologic motion and the non-rigid nature of organs have limited the practical implementation of image fusion for abdominal interventional procedures for diagnostic and prognostic reasons.
Recent studies have shown that fusion of abdominal images from different modalities can improve diagnosis and monitoring of disease progression [14–16]. New hybrid-imaging systems combining positron emission tomography (PET) or single photon emission computed tomography (SPECT) with computed tomography (CT) offer a one-stop examination promoting the diagnostic and prognostic potentials for extra-cranial applications of image fusion in cancer [17–21]. Image fusion has proven useful for evaluation of patients with cancer supporting diagnosis, staging, treatment planning, monitoring the response to therapy including disease progression . Minimally invasive image-guided therapy like radiofrequency thermal ablation is being routinely used, especially in the liver, lung, bone and kidney [23–26] and improves survival for certain patients .
Optimal outcomes of percutaneous radiofrequency ablation (RFA) are highly dependent upon accurate targeting of the neoplastic tissue and monitoring of the resulting thermal lesion. Success of treatment is intimately linked to the volumetric spatial relationship of neoplastic tissue to the thermal lesion margins. An accurate spatial understanding of this relationship that is readily accessible may provide feedback during pre-treatment planning, procedural navigation, early detection of re-growth improving prognosis. Ideal image guidance for RFA allows accurate probe placement for sphere-packing with sufficient overlap to avoid gaps of sub-lethal heating, and to treat a small margin of normal tissue beyond the neoplastic tissue borders. This is a challenging task and is prone to human error. The procedure is to be followed by repeated scanning and look for disease progression.
Image fusion was studied with an image processing software used before, during, and after RFA interventions. Fusion of morphologic and functional image data might improve spatial appreciation and visualization of tumor and its relation to thermal lesion margins.
All reported patients were enrolled under investigational protocols that were approved by the Investigational Review Boards (IRB) of the NIH. Written informed consent was obtained from all patients prior to the procedures. For this report patients and images were selected if they could demonstrate the feasibility and value of the presented fusion technique.
Morphologic imaging was performed with and without contrast enhancement using CTi and Light Speed CT (GE Medical Systems, Milwaukee, WI) and 1.5 Tesla magnetic resonance imaging (MRI) (GE Medical Systems, Milwaukee, WI). Functional imaging included fluoro-18 labeled deoxyglucose (FDG)-PET (GE Medical Systems, Milwaukee, WI) and Dynamic Contrast enchanced (DCE)-MRI using the above mentioned scanner. Prior to RFA (pre-treatment phase) each patient underwent a contrast-enhanced CT. Some patients also had PET and contrast enhanced MRI or DCE-MRI scans. After the initial RFA treatment was completed, a 50–100 ml bolus of iodinated contrast medium was administered intravenously and a CT of the target organ was performed (procedural phase). After two month patients were followed up with CT and in some cases additionaly with PET or DCE-MRI (Post-treatment phase).
Source data was archived on the hospital picture archive and communication system (PACS, Kodak, Rochester, NY) in the standard imaging DICOM (Digital imaging and communications in medicine) format (Fig. 1). Relevant data sets were retrieved from the PACS onto a personal computer (PC) workstation (1.4 GHz processor, 512 MB RAM, MS Windows® 2000 Professional).
Image fusion of morphologic and functional data was performed prior, during and after RFA between the same (intra-modality) and different modalities (inter-modality) (Fig. 2). Intra- and inter-modality registration was carried out using both semi-automated landmark-based methods least squares , thin-plate-splines  and automatic voxel-similarity method, an optimized automated registration (OAR) [30, 31], available in the application MIPAV (Medical Image Processing and Visualization, National Institutes of Health, Bethesda, MD) using the above described workstation. Registration algorithms used included: 1) least-squares rigid registration, 2) thin-plate spline elastic registration, and 3) automated voxel-similarity measure affine registration using either correlation ratio or mutual information cost functions. Intra-modality registration of pre- and post-treatment CT datasets was accomplished using an automatic affine (12 degrees of freedom) registration algorithm. This intra-modality image fusion process used a correlation-ratio voxel-simularity cost function to guide the registration of the images. To enhance the visualization of two fused images, MIPAV provides many tools that allow the user to independently personalize the contrast, brightness and lookup tables for each dataset. Adjustment and blending the amount each image contributes to the final combined display can be performed in real-time.
Different thresholds and color lookup-tables were evaluated for visualization regarding the imaging modality and combination of data sets. Image fusion, volume and surface rendering including multi-planar visualization were performed on the same workstation using the above mentioned MIPAV software.
A laptop PC (1.4 GHz processor, 512 MB RAM, Windows® 2000 Professional) was used for intra-procedural fusions and for intra-procedural display (see Fig. 1).
Fusion results varied in accuracy depending upon organ shift, respiratory variations, lesion or organ shrinkage, and positional changes. The semi-manual landmark methods (least squares, thin-plate splines) required longer setup times due to selection of landmarks (~10–20 min without cropping). This was less practical for intra-procedural navigation, and required a trained user. However, applying the automated voxel-similarity affine registration algorithm to cropped volumes produced reasonable processing times, facilitating fusion during treatments (3–5 min with cropping). Cropping the image had two major advantages: first, cropping significantly reduced processing time and second, cropping localized the registration of the images, reducing non-linear distortion artifacts, thus improving the affine registration process. The voxel-similarity registration technique was the fastest and thus potentially most useful method for registration during treatments (Fig. 3). However, the landmark methods have a built-in internal control, and were better in selected cases for specific patients (Fig. 4, ,5).5). The landmark-based elastic method (thin plate spline) was more accurate for registration of organ shift in some cases (Fig. 6). However, elastic methods alter the actual imaging data, and thus have potential for error.
The CT scans provided primarily morphologic information on organ anatomy, tumor environment, adjacent large vasculature, and vascular supply (see Fig. 3; 4, a). Functional imaging by PET and DCE-MRI scans allowed functional lesion assessment of the metabolic (see Fig. 4, b) or pharmacokinetic microvascular status of the lesion (see Fig. 5, a). RFA needle trajectories were planned based on conventional mental registration, and retrospectively validated with fused images (see Fig. 6, b; Fig. 7). In addition, fused 3D-images with PET or DCE-MRI confirmed or facilitated neoplastic targets. Volume rendering of lesions helped to understand the spatial relation of the tumor and surrounding structures, which was not obvious from the conventional two-dimensional view (see Fig. 4, d). Similar fused combinations localized suspicious residual tumor that would need repeated RFA (see Fig. 5, 6, c). DCE-MRI showed suspicious residual untreated tumor (see Fig. 5, a), which was confirmed with pre treatment PET fused to pre treatment CT (Fig. 5, b).
Fused images especially helped to guide RFA in patients were CT alone could not define recurrent tumor (see Fig. 6, c and d) The procedural images were registered to the pre-treatment images. Fused images of pre-treatment-CT and intra-procedural post-RFA CT rapidly defined adequate treatment margins well using OAR technique (see Fig. 7).
If questionable areas are identified with possible residual tumor, then repeated RFA can be considered before the patient is removed from the room, although fusion was not used prospectively in our study to alter any treatment plan.
Conventional assessment is based on a visual comparison to determine whether the pre-treatment lesions and post-treatment areas are well matched (see Fig. 3, a and b). However, registered and fused images may provide better visualization of subtle differences (see Fig. 3, c and d; ;7,7, ,8,8, ,9).9). Follow-up imaging can be fused to pre-RFA imaging to compare tumor to treatment margin, however this method becomes less useful the longer the follow-up, given that post-RFA lesions will shrink with time (see Fig. 9).
Free network flow of DICOM images was achieved between PACS, imaging scanners and PC workstations. MIPAV provided both volume rendering as well as multi-planar display of fused images for lesion visualization with CT, MRI and PET datasets. Rigid (see Fig. 4; 9, a–c) and elastic (see Fig. 3, Fig. 9, d–f) registration methods were successfully applied intra-operatively but can be challenged due to organ shrinkage at follow up. Choosing accurate and homologous landmarks, in each of the volume datasets was rather time consuming, even more for inter modality registration e.g. PET-CT. The OAR method was faster and appreciated during the procedural phase. Cropping the image had two major advantages; first, cropping significantly reduced processing time. Second, cropping localized the registration, reducing non-linear distortion artifacts, thus improving the affine registration process by narrowing the processing to the volume of interest.
Previous studies have shown that image fusion is a powerful methodology to enhance diagnostic and follow up imaging in cancer patients [30, 32–34]. Different registration algorithms have been used successfully for non-interventional applications in the past [29, 35], and were successfully applied in this study to RFA for liver and kidney tumors. Registration and fusion may be useful during interventional procedures and may assist before, during, and after RFA . Often morphologic and functional imaging studies provide separate and complimentary information. Registration of pre- and post-RFA images may provide another window into the often-subtle spatial relationships between tumor and post-RFA thermal lesion. Conventional interpretation uses mental registration ; however computer processing may provide a more objective and exact view . Image fusion has matured predominately for rigid structures (brain and bone), and many technical details have already been refined, however, several major problems remain while performing image fusion after RFA.
The problem of respiratory motion is inherent to the imaging itself. Image fusion is easily performed on the brain  because the skull is a rigid structure that prohibits significant movements. Unlike the brain, the abdominal cavity is not stationary, and organs can significantly alter their shape and location. These changes can be due to breathing, the position of the patient on the table, organ shift, change in organ shape, hydration status, stomach contents, and the RFA procedure itself etc. Not surprisingly, this caused mis-registration and hampered the fusion process in the kidney and liver in some cases.
Organ shift and shrinkage were encountered, as thermal lesions tend to shrink after RFA. This is problematic for retrospective fusion of post-RFA images to pre-RFA images to assess for adequacy of treatment margin (see Fig. 9). If weeks to months are allowed to pass before post-RFA imaging, then registration may show the now-shrunken thermal lesion to be smaller than the tumor, giving the false impression of inadequate treatment. This occurred repeatedly when we compared 2-month post-RFA images to pre-RFA images in kidney tumor patients, who did not suffer subsequent recurrence years later (see Fig. 7).
The size of the safety margin may influence the utility of this technique . In the liver, a 5–10 mm margin of normal tissue burned may be easier to mentally co-register than a patient with a familial renal cell carcinoma, where only a several mm margin is desirable, to preserve normal kidney function given the predisposition for synchronous and metachronous tumor development over a lifetime. For the latter, this technique may be more useful.
If a tumor only presents during arterial phase imaging, co-registration may enable using the spatial information of that brief arterial phase for localization during a procedure. Image registration lets the physician use off-line prior imaging in the procedure room. Any imaging dataset can be registered to CT space, which can then be used to guide robotic needle placements for point and click tumor destruction . This is a powerful tool that may gain importance in the future as tumor-specific and cell-specific contrast agents are developed. Fusion may also enable biopsy of metabolically active regions of a tumor, which could facilitate more accurate biopsy and improved information on the temporal and spatial evolution of a tumor genomic or proteomic profile. This in turn could help tailor patient-specific drug regimens.
Region cropping proved to be a rapid and simple method of scaling down the large imaging datasets into a computationally workable size. While future optimizations are planned and processing speeds are improving, the registration process took 3–5 min, which makes this technique clinically relevant for intra-procedural monitoring and navigation. Monitoring and navigation during RFA currently suffers from imaging limitations. Ultrasound gas shadows the burn, CT contrast increases risk of renal toxicity, CT fluoroscopy has potentially high radiation doses to the user, and MR thermometry is not widely available, is costly, and requires a RF switch box or alternating imaging Rf-signals with treatment currents. Electromagnetic tracking during RFA may register pre-procedural imaging to the patient for use during needle manipulation and is being further investigated as an alternate method allowing use of pre-procedural imaging during interventions .
Versatile fusion software with multiple available methods of rigid and elastic registration may improve chances for optimal fusion for a given patient [40, 41], as each method has own inherent strengths and weaknesses. However, fusion can facilitate interventions in select scenarios. Further validation is indicated before these techniques can be routinely utilized or applied to navigation systems or treatment planning software.
Current methods of monitoring treatment during RFA are inadequate and may represent the largest technical limitation of RFA today. Early detection of the tumor activity could potentially improve outcomes by allowing for early repeat intervention before regrowth results in a geometrically-unfavorable configuration.
In addition, method 1 (least squares) generates a rigid transformation, which involves 6 degrees of freedom (3 rotations and 3 translations). Method 2 (thin plate splines) is non-linear and can provide a richer registration than method 1 since this method can address non-linear registration problems (e. g., breathing artifacts, organ shift, and organ deformation). However, the accuracy of these two landmark registration methods is sensitive to user training and expertise in choosing landmarks. In addition, it can be time consuming and difficult to find enough landmarks to produce and acceptable registration.
However, the affine voxel-similarity automatic method is invariant to the user and often provides an acceptable result. This method has up to 12 degrees of freedom (3 rotations, 3 translations, 3 scale and 3 skew). While this may yield useful results, this method does not address breathing artifacts very well. Voxel similarity methods use statistics based on comparisons of voxel intensities between two datasets. Correlation ratio and cross-correlation measures are typically used to register intra-modality datasets. Normalized mutual information is typically used for inter-modality registration. Correlation ratio and normalized mutual information cost functions were used in this study for method 3 for intra and inter modality registration, respectively.
To fully visualize two fused images, it is important to be able to adjust the colorization or lookup tables, brightness, and contrast of each image independently. In addition, it is also important to be able to adjust the amount of blending between the two fused images. Having the ability to modify these image attributes greatly improves the visualization of lesions, vessels, and necrotic tissue. Such visualization is vital to the accurate assessment of RFA safety margins (see Fig. 3 and Fig. 7). The most subjectively effective color schemes were saved, which allowed the further automation of routine post-processing steps. In addition, surface-rendering techniques allowed for localization and the visual quantification of both pre-treatment lesion and post-treatment ablation volumes. Image processing and multimodality fusion are mature diagnostic tools that should be further evaluated for potential utility during interventional radiology procedures.
This work was supported in part by the Intramural Research Program of the NIH.
The authors thank Stephen Bacharach, MD, Ingmar Bitter, PhD and Grace Ko, MD for their support.