Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC2847185

Formats

Article sections

Authors

Related links

Med Image Comput Comput Assist Interv. Author manuscript; available in PMC 2010 March 30.

Published in final edited form as:

Med Image Comput Comput Assist Interv. 2007; 10(Pt 1): 933–941.

PMCID: PMC2847185

NIHMSID: NIHMS181613

Yuanjie Zheng,^{1,}^{2} Jingyi Yu,^{2} Chandra Kambhamettu,^{2} Sarah Englander,^{1} Mitchell D. Schnall,^{1} and Dinggang Shen^{1}

See other articles in PMC that cite the published article.

Dynamic enhancement causes serious problems for registration of contrast enhanced breast MRI, due to variable uptakes of agent on different tissues or even same tissues in the breast. We present an iterative optimization algorithm to de-enhance the dynamic contrast-enhanced breast MRI and then register them for avoiding the effects of enhancement on image registration. In particular, the spatially varying enhancements are modeled by a Markov Random Field, and estimated by a locally smooth function with boundaries using a graph cut algorithm. The de-enhanced images are then registered by conventional B-spline based registration algorithm. These two steps benefit from each other and are repeated until the results converge. Experimental results show that our two-step registration algorithm performs much better than conventional mutual information based registration algorithm. Also, the effects of tumor shrinking in the conventional registration algorithms can be effectively avoided by our registration algorithm.

Image registration is a fundamental problem in medical imaging. Classical image registration techniques assume corresponding pixels in two images have consistent intensities or at least are strongly correlated [1]. For instance, it is commonly assumed that if two pixels have the same intensity in one image, they will have the same intensity in the other image [2]. However, this assumption is not always valid in many applications. For example, in dynamic contrast enhanced (DCE) magnetic resonance (MR) images, tumor and non-tumor tissues may have similar intensities at a pre-contrast image, but then appear very dissimilar at post-contrast images due to variable signal enhancements [3,2]. Moreover, even same tissue (such as tumor tissue) [4] can appear different at various contrast enhancement stages, due to different uptakes of the contrast agent. These spatially varying enhancements lead to serious problems for the conventional registration algorithms which assume consistent intensities or strongly correlated intensities in the two images under registration [5]. One typical problem is that a tumor region can be significantly deformed during registration of DCE-MR images in order to maximize the mutual information as reported in [6].

Many research efforts have been proposed to design good correlation metrics in order to cancel out intensity inconsistencies across the images. Normalized mutual information (NMI) [7,3] has been used to account for the change of image intensity and contrast in registering DCE-MR images. However, NMI only works well in the regions with similar enhancement characteristics [8] and poorly handles the images with spatially varying enhancement like the DCE-MR breast images [4]. Model-based approaches, such as the pharmacokinetic (PK) model [2], have also been used to characterize pixel-wise enhancement-time signal. In theory, one can combine the PK model with conventional registration algorithms to get better performance, provided that the temporal enhanced signals comply well with the PK model. In practice, the PK model alone is often insufficient to model the intensity variations. Although the PK model with addition of a noise term can potentially better represent temporal enhanced signals [9], the noise term alone is insufficient to model inconsistency of intensity enhancements between the two images.

In this paper, we present a different approach to solve the intensity inconsistency of DCE-MR images. Instead of finding the physical model that represent this inconsistency, we set out to find the intensity transformation between the images, i.e., estimating enhancements. Specifically, we calculate the ratio of the intensity for each tissue point over the two images. The resulting ratio image is called the enhancement map. For perfectly registered images, the enhancement map is simply the ratio between the reference and the target images. For time-lapse DCE-MR images, computing the enhancement map requires registration between the target and the reference image due to possible patient motion.

We model the enhancement map by a Markov Random Field, and estimate it by a locally smooth function with boundaries using a graph cut algorithm [10]. Then, we use the estimated enhancement map to eliminate the enhancements between two images under registration, and further use a B-spline based registration method to register those de-enhanced images. These two steps are incorporated into an iterative optimization algorithm, and are repeated iteratively until convergence, similar to [11]. Experimental results show that the elimination of spatially variable enhancements in the DCE-MR images can significantly improve the registration accuracy, compared to the conventional mutual information based registration algorithms [7].

Given two images *I* and *J*, our goal is to find the correspondence mapping *T* that warps every pixel from *I* to *J*. We use notion *T* (*I*) to represent the resulting image after applying *T* on *I*. An ideal registration should perfectly align *T* (*I*) with *J*. However, *T* (*I*) and *J* can be different due to intensity changes η between *I* and *J*, e.g., enhancement variations in DCE-MR breast images. In addition, both *I* and *J* can be corrupted by noise ε. Therefore, registration methods that directly measure similarity of pixels intensities between the images might fail.

Notice, if we know η in prior, we can simply cancel out the enhancement between *I* and *J* and then apply existing registration algorithms to recover *T*. Similarly, if we know *T* in prior, we can then solve η using *T* (*I*) and *J*. However, neither η nor *T* is known to us. Therefore, our goal is to simultaneously find the optimal *T* and η by maximizing a posteriori (MAP) given *I* and *J*:

$$T,\eta =\text{arg}\phantom{\rule{thinmathspace}{0ex}}\underset{T,\eta}{\text{max}}\phantom{\rule{thinmathspace}{0ex}}\mathcal{P}\left(T,\eta |I,J\right)$$

(1)

We propose an iterative optimization algorithm to approximate the MAP of *T* and η. We start with a rough registration using existing algorithms. At each iteration, we first estimate the optimal η given the current estimation of *T*. We then re-compute the registration by cancelling out η in *J*. The iteration continues until the results converge.

Assume we have estimated the registration *T* from *I* to *J*, our goal is to compute the enhancement map η between *I*′ and *J*, where *I′* = *T* (*I*). The enhancement is defined as

$${C}^{\prime}\left(s\right)=\frac{J\left(s\right)-{I}^{\prime}\left(s\right)}{{I}^{\prime}\left(s\right)},\text{}s\in \Omega $$

(2)

where *s* represent the pixels inside image region Ω.

Enhancement map has been widely used to analyze DCE-MR images in breast cancer diagnosis [12,4]. We assume the enhancements change spatially in a smooth way, with discontinuities appeared between different kinds of enhancements, e.g., tumor boundaries.

If the registration *T* is exact, then η can be directly computed using equation (2). In practice, *T* can be inaccurate and both images are corrupted by noise ε. Notice, the spatial smoothness and tumor boundary discontinuity of the enhancement map are very similar to those seen in disparity maps in computer vision. Recent research [10] have shown that fields satisfying such properties can be modeled as a Markov Random Field. Methods such as belief propagation or graph-cut are very efficient to solve the MAP of Markov Random Fields.

We start with an initial estimation of the enhancement map by computing *C′* using equation (2). We then discretize the range of the enhancements into a finite number (e.g., 15) of levels, which are initially obtained by clustering with k-means algorithm. Our goal is to assign every pixel in *C′* with an enhancement label and find the MAP of the enhancement MRF η. In our implementation, we modify the graph cut algorithm [10] to compute η that maximizes the labeling consistency and overall smoothness. The energy function we choose to minimize is defined as

$$E={\displaystyle \sum _{s\in \Omega}{E}_{1}\left({\eta}^{\prime}\left(s\right)\right)+{\lambda}_{1}}\phantom{\rule{thinmathspace}{0ex}}{\displaystyle \sum _{<s,t>\in \mathcal{N}}{E}_{2}\left({\eta}^{\prime}\left(s\right),{\eta}^{\prime}\left(t\right)\right)+{\lambda}_{2}}\phantom{\rule{thinmathspace}{0ex}}{\displaystyle \sum _{<s,t>\in \mathcal{N}}{E}_{3}\left({\eta}^{\prime}\left(s\right),{\eta}^{\prime}\left(t\right)\right)}$$

(3)

*E*_{1} computes the difference between the assigned level value and enhancement computed by equation (2) for all pixels, which is defined as:

$${E}_{1}\left({\eta}^{\prime}\left(s\right)\right)=\mathcal{G}\left(|{\eta}^{\prime}\left(s\right)-{C}^{\prime}\left(s\right)|\right)$$

(4)

where $\mathcal{G}\left(x\right)=\frac{x}{x+1}$.

*E*_{2} corresponds to the smoothness term in standard graph cut algorithms. It measures the consistency of the enhancement levels between the neighboring pixels in :

$${E}_{2}\left({\eta}^{\prime}\left(s\right),{\eta}^{\prime}\left(t\right)\right)=1-\delta \left({\eta}^{\prime}\left(s\right)-{\eta}^{\prime}\left(t\right)\right)$$

(5)

where δ is a Kronecker delta function.

Finally, we introduce a new term *E*_{3} that further constrains the enhancement variations between pixels, where

$${E}_{3}\left({\eta}^{\prime}\left(s\right),{\eta}^{\prime}\left(t\right)\right)=\mathcal{G}\left(|{\eta}^{\prime}\left(s\right)-{\eta}^{\prime}\left(t\right)|\right).$$

(6)

The use of *E*_{3} guarantees that even if the neighboring pixels have different enhancement levels, their differences should be constrained within a small range. In our experiments, we find the use of *E*_{3} is critical to maintain spatial smoothness of the recovered enhancement map.

In equation (3), λ_{1} and λ_{2} are used for adjusting the relative importance of the three energy terms. They are both set to 1 in our experiments. We also find 15 levels clustered by k-means are sufficient to accurately approximate an enhancement map for a typical pair of DCE-MR breast images. This can be shown from the results of enhancement map estimated by our graph cut on a pair of typical enhanced images in Fig. 1.

Demonstration of enhancement estimation result by our method. (a) Pre-contrast image. (b) Post-contrast image. (c) De-enhanced post-contrast image. (d) Initial enhancement map calculated from (a) to (b). (e) Enhancement map clustered by a k-means algorithm. **...**

Once we recover the enhancement map η between *I′* and *J*, we can then “de-enhance” *J* to the same level of *I′* using η. Notice *J _{de−enhanced}* should be approximately the same as

$${J}_{\mathit{\text{de}}-\mathit{\text{enhanced}}}\approx {I}^{\prime}=\frac{J}{\eta +1}\text{}s\in \Omega .$$

(7)

In Fig. 1, we use the estimated η to de-enhance the post-contrast MR breast image. The resulting image maintains similar spatial characteristics as the unenhanced one while preserving important features such as tumor boundaries.

We have developed a two-step iterative optimization algorithm to simultaneously recover the MAP registration *T* ^{*} and enhancement map η^{*}. We start with an initial registration using the conventional B-spline based non-rigid registration on the pre-contrast and the post-contrast MR images [7]. We then:

- Find the initial estimate η′ for η based on methods explained in section 2.3.
- Repeat below steps until convergence:
- Compute
*T′*= arg max_{T}(*T*|η′,*I, J*) - Compute η′ = arg max
_{η}(η|*T′, I, J*)

In step (a) of each iteration, we de-enhance the target image using equation (7), to obtain *J _{de−enhanced}*. We then estimate

Fig. 2 illustrates the intermediate results using our method on MR breast images for registration and enhancement map estimation. From the results, we can see the steps of enhancement estimation and registration help each other for achieving better results.

The performance of our enhancement estimation and image registration algorithm is extensively evaluated on both simulated data and real data as described by separate sections next.

For generating the simulated dataset, we first select a pre-contrast image of a real patient, such as the one shown in Fig. 1 (a). Second, we use the enhancements computed between this pre-contrast image and one post-contrast image of the same patient by equation (2) as a simulated enhancement map, to enhance our selected pre-contrast image and obtain a simulated post-contrast image. Third, we further use simulated motions, represented by B-Splines, to deform this simulated post-contrast image. Finally, we add Gaussian noise (with zero mean and standard deviation of 20) to this simulated post-contrast image.

The pre-contrast image and its final simulated post-contrast image are shown in Fig. 3 (a) and (b), respectively. To be clear, we only show the region around tumor. Notice that both simulated enhancement map and motions are pre-known.

Results on simulated data. (a) A pre-contrast image with tumor, selected from Fig. 1 (a). (b) A simulated post-contrast image, using simulated enhancement in (f). (c) An de-enhanced post-contrast image by our algorithm. (d) Warped pre-contrast image using **...**

We have compared our algorithm with the NMI-based method on this simulated dataset. Notice that both algorithms use the B-spline-based deformation representation [7] with 1 degree and 13 control points for each axis. To compensate for the intensity inconsistencies between the pre- and post-contrast images, the NMI-based approach estimates different intensity distributions in the two images while our algorithm first computes and removes the enhancement map and then directly uses the intensity difference metric. The results are shown in Fig. 3.

These results demonstrate at least three pieces of better performance of our algorithm. First, after removing the estimated enhancements (Fig. 3(c)), the simulated post-contrast image looks very similar to the original pre-contrast image (Fig. 3(a)). This shows the effectiveness of our graph cut algorithm in estimating enhancements. Second, the conventional registration algorithm produced errors especially at tumor as reflected by an enlarged tumor (Fig. 3(d)), while our algorithm still worked very well (Fig. 3(e)). Finally, the better registration result by our algorithm can also be observed as more similarity between the estimated (Fig. 3(h)) and the simulated enhancement maps (Fig. 3(f)), compared to that produced by the conventional registration algorithm (Fig. 3(g)). This is further demonstrated by the difference maps between the simulated and the estimated enhancements for our registration algorithm (Fig. 3(j)) and conventional registration algorithm (Fig. 3(i)), respectively.

Quantitatively, our registration algorithm is also better than the conventional registration algorithm. For example, by comparing the simulated motions with the estimated motions, our registration algorithm produced mean error of 0.44 pixel, standard deviation of 0.38 pixel, and maximum error of 2.4 pixels.

In contrast, the conventional registration algorithm produced mean error of 0.82 pixel, standard deviation of 0.57 pixel, and maximum error of 10.8 pixels.

Data acquired from eight patients are also used to further compare the performances of our method and conventional registration algorithm. For each DCE-MR image sequence, the pre-contrast and the last post-contrast images are selected for evaluation of both registration algorithms. Typical results on three subjects are shown in Fig. 4. Each row corresponds to one individual subject. The first column shows the pre-contrast images, while the second column shows the post-contrast images. The de-enhanced images by our method are shown in the third column. It can be observed that the de-enhanced images are very similar to the pre-contrast images in the first column, which indicates the effectiveness of our enhancement estimation method.

Comparisons of our registration algorithm with conventional registration algorithm on real data. Each row corresponds to one individual subject. See text for details.

The difference maps between the warped pre-contrast images and the deenhanced images are shown in the fourth and the fifth columns for conventional registration method and our registration method, respectively. It can be observed that our method generally produces smaller difference maps, indicating better registration results.

Our algorithm completes the registration within 15 seconds on a PC of 1.2 GHz Intel Pentium M CPU and 2GB memory for images of a resolution 512 × 512 while the conventional algorithm takes about 3 seconds. For all the experiments demonstrated in this paper, our two-step optimization converges after 3 iterations.

We have presented an iterative two-step optimization algorithm to simultaneously estimate dynamic enhancements in DCE-MR images and eliminate them for robust registration. The estimation of dynamic enhancement map is particularly formulated by a MRF model, and completed by a graph cut algorithm. The experimental results on both real and simulated datasets show the relative accuracy of registration after de-enhancing the DCE-MR images by our method. Also, by comparing with conventional registration methods using NMI, our registration algorithm can effectively register the tumor regions across different images, thus potentially avoiding the effects of tumor shrinking or enlarging that commonly happen in conventional registration methods. In the future, we will test our algorithm on more datasets available in our institute, adapt it to 3D case, and re-formulate it for group-wise registration of multiple contrast-enhanced breast images.

1. Maintz J, Viergever M. A Survey of Medical Image Registration. Medical Image Analysis. 1998;2:1–36. [PubMed]

2. Hayton P, et al. Analysis of dynamic MR breast images using a model of contrast enhancement. Medical Image Analysis. 1997;1:207–224. [PubMed]

3. Tanner C, et al. Volume and shape preservation of enhancing lesions when applying non-rigid registration to a time series of contrast enhancing MR breast images. MICCAI 2000. 2000:327–337.

4. Schnall MD, Ikeda DM. Lesions diagnosis working group report. Journal of Magnetic Resonance Imaging. 1999;10:982–990. [PubMed]

5. Papademetris X. Integrated intensity and point-feature nonrigid registration. In: Barillot C, Haynor DR, Hellier P, editors. MICCAI 2004. LNCS. vol. 3216. Heidelberg: Springer; 2004. [PMC free article] [PubMed]

6. Rohlfing R, et al. Volume-preserving nonrigid registration of mr breast images using free-form deformation with an incompressibility constraint. IEEE Trans. Med. Imaging. 2003;22:730–741. [PubMed]

7. Rueckert D, et al. Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imag. 1999;18:712–721. [PubMed]

8. Pluim JPW, et al. Mutual information based registration of medical images: survey. IEEE Trans. Med. Imaging. 2003;22:986–1004. [PubMed]

9. Chen X, et al. Simultanous segmentation and registration of contrast-enhanced breast MRI. In: Christensen GE, Sonka M, editors. IPMI 2005. LNCS. vol. 3565. Heidelberg: Springer; 2005.

10. Boykov Y, et al. Fast approximate energy minimization via graph cuts. IEEE Trans. PAMI. 2001;23:1222–1239.

11. Wells WM, et al. Adaptive segmentation of MRI data. IEEE Trans. Med. Imaging. 2001;20:1398–1410. [PubMed]

12. Chen X, et al. Simultanous segmentation and registration for medical image. In: Barillot C, Haynor DR, Hellier P, editors. MICCAI 2004. LNCS. vol. 3216. Heidelberg: Springer; 2004.

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |