Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Proc IEEE Int Symp Biomed Imaging. Author manuscript; available in PMC 2010 July 14.
Published in final edited form as:
Proc IEEE Int Symp Biomed Imaging. 2010 April 1; 2010: 964–967.
doi:  10.1109/ISBI.2010.5490148
PMCID: PMC2903900



Neonatal brain MRI segmentation is challenging due to the poor image quality. Existing population atlases used for guiding segmentation are usually constructed by averaging all images in a population with no preference. However, such approaches diminish the important local inter-subject structural variability. In this paper, we propose a multi-region-multi-reference strategy for atlas building from a population. In brief, the brain is first parcellated into multiple anatomical regions, and for each region, the population images are classified into different sub-populations. The exemplars in sub-populations serve as structural references when determining the most suitable regional atlas for a to-be-segmented image. A final atlas is generated by combining all selected regional atlases, and a joint registration-segmentation strategy is employed for tissue segmentation. Experimental results demonstrate that segmentation with our atlas achieves high average tissue overlap rates with manual golden standard of 0.86 (SD 0.02) for gray matter (GM) and 0.83 (SD 0.03) for white matter (WM), and outperforms other atlases in comparison.

Index Terms: Tissue segmentation, multiple atlases, joint registration-segmentation, neonatal imaging


Brain tissue segmentation, which classifies brain tissues into meaningful structures such as gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF), is a key step in medical imaging analysis. It has been widely employed to quantify brain tissue volumes, to detect tissue boundaries, and to aid cortical surface evaluation. Numerous methods have been proposed for image segmentation [1]. However, many of them cannot be directly extended to the case of pediatric brain images, especially neonates, owing to confounding factors such as their low spatial resolution, insufficient tissue contrast, and ambiguous tissue intensity distributions.

Due to these difficulties, image intensity alone is deemed insufficient for effective neonatal brain MRI segmentation, and there is now thus a growing acceptance of knowledge-based algorithms [1]. Among them, a popular method is to summarize segmentation results from pre-segmented images, to construct a probability atlas, comprising of GM, WM, and CSF probability maps, and then employ this atlas as prior knowledge to aid segmentation. To build such an atlas, 3 strategies are commonly used [2]: 1) single individual atlas; 2) average-shape atlas; and 3) multiple individual atlases with decision fusion. Using only a single subject as the atlas provides better anatomical details, but it will inevitably bias the segmentation results when applied to other subjects. The methods in category 3 apply the individual-atlas-based segmentation multiple times with different atlas subjects, and then fuse the multiple segmentations into a final result with majority voting or more sophisticated methods like STAPLE [3]. The major concern of this technique is that its computation cost is quite high due to the need of multiple segmentations. Most current neonatal segmentation methods fall into the category 2. For example, Prastawa et al. [4] constructed an atlas by averaging 3 semi-automatic segmented neonatal images after alignment using affine transformation. Weisenfeld et al. [5] obtained an unbiased atlas by averaging the probability maps of 20 newborn subjects, which are non-rigidly aligned with a simultaneous group-wise registration. In summary, these atlases are built by averaging a group of spatially-normalized images equally, with no preference among the subject images. However, equally averaging subject images in spite of existing inter-subject variability will lead to a blur atlas with reduced capacity to guide segmentation, especially the fine anatomical structures. Some recent studies [6] indicate that building atlas by choosing subjects from a population by image similarity or even demographic measures like age yields higher segmentation performance than by random selection. Motivated by this, we present here a multi-region-multi-reference approach, which dynamically estimates multiple atlases for different anatomical regions, with the final goal of constructing a subject-specific atlas for more effective neonatal segmentation.

We address two issues in this paper. First, taking brain as a single entity when averaging all images has limitations. This approach assigns weights globally to all voxels irrespective of their local structural characteristics. A local weighting strategy is therefore desired to better represent the different local shape patterns in the brain. We perform an anatomical meaningful parcellation to separate the brain into multiple subregions so that the following atlas building can be performed for each subregion separately. Second, for images in each subregion, their anatomical patterns could be learned. We employ a recently proposed exemplar-based clustering technique called affinity propagation [7] to cluster the images in each subregion based on regional anatomical shape information. The exemplars of each subregion are obtained and then serve as references with which comparison can be made with new query subject to determine the most likely regional probability atlas. These regional atlases are then combined to form a complete brain atlas to guide tissue segmentation. Coupled with a joint registration-segmentation strategy, an adaptive subject-specific atlas as such can be shown to give significant improvement in neonatal brain MRI segmentation, as validated by our experimental results.


A flow chart of the proposed framework is shown in Fig. 1, composed of three steps. Step 1: All presegmented subjects in a given population are normalized into a common coordinate space, and averaged to construct an average-shape atlas (referred to as population atlas). Step 2: The population atlas is then parcellated into multiple anatomical subregions. Based on this parcellation, the individual subjects are subdivided, and the subdivisions are used to construct a set of regional exemplars and their corresponding probability atlases, named collectively as the multi-region-multi-reference atlases. Step 3: Each query subject is compared with the multi-region-multi-reference atlases to determine the most likely regional probability atlases, which are finally combined together to form a subject-specific atlas for guiding segmentation. A joint registration-segmentation strategy is then adopted for more effective segmentation of the query image. Details are given in the following subsections.

Fig. 1
The multi-region-multi-reference framework for neonatal brain MRI segmentation.

2.1. Population normalization

Prior knowledge to aid segmentation is gathered from a previous processed population, involving 68 neonatal subjects (38 males and 30 females) scanned at 1.3 (SD 0.7) months. These neonatal images are already segmented by a longitudinal guided technique [8] and their GM, WM and CSF probability maps are obtained. The query subjects to be segmented in our experiments are not included in this dataset.

To normalize all subjects onto a same coordinate space, two steps are performed. Affine transform is first used for rough alignment. Then, a nonlinear registration is performed to more effectively remove the inter-subject variability. To avoid the registration bias by preselecting template, we adopt a group-wise normalization approach, similar to [9], to transform all subjects to their space center, to which all subjects can be warped with minimum transformation.

Specifically, we first average the affine transformed images {Si, i = 1, …, N} to obtain an initial template T0. Then a nonlinear registration algorithm HAMMER [10] is performed to register each subject to the current template Tt−1, resulting in the warped subjects Sit and taking their averaged image as updated template Tt. By iteratively alternating between subject alignment and template generation, the registration process will stop when the difference between previous and current template is lower than a certain threshold. In this way, all images will eventually be warped onto a common space.

A final template, referred to as population atlas in this paper, can then be obtained by averaging all normalized subjects with equal weight. The atlas created using this approach is generally blurred and lacks anatomical details. We will describe in the next section a better approach to construct an atlas from the population.

2.2. Learning multi-region-multi-reference atlases

There are obviously many methods to parcellate a brain image, such as using a labeled atlas, regular grid, and watershed segmentation. We employ the watershed segmentation because it is data-driven and so the regions with similar anatomy can be grouped together. Before parcellation, we perform an edge-preserving anisotropic diffusion to adaptively smooth the population atlas described in subsection 2.1 for removing noise, so that the possible oversegmentation due to image noise could be minimized. It is worth noting that, more iterations of anisotropic diffusion will result in a more homogenous image, and the parcellated regions are thus larger in size and smaller in number. Too few regions cannot capture the various anatomical patterns while too many regions may raise a problem of heavy computation cost. In this paper, we apply a moderate level of smoothing and the population atlas is then parcellated into 76 subregions. This identical parcellation is directly propagated to the population because they are in the same space, and so all images can be equally parcellated.

For each subregion, we use affinity propagation [7] to cluster the population into sub-populations and, at the same time, determine their respective exemplars. Conventional clustering methods such as the k-means algorithm require that the number of clusters to be determined a priori, and the centers of the clusters are often randomly initialized. In contrast, affinity propagation initially deems all data points as potential exemplars and determines among them a suitable number of exemplars via a message passing mechanism. To use affinity propagation, we need to define first a pairwise image similarity for a given pair of intersubject subregions. In this paper, we use the popular mutual information (MI) to compute the pairwise similarity matrix.

Upon computing the pairwise similarity values of all the N subjects in the population, an N by N similarity matrix S can be obtained for each subregion. The diagonal elements of similarity matrix S are defined as “preferences”. The higher preference Skk, the larger probability of corresponding subject image k of becoming an exemplar. In our case, the preferences are initialized as the median of input similarities and all subjects are given the same probability. For a better understanding of this technique, we include the definitions of responsibility and availability below, although the details can be found in [7].

The responsibility Ri,k, sent from a data point i to a candidate exemplar point k, reflects the accumulated evidence of how well-suited point k is to serve as the exemplar for point i:


where the initial availabilities Ai,k are set to zero.

The “availability” Ai,k, sent from a candidate exemplar point k to point i, reflects the accumulated evidence of how appropriate it would be for point i to choose point k as its exemplar:


where the self-availability Ak,k is updated by:


Ri,k and Ak,k are alternatively and iteratively updated, and this stops when local decisions stay constant for a few iterations. The representativeness for each subject can then be defined:


where Ei,k can be seen as the updated similarity matrix. The diagonal elements Ek,k are taken as exemplars if larger than zero.

For each exemplar, its representativeness values are used as weights to obtain regional weighted average of the subjects in the population and construct the regional sub-population probability atlas (composed of GM, WM, and CSF probability maps). By doing so, the multi-region-multi-reference atlases can be constructed.

2.3. Query image segmentation

The query image is aligned to the template space of subsection 2.1 by an intensity-based nonlinear registration [11]. To construct a subject-specific atlas for the query image, the best match exemplar for each image region is determined from the multi-region-multi-reference atlases, and the corresponding regional probability atlas is taken as the most likely atlas for that particular region. All the regional atlases are then combined to form a complete whole-image probability atlas, serving as a prior for segmentation guidance of the image.

After obtaining the subject-specific atlas, a joint registration-segmentation strategy [12] is adopted for more effective segmentation of neonatal brain images. In brief, the algorithm involves alternating between atlas-to-subject registration, atlas-based tissue segmentation, and bias correction. The atlas represents the tissue prior probabilities, modeled by a mixture of Gaussians (MOG), for each voxel in an image. Bayes rule is employed to combine these prior probabilities with the WM, GM, and CSF probabilities estimated from voxel intensities to obtain the posterior probabilities, which are taken as the voxel tissue membership values. Based on the segmentation results, the registration between the atlas and the subject is refined by HAMMER [10] for more accurate segmentation. The registration step corrects the misalignment between the subject and the atlas, which, when not corrected, will have adverse effect on the segmentation, because tissue spatial distribution statistics cannot be accurately determined. These processes are iterated until the stopping criteria are met, e.g., the segmentation results do not change for a few iterations.


The performance of the proposed multi-region-multi-reference framework for neonatal segmentation is evaluated using T2 images of 10 neonatal subjects (6 males and 4 females), with postnatal age ranging from 26 to 60 days. MR images of these neonates were acquired using a 3T head-only MR scanner. For evaluation purpose, 2 sagittal, 3 coronal, and 3 transverse slices of all the images were manually segmented by an expert. Results yielded by the proposed segmentation algorithm were compared with that of manual segmentation. The Dice ratio (DR), DR = 2|S1S2|/(|S1| + |S2|), is used for measuring tissue overlap rate for manual segmentation S1 and automatic segmentation S2. We further evaluated our approach with two other atlases. The first method (referred to as Population-A), uses a population atlas, which was provided by Altaye et al. [13], as guidance for segmentation. The atlas was created from 76 infants with ages ranging from 9 to 15 months. The second method (referred to as Population-B), uses the population atlas which is generated using the approach described in Section 2.1. For fair comparison, the same joint registration-segmentation strategy is used for all the atlases to segment the brain images. Shown in Fig. 3, (b)–(d) are the GM probability maps of Population-A, Population-B, and Proposed method, respectively; (f)–(h) are their final segmentation results. By visually comparing with the original T2 image and its manual segmentation result, it can be seen that the proposed method provides a more detailed segmentation, especially for the fine structures.

Fig. 3
GM probability maps and segmentation results. (a) Original T2 image, (e) manual segmentation, (b)–(d) probability maps of Population-A, Population-B and the proposed method, and (f)–(h) the respective final segmentation results, respectively. ...

Quantitative results are shown in Fig. 4. Using the manual segmentations as ground truth, the proposed method yields significant higher average tissue overlap rates with GM and WM as Dice ratio of 0.86 and 0.83, compared to 0.81(p=0.001) and 0.75 (p<0.001) by Population-A, and 0.84 (p=0.04) and 0.78 (p=0.02) by Population-B. No significant difference was found between Population-A and Population-B.

Fig. 4
The Dice ratios comparison for the segmentations obtained using Population-A, Population-B, and Proposed method, respectively.


A novel multi-region-multi-reference framework for neonatal brain image segmentation is proposed in this paper. Our proposed atlas is spatially adaptive, and multiple atlases are selected for better representing the local shape variations. This multi-region-multi-reference approach is able to generate an anatomically similar atlas for the query image, thus benefiting subsequent atlas-based segmentation. Experimental results demonstrate that our method yields the highest agreement with manual segmentations, and outperforms two population-atlas based segmentation methods.

Fig. 2
The surrounding images depict the exemplars and their corresponding WM probability map for a given subregion. At the center is the query subject overlaid with its parcellation. By selecting the best matched exemplar for the query image in each subregion, ...


This work was supported in part by grants EB006733, EB008760, EB008374, EB009634, NS055754, MH064065, and HD053000.


1. Pham DL, Xu C, Prince JL. A survey of current methods in medical image segmentation. Annual Review of Biomedical Engineering. 2000;2:315–337. [PubMed]
2. Rohlfing T, Brandt R, Menzel R, Maurer CR., Jr Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains. Neuroimage. 2004;21:1428–1442. [PubMed]
3. Warfield SK, Zou KH, Wells WM. Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Transactions on Medical Imaging. 2004;23:903–921. [PMC free article] [PubMed]
4. Prastawa M, Gilmore JH, Lin W, Gerig G. Automatic segmentation of MR images of the developing newborn brain. Medical Image Analysis. 2005;9:457–466. [PubMed]
5. Weisenfeld NI, Mewes AUJ, Warfield SK. Segmentation of newborn brain MRI. Proceedings of the 3rd IEEE International Symposium on Biomedical Imaging: Macro to Nano; 2006. pp. 766–769.
6. Aljabar P, Heckemann RA, Hammers A, Hajnal JV, Rueckert D. Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy. Neuroimage. 2009;46:726–738. [PubMed]
7. Frey BJ, Dueck D. Clustering by passing messages between data points. Science. 2007;315:972–976. [PubMed]
8. Shi F, Fan Y, Tang S, Gilmore JH, Lin W, Shen D. Neonatal brain image segmentation in longitudinal MRI studies. Neuroimage. 2010;49:391–400. [PMC free article] [PubMed]
9. Joshi S, Davis B, Jomier M, Gerig G. Unbiased diffeomorphic atlas construction for computational anatomy. Neuroimage. 2004;23:151–160. [PubMed]
10. Shen D, Davatzikos C. HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Transactions on Medical Imaging. 2002;21:1421–1439. [PubMed]
11. Shen D. Image registration by local histogram matching. Pattern Recognition. 2007;40:1161–1172.
12. Ashburner J, Friston KJ. Unified segmentation. Neuroimage. 2005;26:839–851. [PubMed]
13. Altaye M, Holland SK, Wilke M, Gaser C. Infant brain probability templates for MRI segmentation and normalization. Neuroimage. 2008;43:721–730. [PMC free article] [PubMed]