|Home | About | Journals | Submit | Contact Us | Français|
Automatic segmentation of cortical and subcortical structures is commonplace in brain MRI literature and is frequently used as the first step towards quantitative analysis of structural and functional neuroimaging. Most approaches to brain structure segmentation are based on propagation of anatomical information from example MRI datasets, called atlases or templates, that are manually labeled by experts. The accuracy of automatic segmentation is usually validated against the “bronze” standard of manual segmentation of test MRI datasets. However, good performance vis-a-vis manual segmentation does not imply accuracy relative to the underlying true anatomical boundaries. In the context of segmentation of hippocampal subfields and functionally related medial temporal lobe cortical subregions, we explore the challenges associated with validating existing automatic segmentation techniques against underlying histologically-derived anatomical “gold” standard; and, further, developing automatic in vivo MRI segmentation techniques informed by histological imaging.
The medial temporal lobe (MTL) is a brain region that has received considerable attention in the neuroimaging literature due to its primary involvement in the declarative memory and its early selective vulnerability to pathologies associated with dementia, epilepsy, and other neurological and psychiatric disorders . The MTL structures primarily involved in memory are the hippocampus and the parahippocampal gyrus. The hippocampus is a complex and heterogeneous structure that is constituted by several anatomically and functionally distinct subfields, including cornu ammonis 1–3 (CA1-3), dentate gyrus (DG), and subiculum. The parahippocampal gyrus is also formed by functionally distinct subregions: the entorhinal cortex (ERC), perirhinal cortex (PRC), and parahippocampal cortex (PHC). Advances in MRI technology have made it possible to visualize different hippocampal layers in routine MRI scans, which led to increased interest in recent years in quantitative analysis of hippocampal subfields and extrahippocampal MTL subregions using high-resolution structural and functional MRI . Quantitative analysis usually requires the structure of interest, in this case MTL subregions, to be segmented. Over the last decade, over 20 protocols for manual segmentation of MTL subregions have been developed , , and three automatic segmentation techniques have become available , , . However, currently, the segmentation of MTL subregions on in vivo MRI, either manual or automated, is based on heuristic rules, as many of the actual anatomical boundaries between subregions cannot be observed consistently. Presently, both manual segmentation protocols and automatic segmentation methods are validated in terms of reliability; for manual segmentation the standard test is to measure the agreement (volume, overlap) between segmentations of the same image by two or more experts (inter-rater reliability), or by the same expert at different times (intra-rater reliability). For automatic segmentation, the standard is to report agreement with manual segmentation. Such reliability measures, while not without value, do not reflect the anatomical accuracy of the segmentation. The resulting uncertainty about the accuracy of in vivo segmentation poses challenges for interpretation of the results of MTL neuroimaging studies, and especially for relating these results to broader neuroscience literature.
Therefore, important goals for the MTL neuroimaging field are (1) to use anatomical accuracy, and not only reliability, for validation of in vivo MRI segmentation; and (2) to develop more anatomically accurate manual segmentation protocols and automatic segmentation techniques. We believe that meeting both goals will require extensive use of ex vivo histological imaging and fusion of information between the in vivo MRI and ex vivo histology domains. This paper describes potential strategies for addressing these goals, particularly in the context of automatic segmentation.
In T1-weighted MRI scans with approximately 1mm × 1mm × 1mm resolution, arguably the most common type of scan in brain morphometry studies, the hippocampus appears largely homogeneous. However, much better contrast between hippocampal layers can be attained using T2-weighted imaging with slices oriented along the main axis of the hippocampus, and high in-plane resolution (e.g., 0.4mm × 0.4mm × 2mm). Interestingly, these T2 scans also reveal much better contrast between meninges and the MTL cortex, as shown in Fig. 1. Virtually all manual segmentation protocols for hippocampal subfields and MTL subregions use these high-resolution T2-weighted MRI scans. However, recent comparison of 20 such protocols  revealed considerable variation in the placement of subfield boundaries, and a broad community effort is currently underway to develop a harmonized protocol that would combine strong reliability with anatomical accuracy (http://hippocampalsubfields.com).
A few techniques for automatic segmentation of MTL substructures from in vivo MRI have been developed. In , , a multi-atlas segmentation algorithm “ASHS” (Fig. 2) is used to propagate subregion labels from a set of 20–30 expert-labeled T2-weighted MRI scans to unlabeled MRI scans. Comparison with manual segmentation yields Dice coefficient of 0.8 and higher for large hippocampal subfields CA1 and DG and 0.75 and higher for subiculum, ERC and PRC. A multi-atlas technique that requires very few expert labeled atlases was separately developed for this problem in , but yielded lower Dice coefficients (0.5–0.6 range) in comparison to manual segmentation. Many studies have also quantified hippocampal subfield using 1mm × 1mm × 1mm, using either shape analysis techniques  or a recent expectation-maximization segmentation technique with a high-resolution ex vivo atlas which is implemented in the FreeSurfer software . However, given the difficulty in performing manual segmentation in T1-weighted MRI, this segmentation approach has not yet been evaluated in terms of reliability relative to manual segmentation. A particular limitation of these automated techniques is that they have not been validated against the anatomical ground truth, but, at best, against manual segmentation with heuristic protocols.
Since in vivo MRI does not offer sufficient features to unequivocally determine MTL subregional boundaries, development of the anatomical ground truth will require histological imaging of ex vivo specimens. Furthermore, it will require datasets in which in vivo MRI can be linked and mapped to the underlying histological imaging.1 This poses two challenges. Firstly, in vivo MRI and ex vivo histology would need to be obtained for the same set of subjects. Such a study can be done prospectively, for example by recruiting patients with terminal disease; or retrospectively, by obtaining specimens from autopsies in patients who had in vivo MRI as part of their clinical care or research study participation during life. Our group has taken the second approach. Over 1000 older adults participating in research studies of dementia, or seen at our memory clinic, have had T2-weighted high-resolution MRI scans of the MTL done in the past five years. Many of these patients and research study participants have consented to autopsy as part of related research studies. As the result, in approximately 30% of autopsies performed at the Penn Center for Neurodegenerative Research, these in vivo T2-weighted MRI are available retrospectively. We have obtained intact MTL specimens in 10 such autopsies in a project that is ongoing.
The second challenge involves finding spatial correspondences between in vivo MRI to histology for each subject. Histology offers virtually unlimited resolution, but it is two-dimensional, has very different contrast characteristics from MRI, and the physical processing of tissue causes shrinking, tearing, and other artifacts that are not present in the MRI. Furthermore, in the interval between when in vivo MRI and histology are acquired, the MTL may undergo changes due to aging, the dying process, brain extraction, and fixation. Directly mapping histology to in vivo MRI would seem to be a daunting challenge. However, high-resolution MRI of the ex vivo specimens prior to histology can offer an intermediate three-dimensional image that shares appearance characteristics with the in vivo MRI and is closer in structure and resolution to the histology. Furthermore, ex vivo MRI not only has the potential to facilitate the mapping of information between in vivo MRI and histology, but also to study the anatomical variability of the MTL at greater detail in 3D.
We have collected ex vivo MRI of the intact MTL in 10 autopsy cases in which subjects had antemortem in vivo MRI within last 3 years of life. Subjects were diagnosed with either frontotemporal dementia or prodromal or probable AD during life. MRI scans with 0.2 × 0.2 × 0.2mm3 resolution were acquired overnight on a Varian 9.4 tesla scanner. Scans were co-registered by manually tracing the outline of the hippocampus in both sets of scans, and applying rigid followed by deformable registration  with large amounts of regularization to the resulting binary images. This resulted in good alignment of hippocampal layers between the in vivo and ex vivo images in all subjects across the whole hippocampus (median distance between the mid-surfaces of the hypointense myelinated hippocampal layers traced manually in the ex vivo and in vivo scans was 0.29 mm after rigid alignment and 0.25 after deformable alignment – less than the in vivo MRI voxel size). Fig. 3 compares in vivo and ex vivo MRI of the same subject side by side.
Ex vivo MRI and histology was performed on 10 MTL specimens (this is a different set of specimens from the one above, and in vivo MRI is available for only 2 of these 10 cases). For each specimen, an MRI scan covering the whole MTL was obtained with 0.2mm × 0.2mm × 0.2mm resolution, after which the MTL was cut into 1cm thick slabs, each slab again scanned at 0.2mm × 0.2mm × 0.2mm resolution, embedded in paraffin, sectioned at approximately 0.2mm intervals, prepared with the Kluver-Barrera stain , and scanned at 0.5μm × 0.5μm resolution.
While previously we employed an automatic registration pipeline  for 3D reconstruction of histology stacks and alignment to the 3D MRI volume, current work relies on manual reconstruction using an interactive tool HistoloZee (www.nitrc.org/projects/historecon). The interactive tool allows concurrent visualization of multiple histology slices and MRI, rotation, translation and scaling of histology slices in-plane, 3D rotation and translation of the MRI volume. While much more labor-intensive, we found manual alignment to lead to better overall reconstruction of histology volumes, particularly for slices with tears, partial coverage of the MTL, and poor staining. An example of reconstructed histology is shown in Fig. 4.
Fig. 5 shows a preliminary result comparing ASHS automatic segmentation of the subfields in the anterior hippocampus on in vivo 3 tesla MRI with ex vivo MRI and histology in the same subject. The segmentations show similarities in general, though several boundaries and microstructural transitions cannot be visualized in vivo or reliably segmented. Effort is underway to obtain serial histology in the specimens for which in vivo MRI is available, align histology to the ex vivo MRI using HistoloZee, and validate current automatic segmentation algorithms using 10–20 images.
Beyond validation against anatomical ground truth, the same-subject in vivo/ex vivo imaging dataset can be used to imbibe the ASHS algorithm with richer anatomical information, which in turn should improve its anatomical accuracy. As the first step, the current set of atlases in ASHS, which consists of 29 MRI scans manually labeled using a heuristic rule-based protocol, should be replaced by the in vivo MRI scans of subjects for whom ex vivo histology is available, with anatomical labels derived from histology and mapped onto the in vivo MRI domain using within-subject deformable registration. However, ex vivo MRI offers additional potential improvements for registration, by providing a high-resolution 3D anatomical reference that can be used as a prior to guide and constrain in vivo segmentation. Toward building such a prior, we have used groupwise deformable registration  to construct an unbiased template of the hippocampus from a set of 25 ex vivo MRI scans (Fig. 6). This template captures the average hippocampal shape and the primary modes of variation in hippocampal shape. This information can in turn be used to constrain ASHS segmentation to stay within the range of feasible topology and shape.
Fig. 7 shows a pilot result illustrating the feasibility of integrating ex vivo data into ASHS. Since histology is not yet available for most of the in vivo/ex vivo dataset, this result was generated using 8 pairs of in vivo and manually labeled ex vivo MRI scans of the same subjects. A target image was first labeled by multi-atlas label fusion , with in vivo MRIs used as atlases and anatomical labels propagated from the ex vivo MRI to in vivo MRI. The ex vivo MRI template in Fig. 6 was then deformed to the multi-atlas segmentation, providing a more detailed and topologically consistent segmentation of the target image. The added advantage of the template is that it allows the mapping of other histologically derived information into the target image domain. As a proof of concept, Fig. 7(v) projects a map of cell body density, extracted from a histology dataset and mapped into the ex vivo MRI template, into the target in vivo MRI image.
Validating automatic and manual segmentation using ground truth derived from histology is the best way to ensure anatomical accuracy of these techniques. Obtaining and aligning histology, ex vivo MRI and in vivo MRI in the same subjects offers a way to perform such validation, and even to integrate histologically-derived anatomical information into the automatic in vivo segmentation algorithm. However, doing so requires a great deal of data collection, annotation, and registration effort.
This work was supported by NIH grants R01 AG037376 and R01 EB017255.
1One important domain in which there are alternatives to ex vivo validation is epilepsy, where the MTL is frequently surgically removed. In , the authors analyzed in vivo MRI and histology data in 15 subjects.