Fundus camera imaging of the retina is widely used to document ophthalmologic disorders including diabetic retinopathy, glaucoma, and age-related macular degeneration. The retinal images typically have a limited field of view due mainly to the curvedness of human retina, so multiple images are to be joined together using image registration technique to form a montage with a larger field of view. A variety of methods for retinal image registration have been proposed, but evaluating such methods objectively is difficult due to the lack of a reference standard for the true alignment of the individual images that make up the montage. A method of generating simulated retinal image set by modeling geometric distortions due to the eye geometry and the image acquisition process is described in this paper. We also present the validation tool for any retinal image registration method by tracing back the distortion path and accessing the geometric misalignment from the coordinate system of reference standard. The quantitative comparison for different registration methods is given in the experiment, so the registration performance is evaluated in an objective manner.
Consistent scan prescription for MRI of the knee is very important for accurate comparison of images in a longitudinal study. However, consistent scan region selection is difficult due the complexity of the knee joint. We propose a novel method for registering knee images using a mutual information registration algorithm to align images in a baseline and follow-up exam. The output of the registration algorithm, three translations and three Euler angles, is then used to redefine the region to be imaged and acquire an identical oblique imaging volume in the follow-up exam as in the baseline. This algorithm is robust to articulation of the knee and anatomical abnormalities due to disease (e.g. osteophytes). The registration method is performed only on the distal femur and is not affected by the proximal tibia or soft tissues. We have incorporated this approach in a clinical MR system and have demonstrated its utility in automatically obtaining consistent scan regions between baseline and follow-up examinations, thus improving the precision of quantitative evaluation of cartilage. Results show an improvement with prospective registration in the coefficient of variation for cartilage thickness, cartilage volume, and T2 relaxation measurements.
Adaptive optics (AO) is a technology used to improve the performance of optical systems by reducing the effects of optical aberrations. The direct visualization of the photoreceptor cells, capillaries and nerve fiber bundles represents the major benefit of adding AO to retinal imaging. Adaptive optics is opening a new frontier for clinical research in ophthalmology, providing new information on the early pathological changes of the retinal microstructures in various retinal diseases. We have reviewed AO technology for retinal imaging, providing information on the core components of an AO retinal camera. The most commonly used wavefront sensing and correcting elements are discussed. Furthermore, we discuss current applications of AO imaging to a population of healthy adults and to the most frequent causes of blindness, including diabetic retinopathy, age-related macular degeneration and glaucoma. We conclude our work with a discussion on future clinical prospects for AO retinal imaging.
adaptive optical systems; optical sensors; biomedical imaging techniques; eye
This study presents methods to 2-D registration of retinal image sequences and 3-D shape inference from fluorescein images. The Y-feature is a robust geometric entity that is largely invariant across modalities as well as across the temporal grey level variations induced by the propagation of the dye in the vessels. We first present a Y-feature extraction method that finds a set of Y-feature candidates using local image gradient information. A gradient-based approach is then used to align an articulated model of the Y-feature to the candidates more accurately while optimizing a cost function. Using mutual information, fitted Y-features are subsequently matched across images, including colors and fluorescein angiographic frames, for registration. To reconstruct the retinal fundus in 3-D, the extracted Y-features are used to estimate the epipolar geometry with a plane-and-parallax approach. The proposed solution provides a robust estimation of the fundamental matrix suitable for plane-like surfaces, such as the retinal fundus. The mutual information criterion is used to accurately estimate the dense disparity map, while the Y-features are used to estimate the bounds of the range space. Our experimental results validate the proposed method on a set of difficult fluorescein image pairs.
3-D reconstruction; Mutual information; Global registration; Retinal fundus image
We propose and compare different registration approaches to align small-animal PET studies and a procedure to validate the results by means of objective registration consistency measurements.
We have applied a registration algorithm based on information theory, using different approaches to mask the reference image. The registration consistency allows for the detection of incorrect registrations. This methodology has been evaluated on a test dataset (FDG-PET rat brain images).
The results show that a multiresolution two-step registration approach based on the use of the whole image at the low resolution step, while masking the brain at the high resolution step, provides the best robustness (87.5% registration success) and highest accuracy (0.67-mm average).
The major advantages of our approach are minimal user interaction and automatic assessment of the registration error, avoiding visual inspection of the results, thus facilitating the accurate, objective, and rapid analysis of large groups of rodent PET images.
Image registration; Positron emission tomography (PET); Validation; Algorithm; Rats
Detection of vascular bifurcations is a challenging task in multimodal retinal image registration. Existing algorithms based on bifurcations usually fail in correctly aligning poor quality retinal image pairs. To solve this problem, we propose a novel highly distinctive local feature descriptor named partial intensity invariant feature descriptor (PIIFD) and describe a robust automatic retinal image registration framework named Harris-PIIFD. PIIFD is invariant to image rotation, partially invariant to image intensity, affine transformation, and viewpoint/perspective change. Our Harris-PIIFD framework consists of four steps. First, corner points are used as control point candidates instead of bifurcations since corner points are sufficient and uniformly distributed across the image domain. Second, PIIFDs are extracted for all corner points, and a bilateral matching technique is applied to identify corresponding PIIFDs matches between image pairs. Third, incorrect matches are removed and inaccurate matches are refined. Finally, an adaptive transformation is used to register the image pairs. PIIFD is so distinctive that it can be correctly identified even in nonvascular areas. When tested on 168 pairs of multimodal retinal images, the Harris-PIIFD far outperforms existing algorithms in terms of robustness, accuracy, and computational efficiency.
Harris detector; local feature; multimodal registration; partial intensity invariance; retinal images
A multi-modality image registration algorithm for the alignment of myocardial perfusion SPECT (MPS) and coronary computed tomography angiography (CTA) scans is presented in this work. Coronary CTA and MPS provides clinically complementary information in the diagnosis of coronary artery disease. An automated registration algorithm is proposed utilizing segmentation results of MPS volumes, where regions of myocardium and blood pools are extracted and used as an anatomical mask. Using a variational framework, we adopt an energy functional with a piecewise constant image model and optimize it numerically with a gradient descent algorithm. The computational efficiency and robustness of the proposed automatic registration of CTA with MPS have been demonstrated by the experiments that yielded an average error smaller than an MPS voxel size.
multi-modality image registration; image registration; myocardial perfusion SPECT; coronary CTA; variational framework
Optical coherence tomography (OCT) has transformed macular disease practices. This report describes the use of conventional OCT systems for peripheral retinal imaging. Thirty-six eyes with peripheral retinal pathology underwent imaging with conventional OCT systems. In vivo sectional imaging of lattice degeneration, snail-track degeneration, and paving-stone degeneration was performed. Differences were noted between phenotypes of lattice degeneration. Several findings previously unreported in histopathology studies were encountered. Certain anatomic features were seen that could conceivably explain clinical and intraoperative behavior of peripheral lesions. Peripheral OCT imaging helped elucidate clinically ambiguous situations such as retinal breaks, subclinical retinal detachment, retinoschisis, choroidal nevus, and metastasis. Limitations of such scanning included end-gaze nystagmus and far peripheral lesions. This first of its kind study demonstrates the feasibility of peripheral retinal OCT imaging and expands the spectrum of indications for which OCT scanning may be clinically useful.
Choroidal nevus; in vivo; lattice degeneration; optical coherence tomography; peripheral retinal lesions; retinoschisis
Purpose. To present a series of retinal disease cases that were imaged by spectral domain optical coherence tomography (SD-OCT) in order to illustrate the potential and limitations of this new imaging modality.
Methods. The series comprised four selected cases (one case each) of age-related macular degeneration (ARMD), diabetic retinopathy (DR), central retinal artery occlusion (CRAO), and branch retinal vein occlusion (BRVO). Patients were imaged using the Heidelberg Spectralis (Heidelberg Engineering, Germany) in SD-OCT mode. Patients also underwent digital fundus photography and clinical assessment.
Results. SD-OCT imaging of a case of age-related macular degeneration revealed a subfoveal choroidal neovascular membrane with detachment of the retinal pigment epithelium (RPE) and neurosensory retina. Using SD-OCT, the cases of DR and BRVO both exhibited macular edema with cystoid spaces visible in the outer retina.
Conclusions. The ability of SD-OCT to clearly and objectively elucidate subtle morphological changes within the retinal layers provides information that can be used to formulate diagnoses with greater confidence.
Image registration is a key step in a great variety of biomedical imaging applications. It provides the ability to geometrically align one dataset with another, and is a prerequisite for all imaging applications that compare datasets across subjects, imaging modalities, or across time. Registration algorithms also enable the pooling and comparison of experimental findings across laboratories, the construction of population-based brain atlases, and the creation of systems to detect group patterns in structural and functional imaging data. We review the major types of registration approaches used in brain imaging today. We focus on their conceptual basis, the underlying mathematics, and their strengths and weaknesses in different contexts. We describe the major goals of registration, including data fusion, quantification of change, automated image segmentation and labeling, shape measurement, and pathology detection. We indicate that registration algorithms have great potential when used in conjunction with a digital brain atlas, which acts as a reference system in which brain images can be compared for statistical analysis. The resulting armory of registration approaches is fundamental to medical image analysis, and in a brain mapping context provides a means to elucidate clinical, demographic, or functional trends in the anatomy or physiology of the brain.
Brain mapping; Image registration; Brain atlas
Image registration is to produce an entire scene by aligning all the acquired image sequences. A registration algorithm is necessary to tolerance as much as possible for intensity and geometric variation among images. However, captured image views of real scene usually produce unexpected distortions. They are generally derived from the optic characteristics of image sensors or caused by the specific scenes and objects.
Methods and Findings
An analytic registration algorithm considering the deformation is proposed for scenic image applications in this study. After extracting important features by the wavelet-based edge correlation method, an analytic registration approach is then proposed to achieve deformable and accurate matching of point sets. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based Levenberg-Marquardt (FLM) method. It converges evidently faster than most other methods because of its feature-based characteristic.
We validate the performance of proposed method by testing with synthetic and real image sequences acquired by a hand-held digital still camera (DSC) and in comparison with an optical flow-based motion technique in terms of the squared sum of intensity differences (SSD) and correlation coefficient (CC). The results indicate that the proposed method is satisfactory in the registration accuracy and quality of DSC images.
Many important eye diseases as well as systemic diseases manifest themselves in the retina. While a number of other anatomical structures contribute to the process of vision, this review focuses on retinal imaging and image analysis. Following a brief overview of the most prevalent causes of blindness in the industrialized world that includes age-related macular degeneration, diabetic retinopathy, and glaucoma, the review is devoted to retinal imaging and image analysis methods and their clinical implications. Methods for 2-D fundus imaging and techniques for 3-D optical coherence tomography (OCT) imaging are reviewed. Special attention is given to quantitative techniques for analysis of fundus photographs with a focus on clinically relevant assessment of retinal vasculature, identification of retinal lesions, assessment of optic nerve head (ONH) shape, building retinal atlases, and to automated methods for population screening for retinal diseases. A separate section is devoted to 3-D analysis of OCT images, describing methods for segmentation and analysis of retinal layers, retinal vasculature, and 2-D/3-D detection of symptomatic exudate-associated derangements, as well as to OCT-based analysis of ONH morphology and shape. Throughout the paper, aspects of image acquisition, image analysis, and clinical relevance are treated together considering their mutually interlinked relationships.
Computer-aided diagnosis; fundus photography; optical coherence tomography (OCT); population screening; retina
AIMS/BACKGROUND: Age-related macular degeneration (ARMD) is a growing public health problem in Britain; currently its aetiology is unclear. The aim of this study was to test the hypothesis that the age specific incidence of blinding ARMD has increased in Britain in the past 50 years, using data on cause of visual loss in people registered as blind, published every 10 years since 1950. METHODS: Data were abstracted from published sources for the years 1950, 1960, 1970, and 1980. Data for the standard year, 1990, were provided in a database from the Office of Population Censuses and Surveys. The numbers of new registrations attributed to ARMD per head of population were compared with registrations for cataract, glaucoma, and optic atrophy. Indirect standardisation was used to control for changes in the age structure of the population over time. RESULTS: After controlling for changes in the age structure of the population, registration rates for all causes, cataract, glaucoma, and optic atrophy have decreased while registrations attributed to ARMD have increased in the order of 30-40%. CONCLUSIONS: These findings are compatible with the hypothesis that the incidence of ARMD is increasing in Britain. It is difficult to exclude potential sources of bias in these data, however, particularly with respect to classification and coding of cause; more reliable population based data on ARMD in Britain are needed.
Retinal lesions caused by eye diseases such as glaucoma and age-related macular degeneration can, over time, eliminate stimulation of parts of the visual cortex. This could lead to degeneration of inactive cortical neuronal tissue, but this has not been established in humans. Here, we used magnetic resonance imaging to assess the effects of prolonged sensory deprivation in human visual cortex. High-resolution anatomical magnetic resonance images were obtained in subjects with foveal (age-related macular degeneration) and peripheral (glaucoma) retinal lesions as well as age-matched controls. Comparison of grey matter between patient and control groups revealed density reductions in the approximate retinal lesion projection zones in visual cortex. This indicates that long-term cortical deprivation, due to retinal lesions acquired later in life, is associated with retinotopic-specific neuronal degeneration of visual cortex. Such degeneration could interfere with therapeutic strategies such as the future application of artificial retinal implants to overcome lesion-induced visual impairment.
macular degeneration; glaucoma; visual field; visual cortex; voxel-based morphometry; grey matter density
Accurate measurement of longitudinal changes of anatomical structure is important and challenging in many clinical studies. Also, for identification of disease-affected regions due to the brain disease, it is extremely necessary to register a population data to the common space simultaneously. In this paper, we propose a new method for simultaneous longitudinal and groupwise registration of a set of longitudinal data acquired from multiple subjects. Our goal is to 1) consistently measure the longitudinal changes from a sequence of longitudinal data acquired from the same subject; and 2) jointly align all image data (acquired from all time points of all subjects) to a hidden common space. To achieve these two goals, we first introduce a set of temporal fiber bundles to explore the spatial-temporal behavior of anatomical changes in each longitudinal data of the same subject. Then, a probabilistic model is built upon the hidden state of spatial smoothness and temporal continuity on the fibers. Finally, the transformation fields that connect each time-point image of each subject to the common space are simultaneously estimated by the expectation maximization (EM) approach, via the maximum a posterior (MAP) estimation of probabilistic models. Promising results are obtained to quantitatively measure the longitudinal changes of hippocampus volume, indicating better performance of our method than the conventional pairwise methods.
This paper proposes an algorithm to register OCT fundus images (OFIs) with color fundus photographs (CFPs). This makes it possible to correlate retinal features across the different imaging modalities. Blood vessel ridges are taken as features for registration. A specially defined distance, incorporating information of normal direction of blood vessel ridge pixels, is designed to calculate the distance between each pair of pixels to be matched in the pair image. Based on this distance a similarity function between the pair image is defined. Brute force search is used for a coarse registration and then an Iterative Closest Point (ICP) algorithm for a more accurate registration. The registration algorithm was tested on a sample set containing images of both normal eyes and eyes with pathologies. Three transformation models (similarity, affine and quadratic models) were tested on all image pairs respectively. The experimental results showed that the registration algorithm worked well. The average root mean square errors for the affine model are 31 µm (normal) and 59 µm (eyes with disease). The proposed algorithm can be easily adapted to registration for other modality retinal images.
(100.4994) Pattern recognition, image transforms; (100.2960) Image analysis; (030.0030) Coherence and statistical optics
In this paper, a novel image registration method is proposed to achieve accurate registration between images having large shape differences with the help of a set of appropriate intermediate templates. We first demonstrate that directionality is a key factor in both pairwise image registration and groupwise registration, which is defined in this paper to describe the influence of the registration direction and paths on the registration performance. In our solution, the intermediate template selection and intermediate template guided registration are two coherent steps with directionality being considered. To take advantage of the directionality, a directed graph is built based on the asymmetric distance defined on all ordered image pairs in the image population, which is fundamentally different from the undirected graph with symmetric distance metrics in all previous methods, and the shortest distance between template and subject on the directed graph is calculated. The allocated directed path can be thus utilized to better guide the registration by successively registering the subject through the intermediate templates one by one on the path towards the template. The proposed directed graph based solution can also be used in groupwise registration. Specifically, by building a minimum spanning arborescence (MSA) on the directed graph, the population center, i.e., a selected template, as well as the directed registration paths from all the rest of images to the population center, is determined simultaneously. The performance of directed graph based registration algorithm is demonstrated by the spatial normalization on both synthetic dataset and real brain MR images. It is shown that our method can achieve more accurate registration results than both the undirected graph based solution and the direct pairwise registration.
Image registration; directionality; directed graph; minimum spanning arborescence (MSA); intermediate templates; groupwise registration
Drusen are common features in the ageing macula associated with exudative Age-Related Macular Degeneration (ARMD). They are visible in retinal images and their quantitative analysis is important in the follow up of the ARMD. However, their evaluation is fastidious and difficult to reproduce when performed manually.
This article proposes a methodology for Automatic Drusen Deposits Detection and quantification in Retinal Images (AD3RI) by using digital image processing techniques. It includes an image pre-processing method to correct the uneven illumination and to normalize the intensity contrast with smoothing splines. The drusen detection uses a gradient based segmentation algorithm that isolates drusen and provides basic drusen characterization to the modelling stage. The detected drusen are then fitted by Modified Gaussian functions, producing a model of the image that is used to evaluate the affected area.
Twenty two images were graded by eight experts, with the aid of a custom made software and compared with AD3RI. This comparison was based both on the total area and on the pixel-to-pixel analysis. The coefficient of variation, the intraclass correlation coefficient, the sensitivity, the specificity and the kappa coefficient were calculated.
The ground truth used in this study was the experts' average grading. In order to evaluate the proposed methodology three indicators were defined: AD3RI compared to the ground truth (A2G); each expert compared to the other experts (E2E) and a standard Global Threshold method compared to the ground truth (T2G).
The results obtained for the three indicators, A2G, E2E and T2G, were: coefficient of variation 28.8 %, 22.5 % and 41.1 %, intraclass correlation coefficient 0.92, 0.88 and 0.67, sensitivity 0.68, 0.67 and 0.74, specificity 0.96, 0.97 and 0.94, and kappa coefficient 0.58, 0.60 and 0.49, respectively.
The gradings produced by AD3RI obtained an agreement with the ground truth similar to the experts (with a higher reproducibility) and significantly better than the Threshold Method. Despite the higher sensitivity of the Threshold method, explained by its over segmentation bias, it has lower specificity and lower kappa coefficient. Therefore, it can be concluded that AD3RI accurately quantifies drusen, using a reproducible method with benefits for ARMD evaluation and follow-up.
Statistical models of deformations (SMD) capture the variability of deformations from the template image onto a group of sample images and can be used to constrain the traditional deformable registration algorithms to improve their robustness and accuracy. This paper employs a wavelet-PCA-based SMD to constrain the traditional deformable registration based on the Bayesian framework. The template image is adaptively warped by an intermediate deformation field generated based on the SMD during the registration procedure, and the traditional deformable registration is performed to register the intermediate template image with the input subject image. Since the intermediate template image is much more similar to the subject image, and the deformation is relatively small and local, it is less likely to be stuck into undesired local minimum using the same deformable registration in this framework. Experiments show that the proposed statistically-constrained deformable registration framework is more robust and accurate than the conventional registration.
biomedical image processing; magnetic resonance imaging; image registration; statistical model
An upgraded optical coherence tomography system with integrated retinal tracker (TOCT) was developed. The upgraded system uses improved components to extend the tracking bandwidth, fully integrates the tracking hardware into the optical head of the clinical OCT system, and operates from a single software platform. The system was able to achieve transverse scan registration with sub-pixel accuracy (~10 μm). We demonstrate several advanced scan sequences with the TOCT, including composite scans averaged (co-added) from multiple B-scans taken consecutively and several hours apart, en face images collected by summing the A-scans of circular, line, and raster scans, and three-dimensional (3D) retinal maps of the fovea and optic disc. The new system achieves highly accurate OCT scan registration yielding composite images with significantly improved spatial resolution, increased signal-to-noise ratio, and reduced speckle while maintaining well-defined boundaries and sharp fine structure compared to single scans. Precise re-registration of multiple scans over separate imaging sessions demonstrates TOCT utility for longitudinal studies. En face images and 3D data cubes generated from these data reveal high fidelity image registration with tracking, despite scan durations of more than one minute.
The impact of retinal pathology detected by high-resolution imaging on vision remains largely unexplored. Therefore, the aim of the study was to achieve high-resolution structure-function correlation of the human macula in vivo.
To obtain high-resolution tomographic and topographic images of the macula spectral-domain optical coherence tomography (SD-OCT) and confocal scanning laser ophthalmoscopy (cSLO), respectively, were used. Functional mapping of the macula was obtained by using fundus-controlled microperimetry. Custom software allowed for co-registration of the fundus mapped microperimetry coordinates with both SD-OCT and cSLO datasets. The method was applied in a cross-sectional observational study of retinal diseases and in a clinical trial investigating the effectiveness of intravitreal ranibizumab in macular telangietasia type 2. There was a significant relationship between outer retinal thickness and retinal sensitivity (p<0.001) and neurodegeneration leaving less than about 50 µm of parafoveal outer retinal thickness completely abolished light sensitivity. In contrast, functional preservation was found if neurodegeneration spared the photoreceptors, but caused quite extensive disruption of the inner retina. Longitudinal data revealed that small lesions affecting the photoreceptor layer typically precede functional detection but later cause severe loss of light sensitivity. Ranibizumab was shown to be ineffective to prevent such functional loss in macular telangietasia type 2.
Since there is a general need for efficient monitoring of the effectiveness of therapy in neurodegenerative diseases of the retina and since SD-OCT imaging is becoming more widely available, surrogate endpoints derived from such structure-function correlation may become highly relevant in future clinical trials.
A biomechanical model-based deformable image registration incorporating specimen-specific changes in material properties is optimized and evaluated for correlating histology of clinical prostatectomy specimens with in vivo MRI. In this methodology, a three-step registration based on biomechanics calculates the transformations between histology and fixed, fixed and fresh, and fresh and in vivo states. A heterogeneous linear elastic material model is constructed based on magnetic resonance elastography (MRE) results. The ex vivo tissue MRE data provide specimen-specific information for the fresh and fixed tissue to account for the changes due to fixation. The accuracy of the algorithm was quantified by calculating the target registration error (TRE) by identifying naturally occurring anatomical points within the prostate in each image. TRE were improved with the deformable registration algorithm compared to rigid registration alone. The qualitative assessment also showed a good alignment between histology and MRI after the proposed deformable registration.
Biomechanical models; correlative pathology; deformable registration; finite element model; magnetic resonance elastography
Motivated by the observation that a retinal fundus image may contain some unique geometric structures within
its vascular trees which can be utilized for feature matching, in this paper, we proposed a graph-based registration
framework called GM-ICP to align pairwise retinal images. First, the retinal vessels are automatically detected and
represented as vascular structure graphs. A graph matching is then performed to find global correspondences between
vascular bifurcations. Finally, a revised ICP algorithm incorporating with quadratic transformation model is used at
fine level to register vessel shape models. In order to eliminate the incorrect matches from global correspondence
set obtained via graph matching, we proposed a structure-based sample consensus (STRUCT-SAC) algorithm. The
advantages of our approach are threefold: (1) global optimum solution can be achieved with graph matching; (2)
our method is invariant to linear geometric transformations; and (3) heavy local feature descriptors are not required.
The effectiveness of our method is demonstrated by the experiments with 48 pairs retinal images collected from
We propose a novel Bayesian registration formulation in which image location is represented as a latent random variable. Location is marginalized to determine the maximum a priori (MAP) transform between images, which results in registration that is more robust than the alternatives of omitting locality (i.e. global registration) or jointly maximizing locality and transform (i.e. iconic registration). A mathematical link is established between the Bayesian registration formulation and the mutual information (MI) similarity measure. This leads to a novel technique for selecting informative image regions for registration, based on the MI of image intensity and spatial location. Experimental results demonstrate the effectiveness of the marginalization formulation and the MI-based region selection technique for ultrasound (US) to magnetic resonance (MR) registration in an image-guided neurosurgical application.
Accurate 3D/2D vessel registration is complicated by issues of image quality, occlusion, and other problems. This study performs a quantitative comparison of 3D/2D vessel registration in which vessels segmented from preoperative CT or MR are registered with biplane x-ray angiograms by either a) simultaneous two-view registration with advance calculation of the relative pose of the two views, or b) sequential registration with each view. We conclude on the basis of phantom studies that, even in the absence of image errors, simultaneous two-view registration is more accurate than sequential registration. In more complex settings, including clinical conditions, the relative accuracy of simultaneous two-view registration is even greater.