|Home | About | Journals | Submit | Contact Us | Français|
For eye diseases, such as glaucoma and age-related macular degeneration (ARMD), involved in long-term degeneration procedure, longitudinal comparison of retinal images is a common step for reliable diagnosis of these kinds of diseases.
To provide a retinal image registration approach for longitudinal retinal image alignment and comparison.
Two image registration solutions were proposed for facing different image qualities of retinal images to make the registration methods more robust and feasible in a clinical application system.
Thirty pairs of longitudinal retinal images were used for the registration test. The experiments showed both solutions provided good performance for the accurate image registrations with efficiency.
We proposed a set of retinal image registration solutions for longitudinal retinal image observation and comparison targeting a clinical application environment.
Benefiting from modern medical imaging techniques, retinal images are able to provide large amounts of valuable information, especially related to eye disease diagnosis, such as glaucoma and ARMD. Both diseases usually go through a long-term degeneration process. Therefore, a longitudinal comparison of retinal images combined with a patient’s medical history is a common step for a reliable diagnosis of these kinds of diseases by ophthalmologists. Image alignment or registration is a key step for aligning two images at the same position for the longitudinal comparison qualitatively or quantitatively.1 In the past decade, a wide range of retinal image registration methods have been published focusing on multi-modal image registration2-4 and single-modal image registration.5 In this paper, we focus on colour fundus image registration. In this area, most papers proposed using geometrical features, as blood vessel bifurcations and crossover points and local features, in the registration methods.6-11Some papers also used region information for the registrations.12 Similarity, affine, quadratic and radial distortion correction models (RADIC) were used as transform models in the registration methods.13 One important application area of the registration methods is multiple visual field image registration7,14, the other longitudinal image registration.1
In this paper, we propose two solutions of retinal image registration for facing a clinical application environment. Solution 1 proposes a novel blood vessel enhancement method for blood vessel based retinal image registration. Solution 2 adopts a region-based registration method while blood vessel features are not fully available in visual fields for some fundus cameras. The solutions aim at providing longitudinal retinal image alignment methods for image comparison in our clinical decision support system.
In the image processing area, aligning two images captured or scanned from different time points is usually an image registration issue. In this paper, two image registration solutions are proposed for facing different image qualities of retinal images (as in Figure 1, the image in the left panel showing good quality with full blood vessels; the image in the right panel showing good quality in its optic disk region but lack of blood vessel information in its peripheral region) in order to make the registration methods more robust and feasible in our system. Solution 1 is a blood vessel based registration method, solution 2 optic-disk-region intensity based registration method. Both methods use green-channel images extracted from their original colour fundus images (around 615x615 pixels in the visual field region) as candidates in the following registration processes.
Solution 1: Retinal blood vessel based image registration
Retinal blood vessels can cover the majority of a retinal image and generally retain a static retinal location even in a diseased eye. This feature makes retinal blood vessels ideal for retinal image registration. The blood vessel based image registration method requires that the blood vessels can be extracted from the retinal image correctly and used as a feature structure in the registration procedure. Therefore, good image quality is necessary.
Prior to the registration, a novel blood vessel enhancement step is proposed for the image pre-processing. A local entropy thresholding method is applied for extracting blood vessels from the processed image.
Black Top-hat method is firstly used for the initial enhancement of the blood vessels in the green-channel image. The morphological method computes the difference between the closing of the image and itself to enhance the objects (blood vessels) whose diameters are less than that of the structuring element used in the morphological method. A Gaussian matched filtering method is then applied for further enhancing the blood vessels in the output of the Black Top-hat operation.According to Chaudhun et al15, the intensity profile of the cross-section of blood vessels can be approximated by a Gaussian curve. A 2-Dimensional Gaussian matched filter is applied as:
where L is the common length of the segment of a piece blood vessel. Twelve directional Gaussian kernels are designed (15 degree interval between the neighbouring kernels) to process the image for matching the blood vessel segments with arbitrary orientations. The final enhanced image is constructed by assigning each pixel with the maximum grey value from the 12 filtered images at the same position.
A local entropy thresholding method7 is applied for the blood vessel extraction from the above enhanced retinal image. If the blood vessels are considered as the object to be detected, the local entropy thresholding method is to maximise a second-order local entropy of the object and the background, for computing an optimal threshold, as:
are the second-order object entropy and second-order background entropy and P is normalised co-occurrence probability of pixel intensities. A binary operation can be easily implemented on the enhanced image by the computed threshold.
An image registration method is developed to apply a translation registration firstly for approaching an initial alignment of the two blood vessel images then applying an affine registration further for a more accurate image registration. Considering the candidates are binary images with blood vessels (foreground) and their background, in the translation registration procedure, One-Plus-One Evolutionary strategy is chosen as the optimiser, and the similarity of matched cardinality as the metric. In the affine registration procedure, a step gradient descent method is chosen as its optimiser and a mean squares difference is used as the metric. In the registration process, nearest-neighbour interpolation method is adopted because of the binary images.
Solution 2: Optic-disk-region intensity based registration
Solution 2 is a complementary to solution 1, in the situation that the blood vessels cannot be extracted fully in the retinal visual field for some kind of images with low image quality or illumination in the peripheral region (right image in Figure 1).
The solution can be briefly described as two steps: (1) searching the optic-disk region in each image and defining a rectangular region, including the optic disk and its surrounding region, as registration candidate; (2) applying image intensity based translation and affine registration methods on the detected optic-disk-region candidates from two images for their image registration.
The difference of the registration process from that in solution 1 is that the mutual information computed from the two optic-disk-region candidates is used as the metric for both translation and affine registrations. A linear interpolation method is adopted during the registration because of the grey value images.
The obtained transformation matrices from the above registration methods are then applied to warp the original colour image or green-channel image for aligning two images for comparison and diagnostic purpose.
The proposed solutions were implemented in C++ language. 15 pairs of longitudinal retinal images (7 pairs – time interval was around 1 year) were used for the blood vessel based registration test. Another 15 pairs of longitudinal retinal images with lower blood vessel information were used for the intensity based registration test.
In the experiment, for per registration pair, it took about 58s to run solution 1 and about 19s to run solution 2. This result shows the efficiency of the methods and the running time of the registrations is acceptable in a clinical application.
Figure 2 illustrates a successfully extracted binary blood vessel image by applying the blood vessel enhancement and extraction method. The good performance of the method has been shown in the extracted blood vessels (Figure 2c). For quantitatively validating the accuracy of the proposed registration methods, corresponding feature points from a pair of registered images were used to compute registration errors. First, the blood vessel medial line images were generated from their registered blood vessel images. Feature points (number of points > 10; located at bifurcations and crossover points of the blood vessels; covering the most area of the image) were manually labelled from each pair of registered blood vessel medial line images (Figure 3 showing feature points on one blood vessel medial line image). The corresponding feature points from a pair of registered medial line images were used to compute the corresponding point distances (pixels), which were used as registration errors. The similar evaluation method was applied for the solution 2. However, the corresponding points were manually picked up only on the blood vessel bifurcations and crossover points from the green channel images (Figure 4) rather from blood vessel medial line images.
For each pair of registered images, mean and standard deviation of the distance errors from the selected corresponding feature points were computed. The statistical result of the registration errors from 15-pair registered images by solution 1 is shown in Figure 5. Among the mean errors of the 15 registration pairs, the maximum value is 4 pixels, the minimum 1.2 pixels. Figure 6 shows the statistical result of the registered errors from 15-pair images by solution 2. The maximum mean error is 2.7 pixels, the minimum 1.1 pixels. The experiment has shown that both solutions provided good performance for the accurate image registration.
Figure 7 shows two images before registration (Figure 7a) and the results after applying registration warping, overlapping area clipping and image colour matching (Figure 7b). The registered images can be used further for qualitative comparison by “flickering” display, which is commonly adopted in a clinical environment, or quantitative comparison by image subtraction, a potential application area in clinic. Figure 8 shows the integration of the solutions into our Western Australia Pilbara project “Remote-I”. The web-based application could display two registered images alternately (lower right pane of Figure 8) with a time interval automatically or manually by operating a slide control. An observer could easily find and compare the difference of two longitudinal images based on the image registration technique.
The registration methods proposed in the paper are aimed at a clinical application environment. Compared to the feature-based registration methods3,4 feature-based registration methods, which are aimed at registering small-overlapping images from multiple visual fields by detected features or landmarks (bifurcation or crossover points in blood vessels), our methods provided a blood vessel enhancement method and directly used blood vessels (solution 1) and image intensity (solution 2) for the registration. Our methods saved time for feature detection, thereby providing a more efficient process with acceptable registration accuracy in clinical application environments. The recently published paper by Tsai et al used feature-based registration by DB-ICP algorithm but targeted multimodal image registration application. Whether our methods can be used in this area will be evaluated in future but this was not this paper’s objectives.
For the differences in error metrics and specific characteristics of registration methods, it is difficult to compare their performance. An objective expert-independent method was proposed for validating similarity, affine, quadratic and RADIC models in the retinal image registration processes13Because our registration solutions target longitudinal image registration and comparison with large overlapping area, we use the point displacement error measurement (DEM) in the local area (overlapping area) from the results of the method13 as a reference for registration accuracy. In Figure 13 in the objective validation method13 method, If considering the overlapping percentage from 40% to 60% as large overlapping areas, we can find the ranges of the mean errors for similarity, affine, quadratic and RADIC models are: 2.2 to 3 pixels, 2 to 2.8 pixels, 3 to 3.6 pixels and 1 to 1.6 pixels, respectively. Compared to the data, the statistical results from our registration methods (15 registration pairs respectively) show the mean registration errors are 2.5 pixels from solution 1 and 1.5 pixels from solution 2. It has shown the performance of our methods is acceptable and our methods have applicable value in our clinical decision system.
Solution 1 provided a full visual field registration between two images; therefore image comparison could be applied on the overlapping areas between them. Solution 2 targeted a situation of retinal image registration with low image quality and image illumination, which happened by using some non-mydriatic cameras in our clinical application but was less discussed in previous retinal image registration methods. Solution 2 provided an alternative and efficient registration method and can be more suitable for the comparison of optic disk areas from two registered images. The colour-matching method applied after the registration provided a more intuitive comparison of two registered images for ophthalmologists’ review (referring to the difference between Figure 7a and Figure 7b).
We proposed two retinal image registration solutions for longitudinal retinal image observation and comparison. Solution 1 used a novel blood vessel enhance method for feature-based registration. Solution 2 used optic-disc region for region based registration. Two groups of image pairs (each group with 15 image pairs) were chosen for testing the performance of the two solutions. The experimental results have demonstrated the accuracy and robustness of the registration methods with the efficiency, which means that the proposed methods have potential value for clinical application. We have integrated the methods into our “Remote-I” clinical decision support system for further clinical evaluation.
Not commissioned. Externally peer reviewed.
CONFLICTS OF INTEREST
The authors have no competing interests.
This research was funded by Pilbara Development Commission, Government of Western Australia.
WACHS Research Ethics Committee
University of Western Australia Human Research Ethics
Please cite this paper as: Xiao D, Vignarajan J, Lock J, Frost S, Tay-Kearney M, Kanagasingam Y. Retinal image registration and comparison for clinical decision support. AMJ 2012, 5, 9, 507-5012. http//dx.doi.org/10.4066/AMJ.2012.1364