Multispectral retinal image analysis: a novel non-invasive tool for retinal imaging

doi:10.1038/eye.2011.202

Eye (Lond). 2011 December; 25(12): 1562–1569.

Published online 2011 September 9. doi: 10.1038/eye.2011.202

PMCID: PMC3234460

Multispectral retinal image analysis: a novel non-invasive tool for retinal imaging

A Calcagni,^1,²^* J M Gibson,^2,³ I B Styles,¹ E Claridge,¹ and F Orihuela-Espina¹

¹University of Birmingham, School of Computer Science, Birmingham, West Midlands, UK

²Birmingham and Midland Eye Centre, Sandwell and West Birmingham Hospitals NHS Trust, City Hospital, Birmingham, West Midlands, UK

³Aston University, School of Life and Health Sciences, Birmingham, West Midlands, UK

^*Birmingham and Midland Eye Centre, Dudley Road, Birmingham, B18 7QU, UK. Tel: +44(0)121 554 3801; Fax: +44(0)121 554 6853. E-mail: a.s.calcagni/at/cs.bham.ac.uk

Received March 7, 2011; Accepted June 27, 2011.

Abstract

Purpose

To develop a non-invasive method for quantification of blood and pigment distributions across the posterior pole of the fundus from multispectral images using a computer-generated reflectance model of the fundus.

Methods

A computer model was developed to simulate light interaction with the fundus at different wavelengths. The distribution of macular pigment (MP) and retinal haemoglobins in the fundus was obtained by comparing the model predictions with multispectral image data at each pixel. Fundus images were acquired from 16 healthy subjects from various ethnic backgrounds and parametric maps showing the distribution of MP and of retinal haemoglobins throughout the posterior pole were computed.

Results

The relative distributions of MP and retinal haemoglobins in the subjects were successfully derived from multispectral images acquired at wavelengths 507, 525, 552, 585, 596, and 611

nm, providing certain conditions were met and eye movement between exposures was minimal. Recovery of other fundus pigments was not feasible and further development of the imaging technique and refinement of the software are necessary to understand the full potential of multispectral retinal image analysis.

Conclusion

The distributions of MP and retinal haemoglobins obtained in this preliminary investigation are in good agreement with published data on normal subjects. The ongoing development of the imaging system should allow for absolute parameter values to be computed. A further study will investigate subjects with known pathologies to determine the effectiveness of the method as a screening and diagnostic tool.

Keywords: retina, imaging, macular pigment, theoretical models, Monte Carlo simulation

Introduction

The colours observed in images of the ocular fundus depend on the reflection of light from the different parts of the fundus and this in turn depends on the distribution, quantities, and characteristic optical absorption properties of its pigments. Theoretically, it should therefore be simple to identify and quantify one pigment from another, but in practice this is often impossible because of the limited sensitivity of both the human eye and currently available imaging systems, and the masking effect that one pigment may have on another.

There are five main pigments in the fundus: macular pigment (MP), retinal blood, retinal pigment epithelium (RPE) melanin, choroidal blood, and choroidal melanin.

The investigators report the preliminary results of a promising, novel technique, multispectral retinal image analysis (MRIA), which is capable of objectively producing maps showing the distribution and measured quantities of MP and other retinal pigments across the posterior pole, by combining multispectral imaging with sophisticated computer analysis. It uses a Monte Carlo computer simulation of the passage of light through the fundus tissues, in order to establish a link between the tissue composition and fundus reflectance at different wavelengths. This is based on the knowledge that when light strikes a tissue, it can be reflected, absorbed, scattered, transmitted, or remitted depending on the wavelength of incident light and tissue histology, and image formation is determined by the amount of each wavelength reflected.

Materials and methods

Using detailed knowledge of the structure and histology of the human fundus and the optical properties (reflectance, absorbance, transmission, scatter, and remission) of its single constituents, a computer-based Monte Carlo simulation was used to construct a model of fundus reflectance (forward model); the model was successively employed to select the most appropriate wavelengths for minimising error in deducing histological information; these wavelengths were then adopted to acquire images from healthy and pathological fundi, in an attempt to extract quantitative histological information (inverse model). Fundus images were obtained using a modified fundus camera (Zeiss RCM250, Carl Zeiss, Jena, Germany).

A comprehensive description of the principles and techniques underpinning this work and the modifications to the fundus camera have been previously reported in detail elsewhere.¹

In summary, the parameter values can be recovered according to the following procedure:

Images are acquired through the selected filters (507, 525, 552, 585, 596, and 611nm).
The images are spatially registered to correct for involuntary movements during the acquisition process and corrected for exposure, gain, and offset.
Five illumination-normalised images (image ratios) are generated by dividing each of the first five images by the image acquired at 611nm; hence for each point of the posterior pole, five normalised image values are extracted. Image ratios are used to compensate for the uneven illumination of the fundus, secondary to the spherical nature of the globe.
The concentrations associated with the five normalised image values for each point of the image are calculated using the inverse model.
Five new images, one for each of the parameters, are generated (‘parametric maps').

To recover the five parameters at each point of the posterior pole, images acquired from all subjects need to be compared with the reflectance model.

The reflectance model can be thought of as a database where for each combination of the five histological parameters there is one, and only one, record that stores the five illumination-normalised image values, and every possible combination of concentrations of the five parameters is represented within the database.

Assuming the model is correct, for each set of values obtained from a set of fundus images there should be a closely corresponding record in the database.

The whole procedure will therefore generate five parametric maps, representing respectively the concentration of MP, retinal haemoglobins, RPE melanin, choroidal haemoglobins, and choroidal melanin at each point of the image.

Images were acquired from 16 healthy volunteers (10 males, 6 females, mean age=35.9; age range 26–64); a set of images was also acquired from one subject with retinal haemorrhages secondary to choroidal neovascular membrane, to validate MRIA as a tool capable of flagging localised increases in retinal haemoglobins.

In all healthy participants, best-corrected visual acuity was 0.0 or better using the logarithm of the minimum angle of resolution (LogMAR) acuity chart and there was no clinical evidence of ocular disease.

Following a full ocular examination by an ophthalmologist before and after pupil dilatation with Tropicamide 1% and Phenylephrine 2.5% eye drops, the subject's head was stabilised using the chin rest and forehead support of the fundus camera. The subject was asked to look at a fixation light while the operator adjusted the image alignment and focus. The acquisition of the set of six multispectral images required approximately 5

s during which the subject was asked to maintain fixation.

The investigators certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during this research and the study protocol adhered to the tenets of the Declaration of Helsinki.

Results

Computer simulation analysis: effect of the fundus composition on the remitted spectra

Figure 1 shows a typical output of the reflectance model generated by the Monte Carlo simulation: in each of the graphs, four of the five parameters are constant, and the fifth parameter is varied. Similar graphs were plotted for each possible combination of the five histological parameters being assessed.

Figure 1

Calculated variation of fundus reflectance in MP (a), RPE melanin (b), retinal haemoglobin (c), choroidal melanin (d), and choroidal haemoglobins (e), maintaining other four parameters constant.

As the concentration of MP is increased (Figure 1a), the reflectance is attenuated in the blue and green spectral regions, with no effect in the red, since MP does not absorb at wavelengths longer than 534

nm.

It is interesting to note that although MP peak absorbance is at 460

nm, the software algorithm used to select the most appropriate six wavelengths to recover the five parameters did not include 460

nm; multispectral retinal image analysis assesses the fundus pigments together and the selection of filters is based on sophisticated computer analysis aimed at minimising error in the recovery of the various pigments and avoiding use of dangerous light levels; when the inverse model to be applied was calculated, the software established 460

nm would not be an appropriate wavelength and 507, 525, 552, 585, 596, and 611

nm were found to be best suited to achieve these fundamental goals.

It is also intriguing that variations of melanin and haemoglobins have a different effect on the remitted light according to whether the pigments are located in the retina (Figures 1b and c) or choroid (Figures 1d and e) and this is believed to be due to the different environments the pigments find themselves and the properties of the surrounding tissues, in particular scatter, which will obviously differ according to whether the pigment lies in the retina, RPE or choroid.

Image analysis

The inverse model was applied to image data from each set of images in order to compute maps showing the spatial variation and magnitudes of the parameter values. The initial results suggested a significant mismatch between the model data and the image data. This was corrected by applying an empirical correction factor, as described in Styles et al.¹ The consequence of this correction factor is that the absolute values of the recovered parameters are incorrect; the relative values, however, maintain the correct relationship; that is, when the concentration of one of the parameters at a given retinal point A is greater than the concentration at a second given point B, the parameter shown in the computer-generated map at point A will be greater than that at point B.

Potential causes of the model–data mismatch are identified in the discussion section.

Only the parametric maps showing the distribution of MP and retinal haemoglobins were analysed; recovery of the other parameters with sufficient accuracy proved impossible with the equipment available, because of the highly complex relationship between the image ratios and the parameters and the need of excessive exposure times, with consequent introduction of significant movement artefacts, which could not be compensated for after image acquisition; the investigators therefore modified the computer software, so that only MP and haemoglobin could be recovered, while choroidal haemoglobin, and RPE and choroidal melanin were included in the model as variables but quantification was not attempted; this decision was based on the fact that MP and retinal blood have a characteristic distribution in fundi of healthy individuals and could therefore be used to validate MRIA as a tool for quantifying retinal pigments.

Furthermore, recovery of MP and haemoglobin distribution was not possible in all subjects and this is most likely due to problems inherent within the imaging methods; these problems are tackled in detail in the discussion section, but, in brief, the relatively prolonged exposure used to take the images implies that small saccadic movements are unavoidable; while the movement artefact can be overcome with sophisticated registration techniques, constancy of fundus illumination, essential for image normalisation, is inevitably compromised.

When MP and retinal haemoglobin concentrations were computable, the accuracy was in the order of ±0.1

mmol/l.

Analysis of MP distribution

MP, in view of its well-established distribution in non-pathological eyes, is well suited for validating the method described in this paper. Seven healthy subjects in all showed in the parametric map of MP the expected distribution; Figure 2 shows the RGB photographs (first column from left), the computer-generated parametric maps (second column), two-dimensional (third column), and three-dimensional (fourth column) representations of the MP distribution of these eyes; there is an obvious peak of MP at the fovea with a rapid decline outside the foveal area; this is consistent with known histology of the fundus.^{2, 3, 4} The last two sets of images are from the same subject at different times, showing the repeatability of the method.

Figure 2

MP recovery (blue indicates less, red indicates more).

Analysis of retinal haemoglobin distribution

Figures 3a–d show the RGB photographs (first column from left), the computer-generated parametric maps (second column), two-dimensional (third column), and three-dimensional (fourth column) representations of three normal fundi (Figures 3a–c) and of one with retinal haemorrhages (Figure 3d), used to validate MRIA as an effective tool in flagging areas of localised variation in retinal haemoglobins. The normal subjects show a clear depression in correspondence with the foveal avascular zone, as expected. In Figure 3d, the computer-generated parametric maps and the 2D and 3D images highlight a clear increase in blood in the same areas the RGB image shows haemorrhages.

Figure 3

Haemoglobin recovery (blue indicates less, red indicates more; (a–c) normal fundi; (d) fundus with haemorrhages).

There is frequently a false positive at the optic disc, the different histology of this area implies that it is not represented in the model and hence falls outside its bounds and cannot be interpreted correctly. Furthermore, artefacts because of reflection from the inner limiting membrane can be noticed in some subjects adjacent to the macular area; this problem can be solved using polarised light.

Discussion

The ability to quantify and objectively assess the distribution of the ocular fundus pigments could aid in the assessment and monitoring of pathological conditions of the retina and choroid.

One of the potential roles for the technique described is to provide a reproducible, simple, and objective measurement of MP. The MP confers the characteristic yellow tint around the foveal area in the primate retina; it has a sharp peak at the centre of the fovea and declines exponentially with increasing eccentricity, reaching negligible levels at 4–6° of the visual angle.^{5, 2, 3} Measurement of MP in vivo is possible and has been advocated as a means of assessing the risk of development of age-related macular degeneration in later years.⁶ MP is mainly of dietary origin⁴ and is thought to have a protective role by acting as an optical filter to shorter wavelengths and as a consequence of its antioxidant properties.^{7, 8, 9, 10, 11} Several techniques have been adopted to measure MP in vivo, including Raman detection,¹² scanning laser ophthalmoscope,¹³ reflectometry,^{14, 15, 16, 17} autofluorescence,¹⁴ colour matching,¹⁸ and heterochromatic flicker photometry using ‘Maxwellian' and ‘freeview' systems,^{19, 20} but there are limitations to all these techniques; Raman spectroscopy, scanning laser ophthalmoscopy, reflectometry, and autofluorescence all require complex equipment and have been developed as research tools. The more widespread techniques used in clinical practice to measure MP rely on colour matching or heterochromatic flicker photometry, both of which are dependent on subjective interpretation of an end point by the individual performing the test and this may be difficult in elderly subjects, especially if fixation is poor; there is therefore a need for a simple objective technique that could be applied in a clinical setting.^{21, 22}

Another potential role of the technique described is the assessment of retinal perfusion; detecting variations in the distribution of retinal blood is an essential part of the fundus examination, especially when pathology is concerned. Sometimes it can be evaluated only by fundus fluorescein angiography, an invasive technique that is potentially life-threatening (anaphylactic shock) and an alternative non-invasive method would therefore be of great benefit.

Other groups have used multispectral imaging for the quantification of retinal haemoglobins (for example, see Mordant et al²³), but MRIA offers a completely different approach in that it interprets the images by means of a comparison of the images with a computer-generated reflectance model of the fundus and it reconstructs ‘parametric maps' from the data available, potentially providing the clinician with pixel-by-pixel concentration values.

The results from this study suggest that the described method can potentially be used as a non-invasive tool in quantifying MP, retinal haemoglobins and possibly other retinal and choroidal parameters in normal subjects.

While the results are encouraging, further development is required in several areas.

First, not all model parameters can be recovered equally well at present; this is thought to be due to the high level of non-linearity of the method used in generating the model and inverse model,^{1, 24, 25} hence more exhaustive mathematical analysis is needed and more effective methods for parameter recovery are being investigated.

Second, one of the main drawbacks of the method is the image acquisition system used; the exposure time was too long and image movement artefacts were inevitably introduced; some of these artefacts cannot be compensated for by simple registering of the six images, as constancy of fundus illumination, essential for image normalisation, is inevitably compromised (for further details, see Styles et al¹). Modifications to the equipment have been designed and completed^{26, 27} so that faster image acquisition is possible and these artefacts minimised and this adapted equipment is currently being tested; the investigators are currently using the improved image acquisition system to validate MRIA for MP quantification against the current clinical gold standard, heterochromatic flicker photometry, and also to prove MRIA has an acceptable test–retest variability.

Third, yellowing of the human lens with ageing is a major problem that all fundus imaging techniques need to overcome and this has particular significance in the measurement of MP, which also absorbs in the yellow part of the spectrum. The model accounts for attenuation of light transmission by the lens as a function of age,^{28, 29} but subjects with significant cataract, that is, compromised view of the posterior pole on slit-lamp biomicroscopy, cannot be assessed. The investigators are currently investigating a solution to this problem.

Finally, the need for empirical scaling to ensure that model and image data were coincident is probably the most concerning factor; although improvement in the imaging system used is likely to solve this issue in part, other potential problems with the model itself have been identified, and these require careful consideration in further studies.

The model assumes that the individual layers in the fundus are of constant thickness and are the same in all subjects. This assumption is not correct and variations in the layer thicknesses may well lead to significant changes in the model.
Although the major optical properties of the ocular tissues were taken into account, not all the possible variations within them were considered.

While the model appears to capture the correct trends, it does not include the full variation of normal eyes. This is a consequence of two factors: first, knowledge of in vivo ocular tissue is not exhaustive, especially with regards to the scattering properties of the various layers of the fundus; second, an introduction of too many parameters in the computer simulation would require many more images to be taken and would significantly increase computation times and margin of error. Further investigations, however, are necessary in order to determine the precise cause of the discrepancy between normal data and model data, and other fundus features, for example foveal architecture, may need to be added to the model to obtain reliable and repeatable results.

In conclusion, work presented in this paper demonstrates that clinically relevant histological parameters can be computed from multispectral images of the ocular fundus. The method has been successfully used to map the distribution of MP and haemoglobins in the retina.

There is considerable scope for further development of the methodology described here. The investigators are currently investigating the use of a new imaging system,^{26, 27} and formulating new techniques for analysing the data that take into account unknown factors that may affect the image formation process.

Once these issues have been addressed, MRIA may be applicable to a number of clinical scenarios. Most promising are those retinal and macular disorders, which require monitoring of subtle changes. MRIA's ability to quantify pigments and their distribution may help to identify individuals at risk of developing visual loss, or help ophthalmologists to make informed decisions on the continuation or termination of treatment.

An external file that holds a picture, illustration, etc.
Object name is eye2011202i1.jpg

Acknowledgments

We gratefully acknowledge financial support from the Engineering and Physical Sciences Research Council (EPSRC) under Grant number GR/S09906/01 and from the Advantage West Midlands for funding of the hyperspectral imaging facilities under grant number BGIN3264. We thank: Mr James Osborne for his generous donation of the Zeiss RCM250 fundus camera used in this work; Mr Peter Lundh of the Institute of Ophthalmology provided technical help with the design of the imaging system; the Optometry Department of the Birmingham and Midland Eye Centre for kindly loaning an artificial eye and lenses; the Consultants and medical staff of the Birmingham and Midland Eye Centre for assisting with the selection of suitable subjects.

Notes

The authors declare no conflict of interest.

Footnotes

The preliminary results of this research were presented at the 2005 EVER Congress.

References

Styles IB, Calcagni A, Claridge E, Orihuela-Espina F, Gibson JM. Quantitative analysis of multi-spectral fundus images. Med Image Anal. 2006;10 (4:578–597. [PubMed]
Hammond BR, Jr, Wooten BR, Snodderly DM. Individual variations in the spatial profile of human macular pigment. J Opt Soc Am A Opt Image Sci Vis. 1997;14 (6:1187–1196. [PubMed]
Ciulla TA, Curran Celentano J, Cooper D. Macular pigment optical density in a midwestern sample. Ophthalmology. 2001;108 (4:730–737. [PubMed]
Bone RA, Landrum JT, Friedes LM, Gomez CM, Kilburn MD, Menendez E, et al. Distribution of lutein and zeaxanthin stereoisomers in the human retina. Exp Eye Res. 1997;64 (2:211–218. [PubMed]
Bone RA, Landrum JT, Fernandez L, Tarsis SL. Analysis of the macular pigment by HPLC: retinal distribution and age study. Invest Ophthalmol Vis Sci. 1988;29 (6:843–849. [PubMed]
Carpentier S, Knaus M, Suh M. Associations between lutein, zeaxanthin, and age-related macular degeneration: an overview. Crit Rev Food Sci Nutr. 2009;49 (4:313–326. [PubMed]
Landrum JT, Bone RA, Kilburn MD. The macular pigment: a possible role in protection from age-related macular degeneration. Adv Pharmacol. 1997;38:537–556. [PubMed]
Beatty S, Boulton M, Henson D, Koh HH, Murray IJ. Macular pigment and age related macular degeneration. Br J Ophthalmol. 1999;83 (7:867–877. [PMC free article] [PubMed]
Beatty S, Murray IJ, Henson DB, Carden D, Koh H, Boulton ME. Macular pigment and risk for age-related macular degeneration in subjects from a Northern European population. Invest Ophthalmol Vis Sci. 2001;42 (2:439–446. [PubMed]
Loane E, Kelliher C, Beatty S, Nolan JM. The rationale and evidence base for a protective role of macular pigment in age-related maculopathy. Br J Ophthalmol. 2008;92 (9:1163–1168. [PubMed]
Moeller SM, Parekh N, Tinker L, Ritenbaugh C, Blodi B, Wallace RB, CAREDS Research Study Group et al. Associations between intermediate age-related macular degeneration and lutein and zeaxanthin in the Carotenoids in Age-related Eye Disease Study (CAREDS): ancillary study of the Women's Health Initiative. Arch Ophthalmol. 2006;124 (8:1151–1162. [PubMed]
Bernstein PS, Yoshida MD, Katz NB, McClane RW, Gellermann W. Raman detection of macular carotenoid pigments in intact human retina. Invest Ophthalmol Vis Sci. 1998;39 (11:2003–2011. [PubMed]
Elsner AE, Burns SA, Beausencourt E, Weiter JJ. Foveal cone photopigment distribution: small alterations associated with macular pigment distribution. Invest Ophthalmol Vis Sci. 1998;39 (12:2394–2404. [PubMed]
Delori FC, Goger DG, Hammond BR, Snodderly DM, Burns SA. Macular pigment density measured by autofluorescence spectrometry: comparison with reflectometry and heterochromatic flicker photometry. J Opt Soc Am A Opt Image Sci Vis. 2001;18 (6:1212–1230. [PubMed]
Berendschot TTJM, Goldbohm RA, Klopping WAA, van de Kraats J, van Norel J, van Norren D. Influence of lutein supplementation on macular pigment, assessed with two objective techniques. Invest Ophthalmol Vis Sci. 2000;41 (11:3322–3326. [PubMed]
Elsner A, Burns SA, Beausencourt E, Weiter JJ. Foveal cone photopigment distribution: small alterations associated with macular pigment distribution. Invest Ophthalmol Vis Sci. 1998;39 (12:2394–2404. [PubMed]
van Norren D, Tiemeijer LF. Spectral reflectance of the human eye. Vision Res. 1986;26 (2:313–320. [PubMed]
Ruddock KH. Evidence for macular pigmentation from colour matching data. Vision Res. 1963;61:417–429. [PubMed]
Wooten BR, Hammond BRJ, Land RI, Snodderly DM. A practical method for measuring macular pigment optical density. Invest Ophthalmol Vis Sci. 1999;40 (11:2481–2489. [PubMed]
Snodderly DM, Hammond BR. In vivo psychophysical assessment of nutritional and environmental influences on human ocular tissue: lens and macular pigmentIn: Taylor A (ed).Nutritional and Environmental Influences on the Eye CRC Press: Boca Raton, FL; 1999. ch. 13.
Beatty S, van Kuijk FJ, Chakravarthy U. Macular pigment and age-related macular degeneration: longitudinal data and better techniques of measurement are needed. Invest Ophthalmol Vis Sci. 2008;49 (3:43–45. [PubMed]
Bernstein PS, Delori FC, Richer S, van Kuijk FJ, Wenzel AJ. The value of measurement of macular carotenoid pigment optical densities and distributions in age-related macular degeneration and other retinal disorders. Vision Res. 2010;50 (7:716–728. [PMC free article] [PubMed]
Mordant DJ, Al-Abboud I, Muyo G, Gorman A, Sallam A, Ritchie P, et al. Spectral imaging of the retina. Eye (Lond) 2011;25 (3:309–320. [PMC free article] [PubMed]
Styles IB, Claridge E, Orihuela Espina F, Calcalgni A, Gibson JM. Quantitative interpretation of multispectral fundus imagesIn: Amini AA, Armando Manduca A (eds).Medical Imaging 2005; Physiology, Function, and Structure from Medical ImagesProceedings of SPIE Vol. 5746.SPIE: Bellingham, WA; 2005. 267–278.278.
Haykin S. Neural Networks: A Comprehensive Foundation2nd edn.Prentice Hall: Upper Saddle River, New Jersey, USA; 1999.
Everdell NL, Hebden JC, Claridge E, Styles IB, Calcagni AS. Multispectral imaging of the ocular fundus using LED illumination. Proc of SPIE. 2009;7371 (1:73711C–73711C-6.
Everdell NL, Styles IB, Calcagni A, Gibson J, Hebden J, Claridge E. Multispectral imaging of the ocular fundus using light emitting diode illumination. Rev Sci Instrum. 2010;81 (9:093706. [PubMed]
Boettner EA, Wolter JR. Transmission of the ocular media. Invest Ophthalmol Vis Sci. 1962;1:776–783.
Weale RA. Age and the transmittance of the human crystalline lens. J Physiol. 1988;395:577–587. [PubMed]

Articles from Eye are provided here courtesy of
Nature Publishing Group