|Home | About | Journals | Submit | Contact Us | Français|
Projective transformation is a mathematical correction (implemented in software) used in the remote imaging field to produce distortion-free images. We present the application of projective transformation to correct minor alignment and astigmatism distortions that are inherent in dispersive spectrographs. Patterned white-light images and neon emission spectra were used to produce registration points for the transformation. Raman transects collected on microscopy and fiber-optic systems were corrected using established methods and compared with the same transects corrected using the projective transformation. Even minor distortions have a significant effect on reproducibility and apparent fluorescence background complexity. Simulated Raman spectra were used to optimize the projective transformation algorithm. We demonstrate that the projective transformation reduced the apparent fluorescent background complexity and improved reproducibility of measured parameters of Raman spectra. Distortion correction using a projective transformation provides a major advantage in reducing the background fluorescence complexity even in instrumentation where slit-image distortions and camera rotation were minimized using manual or mechanical means. We expect these advantages should be readily applicable to other spectroscopic modalities using dispersive imaging spectrographs.
Grating spectrographs equipped with two-dimensional (2D) imaging detectors are widely used in microspectroscopy and fiber-optic instrumentation because of their ability to capture many spectra simultaneously.1–6 Ideally, a dispersive imaging spectrograph should record a two-dimensional image where one axis contains the spatial information and the other axis contains the spectral information. In practice, recorded images suffer from slight distortions due to light, which is off the primary optical axis being dispersed, at an angle to the grating normal. Distortions in dispersive instrumentation arise from monochromatic and chromatic aberration of collimating and focusing elements, as well as physical misalignment of optical elements. Most axial transmissive spectrographs use aberration-corrected lenses to minimize these problems. The Offner spectrograph uses a concentric spherical mirror and grating, for which the monochromatic aberrations will cancel if the instrument is perfectly aligned and an aberration-corrected grating is employed.1,7
Since the first axial transmissive spectrograph for Raman spectroscopy was introduced it has become a popular design for imaging Raman spectrographs.1 In this paper we use a Raman system employing an axial transmission spectrograph to illustrate some important correction techniques. In this configuration, major sources of defects in spectroscopic images include misalignment of the imaging detector with respect to the optical dispersion axis, uncorrected slit-image curvature, and other aberrations in the spectrograph.8,9 The spectral and spatial axes are affected by these problems, and recorded spectroscopic images are slightly distorted. These distortions may seem slight, but, as we show, they significantly affect the accuracy and precision of the underlying spectra and the quality of the derived optical images.
Instrument modification to reduce aberrations and software correction of the acquired signal are the two major approaches to reduce these problems. A classic example is the use of toroidal mirrors to reduce astigmatism in the Czerny–Turner spectrograph. Hardware correction using curved input slits was demonstrated in the early days of high-throughput spectrographs. Slit-image curvature (or smile) is where “the image of the slit on the CCD [charge-coupled device] detector has a slope that varies with wavelength”.10 Slit-image curvature is a prominent artifact in imaging Raman spectrographs, and a number of corrective approaches have been described.8,11–14 In the axial transmission design, collection fiber optics can be arranged in an arc with curvature opposite to the slit-image curvature at the detector plane.8,11 This hardware approach will not correct pushbroom microspectrometry or other images collected using free-space optics. Other hardware solutions include curved entrance slits, optical focusing elements specially designed to minimize chromatic aberration in the image plane over the wavelength interval of interest, and camera mounting systems that allow fine rotation adjustments of the imaging detector. Traditional approaches to correct curvature using hardware completely correct curvature at only a single wavelength, while image transformation allows correction across the entire spectral image.
Software correction methods vary between instrument manufacturers, applications, and even members within research groups. Software slit-image curvature correction realigns the data points in each recorded spectrum such that atomic emission lines appear at the same pixel position, regardless of the vertical position along the detector. In these methods the pixel shift due to curvature is measured and the image rows are offset using sinc interpolations to resample the data accordingly.15 Matlab code for this correction has previously been published.16 A similar method was developed for planar array IR microspectrometry and has since been adopted for Raman spectroscopy.17 In these previous works, each spectral row in the image is corrected horizontally to align pixels in each row to a consistent spectral position, based on the assumption that the dispersion axis corresponds adequately to the pixel row axis. Individual spectra are subsequently recovered either from regions where multiple image rows are binned or from individual rows on the imaging detector.3,18
Distorted images are also encountered in remote sensing, and several software corrections have been reported.19–22 In remote sensing, images distorted by tilt, surface curvature, and other distortions are corrected using software to produce orthophotos, in which the distortions have been removed and the scale is uniform. Projective transformation is used to align images taken from different perspectives with different optical systems to produce distortion-free registered images.19,20 This approach is used in satellite imagery to correct for aberrations in the focusing optics of the camera, as well as for the position and angle of the camera relative to the planet surface (including the radius of curvature relative to the field of view). Positions of identifiable markers are measured in recorded images (called measured control points) and their corresponding ideal positions (ideal control points) are used to convert the positions of the pixels in the ideal image to positions in the recorded image. Finally, the pixel intensities in the ideal image are interpolated from the neighboring pixel intensities in the recorded image. Bilinear interpolation is typically used for quantitative applications.
Projective transformation can also be used to correct distortions in spectroscopic images. One embodiment of distortions in spectroscopic images is an increase in the apparent complexity of the fluorescence background. Raman spectra are typically accompanied by background fluorescence, which arises from several sources including auto-fluorescence and degradation or oxidation byproducts. Stray light or reflections may also be misconstrued as fluorescence and contribute to the apparent complexity of the background signal. Fluorescence emission intensities are much lower at near-infrared wavelengths than in the visible. The relative intensity of fluorescence emission profiles decreases at longer wavelengths; however, the emission profiles extend into the near infrared with intensities comparable to those of Raman bands. The near-infrared component of the fluorescence profiles should have simple shapes, approximated by low-order polynomials, because the major fluorescence emission features are located at much lower wavelengths.
However, observed backgrounds in near-infrared Raman spectroscopy are often much more complex than would be expected. These observed fluorescence backgrounds must be corrected, and they can significantly affect quantitative analysis of Raman spectra.
Fluorescence backgrounds in Raman spectroscopy are often corrected using polynomial fitting methods. Background-correction methods based on polynomial fitting can be traced back to at least 1975 and23,24 were incorporated into the Gas chromatography/Infrared Fourier Transform Software (GIFTS).25 The GIFTS background-correction method was later included in the commercially available GRAMS spectral processing package by Thermo-Galactic. Related background-correction methods include fitting a single polynomial26 and taking the minima of several fitted polynomials (the “adaptive minmax” method),27 as well as other background-correction methods.28–33 Consistent among all of the background-correction methods is the principle that the polynomial order required to remove the spectral background can be taken as a rough indication of the background complexity.
In this study, we apply projective transformation to correct for subtle distortions inherent in imaging spectrographs. We show that projective transformation in spectral image preprocessing improves the quality and reproducibility of simulated and experimental spectral images. We present a simple process for collection of spatial and spectral patterns and determination of the control points that is applicable to data collected through imaging optics (such as microscopes), fiber-optic probes, or any other extended source collection systems. The conversion of measured control points to ideal control points is achieved by collecting images with known spatial and spectral patterns. Simulated and experimental Raman data, from microspectroscopy and fiber-optic spectrographs, are used to determine the effects of distortion correction on recovered spectra and to demonstrate the utility of a transform function on the accuracy and reproducibility of recovered spectra. We propose that even slight instrument misalignment leads to a surprising increase in the apparent background complexity of spectra.
Software distortion correction uses polynomial functions to map and correct the spatial distortion in images. Ideal and measured control points are selected and ideal positions are then computed. A polynomial function is calculated to relate the ideal and real control point positions. Rotational misalignment of the detector can be corrected with a first-order polynomial, while slit-image curvature varies with wavelength and is described by both parabolic and inverse cosine terms. Theory supports the use of a second-order polynomial because the slit-image curvature is approximated as a function of the square of the vertical position.12 Mapping functions with higher-order polynomials can be used but are unnecessary to correct slit-image curvature and rotational misalignment. Polynomial functions are calculated for the forward and inverse projective transformations. The forward transform is calculated to convert the measured control point positions to the ideal control point positions. The inverse transform is calculated by interchanging the control point sets, to convert the ideal control point positions to the measured control point positions. A forward transform is used to take the imperfect image and remove distortion, while the inverse transform produces a distorted image from the ideal image.
Polynomial fits were calculated by minimizing the residuals in fitted control points according to:
In Eqs. 1 and 2, x and x′ are the x-positions of control points in the initial and transformed images and y and y′ are the y-positions of control points in the initial and transformed images. Coefficients a and b are estimated using a least square error fit of the coupled Eqs. 1 and 2 describing the x and y positions. The system is over-constrained by using more control points than necessary to calculate the polynomial coefficients. The fitting is iterative, where the polynomial fit is first calculated, and then control points with fitting error significantly greater than the average are deselected to improve the fit. This point rejection process is iterated until no outliers are detected. This process is handled automatically in the Matlab software routine cp2tform, using the second-order polynomial transformation type. After functions are calculated to map pixel positions between the real and ideal images, intensities for each pixel in the ideal image are calculated by interpolating intensities from the measured image. Interpolated intensities are assigned to the appropriate pixel positions in the output image. This procedure is equivalent to extending to two dimensions the quadratic correction in one dimension used in an axial transmissive spectrograph to correct for slit-image curvature. Instead of a one dimensional fitting of the curvature along the x axis, we fit quadratic polynomials to both axes simultaneously so that both the x and y axes are corrected. The procedure is general and will correct errors from any source, provided that they do not result in defocusing the image on the detector. Extension to higher-order polynomials is possible.
The first step in projective transformation is to measure control points in the dispersion and spatial axes. Spatially and spectrally patterned calibration images are recorded and control points are identified where lines of constant spatial position and lines of constant spectral position intersect. For microspectrometry, we used a reflective 1951 USAF glass slide resolution target (NT38-257, Edmund Optics, Barrington, NJ) affixed to a white-light calibration source to measure the spatial pattern and a neon source to measure the spectral pattern. For fiber-optic-coupled spectroscopy, we used the spatial pattern arising from the fiber-optic probe to measure the spatial pattern and a neon source to measure the spectral pattern. We developed a simple procedure to determine control points from data acquired by a Raman microscopy system and a Raman fiber-optic system. The Appendix provides a more detailed description of the methodology used for the determination of control points.
A binary checkerboard image was generated using the Matlab (R2008b, The MathWorks, Natick, MA) function ‘checkerboard’ with intensities 0 and 255 to simulate a horizontal spectral dispersion axis and a vertical spatial axis. This pattern provides improved contrast for visualization of the image distortion. The control points used in this simulation were measured from a fiber-optic-coupled Raman imaging system. From the control points both the forward and inverse projective transformations were computed. The inverse projective transformation was first used to simulate a typical pattern generated by a dispersive Raman instrument. This distorted pattern was then corrected using the forward transform function.
In the projective transformation, pixels in the calculated image were sampled from pixels within the initial image. However, some output image pixels in the forward-transform process would require sampling pixels outside the bounds of the input image due to distortion. Output pixels intensities were identified as “not a number” if they required interpolation of pixel intensities outside the area of the initial input image. In order to accommodate the extra-image pixel sampling, we created a checkerboard image larger than the transformed image. This allowed visualization of curvature over the entire calculated image, including the edges where intensities would otherwise be undefined. For simulated and measured Raman data, initial images had the same pixel dimensions as the detector surface, and undefined points at the edges of the transformed images resulted in the black edges.
Raman data were acquired with two different Raman spectrographs: a 785 nm Raman microscope with a dispersive spectrograph (HoloSpec, Kaiser Optical Systems, Inc., Ann Arbor, MI) and an 830 nm Raman fiber-optic-coupled instrument with a dispersive spectrograph (RXN-1, Kaiser Optical Systems, Inc.). Custom-designed fiber-optic probes were constructed for tissue measurements (FiberTech Optica, Inc., Kitchener ON, Canada). The fiber-coupled data was collected using a pen-like probe in which the illumination and collection fibers were interleaved around a ring approximately 5 mm in diameter at the tip. Raman spectra are collected by positioning the probe approximately 1 to 2 mm from the sample so that the sample is globally illuminated. Collected images were 1024 × 127 pixels for the microscopy system and 1024 × 255 pixels for the fiber-optic system. Raman transects consisted of 127 or 255 spectra and were imported into Matlab for preprocessing using software routines written in-house.34 Cosmic rays present in the data were removed using either a GUI-based routine written in-house for microscopy data or an upper-bound spectrum method for fiber-optic data.34,35 One image frame was collected for the Raman microscopy system and ten image frames were collected for fiber-optic data. Initial recorded Raman spectral images and the spectral images after distortion correction using the appropriate forward projective transformations were compared.
Polynomial spectral baseline correction was performed, and the order of the polynomial required to correct the baseline was used as an indication of background complexity.27,34 A Raman transect of intact equine metacarpal bone acquired with the 785 nm Raman microspectrometer was examined for differences in apparent fluorescence background complexity before and after the image-distortion correction.34 Raman band intensity ratios were used for measuring bone properties; the carbonate ν1 to phosphate ν1 (1070 cm−1/958 cm−1) and phosphate ν1 to amide I (958 cm−1/1655 cm−1) were calculated for images with and without transformation. Band intensities were measured as the maximum over a range of five pixels at the band maxima with subtraction of the minima over five pixels at nearby background regions. Band intensity ratios were then calculated from the specified wavenumber intensities. Band intensities are less susceptible to residual baseline error and are less affected by noise than band fitting in spectral images where individual spectral rows have a low signal-to-noise ratio (less than 10 : 1).36
A Raman spectrum from a fixed and embedded fragment of human iliac crest bone was used to generate a simulated image, which was inverse- and forward-transformed in the same manner as the checkerboard image. The single bone spectrum was repeated over the spatial axis to produce an 811 × 127 pixel Raman image. Each row in the image was independently weighted to simulate varying reflection from the sample surface and intensity variations within the line-shaped laser beam. Intensities were weighted from 70 to 100% of the initial spectral intensity using uniformly distributed random numbers. The inverse projective transformation was used to generate a Raman transect that mimicked the distortion in Raman transects collected by the HoloSpec spectrograph. The distorted image was corrected using the forward transform.
A transcutaneous Raman spectrum from a cadaveric human tibia was used to generate a simulated fiber-optic probe image, which was transformed using the method described above. A simulated 811 × 256 pixel image was generated by repeating a single Raman spectrum across the entire spatial axis. Because spectral images recorded by fiber-optic-coupled Raman imaging systems have a distinctive striped intensity pattern, it was necessary to generate spatial profiles similar to the arrangement of 100 µm core fiber-optic bundles at the spectrograph. Gaussian profiles with a width (standard deviation) of 2 pixels, fiber-to-fiber spacing of 4.75 pixels, and an intensity of 1 were used to simulate the characteristic Gaussian intensity profiles of fiber optics. Dimensions corresponded to the spatial profiles observed from fiber-optic probes where the buffer is stripped from the spectrograph end of each fiber to efficiently pack fibers into the fixed detector height. Spectral intensities for each individual simulated fiber optic were randomly weighted to 85–100% of the initial intensity using uniformly distributed random numbers. This initial simulated Raman fiber-optic image was inverse transformed to simulate the distorted image and then forward transformed to correct for the distortion.
Projective transformation extends to two dimensions the correction that is typically only performed in one dimension to correct slit-image curvature (smile or line curvature). Spatial and spectral axes are simultaneously corrected using a projective transformation to reduce distortions from detector misalignment. We show with experimental and simulated Raman data that minor distortions have large effects on the apparent background complexity of spectra collected with dispersive spectrographs.
The principles of projective transformation are shown on high-contrast images using checkerboard patterns, shown in Fig. 1, as an ideal test image for emphasizing spatial distortion. The initial undistorted checkerboard image is shown in Fig. 1A. Because there is no simple method for producing a spectral checkerboard pattern on a spectrograph, an inverse spatial transform was applied to the checkerboard image to simulate the distortion present in experimental spectral images. The initial checkerboard pattern was larger than the desired output image and the inverse spatial transform resampled an image output area corresponding to the actual detector size, as shown by the rectangle in Fig. 1A. As shown in Fig. 1B, the inverse transform distorted the checkerboard image. Horizontal lines in the checkerboard pattern are slightly rotated and also shifted vertically downwards by 2 pixels over the 1024 horizontal pixels. This can be seen in the top-right corner of Fig. 1B where the bottom of a black square is slightly visible. Horizontal and vertical alignment was restored to the checkerboard image after application of the forward projective transformation, shown in Fig. 1C. Dark boundaries in the top left and bottom right edges of the image were formed because the boundaries were truncated by the transform. The bilinear interpolation used in the projective transformation reduced the spatial and spectral resolution by interpolation between adjacent pixels.
We first applied the projective transformation to Raman microscopy images of equine bone, which does not have uniform fluorescence in the sampling volume. In the example selected for Fig. 2, the CCD had a rotation angle of −0.69 degrees (corresponding to a 12-pixel shift across the 1024 spectral axis), indicating poor alignment of the camera with the dispersion axis. The spectral image was preprocessed using a manual cosmic ray correction step, subtraction of a reference dark image, and normalization by the measured reference white intensity. This spectral image was then processed in different ways. Spectra shown in Fig. 2A have a fluorescence background that varied with position in the image. As shown in Fig. 2B, spectra in the transformation-corrected image had almost identically shaped backgrounds. The adaptive minmax method by Cao et al. was used to correct the baseline using the iterative method and taking the minimum of four fitted polynomial functions in each iteration (first- and second-order polynomials were used, both with and without endpoint constraints).27 This method allows background to be closely fitted with low-order polynomials without any of the edge effects that are otherwise problematic in polynomial baseline corrections. Figures 2C and 2D show the baseline-corrected and normalized spectra corresponding to the spectra in Figs. 2A and 2B. Higher-order polynomials are clearly required to remove additional background complexity from Fig. 2C.
As shown in Fig. 2B, fluorescence backgrounds in biological specimen spectra are less complex and more similar to each other at different positions in the field of view. The Cao baseline-subtraction method was repeated with third- and fourth-order polynomials to further reduce the residual background. The resulting baselined spectra are shown in Figs. 2E and 2F. The image transform correction has reduced the background complexity, and less background correction is required to remove the residual background.
There are many methods for selecting the polynomial order for the correction, such as visually examining the spectra after background correction using different algorithm options (polynomial orders) and selecting the one that appears to have the simplest background. Two simple automated (operator-independent) methods can be used to examine the background-corrected spectra. Principal component analysis (PCA) methods can be used to determine the number of principal components required to describe the data (fewer principal components indicating less background). A simple numeric indicator spectral variance can also be calculated by taking the mean of the standard deviations calculated from each baselined and normalized spectral row. Numeric results for the data shown in Fig. 2 are included in Table I, from which it can be seen that results improve only gradually beyond a third-order correction (indicating that we should select third- and fourth-order polynomials in the Cao method). The total difference in the variance estimate between the first-order correction and higher correction orders is much greater for the initial data than for the transformed data, due to the apparent background complexity.
The reduction of background complexity improved reproducibility in calculating Raman intensity ratios. The means and standard deviations of the ratio of the carbonate ν1 to phosphate ν1 Raman band (1068 cm−1 : 958 cm−1) and the ratio of phosphate ν1 to amide I (958 cm−1 : 1650 cm−1) were calculated from spectra preprocessed by correcting for cosmic rays and dark subtracting. Spectra were then processed with and without the projective transformation. In both cases, the spectra were intensity corrected using a NIST-traceable intensity calibration source, and band intensity ratios were calculated. The mean values of the carbonate-to-phosphate ratio are similar (0.174 initially and 0.157 with the transform). However, the standard deviation in the ratio decreased from ±0.032 to ±0.021 (to 65% of the initial value) after projective transformation correction. The phosphate-to-amide I ratio showed a small increase in the mean value (6.55 to 7.19), and the standard deviation was reduced from ±2.00 to ±1.58 (to 79% of the initial value). Because this image was of a heterogeneous biological specimen, we would not expect a zero standard deviation. However, some of the variation is due to instrumental error. By taking 5-point windows in calculation of the band intensities, we have removed much of the influence of random noise (standard deviations of the individual band intensities for the two methods all varied by less than 2%). The decrease in error after application of the image transform is pronounced. The phosphate-to-amide I ratio is more susceptible to rotational misalignments in the CCD because the bands span a large wavenumber range and is complicated by a lower CCD efficiency for the high wavenumber band (1650 cm−1). As a result, the relative standard deviations are greater for the phosphate-to-amide I ratio. For the carbonate-to-phosphate ratio, the relative standard deviation dropped from 18.4% to 13.4% after application of the image transform. For the mineral-to-matrix ratio, the relative standard deviation dropped from 30.5% to 22.0%.
We expect that the projective transformation method will improve recovery of spectra and may enable detection of very subtle changes in tissue. Background complexity in biological Raman images is routinely reduced using the projective transformation correction. In our laboratory, we have observed reductions in polynomial background complexity required to correct for background signals. With the projective transformation, we now use much lower order (first to third) polynomials for background correction as compared to the high-order (fourth to ninth) polynomials previously required. Low-order polynomials more closely resemble theoretical approximations of biological fluorescence, which should not have a complex shape in the near-infrared region.
As previously observed, band position shifts with respect to CCD row position and must be corrected to compare or sum the spectral information.8,10,11,15,17 Likewise, row-to-row variations in image intensity combine with very slight image rotation to compound the effective background complexity of the spectra more than expected. This is exacerbated in fiber-optic systems because the Gaussian spatial profile of each optical fiber leads to substantial intensity variations from row to row. Likewise, microscopy of specimens with variable optical scattering or absorption coefficients over the relevant spatial scale will show variation of intensity. Biological specimens commonly have optical heterogeneity over relevant microscopy scales (hence the use of microscopy), and exhibit strong row-to-row intensity variations. As a result, even slight rotations between the detector axis and grating dispersion direction influence the effective background complexity.
Compared to microscopy images, 2D images collected with fiber-optic probes can be more complicated to interpret. Nonuniformity may be increased by tissue heterogeneity over a large sampling area and by nonuniform collection efficiency. In the example shown in Fig. 3, the CCD had a rotation angle of 0.11 degrees (a misalignment of less than 2 pixels across the 1024 pixel spectral axis). This misalignment of the CCD to the spectrograph is usually considered negligible. Figure 3 shows representative spectra of exposed human bone from regions in the fiber-optic image that were globally illuminated. The effects of image distortion correction are striking, even for images collected from an instrument with vanishingly small distortions. Figure 3A shows spectra from two regions of a fiber-optic image after preprocessing spectra using traditional preprocessing techniques. A large variability in background signal is observed and bands in the 200 to 600 pixel region are obscured because of the large and varying background. After preprocessing spectra using the corrective transform, shown in Fig. 3B, variability in background is significantly reduced and enables visualization of bands even before background correction.
Sample heterogeneity contributes to calculated variance. In the examples above it is not clear how much of the initial variance is due to sample heterogeneity versus instrumental error. Here the method is demonstrated with a spectral image of Teflon, which is expected to have homogeneous properties. We demonstrate the effect with measurements of Teflon using an offset reflection fiber-optic probe (pen-like probe37), without focusing optics so that the collection cones of the fibers are overlapped. The collection fibers sampled a small area of the Teflon, such that spectra are expected to be identical. Teflon data were collected on a spectrograph with a rotational misalignment of less than two pixels across the detector. Moreover, the peaks located near pixels 350 (732 cm−1) and 700 (1382cm−1) span a short enough range across the spectral axis to make the rotational misalignment less than 1 pixel. In Fig. 4, the ratio of Teflon band intensities at 732 cm−1 and 1382 cm−1 was calculated for each spatial pixel position for an image before performing the transform (left-hand panel) and after the transform was applied (right-hand panel). As shown in the left-hand panel of Fig. 4, the value of the 732 cm−1 : 1382 cm−1 intensity ratio was highly variable across the image (7.32 ± 2.00). After the image transform, the ratio was less variable across the image (6.81 ± 0.40). The image transform reduced the error in ratios recovered at the edges of the image, particularly where individual fibers are in better focus and where row-to-row intensity variations are maximized. The decrease in relative standard deviation of the band ratio from 27.4% to 5.9% across the single image shows the effect of a corrective transform, even for images with nearly imperceptible distortion.
To determine whether the improvement in intensity ratio variation is due to interpolation, we compared the effects of smoothing due to different image correction methods. A major concern was that the interpolation used in the image transform might smooth the spectral image and cause an apparent reduction of variability of the intensity ratio. A degree of smoothing accompanies all software corrections for slit-image curvature and image rotation. Interpolation is inherent to the band-shifting methods used to correct slit-image curvature. However, the interpolation is along only one dimension of the CCD. This will smooth intensity along rows where there is potentially intensity variation due to CCD rotation misalignment (interpolation between pairs of pixels). Software corrections for rotation use an image transformation based on sine and cosine pixel position transformations. Like rotation correction, image transformations using bilinear interpolation combine signal from groups of four pixels according to position of the new virtual pixel within the four surrounding pixels from the initial image.
We compared the degree of smoothing associated with different software correction approaches. A test image was simulated and contained normally (Gaussian) distributed random numbers with a standard deviation of 1. Four methods were applied to the simulated test image: (1) curvature correction only, (2) rotation correction only, (3) rotation correction followed by curvature correction, and (4) projective transformation. The standard deviation was calculated over all pixels in the image, and the results are shown in Table II. The reduction in standard deviation is correlated to the total amount of smoothing, with a lower number associated with a greater smoothing effect.
The initial image (815 × 127 pixels, corresponding to microscopy data) had a measured standard deviation of 0.997. Using only slit-image curvature for the measured microscopy band positions imposed only a moderate amount of smoothing and resulted in a standard deviation of 0.830. Rotational correction for a −0.69 degree rotation reduced the standard deviation to 0.647. Sequential rotational and slit-image curvature corrections produced the greatest level of smoothing, with a standard deviation of 0.570. Projective transformation produced a standard deviation of 0.687, which was similar to the result produced after only rotational correction. Moderate data smoothing is observed for the projective transformation correction, which was expected because the algorithm employs interpolation functions. However, smoothing alone cannot account for the dramatic improvements in band ratio standard deviations for Teflon data. Variability of the Teflon image band ratios decreased to 21% of the initial value after the image transform, far more than explained by interpolation alone (which reduced the standard deviation of the image to 69% of the initial value).
To further demonstrate this, we used PCA to demonstrate that the number of principal components required to model the data decreased after performing the spectral image corrections. The fundamental concept is that there are a limited number of chemical species that additively contribute some spectral component to the experimentally recorded data. Likewise, the fluorescence background should be composed of one or more fluorophores generating a background. The number of linear terms required to model the data is indicative of the number of chemical species contributing to the data. Three different automated methods were used for determining the number of PCA components required to model the Teflon spectral image, results from which are shown in Table III. The three methods used to determine the number of components were significant factor analysis (SFA), residual percent variance (using a 0.1% threshold), and the average eigenvalue or “eigenvalue-one criterion”.38 Though the three methods ultimately select different numbers of components, the transformation-corrected data always resulted in the fewest principal components. Differences are illustrated by the scree plot in Fig. 5, where the initial and transformed data differ substantially. The initial image (without preprocessing) required the most components to model, followed closely by the slit-image curvature corrected image. This trend continued, with the rotated image being composed of fewer linear terms. The combined rotation and slit-image curvature corrected image and the image-transformed images were approximately equal, both being modeled with fewer terms than the other images. Note that the effects of smoothing are also evident in the scree plots. Increased smoothing results in lower global minima, and lines in the scree plots are ordered according to the total level of smoothing in each processing method at higher numbers of PCA components. The method using rotation followed by slit-image curvature correction has the global minimum in the scree plot but also imposes the greatest interpolation cost (as seen in the standard deviations above).
We used simulated microscopy and fiber-optic images to validate the projective transformation as a corrective method for imaging spectrographs. Simulated data with known spectral properties were inverse transformed to introduce distortions that were observed in experimental data. Forward transformation of the distorted simulated images allowed recovery of the initial input spectra with no loss of spectral integrity.
Simulated Raman microscopy images are shown in Fig. 6 and the corresponding spectra in Fig. 7. Importantly, the initial undistorted image was generated using a single spectrum repeated across the entire spatial axis. Intensities of each spectrum in the image were randomly scaled. The spectral and spatial axes in the initial spectral image are perfectly aligned to the image axes, as shown in Fig. 6A. Spectra from the initial image reflect a constant fluorescence pattern and constant relative band intensities throughout the spatial axis, shown in Fig. 7A. After applying the inverse transform, Raman bands are slightly curved along the vertical axis and the image is slightly rotated clockwise. Distortion in the image is imperceptible in Fig. 6B, even with the vertical and horizontal lines added for emphasis; however, there were significant differences in the individual spectra, shown in Fig. 7B. The forward transform was used to correct the distorted image. Individual spectra from the corrected image have a fluorescence background consistent with spectra from the initial undistorted image.
A comparison of spectra in Fig. 7 emphasizes the importance of correcting for distortions even when they are not visually apparent in the recorded dispersion image. Figure 7A shows several rows of spectra from the initial image taken from rows near the middle of the image shown in Fig. 6A. There were significant differences in the apparent fluorescence background, shown in Fig. 7B, and individual spectra no longer appeared as scaled versions of a single spectrum. Apparent differences in the fluorescence background of simulated Raman data result only from collating different spectral regions of spatially adjacent image rows. Distortion also affected row-to-row variation in spectral band intensities because different spatial positions contributed to the intensity of a spectral band in a single pixel row. Figure 7C shows the spectra after the corrective forward transform is applied, where individual spectra are scaled versions of a single spectrum with the same apparent fluorescence background. Minor bands, such as the one near pixel 175, were broadened because of the interpolation process. Exact row positions of spectra in Figs. 6A and 6C were not identical because the images were truncated and the row positions shift slightly during image transformation.
Simulated Raman fiber-optic spectral images are shown in Fig. 8 with the corresponding spectra shown in Fig. 9. The signal from each optical fiber is shown as a horizontal stripe across the image, with many fibers making up the whole image. Because of the inherent brightness contrast in our fiber-optics system, rotational misalignment in fiber-optic-coupled systems is more easily observed than in our Raman microscopy image. Additionally, slit-image curvature is more pronounced because the fiber-optic probe system is fitted with a 256-row CCD, while the microscope is fitted with a 128-row CCD. In the 256-row detector system, the vertical angle between incident light and the central optical axis is increased because a taller input slit is imaged. As a result, the vertical curvature of spectral bands in Fig. 8B is more evident than in Fig. 6B because of the increased vertical detector size (256 pixels vs. 128 pixels). Figure 8C shows the image after the corrective forward transform is applied.
Spectra from the simulated Raman fiber-optic probe are shown in Fig. 9. The initial spectra are scaled versions of a single spectrum. The initial spectrum is shown in Fig. 9A. As shown in Fig. 9B, spectra no longer appeared identical after application of the inverse transform. Significantly, background contributions from the fused-silica optical fibers appeared to vary from spectrum to spectrum. Variations are subtle in Fig. 9B because the spectra are scaled to accommodate the full intensity range. Differences in the spectra manifest as inconsistencies in the vertical spacing between spectra along the wavenumber axis. Upon examination, the most intense Raman features were less than 10% of the background fluorescence intensity and fluctuations in the background intensity were substantial. These relatively large fluctuations in background intensity had a significant effect on the intensity of weak Raman bands in the simulated spectra. The spectra in Fig. 9C are shown after the corrective forward transform is applied, and spectra are consistent with the initial simulation.
We have demonstrated that there are large errors caused by minor CCD/spectrograph misalignments, slit-image curvature, and other residual distortions and aberrations in images collected with dispersive spectrographs. Imperfections are found even in the best-designed and properly aligned spectrographs. For the spectrograph on our microscope system, the camera currently has a barely perceptible rotation angle of 0.12 degrees even after careful alignment. Such imperfections affect the apparent complexity and variability in the fluorescence background and introduce substantial error in Raman peak intensity ratios. To address these problems, we developed a simple method to identify control points along the spectral and spatial axis in microscopy and fiber-optic spectroscopy systems. Control points can be used to correct for minor distortions using a projective transformation routine, even when the distortion is not visually apparent. While this method was tested and validated on Raman spectroscopy instruments, the same approach can be applied to other techniques using dispersive spectrographs.
It is important to note that instruments using an imaging detector to recover a single spectrum using vertical binning over all rows (that are fully resolved along the spatial axis) are not affected by rotational misalignment, provided that a slit-image curvature correction method is used to correct horizontal curvature (hardware or software).11,17 One-dimensional slit-image curvature corrections are sufficient for preventing degradation of spectral resolution in the case of full vertical binning. Dispersive spectrographs using linear detector arrays will be affected by any slight rotational misalignments if the individual detector elements are not taller than the source. Intensity calibration steps should mostly correct these effects, and only increased variability in signal-to-noise across the spectrum should be observed.
The projective transformation spectral correction method is a simple method for overcoming limitations of dispersion spectrograph performance while also enabling improved data processing methods. This method requires no hardware changes to existing systems. The required calibration materials are currently used in most laboratories with Raman spectroscopy systems. Implementation of this method would only require minor changes to protocols for collection of calibration spectra. The software can be implemented in Matlab or other widely used image-processing packages. While the procedure to collect calibration spectra and images for projective transformation may be slightly more complex than typical calibrations, the projective transform function should be unchanged as long as the optical alignment remains unchanged. Hence, the correction function can be calculated once when the instrument is aligned and used until the optical system is realigned or otherwise modified. A secondary benefit to this technique is that the wavelength axis should be perfectly aligned after projective transformation correction. If bands in the atomic emission spectra shift, it is an indication that the instrument alignment has been altered. Thus, projective transformed calibration spectra will provide direct evidence that the instrument alignment has changed.
One limitation of this method is that astigmatism and other defocusing aberrations are not corrected by this one-to-one correspondence mapping method. To do so would require mapping the instrument point-spread function with respect to detector position. An additional limitation of the current Matlab implementation is a lack of correction for changes in intensity resulting from any alterations to the effective detector pixel spacing. Fortunately, alterations to detector pixel spacing should be corrected by the white-light preprocessing step used to compensate for detector sensitivity using a reference broadband source.
We have described a method for correcting geometric errors, but there are other sources of error. These include spatial overlap from adjacent optical fibers and focusing errors that degrade the spatial and spectral resolution. Techniques to compensate for these will benefit from prior correction of the geometric distortions. For example, geometric image correction reduces the overlap of spectral information between adjacent spectral image rows. This effect is exacerbated in fiber-optic probes, where the outer cladding is removed to allow for tight packing of fiber optics at the spectrograph. Using this spatial transform method, the properly registered spectral images can be fit to measured spatial profiles, allowing closely spaced fiber optics with minimal signal contamination. Thus, the projective transformation enables a simple technique to recover fiber-specific data. Preprocessing to mitigate this problem and others will be described in future publications. This approach is expected to improve recovery of spectra from close-packed fiber optics and improve the spatial resolution of imaging spectroscopy.
We acknowledge financial support from the National Institutes of Health through research grant R01AR055222 (MDM), a training grant from NIH CTSA grant UL1RR024986 (KEW), and the Wallace H. Coulter Foundation. We thank Bonnie Nolan and Erin Robertson for preparing human limbs for transcutaneous Raman spectroscopy, Kathryn Dooley for providing Raman spectra of equine bone, and Gurjit Mandair for providing a human bone biopsy Raman spectrum. We thank Matthew Schulmerich for insightful discussions and for providing helpful editorial feedback. We thank Michael Pelletier for insightful email correspondence regarding slit-image curvature correction for Raman imaging.
On the Raman microscopy system, spatial patterns were measured using a 1951 USAF bar target (NT38-257, Edmund Optics, Barrington, NJ). The bar target was affixed onto a NIST-traceable white-light source (HCA, Kaiser Optical Systems, Ann Arbor, MI) to produce a distinctive banded spatial pattern over the broad spectral illumination range. Spectral patterns were measured using a neon atomic emission calibration source (HCA, Kaiser Optical Systems, Ann Arbor, MI). An appropriate atomic emission source should be selected for the spectral region under study. We tested the use of other atomic emission sources, including argon and xenon (Pencil-style calibration lamps, Oriel Instruments, Stratford, CT). While argon provides more bands for the 830 nm excitation Raman system, the convenience of the HCA neon source for instrument portability outweighed the slight improvement from having additional spectral bands and the inconvenience of using atomic emission sources in the pen-lamp format. Spatial and spectral control points were simultaneously measured on fiber-optic systems because the collection fiber optics inherently form spatial patterns at the spectrograph. Vertical bands defining the spatial axis were formed by the overlap of the atomic emission peaks from collection fiber optics. Control points were identified as maxima along the vertical profile of each atomic emission line. These can either be selected automatically by finding maxima in the derivative of the spectral image or by selecting points manually. The pixel resolution of preliminary control points can be refined to sub-pixel accuracy by fitting Gaussian functions to find the center of mass of the relevant spectral image features.
For this work, we selected the control points manually and then used sinc interpolated data to find the absolute maxima for enhancing the selected control point positions to sub-pixel accuracy (1/10th of a pixel). Control points were selected along ten neon bands for the 785 nm Raman image correction, at 830.03, 837.76, 841.84, 849.54, 859.13, 863.46, 865.44, 878.06, 885.39, and 886.58 nm. Intense bands that were approximately (horizontally) evenly spaced across the region of interest of the CCD image were selected, all of which were resolved from neighboring emission bands of comparable intensity. Six control points were selected along the slit axis for each neon band, evenly spaced across the (vertical) CCD axis at intervals of approximately 16 pixels.
In our experience, many approaches can be used to measure control points, and the optimum method depends on the exact calibration data measured. Sub-pixel accuracy is desirable for reducing any error due to the transformation process. We have also used a polynomial fitting method that could be used to determine the control point position at a higher precision, such as 1/100th of a pixel. The polynomial fitting method consisted of fitting a 2D polynomial to a sub-region based on initial crude (integer pixel position) control point positions.
Control points were then generated for an ideal image in which the spectral and spatial axes are parallel to the x and y axes of the image, respectively. Measured control points were arranged in a matrix corresponding to their spatial positions, such that rows contained control points from the same spatial position and columns contained control points from the same spectral position. Ideal control points were calculated by duplicating the experimentally measured control point matrix, then replacing the x positions in each column with the mean x position from that column and replacing the y positions in each row of with the mean y position from that row. To apply the perspective transformation for correcting the distortions in the spectral image, control points were fit to a polynomial function as described in the Materials and Methods section. The ‘cp2tform’ function (Image processing toolbox version 6.2, The MathWorks, Natick, MA) in Matlab (R2008b, The MathWorks, Natick, MA) was used to calculate the forward and inverse perspective transformation functions. Images were transformed using the ‘imtransform’ function (Image processing toolbox version 6.2, The MathWorks, Natick, MA) with the calculated forward or inverse transforms. The image processing toolbox functions used here have been present in the toolbox since at least Matlab release 2008a.
Figures A1 and andA2A2 show examples of the calibration images required for correcting slit-image distortions and slight imperfections in camera–spectrograph alignment. For microscopy instrumentation, the distortion pattern was mapped using a bar target pattern with a white-light calibration source and an atomic emission lamp. Example Raman microscopy calibration images are shown in Fig. A1. The small image at the left of Fig. A1, panel A, is an image of the 1951 USAF bar pattern at the microscope stage. Figure A1, panel A, shows the CCD image of the bar-pattern-masked white-light source where alternating bright and dark stripes show lines of constant spatial position dispersed across the spectrograph detector. Lines of constant spatial position were identified as maxima along each column of pixels after taking the absolute value of the difference between intensities in vertical pixels. The angle between the dispersion axis and the edge of the spectrograph detector can be calculated using the boundaries between the bright and dark stripes. In the example selected for panel A of Fig. A1, the CCD had a rotation angle of −0.69 degrees (corresponding to a 12-pixel shift), indicating poor alignment of the camera to the dispersion axis. This large rotation angle was used so that the rotation angle is visible and is much greater than typical camera rotation in our systems (of about 0.15 degrees or 3 pixels). Panel B of Fig. A1 shows the neon emission spectrum used to calibrate the spectral axis and correct for slit-image curvature. A continuous and nearly uniform pattern of atomic emission spectra was imaged on the CCD detector (the 1951 USAF bar target slide is not used during acquisition of the neon image). Lines of constant spectral position are found as maxima along each row of pixels after preprocessing the image by taking maxima in the recorded image.
For fiber-optic instrumentation, only a neon emission image was required because the arrangement of fiber optics at the spectrograph creates the spatial pattern. Panel A of Fig. A2 shows the neon atomic emission spectral calibration image recorded using a fiber-optic probe consisting of 50 collection fibers each with 100 µm core diameter. Experimentally measured control points for a fiber-optic system are shown in panel B of Fig. A2. Unresolved bands in either the spatial or spectral axis were not used as control points. Control points were calculated as the intersection of the lines of constant spatial and spectral position. Calculated ideal control points (corresponding to the measured control points from Figure A2, panel B) are shown in panel C of Fig. A2. Using the control points, the perspective transformation function was calculated and then interpolation was used to calculate the corrected image. Our experiments showed that bilinear interpolation provided more accurate results than nearest-neighbor or bicubic interpolation (data not shown).