|Home | About | Journals | Submit | Contact Us | Français|
We demonstrate experimentally that the method of scanning holographic microscopy is capable of producing images reconstructed numerically from holograms recorded digitally in the time domain by scanning, with transverse and axial resolutions comparable to those of wide-field or scanning microscopy with the same objective. Furthermore, we show that it is possible to synthesize the point-spread function of scanning holographic microscopy to obtain, with the same objective, holographic reconstructions with a transverse resolution exceeding the Rayleigh limit of the objective up to a factor of 2 in the limit of low numerical aperture. These holographic reconstructions also exhibit an extended depth of focus, the extent of which is adjustable without compromising the transverse resolution.
Scanning holographic microscopy is a technique that was proposed as having the potential of recording the three-dimensional (3D) information of thick specimens in a single two-dimensional (2D) scan, thereby improving the data-acquisition speed compared with methods requiring a 3D scan.1 Speed is a crucial factor in, for example, the in vivo study of the dynamics of biological activities. Scanning holographic microscopy has other potential advantages as well, one of which is the flexibility with which the point-spread function (PSF) of the imaging mode can be engineered with the method of two-pupil interaction.2 This method allows one to synthesize PSFs that are not constrained to be real positive, but can be bipolar as well. This broadens considerably the types of imaging mode accessible to the instrument (e.g., amplitude contrast, quantitative phase contrast, fluorescence contrast). Another unique attribute of scanning holographic microscopy is that it makes it possible to capture holographic information of 3D fluorescent structures. Scanning holographic microscopy is an incoherent holographic process (even if lasers are used as sources for convenience), but because the PSF can be bipolar, the method remains quantitatively sensitive to phase information.3
The holographic approach aims to capture the 3D information of a specimen in a single shot. The data are then deconvolved a posteriori to reconstruct axial sections. This is opposite to the confocal approach in which sharp axial sections of the specimen are captured one at a time. Clearly, the single-shot holographic method is not expected to approach the sectioning capability of the confocal method. It is, however, expected to lead to a significant gain in data-acquisition time. The scanning holographic method is also capable of sectioning, as has been shown theoretically,4 but the sections must be captured one at a time, as in confocal imaging.
The principle of the method has been demonstrated with simulations, and experimentally with macroscopic objects, but its capability on a microscopic scale has not been convincingly demonstrated. The purpose of this paper is to provide the experimental demonstration of two essential aspects of scanning holographic microscopy. First, we want to show that scanning holographic imaging in its most straightforward implementation, as originally proposed, leads to images with quality comparable to that of standard wide-field or scanning microscopy. Next, we want to demonstrate the capability of the method in PSF engineering by illustrating an imaging mode leading to a transverse resolution exceeding the Rayleigh limit of the objective.
The paper is organized as follow. In Section 2 we briefly review the principles of scanning holographic microscopy. A simplified theory of its originally proposed implementation is given in Section 3. In Section 4 we describe the theoretical background for scanning holographic microscopy with a transverse resolution exceeding the Rayleigh limit, and with an extended depth of focus (DOF). In Section 5 we give technical details of the experimental setup, and finally the experimental results are discussed in Section 6.
The basic idea in scanning holographic microscopy is to reverse the order in which conventional images (holographic or not) are being captured in order to take advantage of imaging parameters that are otherwise not accessible. In conventional imaging, an objective is used to produce a magnified image of a specimen on a pixelated, spatially resolving detector (CCD or complementary metal-oxide semiconductor devices, for example). For holographic recording this image must be coherent and made to interfere with a reference beam.5,6 In this way, each object voxel is encoded into a spatial pattern that contains the 3D information of its position in space. That voxel can then be reconstructed digitally by a convolution operation simulating free-space propagation for an appropriate distance.7 In scanning holographic microscopy, the spatial pattern encoding each object voxel is synthesized by the interference of two pupil distributions. A constant frequency offset between the illuminations of the pupils allows one to directly capture the phase of the pattern in the time domain, using heterodyne methods. The pattern is projected through the objective onto the specimen and scanned in a 2D raster. A single nonimaging detector (photomultiplier, photodiode) is used to capture the time-modulated signal. The 2D raster scan effects a mapping of the encoding pattern from space domain to time domain, and the phase of the pattern is extracted by heterodyne methods without any disturbances due to zero-order background or twin-image interference. Figure 1 gives an idea of the arrangement used to implement this idea. The hardware is discussed in detail in Section 5.
In the most straightforward implementation of scanning holographic microscopy, as was originally proposed, the two pupils P1, P2 (Fig. 1) are, respectively, a small aperture (pinhole) and a spherical wave. The waves propagating from the pupils are combined by a beam splitter and form in the plane pattern (see Fig. 1), a magnified version of the pattern that is to be projected onto the specimen. This pattern is the interference of a plane wave and a co-propagating spherical wave. The size of the two waves is limited by apertures matching the size of the pupil of the objective, and the radius of curvature of the spherical wave is chosen to match the numerical aperture of the objective. In this way, the pattern projected onto the specimen is a Fresnel pattern with radius a and focal length z0. The numerical aperture of the pattern, sin α =a/z0, matches that of the objective (α is the half cone angle of the spherical wave). The Fresnel number of the pattern is F=a2/λz0, where λ is the wavelength of the illumination. The Fresnel number is equal to twice the number of interference rings observed in the pattern. Of course with a frequency offset between the two waves, these rings are temporally modulated and invisible to the eye unless the offset frequency is set to zero.
Without the modulation, the hologram captured after a raster scan of the specimen is equivalent to an in-line Gabor hologram recorded at a distance z0 from the object.8 With modulation and heterodyne detection, one captures directly a single-sideband hologram without zero order or twin image. An equivalent way to obtain the single-sideband hologram is to record the entire modulated signal with a fast data-acquisition system and to filter out one sideband digitally in the temporal frequency domain. The hologram is then reconstructed numerically.
The images reconstructed from such a hologram are expected to have the same transverse and axial resolutions as those of a conventional wide-field image obtained with the same objective. That is, the usual Rayleigh limits are
assuming a modest numerical aperture. Theoretical calculations valid for moderate numerical apertures have already been published.3 Extension of these calculations to high numerical aperture, on the basis of well-documented literature,9-12 is being pursued. Needless to say, scanning holographic microscopy with high-numerical-aperture objectives involves encoding patterns with high numerical apertures and possibly low Fresnel numbers that must be calculated exactly. If the digital reconstruction of the hologram has to minimize aberrations, the function used for the reconstruction must include the aberrations of the objective, possibly the aberrations due to the specimen and its environment13,14 and the effects of polarization, and the vectorial nature of the electromagnetic field.9,10 Our goal here is to give only a simple, intuitive description of the system and to explain in Section 3 how a resolution exceeding the Rayleigh limit of the objective is attained. To this end, it is sufficient to assume a paraxial system in which the encoding pattern has a low numerical aperture and a relatively large Fresnel number. Within the limits of these approximations, the scanning pattern projected onto the specimen can be written as
where (r,z) are the transverse and axial coordinates in specimen space, z is measured from the nominal plane of focus and thus represents a defocus distance, and Ω is the temporal frequency offset between the two pupils. The amplitude distributions Aj(r,z) (j=1,2) are given by the Fourier transforms of the pupils P1(ν), P2(ν) defocused by a distance z. (ν) is the transverse spatial-frequency coordinate in the pupil planes.15
The pupils P1(ν), P2(ν) are arbitrary complex amplitude distributions. These distributions can be synthesized by using masks, refractive or diffractive optical elements, or spatial light modulators, allowing dynamic changes. The examples discussed in this paper involve only pupils with circular symmetry. In this case we have
where νmax=sin α /λ is the cutoff frequency of the pupil of the objective and J0 is a zero-order Bessel function of the first kind. After scanning the specimen in a 2D raster, and demodulation of the signal, each specimen voxel of coordinate (r,z) is encoded into a complex pattern,
where * stands for the complex conjugate.
Originally11 it was proposed to use two pupils given by
where δ(ν) is a delta function approximating the small pinhole aperture, and circ(x)=1 for x≤1 and 0 otherwise. This choice of pupils was dictated by the desire to match the performance of scanning holographic microscopy with that of conventional wide-field microscopy. Within the realm of the stated approximations (low numerical aperture and high Fresnel number), it is reasonable to approximate the Fourier transform of a spherical wave of limited extent by another spherical wave of limited extent.16 In these conditions, we have
With small numerical apertures, the change of size due to the defocus distance z can be neglected, and the encoding pattern from Eq. (4) is
The hologram of a specimen is the convolution of the 3D distribution of that specimen with the encoding pattern S(r,z). The reconstruction of the hologram at a focus distance zR is effected digitally by a correlation of the hologram with the encoding pattern at z0+z=zR. The 3D PSF of the reconstruction is thus given by
where stands for a correlation integral. The transfer function of the system is given by the 2D transverse Fourier transform of the PSF. Thus, again within the limit of validity of the stated approximations, the transfer function is given by
The transfer function represents how a spatial frequency ν of the specimen is represented in an image plane with defocus z. Approximation (9) indicates the following two properties. First, the cutoff frequency in the reconstructed image is νmax=sin α /λ and is thus the same as that of the objective. Consequently, the transverse resolution of the reconstruction is expected to also be the same as that of the objective, namely, Δx≈λ/2 sin α. The second property is related to the out-of-focus behavior of the spatial spectrum. With a defocus z, different spatial frequencies acquire different phase shifts, leading to the usual blurred image. The phase shifts are proportional to the defocus distance z, as in conventional imaging. The axial resolution is usually defined as the defocus distance Δz at which the spatial frequency at the edge of the pupil acquires a phase shift of π. In the context of our setup, this criterion is equivalent to defining the DOF as the axial distance at which the Fresnel number of the encoding pattern changes by one unit from its value at the nominal focus. In either case, in the situation discussed, namely, an encoding pattern resulting from the interference of one planar and one spherical wave, the DOF is
the same as that of a conventional imaging system. These values are compared with the results described in Section 4.
With the scheme described in Section 3, the resolution of the reconstructed images is limited by the numerical aperture of the objective in exactly the same way as it is limited in a conventional wide-field image. In conventional imaging, the cone of rays issued from each object voxel is limited by the objective to a half cone angle α, whereas in this implementation of holographic microscopy, the cone of rays forming the projected pattern occupies the same half cone angle α. However, because the imaging steps occur in reverse order in scanning holographic microscopy, it is possible to create an encoding pattern having a numerical aperture larger than that of the objective. Rather than using a pattern formed by the interference of one plane wave and one spherical wave matched to the numerical aperture of the objective, as described in Section 3, we can use two spherical waves with curvatures of opposite signs, both matched to the numerical aperture of the objective. This is the mode of operation sketched in Fig. 1. Because the two waves have opposite curvatures, the encoding pattern projected onto the specimen has a Fresnel number twice as large as that obtained with one planar and one spherical wave. Consequently, in the limit of low numerical apertures and large Fresnel numbers, the reconstruction of a hologram encoded with this pattern is expected to have a transverse resolution half as large as the Rayleigh limit of the objective.16 The images reconstructed from holograms obtained with an encoding pattern resulting from the interference of two spherical waves with opposite curvatures have the additional property exhibiting an extended depth of focus, as will be shown, which may be of interest in certain applications.
Following the same steps as in Section 3 and assuming the same simplifying approximations, the two pupils are now
with νmax=sin α /λ as before. The wave amplitudes interfering in specimen space are approximately given by the two spherical waves:
Neglecting the change of size of these waves with defocus distance z, the encoding pattern is, from Eq. (4),
The 3D PSF is again given by Eq. (8), and the resulting transfer function is given by
Comparing this transfer function with that of approximation (9), obtained with an encoding pattern resulting from the interference of a planar and a spherical wave, two remarkable differences can be pointed out. The first difference is that the cutoff frequency of the transfer function of approximation (14) is twice that of the objective. Namely,
We thus expect a reconstructed image with twice the resolution of the objective, namely, Δx≈λ/4 sin α, at least in the limit of low numerical aperture. The second difference is that the phase acquired by a certain spatial frequency ν with defocus distances grows quadratically with z, as opposed to the linear relationship of conventional imaging [approximation (9)]. The reason for this is that the Fresnel number of the encoding pattern with two waves of opposite curvatures changes only quadratically with defocus rather than linearly. Consequently, we expect the reconstruction of this hologram to exhibit an extended DOF. If we again define the DOF as the defocus distance corresponding to a phase change of π for the spatial frequency νmax at the edge of the pupil, we find16 . Using the relationships sin α =a/z0 and F =2a2/λz0, we can express the DOF as
where F is the Fresnel number of the encoding pattern. Compared with the DOF of Eq. (10), this indicates a DOF extended by a factor .
Figure 1 is a sketch of the scanning holographic microscope that was built. A He-Ne laser beam (λ=633 nm) is split in two parallel beams with beam expanders consisting each of a 10× microscope objective, a 25-μm pinhole, and a 12-cm focal-length achromat as a collimating lens. The attenuators are each made of a half-wave plate in a rotating stage followed by a fixed polarizer. One of the beams goes through an electro-optic phase modulator (Linos LM0202 Phas) driven by a sawtooth waveform (Stanford Research System DS 345) followed by a high-voltage amplifier (FLC Electronics A800). The sawtooth waveform has a peak-to-peak voltage twice the half-wave voltage of the modulator (≈360 V at 633 nm). The eventual elements creating the spatial modulation of the pupils (masks, spatial light modulators) are placed in planes P1 and P2. For the cases discussed in this paper, the pupils are simple clear apertures. The two waves are combined by the beam splitter to create an interference pattern in the plane pattern. The pattern is then reduced in size and projected through the objective onto the specimen. Lenses L1, L2, L3 are achromats with 16-cm focal length.
To create a scanning pattern resulting from the interference of a planar wave and a spherical wave, as discussed in Section 3, one of the beam expanders is removed, and the beam is loosely focused one focal length in front of lens L1. Lens L2 is then positioned axially so as to produce the needed curvature in the pattern aperture. To create a pattern resulting from the interference of two spherical waves with opposite curvatures, the two lenses L1, L2 are displaced axially in opposite directions to produce the needed positive and negative curvatures in the aperture pattern.
The goal of the experiment is to illustrate the differences between the case discussed in Section 3, which is equivalent to conventional wide-field microscopy, and the case discussed in Section 4, which exhibits a higher resolution and an extended DOF. To show these features, it is important to ensure that the net resolution of the entire system is limited by the objective only and not by any other component in the chain. To ensure this, we chose an objective of modest numerical aperture and a sampling rate higher than absolutely necessary to avoid possible aliasing problems or other sampling limitations. The objective is an infinity-corrected Mitutoyo 10× Plan Apo with a working distance ≈3 cm, a focal length of ≈2 cm, and an effective numerical aperture sin α ≈0.2. The sample was scanned in a 2D raster with an X-Y piezo stage (Physik Instrument Hera P-625).
There are two possible ways to demodulate the signal. The first is to use phase-sensitive detection (e.g., a lock-in amplifier). The second is to collect the entire signal and filter it in Fourier space. The latter method was chosen for the results shown in Section 6. The data are collected by a GageScope CS1602 acquisition system, and data manipulation is supported by matlab. Since the collected data are in the form of a continuous signal while the specimen is being scanned, it is necessary to ensure that the modulation of the signal has the same phase at the start of each scanned line. To avoid difficult synchronization between the modulation and the scanning stage, we collected a reference signal synchronously with the data (see Fig. 1). Data and reference signals are treated in exactly the same manner, and the phase of the modulation of each data line is corrected by using the phase of the corresponding reference signal line.
In this way, all possible phase shifts due to mechanical motions or electronic drifts are canceled out.
In the first experiment we used a scanning pattern created by the interference of a plane wave and a spherical wave with a numerical aperture matching that of the objective (N.A.≈0.2), as discussed in Section 3. The pattern projected onto the sample has a diameter≈75 μm, a Fresnel number F≈12, and a numerical aperture sin α ≈0.2. The numerical aperture is limited by the objective, but the other parameters can be varied since the curvature radius z0 of the spherical wave is a free parameter, as described in Section 3. The temporal modulation frequency of the projected pattern is 25 kHz, and the data-acquisition rate is 100 ksample/s. The expected transverse and axial resolutions of the system are those of the objective. That is, Δx≈1.22λ/2 sin α ≈2 μm and Δz ≈λ/sin2 α ≈16 μm. The scanned field is 300 μm ×300 μm. After filtering each line, the hologram consists of an array of 3000×3000 samples. This is more than needed to accommodate the expected resolution, but this was chosen to avoid all possible undersampling problems.
Figure 2(a) shows the phase of a hologram of a 1-μm-diameter pinhole. Since the pinhole size is smaller than the expected transverse resolution, this hologram is a good representation of the complex encoding pattern. Figure 2(a) confirms that the encoding pattern has indeed a Fresnel number F≈12, a diameter 2a75 μm, and thus a numerical aperture sin α ≈0.2. The autocorrelation of this pattern, after equalization of its absolute value, provides an experimental measure of the PSF of the system. A trace through the autocorrelation of the pattern is shown in Fig. 2(b). Its FWHM is Δ≈2.15 μm, which compares favorably with the resolution Δx≈2 μm expected from theory. The reconstruction of a 300 μm×300 μm field of Mucor zygotes (mounted slide from Carolina Biological Supply) is shown in Fig. 3. This reconstruction was “focused” on the large zygote. The three smaller ones to the right are out of focus, and the small one to the left is actually below the large one and severely out of focus.
The setup was then transformed as explained in Section 5 to create a scanning pattern resulting from the interference of two spherical waves of opposite curvatures, as discussed in Section 4. All other parameters including the effective numerical aperture of the objective are unchanged. From Section 4, we expect an encoding pattern with the same size and a Fresnel number twice as large. This is indeed confirmed by the phase of the hologram of the 1-μm pinhole shown in Fig. 4(a). The encoding pattern has a Fresnel number F≈24 as expected and a diameter 2a75 μm. The autocorrelation of this pattern, after amplitude equalization, is shown in Fig. 4(b). This should again be a fair representation of the system's experimental PSF. The FWHM of the autocorrelation is Δ ≈1.1 μm and compares well with the theoretically expected resolution Δx≈1 μm. This represents an improvement of a factor of 2 compared with the Rayleigh limit of the objective.
Figure 5 shows the reconstruction of the same 300 μm×300 μm field of Mucor zygotes. The gain of resolution beyond the Rayleigh limit of the objective is readily observable by comparison of Figs. Figs.55 and and3,3, which were obtained with the exact same objective. Another striking difference between Figs. Figs.33 and and55 is the extended DOF of Fig. 5. From Section 4, the DOF of the reconstruction is expected to be larger by a faction compared with the DOF of the objective, which was ≈16 μm. With a DOF of ≈80 μm, not only the big zygote on which the reconstruction was “focused” is sharp, but the three smaller ones on the right side are also in focus. A useful feature of the method is that the DOF can actually be adjusted to match a certain specimen by varying the Fresnel number of the encoding pattern without changing its numerical aperture. In that way, the DOF can be changed without affecting the transverse resolution.
We have shown that the method of scanning holographic microscopy is capable of producing reconstructed images comparable to wide-field or conventional scanning microscopy. The potential advantages of scanning holographic microscopy include the capture of 3D information in a single 2D scan, the simultaneous recording of multifunctional data (including, for example, fluorescence contrast of 3D specimens and quantitative phase contrast), and the possibility of synthesizing the PSF of the system by using two-pupil-interaction methods. With modern spatial light modulator technology, it becomes possible to synthesize and vary dynamically arbitrary bipolar PSFs adapted to a particular imaging need.
As a simple but significant example, we have shown that the transverse resolution of scanning holographic microscopy is not necessarily constrained by the Rayleigh limit of the objective, but can be improved up to a factor of 2 in the limit of low numerical apertures. In the case demonstrated here, the reconstructed images also exhibit an extended DOF that can be adapted to match a particular specimen without compromising the gain in transverse resolution beyond the Rayleigh limit. These attributes, which are not readily achievable with conventional microscopy, can be of importance in a number of biological applications.
This work was supported by National Institutes of Health grant No. 5 R21 RR018440-02.