Home | About | Journals | Submit | Contact Us | Français |

**|**Wiley-Blackwell Online Open**|**PMC3447539

Formats

Article sections

- Abstract
- Introduction
- Path length difference estimation
- OTF separation and synthesis
- Results
- Discussion
- References

Authors

Related links

Journal of Microscopy

J Microsc. 2012 June; 246(3): 229–236.

PMCID: PMC3447539

NIHMSID: NIHMS387745

Correspondence to: John W. Sedat, Department of Biochemistry and Biophysics, University of California, San Francisco, CA 94158, U.S.A. Tel: +1-415-476-4156; fax: +1-415-514-4242; e-mail: ude.fscu.gsm@tades

Received 2011 August 2; Accepted 2012 January 30.

Copyright © 2012 The Authors Journal of Microscopy © 2012 Wadsworth Center, New York State Department of Health

Re-use of this article is permitted in accordance with the Creative Commons Deed, Attribution 2.5, which does not permit commercial exploitation.

In an interferometer-based fluorescence microscope, a beam splitter is often used to combine two emission wavefronts interferometrically. There are two perpendicular paths along which the interference fringes can propagate and normally only one is used for imaging. However, the other path also contains useful information. Here we introduced a second camera to our interferometer-based three-dimensional structured-illumination microscope (I^{5}S) to capture the fringes along the normally unused path, which are out of phase by π relative to the fringes along the other path. Based on this complementary phase relationship and the well-defined phase interrelationships among the I^{5}S data components, we can deduce and then computationally eliminate the path length errors within the interferometer loop using the simultaneously recorded fringes along the two imaging paths. This self-correction capability can greatly relax the requirement for eliminating the path length differences before and maintaining that status during each imaging session, which are practically challenging tasks. Experimental data is shown to support the theory.

We reported earlier a resolution-enhancing scheme for three-dimensional (3D) widefield fluorescence microscope, I^{5}S (Shao *et al.*, 2008), that greatly improves the resolution of optical microscopy to nearly 100 nm in all three dimensions. It combines structured-illumination microscopy (SIM; Heintzmann & Cremer, 1999; Gustafsson *et al.*, 2000; Gustafsson, 2000; Gustafsson *et al.*, 2008) with an interferometer using two opposing objective lenses (Hell & Stelzer, 1992; Gustafsson *et al.*, 1999). In SIM, the sample is illuminated by a periodic pattern, which mixes with the otherwise undetectable high-resolution information and, in spatial frequency space, moves it into the passband of the microscope in the form of moiré fringes. The down translated high-resolution information is then computationally recovered to produce an image with extended resolution (Heintzmann & Cremer, 1999; Gustafsson *et al.*, 2000; Gustafsson, 2000; Gustafsson *et al.*, 2008). SIM can double the conventional resolution in all three dimensions, resulting in a resolution of about 100 nm laterally and 300 nm axially (Gustafsson *et al.*, 2008). I^{5}S is an extension of SIM in which a second objective lens is used and placed on the opposite side of the sample, and the two apertures are combined interferometrically (Hell & Stelzer, 1992; Gustafsson *et al.*, 1999; Shao *et al.*, 2008). As a result, the axial resolution is greatly improved both because the set of light-gathering angles of the microscope is enlarged and because the highest axial spatial frequencies in the illumination pattern are increased due to the interference of the illumination beams coming from the two opposing objectives. An isotropic 100-nm-scale resolution is achieved by this technique in all dimensions (Shao *et al.*, 2008).

In I^{5}S, a collimated laser illumination beam is diffracted by a transmission phase grating and the central three diffraction orders are split by a beam splitter, resulting in six mutually coherent illumination beams being incident on the sample, one triplet of beams passing through each objective lens (Fig. 1). The illumination pattern that results from the interference of these six beams consists of 19 frequency components (Fig. 2D) and can be expressed as (Shao *et al.*, 2008)

The schematic drawing of an I^{5}S microscope. The illumination light passes first through a transmission grating, which diffracts it into three beams (green lines), and then through a beam splitter, which splits each beam and directs three beams to each **...**

Principles of I^{5}S explained in frequency space. (A) The generalized pupil function (GPF) of an I^{5}S/I^{5}M microscope, i.e. the portions of the wavefront detectable through the two objective lenses. In a conventional widefield microscope, only one of the **...**

(1)

where **r**_{xy} represents the lateral coordinates (x, y), **p** is the lateral component of the wave vector of each beam not coincident with the optical axis *k*_{z}(or the side beams), *I _{m}*(

(2)

where is the sample information, *O _{m}* (Figs 3A–C) is the

The raw data of SIM, including I^{5}S, is the summation of five components (Eq. (2)). These components can be separated given a series of images taken with five different lateral pattern's phases ( in Eqs (1) and (2); Gustafsson *et al.*, 2008; Shao *et al.*, 2008). Because the lateral pattern in our implementation is one-dimensional (1D), the same pattern needs to be applied multiple times with different angles (i.e. **p** in Eqs (1) and (2) with the same magnitude but different orientations) so as to isotropically extend the lateral resolution. A final image with 100 nm resolution in all three dimensions (Fig. 2E) is reconstructed by positioning all the separated information components back to their original locations in frequency space and then correcting for amplitude differences via a generalized Wiener filter (Gustafsson *et al.*, 2008; Shao *et al.*, 2008).

For I^{5}S reconstruction to work, it is required that the raw image is acquired under the same conditions as the measured effective OTFs. In I^{5}S or any interferometric set-up, one condition prone to variation in different imaging sessions is the path length difference of the interferometer loop. Before every experiment, significant time must be spent on digitally adjusting this path length difference to reach zero, which gives the maximum axial fringe contrast. Even if this preimaging adjustment is performed very accurately, instrumental drift will have caused different path length differences, i.e. different imaging conditions, in the actual data set and in the data set taken for the OTF determination, resulting in axially ringing artefacts in the reconstruction because of the unmatched OTF used. To make I^{5}S easier to use, we need a scheme that can relax this very stringent requirement on path length difference consistency without inducing artefacts.

Another disadvantage of the current I^{5}S set-up is that only the imaging interference fringes at one of the beam splitter's two exit ports are recorded, meaning half of the emission light is discarded. Capturing both halves of the emission light simultaneously would significantly improve the signal-to-noise ratio. Furthermore, because of the complementary phase relationship between the fringes at the two beam splitter exits (to be discussed next), the actual path length difference within the interferometer loop can be estimated from a pair of 3D images simultaneously acquired through the two exits. With this quantity known, one can then either choose a matching set of OTFs from a library of OTFs or, in a more attractive solution, synthesize a set of OTFs that can account for the estimated path length difference, and use those to reconstruct the data set. As a result, the requirement for achieving a specific path length difference for each experiment can be relaxed.

First, we discuss how the path length difference can be estimated from an I^{5}S data set. Given a known path length difference δ, the detection OTF of I^{5}S can be divided into three regions based on different phases: *b*, 0 and −*b*, where *b*= 2*πδ*/λ_{em} and λ_{em} is the emission wavelength (Fig. 4A). These values are because of the fact that the detection OTF is an autocorrelation of the generalized pupil function (GPF; Fig. 2A) and the relative phase difference between the two shells of GPF (i.e. the two wavefronts collected separately by the two objective lenses) is *b*. Likewise for the illumination components (Fig. 4B), three groups can be distinguished based on the components’ phases: *a*, 0 and −*a*, where *a*= 2*πδ*/λ_{ex} and λ_{ex} is the excitation wavelength. Because the effective I^{5}S OTF *O _{m}* is the convolution of the detection OTF with the

Because the *m*th-order component of the I^{5}S raw data is a subregion of the sample's Fourier transform—corresponding to the support of *O _{m}* shifted by

(3)

where and are OTFs that are acquired with zero path length difference and is the *m*th-order component separated from Eq. (2) (the zero path length difference requirement on can be satisfied even for OTFs acquired with nonzero path length difference, to be discussed in *OTF separation and synthesis*). Linear regression can then be applied to solve Eq. (3) for *a* using all the frequencies within the overlap. This seems to be a straightforward method to estimate the path length difference, except that it requires a clean overlap between only the two central band-derived regions of the *m*= 0 and 1 components pointed to by the arrows in Figure 5. Unfortunately, superimposed on those two central band-derived regions are other signals derived from the sidebands of the detection OTF (Figs 3 and 4C–E), which make Eq. (3) invalid. Hence, we first need a solution for obtaining the uncontaminated overlaps as depicted in Figure 5 before the path length difference can be estimated using Eq. (3).

It can be proved (see Appendix) that in I^{5}S, when the two emission wavefronts collected from the two objective lenses are coherently combined at the beam splitter, the phase of the interference fringes at the primary exit port (i.e. the illumination entry port, Fig. 1) of the beam splitter is exactly π different than that at the other exit port (the secondary exit). This results in the following relation between the detection OTFs acquired at the primary and the secondary exits: when the phases are ±*b* in the sidebands of the former, the phases are ±(*b*+π) in the sidebands of the latter. Then if I^{5}S effective OTFs are acquired through the two exits, the same π phase difference exists between the two sets of OTFs in those regions because of the convolution of the illumination components with the sidebands of the detection OTF. Furthermore, as an image in frequency space is a multiplication of the object's Fourier transform by the OTF, this π phase difference in those sideband-derived regions is also there between two I^{5}S data sets acquired simultaneously through the two exits for any sample. Consequently, the addition (or subtraction) of these two data sets results in only the central (or side) band-derived regions being nonzero in frequency space, provided that correct intensity normalization between the two images has been applied. If the addition image is used for extracting the overlap shown in Figure 5, the overlap would not be contaminated by the superimposed sideband-derived signals and therefore, the path length difference can be cleanly estimated as described in Eq. (3).

To implement the above idea, a second imaging path was added to I^{5}S to capture the interference fringes through the secondary exit of the beam splitter (CCD 2 in Fig. 1). A pair of I^{5}S point spread functions (PSF) was acquired simultaneously by the two CCDs and the rotationally averaged effective OTFs were generated (Gustafsson *et al.*, 2008; Shao *et al.*, 2008). The summations and subtractions of these two sets of OTFs are shown in Figure 6, which, as expected, contain mostly the subregions derived only from the central band and sidebands of the detection OTF, respectively.

The effective I^{5}S OTFs were acquired through the primary and secondary beam splitter exit and rotationally averaged, and their additions and subtractions, for *m*= 0, ±1 and ±2, are shown in (A)–(C) and (D)–(E), respectively. **...**

To estimate the path length difference in each data set, a pair of raw data sets was acquired simultaneously by the two cameras. The image pair was then computationally aligned with each other to compensate the residual misalignment between the imaging paths. The misalignment, mainly because of lateral shift, rotation and magnification, only needs to be calibrated once every few months using scattered beads samples. The result of the calibration, a two-dimensional affine transform matrix, was saved for aligning the image pairs in subsequent experiments. The intensity ratio between the image pair after background subtraction was also calibrated and saved for future use. After alignment and intensity correction, each raw image was separated into the five information components (Eq. (2)), each corresponding to a different *m* of Eq. (2) (Shao *et al.*, 2008), and the *m*= 0 and 1 components of the images acquired by one camera were added to their counterparts acquired by the other, resulting in the new *m*= 0 and 1 components with little sideband-derived contamination. The overlap between the red subregion of the *m*= 0 component and the green subregion of the *m*= 1 component (Fig. 5) was then extracted and used for the estimation of the path length difference based on Eq. (3).

Given the estimated path length difference, we still need a scheme for image reconstruction to take advantage of it. Here we introduce such a scheme based on the fact that I^{5}S effective OTFs are the convolution of the detection OTF *O* with the 1D axial illumination structures (Eq. (2)). Suppose a measured I^{5}S OTF can be decomposed into *O* and . Once the path length difference is estimated from a raw image, then it is trivial to modify the phases of *O* and by multiplying to the components of nonzero phases (Fig. 4B) and to the sidebands of *O*(Fig. 4A). The phase-modulated *O* and can then be convolved together to synthesize a set of OTFs that can account for the path length difference specific to the raw image.

To separate I^{5}S OTFs into two convolving parts, one of the parts has to be known. There is a way to measure the detection OTF and I^{5}S effective OTFs almost simultaneously as long as the mirror M3 in Figure 1 also transmits partially (20% is what we used) so that a conventional illumination path can be introduced through that mirror. With this new set-up (Fig. 1), the 3D I^{5}S effective PSF and detection PSF can be acquired in an interleaved manner by alternating between the conventional and the structured-illumination path at each defocus step. We can then obtain rotationally averaged I^{5}S effective OTFs and detection OTF from these PSFs.

The OTF separation problem is formulated as solving for in given *O _{m}*(

Figure 7 shows the results of separating I^{5}S illumination structures from the measured I^{5}S effective OTFs as discussed above. The three curves in Figure 7 are the amplitudes of I^{5}S illumination structure versus *k _{z}* in frequency space for

I^{5}S illumination structure axial profiles in frequency space, obtained by deconvolving measured I^{5}S detection OTF from the I^{5}S effective OTFs. Black, red and blue curves correspond to order *m*= 0, ±1 and ±2 components, respectively.

Two pairs of dual-camera I^{5}S images of a single-bead sample were acquired. Nonzero path length differences were intentionally introduced when acquiring these data sets. The reconstructions using a standard set of effective OTFs taken with little path length difference present show the expected ringing artefacts in the axial intensity profile through the bead centre (Figs 8A and andC).C). The different polarities of the asymmetry in those plots indicate either positive or negative path length difference in the loop. For each data set, we performed phase difference estimation (the results are 0.15 and –0.9 radians for data set shown in Figs 8A and andC,C, respectively) and data set-specific OTF synthesis as described above, and then reconstructed the images acquired through the primary beam splitter exit using the synthesized OTFs. The results are also displayed as axial intensity profiles in Figures 8B and andD,D, which exhibit much less ringing than Figures 8A and andCC.

We introduced a method to estimate the path length difference present in each I^{5}S data set with the assistance of a second camera recording fluorescence emission's interference fringes through the secondary exit of the beam splitter. The estimation method exploits the π phase difference relationship between the imaging interference fringes at the primary and secondary beam splitter exits and the well-defined Fourier space phase maps for all the information components constituting an I^{5}S data set. Another way to think about the π phase difference is to recall that when the primary beam splitter exit has constructive interferences information, the new secondary beam splitter exit will have destructive interferences information. With the estimated phase difference, a data set specific set of OTFs can then be synthesized, because I^{5}S effective OTFs are separable into a detection OTF convolving with the axial illumination structures, and used for reconstructing the data set. We have used a single bead imaging experiment to prove the principle. Reconstruction using a synthesized, as opposed to a standard, set of OTFs successfully removes most of the axial ringing artefacts that are because of an unknown path length difference. One major difficulty of I^{5}S or any other interferometric microscope is the strict requirement on path length difference adjustment. Relaxing this requirement will significantly enhance the real-world applicability of I^{5}S to biology. The basic principle demonstrated in this paper lays the groundwork for a scheme that can achieve such a goal. It does not, however, affect other factors limiting the application of I^{5}S to biology, including the requirement of refractive index matching among the mounting medium, the immersion medium and the sample itself, and the requirement that the two objective lenses are closely matched in their optical properties. In our demonstration, the fluorescent bead was mounted in the same medium as the high refractive index (1.51) immersion medium.

As mentioned earlier, the image pair recorded on two cameras can also be used simply for improving the signal to noise. This can be accomplished by including in the generalized Wiener filter (Gustafsson *et al.*, 2008) the additional terms corresponding to the images and OTFs acquired by the second camera once the image pair is aligned.

We dedicate this paper to the late Dr. Mats Gustafsson. This work was supported in part by the Keck Laboratory for Advanced Microscopy, the National Institutes of Health (GM25201 to J.W.S. and GM31627 to D.A.A.), the Howard Hughes Medical Institute (D.A.A.) and by the National Science Foundation through the Center for Biophotonics, an NSF Science and Technology Center that is managed by the University of California, Davis, under Cooperative Agreement No. PHY 0120999.

Here we prove the π phase difference between the interference fringes at the primary and secondary exit port of the beam splitter. Two parts of a wavefront with intensities *I*_{1} and *I*_{2} combine at the beam splitter after traversing the two arms of the interferometer. Each of them is split into halves by the beam splitter, with one-half transmitting through and the other half reflected by the beam splitter (Fig. 9). The reflected half of wavefront 1 either has phase 1 +π, where 1 is the phase before the split, if the reflection occurs at the glass–glue interface, or 1 + 2θ if the reflection occurs at the glue–glass interface according to the Stokes relations (Hecht, 1998, Chapter 4.10), where θ is the phase delay because of the glue. The reflected half of wavefront 1 interferes at the primary exit with the transmitted half of wavefront 2, whose phase is 2 +θ, and thus the interference intensity is (Born & Wolf, 1980, Chapter 7.2) either *I*_{p}=*I*_{1}/2 +*I*_{2}/2 + (*I*_{1}*I*_{2})^{1/2}·cos(1 +π–2 –θ), denoted Case 1a, or *I*_{p}=*I*_{1}/2 +*I*_{2}/2 + (*I*_{1}*I*_{2})^{1/2}·cos(1 +θ–2), denoted as Case 1b. Likewise, the reflected half of wavefront 2 either has phase 2 +π or 2 + 2θ, depending on where the reflection occurs, and when it interferes with the transmitted half of wavefront 1 at the secondary exit, whose phase is 1 +θ, the interference intensity is either *I*_{s}=*I*_{1}/2 +*I*_{2}/2 + (*I*_{1}*I*_{2})^{1/2}·cos(1 +θ–2 –π), denoted Case 2a, or *I*_{s}=*I*_{1}/2 +*I*_{2}/2 + (*I*_{1}*I*_{2})^{1/2}·cos(1 –2 –θ), denoted Case 2b. Because of energy conservation, i.e. that *I*_{1}+*I*_{2} must equal *I*_{p}+*I*_{s}, it is mandatory that Case 1a and 2b or Case 1b and 2a co-occur so that the cos(·) terms cancel out. In either case, it means the phases of the interference fringes at the primary and secondary exit port are different by π.

- Bertero M, Boccacci P.
*Introduction to Inverse Problems in Imaging*. Philadelphia, PA: Institute of Physics Publishing; 1998. - Born M, Wolf E.
*Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light*. 6th edn. Oxford: Pergamon Press; 1980. - Gustafsson MG. Surpassing the lateral resolution limit by a factor of two using structured-illumination microscopy. J. Microsc. 2000;198:82–87. [PubMed]
- Gustafsson MG, Agard DA, Sedat JW. I
^{5}M: 3D widefield light microscopy with better than 100 nm axial resolution. J. Microsc. 1999;195:10–16. [PubMed] - Gustafsson MG, Shao L, Carlton PM, et al. Three-dimensional resolution doubling in widefield fluorescence microscopy by structured-illumination. Biophys. J. 2008;94:4957–4970. [PubMed]
- Gustafsson MGL, Agard DA, Sedat JW. Doubling the lateral resolution of wide-field fluorescence microscopy using structured-illumination. Proc. SPIE. 2000;3919:141–150.
- Hecht E.
*Optics*. 3rd edn. Reading, MA: Addison-Wesley; 1998. - Heintzmann R, Cremer C. Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating. Proc. SPIE. 1999;3568:185–196.
- Hell S, Stelzer EHK. Properties of a 4pi confocal fluorescence microscope. J. Opt. Soc. Am. A. 1992;9:2159–2166.
- Shao L, Isaac B, Uzawa S, Agard DA, Sedat JW, Gustafsson MG. I
^{5}S: widefield light microscopy with 100-nm-scale resolution in three dimensions. Biophys. J. 2008;94:4971–4983. [PubMed]

Articles from Wiley-Blackwell Online Open are provided here courtesy of **Wiley-Blackwell, John Wiley & Sons**

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |