PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Phys Rev Lett. Author manuscript; available in PMC Dec 13, 2012.
Published in final edited form as:
Published online Oct 17, 2012.
PMCID: PMC3521605
NIHMSID: NIHMS419101
Unified Resolution Bounds for Conventional and Stochastic Localization Fluorescence Microscopy
Eran A. Mukamel1,3* and Mark J. Schnitzer2,4
1Department of Physics, Stanford University, Stanford, California 94305, USA
2Departments of Applied Physics and Biology, Stanford University, Stanford, California 94305, USA
3Center for Brain Science, Harvard University, Cambridge, Massachusetts 02138, USA; Center for Theoretical Biological Physics, University of California, San Diego, California 92093, USA
4Howard Hughes Medical Institute, Stanford University, Stanford, California 94305, USA
*eran/at/post.harvard.edu
mschnitz/at/stanford.edu
Superresolution microscopy enables imaging in the optical far field with ~20 nm-scale resolution. However, classical concepts of resolution using point-spread and modulation-transfer functions fail to describe the physical limits of superresolution techniques based on stochastic localization of single emitters. Prior treatments of stochastic localization microscopy have defined how accurately a single emitter’s position can be determined, but these bounds are restricted to sparse emitters, do not describe conventional microscopy, and fail to provide unified concepts of resolution for all optical methods. Here we introduce a measure of resolution, the information transfer function (ITF), that gives physical limits for conventional and stochastic localization techniques. The ITF bounds the accuracy of image determination as a function of spatial frequency and for conventional microscopy is proportional to the square of the modulation-transfer function. We use the ITF to describe how emitter density and photon counts affect imaging performance across the continuum from conventional to superresolution microscopy, without assuming emitters are sparse. This unified physical description of resolution facilitates experimental choices and designs of image reconstruction algorithms.
Exploring the physical bounds on light microscopy has been a mainstay activity in theoretical optics. In classical [1, 2] and in most modern treatments [3] the limits on resolution have been derived using deterministic field theories for optical diffraction. This ignores photon shot noise, the dominant noise source in perhaps most modern microscopy applications. Digital image processing algorithms that take noise into account can achieve subpixel superresolution by combining multiple conventional images of a single sample or scene [4, 5]. However, the poor performance of these techniques in recovering high spatial frequency information led to the recognition that prior information about image structure is generally essential for attaining superresolution [6]. Yet, existing theories do not provide physical limits on image estimation based on the noise and prior information available from specific measurement schemes.
Here we study the physics of superresolution imaging by stochastic localization of single molecules, as in photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM), which achieve superresolution via the spatial localization of single fluorescent molecules [710]. We stress that stochastic localization is a distinct measurement and computational analysis procedure that exploits the prior knowledge the active fluorophores are dilute. Previous theories of superresolution from digital image processing do not describe the specific form of prior information that has allowed stochastic localization techniques to successfully achieve tenfold or better superresolution beyond the conventional diffraction limit. Our theory provides unified performance limits for stochastic localization, conventional microscopy, and the entire class of digital image processing algorithms for subpixel superresolution.
Our work builds on earlier applications of estimation theory that amended classical notions of resolution by including fluctuations in the propagating fields [11, 12]. Instead, we consider photon statistics and model the stochastic activation and measurement of individual molecules. Our resolution measure, termed here the information transfer function (ITF), limits how well one can determine a specimen’s features at each spatial frequency. The ITF thus replaces the modulation-transfer function (MTF) but with an expanded range of explanation, covering conventional and stochastic localization microscopy. Importantly, for conventional microscopy the ITF reduces to the square of the MTF, scaled by the number of observed photons, thereby preserving physicists’ basic intuitions about imaging. Overall, our formalism unifies long-standing work on the spatial filtering properties of optical imaging with recent efforts to describe the limits imposed by photon shot noise.
We consider an image taken from a specimen of m fluorophores, all of the same color and located in the sample plane at {xj}. The experimental data are the counts, {Nl}, of photons detected at pixel l which is located in the image plane at yl, l = 1…L. The probability, P, of observing the data is
equation M1
(1)
where pj is the intensity of the jth emitter, Δyl is the pixel area, and the point-spread function (PSF), h, is normalized, equation M2. We idealize the detector by assuming uniformly distributed pixels with infinitesimal area, Δyl → 0, so thatNl [set membership] {0,1}. This neglects the effect of finite pixel size. In this limit we can rewrite Eq. (1) as a function of the locations in the image plane, {zk}, of N = ∑lNl detected photons: P({Nl}|{xj}) = P({zk}|N, {xj})P(N), where P(N) is the probability that N photons are detected, and
equation M3
(2)
This satisfies ∫P({zk}|N, {xj})dz1dzN = 1. Localization microscopy finds estimates, {x̂j}, of {xj} based on the observations, {zk}.
To study the limits of this technique, we use a key tool from estimation theory, the Cramér-Rao lower bound [13] (CRLB), which bounds the accuracy of any estimator, {qi}, of a set of parameters, {qi}: [E − J−1] ≥ 0. Here Eij [equivalent] left angle bracket(qiqi)(qjqj)*right angle bracket is the estimator covariance matrix, equation M4 is the Fisher information matrix, and qi is assumed unbiased, i.e., left angle bracketqiright angle bracket = qi. The angular brackets, left angle bracket···right angle bracket, denote averages over the stochastic locations of detected photons, distributed according to Eq. (2). The CRLB states [EJ−1] is positive semidefinite, implying that the diagonal elements (the mean squared errors) obey Eii = left angle bracket|qiqi|2right angle bracket ≥ [J−1]ii. This inequality bounds estimator accuracy for any set of image parameters, such as the emitters’ locations (qi [equivalent] xi) [14, 15], or, as in our theory, the real-space [qx [equivalent] I(x)] or spatial frequency [qk [equivalent] I(k)] components.
Previous applications of the CRLB to emitter localization did not yield limits on how well entire images can be determined, nor formalism to supersede the PSF and MTF. To illustrate, consider a 2D scene composed of two equally bright emitters [14]. We choose a coordinate system in which the emitters lie on the x axis at (x, y) = (±dx, 0). J is diagonal in the basis given by the mean position and separation vector [Fig. 1(a), inset]. Figure 1(a) shows the corresponding CRLBs in units of the width of the Gaussian PSF, σ [16]. Similar results hold for an Airy disk PSF, and when the two emitters are not equally bright [16]. If dx [dbl greater-than sign] σ, Jequation M5 for each coordinate. When dx [double less-than sign] σ, one cannot unambiguously assign each photon to its correct emitter and Jdy,dy, Jdx,dx both vanish. This is why in stochastic localization microscopy one typically uses a very dilute density of simultaneously active emitters, ρa [double less-than sign] σ−2, to prevent emitters’ images overlapping.
FIG. 1
FIG. 1
Real-space analysis of localization accuracy depends on the configuration of emitters. (a) A scene with two emitters has 4 degrees of freedom (inset). The Cramér-Rao lower bound (CRLB) for estimation of each of the 4 coordinates (solid lines; (more ...)
This analysis extends to scenes of more emitters [Figs. 1(b)–1(d)], but in the general case localization accuracy varies with the scene’s details. The Fisher information for m emitters is a 2m × 2m matrix whose entries depend on the emitters’ 2m position coordinates. For m > 2 there is no general coordinate system diagonalizing J. A resolution measure that varies from sample to sample is inconsistent with traditional concepts of resolution and has limited practical utility.
We introduce a measure, the ITF, that gives resolution bounds for all specimens and generalizes the MTF to account for photon statistics. The ITF provides the CRLB for image estimation as a function of spatial frequency; this conception of resolution applies equally well to conventional or stochastic localization microscopy.
Consider an image scene, represented by a positive intensity function, I(x), normalized to have unit integral. In biological microscopy, I may represent the spatial probability density of fluorescently labeled cellular or protein structures. We seek a bound on the squared error of an unbiased estimator, Î(k). The ITF, F(k), bounds the error in estimating the image frequency components: EI(k) [equivalent] left angle bracket|Î(k) − I(k)|2right angle bracket ≥ 1/F(k). (In practice it may be impossible to define unbiased estimators for some image components, especially at high spatial frequencies [16]. A generalization of the CRLB relates F(k) to the minimum variance of a biased estimator [13]. Even when no unbiased estimator exists, the ITF provides an objective figure of merit for image resolution that is based on the measurement statistics.)
Physical intuition suggests the ITF should vanish at large k and be proportional to N, since each photon carries equal Fisher information. We first calculate the ITF exactly using three key simplifying assumptions. First, we ignore photons that cannot be assigned to an emitter due to overlap; the case of overlapping emitters will be considered below. This assumption simplifies the probability, Eq. (2). If nj is the number of detected photons from the jth emitter, and assuming the fluorophores’ locations are randomly sampled from the distribution, I(x), we have [16]:
equation M6
(3)
Second, assuming the PSF, h, is Gaussian, we can compute the integral:
equation M7
(4)
where equation M8 is the centroid of the observed photons’ locations, and Ĩnj (x̂j) [equivalent]dxjI(xj)e−(nj/2σ2)(xjx̂j)2 (nj/2πσ2) is a blurred version of the true image with blurring scale equation M9 [16]. Third, we assume each emitter emits the same number of photons, nj = n; numerical results show that a more realistic calculation with Poisson-distributed nj has a nearly identical result [16]. We find
equation M10
(5)
Using the normalization R of the image and the PSF, ∫dx̂Ĩn(x̂) = 1, we invert the Fisher information matrix to compute the ITF:
equation M11
(6)
where A is the field of view area and ρe is the total density of emitters; see Fig. 2(a). (Note ρe appearing here is the density of all emitters observed in the experiment, whereas ρa introduced above is the density of simultaneously activated emitters in a single time frame.)
FIG. 2
FIG. 2
The information transfer function, F(k), bounds image estimation accuracy. (a) F0(k) for conventional imaging (n = 1 photon/emitter; blue and red curves) and superresolution (n = 100, green and cyan). (b) Mean squared localization error for a truncated (more ...)
This result is derived more explicitly in [16] and shows that the ITF is an image-independent, universal limit valid for any scene. Equation (6) shows that superresolution hinges on the ability to have n [dbl greater-than sign] 1; in practice n ~ 100−104 or more. In conventional microscopy, individual emitters may contribute more than one photon to the image, but since photons cannot be reliably assigned to the correct emitter their statistics are equivalent to the case n = 1. Increasing the emitter labeling density, ρe, reduces the error variance at all spatial frequencies but has little effect on the threshold frequency at which the ITF becomes vanishingly small [Fig. 2(a)].
We can also compute the ITF exactly, with no restriction on the PSF, for conventional microscopy (n = 1). In this case Eq. (3) factorizes into a product of independent contributions from each photon, and the ITF is [16]: F(k) = N|h(k)|2. This result connects the ITF directly to the conventional MTF.
Both of these exact results are special cases of our main finding, which is a general bound on the ITF for any PSF. If the MTF is bounded by a Gaussian, |h(k)| ≤ e−(2πkσ)2/2, then we find [16]:
equation M12
(7)
where C is a constant that depends on the image but not on k, A or ρe.
We now consider how the ITF applies in the case the emitters’ images overlap, which is common when the density of active emitters, ρa, approaches σ−2. Several computational procedures have recently been proposed for analyzing data from such crowded scenes to speed up superresolution image acquisition, but the fundamental limits on resolution and speed remain unknown [1721]. We show that the spatial frequency-dependent ITF is generally applicable and smoothly depends on ρa.
To derive how the ITF depends on ρa, we must specialize to images consisting of m emitters. (More generally, the following model applies to images of M emitters obtained in M/m distinct rounds with exactly m active emitters per round. Each round contributes independent information, so the total ITF is the sum of the ITF for each.) The image is equation M13 in real space, or equation M14 in the frequency domain. We can compute F(k) through its relation to the real-space Fisher information matrix [13, 16]: 1/F(k) [equivalent] equation M15. Conventional microscopy corresponds to ρa [dbl greater-than sign] σ−2. In this limit we may approximate the Poisson statistics of photon shot noise as additive Gaussian fluctuations in intensity, valid when many photons are detected per pixel [16]. The result matches the form derived above for general images when n = 1 photon is observed per emitter [Fig. 3(a), solid black curve].
FIG. 3
FIG. 3
ITF for images of discrete emitters reveals the optimal density. (a) ITF for a fixed number of photons, N, with an Airy disk PSF. The Fisher information about emitter location for an isolated emitter is J(1) = σ−2 per photon, and N = (more ...)
We also reconsider superresolution microscopy with sparse emitters, ρa [double less-than sign] σ−2. Here the Fisher information matrix is diagonal, Jij = nJ(1)δij, where J(1) is the Fisher information per photon for localizing a single emitter [16]. The ITF, equation M16, declines only quadratically with k [Fig. 3(a), black dot-dashed line]. Prior knowledge that the image structure consists of m discrete emitters improves estimation as compared with a completely general image. Hence, the ITF here is strictly greater than the previous bound, Eq. (7), derived for a general image with n > 1. However, in both cases, the highest resolvable spatial frequency is proportional to equation M17.
Besides providing a unified formalism for these two extremes, the ITF can quantify the trade-off between image resolution and acquisition time as a function of the density of activated emitters. To do this, we computed the ITF for simulated specimens with emitters at a range of densities. The ITF smoothly transitions between the low and high emitter density regimes, Fig. 3(a). In live-cell imaging, the time to acquire an image, T, is a constrained resource. For fixed T, the number of detected photons scales asN [proportional, variant] BρaT, where B is the average emitter brightness. There is thus a trade-off between achieving superresolution, by working at low ρa, and increasing the signal-to-noise ratio by collecting more photons [Figs. 3(b) and 3(c)]. Anextremely low density of emitters is suboptimal, with F(k) falling off as ρa (blue curve) for frequencies above the diffraction limit (vertical dashed line). Our simulations reveal the optimal active emitter density, equation M18, that maximizes the ITF for each k, Fig. 3(d). The sum of the ITF over all frequencies greater than the conventional diffraction limit, Fig. 3(e), is maximum around equation M19, which provides sufficient separability for localizing neighboring emitters without wasting time by overly diluting the emitters.
In summary, we used estimation theory to calculate the ITF, a unified bound on image estimation accuracy, for stochastic localization and conventional fluorescence microscopy. For completely general images, the key parameter is the number of observed photons per emitter, n. In conventional microscopy (n = 1) the ITF reduces to the square of the MTF times the total number of observed photons. This result provides a performance bound for superresolution techniques that combine multiple, shifted images or photographs [4, 5]. Such techniques can overcome undersampling due to finite pixels, but the fundamental resolution limit imposed by optical diffraction is described by the ITF for conventional microscopy [6].
Overall, the ITF formalism reveals fundamental resolution limits on fluorescence microscopy, unifying the conventional concepts of resolution with those developed separately for stochastic localization microscopy. The ITF also fulfills the long-standing goal of characterizing how resolution depends on the unavoidable limitations imposed by photon statistics [3, 11, 12]. Further, the ITF facilitates principled choices of experimental parameters that optimize imaging performance at any chosen length scale. Although improved localization and image estimation algorithms will surely continue to emerge [1721], our work already provides the fundamental physical limits on the possible improvements. The ITF bounds the performance of any estimation procedure in fluorescence microscopy, regardless of emitter density.
Supplementary Material
Supplementary Info
Acknowledgments
The authors would like to thank Morteza Shahram, Haim Sompolinsky, and Markus Meister for helpful discussions, as well as the Swartz Foundation (E. A. M.) and an NIH Director’s Pioneer Award (M. J. S.) for funding.
1. Abbe E. Archiv für Mikroskopische Anatomie. 1873;9:413.
2. Rayleigh L, Strutt JW. Philos. Mag. 1879;8:261.
3. den Dekker A, van den Bos A. J. Opt.Soc. Am. A. 1997;14:547. [PubMed]
4. Robinson D, Milanfar P. IEEE Trans. Image Process. 2006;15:1413. [PubMed]
5. Farsiu S, Robinson D, Elad M, Milanfar P. Int. J. Imaging Syst. Technol. 2004;14:47.
6. Baker S, Kanade T. IEEE Trans. Pattern Anal. Mach. Intell. 2002;24:1167.
7. Betzig E, Patterson GH, Sougrat R, Lindwasser OW, Olenych S, Bonifacino JS, Davidson MW, Lippincott-Schwartz J, Hess HF. Science. 2006;313:1642. [PubMed]
8. Rust MJ, Bates M, Zhuang X. Nat. Methods. 2006;3:793. [PMC free article] [PubMed]
9. Hess ST, Girirajan TPK, Mason MD. Biophys. J. 2006;91:4258. [PubMed]
10. Moerner WE. Proc. Natl. Acad. Sci. U.S.A. 2007;104:12596. [PubMed]
11. Helstrom C. J. Opt. Soc. Am. 1970;60:659.
12. Barrett HH, Denny JL, Wagner RF, Myers KJ. J. Opt. Soc. Am. A. 1995;12:834. [PubMed]
13. Kay SM. Fundamentals of Statistical Signal Processing. Englewood Cliffs, NJ: Prentice-Hall; 1993.
14. Ram S, Ward ES, Ober RJ. Proc. Natl. Acad. Sci. U.S.A. 2006;103:4457. [PubMed]
15. Watkins L, Yang H. Biophys. J. 2004;86:4015. [PubMed]
16. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevLett.109.168102 for supplementary methods and figures.
17. Huang F, Schwartz SL, Byars JM, Lidke KA. Biomed. Opt. Express. 2011;2:1377. [PMC free article] [PubMed]
18. Holden SJ, Uphoff S, Kapanidis AN. Nat. Methods. 2011;8:279. [PubMed]
19. Cox S, Rosten E, Monypenny J, Jovanovic-Talisman T, Burnette DT, Lippincott-Schwartz J, Jones GE, Heintzmann R. Nat. Methods. 2011;9:195. [PMC free article] [PubMed]
20. Burnette DT, Sengupta P, Dai Y, Lippincott-Schwartz J, Kachar B. Proc. Natl. Acad. Sci. U.S.A. 2011;108:21081. [PubMed]
21. Mukamel EA, Babcock H, Zhuang X. Biophys. J. 2012;102:2391. [PubMed]
22. Bates M, Huang B, Dempsey GT, Zhuang X. Science. 2007;317:1749. [PMC free article] [PubMed]