|Home | About | Journals | Submit | Contact Us | Français|
We discuss unique features of lens-free computational imaging tools and report some of their emerging results for wide-field on-chip microscopy, such as the achievement of a numerical aperture (NA) of ~0.8–0.9 across a field of view (FOV) of more than 20 mm2 or an NA of ~0.1 across a FOV of ~18 cm2, which corresponds to an image with more than 1.5 gigapixels. We also discuss the current challenges that these computational on-chip microscopes face, shedding light on their future directions and applications.
Lens-free on-chip imaging refers to using a digital optoelectronic sensor array, such as a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) chip to directly sample the light transmitted through a specimen without the use of any imaging lenses between the object and the sensor planes1–25. The hardware for such an imaging geometry is significantly simpler and much more compact and lightweight than that of conventional lens-based microscopy. In addition, this geometry, as will be detailed later on, can decouple imaging FOV and resolution from each other, creating unique microscopes that can achieve improved resolution and FOV at the same time. The advancements in this type of microscopy are being spearheaded by the development of sensor chips that are continually being improved and introduced into consumer electronics products, particularly cell phones and high-end digital cameras.
For a lens-free on-chip microscope, there are various design choices that one can select from. Leaving the discussion of lens-free fluorescence on-chip imaging techniques26–29 to later sections, in general we can categorize bright-field lens-free microscopes into two main streams: (i) contact-mode shadow imaging–based microscopes18–21 and (ii) diffraction-based lens-free microscopes1–17. The first group of lens-free microscopes is designed to minimize the distance (ideally less than 1 μm) between the sample and the active region of the sensor array (or an aperture array in some cases18,19) so that diffraction can be significantly reduced. Therefore, these contact-mode lens-free optical microscopes sample the transmitted light through the objects that are placed on a sensor array, effectively capturing the shadows of the objects. Under the assumption that optical diffraction within the object body and between the object and the sensor active region can both be ignored, these object shadows represent two-dimensional (2D) images of the specimens. To mitigate pixelation-related artifacts in the digital sampling of these transmission shadow images, earlier designs of such lens-free microscopes used the motion of the specimens within a microfluidic channel so that a smaller effective pixel size could be created from a time sequence of shadow images, thus improving the spatial resolution18,19,21. For stationary or slowly moving samples on a chip, however, shifting of the light source6,7 can be used to digitally control the movements of these lens-free object shadows on the sensor array as a function of the source position and can also lead to the synthesis of higher-resolution shadow images20.
The second category of lens-free microscopes relies on computation (on the basis of, for example, digital holography1–17 or coherent diffractive imaging techniques30–36) to partially undo the effects of diffraction that occur between the object and the detector planes. Therefore, unlike contact-mode shadow-imaging approaches, a sizeable distance between the objects and the sensor chip can be accommodated, which also permits 3D imaging of large sample volumes, where objects at different heights can be simultaneously imaged. In this second group of lens-free microscopes, the scattered light from each object interferes with itself and with the unscattered background light (if it exists) to create an interference pattern, which is then digitally processed to reconstruct an image of the object1–17,37–42.
In this Perspective, we expand on lens-free holographic-microscope designs, some of which use a spatially and temporally coherent light source such as a laser that is filtered by a small pinhole (≤1–2 μm)1–4, whereas others rely on partially coherent illumination provided by, for example, light-emitting diodes (LEDs)5–14,43. We focus on the latter and present the unique features of such partially coherent lens-free optical microscopy tools that operate under unit magnification, in which the sample is ‘on-chip’ (Fig. 1); we report some of the emerging results that they provide for wide-field imaging needs, achieving, for example, an NA of ~0.8–0.9 with a half-pitch resolution of ~300–350 nm across an FOV of >20 mm2 (that is, >5 mm × 4 mm) or an NA of ~0.1 across an FOV of ~18 cm2 (~4.9 cm × 3.7 cm), which corresponds to an image with more than 1.5 billion useful pixels. We also present some of the current challenges that these computational on-chip microscopes face, and we compare different approaches to shed light on their future directions and applications.
In a partially coherent holographic on-chip microscope (Fig. 1), the source can simply be an LED or an array of LEDs5,7,10. In case wavelength tunability is desired, a monochromator can also be used that is coupled to a multimode fiber. The spectral bandwidth of the source can vary from a few nanometers to 20–30 nm depending on the sample-to-detector distance and the resolution requirement of the system. Because the sample plane is close to the detector plane (typically ≤0.1–2 mm), the scattered light rays and the background light can still interfere at the sensor chip even though the temporal coherence lengths of such broadband sources are significantly shorter than those of lasers.
In addition to temporal-coherence requirements, spatial coherence of the illumination is also critical in lens-free holographic microscopy (which is also true for contact imaging, as will be detailed later on). Under unit fringe magnification, because the lens-free holographic shadows of objects do not spread across the entire sensor active area, the spatial coherence diameter that is required at the sampling plane is small5, typically less than 0.5–1 mm. This implies that an LED can be directly coupled to a large-core fiber-optic cable or a simple pinhole (for example, 50–100 μm in diameter) without the use of any mechanical alignment stage or light-coupling optics. This makes alignment and operation of a partially coherent lens-free holographic microscope straightforward. Because coherence is now used as a gating function, this choice of partial coherence, besides offering simplicity of alignment and cost-effectiveness, helps reduce speckle and multiple-reflection-interference noise terms as well as cross-interference among the diffraction patterns of objects, which is in general a source of artifact for holography44.
Apart from the illumination end, the other key component involved in a lens-free microscope is the optoelectronic sensor array that is used to sample the transmitted light pattern from each specimen. Under unit magnification, the imaging FOV of a lens-free holographic microscope and some types of contact-mode microscopes is equal to the active area of the sensor chip, which implies that by using state-of-the-art CCD and CMOS chips (monochrome or color), one can achieve a wide range of FOVs varying from 20 mm2 to more than 15 cm2. These numbers constitute significantly wider imaging areas than are offered by standard objective-lenses used in conventional optical microscopes. Note that microlenses involved in CMOS or CCD sensor arrays do not form an image per lens; instead they affect the photon collection efficiency of individual pixels.
Another key feature is the pixel size, which directly influences the spatial resolution that can be achieved. For both holographic and contact-mode lens-free on-chip imaging techniques, a smaller pixel size will help us achieve better resolution unless the pixel design exhibits severe angular distortions that create aberrations for oblique rays, which correspond to high numerical apertures. Because the area of the pixel shrinks with the square of its size, to claim a large active area or imaging FOV, more megapixels would also be needed for an ideal lens-free on-chip microscope. Fortunately, adding more megapixels to image sensors is already a major trend in consumer electronics, mostly driven by the massive volume of camera-phones, which have recently started to use ~40-megapixel imagers.
Lens-free on-chip imaging has two important reconstruction blocks that are needed to visualize an object’s image. The first of these computational blocks, termed ‘pixel super-resolution’45–47, is used to overcome the resolution limitation due to the pixel size and is required to achieve subpixel spatial resolution in lens-free on-chip imaging6–10. Note that the term ‘pixel super-resolution’ refers to ‘de-aliasing’ that is used to mitigate the unit magnification of on-chip imaging geometry and should not be confused with the recent literature on surpassing the diffraction limit of light. Contact-mode and holographic lens-free microscopes share this common step to digitally embed more pixels to an image by, for example, shifting the light source and capturing multiple (typically ~10–100) subpixel-shifted lens-free images of the same static object6,7. Alternatively, the motion of the object within microfluidic devices can also be used for the same pixel super-resolution step to resolve finer features of an object8,13,21.
The second reconstruction block, which follows the pixel super-resolution step, is required only for diffraction-based lens-free microscopes, in which there is a considerable distance between the objects and the detector array such that diffraction cannot be ignored at optical frequencies. In this second computation step, the image of the specimen can be reconstructed from its super-resolved interference pattern by using iterative phase reconstruction48 or twin-image elimination algorithms49 that are commonly used in digital holography literature, as well as other reconstruction algorithms borrowed from, for example, coherent diffraction imaging techniques37–42. As a result of this step, both amplitude and phase images of the object can be generated, the latter of which might be especially important for imaging weakly scattering transparent specimens such as submicron bacteria or parasites7.
Lens-free on-chip microscopy can use additional computational tools to move from 2D cross-sectional images to a lens-free tomogram of the object by merging the spatial-frequency information of different illumination angles13,14,50. Here we should emphasize that the ability of lens-free in-line holography to digitally propagate a field over a long depth of focus does not immediately permit tomographic imaging of objects unless multiangle illumination13,14, ptychographic tomography51 or a compressive holography–based approach15,52 is used.
Note also that such computational blocks are not needed for a conventional lens-based microscope, which can provide immediate visualization of specimen through the eyepiece. Although this can be considered an important limitation of computational microscopy tools in certain settings, the recent advances in microprocessors such as graphics processing units (GPUs) that are now appearing even on our cell phones make computation extremely cost-effective and widely accessible globally, which we believe would be a key enabler for lens-free imaging techniques to scale up.
After covering some of the basic features of lens-free on-chip microscopes, next we would like to give examples of their state-of-the-art performance. Lens-free computational microscopy permits imaging at a large space-bandwidth product corresponding to a reconstructed image with more than 1.5 gigapixels. Using a state-of-the-art CCD enables imaging an ultra-wide FOV of ~18 cm2 (~4.9 cm × 3.7 cm) with a half-pitch resolution of ~2.19 μm, which contains >1.5 billion useful pixels, assuming 2 pixels define the minimum feature size (Fig. 2). In this case, the monochrome CCD chip itself (Kodak, KAF-39000) has ~40 megapixels, where each pixel is physically 6.8 μm wide. However, by using pixel super-resolution algorithms, a deeply subpixel resolution corresponding to an NA of ~0.1 can be achieved across the entire active area of the CCD chip (a FOV of ~18 cm2, Fig. 2). In comparison, a conventional objective lens with a similar NA would typically have a FOV that is a few square millimeters, with typically less than 10 megapixels. Although a customized microscope lens design based on conventional optics53 or mechanical scanning of the sample or lens could be used to enlarge the FOV, it would be a relatively complicated, bulky and costly solution to achieve such a wide imaging area.
At the other extreme, using a state-of-the-art CMOS chip instead of a CCD, one can achieve a half-pitch resolution of <350 nm in air (illumination wavelength, 530 nm) across an FOV of ~20.5 mm2 (that is, ~5.21 mm × 3.94 mm). In this case (Fig. 3), the CMOS chip (Sony Corp.) is a 16-megapixel color (RGB) sensor with a pixel size of ~1.1 μm that is manufactured for use in cell phone cameras. Once again, using pixel superresolution algorithms (only for the green pixels of the color CMOS chip) achieves deeply subpixel resolution corresponding to an NA of ~0.8 (in air) across the entire active area of the CMOS sensor chip (FOV > 20 mm2, Fig. 3).
One unique aspect of lens-free computational on-chip imaging is the fact that these quoted numbers will immediately improve as new sensor arrays become available. The rapid advancement that we experience in sensor array technologies is driven mostly by the cell phone and digital camera manufacturers, which produce more than 1 billion new camera modules every year, placing lens-free on-chip microscopy in a unique position to follow this trend.
Equally important is that the imaging geometry (Fig. 1) decouples spatial resolution from FOV. Stated differently, as more mega-pixels are introduced onto the same chip, while the pixel size is kept the same (or even smaller), it is possible to achieve a larger FOV without sacrificing resolution (or keep the same FOV with improved resolution). Therefore, the current trend in the image sensor industry toward smaller pixel size and higher-megapixel imager chips will continue to improve the resolution and FOV of lens-free computational microscopes, providing us a unique on-chip microscopy platform in which the resolution and FOV are not necessarily tied to each other.
Resolving microscale features of biological specimen in both space and time, lens-free on-chip microscopes have the potential to influence almost all the fields that their conventional lens-based counterparts are used for, including imaging, screening and tracking of cells and microorganisms in applications ranging from high-throughput screening to lab-on-a-chip technologies5–12,20,54,55. Because conventional wide-field optical microscopy has itself gone through a renaissance during the last two decades, an extensive comparison between lens-free on-chip microscopy tools and conventional lens-based optical microscopes would need to be rather detailed to be fair. However, one can highlight the following features and relative advantages of on-chip microscopes over their conventional lens-based counterparts: (i) decoupling of FOV from resolution, which enables the space-bandwidth product of the on-chip microscope to easily scale up with rapid advancements in optoelectronic sensor-array technologies; (ii) larger FOV and depth-of-field for 3D screening of enlarged volumes; (iii) compactness and lighter weight, which lead to advances such as better integration with lab-on-a-chip platforms; and (iv) design simplicity and cost-effectiveness. On the other hand, compared to conventional microscopy, lens-free on-chip microscopy still faces several key challenges, such as limited spatial resolution, difficulty in imaging fluorescent and thick specimens or the lack of standardized computational tools, which may, in practical terms, limit the reconstruction and image visualization times. To further improve the performance of on-chip microscopy and widen its application areas, such challenges will need to be addressed. We discuss these challenges next.
Despite their large imaging areas, the state-of-the-art resolution for lens-free on-chip microscopy is still not diffraction limited. Improving the numerical aperture from its current level of ~0.8 to 1 or even higher could potentially be achieved using liquids with higher refractive indices (for example, by oil immersion1) that fill air gaps between the object and detector planes to increase the spatial-frequency passband of the system. Figure 3 illustrates that based on this oil-immersion method and a 1.1-μm CMOS imager, we can improve the numerical aperture to ~0.9, which corresponds to a half-pitch resolution of ~300 nm under an illumination wavelength of 530 nm.
Further resolution improvements would probably necessitate new sensor chips that have submicron pixel sizes. Though these seem to be on the horizon, especially with the next-generation CMOS imager chips that are being developed, a factor that is important for on-chip microscopy but less so for the mainstream use of sensor arrays (for example, in digital cameras) is the angular distortion of pixels. At high numerical apertures, the pixels of a sensor array typically exhibit artifacts because oblique light rays experience much higher losses than straight rays and, more importantly, can end up generating a signal in neighboring pixels, which distorts accurate sampling of object transmission patterns. Therefore, special attention has to be given to the design of lens-free imager chips in terms of their angular response, which is not as critical for cell phone camera applications because of the presence and the lower NA of the imaging lens. To reduce pixel-related imaging aberrations and get closer to the diffraction limit, new sensor chips that have submicron pixel size and a decent external quantum efficiency—which is not strongly dependent on illumination angle—are needed.
In addition, new signal processing approaches that rely on sparse signal recovery and compressive sampling algorithms15,16,26,37 are also promising directions that can be combined with pixel super-resolution schemes to further improve the resolution of lens-free on-chip microscopes.
For a transmission imaging geometry, sample density (that is, the number of scattering objects—for example, cells—per unit volume or area) can cause issues in lens-free on-chip imaging performance for both holographic and contact imaging schemes. For contact shadow imaging, if the density of the specimen—for example, of nonadherent cells or multicellular organisms—increases in 3D, it creates aberrations because the objects will effectively move away from the sensor active area, thus increasing the contribution of diffraction to shadow images, which directly reduces the resolution and creates aberrations when applying pixel super-resolution algorithms20,21. In the case of high-density planar 2D objects (such as adherent cells, for which specimen thickness can be ≤1 μm), contact imaging would still exhibit aberrations and artifacts due to (i) shadow stretching at the active plane of the sensor chip as a function of illumination angle, (ii) change of object cross-section and its shadow as a function of illumination angle and (iii) partial interference of these dense object shadows with each other. The first two limitations mentioned above are inherent to the technique, as contact imaging of static objects requires large illumination angles (for example, up to ±60°) to achieve subpixel shadow shifts20. The third limitation is related to partial coherence of illumination and needs to be carefully analyzed in high-resolution contact imaging even if the coherence diameter at the active region of the detector array is, for example, less than 1–2 μm. Stated differently, contact-mode lens-free imaging that uses pixel super-resolution techniques could exhibit ‘artificial’ submicron features under even a very small coherence diameter.
Similar limitations also apply to holographic lens-free on-chip imaging in the case of dense samples. For lens-free in-line holographic imaging, as the density of the samples gets higher (as in the case of confluent cell cultures or a tissue slice, for example), the background light (which acts as a reference wave) gets distorted. One potential solution for this issue is to increase the distance between the sample and the detector chip so that a beam splitter can reflect an unscattered reference beam onto the sensor array56. This, however, would necessitate the use of increased temporal and spatial coherence for illumination and complicate the setup in terms of alignment and size. Because of the increased distance between the sample and sensor planes, it would also reduce the effective FOV, especially for large-area CCD chips.
Another solution to this object density issue that has been applied to wide-field on-chip holographic microscopy is multi-height lens-free imaging, in which the sensor array records the lens-free diffraction holograms of the specimen at different heights. By iteratively propagating back and forth between 3–5 different heights, for example, phase and amplitude images of dense objects can be reconstructed without the need for any spatial filtering10,41. The disadvantage of this approach is that more measurements are now required for the same sample, which also necessitates the use of additional computation to align the FOV of each multiheight lens-free hologram to others.
Whereas dense and transparent specimens can be handled using lens-free on-chip microscopy tools as discussed above, opaque samples cannot be imaged using transmission microscopy modalities in general. For relatively thick and nontransparent samples, such as tissue, reflection imaging would be needed. There are previous reports on lens-free reflection imaging methods17,57; however, these approaches have a smaller FOV (for example, ~9 mm2) than that of lens-free transmission on-chip imaging, which is mostly because of significantly increased distance between the specimen and the detector array in reflection imaging geometry. Furthermore, although these reflection imaging approaches are lens-free, they still rely on a beam splitter cube to channel the reflected object field onto a sensor array. As the active area of the sensor chip gets larger (for example, >10 cm2), the size of such a beam splitter would also grow, which would further increase the distance between the specimen and the detector planes, potentially causing signal-to-noise ratio limitations for submicron features of the object. Therefore, unlike its transmission counterpart, lens-free reflection microscopy is less suitable for reaching extreme FOVs (>10–20 cm2) that are achievable with the state-of-the-art CCD chips.
Our lens-free imaging discussions so far have been limited to bright-field microscopy on a chip. Fluorescence imaging, however, is another important modality that needs to be merged onto the same lens-free on-chip microscope for various applications that demand it—for example, when molecular probes are used to bring functionality, specificity and sensitivity to the imaging platform. To this end, there have been efforts to create dual-mode lens-free microscopes that can switch back and forth between bright-field and fluorescence imaging27. However, the resolution and image quality of these existing lens-free fluorescence imaging solutions are not yet as competitive as those of their bright-field counterparts26,28,58. There are several reasons behind this limited performance of lens-free fluorescence imaging. First, fluorescence emission is not directional, and the signal strength therefore drops much faster as a function of vertical distance in a lens-free fluorescence imaging geometry as compared to its transmission counterpart. This lower signal-to-noise ratio requires placing the labeled specimens rather close to the sensor chip surface (for example, <400–500 μm). Second, decent filtering of the excitation light on a chip is challenging. In a lens-free configuration, thin film–based standard fluorescence filters would not work as desired because the randomly scattered excitation light would not be collimated, unlike in conventional lens-based fluorescence microscopes. Fluorescence lens-free microscopy thus requires relatively thicker absorption-based filters, which increase the distance between the sample and the detector active area at the expense of reducing the achievable resolution. Third, fluorescence emission is spatially and temporally incoherent, and so holographic digital reconstruction and related pixel super-resolution techniques that are based on source shifting are not useful in this case. Although recently emerging sparse signal recovery techniques (based on compressive sampling, for example) have been used to decode lens-free fluorescence images28, their resolution level is still relatively coarse (~3–4 μm), mostly because of the reduced signal-to-noise ratio achieved in fluorescence on-chip imaging.
Though the issues discussed above are creating technical obstacles for fluorescence imaging to achieve submicron resolution over a large imaging area, which is the characteristic signature of lens-free on-chip imaging, these difficulties do not constitute fundamental challenges. Systematic improvements in (i) detection signal-to-noise ratio through technologies such as actively cooled sensor chips, (ii) absorption filter performance by use of better dyes, (iii) illumination schemes involving, for example, the use of structured excitation light and (iv) incoherent signal recovery algorithms that exploit sparsity of fluorescence images could help us achieve high-resolution dual-mode lens-free on-chip microscopes that can switch between fluorescence and bright-field imaging of large FOVs. Once lens-free microscopy is combined with color sensor arrays (for example, RGB CMOS chips, Fig. 3), multicolor lens-free fluorescence imaging could also be achieved, which might extend the use of these on-chip microscopes to various applications such as fluorescence cytometry.
The imaging speed of lens-free on-chip microscopes is limited by the frame rate of the sensor array, which varies from ~1 frame per second (f.p.s.) to >50 f.p.s. depending on the exposure time and the megapixel count of the sensor chip. Under appropriate illumination conditions, which can be satisfied by even an LED, the frame rate can be significantly increased to >200–300 f.p.s. by digitally selecting a smaller FOV within the active area of the chip. This also implies that high-throughput scanning of large sample areas and volumes at high frame rates can be achieved by digitally moving the region of interest within the active area of the sensor chip. This could be quite useful for observing spatiotemporal dynamics of fast-moving microorganisms such as sperm12 across large sample volumes and for collecting extensive statistics on their natural swimming behavior and response to various external stimuli.
Regarding imaging speeds, an important consideration is that the surface temperature of the sensor chip can easily reach above 40 °C after ‘continuous’ operation for, for example, more than 10 min. This heating poses an important temporal limitation, especially for contact imaging approaches, which might not succeed in cooling the immediate top surface of the imager chip despite using cooling circuitry or heat sinks. This concern also exists for lens-free holographic approaches, but to a lesser degree because of the greater distance (for example, 1–2 mm) between the sample and the sensor chip.
The different versions of lens-free on-chip microscopes can vary greatly in design. What makes this picture even more confusing is that the reconstruction methods also vary depending on the imaging architecture. This complexity poses challenges for the wide-scale use of these emerging computational microscopy tools, especially in the biomedical sciences. This limitation can be overcome by the standardization of reconstruction algorithms through the creation of modular software blocks (running on GPUs, for example) that are integrated with each other so as to handle various possible lens-free imaging configurations. As a starting point, an open-source computational library for holographic reconstruction is already available59.
In conclusion, lens-free computational imaging tools are becoming more capable of creating wide-field microscopic images of specimens located on a chip. Compared to conventional lens-based microscopes, lens-free microscopes are in general simpler in hardware and more compact and lightweight while also achieving a larger FOV and depth-of-field for 3D screening of enlarged volumes. With further improvements in fluorescence and reflection-based lens-free on-chip microscopes and development of standardized reconstruction software that can better handle sample density issues and variations in lens-free imaging designs, we believe that computational on-chip microscopy will continue to find broader applications in the physical and biomedical sciences, among others. Once lens-free microscopy overcomes these limitations and successfully scales up to a large number of users, massive amounts of microscopy data within various disciplines could be generated. This, in turn, could present its own challenges, adding more to our existing ‘big data’ problem60. This challenge, however, will also create new opportunities for microscopic analysis in general and could potentially be addressed through machine learning as well as crowd-sourcing strategies61.
A.O. gratefully acknowledges the support of the Presidential Early Career Award for Scientists and Engineers, Army Research Office Young Investigator Award, National Science Foundation CAREER Award, Office of Naval Research Young Investigator Award and National Institutes of Health Director’s New Innovator Award DP2OD006427 from the Office of the Director, National Institutes of Health.
COMPETING FINANCIAL INTERESTS
The authors declare competing financial interests: details are available in the online version of the paper.
Reprints and permissions information is available online at http://www.nature.com/reprints/index.html.