PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of optexpOSAOptics InfoBaseThis ArticleThis JournalSearchAuthor Information
Optics Express
 
Opt Express. 2012 January 30; 20(3): 3129–3143.
Published online 2012 January 25. doi:  10.1364/OE.20.003129
PMCID: PMC3364049
NIHMSID: NIHMS377264

Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy

Alon Greenbaum 1 and Aydogan Ozcan 1 , 2 , 3 , *

Abstract

Lensfree in-line holographic microscopy offers sub-micron resolution over a large field-of-view (e.g., ~24 mm2) with a cost-effective and compact design suitable for field use. However, it is limited to relatively low-density samples. To mitigate this limitation, we demonstrate an on-chip imaging approach based on pixel super-resolution and phase recovery, which iterates among multiple lensfree intensity measurements, each having a slightly different sample-to-sensor distance. By digitally aligning and registering these lensfree intensity measurements, phase and amplitude images of dense and connected specimens can be iteratively reconstructed over a large field-of-view of ~24 mm2 without the use of any spatial masks. We demonstrate the success of this multi-height in-line holographic approach by imaging dense Papanicolaou smears (i.e., Pap smears) and blood samples.

OCIS codes. (090.1995) Digital holography, (170.3880) Medical and biological imaging

1. Introduction

Optical microscopy has been the workhorse of various fields including physical sciences, engineering, biology and medicine. However, existing optical microscopes are still relatively bulky and expensive, making them by and large restricted to laboratory settings. In recent years, however, there has been an extensive research effort to create alternative microscopy tools that are based on computational tools such as holographic imaging [‎1-‎14]. As a result of these efforts, several new microscopy modalities were introduced to create simpler, more compact and cost-effective designs that can even be used in remote locations and field settings [‎15-‎30].

A recent development toward the same goal is the lensfree super-resolution holographic microscope [‎31, ‎32] which achieves sub-micron spatial resolution over a large field-of-view of e.g., ~24 mm2. It works based on partially-coherent lensfree digital in-line holography using multiple light sources (e.g., light-emitting diodes - LEDs) placed at ~3-6 cm away from the sample plane such that at a given time only a single source illuminates the objects, projecting in-line holograms of the specimens onto a CMOS sensor-chip. Since the objects are placed very close to the sensor chip (e.g., ~1-2 mm) the entire active area of the sensor becomes our imaging field-of-view, and the fringe-magnification is unit. As a result of this, these holographic diffraction signatures are unfortunately under-sampled due to the limited pixel size at the CMOS chip (e.g., ~2-3 µm). To mitigate this pixel size limitation on spatial resolution, several lensfree holograms of the same static scene are recorded as different LEDs are turned on and off, which creates sub-pixel shifted holograms of the specimens. By using pixel super-resolution techniques, these sub-pixel shifted under-sampled holograms can be digitally put together to resolve/sample much larger portion of the higher spatial frequency oscillations within the lensfree object hologram [‎31, ‎32].

This super-resolved (SR) in-line hologram, however, still suffers from twin-image artifact, which is common to all in-line hologram recording geometries [‎33]. In our earlier work, we demonstrated the use of an iterative object-support based phase recovery method [‎34, ‎35] to eliminate this twin-image artifact creating wide-field microscopic images of samples. This twin-image elimination method, however, requires as input the location estimations of the objects within the imaging field-of-view. For this end, a simple threshold or a segmentation algorithm can be used to automatically estimate the objects’ locations (creating the object-support) for relatively sparse samples. However, in denser specimens, this object support is difficult to estimate which can create challenges in removal of the twin-image artifact.

To overcome these object-support related imaging challenges for dense and connected specimens, here we demonstrate a new approach for pixel super-resolution holographic microscopy that uses multiple (e.g., 2-5) lensfree intensity measurements that are each captured at a different height (i.e., Z2) from the detector-array (see Fig. 1 ). Stated differently, each lensfree super-resolved hologram is synthesized with ~30-70 µm change in the relative height of the object with respect to the detector-chip surface, after which they are digitally registered and aligned to each other to take into account possible rotations and shifts among these in-line holograms. These co-registered super-resolved holograms, corresponding to different object heights, are then iteratively processed [‎36] to recover the missing optical phase so that microscopic images of the specimens can be automatically reconstructed without the need for any spatial masking steps. Therefore, this multi-height holographic approach eliminates the need to estimate the object-support at the sample plane cleaning the twin-image artifacts of our in-line geometry over a large field-of-view of e.g., ~24 mm2 even for dense and connected specimens.

Fig. 1
Schematic diagram of the multi-height pixel super-resolution based lensfree on-chip imaging set-up. A partially-coherent source (emanating from a 0.1 mm core fiber-optic cable) creates lensfree in-line holograms of the samples, which are sampled using ...

Compared to previously reported results, this work demonstrates the first implementation of maskless multi-height phase recovery in partially-coherent lensfree optical microscopy on a chip. Furthermore, this is the first time that pixel-super resolution has been implemented in multi-height phase recovery to digitally mitigate twin image artifact in lensfree in-line holography. A key to the success of pixel super-resolved multi-height phase recovery is actually the use of partially-coherent illumination (both spatially and temporally) as opposedto coherent illumination. Considering that under highly coherent illumination, speckle noise and multiple reflection interference artifacts would create fundamental challenges for digital registration of sub-pixel shifted lensfree holograms of a given height to each other as well as to lensfree holograms of different heights, our partially-coherent illumination scheme is rather important for enabling maskless reconstructions over a large field-of-view of ~24 mm2. Based on its unique hologram recording geometry with unit fringe magnification (Fig. 1), the presented technique can work with a spectral illumination bandwidth of e.g., ~5-10 nm and a spatial coherence diameter of e.g., < 0.5 mm at the detector plane. As a result, speckle and multiple reflection interference artifacts can be minimized, which is the key for multi-height pixel super-resolved lensfree on-chip imaging over large field-of-views as demonstrated in this work. We validated the superior performance of this approach by imaging dense Papanicolaou smears (i.e., Pap smears or Pap tests, which are used to screen cervical cancer by detecting premalignant and/or malignant cells in the endocervical canal) as well as blood samples. Providing a light-weight and cost-effective design, this multi-height holographic on-chip imaging platform could be rather useful for wide-field microscopy and pathology needs in resource poor locations as well as in field conditions.

2. Overview of phase recovery methods

In digital holography, the optical phase information of the scattered object field cannot be directly measured and is actually encoded into intensity oscillations of the recorded hologram. Therefore, for reconstruction of images using digital holographic data, phase recovery is of paramount importance. There have been several different approaches to tackle this important problem, and depending on the hologram recording scheme and its complexity, the degree of success varies [‎36-‎42].

The goal of phase recovery in our context is to extract a complex-valued object function from the intensity of its diffraction pattern. Note that in our partially-coherent holographic microscopy scheme described in Fig. 1, since the specimens are placed rather close to the detector array (e.g., Z2 ~0.7-1 mm), the object field-of-view roughly equals to the detector active area. And due to partial-coherence of illumination there is no longer a single Fourier transform relationship between the entire object and detector planes. In fact, using the transfer function of free-space one can digitally propagate back and forth between the object and detector planes through two successive Fourier transform operations. Therefore, the number of effective pixels (P) at the diffraction sampling plane equals to the number of useful pixels at the object plane. On a related note, the basic function of pixel super-resolution techniques in lensfree digital in-line holography is actually to increase the value of P beyond what the sensor chip can provide at the circuit level [‎31, ‎32].

As a result of this, for a complex-valued object function, phase recovery problem becomes undetermined by a factor of 2 since there are 2 × P pixels defining the object function (P pixels for the real part and P pixels for the imaginary part), whereas there are only P pixels in the measurement matrix [‎43, ‎44]. In order to solve this underdetermined phase recovery problem, new information about the object function needs to be acquired and incorporated as a constraint on the solution space.

To provide a solution to this important problem, numerous iterative phase recovery algorithms were devised [‎37, ‎45-‎50], where various different types of information about the object were used as constraints. These object constraints, together with the measured diffraction pattern were enforced at each step of the iterations, gradually converging to the missing 2D phase information [‎51]. The object-support constraint, which specifies the locations of the objects as a binary mask, is one of the most commonly used constraints [‎34], which also formed the basis of some of our earlier work [‎28, ‎29, ‎52]. This binary mask is a matrix, where a value of one is assigned to the binary mask wherever the object is located, and a zero is assigned wherever the background appears. This way, the object-support constraint or the spatial mask provides the coordinates for which the object function is zero, which effectively reduces the number of unknowns in our underdetermined phase recovery problem.

While quite effective [‎42, ‎53], object-support based iterative phase recovery techniques encounter practical challenges in digital in-line holography when the density of the sample is relatively high, which makes it difficult to create an appropriate binary mask that can effectively reduce the number of unknowns at the object plane. This issue can affect convergence of iterative phase recovery techniques, yielding sub-optimum reconstructions of dense object functions. To overcome these hurdles, several methods were devised which in practice are restricted to sparse objects and/or to relatively small imaging field-of-views [‎54-‎56].

In this work, we utilized an iterative phase recovery method together with a pixel super-resolution technique that uses multiple diffraction intensity measurements of the objects captured at different planes along the light propagation direction. Each additional lensfree diffraction measurement captured at a different object height brings a new set of equations to help us solve this underdetermined phase recovery problem without the use of any object-support constraint or spatial masks. In the literature, there are various methods that one can use to retrieve the lost phase information from two or more diffraction intensity measurements, such as transport-of-intensity equation (TIE) based methods [‎40, ‎41, ‎57, ‎58], iterative methods which use these diffraction measurements as successive amplitude constraints [‎36, ‎59] and other non-iterative methods [‎60, ‎61]. Furthermore, multiple defocused images are also utilized in phase diversity methods, where phase aberrations of incoherent imaging systems can be characterized [‎62-‎64]. In this work, we followed the iterative method presented in reference [‎36] due to its simplicity and robustness. Refer to Sections 4 and 5 for further details of implementation.

3. Pixel super-resolution (PSR) based multi-height lensfree holographic imaging

Our pixel super-resolved multi-height imaging set-up is illustrated in Fig. 1. It is composed of a partially-coherent light source (Xenon lamp attached to a monochromator, with bandwidth of ~5 nm), that is coupled to a multi-mode fiber (100 µm core diameter). The fiber-tip to sample distance (Z1) is ~10 cm, while the sample to sensor distance (Z2) is ~0.7-1 mm (Fig. 1). The detector is a CMOS chip (Aptina MT9P031, 5 Mega-pixel) with a 2.2µm pixel size. Since we employ Z1 >> Z2, the imaging field-of-view (FOV) is only limited by the CMOS sensor’s active area (i.e., ~24 mm2 in our case) [‎55].

As discussed in our introduction, in this hologram recording geometry, the main limitation on resolution is the limited pixel size (2.2 µm) at the detector-array, which causes the higher frequency oscillations in our lensfree in-line holograms to be undersampled. To overcome this undersampling issue, we employ source-shifting [‎31, ‎32] to create sub-pixel shifted replicas of the same holographic pattern on the CMOS array such that by using PSR we can digitally synthesize an effectively much smaller pixel size [‎32]. Here we should emphasize that without employing any source shifting or pixel super-resolution techniques, a sub-pixel spatial resolution can already be achieved using our unit fringe magnification hologram recording geometry shown in Fig. 1 (see e.g., references ‎29 and ‎52). In this respect, despite its unit magnification, our holographic imaging set-up has significant advantages compared to a hypothetical “perfect” near-field sampling experiment that utilizes the same pixel size at the detector array to sample the object field [‎29, ‎52]. The main idea behind PSR to further improve spatial resolution is to incorporate few lower-resolution (LR) images of the same object into one SR image [‎65-‎67]. In our multi-height implementation, a slight translation of the fiber-tip in the plane parallel to the CMOS sensor will result in a sub-pixel translated image, during which a single LR lensfree hologram is captured. By translating the source multiple times, an image stack is created, where each lensfree image in this stack is slightly shifted with respect to the others (see Fig. 1). Note also that since Z1 >> Z2 there is a large demagnification factor (e.g., >100) between the source-shifting plane and the detector plane so that relatively large shifts at the source plane translate into much smaller shifts at the detector plane, which makes our multi-height imaging approach rather convenient to operate without the need for any fine mechanical alignment or scanners. From this acquired stack of LR lensfree holograms, an SR hologram is synthesized (individually for each object height), which is effectively equivalent to recording the same object hologram with a smaller pixel size detector-array [‎31, ‎32].

4. Image registration among multi-height lensfree SR holograms

In our multi-height imaging based iterative reconstruction scheme, we synthesize M PSR holograms, one for each object height (Z2) - see Fig. 2 . However, among these M super-resolved holograms there might be spatial discrepancies since the object might have shifted and/or rotated between the acquisitions of each lensfree image stack. Therefore, we perform a digital registration step after the synthesis of these SR holograms to ensure that each hologram essentially looks at the same object field-of-view. This registration procedure is based on arbitrary selection of three points (control-points) in one hologram (i.e., the reference hologram, typically the one with the lowest Z2-distance) and finding the matching three points in the other M-1 lensfree holographic SR images. These control points should be close to the corners of the image in order to achieve small registration errors (~2 μm or less) over relatively large distances (e.g., ~5 mm). Moreover, these control points should better be circularly symmetric since normalized-correlation, which is used to find the shift of each control-point between the images, is sensitive to rotation. Quite conveniently, the acquired lensfree images are in-line holograms, and an ergo isolated object (like a dust particle) in the object plane resembles an airy function in the hologram domain and exhibits circular symmetry. Therefore, in-line lensfree holograms of these dust particles or substrate defects, which are common in most samples, are desirable candidates to serve as control points. If needed, isolated spatial marks (in the form of e.g., dots) can also be placed to the corners of a sample slide to serve as predetermined control points.

Fig. 2
(a) A schematic portraying the image processing that is conducted after acquiring M LR lensfree image stacks at different Z2-distances. Each image stack is used to create one super-resolved lensfree hologram. These high-resolution holograms are then registered ...

After selection of the control points, a small area (e.g., ~30 × 30 μm) around each control point is cropped and digitally interpolated (~4-6 times) to serve as a normalized correlation template. Furthermore, for accurately finding the coordinate shift of each control point among M images, lensfree holographic images have to be positioned in the same Z2-distance. Therefore, the difference in the Z2-distance between lensfree holograms acquired at different heights is evaluated by an auto-focus algorithm [‎68, ‎69], which permits us to digitally propagate the selected correlation templates to the same Z2-distance, where normalized correlations are calculated to find the coordinate shifts between the control points in each image. After each control-point’s coordinates are known in every image, M-1 affine transformations are built to register these lensfree holographic images to the reference image, such that all the lensfree holograms acquired at M different heights can be digitally registered to each other.

5. Phase recovery using M PSR lensfree holograms

In order to retrieve the phase information and consequently eliminate the twin image artifact in our multi-height lensfree imaging approach, a modified Gerchberg-Saxton (GS) algorithm is utilized [‎59]. This modified GS algorithm is based on M intensity measurements that are taken at different Z2 planes (see Fig. 1). The lowest measurement plane is typically taken with a Z2 value of ~700 µm (which will be referred as intensity measurement #1). Note that to achieve a measurement with a lower Z2 distance, the sensor’s protective glass will have to be removed. The highest intensity measurement is typically taken with a Z2 value of ~1 mm (referred as intensity measurement #M). The distance ([increment]Z) between two sequential intensity measurements is ~50 µm. We achieved this [increment]Z value by using glass cover slips of different thicknesses.

In our iterative phase recovery process, we used the transfer function of free-space [‎29] to propagate the fields back and forth, where the amplitudes of the M intensity measurements serve as constraints at each plane. To initiate the phase recovery process, a zero-phase is assigned to the object intensity measurement. One iteration during this phase-recovery process can be described as follows: Intensity measurement #1 is forward propagated (with zero initial phase) to the plane of intensity measurement #2 (see Fig. 2 green arrow). Then, the amplitude constraint in measurement #2 is enforced while the calculated phase resulting from forward propagation remains unchanged. The resulting complex field is then forward propagated to the plane of intensity measurement #3, where once again the amplitude constraint in measurement #3 is enforced while the calculated phase resulting from forward propagation remains unchanged. This process continues until reaching the plane of intensity measurement #M. Then instead of forward propagating the fields of the previous stages, back propagation is used (see Fig. 2 yellow arrow). The complex field of plane #M is back propagated to the plane of intensity measurement #M-1. Then, the amplitude constraint in measurement #M-1 is enforced while the resulting phase remains unchanged. The same iteration continues until we reach the plane of intensity measurement #1. When one complete iteration is achieved (by reaching back to the plane of intensity measurement #1), the complex field that is derived in the last step will serve as the input to the next iteration. Typically, 1-70 iterations are required for satisfactory results, and for final object reconstruction step the acquired complex field of any one of the measurement planes can be back propagated to the object plane to retrieve both phase and amplitude images of the dense specimens on the chip.

6. Sample preparation

Blood smear samples were prepared using whole blood (UCLA Blood Bank, USA), where the samples were diluted (×2) with RPMI (Thermo Scientific, Catalog #: SH3002701) in room temperature. Then 5μL of the diluted blood was dropped on a type-one glass cover slip (Fisher Scientific Catalog # 12-548-A). The blood droplet was then smeared by a second cover slip by applying a constant force. The sample was then left to dry in air for ~10 minutes before being fixed and stained by HEMA 3 Wright-Giemsa staining kit (Fisher Diagnostics). The Papanicolaou smear (Pap smear) was prepared using a standard SurePath (BD Inc.) procedure [‎70].

7. Experimental results and discussion

A schematic diagram of our lensfree multi-height imaging set-up can be seen in Fig. 1. The set-up is composed of a partially-coherent light source (~5 nm bandwidth centered at 550 nm), glass cover slips with different thicknesses and a CMOS detector-array. The set-up is rather simple to operate without any complicated alignment. For reconstructing dense samples where object-support based phase-recovery approaches face challenges, different lensfree intensity measurements of the sample are acquired at different Z2 distances.

The Z2 distance is controlled by placing glass cover slips with different thicknesses between the sample and the sensor-chip. The thicknesses of our glass cover slips varied between 50 µm and 250 µm, hence the corresponding Z2 distances varied between ~0.7 mm and ~1 mm. Each lensfree intensity measurement is sampled by our CMOS sensor with 2.2 µm pixel size. This relatively large pixel size can cause undersampling issues; therefore, a PSR method is applied in order to effectively decrease the detector pixel size [‎31, ‎32]. For each Z2-distance a lower-resolution (LR) image stack is captured, where each image in this stack is sub-pixel shifted with respect to the other images in the stack. These sub-pixel shifts are achieved by a slight translation of the fiber-tip position between two sequential images (see Fig. 1. upper left inset).

Figure 2 depicts the image processing steps after image acquisition. For each Z2-distance, one SR hologram is synthesized from the LR image-stack (typically 16 images in each stack). Then, these M SR images are registered to each other (see Section 4) to account for rotation, translation and shear that may occur during placing/replacing the glass cover slips between each lensfree measurement. For the reconstruction procedure (iterative phase recovery, see Section 5), these M SR intensity measurements are utilized as amplitude constraints (typically 1-70 iterations are required for satisfactory results). Once the phase is iteratively recovered, any one of these M SR images is low-pass filtered (Butterworth 2nd order) to eliminate high frequency noise and the resulting complex image is back propagated to the object plane, retrieving both phase and amplitude images of the specimens on the chip (see Fig. 2).

Figure 3 illustrates the benefits of using the above outlined multi-height lensfree imagingapproach for a blood smear sample. Figure 3(a) shows a full FOV (~24 mm2) LR lensfree hologram as captured by the CMOS sensor. The green dashed rectangle focuses on an area that is rather dense; however the blood cells are still organized as a mono-layer, suitable for imaging. The reconstruction results of this dense blood smear using five different Z2-distances (711μm, 767μm, 821μm, 876μm and 946μm) are shown in Fig. 3(b). These five Z2-distances/heights are automatically evaluated by using an auto-focus algorithm [‎68, ‎69]. The reconstruction results of Fig. 3(b) provide a decent agreement to a 10 × microscope objective comparison image shown in Fig. 3(c). Figures 3(d), 3(e) and 3(f) provide images of zoomed areas (taken from the dashed blue rectangle in Fig. 3(b)) of single height back propagation image, multi-height reconstruction image and a 20 × microscope objective comparison image, respectively. The back propagated single height image (Fig. 3(d)) has lower contrast, and it is hard to evaluate the locations of the RBCs for spatial masking purposes. Therefore support-based phase recovery would not be effective in this case. On the other hand, the multi-height amplitude image (Fig. 3(e)) has significantly improved contrast, and individual RBCs can be identified and resolved even in dense clusters. It is important to emphasize that these multi-height reconstruction images shown in Figs. 3(b,e) are obtained without the use of any spatial masking or any other prior information regarding the sample.

Fig. 3
(a) Full FOV, LR hologram. (b) Multi-height based PSR lensfree amplitude image of a dense RBC smear is shown. This lensfree image was reconstructed using five different heights (λ = 550nm). The FOV corresponds to the green dashed rectangular in ...

After these blood smear experiments, next we imaged Pap smears (based on SurePath automated slide preparation [‎70]) using the same multi-height imaging set-up. Figure 4 summarizes our imaging results for this Pap smear. Because of the density of the specimen, the reconstruction of this image is a challenging task for any phase recovery method. Figure 4(a) shows the multi-height phase image, which is recovered using lensfree measurements from five different heights (754μm, 769μm, 857μm, 906μm and 996μm - these Z2-distances were automatically determined using an auto-focus algorithm [‎68, ‎69]). Figures 4 (c,h) and 4(d,i) show zoomed images of the same Pap smear sample, for amplitude and phase channels, respectively. In these reconstructed multi-height images the cell morphology is clear and their boundaries can clearly be seen and separated from the background. Moreover, minor overlaps among the cells do not constitute a limitation in this method. As a comparison, Fig. 4(b) depicts a single height back propagated phase image corresponding to one of the Z2 measurements (the FOV is the same as in Fig. 4(a)). It is evident that distinguishing the cells from the background is a difficult task in this dense reconstructed image. To better provide a comparison, Figs. 4 (f,k) and (g,l) also show zoomed images of the same Pap smear sample, for phase and amplitude channels, respectively, calculated using back propagation of a single height image. Compared to Figs. 4(d,i) and 4(c,h), these single height back projection images show significant spatial distortion due to the density of the cells. Figures 4(e) and 4(j) also provide 40 × objective lens (0.65NA) microscope comparison images for the same zoomed regions, clearly providing a decent match to our multi-height reconstruction results shown in Figs. 4(d,i) and (c,h). Especially note the enhanced contrast of the cell boundaries in our phase images ( Figs. 4(d,i) ), which is complementary to the spatial information coming from our amplitude images ( Figs. 4(c,h) ). This complementary set of information that is conveyed by the amplitude and phase images might facilitate detection of abnormal cells within a Pap test that are characterized for instance by a high nuclear-cytoplasmic ratio. It is also important to note that all the phase images reported in our manuscript are wrapped; hence, for the multi-height reconstructed phase images, phase jumps should be expected in absorbing areas of the cells (e.g. nuclei), where phase would not be properly defined. Contrary to the phase images, these absorbing areas will be of high contrast in their corresponding lensfree amplitude images, which once again emphasizes the complementary nature of phase and amplitude lensfree image channels.

Fig. 4
(a) Multi-height based PSR lensfree phase image of a Pap test is shown. This image was reconstructed using five heights. 36 iterations were used during phase recovery (λ = 550nm). (b) Single height back propagated PSR phase image is shown. (c) ...

Next we investigated how the number of intensity measurements used in our iterative reconstruction process affects the image quality (see Fig. 5 ). To provide a fair comparison (i.e., to better isolate the source of improvement in image quality), a total of 144 Fourier transform pairs were used in each case, regardless of the number of intensity measurements employed in the multi-height based phase recovery. Moreover, all the phase images are wrapped and the same global phase was assigned to all of the images to avoid different phase jumps in different images. Figure 5(a) shows a single height back propagated phase image. When a second intensity measurement is added, multi-height based iterative phase recovery approach can be utilized. Consequently, the recovered phase image after 72 iterations (see Fig. 5(b)) looks significantly better than the phase image of Fig. 5(a). A further improvement in image quality is achieved by adding a third intensity measurement to the multi-height phase recovery process (Fig. 5(c)). After 36 iterations (i.e., corresponding to a total of 144 Fourier transform pairs as before), the cells that were hidden in the noisy background are now visible (see white arrows in Fig. 5(c)). A moderate improvement is noticed in the image contrast when adding more intensity measurements, as can be seen in the reconstructed multi-height phase images from four and five heights (Fig. 5(d) and 5(e), respectively). Note that in these two cases, 24 and 16 iterations were used, respectively, so that the total number of Fourier transform operations remains the same in all reconstructions shown in Figs. 5(b-e), which helps us to isolate the source of the phase reconstruction improvement and relate it to multiple height measurements rather than the number of back-and-forth digital propagation operations. Figure 5(f) shows a microscope comparison image (10 × , 0.25NA) for the same region of interest. Note also that the cell’s boundaries are more visible in our phase images, while the absorbing nuclei of the cells are better visualized in our amplitude images as illustrated in Fig. 4.

Fig. 5
Pap smear reconstruction results acquired for different number of lensfree diffraction intensities. (a) Back propagated image from one PSR lensfree hologram. (b), (c), (d) and (e) Multi-height based PSR lensfree phase images from two, three, four and ...

After validating the usefulness of pixel super-resolved multi-height based phase recovery approach with dense blood smears and Pap tests, we experimentally tested its impact on the reconstructed image quality. An important question that we aimed to address with this additional experiment was whether or not the digital cross registration process among different Z2 lensfree holograms results in spatial smearing of our reconstructed images. Therefore we compared the imaging performance of our multi-height reconstruction results against a single back-propagated super resolved hologram. For this end, we imaged an isolated ‘UCLA’ pattern that was etched on a glass slide using focused ion beam (FIB) milling, where the letters ‘U’ and ‘C’ are ~1 µm apart. We emphasize here that for such an isolated object multi-height based image reconstruction is not necessary. Since we aim to understand the impact of multi-height cross registration related issues, in this final experiment we chose an isolated object (‘UCLA’) so that the back-propagation result of a single height SR hologram could work for comparison purposes. This is quite different from the dense objects/specimens reported in Figs. 3--5,5, where back-propagation of a single height lensfree PSR hologram fails, requiring the use of multiple height measurements.

For this final experiment, the single height back propagated SR holographic image is shown in Fig. 6(a) , where the letters ‘U’ and ‘C’ are clearly separated. The ‘UCLA’ pattern is spatially isolated from nearby objects, and therefore for this small isolated FOV phase recovery is not necessary as emphasized earlier. Figures 6(b), 6(c), 6(d) and 6(e) show multi-height based reconstructed amplitude images, for two, three, four and five different heights, respectively (λ = 490nm). For fair comparison among these recoveries, once again the number of Fourier transform pairs was kept constant in each case, as a result of which each reconstruction used a different number of iterations (60, 30, 20 and 15 iterations, respectively). It is evident that the letters ‘U’ and ‘C’ are clearly separated in all of these images, which is an indication of our success in cross registration of different height super-resolved holograms to each other so that spatial smearing affects due to possible inconsistencies among different Z2 lensless holograms are minimized. A microscope comparison image of the same “UCLA” pattern can also be seen in Fig. 6(f), acquired using a 40x objective lens (0.65 NA).

Fig. 6
Adding intensity measurements from different Z2 distances does not degrade the image resolution. An important question that we aimed to address with this additional experiment was whether or not the digital cross registration process among different height ...

8. Conclusions

We demonstrated that by incorporating pixel super-resolved partially-coherent lensfree imaging with a multi-height phase recovery approach, dense and connected samples can be reconstructed over a large FOV (~24 mm2). The multi-height iterative phase recovery approach uses multiple super-resolved intensity measurements, which are acquired at different sample-to-sensor distances. Unlike mainstream object-support based phase recovery methods, the multi-height based phase recovery approach does not need a spatial mask for objects. As a proof of concept, this pixel super-resolved partially-coherent multi-height lensfree imaging approach was tested on Pap smears and whole blood samples. Color imaging results are not shown here, however using the same multi-height lensfree imaging approach at three different wavelengths (e.g., λ = 450 nm, 550 nm and 650 nm) and combining these lensfree images to generate color images of the specimens should be straightforward (see e.g., Ref. [71]). Finally, we should also mention that translating the same lensfree microscopy platform into reflection geometry is also feasible, although a reduction in imaging field of view would be expected as demonstrated in Ref. [72].

Acknowledgments

A. Ozcan gratefully acknowledges the support of ARO Young Investigator Award, NSF CAREER Award, the ONR Young Investigator Award 2009 and the NIH Director's New Innovator Award DP2OD006427 from the Office of The Director, NIH. The authors also acknowledge the support of the Gates Foundation, Vodafone Americas Foundation, and NSF BISH program (under Awards # 0754880 and 0930501). The authors also acknowledge Gabriel Biener, Ting-Wei Su, Serhan Isikman and Onur Mudanyali for their assistance with the manuscript.

References and links

1. Brady D. J., Choi K., Marks D. L., Horisaki R., Lim S., “Compressive holography,” Opt. Express 17(15), 13040–13049 (2009). doi: 10.1364/OE.17.013040. [PubMed] [Cross Ref]
2. Hahn J., Lim S., Choi K., Horisaki R., Brady D. J., “Video-rate compressive holographic microscopic tomography,” Opt. Express 19(8), 7289–7298 (2011). doi: 10.1364/OE.19.007289. [PubMed] [Cross Ref]
3. Waller L., Tsang M., Ponda S., Yang S. Y., Barbastathis G., “Phase and amplitude imaging from noisy images by Kalman filtering,” Opt. Express 19(3), 2805–2814 (2011). doi: 10.1364/OE.19.002805. [PubMed] [Cross Ref]
4. Wang Z., Millet L., Mir M., Ding H., Unarunotai S., Rogers J., Gillette M. U., Popescu G., “Spatial light interference microscopy (SLIM),” Opt. Express 19(2), 1016–1026 (2011). doi: 10.1364/OE.19.001016. [PMC free article] [PubMed] [Cross Ref]
5. Wang Z., Marks D. L., Carney P. S., Millet L. J., Gillette M. U., Mihi A., Braun P. V., Shen Z., Prasanth S. G., Popescu G., “Spatial light interference tomography (SLIT),” Opt. Express 19(21), 19907–19918 (2011). doi: 10.1364/OE.19.019907. [PMC free article] [PubMed] [Cross Ref]
6. Rosen J., Brooker G., “Digital spatially incoherent Fresnel holography,” Opt. Lett. 32(8), 912–914 (2007). doi: 10.1364/OL.32.000912. [PubMed] [Cross Ref]
7. Poon T. C., “Optical scanning holography - a review of recent progress,” J. Opt. Soc. Kor. 13(4), 406–415 (2009). doi: 10.3807/JOSK.2009.13.4.406. [Cross Ref]
8. Mann C., Yu L., Lo C. M., Kim M., “High-resolution quantitative phase-contrast microscopy by digital holography,” Opt. Express 13(22), 8693–8698 (2005). doi: 10.1364/OPEX.13.008693. [PubMed] [Cross Ref]
9. Hsieh C. L., Grange R., Pu Y., Psaltis D., “Three-dimensional harmonic holographic microcopy using nanoparticles as probes for cell imaging,” Opt. Express 17(4), 2880–2891 (2009). doi: 10.1364/OE.17.002880. [PubMed] [Cross Ref]
10. Shi K., Li H., Xu Q., Psaltis D., Liu Z., “Coherent anti-Stokes Raman holography for chemically selective single-shot nonscanning 3D imaging,” Phys. Rev. Lett. 104(9), 093902 (2010). doi: 10.1103/PhysRevLett.104.093902. [PubMed] [Cross Ref]
11. Kikuchi Y., Barada D., Kiire T., Yatagai T., “Doppler phase-shifting digital holography and its application to surface shape measurement,” Opt. Lett. 35(10), 1548–1550 (2010). doi: 10.1364/OL.35.001548. [PubMed] [Cross Ref]
12. Xu W., Jericho M. H., Meinertzhagen I. A., Kreuzer H. J., “Digital in-line holography for biological applications,” Proc. Natl. Acad. Sci. U.S.A. 98(20), 11301–11305 (2001). doi: 10.1073/pnas.191361398. [PubMed] [Cross Ref]
13. Garcia-Sucerquia J., Xu W., Jericho S. K., Klages P., Jericho M. H., Kreuzer H. J., “Digital in-line holographic microscopy,” Appl. Opt. 45(5), 836–850 (2006). doi: 10.1364/AO.45.000836. [PubMed] [Cross Ref]
14. Xu W., Jericho M. H., Kreuzer H. J., Meinertzhagen I. A., “Tracking particles in four dimensions with in-line holographic microscopy,” Opt. Lett. 28(3), 164–166 (2003). doi: 10.1364/OL.28.000164. [PubMed] [Cross Ref]
15. Rodriguez W. R., Christodoulides N., Floriano P. N., Graham S., Mohanty S., Dixon M., Hsiang M., Peter T., Zavahir S., Thior I., Romanovicz D., Bernard B., Goodey A. P., Walker B. D., McDevitt J. T., “A microchip CD4 counting method for HIV monitoring in resource-poor settings,” PLoS Med. 2(7), e182 (2005). doi: 10.1371/journal.pmed.0020182. [PMC free article] [PubMed] [Cross Ref]
16. Rahman M. S., Ingole N., Roblyer D., Stepanek V., Richards-Kortum R., Gillenwater A., Shastri S., Chaturvedi P., “Evaluation of a low-cost, portable imaging system for early detection of oral cancer,” Head Neck Oncol 2(1), 10 (2010). doi: 10.1186/1758-3284-2-10. [PMC free article] [PubMed] [Cross Ref]
17. Pang S., Cui X., DeModena J., Wang Y. M., Sternberg P., Yang C., “Implementation of a color-capable optofluidic microscope on a RGB CMOS color sensor chip substrate,” Lab Chip 10(4), 411–414 (2010). doi: 10.1039/b919004j. [PubMed] [Cross Ref]
18. Balsam J., Ossandon M., Kostov Y., Bruck H. A., Rasooly A., “Lensless CCD-based fluorometer using a micromachined optical Söller collimator,” Lab Chip 11(5), 941–949 (2011). doi: 10.1039/c0lc00431f. [PubMed] [Cross Ref]
19. Breslauer D. N., Maamari R. N., Switz N. A., Lam W. A., Fletcher D. A., “Mobile phone based clinical microscopy for global health applications,” PLoS ONE 4(7), e6320 (2009). doi: 10.1371/journal.pone.0006320. [PMC free article] [PubMed] [Cross Ref]
20. Hauck T. S., Giri S., Gao Y., Chan W. C. W., “Nanotechnology diagnostics for infectious diseases prevalent in developing countries,” Adv. Drug Deliv. Rev. 62(4-5), 438–448 (2010). doi: 10.1016/j.addr.2009.11.015. [PubMed] [Cross Ref]
21. Li X., Tian J., Nguyen T., Shen W., “Paper-based microfluidic devices by plasma treatment,” Anal. Chem. 80(23), 9131–9134 (2008). doi: 10.1021/ac801729t. [PubMed] [Cross Ref]
22. Vannahme C., Klinkhammer S., Lemmer U., Mappes T., “Plastic lab-on-a-chip for fluorescence excitation with integrated organic semiconductor lasers,” Opt. Express 19(9), 8179–8186 (2011). doi: 10.1364/OE.19.008179. [PubMed] [Cross Ref]
23. Miller A. R., Davis G. L., Oden Z. M., Razavi M. R., Fateh A., Ghazanfari M., Abdolrahimi F., Poorazar S., Sakhaie F., Olsen R. J., Bahrmand A. R., Pierce M. C., Graviss E. A., Richards-Kortum R., “Portable, battery-operated, low-cost, bright field and fluorescence microscope,” PLoS ONE 5(8), e11890 (2010). doi: 10.1371/journal.pone.0011890. [PMC free article] [PubMed] [Cross Ref]
24. Wong D. T., “Salivary diagnostics powered by nanotechnologies, proteomics and genomics,” J. Am. Dent. Assoc. 137(3), 313–321 (2006). [PubMed]
25. Desai D., Wu G., Zaman M. H., “Tackling HIV through robust diagnostics in the developing world: current status and future opportunities,” Lab Chip 11(2), 194–211 (2011). doi: 10.1039/c0lc00340a. [PubMed] [Cross Ref]
26. Goddard G., Martin J. C., Graves S. W., Kaduchak G., “Ultrasonic particle-concentration for sheathless focusing of particles for analysis in a flow cytometer,” Cytometry A 69A(2), 66–74 (2006). doi: 10.1002/cyto.a.20205. [PubMed] [Cross Ref]
27. Kuhn W., Armstrong D., Atteberry S., Dewbrey E., Smith D., Hooper N., “Usefulness of the ParalensTM fluorescent microscope adaptor for the identification of mycobacteria in both field and laboratory settings,” Open Microbiol J 4(1), 30–33 (2010). doi: 10.2174/1874285801004010030. [PMC free article] [PubMed] [Cross Ref]
28. Isikman S. O., Bishara W., Mavandadi S., Yu F. W., Feng S., Lau R., Ozcan A., “Lens-free optical tomographic microscope with a large imaging volume on a chip,” Proc. Natl. Acad. Sci. U.S.A. 108(18), 7296–7301 (2011). doi: 10.1073/pnas.1015638108. [PubMed] [Cross Ref]
29. Mudanyali O., Tseng D., Oh C., Isikman S. O., Sencan I., Bishara W., Oztoprak C., Seo S., Khademhosseini B., Ozcan A., “Compact, light-weight and cost-effective microscope based on lensless incoherent holography for telemedicine applications,” Lab Chip 10(11), 1417–1428 (2010). doi: 10.1039/c000453g. [PMC free article] [PubMed] [Cross Ref]
30. Zhu H., Yaglidere O., Su T. W., Tseng D., Ozcan A., “Cost-effective and compact wide-field fluorescent imaging on a cell-phone,” Lab Chip 11(2), 315–322 (2011). doi: 10.1039/c0lc00358a. [PMC free article] [PubMed] [Cross Ref]
31. Bishara W., Su T. W., Coskun A. F., Ozcan A., “Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution,” Opt. Express 18(11), 11181–11191 (2010). doi: 10.1364/OE.18.011181. [PMC free article] [PubMed] [Cross Ref]
32. Bishara W., Sikora U., Mudanyali O., Su T. W., Yaglidere O., Luckhart S., Ozcan A., “Holographic pixel super-resolution in portable lensless on-chip microscopy using a fiber-optic array,” Lab Chip 11(7), 1276–1279 (2011). doi: 10.1039/c0lc00684j. [PMC free article] [PubMed] [Cross Ref]
33. J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts & Company Publishers, 2005).
34. Fienup J. R., “Reconstruction of an object from the modulus of its Fourier transform,” Opt. Lett. 3(1), 27–29 (1978). doi: 10.1364/OL.3.000027. [PubMed] [Cross Ref]
35. Koren G., Polack F., Joyeux D., “Iterative algorithms for twin-image elimination in in-line holography using finite-support constraints,” J. Opt. Soc. Am. A 10(3), 423–433 (1993). doi: 10.1364/JOSAA.10.000423. [Cross Ref]
36. Allen L. J., Oxley M. P., “Phase retrieval from series of images obtained by defocus variation,” Opt. Commun. 199(1-4), 65–75 (2001). doi: 10.1016/S0030-4018(01)01556-5. [Cross Ref]
37. Elser V., “Phase retrieval by iterated projections,” J. Opt. Soc. Am. A 20(1), 40–55 (2003). doi: 10.1364/JOSAA.20.000040. [PubMed] [Cross Ref]
38. Faulkner H. M. L., Rodenburg J. M., “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93(2), 023903 (2004). doi: 10.1103/PhysRevLett.93.023903. [PubMed] [Cross Ref]
39. Maiden A. M., Rodenburg J. M., Humphry M. J., “Optical ptychography: a practical implementation with useful resolution,” Opt. Lett. 35(15), 2585–2587 (2010). doi: 10.1364/OL.35.002585. [PubMed] [Cross Ref]
40. Waller L., Tian L., Barbastathis G., “Transport of Intensity phase-amplitude imaging with higher order intensity derivatives,” Opt. Express 18(12), 12552–12561 (2010). doi: 10.1364/OE.18.012552. [PubMed] [Cross Ref]
41. Barone-Nugent E. D., Barty A., Nugent K. A., “Quantitative phase-amplitude microscopy I: optical microscopy,” J. Microsc. 206(3), 194–203 (2002). doi: 10.1046/j.1365-2818.2002.01027.x. [PubMed] [Cross Ref]
42. P. Thibault and C. University, Algorithmic Methods in Diffraction Microscopy(Cornell University, 2007).
43. Miao J., Kirz J., Sayre D., “The oversampling phasing method,” Acta Crystallogr. D Biol. Crystallogr. 56(10), 1312–1315 (2000). doi: 10.1107/S0907444900008970. [PubMed] [Cross Ref]
44. Miao J., Sayre D., Chapman H. N., “Phase retrieval from the magnitude of the Fourier transforms of nonperiodic objects,” J. Opt. Soc. Am. A 15(6), 1662–1669 (1998). doi: 10.1364/JOSAA.15.001662. [Cross Ref]
45. Marchesini S., “Invited article: a unified evaluation of iterative projection algorithms for phase retrieval,” Rev. Sci. Instrum. 78(1), 011301 (2007). doi: 10.1063/1.2403783. [PubMed] [Cross Ref]
46. Fienup J. R., “Phase retrieval algorithms: a comparison,” Appl. Opt. 21(15), 2758–2769 (1982). doi: 10.1364/AO.21.002758. [PubMed] [Cross Ref]
47. Latychevskaia T., Fink H. W., “Solution to the twin image problem in holography,” Phys. Rev. Lett. 98(23), 233901 (2007). doi: 10.1103/PhysRevLett.98.233901. [PubMed] [Cross Ref]
48. Millane R., “Phase retrieval in crystallography and optics,” J. Opt. Soc. Am. A 7(3), 394–411 (1990). doi: 10.1364/JOSAA.7.000394. [Cross Ref]
49. Zhang F., Pedrini G., Osten W., “Phase retrieval of arbitrary complex-valued fields through aperture-plane modulation,” Phys. Rev. A 75(4), 043805 (2007). doi: 10.1103/PhysRevA.75.043805. [Cross Ref]
50. Chen C. C., Miao J., Wang C. W., Lee T. K., “Application of optimization technique to noncrystalline x-ray diffraction microscopy: guided hybrid input-output method,” Phys. Rev. B 76(6), 064113 (2007). doi: 10.1103/PhysRevB.76.064113. [Cross Ref]
51. Fienup J. R., Wackerman C. C., “Phase-retrieval stagnation problems and solutions,” J. Opt. Soc. Am. A 3(11), 1897–1907 (1986). doi: 10.1364/JOSAA.3.001897. [Cross Ref]
52. Oh C., Isikman S. O., Khademhosseinieh B., Ozcan A., “On-chip differential interference contrast microscopy using lensless digital holography,” Opt. Express 18(5), 4717–4726 (2010). doi: 10.1364/OE.18.004717. [PMC free article] [PubMed] [Cross Ref]
53. Mudanyali O., Bishara W., Ozcan A., “Lensfree super-resolution holographic microscopy using wetting films on a chip,” Opt. Express 19(18), 17378–17389 (2011). doi: 10.1364/OE.19.017378. [PMC free article] [PubMed] [Cross Ref]
54. Fienup J. R., Kowalczyk A. M., “Phase retrieval for a complex-valued object by using a low-resolution image,” J. Opt. Soc. Am. A 7(3), 450–458 (1990). doi: 10.1364/JOSAA.7.000450. [Cross Ref]
55. Biener G., Greenbaum A., Isikman S. O., Lee K., Tseng D., Ozcan A., “Combined reflection and transmission microscope for telemedicine applications in field settings,” Lab Chip 11(16), 2738–2743 (2011). doi: 10.1039/c1lc20169g. [PMC free article] [PubMed] [Cross Ref]
56. Crimmins T. R., Fienup J. R., Thelen B. J., “Improved bounds on object support from autocorrelation support and application to phase retrieval,” J. Opt. Soc. Am. A 7(1), 3–13 (1990). doi: 10.1364/JOSAA.7.000003. [Cross Ref]
57. Paganin D., Nugent K. A., “Noninterferometric phase imaging with partially-coherent light,” Phys. Rev. Lett. 80(12), 2586–2589 (1998). doi: 10.1103/PhysRevLett.80.2586. [Cross Ref]
58. Zysk A. M., Schoonover R. W., Carney P. S., Anastasio M. A., “Transport of intensity and spectrum for partially coherent fields,” Opt. Lett. 35(13), 2239–2241 (2010). doi: 10.1364/OL.35.002239. [PMC free article] [PubMed] [Cross Ref]
59. Gerchberg R. W., Saxton W. O., “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik (Stuttg.) 35, 237–246 (1971).
60. Zhang Y., Pedrini G., Osten W., Tiziani H. J., “Reconstruction of in-line digital holograms from two intensity measurements,” Opt. Lett. 29(15), 1787–1789 (2004). doi: 10.1364/OL.29.001787. [PubMed] [Cross Ref]
61. Das B., Yelleswarapu C. S., “Dual plane in-line digital holographic microscopy,” Opt. Lett. 35(20), 3426–3428 (2010). doi: 10.1364/OL.35.003426. [PubMed] [Cross Ref]
62. Paxman R. G., Schulz T. J., Fienup J. R., “Joint estimation of object and aberrations by using phase diversity,” J. Opt. Soc. Am. A 9(7), 1072–1085 (1992). doi: 10.1364/JOSAA.9.001072. [Cross Ref]
63. Vogel C. R., Chanb T., Plemmons R., “Fast algorithms for phase diversity-based blind deconvolution,” Proc. SPIE 3353, 994–1005 (1998). doi: 10.1117/12.321720. [Cross Ref]
64. Noort M., Voort L. R. V., Löfdahl M. G., “Solar image restoration by use of multi-frame blind de-convolution with multiple objects and phase diversity,” Sol. Phys. 228(1-2), 191–215 (2005). doi: 10.1007/s11207-005-5782-z. [Cross Ref]
65. Hardie R. C., Barnard K. J., Armstrong E. E., “Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,” IEEE Trans. Image Process. 6(12), 1621–1633 (1997). doi: 10.1109/83.650116. [PubMed] [Cross Ref]
66. Park S., Park M., Kang M., “Super-resolution image reconstruction: a technical overview,” IEEE Sig. Process. Mag. 20(3), 21–36 (2003). doi: 10.1109/MSP.2003.1203207. [Cross Ref]
67. Hardie R. C., Barnard K. J., Bognar J. G., Armstrong E. E., Watson E. A., “High resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247 (1998). doi: 10.1117/1.601623. [Cross Ref]
68. J. L. Pech-Pacheco, G. Cristóbal, J. Chamorro-Martínez, and J. Fernández-Valdivia, “Diatom autofocusing in brightfield microscopy: a comparative study,” in Proceedings of 15th International Conference On Pattern Recognition,IEEE Computer Society, 2000 pp. 314–317.
69. Mudanyali O., Oztoprak C., Tseng D., Erlinger A., Ozcan A., “Detection of waterborne parasites using field-portable and cost-effective lensfree microscopy,” Lab Chip 10(18), 2419–2423 (2010). doi: 10.1039/c004829a. [PMC free article] [PubMed] [Cross Ref]
70. Fremont‐Smith M., Marino J., Griffin B., Spencer L., Bolick D., “Comparison of the SurepathTM liquid‐based Papanicolaou smear with the conventional Papanicolaou smear in a multisite direct‐to‐vial study,” Cancer Cytopathol. 102(5), 269–279 (2004). doi: 10.1002/cncr.20599. [PubMed] [Cross Ref]
71. Isikman S. O., Sencan I., Mudanyali O., Bishara W., Oztoprak C., Ozcan A., “Color and monochrome lensless on-chip imaging of Caenorhabditis elegans over a wide field-of-view,” Lab Chip 10(9), 1109–1112 (2010). doi: 10.1039/c001200a. [PMC free article] [PubMed] [Cross Ref]
72. Lee M., Yaglidere O., Ozcan A., “Field-portable reflection and transmission microscopy based on lensless holography,” Biomed. Opt. Express 2(9), 2721–2730 (2011). doi: 10.1364/BOE.2.002721. [PMC free article] [PubMed] [Cross Ref]

Articles from Optics Express are provided here courtesy of Optical Society of America