|Home | About | Journals | Submit | Contact Us | Français|
We demonstrate lensfree digital microscopy on a cellphone. This compact and light-weight holographic microscope installed on a cellphone does not utilize any lenses, lasers or other bulky optical components and it may offer a cost-effective tool for telemedicine applications to address various global health challenges. Weighing ~38 grams (<1.4 ounces), this lensfree imaging platform can be mechanically attached to the camera unit of a cellphone where the samples are loaded from the side, and are vertically illuminated by a simple light-emitting diode (LED). This incoherent LED light is then scattered from each micro-object to coherently interfere with the background light, creating the lensfree hologram of each object on the detector array of the cellphone. These holographic signatures captured by the cellphone permit reconstruction of microscopic images of the objects through rapid digital processing. We report the performance of this lensfree cellphone microscope by imaging various sized micro-particles, as well as red blood cells, white blood cells, platelets and a waterborne parasite (Giardia lamblia).
Today there are more than 4 billion cellphone users in the world, and the majority of these cellphones are being used in the developing parts of the world. In addition to this, 80% of the world population currently lives in locations that are already covered through wireless communication links such as GSM networks.1 These numbers will further grow in the near-future, where it is predicted that by 2015 more than 86% of the world population will own at least one cellphone.2
This massive volume of wireless phone communication brings an enormous cost-reduction to cellphones despite their sophisticated hardware and software capabilities. Quite importantly, most of these existing cellphones are already equipped with advanced digital imaging and sensing platforms that can be utilized for various health monitoring applications. As also detailed in a recent article,3 utilizing this advanced state-of-the-art cellphone technology towards medical diagnostics, imaging and/or sensing applications can offer numerous opportunities to improve health care especially in the developing world where medical facilities and infrastructure are extremely limited or even do not exist. For this revolution in health care to occur, lab-on-a-chip systems that can potentially be miniaturized to the level of a handheld wireless unit are needed. Ideally, such systems should be compatible with the existing designs of cellphones, which will make their wide-spread use in the developing world much more cost-effective.
In the mean time, several diagnostic tests today are still conducted using conventional optical microscopes that are not always compatible with compact wireless systems to be used in resource limited settings. To address this important need, an important research direction is to miniaturize the existing designs of lens-based microscopes to be compatible with cellphones, while making use of the installed digital camera of the phone. Recent results successfully demonstrated both the significance and the feasibility of this approach.4
To provide a complementary effort to this important need in telemedicine, here we illustrate the proof-of-concept of an alternative digital microscopy platform running on a cellphone. This holographic cellphone microscope does not utilize any lenses, lasers or other bulky optical components which greatly simplifies its architecture making it extremely compact and lightweight, such that only ~38 grams (~1.34 ounces) of attachment to the cellphone is required – see Fig. 1. Instead of using a coherent light source (e.g., a laser) as in conventional holography approaches,5,6 here we utilize an inexpensive light-emitting-diode (LED) to vertically illuminate the samples, which can be mechanically loaded into the cellphone from the side as illustrated in Fig. 1. This incoherent LED light is initially filtered by passing it through a large aperture of ~100 μm diameter to better control the spatial coherence of the illumination at the sensor plane. This large aperture also provides orders of magnitude improved light transmission efficiency as well as tolerance to misalignments. This spatially filtered LED light, after travelling in air a distance of ~4 cm (see Fig. 1b), interacts with the sample of interest, where each cell/particle within the sample scatters and refracts the incoming LED light based on its size, 3D morphology, sub-cellular elements, and refractive index. The interference of the light waves that passed through the cells with the unscattered LED light creates the hologram of each cell, which is detected using the CMOS (complementary metal-oxide semiconductor) detector array that is already installed on the cellphone camera unit. The lensfree hologram of each cell is extremely rich (despite the simplicity of the recording geometry) and permits rapid reconstruction of its microscopic image through digital processing.
In this manuscript, we demonstrate the performance of this lensfree cellphone microscope by successfully imaging various sized micro-particles, as well as red blood cells, white blood cells, platelets and a waterborne parasite such as a Giardia lamblia cyst. We believe that this compact and light-weight microscopy platform running on a cellphone could be exceedingly important especially for various global health problems by permitting infectious disease diagnosis from bodily fluids, as well as screening of the quality of water resources.
To convert a cellphone into a microscope to be used in tele-medicine applications there are several approaches that one can take. If the cellphone does not have an installed camera unit on it, one can create an ultra-compact and light-weight digital microscope that can attach to the cellphone through e.g., a USB interface. Such a unique combination of a cellphone and a compact microscope can be used for telemedicine purposes to wirelessly transmit the acquired images to a remote location such as a central hospital or a clinic. In this work, however, we have explored the use of the existing digital camera unit of the cell-phone to create a microscope as part of the cellphone. For this task, we have adapted lensfree holographic on-chip imaging7–9 and made it compatible with an existing camera-cellphone to conduct microscopy on a cellphone.
Lensfree holographic microscopy can be performed in various different formats depending on the choice of the light source, the detector-array, as well as the hologram recording geometry.7–14 In this work, rather than a laser source, we used an inexpensive LED (at ~587 nm with a spectral width of ~20 nm) for illumination of the samples. This choice not only reduced the device cost, but also significantly weakened the speckle noise and the multiple-interference artifacts from our images. Furthermore, with a unit fringe magnification geometry,7–9 we used the color sensor installed on the camera-phone to record the lensfree holograms of the cells. This color sensor chip, unlike a monochrome one, has color filters at each pixel yielding what is known as the Bayer pattern composed of a periodic array of red-green-blue (RGB) pixels. In a regular lensfree holographic microscope, a color sensor would hardly be the optimal choice, since not all the pixels would receive enough light under quasi-monochromatic illumination, which in our case is at ~587 nm. To handle this issue of hologram distortion due to the Bayer pattern of the camera-phone, our digital image reconstruction process involved an extra step of converting the raw format (Bayer Pattern Image) into a monochrome equivalent image before conducting holographic reconstruction of the cells’ images. The technical details of this holographic processing of the color sensor output will be provided in the Methods section.
To demonstrate the performance of our lensfree cellphone microscope, we imaged several micro-particles (with diameters of 3, 7 and 10 μm), as well as red blood cells, white blood cells, and platelets, the results of which are summarized in Fig. 2. In these experiments, the disposable samples were inserted into the lensless cellphone microscope from the side for detection of their holographic shadows, illustrated in Fig. 2, right column. These holographic shadows were then digitally processed using a custom-developed reconstruction algorithm to create the microscopic images of the samples shown in the same figure within the middle column. Because of its significance for global health, we also imaged a waterborne parasite, namely a Giardia lamblia cyst, with our lensfree cellphone microscope, the result of which is also illustrated in Fig. 2, bottom row. For comparison purposes, the same objects were also imaged using a conventional lens-based microscope (see Fig. 2, left column), providing a decent match to our imaging results. The spatial resolution of our reconstructions, under unit fringe magnification, is limited by the pixel size at the sensor, which in our case was ~2.2 μm for each color pixel. For a monochrome sensor of the same pixel size, we can normally achieve a sub-pixel resolution of ~1.5–2 μm,8,9 however, the distortions and the lower signal to noise ratio introduced by the color filters under quasi-monochromatic illumination relatively degrades the spatial resolution when compared to a monochrome sensor. With recently emerging sub-micron pixel CMOS sensors,15 the resolution could be further enhanced to claim <1 μm.
An important benefit of this lensfree cellphone microscope towards telemedicine and global health related applications could be to bring the function of microscopy to remote locations for performing more accurate medical diagnostics or even for screening of water quality in resource poor environments. For this end, in an ideal setting, the cellphone itself should be used not only for the actual holographic image acquisition, but also for wireless transmission of the raw images together with other related information (such as demographic data of the patient, the location, etc.) to a central computer installed in e.g., a clinic or a hospital for gathering larger sets of data from remote locations. This implies that the holographic reconstruction process can ideally be performed remotely such that the computational burden on the cellphone hardware can be significantly reduced. Using a cost-effective graphics processing unit (GPU) installed in e.g., a central hospital, the digital image reconstruction from the raw holograms captured by our cellphone microscope can be achieved within less than 1 s, i.e., the end-users can quickly have access to the reconstructed microscopic images through wireless communication.
The above discussed model of operation surely relaxes the hardware and software requirements on the cellphone unit, which is the right step to further reduce the cost, size and the weight of the lensless microscopy platform. However, it puts the entire burden on wireless data communication, which can be expensive and time consuming depending on the status of the network. To mitigate this problem, we investigated the compressibility of our raw holographic images to better estimate the minimum amount of digital holographic data to be transmitted over the wireless link without an apparent loss of resolution in the microscope image. As a result of this study (see Fig. 3), we found out that the recorded holograms with our lensfree cellphone microscope are highly compressible, such that for one pixel of the holographic image it would only be necessary to transmit (on average) ~3 Bits in PNG (portable network graphics) format, which provides lossless compression. This implies that 1 Mega Pixel worth of raw holographic data would require transmission of only ~0.375 Mbytes. In other words, to transmit a holographic image that contains the microscopic information corresponding to a field of view of ~5 mm2, we would need to transfer only 0.375 Mbytes over the wireless network. Fig. 3 summarizes these results by showing how the reconstructed image is affected as one quantizes the holographic image using different bit depths and saves it in PNG format. These results imply that we can quickly communicate back and forth between the lensfree cellphone microscope and the central processing unit with much smaller data rates (compatible with GSM networks) reducing the total cost of wireless data transfer without degrading the microscopic image quality.
A digital color image is represented as an array of pixels, with each pixel represented by a mixture of primary colors. The standard primary colors used by most of the consumer cameras are red, green and blue (RGB). In an ideal case, these colors can be recorded separately by splitting the light beam onto three different sensors, each recording one color. However, for cost reasons, a mobile phone camera uses a single image sensor chip which is covered by a Color Filter Array (CFA) designed in a variety of patterns. The most widely used CFA pattern in the image acquisition industry is called the Bayer pattern which employs a repeating 2 × 2 pattern consisting of one blue, one red and two green filters. Therefore, the raw output of a sensor using Bayer Pattern CFA, which is usually called the Bayer Pattern Image, is made of pixels which carry information regarding one of the three primary channels. The process of merging these 3 channels in order to obtain a full-color image is called demosaicing. There is an ample amount of literature on different methods for demosaicing each of which answers the needs of different applications.16 However, for the purpose of holographic cellphone microscopy as illustrated in Fig. 1, such standard demosaicing algorithms would wash out high frequency amplitude oscillations which are crucial in holographic reconstruction process. Therefore, the usage of the recorded information in its most pure format has a significant advantage of preventing any undesired artifacts that might be introduced by conventional demosaicing algorithms. For preserving the holographic diffraction signatures of microscopic objects, we developed our own demosaicing algorithm (summarized in Fig. 4) to obtain grayscale images with least distortion to the acquired holographic patterns. Unlike conventional demosaicing algorithms where it is aimed to output an RGB image by interpolating missing channels at each pixel while preserving inter-pixel as well as inter-channel correlation, the main aim of our demosaicing algorithm was to maximize spatial correlation. Therefore, the raw output of the cellphone is treated as a monochrome image which has patterned artifacts to be ameliorated (see Fig. 4).
For a lensfree holographic pattern sampled by a color-sensor, the illumination wavelength is quite important in assuring an optimal spatial sampling performance. As stated earlier, 50% of the pixels on a color sensor which uses Bayer Pattern CFA are responsive to green, 25% to blue and 25% to red. Because it is desired to have as many un-saturated pixels above noise level as possible, the wavelength of our LED source is selected to be in a band where both red and green pixels have high detection efficiency. Therefore, an LED at ~587 nm was used have decent performance for the red and green channels. However, under this quasi-monochromatic illumination, the resulting raw holographic image at the color-sensor mainly suffers from two artifacts. First, even though red and green channels carry information with high signal to noise ratio, they are not equally illuminated and therefore equalization needs to be carried out between the values belonging to these two channels. Second, as a result of selecting a wavelength at which blue pixels are not sensitive enough, the third channel (blue) is highly corrupted by the noise. Hence it is required to predict all the blue pixels using neighboring green and red pixels.
In this work, the detection imbalance between the intensity levels of green and red channels is compensated using a background image acquired with identical illumination conditions that were used for capture of lensfree holograms of the objects. This background image provides a normalization coefficient matrix which determines the scaling factor for each pixel on the holographic image. This method not only equalizes the green and red channels, but also compensates for any potential artifact caused by non-uniform illumination at the sensor plane.
Once this channel equalization step is done, the remaining problem is the prediction of the missing blue channel. Our approach for interpolation of the blue pixels includes an estimation step, which is done by using an edge-aware interpolation, followed by a refinement step which improves this initial prediction iteratively by using an altered version of the phase recovery method17–19 that we adapted for reconstruction of lensfree holographic images.8,9
When a larger block of 3 × 3 pixels is considered, this missing channel prediction problem may also be interpreted as estimation of a missing pixel (blue) that is surrounded by 8 known pixels (red and green). The simplest way to estimate this unknown pixel is straight-forward averaging of all the eight neighboring pixels. However, such an approach would oversee high frequency changes in the lensfree hologram. Instead, we used an edge-aware interpolation algorithm,20 which adjusts the estimation of the missing pixels based on the magnitudes of the spatial derivatives in each of the four directions. To be more specific, if the missing blue pixel is denoted by X and the surrounding pixels by Zij where i represents one of the four main directions (2 along x and y, and 2 along the diagonal directions), and j represents one of two known pixels in each direction, then X can be estimated as a weighted sum of Zij’s:
where the weighting factors are calculated using the spatial derivative in each one of the four directions:
In the last equation, α is a normalization constant such that the sum of all the weighting factors for each blue pixel is equal to 1. As a result of this choice of weighting factors, the linear interpolation at each direction is penalized by using the magnitude of the spatial derivative in that direction. Such an edge-aware interpolation approach ensures that the missing blue pixel estimation is carried out by weighing the pixel values on directions with lower gradient more than others.
Even though the output of the above described process becomes free from Bayer pattern artifacts with high spatial correlation over neighboring pixels, some discontinuity and blurring effects may still be observed, especially over rapidly oscillating fringes of the cell holograms. In other words, at this stage of our algorithm, the problem is now reduced to a point where only a finer set of oscillations, which defines the object at the hologram domain, needs to be corrected to obtain an acceptable image reconstruction. To provide a solution to this need, our hologram phase recovery method8,9 was modified and utilized for further refinement of the predicted blue pixels. In its original version, this phase recovery method was used for digital recovery of the lost phase information from the hologram intensity. It is based on back propagating the amplitude of the hologram onto object plane and replacing the complex field outside of the object support with a background value; then propagating back to the detector plane and updating the amplitude of the complex field to be the same as the original measured one. In the adjusted version of this iterative phase recovery approach, a second spatial mask which identifies the locations of the predicted blue pixels is applied on the hologram plane, enabling the update of the amplitude as well as the phase information exclusively for these blue pixels. The iterations stop when the average change over amplitude of the pixels at a given iteration reaches a minimum threshold. In this work the number of such iterations needed for convergence was typically ~10.
In summary, through the above described numerical approach, the amplitude of the estimated blue pixels is refined iteratively, converging to better defined holographic oscillations with higher contrast. The resulting cleaned holograms, as illustrated in Fig. 2, right column, are then processed to provide digital images of microscopic objects using our holographic reconstruction algorithm.8,9
We used MotoZine ZN5 from Motorola as the starting base of our lensfree cellphone microscope. However, we should also emphasize that the presented technique can easily be installed on various other camera cell-phones (independent of the operating system, the communication protocol or the service provider). This cellphone has a 5 Mpixel color (RGB) sensor installed on it, which we used to capture the raw lensfree holograms as described in earlier sections. The LED source (OSRAM Opto Semiconductors Inc., Part# LY E63B-CBEA-26-1-Z – Center wavelength: 587 nm, Bandwidth: ~20 nm) is butt-coupled to a ~100 μm pinhole without the use of any focusing or alignment optics, illuminating the entire FOV of ~23.8 mm2 of the cell-phone camera chip. Following Fig. 1, typical distances used in our design are ~4 cm between the source and the sample; and <2 mm between the sample and the sensor planes. The LED source is powered through a flat-battery installed at the top of the unit. The disposable samples to be imaged are loaded into the lensfree cellphone microscope within a mechanical tray from the side as illustrated in Fig. 1. This entire holographic microscope unit that attaches to the cellphone body weighs ~38 grams (~1.34 ounces) including the battery, the LED, the sample tray and the other mechanical components (see Fig. 1b).
In terms of the cost of the cellphone modification, excluding labor and the custom-made plastic components shown in Fig. 1, the other significant components are the LED (0.35 USD per piece for <10 units; 0.18 USD per piece for ~2000 units) and the flat-battery (0.2 USD per piece for ≥100 units), the cost of which can be further reduced through mass-production.
For blood imaging experiments, whole blood samples were treated with 2.0 mg EDTA/ml; and 1 μL of sample was dropped on the top of a type 1 glass cover slip and another type 1 cover slip was used for spreading and smearing the blood droplet over the entire cover slip with about 30 degree of smearing angle. The smeared specimen was air-dried for 5 min before being fixed and stained by HEMA 3 Wright-Giemsa staining kit (Fisher Diagnostics). Dipping dried samples into three Coplin jars which contain methanol based HEMA 3 fixative solution, eosinophilic staining solution (HEMA 3 solution I) and basophilic solution (HEMA 3 solution II), respectively, was performed five times in a row for one second each step. Then, the specimen was rinsed with de-ionized water and air-dried again before being imaged using the lensfree cellphone microscope. These blood smear samples can also be stored in a fridge for at least a few weeks for further imaging.
For the waterborne parasite and micro-particle imaging experiments, the samples were imaged within a solution. Giardia lamblia cysts (WaterBorne Inc., USA) were fixed in 5% Formalin/PBS at pH 7.4/0.01% Tween 20. Polystyrene microbeads (Duke Scientific, USA) were suspended in 1× phosphate buffered saline (PBS) solution. Diluted sample solutions are placed using a micro pipette between two identical glass cover slides or within a micro-fluidic channel to be imaged. Other details of sample preparation steps can be found in ref. 21.
In conclusion, we have demonstrated lensfree digital microscopy on a cellphone. This compact and light-weight holographic microscope installed on a cellphone does not utilize any lenses, lasers or other bulky optical components. Weighing ~38 grams, this cost-effective microscopy platform can be mechanically attached to the camera unit of a cellphone, where the samples are loaded from the side, and are vertically illuminated by a simple LED. This incoherent LED light is then scattered from each micro-object to coherently interfere with the background light creating the lensfree hologram of each object on the detector array of the cellphone. Each of these holographic signatures captured by the cellphone permits reconstruction of its microscopic image through rapid digital processing. We have successfully demonstrated the performance of this lensfree cell-phone microscope by imaging various sized micro-particles, as well as red blood cells, white blood cells, platelets and water-borne parasites such as Giardia lamblia cysts. We believe that this compact and light-weight microscopy platform running on a cellphone could be very important especially for various global health problems by providing infectious disease diagnosis from bodily fluids, as well as rapid screening of the quality of water resources.
The authors acknowledge the support of the Okawa Foundation, Vodafone Americas Foundation, DARPA DSO (under 56556-MS-DRP), NSF BISH program (under Awards # 0754880 and 0930501), NIH (under 1R21EB009222-01 and the NIH Director’s New Innovator Award – Award Number DP2OD006427 from the Office of The Director, National Institutes of Health), AFOSR (under Project # 08NE255). A. Ozcan also gratefully acknowledges the support of the Office of Naval Research (ONR) under the Young Investigator Award 2009.