Bundle adjustment (BA) is a common estimation algorithm that is widely used in machine vision as the last step in a feature-based three-dimensional (3D) reconstruction algorithm. BA is essentially a non-convex non-linear least-square problem that can simultaneously solve the 3D coordinates of all the feature points describing the scene geometry, as well as the parameters of the camera. The conventional BA takes a parameter either as a fixed value or as an unconstrained variable based on whether the parameter is known or not. In cases where the known parameters are inaccurate but constrained in a range, conventional BA results in an incorrect 3D reconstruction by using these parameters as fixed values. On the other hand, these inaccurate parameters can be treated as unknown variables, but this does not exploit the knowledge of the constraints, and the resulting reconstruction can be erroneous since the BA optimization halts at a dramatically incorrect local minimum due to its non-convexity. In many practical 3D reconstruction applications, unknown variables with range constraints are usually available, such as a measurement with a range of uncertainty or a bounded estimate. Thus to better utilize these pre-known, constrained, but inaccurate parameters, a bound constrained bundle adjustment (BCBA) algorithm is proposed, developed and tested in this study. A scanning fiber endoscope (the camera) is used to capture a sequence of images above a surgery phantom (the object) of known geometry. 3D virtual models are reconstructed based on these images and then compared with the ground truth. The experimental results demonstrate BCBA can achieve a more reliable, rapid, and accurate 3D reconstruction than conventional bundle adjustment.
(150.1135) Algorithms; (150.0150) Machine vision; (110.6880) Three-dimensional image acquisition; (100.3010) Image reconstruction techniques
Although the single-shot focus scanning technique (SSFS) has been experimentally demonstrated for extended depth of field (EDOF) imaging, few work has been performed to characterize its imaging properties and limitations. In this paper, based on an analytical model of a SSFS system, we examined the properties of the system response and the restored image quality in relation to the axial position of the object, scan range, and signal-to-noise ratio, and demonstrated the properties via a prototype of 10 × 0.25 NA microscope system. We quantified the full range of the achievable EDOF is equivalent to the focus scan range. We further demonstrated that the restored image quality can be improved by extending the focus scan range by a distance equivalent to twice of the standard DOF. For example, in a focus-scanning microscope with a ± 15 μm standard DOF, a 120 μm focus scan range can obtain a ± 60 μm EDOF, but a 150 μm scan range affords noticeably better EDOF images for the same EDOF range. These results provide guidelines for designing and implementing EDOF systems using SSFS technique.
(110.0180) Microscopy; (110.4850) Optical transfer functions; (100.3020) Image reconstruction-restoration; (080.3620) Lens system design
We present a polarization rotation and coupling scheme that rotates a TE0 mode in a silicon waveguide and simultaneously couples the rotated mode to a hybrid plasmonic (HP0) waveguide mode. Such a polarization rotation can be realized with a partially etched asymmetric hybrid plasmonic waveguide consisting of a silicon strip waveguide, a thin oxide spacer, and a metal cap made from copper, gold, silver or aluminum. Two implementations, one with and one without the tapering of the metal cap are presented, and different taper shapes (linear and exponential) are also analyzed. The devices have large 3 dB conversion bandwidths (over 200 nm at near infrared) and short length (< 5 μm), and achieve a maximum coupling factor of ∼ 78% with a linearly tapered silver metal cap.
(230.3120) Integrated optics devices; (250.5403) Plasmonics; (130.5440) Polarization-selective devices
We propose methods to align interferograms affected by trigger jitter to a reference interferogram based on the information (amplitude/phase) at a fixed-pattern noise location to reduce residual fixed-pattern noise and improve the phase stability of swept source optical coherence tomography (SS-OCT) systems. One proposed method achieved this by introducing a wavenumber shift (k-shift) in the interferograms of interest and searching for the k-shift that minimized the fixed-pattern noise amplitude. The other method calculated the relative k-shift using the phase information at the residual fixed-pattern noise location. Repeating this wavenumber alignment procedure for all A-lines of interest produced fixed-pattern noise free and phase stable OCT images. A system incorporating these correction routines was used for human retina OCT and Doppler OCT imaging. The results from the two methods were compared, and it was found that the intensity-based method provided better results.
(170.4500) Optical coherence tomography; (170.3880) Medical and biological imaging; (350.5030) Phase
Stereoscopic 3D (S3D) displays use spatial or temporal interlacing to send different images to the two eyes. Temporal interlacing delivers images to the left and right eyes alternately in time; it has high effective spatial resolution but is prone to temporal artifacts. Spatial interlacing delivers even pixel rows to one eye and odd rows to the other eye simultaneously; it is subject to spatial limitations such as reduced spatial resolution. We propose a spatiotemporal-interlacing protocol that interlaces the left- and right-eye views spatially, but with the rows being delivered to each eye alternating with each frame. We performed psychophysical experiments and found that flicker, motion artifacts, and depth distortion are substantially reduced relative to the temporal-interlacing protocol, and spatial resolution is better than in the spatial-interlacing protocol. Thus, the spatiotemporal-interlacing protocol retains the benefits of spatial and temporal interlacing while minimizing or even eliminating the drawbacks.
(330.1400) Vision - binocular and stereopsis; (330.6130) Spatial resolution; (330.5510) Psychophysics; (120.2040) Displays; (100.2000) Digital image processing
Detection of blood flow inside the tissue sample can be achieved by measuring the local change of complex signal over time in angiographic optical coherence tomography (OCT). In conventional angiographic OCT, the transverse displacement of the imaging beam during the time interval between a pair of OCT signal measurements must be significantly reduced to minimize the noise due to the beam scanning-induced phase decorrelation at the expense of the imaging speed. Recent introduction of dual-beam scan method either using polarization encoding or two identical imaging systems in spectral-domain (SD) OCT scheme shows potential for high-sensitivity vasculature imaging without suffering from spurious phase noise caused by the beam scanning-induced spatial decorrelation. In this paper, we present multi-functional angiographic optical frequency domain imaging (OFDI) using frequency-multiplexed dual-beam illumination. This frequency multiplexing scheme, utilizing unique features of OFDI, provides spatially separated dual imaging beams occupying distinct electrical frequency bands that can be demultiplexed in the frequency domain processing. We demonstrate the 3D multi-functional imaging of the normal mouse skin in the dorsal skin fold chamber visualizing distinct layer structures from the intensity imaging, information about mechanical integrity from the polarization-sensitive imaging, and depth-resolved microvasculature from the angiographic imaging that are simultaneously acquired and automatically co-registered.
(110.4500) Optical coherence tomography; (170.3340) Laser Doppler velocimetry; (170.3880) Medical and biological imaging; (170.4500) Optical coherence tomography
Fluorescence microscopy is a photon-limited imaging modality that allows the study of subcellular objects and processes with high specificity. The best possible accuracy (standard deviation) with which an object of interest can be localized when imaged using a fluorescence microscope is typically calculated using the Cramér-Rao lower bound, that is, the inverse of the Fisher information. However, the current approach for the calculation of the best possible localization accuracy relies on an analytical expression for the image of the object. This can pose practical challenges since it is often difficult to find appropriate analytical models for the images of general objects. In this study, we instead develop an approach that directly uses an experimentally collected image set to calculate the best possible localization accuracy for a general subcellular object. In this approach, we fit splines, i.e. smoothly connected piecewise polynomials, to the experimentally collected image set to provide a continuous model of the object, which can then be used for the calculation of the best possible localization accuracy. Due to its practical importance, we investigate in detail the application of the proposed approach in single molecule fluorescence microscopy. In this case, the object of interest is a point source and, therefore, the acquired image set pertains to an experimental point spread function.
(100.2000) Digital image processing; (180.2520) Fluorescence microscopy; (030.5290) Photon statistics; (180.6900) Three-dimensional microscopy
Matching wavefront correctors and wavefront sensors by minimizing the condition number and mean wavefront variance is proposed. The particular cases of two continuous-sheet deformable mirrors and a Shack-Hartmann wavefront sensor with square packing geometry are studied in the presence of photon noise, background noise and electronics noise. Optimal number of lenslets across each actuator are obtained for both deformable mirrors, and a simple experimental procedure for optimal alignment is described. The results show that high-performance adaptive optics can be achieved even with low cost off-the-shelf Shack-Hartmann arrays with lenslet spacing that do not necessarily match those of the wavefront correcting elements.
A multi-source inverse-geometry CT (MS-IGCT) system consists of a small 2D detector array and multiple x-ray sources. During data acquisition, each source is activated sequentially, and may have random source intensity fluctuations relative to their respective nominal intensity. While a conventional 3rd generation CT system uses a reference channel to monitor the source intensity fluctuation, the MS-IGCT system source illuminates a small portion of the entire field-of-view (FOV). Therefore, it is difficult for all sources to illuminate the reference channel and the projection data computed by standard normalization using flat field data of each source contains error and can cause significant artifacts. In this work, we present a raw data normalization algorithm to reduce the image artifacts caused by source intensity fluctuation. The proposed method was tested using computer simulations with a uniform water phantom and a Shepp-Logan phantom, and experimental data of an ice-filled PMMA phantom and a rabbit. The effect on image resolution and robustness of the noise were tested using MTF and standard deviation of the reconstructed noise image. With the intensity fluctuation and no correction, reconstructed images from simulation and experimental data show high frequency artifacts and ring artifacts which are removed effectively using the proposed method. It is also observed that the proposed method does not degrade the image resolution and is very robust to the presence of noise.
(170.0110) Imaging systems; (170.3880) Medical and biological imaging; (170.6960) Tomography
We report an experimental technique where one uses a standard silica fiber as a cylindrical whispering gallery mode (WGM) resonator to sense airborne nanoscale aerosols produced by electric arc welding. We find that the accumulation of aerosols on the resonator surface induces a measurable red-shift in resonance frequency, and establish an empirical relation that links the magnitude of resonance shift with the amount of aerosol deposition. The WGM quality factors, by contrast, do not decrease significantly, even for samples with a large percentage of surface area covered by aerosols. Our experimental results are discussed and compared with existing literature on WGM-based nanoparticle sensing.
(060.2370) Fiber optics sensors; (140.3948) Microcavity devices; (230.5750) Resonators; (240.6690) Surface waves
In this paper, we propose a design for a 2D slab photonic crystal (PhC) virus sensor and an associated signal analysis methodology that together enable single-virus detection while rejecting false positives that occur due to non-specific interactions of serum proteins and small molecules with the sensor surface. The slab-PhC design takes advantage of both the optical and geometrical properties of its incorporated structures by physically limiting virus infiltration to only the most sensitive region of the PhC sensor, while allowing simultaneous measurement of both site-selective virus infiltration and non-specific small molecule accumulation across the device. Notably, the proposed sensor transducer is compatible with both standard semiconductor fabrication procedures and lab-on-a-chip style microfluidic delivery systems. 3D finite-difference time-domain electromagnetic field computation results are presented, the outcomes of which indicate that both specific (target) virus capture and non-specific (non-target) binding can be simultaneously measured and discerned from one another. This type of capacity for background-corrected, single-pathogen target detection would provide a new and novel advancement toward sensitive, label-free virus diagnostics.
(280.1415) Biological sensing and sensors; (230.5298) Photonic crystals; (350.4238) Nanophotonics and photonic crystals; (230.3990) Micro-optical devices; (130.5296) Photonic crystal waveguides; (230.5750) Resonators
Phase spatial light modulators (SLMs) are widely used for generating multifocal three-dimensional (3D) illumination patterns, but these are limited to a field of view constrained by the pixel count or size of the SLM. Further, with two-photon SLM-based excitation, increasing the number of focal spots penalizes the total signal linearly—requiring more laser power than is available or can be tolerated by the sample. Here we analyze and demonstrate a method of using galvanometer mirrors to time-sequentially reposition multiple 3D holograms, both extending the field of view and increasing the total time-averaged two-photon signal. We apply our approach to 3D two-photon in vivo neuronal calcium imaging.
Ptychography is a coherent diffraction imaging (CDI) method for extended objects in which diffraction patterns are acquired sequentially from overlapping coherent illumination spots. The object’s complex transmission function can be reconstructed from those diffraction patterns at a spatial resolution limited only by the scattering strength of the object and the detector geometry. Most experiments to date have positioned the illumination spots on the sample using a move-settle-measure sequence in which the move and settle steps can take longer to complete than the measure step. We describe here the use of a continuous “fly-scan” mode for ptychographic data collection in which the sample is moved continuously, so that the experiment resembles one of integrating the diffraction patterns from multiple probe positions. This allows one to use multiple probe mode reconstruction methods to obtain an image of the object and also of the illumination function. We show in simulations, and in x-ray imaging experiments, some of the characteristics of fly-scan ptychography, including a factor of 25 reduction in the data acquisition time. This approach will become increasingly important as brighter x-ray sources are developed, such as diffraction limited storage rings.
(100.5070) Phase retrieval; (100.3010) Image reconstruction techniques; (180.5810) Scanning microscopy; (180.7460) X-ray microscopy
We provide a framework for compensating errors within passive optical quadrature demodulation circuits used in swept-source optical coherence tomography (OCT). Quadrature demodulation allows for detection of both the real and imaginary components of an interference fringe, and this information separates signals from positive and negative depth spaces. To achieve a high extinction (∼60 dB) between these positive and negative signals, the demodulation error must be less than 0.1% in amplitude and phase. It is difficult to construct a system that achieves this low error across the wide spectral and RF bandwidths of high-speed swept-source systems. In a prior work, post-processing methods for removing residual spectral errors were described. Here, we identify the importance of a second class of errors originating in the RF domain, and present a comprehensive framework for compensating both spectral and RF errors. Using this framework, extinctions >60 dB are demonstrated. A stability analysis shows that calibration parameters associated with RF errors are accurate for many days, while those associated with spectral errors must be updated prior to each imaging session. Empirical procedures to derive both RF and spectral calibration parameters simultaneously and to update spectral calibration parameters are presented. These algorithms provide the basis for using passive optical quadrature demodulation circuits with high speed and wide-bandwidth swept-source OCT systems.
(110.4500) Optical coherence tomography; (120.3180) Interferometry; (100.0100) Image processing
Three-dimensional super-resolution imaging in thick, semi-transparent biological specimens is hindered by light scattering, which increases background and degrades both contrast and optical sectioning. We describe a simple method that mitigates these issues, improving image quality in our recently developed two-photon instant structured illumination microscope without requiring any hardware modifications to the instrument. By exciting the specimen with three laterally-structured, phase-shifted illumination patterns and post-processing the resulting images, we digitally remove both scattered and out-of-focus emissions that would otherwise contaminate our raw data. We demonstrate the improved performance of our approach in biological samples, including pollen grains, primary mouse aortic endothelial cells cultured in a three–dimensional collagen matrix and live tumor-like cell spheroids.
(110.0180) Microscopy; (100.6640) Superresolution; (170.2520) Fluorescence microscopy; (180.0180) Microscopy
We demonstrate the proof of concept of a novel Fourier-domain optical coherence tomography contrast mechanism using gold nanorod contrast agents and a spectral fractionation processing technique. The methodology detects the spectral shift of the backscattered light from the nanorods by comparing the ratio between the short and long wavelength halves of the optical coherence tomography signal intensity. Spectral fractionation further divides the halves into sub-bands to improve spectral contrast and suppress speckle noise. Herein, we show that this technique can detect gold nanorods in intralipid tissue phantoms. Furthermore, cellular labeling by gold nanorods was demonstrated using retinal pigment epithelial cells in vitro.
(170.4500) Optical coherence tomography; (170.3880) Medical and biological imaging
Measurements of the orientational freedom with which a single molecule may rotate or ‘wobble’ about a fixed axis have provided researchers invaluable clues about the underlying behavior of a variety of biological systems. In this paper, we propose a measurement and data analysis procedure based on a widefield fluorescence microscope image for quantitatively distinguishing individual molecules that exhibit varying degrees of rotational mobility. Our proposed technique is especially applicable to cases in which the molecule undergoes rotational motions on a timescale much faster than the framerate of the camera used to record fluorescence images. Unlike currently available methods, sophisticated hardware for modulating the polarization of light illuminating the sample is not required. Additional polarization optics may be inserted in the microscope’s imaging pathway to achieve superior measurement precision, but are not essential. We present a theoretical analysis, and benchmark our technique with numerical simulations using typical experimental parameters for single-molecule imaging.
(170.2520) Fluorescence microscopy; (100.0100) Image processing
A three dimensional (3D) pupil is an optical element, most commonly implemented on a volume hologram, that processes the incident optical field on a 3D fashion. Here we analyze the diffraction properties of a 3D pupil with finite lateral aperture in the 4-f imaging system configuration, using the Wigner Distribution Function (WDF) formulation. Since 3D imaging pupil is finite in both lateral and longitudinal directions, the WDF of the volume holographic 4-f imager theoretically predicts distinct Bragg diffraction patterns in phase space. These result in asymmetric profiles of diffracted coherent point spread function between degenerate diffraction and Bragg diffraction, elucidating the fundamental performance of volume holographic imaging. Experimental measurements are also presented, confirming the theoretical predictions.
(180.6900) Three-dimensional microscopy; (090.7330) Volume gratings; (080.5084) Phase space methods of analysis
Förster Resonance Energy Transfer (FRET) based measurements that calculate the stoichiometry of intermolecular interactions in living cells have recently been demonstrated, where the technique utilizes selective one-photon excitation of donor and acceptor fluorophores to isolate the pure FRET signal. Here, we present work towards extending this FRET stoichiometry method to employ two-photon excitation using a pulse-shaping methodology. In pulse-shaping, frequency-dependent phases are applied to a broadband femtosecond laser pulse to tailor the two-photon excitation conditions to preferentially excite donor and acceptor fluorophores. We have also generalized the existing stoichiometry theory to account for additional cross-talk terms that are non-vanishing under two-photon excitation conditions. Using the generalized theory we demonstrate two-photon FRET stoichiometry in live COS-7 cells expressing fluorescent proteins mAmetrine as the donor and tdTomato as the acceptor.
(180.4315) Nonlinear microscopy; (180.2520) Fluorescence microscopy; (320.5540) Pulse shaping; (100.2960) Image analysis; (170.1530) Cell analysis; (190.4180) Multiphoton processes
A compact waveguide display system integrating freeform elements and volume holograms is presented here for the first time. The use of freeform elements can broaden the field of view, which limits the applications of a holographic waveguide. An optimized system can achieve a diagonal field of view of 45° when the thickness of the waveguide planar is 3mm. Freeform-elements in-coupler and the volume holograms out-coupler were designed in detail in our study, and the influence of grating configurations on diffraction efficiency was analyzed thoroughly. The off-axis aberrations were well compensated by the in-coupler and the diffraction efficiency of the optimized waveguide display system could reach 87.57%. With integrated design, stability and reliability of this monochromatic display system were achieved and the alignment of the system was easily controlled by the record of the volume holograms, which makes mass production possible.
(090.2870) Holographic display; (090.2890) Holographic optical elements; (090.7330) Volume gratings; (220.2740) Geometric optical design; (220.4830) Systems design
Integrated optical devices comprised of multiple material systems are able to achieve unique performance characteristics, enabling applications in sensing and in telecommunications. Due to ease of fabrication, the majority of previous work has focused on polymer-dielectric or polymer-semiconductor systems. However, the environmental stability of polymers is limited. In the present work, a hybrid device comprised of an indium tin oxide (ITO) coating on a silicon dioxide toroidal resonant cavity is fabricated. Finite element method simulations of the optical field in the multi-material device are performed, and the optical mode profile is significantly altered by the high index film. The quality factor is also measured and is material loss limited. Additionally, its performance as a temperature sensor is characterized. Due to the high thermo-optic coefficient of ITO and the localization of the optical field in the ITO layer, the hybrid temperature sensor demonstrates a nearly 3-fold improvement in performance over the conventional silica device.
(230.5750) Resonators; (130.3120) Integrated optics devices; (160.6840) Thermo-optical materials; (160.6060) Solgel
Ptychography is an imaging method whereby a coherent beam is scanned across an object, and an image is obtained by iterative phasing of the set of diffraction patterns. It is able to be used to image extended objects at a resolution limited by scattering strength of the object and detector geometry, rather than at an optics-imposed limit. As technical advances allow larger fields to be imaged, computational challenges arise for reconstructing the correspondingly larger data volumes, yet at the same time there is also a need to deliver reconstructed images immediately so that one can evaluate the next steps to take in an experiment. Here we present a parallel method for real-time ptychographic phase retrieval. It uses a hybrid parallel strategy to divide the computation between multiple graphics processing units (GPUs) and then employs novel techniques to merge sub-datasets into a single complex phase and amplitude image. Results are shown on a simulated specimen and a real dataset from an X-ray experiment conducted at a synchrotron light source.
(100.5070) Phase retrieval; (110.3010) Image reconstruction techniques; (200.4960) Parallel processing; (170.7440) X-ray imaging
Abstract: Better understanding of bacteria environment interactions in the context of biofilm formation requires accurate 3-dimentional measurements of bacteria motility. Digital Holographic Microscopy (DHM) has demonstrated its capability in resolving 3D distribution and mobility of particulates in a dense suspension. Due to their low scattering efficiency, bacteria are substantially difficult to be imaged by DHM. In this paper, we introduce a novel correlation-based de-noising algorithm to remove the background noise and enhance the quality of the hologram. Implemented in conjunction with DHM, we demonstrate that the method allows DHM to resolve 3-D E. coli bacteria locations of a dense suspension (>107 cells/ml) with submicron resolutions (<0.5 µm) over substantial depth and to obtain thousands of 3D cell trajectories.
(090.1995) Digital holography; (100.2980) Image enhancement; (100.6890) Three-dimensional image processing; (180.6900) Three-dimensional microscopy
Temporal interlacing is a method for presenting stereoscopic 3D content whereby the two eyes’ views are presented at different times and optical filtering selectively delivers the appropriate view to each eye. This approach is prone to distortions in perceived depth because the visual system can interpret the temporal delay between binocular views as spatial disparity. We propose a novel color-interlacing display protocol that reverses the order of binocular presentation for the green primary but maintains the order for the red and blue primaries: During the first sub-frame, the left eye sees the green component of the left-eye view and the right eye sees the red and blue components of the right-eye view, and vice versa during the second sub-frame. The proposed method distributes the luminance of each eye’s view more evenly over time. Because disparity estimation is based primarily on luminance information, a more even distribution of luminance over time should reduce depth distortion. We conducted a psychophysical experiment to test these expectations and indeed found that less depth distortion occurs with color interlacing than temporal interlacing.
(330.1400) Vision - binocular and stereopsis; (330.1690) Color
The Lagrange invariant is a well-known law for optical imaging systems formulated in the frame of ray optics. In this study, we reformulate this law in terms of wave optics and relate it to the resolution limits of various imaging systems. Furthermore, this modified Lagrange invariant is generalized for imaging along the z axis, resulting with the axial Lagrange invariant which can be used to analyze the axial resolution of various imaging systems. To demonstrate the effectiveness of the theory, analysis of the lateral and the axial imaging resolutions is provided for Fresnel incoherent correlation holography (FINCH) systems.
(170.0110) Imaging systems; (090.1995) Digital holography; (110.6880) Three-dimensional image acquisition; (100.3010) Image reconstruction techniques; (070.6120) Spatial light modulators; (090.2880) Holographic interferometry; (090.1970) Diffractive optics