|Home | About | Journals | Submit | Contact Us | Français|
We present a method for computational reconstruction of the 3-D morphology of biological objects, such as cells, cell conjugates or 3-D arrangements of tissue structures, using data from high-resolution microscopy modalities. The method is based on the iterative optimization of Voronoi representations of the spatial structures. The reconstructions of biological surfaces automatically adapt to morphological features of varying complexity with flexible degrees of resolution. We show how 3-D confocal images of single cells can be used to generate numerical representations of cellular membranes that may serve as the basis for realistic, spatially resolved computational models of membrane processes or intracellular signaling. Another example shows how the protocol can be used to reconstruct tissue boundaries from segmented two-photon image data that facilitate the quantitative analysis of lymphocyte migration behavior in relation to microanatomical structures. Processing time is of the order of minutes depending on data features and reconstruction parameters.
The increased availability and use of advanced 3-D imaging techniques such as fluorescence confocal and multi-photon or electron tomographic microscopy has enabled researchers to investigate biological structures and processes with a high degree of spatial resolution. The applications range from high-resolution mapping of (sub-)cellular morphology and dynamic tracking of fluorescently labeled proteins in sub-cellular compartments to in situ imaging of cell population behavior1–3. Because many biological processes are closely related to and influenced by the spatial context in which they occur – information made accessible by 3-D microscopy – it is essential to include these spatial properties in the analysis of such data. Moreover, because data sets acquired in these experiments are usually large and the relevant biological objects are numerous and/or the spatial properties complex, manual analyses are laborious and frequently involve subjective choices that render them problematic for quantitative data analysis. Here, we describe the application of an approach, implemented in a user-friendly software tool, that allows for the automated three-dimensional reconstruction of the surfaces of biological objects ranging from (sub-) cellular membranes to tissue/organ boundaries and the subsequent integration of these reconstructions with automated tissue-contextual cell migration data analysis and modeling.
Many different strategies for computational surface reconstruction have been developed, frequently based either on higher-order polygonal 4, 5 or triangular 6, 7 surface meshes (for a review, see e.g. 8). Most approaches were designed mainly for visualization purposes in software used to process microscopy data, as for instance in Imaris® (Bitplane). The difference between those approaches and the technique introduced here are that our approach uses adaptive resolution of surface features, can reduce artefacts resulting from lower out-of-plane resolution, and is capable of high-quality mesh generation required for computational modeling. The price that has to be paid for the combination of these advantages is that the iterative optimization procedure may cause longer processing times compared to conventional approaches if high mesh resolutions are desired. Our method, used to obtain the results published in ref. 9 is based on the geometric concept of ‘Voronoi diagrams10’ that combines the concepts of polygonal and triangular meshes and offers specific advantages for numerical simulations 11. In two dimensions, a Voronoi diagram (also called Dirichlet tesselation12) of a set of points, here called ‘vertices’, is constructed by subdividing the area containing the vertices into geometric mesh elements in such a way that the Voronoi element of each vertex comprises the region surrounding the vertex that is closer to this than to any other vertex13 (Fig. 1). Because Voronoi diagrams can be computed for arbitrary vertex distributions, their shapes can also be highly variable (Fig. 1A). There exist, however, vertex distributions for which the Voronoi diagrams have properties that are particularly desirable for computational analyses: the variation of the distances between neighboring vertices is minimized (equally spaced mesh) and the ratio of the element circumference and element area are minimal. In 2-D, these optimized Voronoi diagrams, or meshes, are hexagonal lattices (Fig. 1D). They occur naturally, for instance, in bee honeycombs, minimizing the material needed for building robust planar structures. Voronoi-like shapes generated during isotropic growth or diffusion processes starting from initial seed points, as in turtle carapaces or giraffe fur, show similar but sometimes less optimized hexagonal structures. Perfectly hexagonal structures can be viewed as limiting cases toward which ‘real’ meshes, i. e. 2-D meshes constrained by boundaries or embedded in 3-D, may evolve if appropriate algorithms are used. Such meshes are called ‘optimal’ (or also ‘high-quality’ meshes).
An important characteristic of optimal Voronoi meshes is that the center points (also called forming points) have the same coordinates as the centroids of the elements. While vertex distributions resulting in optimal Voronoi meshes can be easily generated in a two dimensional plane without boundaries or with rectangular borders, this task is nontrivial if curved boundaries are present or for curved surfaces embedded in 3-D, which is the case for computational reconstructions of cell/tissue surfaces. The computational method we describe here uses an iterative optimization algorithm that produces optimal meshes for arbitrary shapes and boundaries from arbitrary initial vertex distributions. Initial “seed” vertices are distributed randomly within the confines of the surface layer defined by a 3-dimensional binary mask created from microscopy image series (Figure 2). Then the Voronoi surface elements are computed for the initial distribution and the vertices are subsequently moved to the centroids of these elements. In the next step the Voronoi elements are recomputed for the centroids, which starts an iterative process (computation of Voronoi diagram → computation of Voronoi centroid → replacement of Voronoi center vertex with centroid ) that converges to an optimal Voronoi mesh with hexagonal element geometry (Figures 1E,F and 3).
Our approach permits to automatically and adaptively produce optimized meshes and resolve morphological details of varying complexity with an optimal number of elements for a given desired accuracy. More precisely, our method offers two reconstruction alternatives that may also be combined. The first option requires the definition of the total number of elements/vertices used in the reconstruction. The method then iteratively optimizes the distribution of the vertices so that a uniform (high-quality) mesh is generated (Figure 4 and Box 1 ‘Description of the reconstruction algorithm’), which approximates the surface with an accuracy not lower than the average distance between neighboring vertices. The second (and default) option allows for the automatic adaptation of the reconstruction to local differences in spatial detail in the image data and ensures that a certain accuracy of the surface reconstruction is reached with a minimal number of elements (Figure 5). This approach is based on the computation of local quality measures (for instance, angles between surface elements or Euclidean vs. geodesic vertex distances) and uses local mesh refinement if quality criteria are not met (see Box 2 ‘Local adaptivity’ for details). These numerical Voronoi reconstructions are useful whenever quantitatively precise representations of spatial features have to be extracted from 3D microscopy images. Here we focus on two example applications at distinct scales of biological resolution.
The following steps are performed automatically without any user intervention and are listed here only to present details of the algorithm.
The reconstruction algorithm without local adaptivity results in a homogeneous mesh, i.e., the mesh vertices are equally distributed over the surface resulting in an optimal mesh approximating the surface with a accuracy given by the mesh constant (average distance between neighboring vertices) (Supplementary Figure 1). With locally adaptive reconstruction, the surface reconstruction adaptively resolves morphological details until a certain, definable accuracy is reached. This is achieved by computing the angles between every pair of adjacent triangles. If this angle is smaller than a certain threshold, the mesh is locally refined by inserting an additional vertex at the center of the common side of the two adjacent triangles. To maintain a locally optimized grid structure, one ‘regular’ iteration step is performed following the insertion of additional vertices. This adaptivity generates surface parameterization of higher accuracy with a locally higher density of vertices in regions of higher spatial complexity, while limiting the total number of elements (Figure 5). The relevant options and parameters can be set in the MorphologyModeler Params menu.
The first example describes surface reconstruction based on microscopy images of cellular morphology. Computational studies in cell biology increasingly take into account that many aspects of cellular function cannot be understood or modeled without proper treatment of cellular morphology14–17. Analyzing or simulating intra-cellular biochemical dynamics, such as non-isotropic reaction-diffusion processes in the cytoplasm or on the cell membrane or intra-cellular transport mechanisms, requires computational representations (spatial discretizations) of cellular morphology. The elements of high quality Voronoi meshes are particularly suitable for such discretizations because they ensure locally homogeneous mesh spacings even for complex cell morphologies and the lines connecting the Voronoi centroids are perpendicular to the element boundaries, properties that greatly facilitate mathematical analyses and simulation of diffusion and fluxes18. Because our reconstruction method can adapt its spatial resolution to the morphological properties of the cells being analyzed, we obtain high quality representations of cellular structure with a minimal number of volume elements, thereby minimizing computational costs of numerical analyses and simulations (Fig. 5).
On a slightly higher spatial scale, analyzing the dynamic organization of physically interacting cell populations in specialized tissues requires computational representations of micro-anatomical structures and cellular motion tracks. Recent advances in in-situ two-photon microscopy have facilitated the investigation of cell behavior dynamics within complex tissue environments. These technological developments have been particularly beneficial for immunological research, as the analysis of migration and interaction of leucocytes, lymphocytes, and antigen presenting cells in secondary lymphoid organs and at sites of chronic inflammation is critical for a deeper understanding of the immune response19. Because the microanatomy of secondary lymphoid organs reflects the compartmentalization of T and B cells and antigen presenting cells and provides guidance for entry, internal movement, and exit of cells, cell migration analysis must take into account the micro-anatomical context. In the tissue surface reconstruction example presented here we specifically look at germinal centers (GCs), which are organized lymphoid tissue structures where high affinity antibody responses are generated. Lymphocytes that occupy GCs mainly include antigen-specific B and T cells, while naïve B cells are largely excluded from the GC proper. Intravital 2-photon microscopy has been increasingly utilized to understand how the GC tissue structure is dynamically organized and maintained and how the trafficking of its cellular components is controlled20–22. To analyze imaging data from such studies, it is often necessary to define GC-related tissue domains. Typically, this is accomplished by manual annotation of tissue boundaries on optical sections according to relevant fluorescently-labeled tissue landmarks. Ideally, these boundaries defined on series of axially adjacent 2-D imaging planes are further digitally combined to allow visualization and mathematic description of the reconstructed tissue structure in full 3-D space (Figures 7). The tissue reconstructions generated using this protocol have been used in a recent research publication9 to quantitatively analyze the cellular migration behavior of lymphocytes in the context of germinal center tissue structure. In that study computing and comparing the speed and location/distance of a SLAM-associated-protein and wild-type lymphocytes relative to the virtual germinal center surface over each track length permitted the automatic quantification and classification of the cell behaviour. The analysis revealed that in contrast to wild-type, SAP KO T lymphocytes are incapable of crossing the border of and moving into germinal centers (see 9 for details).
In addition to these types of applications, our method may improve the computational efficiency of standard surface reconstruction applications, such as morphological measurements or surface visualization, because it minimizes the number of surface elements for a given reconstruction accuracy. Conventional approaches, on the other hand, use triangular meshes with high homogeneous resolution and cannot locally adapt the mesh to surface features.
The surface reconstruction protocol described in the procedure may be used with any kind of segmented image data whose format represents an object whose surface is to be reconstructed with intensity values equal to or larger than 1 while the background pixels are zero. The images have to be provided in individual z -stack files and may have the following formats: JPEG, TIFF, or PNG. Files have to be named using a prefix followed by numbers indicating the z slice (the index format is detected automatically) and the corresponding image format suffix.
To generate the binary segmentations of the 3-D confocal microscopy images of the T cell used in our first example application, we manually defined an intensity threshold to differentiate between cell and background pixels using standard image processing software (e. g. Photoshop or GIMP) and applied that threshold to the whole z-stack (Figure 2). In case of image data of lower quality, for instance due to z-stack intensity inhomogeneities, we recommend using automatic segmentation software (see Reagents). To define binary masks of the germinal center area on individual imaging planes, which serve as the input for the tessellation program, we took advantage of the fact that naive B cells as a population are substantially excluded from GCs (Figure 7A). Therefore, fluorescent signals associated with naïve B cells were time-averaged for each imaging time series, and the follicular area and approximation of the GC–mantle border on individual optical sections were manually marked using the Photoshop mask function. Such marked images were then transformed into binary images in Photoshop, with pixel size and dimension maintained. The resulting z-stack of segmented germinal center masks essentially represents the average configuration of the visualized GC during the imaging period (Figure 7B,C).
Since the Voronoi mesh computation utilizes discrete voxel coordinates given by the image data, the precision of the Voronoi centroid computation decreases when a Voronoi element contains only relatively few voxels, i.e., the Voronoi resolution comes close to the image resolution. However, this is not a limitation of the reconstruction method, but related to the image quality and if necessary has to be overcome by an increase in image resolution. Low image resolutions mainly pose a problem in form of discrepancies between a higher in-plane and a lower out-of-plane (z-direction) resolution and visible ‘steps’ or ‘kinks’ may occur along the z direction. One intrinsic feature of the method presented here is that such z-direction ‘steps’ can be automatically interpolated. However, this only works as long as the z-steps are smaller than the reconstruction mesh element size (see Supp. Fig. 2, 3). If the Voronoi mesh elements have a higher resolution than the z-resolution of the image data, the z-steps are ‘precisely’ reconstructed (see Supp. Fig. 3 and 4), an unwanted phenomenon that is present in surface visualizations of microscopy data performed with most conventional software packages (e.g. Imaris®) that are widely used in biological microscopy that do not offer adaptive reconstructions. Thus, there is an optimum number of surface vertices that may correct limitations of microscopy data and more accurately represent the actual, real biological surface than higher-resolution surface meshes. In the method described here, this problem is automatically avoided when the number of vertices is adaptively adjusted (option B), but it has to be taken into account if the number of surface vertices used for the reconstruction is defined by the user (more is not necessarily better!). If adaptive reconstruction (option B) is used it has to be kept in mind that setting the quality parameter to high values (approaching the maximum 1.0) may in certain cases lead to very long processing times due to a large number of iterations. We therefore recommend starting with medium quality values.
The computational time required to perform the task outlined above depends on the size of the microscopy data set, the intended reconstruction accuracy and the performance of the computer hardware. Typical times using recent computers for surface reconstruction are on the order of minutes.
Application of this protocol to segmented image data produces surface tesselations that consist of mesh points (‘vertices’) and a mesh topology (vertex connections, ‘edges’). The general features of the results that can be anticipated from the surface reconstruction protocol in general depend on the features of the true object, the image resolution with which the object is represented and the choice of the reconstruction parameters and their combination: First, it can be expected that with increasing number of Voronoi mesh elements the image data is more accurately reconstructed. We call this type of accuracy image-related accuracy and define it as the distance of the centroids of the triangular mesh that is formed by connected the Voronoi center points (the ‘dual’ mesh) from the voxel image surface. The second type of reconstruction quality is the accuracy with which the true object is represented by the surface tesselation, called ‘object-related accuracy’. While an increasing mesh resolution always leads to an increasing image-related accuracy, an increase in the number of mesh elements does not always produce better object-related accuracy and in such cases, a lower number of mesh elements might lead to a higher-object accuracy due to smoothing effects (see also ‘Troubleshooting’ and Supp. Fig. 3 and 4).
The iterative optimization process (both the homogenous and locally adaptive options) converges to optimal (high-quality) meshes (see Fig. 1), if the resolution of the voxel data is sufficiently fine in relation to the mesh element size (see also ‘Troubleshooting’). The effect of locally adaptive surface reconstruction on the results depends on the spatial properties of the image data. A spherical object will produce a result similar to that of homogeneous reconstruction, because the iterative refinement process depends on local membrane curvature, which is constant on a sphere. However, when objects have special spatial features (which are resolved in the image data), the local adaptivity produces inhomogeneously resolved surface meshes. An example for the latter case is the T cell surface reconstruction in which the membrane protrusion (shown in the lower right part of the cell in Fig. 5C and middle part of Fig. 5D) is automatically resolved by local adaptive mesh refinement.
The results of the surface reconstruction of the T cell alone, the T-B-cell conjugate and the germinal center border reconstruction are presented in Figures 4–6. Figure 4 (+ Supp. Video 1) and Fig. 5 (+ Supp. Video 2) offer a comparison between adaptive and homogeneous reconstruction and illustrate the optimization process. See also Supplementary Figures for additional examples. Supplementary Figure 5 shows example results obtained when two separate surfaces present in the same data set are reconstructed simultaneously. If regions exist where the distance between the surfaces is equal to or smaller than the reconstruction mesh spacing the separate surfaces might merge/attach to each other. This problem can be solved by increasing the mesh resolution or processing the surfaces separately. Exemplary results of the cell tracking analysis can be found in the supplementary material section ‘cell track analysis’, Figure 7 Supplementary Figure 6.
This research was supported by the Intramural Research Program of NIAID, NIH.
COMPETING FINANCIAL INTERESTS STATEMENT
The authors declare that they have no competing financial interests.