Understanding how neural stem cells (NSCs) give rise to different types of progeny is a fundamental question in developmental biology. Lineage analysis is a key aspect of quantifying the patterns of development for stem cells. The importance of gathering lineage information is well-exemplified in the blood system where lineage analysis revealed that differentiated blood cells are derived from hematopoietic stem cells and identified factors that regulate the different types of blood cells generated information that was vital for design of blood disorder therapies1,2
. In addition to lineage analysis, another key aspect of NSC behavior is the pattern of changes in the motion and morphology of an individual cell, patterns that have been shown to enable accurate prediction of the type of progeny an NSC will produce before the cell divides. In previous research we developed a software methodology called Algorithmic Information Theoretic Prediction and Discovery (AITPD)3,4
. AITPD was applied to the segmentation and tracking results of NSC image sequence data where it discovered that NSCs with different cell fate outcomes exhibited visually subtle differences in the patterns of motion and morphology, and subsequently used those differences to accurately predict cell fate from the segmentation and tracking results.
There is a compelling and widespread need for computer automated tools to generate lineage trees from time-lapse images, and to produce quantitative phenotypic measurements. Due to the visual ambiguity inherent in image sequence data showing proliferating cells, such automated tools will inevitably make mistakes. Mistakes in the tracking or lineaging of NSCs can corrupt the ultimate statistical analysis. There is a pressing need for methods to automatically identify easily correct errors in the automated image analysis results. Here we present a software tool named LEVER (Lineage Editing and Validition) designed to quantify the lineage and the patterns of dynamic behaviors of NSC development. LEVER contains two integrated modules. The first performs automatic image segmentation (delineation), tracking and lineaging of the stem cells. The second module is an interactive software application that allows the user to inspect and edit the results of the automated image analysis. Integration of the two modules enables the automatic image analysis tools to learn from the manual edits, making the best possible use of human-provided corrections. In addition to generating lineage trees from the time lapse-image sequence data, LEVER accurately segments and tracks NSCs through the image sequence, extracting the segmentation and tracking information required to apply AITPD to wider problems relating to the analysis of NSC development. illustrates the LEVER process.
Figure 1 Schematic illustrating the steps of the protocol. The sequence of steps to image and analyze clonal development. The corresponding protocol steps are indicated in bold adjacent to each step on the schematic. NPCs are isolated and imaged through the end (more ...)
Development of the protocol
The automated segmentation, tracking and lineaging modules of LEVER are based originally on the NSC lineaging tools developed by Al Kofahi, et al.5
. The segmentation and tracking was improved in Cohen, et al.4
to allow the tracking algorithm to automatically make corrections to segmentation errors. The tracking algorithm was also improved to incorporate multiple image frames when solving the data association problem between segmentation results and tracks.
This work improves further on the segmentation algorithm from these previous implementations by replacing the watershed transform that was used to separate touching cells with a thresholded morphological gradient, improving the ability to separate touching cells while reducing the over-segmentation associated with the watershed transform 6
. This work also improves on these previous implementations by replacing the bipartite assignment step that was used to optimally assign existing tracks to future sequences of segmentations. The tracking algorithm used by the LEVER program uses an approach we term Multitemporal Association Tracking (MAT). MAT performs a minimum spanning tree optimization7
instead of using a bipartite graph matching approach to associate existing tracks with multiple frames of segmentation results. MAT eliminates the need to construct explicit probabilistic models of segmentation errors, instead using known stem cell behaviors to predict the most likely sequence of frames. Supplementary Table 1
shows a comparison between the MAT tracking approach and the bipartite matching tracker that was used in Cohen, et al.4
. LEVER also exploits user supplied edits to automatically correct related segmentation errors. This inference-based approach to learning8
from user supplied edits was inspired by our prior work on retinal progenitor cells4
, where tracking results were used to automatically correct segmentation errors.
In the LEVER program we have combined the MAT algorithm with the inspection and editing tools in order to optimally utilize manual edits for correcting the segmentation and tracking results. Importantly, each time the user performs an edit operation, MAT recalculates the segmentation and tracking for all cells affected by the manual edit throughout the image sequence. shows the LEVER user interface, with the image sequence data and automated segmentation and tracking results displayed in one window, and the lineage tree displayed in a second window. Edits and navigation of the spatiotemporal data can be done on either window. shows how a user can correct a segmentation error by simply specifying the correct number of cells in a region. Such corrections are then used to identify and correct related errors in subsequent image frames.
Figure 2 The LEVER user interface. Representative views showing the lineage and image data windows from the LEVER program. (A) The lineage window shows the lineage tree for the currently selected clone. (B) The image data window shows the image sequence data with (more ...)
Figure 3 Segmentation errors are easily corrected. Example image frames showing the correction of a segmentation error and the subsequent automatic correction of related errors. (A) The only user input required to fix a segmentation error is the correct number (more ...)
Here we describe a protocol that uses the LEVER computer program to automate lineaging for embryonic NSCs. To adapt the protocol for other cell types, the culture medium, growth factors, and coating of tissue culture dishes must be optimized. In addition, the division rate and proliferative potential of the cells must be considered. For highly proliferative cells that produce large clones, the initial plating density should be low to minimize clonal progeny overlapping and cells crawling on top of each other, which impedes tracking. In addition, cell motility is an important factor because cells can move out of the frame. If cells are highly motile, conditions that limit cell movement to within the field of view can be used, for example by etching small islands of appropriate adhesive surfaces9
. Increasing the temporal resolution of imaging can make it easier to track highly motile cells. Depending on the cell cycle length and motility, the time of recording should be adjusted. For some cells, live imaging for example with specific promoter driven fluorescent tags, such as eGFP, can be acquired (e.g.
once per hour). This is a useful technique that provides real-time information on phenotype, but is limited by cell sensitivity to exciting wavelengths. To elucidate the progeny phenotype at the end of clonal development, immunohistochemistry is performed using specific markers appropriate for the particular cell type being imaged. The LEVER program is applicable to any image sequence for which a human observer is able to specify the correct tracking and lineaging. Applications of LEVER are limited to cell culture densities at which it is possible for a human observer to validate and correct the automated results.
The segmentation algorithm described here is specific to phase contrast microscopy. Applications using a different imaging modality such as fluorescence microscopy will require a custom segmentation algorithm, or should investigate the use of either manual lineaging or the protocol developed by Murray et al.
. Although the segmentation algorithm has been optimized for mouse embryonic NSCs, it is still possible to apply directly to other cell types. For example, in other current biological studies we have applied the segmentation algorithm to adult NSCs, and to hematopoietic stem cells. Because the adult NSCs tend to “clump” together, these image sequences require more manual editing to separate touching cells. Since the adult NSC clones being analyzed are smaller than embryonic clones, the overall editing time is similar. For the hematopoietic stem cells, because the cells are extremely motile we increased the temporal resolution of imaging to one frame per minute. This makes it easier for the tracker (and also for the human observer) to establish temporal correspondences between cells. The increased temporal resolution comes at a cost of decreased microscope throughput due to a reduction of the number of movies that can be captured simultaneously and also increases the data storage requirement and time required for application of the protocol. As a “rule of thumb”, temporal resolution should be chosen to be no less frequent than the cell radius divided by the maximum cellular velocity. The use of phase contrast rather than fluorescence imaging has eliminated issues related to phototoxicity for the cells imaged using this protocol, enabling long term high frequency image capture. At the maximum temporal resolution of imaging, the use of a deep red filter in the light path has allowed long term survivability under continuous illumination of a single field without impacting subsequent image analysis.
Experimental treatments and biological controls can be imaged from the same culture plate during the same period of image acquisition. Software controls consist of the sample data and results (Data Sets 1 and 2, which are available for download along with the LEVER executables on the webpage: https://pantherfile.uwm.edu/cohena/www/lever.html
.). In adapting the protocol to other applications, users should compare their images to the sample images for cellular appearance and dynamics, as well as for contrast, illumination and background. Results can be compared to those obtained from the sample data.
Applications of the method
The full diversity of potential applications for this protocol include the segmentation, tracking and lineaging of proliferating cells imaged using phase contrast microscopy. The software implementation of the protocol presented here has been applied to segmenting, tracking and lineaging 2-D phase contrast image sequences of adult and embryonic murine NSCs as well as to rat retinal progenitor cells and oligodendrocyte precursors4
. The MAT tracking algorithm described here has also been applied to quantifying organelle transport deficiencies in Huntington’s disease11
. The cell culture and staining steps of the protocol have been applied to the study of timing in cortical neurogenesis12
. Our image analysis algorithms can adapt to normal variations in imaging conditions. In order to analyze other cell types imaged by other protocols, some modifications and adjustments to the segmentation algorithm may be required. With the exception of the segmentation algorithm, all of the LEVER program including the tracking, lineaging and editing user interface, can be applied to any type of stem or tumorigenic cell.
Comparison with other methods
Recent research involving stem cell lineage analysis has used manual tracking to follow individual cells through the image sequence and establish parent-daughter cell relationships. In Gomes et al.13
, thousands of rat retinal progenitor cells were tracked manually to reconstruct their lineages. While manual tracking to generate lineages allows the analysis of population dynamics related to the lineage tree, the approach is not only tedious but it also fails to capture important properties of shape and motion for the cells being analyzed. Additionally, manually generated lineages are challenging to validate as it can be difficult to establish correspondences between the lineage and the image data. In Eilken, et al.14
, a software tool was developed that allows users to manually specify temporal correspondences between cells in every pair of image frames. This approach allows for the capture of attributes related to cell motion and shape, and also provides correspondences between the lineage tree and the image data. This protocol represents a significant reduction in the amount of labor required compared to manual segmentation approaches. The error rate for the automated image analysis portion of the protocol is approximately 1–2% (see Supplementary Table 1
). In contrast, manual segmentation would require editing 100% of the image data.In order to enable high throughput analysis of large quantities of stem cell data necessary for analyzing large populations, automated tools are required.
A number of automated stem cell lineaging trackers have been developed. In particular, a system for automatically lineaging and editing the lineage for fluorescently image sequences of C. elegans embryos was described in previous a Nature Protocols publication10
. The protocol here differs in that it is designed for analyzing phase contrast vertebrate NSCs rather than fluorescence images of C. elegans embryos. Additionally, the tracking approach described in that protocol used a nearest neighbor strategy to associate segmentation results with tracks. Nearest neighbors data association approaches in multitarget tracking are error-prone under high target and noise density. The MAT tracking approach utilized by LEVER solves the data association problem optimally across multiple image frames simultaneously, improving performance significantly under high target and noise density, as shown in Supplementary Table 1
. Other sophisticated phase contrast vertebrate stem cell tracking programs have also been developed, but to our knowledge none are available under an open-source freeware model. Li et al.
describe a tracking approach that achieves an error rate comparable to MAT but at the cost of significant implementation complexity 15
. Chen et al.16
implemented segmentation and tracking for 3-D fluorescence image sequences of thymocytes that included a statistically guided edit-based validation system similar to the protocol described here, although the source code was not released. Our approach here is designed for phase contrast rather than fluorescence images and uses a more sophisticated tracking algorithm (MAT) compared to their bipartite assignment tracking algorithm. Finally, the protocol presented here is unique in that the segmentation, tracking and lineaging algorithms are tightly coupled with the editing application allowing the program to learn from user edits and automatically correct related errors. Here we are releasing both the automatic image analysis and the user interface portions of the LEVER program under an open-source license with the intention that future developments and improvements in stem cell segmentation and tracking can be integrated into the LEVER program in order to make those improvements available to the wider biological community using a consistent user interface.
Progenitor cells that are difficult to grow as adherent cultures, that grow on top of one another, are highly motile or are particularly photosensitive will be challenging. In addition, dense cultures are difficult to segment and follow, so this protocol is most suitable for cells that can be plated initially at lower density then proliferate to form a low to moderately dense culture.
One limitation of the current implementation of the LEVER computer program is that it requires a computer running the Windows operating system. We do provide all of the LEVER source code so that anyone with access to a C++ compiler and the MATLAB environment for a particular operating system should be able to use LEVER with that operating system. Our use of the GPL free software license will allow such a modified version of LEVER to be redistributed. The authors are not aware of any issues that would prevent LEVER from running on a platform other than Windows, but this has not been tested to date.
There are two additional limitations to the protocol presented here. First, the segmentation is specific to the visual appearance of the cells being segmented. For cells that are imaged by other imaging protocols, the segmentation algorithm may not work as well. Such image data may still be processed but will require additional manual editing subsequent to the automated image analysis. In order to mitigate this limitation, we have implemented the segmentation in MATLAB in the hopes that it will ease the creation of new segmentation algorithms for other types of stem cells. The second limitation is that the segmentation algorithm can take a long time to run on older computers. On our reference computer which contains a dual quad core Intel Xeon X5570 with 16 processing cores and 24 GB of RAM, segmentation fully exploits the parallel processing capabilities of the machine and typically takes 5 to 15 min.