Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Nat Protoc. Author manuscript; available in PMC 2012 June 1.
Published in final edited form as:
PMCID: PMC3313840

Vertebrate Neural Stem Cell Segmentation, Tracking and Lineaging with Validation and Editing


This protocol and the accompanying software program called LEVER enable quantitative automated analysis of phase contrast time-lapse images of cultured neural stem cells. Images are captured at 5 min. intervals over a period of 5 to 15 days as the cells proliferate and differentiate. LEVER automatically segments, tracks and generates lineage trees of the stem cells from the image sequence. In addition to generating lineage trees capturing the population dynamics of clonal development, LEVER extracts quantitative phenotypic measurements of cell location, shape, movement, and size. When available, the system can include biomolecular markers imaged using fluorescence. It then displays the results to the user for highly efficient inspection and editing to correct any errors in the segmentation, tracking or lineaging. In order to enable high-throughput inspection, LEVER incorporates features for rapid identification of errors, and learning from user-supplied corrections to automatically identify and correct related errors.

Keywords: stem cell, progenitor cell, clone, lineage, time-lapse recording, image sequence analysis, cell dynamics, lineage editing, stem cell tracking edit


Understanding how neural stem cells (NSCs) give rise to different types of progeny is a fundamental question in developmental biology. Lineage analysis is a key aspect of quantifying the patterns of development for stem cells. The importance of gathering lineage information is well-exemplified in the blood system where lineage analysis revealed that differentiated blood cells are derived from hematopoietic stem cells and identified factors that regulate the different types of blood cells generated information that was vital for design of blood disorder therapies1,2. In addition to lineage analysis, another key aspect of NSC behavior is the pattern of changes in the motion and morphology of an individual cell, patterns that have been shown to enable accurate prediction of the type of progeny an NSC will produce before the cell divides. In previous research we developed a software methodology called Algorithmic Information Theoretic Prediction and Discovery (AITPD)3,4. AITPD was applied to the segmentation and tracking results of NSC image sequence data where it discovered that NSCs with different cell fate outcomes exhibited visually subtle differences in the patterns of motion and morphology, and subsequently used those differences to accurately predict cell fate from the segmentation and tracking results.

There is a compelling and widespread need for computer automated tools to generate lineage trees from time-lapse images, and to produce quantitative phenotypic measurements. Due to the visual ambiguity inherent in image sequence data showing proliferating cells, such automated tools will inevitably make mistakes. Mistakes in the tracking or lineaging of NSCs can corrupt the ultimate statistical analysis. There is a pressing need for methods to automatically identify easily correct errors in the automated image analysis results. Here we present a software tool named LEVER (Lineage Editing and Validition) designed to quantify the lineage and the patterns of dynamic behaviors of NSC development. LEVER contains two integrated modules. The first performs automatic image segmentation (delineation), tracking and lineaging of the stem cells. The second module is an interactive software application that allows the user to inspect and edit the results of the automated image analysis. Integration of the two modules enables the automatic image analysis tools to learn from the manual edits, making the best possible use of human-provided corrections. In addition to generating lineage trees from the time lapse-image sequence data, LEVER accurately segments and tracks NSCs through the image sequence, extracting the segmentation and tracking information required to apply AITPD to wider problems relating to the analysis of NSC development. Figure 1 illustrates the LEVER process.

Figure 1
Schematic illustrating the steps of the protocol. The sequence of steps to image and analyze clonal development. The corresponding protocol steps are indicated in bold adjacent to each step on the schematic. NPCs are isolated and imaged through the end ...

Development of the protocol

The automated segmentation, tracking and lineaging modules of LEVER are based originally on the NSC lineaging tools developed by Al Kofahi, et al.5. The segmentation and tracking was improved in Cohen, et al.4 to allow the tracking algorithm to automatically make corrections to segmentation errors. The tracking algorithm was also improved to incorporate multiple image frames when solving the data association problem between segmentation results and tracks.

This work improves further on the segmentation algorithm from these previous implementations by replacing the watershed transform that was used to separate touching cells with a thresholded morphological gradient, improving the ability to separate touching cells while reducing the over-segmentation associated with the watershed transform 6. This work also improves on these previous implementations by replacing the bipartite assignment step that was used to optimally assign existing tracks to future sequences of segmentations. The tracking algorithm used by the LEVER program uses an approach we term Multitemporal Association Tracking (MAT). MAT performs a minimum spanning tree optimization7 instead of using a bipartite graph matching approach to associate existing tracks with multiple frames of segmentation results. MAT eliminates the need to construct explicit probabilistic models of segmentation errors, instead using known stem cell behaviors to predict the most likely sequence of frames. Supplementary Table 1 shows a comparison between the MAT tracking approach and the bipartite matching tracker that was used in Cohen, et al.4. LEVER also exploits user supplied edits to automatically correct related segmentation errors. This inference-based approach to learning8 from user supplied edits was inspired by our prior work on retinal progenitor cells4, where tracking results were used to automatically correct segmentation errors.

In the LEVER program we have combined the MAT algorithm with the inspection and editing tools in order to optimally utilize manual edits for correcting the segmentation and tracking results. Importantly, each time the user performs an edit operation, MAT recalculates the segmentation and tracking for all cells affected by the manual edit throughout the image sequence. Figure 2 shows the LEVER user interface, with the image sequence data and automated segmentation and tracking results displayed in one window, and the lineage tree displayed in a second window. Edits and navigation of the spatiotemporal data can be done on either window. Figure 3 shows how a user can correct a segmentation error by simply specifying the correct number of cells in a region. Such corrections are then used to identify and correct related errors in subsequent image frames.

Figure 2
The LEVER user interface. Representative views showing the lineage and image data windows from the LEVER program. (A) The lineage window shows the lineage tree for the currently selected clone. (B) The image data window shows the image sequence data with ...
Figure 3
Segmentation errors are easily corrected. Example image frames showing the correction of a segmentation error and the subsequent automatic correction of related errors. (A) The only user input required to fix a segmentation error is the correct number ...

Experimental Design

Here we describe a protocol that uses the LEVER computer program to automate lineaging for embryonic NSCs. To adapt the protocol for other cell types, the culture medium, growth factors, and coating of tissue culture dishes must be optimized. In addition, the division rate and proliferative potential of the cells must be considered. For highly proliferative cells that produce large clones, the initial plating density should be low to minimize clonal progeny overlapping and cells crawling on top of each other, which impedes tracking. In addition, cell motility is an important factor because cells can move out of the frame. If cells are highly motile, conditions that limit cell movement to within the field of view can be used, for example by etching small islands of appropriate adhesive surfaces9. Increasing the temporal resolution of imaging can make it easier to track highly motile cells. Depending on the cell cycle length and motility, the time of recording should be adjusted. For some cells, live imaging for example with specific promoter driven fluorescent tags, such as eGFP, can be acquired (e.g. once per hour). This is a useful technique that provides real-time information on phenotype, but is limited by cell sensitivity to exciting wavelengths. To elucidate the progeny phenotype at the end of clonal development, immunohistochemistry is performed using specific markers appropriate for the particular cell type being imaged. The LEVER program is applicable to any image sequence for which a human observer is able to specify the correct tracking and lineaging. Applications of LEVER are limited to cell culture densities at which it is possible for a human observer to validate and correct the automated results.

The segmentation algorithm described here is specific to phase contrast microscopy. Applications using a different imaging modality such as fluorescence microscopy will require a custom segmentation algorithm, or should investigate the use of either manual lineaging or the protocol developed by Murray et al. 10. Although the segmentation algorithm has been optimized for mouse embryonic NSCs, it is still possible to apply directly to other cell types. For example, in other current biological studies we have applied the segmentation algorithm to adult NSCs, and to hematopoietic stem cells. Because the adult NSCs tend to “clump” together, these image sequences require more manual editing to separate touching cells. Since the adult NSC clones being analyzed are smaller than embryonic clones, the overall editing time is similar. For the hematopoietic stem cells, because the cells are extremely motile we increased the temporal resolution of imaging to one frame per minute. This makes it easier for the tracker (and also for the human observer) to establish temporal correspondences between cells. The increased temporal resolution comes at a cost of decreased microscope throughput due to a reduction of the number of movies that can be captured simultaneously and also increases the data storage requirement and time required for application of the protocol. As a “rule of thumb”, temporal resolution should be chosen to be no less frequent than the cell radius divided by the maximum cellular velocity. The use of phase contrast rather than fluorescence imaging has eliminated issues related to phototoxicity for the cells imaged using this protocol, enabling long term high frequency image capture. At the maximum temporal resolution of imaging, the use of a deep red filter in the light path has allowed long term survivability under continuous illumination of a single field without impacting subsequent image analysis.


Experimental treatments and biological controls can be imaged from the same culture plate during the same period of image acquisition. Software controls consist of the sample data and results (Data Sets 1 and 2, which are available for download along with the LEVER executables on the webpage: In adapting the protocol to other applications, users should compare their images to the sample images for cellular appearance and dynamics, as well as for contrast, illumination and background. Results can be compared to those obtained from the sample data.

Applications of the method

The full diversity of potential applications for this protocol include the segmentation, tracking and lineaging of proliferating cells imaged using phase contrast microscopy. The software implementation of the protocol presented here has been applied to segmenting, tracking and lineaging 2-D phase contrast image sequences of adult and embryonic murine NSCs as well as to rat retinal progenitor cells and oligodendrocyte precursors4. The MAT tracking algorithm described here has also been applied to quantifying organelle transport deficiencies in Huntington’s disease11. The cell culture and staining steps of the protocol have been applied to the study of timing in cortical neurogenesis12. Our image analysis algorithms can adapt to normal variations in imaging conditions. In order to analyze other cell types imaged by other protocols, some modifications and adjustments to the segmentation algorithm may be required. With the exception of the segmentation algorithm, all of the LEVER program including the tracking, lineaging and editing user interface, can be applied to any type of stem or tumorigenic cell.

Comparison with other methods

Recent research involving stem cell lineage analysis has used manual tracking to follow individual cells through the image sequence and establish parent-daughter cell relationships. In Gomes et al.13, thousands of rat retinal progenitor cells were tracked manually to reconstruct their lineages. While manual tracking to generate lineages allows the analysis of population dynamics related to the lineage tree, the approach is not only tedious but it also fails to capture important properties of shape and motion for the cells being analyzed. Additionally, manually generated lineages are challenging to validate as it can be difficult to establish correspondences between the lineage and the image data. In Eilken, et al.14, a software tool was developed that allows users to manually specify temporal correspondences between cells in every pair of image frames. This approach allows for the capture of attributes related to cell motion and shape, and also provides correspondences between the lineage tree and the image data. This protocol represents a significant reduction in the amount of labor required compared to manual segmentation approaches. The error rate for the automated image analysis portion of the protocol is approximately 1–2% (see Supplementary Table 1). In contrast, manual segmentation would require editing 100% of the image data.In order to enable high throughput analysis of large quantities of stem cell data necessary for analyzing large populations, automated tools are required.

A number of automated stem cell lineaging trackers have been developed. In particular, a system for automatically lineaging and editing the lineage for fluorescently image sequences of C. elegans embryos was described in previous a Nature Protocols publication10. The protocol here differs in that it is designed for analyzing phase contrast vertebrate NSCs rather than fluorescence images of C. elegans embryos. Additionally, the tracking approach described in that protocol used a nearest neighbor strategy to associate segmentation results with tracks. Nearest neighbors data association approaches in multitarget tracking are error-prone under high target and noise density. The MAT tracking approach utilized by LEVER solves the data association problem optimally across multiple image frames simultaneously, improving performance significantly under high target and noise density, as shown in Supplementary Table 1. Other sophisticated phase contrast vertebrate stem cell tracking programs have also been developed, but to our knowledge none are available under an open-source freeware model. Li et al. describe a tracking approach that achieves an error rate comparable to MAT but at the cost of significant implementation complexity 15. Chen et al.16 implemented segmentation and tracking for 3-D fluorescence image sequences of thymocytes that included a statistically guided edit-based validation system similar to the protocol described here, although the source code was not released. Our approach here is designed for phase contrast rather than fluorescence images and uses a more sophisticated tracking algorithm (MAT) compared to their bipartite assignment tracking algorithm. Finally, the protocol presented here is unique in that the segmentation, tracking and lineaging algorithms are tightly coupled with the editing application allowing the program to learn from user edits and automatically correct related errors. Here we are releasing both the automatic image analysis and the user interface portions of the LEVER program under an open-source license with the intention that future developments and improvements in stem cell segmentation and tracking can be integrated into the LEVER program in order to make those improvements available to the wider biological community using a consistent user interface.


Progenitor cells that are difficult to grow as adherent cultures, that grow on top of one another, are highly motile or are particularly photosensitive will be challenging. In addition, dense cultures are difficult to segment and follow, so this protocol is most suitable for cells that can be plated initially at lower density then proliferate to form a low to moderately dense culture.

One limitation of the current implementation of the LEVER computer program is that it requires a computer running the Windows operating system. We do provide all of the LEVER source code so that anyone with access to a C++ compiler and the MATLAB environment for a particular operating system should be able to use LEVER with that operating system. Our use of the GPL free software license will allow such a modified version of LEVER to be redistributed. The authors are not aware of any issues that would prevent LEVER from running on a platform other than Windows, but this has not been tested to date.

There are two additional limitations to the protocol presented here. First, the segmentation is specific to the visual appearance of the cells being segmented. For cells that are imaged by other imaging protocols, the segmentation algorithm may not work as well. Such image data may still be processed but will require additional manual editing subsequent to the automated image analysis. In order to mitigate this limitation, we have implemented the segmentation in MATLAB in the hopes that it will ease the creation of new segmentation algorithms for other types of stem cells. The second limitation is that the segmentation algorithm can take a long time to run on older computers. On our reference computer which contains a dual quad core Intel Xeon X5570 with 16 processing cores and 24 GB of RAM, segmentation fully exploits the parallel processing capabilities of the machine and typically takes 5 to 15 min.



Tissue culture

  • DMEM (Invitrogen #11960-051)
  • L-Glutamine (Invitrogen #25030-081)
  • Na-Pyruvate (Invitrogen #15039-019)
  • N2 Supplement (Invitrogen #17502-04)
  • B27 Supplement (Invitrogen # 17504-044)
  • FGF2 (Invitrogen #13256-029)
  • Papain (Worthington #3126)
  • poly-l-lysine (Sigma #P4707)
  • DNAse (Sigma # DN25)
  • KCl (Sigma #P5405)
  • NaOH (Sigma #S5881)
  • NAH2PO4 (Sigma #S3139)
  • MgCL2 (Sigma #M2670)
  • Glucose (Sigma # G6152)
  • Sorbital (Sigma #S3889)


  • PIPES (Sigma #P1851)
  • HEPES (Sigma #H3375)
  • EGTA (Sigma #E4378)
  • MgCl2.6H2O (Sigma #M2670)
  • KOH (Fisher #P251)
  • Paraformaldehyde (PolyScience #380)
  • 10X PBS (Invitrogen # AM9624)
  • Triton X-100 (Sigma T9284)
  • Normal goat serum (Vector #S-1000)
  • B-tubulin III antibody (Sigma #T8660)
  • Cy5-conjugated goat-anti-mouse IgG2b (Invitrogen # A21248)
  • DAPI (Invitrogen A21248)



  • Zeiss Axio-Observer Inverted Z1 microscope with phase contrast and fluorescence capabilities
  • Hamamatsu digital camera
  • Pecon incubation chamber with controlled temperature at 37°C, 98% humidity and 5% CO2
  • Motorized XYZ stage for imaging multiple points
  • Zeiss Axiovision software

Tissue culture

  • Dissection hood
  • Dissection microscope
  • Brain dissection instruments
  • Biological safety cabinet
  • Centrifuge
  • Oven set at 37°C
  • CO2 incubator (37°C, 5% CO2 )
  • Refrigerator
  • −20° freezer
  • Certified fume hood
  • Hemocytometer
  • Pipettors
  • 100mm petri dish (Krackeler 97-4002)
  • 24 well flat bottomed tissue culture treated plates (Corning #83-3526)
  • 15 ml conical tubes (Sarstedt-red cap #62.554.002)
  • 50ml filter systems (Costar #430320)
  • 5ml syringes (BD #301603)
  • 0.22μm syringe filters (Krackler Scientific #431219)
  • 25cm filter paper (Fischer #9-801G)

Computer equipment

Suggested computer system

Windows 7 operating system, dual quad core Xeon X5570 processors (16 cores total with hyperthreading), 24 GB RAM, 2 TB ESATA external hard drive for image data storage, dual monitors. All timing information in the procedure below is based on this reference system.

Minimal computer requirements

Windows operating system (XP or later) for running LEVER.exe. Approximately 2 GB free disk space is required for the executables and sample data.

Downloading the software

Compiled Windows executables as well as all of the LEVER source code and sample image data and results (Data sets 1 and 2) can be downloaded from The software will be released under the GNU public license. Details of the license are described in the source code, but in essence it allows a user to use, modify and redistribute LEVER as long as the open source aspect is maintained.



Using aseptic technique in a biological safety cabinet, dilute 100mls of 10X PBS solution with 900mls of sterile water. This can be stored at room temperature for up to three years.

Hibernation (HIB) Buffer

To 1L sterile water add 2.4g KCl, 0.2g NAOH, 0.6g NAH2PO4, 0.1g MgCL2, 2.2g Na-pyruvate, 1.0g glucose, and 36.4g sorbitol. pH to 7.3 with NAOH. This can be stored at room temperature for up to one month.

Serum Free Culture Medium

Prepare using aseptic technique in a biological safety cabinet. For every 10mls of medium use 9.5mls DMEM, add 100μl L-glutamine, 100μl Na-Pyruvate, 100μl N2 supplement, 200μl B27 supplement, 10μl FGF2. Sterile filter using a 50 ml conical filter system. This can be stored at 4°C overnight.

Papain solution

To 5mls of DMEM add 10 units papain and 15μl DNase. Sterile filter through a 0.22μm syringe attached to a 5ml syringe. Incubate at 37°C for 30 minutes and use within an hour.

2X Phem Buffer

In a flask add 450 ml sterile water and a magnetic stir bar. Place on a stirrer. Add 40 pellets KOH (the KOH adjusts the pH to dissolve the PIPES buffer). Add in order: 18.15g PIPES, 5.95g HEPES, 3.80g EGTA, 0.812g MgCl2.6H2O. Add pellets of KOH one at a time until all PIPES is dissolved, then bring up to 500mls with sterile water. This should be used in the same day to make 4% Phem Fix. Recipe follows.


In a certified fume hood heat 450mls water to 60–70°C on a heated stirrer, add 40g of paraformaldehyde while stirring with a magnetic stir bar. Turn off heat and allow to dissolve. Add 1–2 drops 10N NaOH (or 1 pellet) to clear the solution and cool to room temperature. This should be used in the same day to make 4% Phem Fix. Recipe follows.

4% PHEM Fix

Combine 500ml 2X Phem Buffer with 500ml Paraformaldehyde. Check pH with pH strips in fume hood and adjust pH to 7.4 with NaOH. Vacuum filter through filter paper. This can be aliqouted and stored at −20°C up to 6 months.

Antibody Diluent

1X PBS with 0.1% Triton X-100 This can be stored at 4°C for 2 weeks.

Blocking solution

1X PBS with 0.1% Triton X-100 plus 10% normal goat serum. This can be stored at 4 °C for up to one week.


Preparing and imaging NPCs [diamond] timing : 5–15 days

  • 1)
    In a biological safety cabinet, coat a 24 well plate with poly-l-lysine by pipetting 500ul of poly-l-lysine into 8 wells and incubate at room temperature for 30 minutes. Rinse plates three times with sterile water and allow to dry.
  • 2)
    To isolate NPCs from the cerebral cortex of embryonic day (E)12 Swiss Webster mouse embryos, euthanize pregnant dams by cervical dislocation and remove E12 pups to a petri dish with HIB Buffer.
    CRITICAL STEP!: All animal procedures were approved by the University at Albany, Institutional Animal Care and Use Committee.
  • 3)
    Under a dissection scope in a clean dissection hood, euthanize embryos by decapitation and dissect cerebral cortices and place in HIB on ice.
  • 4)
    Transfer cortices to a 15 ml conical tube and allow them to sink to the bottom of the tube.
  • 5)
    Remove supernatant and add 5 mls papain solution and incubate on a rocker at 37°C for 30 minutes.
  • 6)
    Wash cells with DMEM three times by centrifugation at 405G for 10 minutes at 4°C.
  • 7)
    Count cells using a hemocytometer and plate at a density of 2000 cells/well in 8 wells of a poly-l-lysine coated 24 well tissue culture dish (from Step 1) in serum free culture medium as described previously17,18.
  • 8)
    Allow cells to settle in an incubator and then place under the Zeiss Axio-Observer Inverted Z1 microscope equipped with an incubation chamber. Open the Ziess Axiovision software and set up the experimental parameters using the multidimensional acquisitions tool. Specific instructions on how to use Ziess Axiovision can be found at$file/avguide.pdf. Image nine fields/well every five minutes with a 10X phase objective and 32ms shutter time for 5 to 15 days, refeeding every three days or as needed. CRITICAL STEP! Save all the field coordinates used.
  • 9)
    Export zvi image files from the microscope software as individual TIFF files. The file names must be of the form “datasetname_tXXX.tif”, where XXX is the zero-padded time stamp for the image. For example, “E120527090025_05_t001. tif “ and “E120527090025_05_t500. tif “ are both appropriate filenames. CRITICAL STEP! LEVER is able to work with an arbitrary sized timestamp (number of X’s), but the number of digits in the timestamp must be fixed for every image in the sequence. Users of microscope software other than the Zeiss Axiovision may need to write a batch file or conversion script to make sure the filename is in the appropriate format. A freeware imaging tool called “Irfanview” can be used for batch renaming.
  • 10)
    To stain cells for cell type specific markers such as the neuronal marker β-tubulin III in order to identify the progeny, firstly remove culture medium and fix cells with 0.5mls per well 4% PHEM fix at room temperature for 20 minutes.
  • 11)
    Incubate cells in 0.5 mls/ well blocking buffer for 30 minutes at room temperature.
  • 12)
    Remove blocking buffer and incubate cells with 0.5mls/well anti-β-tubulin III antibody diluted 1:600 in antibody diluent (4.0mls total volume) overnight at 4°C.
  • 13)
    Remove antibody solution, rinse with 0.5–1.0mls/well PBS and then incubate with 0.5mls/well Cy-5 conjugated goat anti-mouse IgG2B diluted 1:1000 in antibody diluents (4.0 mls total volume) for one hour at room temperature.
  • 14)
    Remove antibody solution and rinse three times in 0.5–1.0mls/ well PBS.
  • 15)
    Remove PBS and incubate in 0.5mls/well DAPI diluted 1:1000 in antibody diluents (4.0mls total) for 5 minutes at room temperature to label nuclei.
  • 16)
    Register the staining information with the timelapse recording by placing the tissue culture dish back on the microscope and revisiting the fields that were time-lapse recorded using the field coordinates saved in Step 1 in the Axiovision software. Capture fluorescent and phase contrast images for each marker per field. This information can be edited into the final lineage tree (see Step 23).

Installing the software [diamond] timing : 5–10 minutes

  • 17)
    Download the LEVER executables and/or source code from An example data set (Data Set 1) is also available for download. A reduced size data set (Data Set 2) is available for download in cases where available disk storage space is limited. Install the Microsoft Visual C++ runtime from the link on the webpage. If you do not have MATLAB version 2009b or later installed on your computer, you must also download and install the MATLAB runtime (MCRInstaller.exe, 170MB). Full instructions for installation are given on the webpage.
  • 18)
    Launch the software by double clicking on the LEVER.exe program from the folder where you installed the executables. The first time that the software executes there may be a one-time delay of up to 1 min. for LEVER to initialize. Alternatively, you may launch LEVER from within MATLAB by browsing to the source code folder and executing LEVER.m.

Segmenting, tracking and lineaging [diamond] timing : 5–15 minutes

  • 19)
    Select the first image in the data set that you wish to edit or process by browsing to the folder where you exported the image sequence data to (from Step 9). After the first image is selected, a dialog box allows you to either “Run Segmentation and Tracking” or to “Use Existing Data”. To begin processing a new data set, choose “Run Segmentation and Tracking” and choose a location and filename for the segmentation, tracking and lineaging results file. To continue with editing an existing data set or to try the software with the sample data, choose “Use Existing Data” and proceed to Step 20. CRITICAL STEP! Note that the segmentation algorithm utilizes all available computing resources. While the segmentation is running, the performance of other applications on this computer will be impacted. In particular, do not attempt to edit data while segmentation is running.
    [diamond] Pause point. Tracking and lineaging results are stored at this point for later editing. Proceed to Step 20 to begin editing these results.

Editing segmentation, tracking and lineaging results [diamond] timing : 30 minutes - 4 hours

  • 20)
    Select the results file for the image sequence of interest that was processed in Step 19. The name of the image sequence is posted on the title bar of the open file dialog box. If the data set name from the image data does not match the default naming convention LEVER will post a warning dialog.
    ! CRITICAL STEP Check that the results file chosen at this step matches the image sequence chosen in Step 19. LEVER will potentially allow (after posting a warning if the filenames do not match) segmentation, tracking and lineaging results to be displayed on the wrong image sequence data.
  • 21)
    Visualize results and navigate around the LEVER image and lineage windows using both the mouse and keyboard. Ideally, maximize each of the LEVER windows on separate displays. Activate the zoom-in tool and then click a point in the window to zoom centered at that point. This tool is especially useful as it allows the image data to be viewed in large detail. It is also possible to click and drag with the zoom-in tool activated to specify a rectangular region to zoom in on. Activate the pan tool and then click and drag to move the display data. Double-click with any of the zoom-in, zoom-out or pan tools selected to reset the window view. Left clicking on a cell in the image data window will display the lineage for that cell in the lineage window. The current image frame for LEVER can be controlled from either window. The current image time is indicated by the red line in the lineage window. Left clicking in the lineage window will move the red line. It is also possible to change the current time using the keyboard up and down arrow keys or the mouse scroll wheel to increment or decrement the current time. The PgUp and PgDn buttons move the current time forward or backward by five image frames. Playing the movie as an animation can be toggled by using the “Play” option from the “View” menu in either window. The “View” menu also allows you to enter the image frame you wish to view. Additionally, the “View” menu allows you to toggle the display of segmentation and tracking results. This feature is particularly helpful when trying to resolve visually ambiguous cell dynamics.
    ! CRITICAL STEP After you have completed using any of the view navigation tools you must be sure to deactivate that tool or subsequent keyboard or mouse inputs will be directed to the view navigation tool rather than to the LEVER application. Clicking an active tool button will deactivate the tool.
  • 22)
    Editing of segmentation results, tracking results, or lineaging to correct errors can be done following options A, B or C, respectively. Note that it is not necessary to fully correct all segmentations. Correct the segmentation as needed to enable the lineage tree to be correctly captured. For example, in cases where a mitosis is not visually apparent, a single segmentation can be split into two to allow the daughter cells to be labeled properly on the lineage tree.

A) Editing segmentation results to correct segmentation errors

  1. If multiple cells were segmented as a single cell, right-click on the single cell, select “Add Cells” from the context menu and then choose the number of cells to add.
  2. If a segmentation was missed completely, right-click on the background at the point where the segmentation should go, select “Add Cells” from the context menu and then choose “1” cells to add.
  3. Holding down the “shift” key while clicking a cell will split a single cell into two. Holding down the “ctrl” key while clicking will select multiple cells, indicated by dashed lines, which can then be deleted (“del” key) or merged (“enter” key).
    ! CRITICAL STEP Some edit operations (particularly segmentation corrections) will impact future results. Once you are done editing a frame, be sure to click the “Learn from Edits”, if it is enabled, to propagate your changes to future image frames.

B) Editing tracking results to correct tracking errors

  1. Tracking errors can be corrected either using “drag and drop” or by entering the correct tracking label directly. For “drag-and-drop”, if tracks were swapped between two cells, simply click on one cell and while holding the mouse drag over the top of the other cell and release. To enter tracking label directly, right-click on the cell and choose “Change Label”.

C) Editing lineage results to correct lineage errors

  1. Lineage errors can be corrected by adding or removing mitoses on either window. In the lineage window, right-click on the circular cell label and select “Remove Mitosis”. You will be prompted to select the daughter cell whose connection to the parent will be preserved. To add a mitosis, right-click on the vertical line defining the cell on the lineage tree and select “Add Mitosis”. Specify the sister cell and select “OK”. The same operations can be performed on the image data window as well by right-clicking within the cell segmentation outline.
    [diamond] Pause point. Editing may be saved and continued at any point until the lineage is fully corrected.

Exporting results [diamond] timing: 5–15 minutes

  • 23)
    Phenotype information can be specified for individual cells. Right-click on a cell and select the “Phenotype” option. The phenotype of “died” is included by default. Select “Create new phenotype…” to enter a new phenotype. When typing in the phenotype, standard LaTeX formatting applies, e.g. you can type “\beta” to display the symbol “β”. The phenotypes for the cells within a clone are drawn on the lineage tree for that clone. Select “View/Labels” to hide the tracking colors on the lineage tree making phenotype data easier to visualize or for export.
  • 24)
    Once all phenotype information has been entered choose “File/Export Lineage Tree” to create a new window showing the lineage without tracking numbers. You can print or save this lineage to a file. Finally, the legend showing phenotype information can be manipulated using the “arrow” tool found on the toolbar, allowing its location or font to be changed.
  • 25)
    A log of every editing operation, the user that performed the edit and the time (in seconds) the edit operation required is stored in the log file. The log file is given the same name as the data set being edited, but with the extension “_log.csv”. This file is stored as a comma separated value file and can be opened viewed and edited directly using any spreadsheet application.
    ! CRITICAL STEP LEVER will be unable to write to the log file if it is open for viewing in a spreadsheet program and will display a warning message prompting you to either close the log file or create a new one.
  • 26)
    Once a lineage has been completely edited, a movie combining the image data window with the lineage window using the current view settings can be exported. Choose “File/Export AVI” from the LEVER menu. The movie is exported as a cinepak compressed “.avi” file. The video shows the lineage tree in one half of the frame and the image data with segmentation and tracking overlaid in the other half. The current view settings from the lineage and image windows are preserved in the video; these should be set as desired prior to exporting. A free application for Microsoft called “Windows movie maker” can then be used to provide additional compression for different playback environments. Supplementary videos 1 and 2 were generated using this mechanism. Supplementary video 3 was generated by combining Supplementary video 2 and an adult NSC image sequence using a custom MATLAB script.
  • 27)
    The lineage data along with shape and motion parameters for each cell can be exported to a format suitable for opening with any spreadsheet program. Choosing “Export Cell Metrics” from the File menu will generate a comma separated value (“.CSV”) file containing the tracking label, parent, sibling offspring and root (originating cell for the lineage tree) labels, lifespan and the mean, standard deviation, maximum and minimum values for cell velocity and size. An example of this information is available in the Supplementary Results.
  • 28)
    The tracking results can be exported for use by the Algorithmic Information Theoretic Prediction and Discovery software tools4 by choosing “Export AITPD data” from the File menu. Analysis of the stem cell dynamic behaviors using AITPD requires additional MATLAB programming scripts. Details on using AITPD can be found in Cohen, et al.4


At the completion of this protocol, a user can expect to have gone from time-lapse image sequence data showing the development of one or more vertebrate NSCs imaged using phase contrast microscopy to a fully corrected lineage tree. A log file containing a complete list of the edits, with time, date and the id of the user creating the edits is generated. A movie showing the lineage tree simultaneously with the image sequence data with segmentation and tracking overlaid throughout the image sequence can be generated. Lineage statistics for every cell in the movie including parent-daughter cell relationships, and cell lifetimes as well as cell velocity and size information can be exported to a spreadsheet program for further analysis. Thus, digitization of the image sequence by LEVER will enable in depth statistical analysis of parameters such as changes in cell cycle time, cell motility, cell phenotype and overall lineage structure.

Figure 4 shows an example of a LEVER generated lineage tree with phenotypic information for Data set 1 (left panel). This lineage required approximately 30 min. to fully correct. Figure 4 also shows an example plot generated from the exported data showing average velocity, size and cell cycle time for the each of the NSCs in the lineage tree arranged by increasing order of division number (right panel). The full segmentation, tracking and lineaging results can also be exported for more sensitive analysis using the Algorithmic Information Theoretic Prediction and Discovery software tools4. These software tools were previously applied to predicting cell fate of individual NSCs, but have not yet been applied to studying clonal properties of development because of the difficulty in obtaining fully corrected lineages. The protocol presented here will enable the comprehensive study of the temporal dynamics of cell cycle time and population growth, motion, morphology and associations between cells across large numbers of clones.

Figure 4
Lineage tree and automatically generated cell statistics. Outcomes from the LEVER program include a fully corrected lineage tree as well as exported statistics describing cell cycle time as well as size and motion parameters for all of the cells on the ...


Steps 1–16: The image acquisition portion of the protocol typically requires 5 to 15 days for NSCs, but this could vary depending on the cell cycle time, proliferative potential and survival of the cell type under study. During this timeframe, the system can capture up to 100 movies simultaneously. Each movie typically contains a few clones.

Steps 17–18: installing the software takes 5–10 min.

Step 19: Obtaining segmentation, tracking and lineage results takes 5–15 min. Note that the time requirement will depend on the specific image sequence and on the computer used. The time requirement will also scale (nearly) linearly with the number of processor cores available. It is suggested that these steps be run on a dual quad core processor if possible.

Steps 20–22: Editing results for a single clone takes from 30 min. to four hours depending on the number of cells in the clone.

Steps 23–28: Exporting the results takes 5–15 min.

This sample timing information is for a 504 frame image sequence, with each image containing 1344 × 1024 pixels, an average of 46 cells per frame, and a maximum of 120 cells in a single frame.


Troubleshooting advice can be found in Table 1.

Table 1
Troubleshooting approaches for specific procedural steps.

Supplementary Material

Supplementary Table 1

Comparison of the MAT tracking algorithm used by LEVER with the bipartite matching tracker originally developed in Al Kofahi, et al.5, and refined in Cohen, et al.4 Although automatic segmentation, tracking and lineaging algorithms continue to improve, even very low error rates translate to a significant number of editing operations in order to generate a fully corrected lineage tree.

Supplementary results

This Excel spreadsheet contains the results of exporting cell metrics from LEVER for Data Set 1. The data was sorted by field “origin cell”. All cells whose origin cell was 243 (the lineage that we wish to analyze) were copied to the second worksheet labeled “clone 243”. These cells were then sorted by the “first frame” field. The cell cycle time, average speed and average area were then converted to appropriate units and plotted to produce panel B of Figure 4.

supplementary video 1

This video shows the corrected lineage corresponding to Supplementary Data Set 1. This video was generated directly from LEVER (protocol, Step 26).

supplementary video 2

This video shows a second corrected lineage. The image sequence data for this lineage was generated as described Step 1 of this protocol, but was not provided due to space restrictions on the web server. This video was also generated directly from the LEVER program. This lineage was used to create Figure 2.

supplementary video 3

This video shows an adult NSC lineage in the top frame and an embryonic NSC lineage below. The embryonic lineage is the same shown in Supplementary Video 2. Both lineages are zoomed to show cells on the lineage more closely. These movies were generated individually from the LEVER program. A separate MATLAB script then combined the two movies and added the white 25 μm scale bars.


This project was supported by the University of Wisconsin-Milwaukee, and by grant number NS033529 from the National Institutes of Health (NIH) and by New York State Department of Health (NYSDOH) Contract C024352 from the Empire State Stem Cell Fund and the Regenerative Research Foundation. This project was also supported by NIH grant number R01EB005157 and by Grant Number R01NS076709 from the National Institute Of Neurological Disorders And Stroke. The authors would like to thank Samareh Shahmohammadi, Ryan Ference, Aislyn DiRisio, Sarah Hardwick, Sarah M. Goderie, Erin McAuley, Raymond Futia and Claire Davenport for their help with cell culture and lineage validation.



M.W., E.W., B.R. and A.R.C. designed and implemented the computational tools and algorithms, prepared the figures and wrote the manuscript. R. A. N. A. did cell preparation and maintenance, helped with staining and verified lineage trees. S.K.G., E.K. and S.T. cultured and imaged the NSCs, provided manual lineage reconstructions for comparison, edited the automated lineages, made suggestions on the design of the software tools from a biological perspective and helped prepare figures and write the manuscript.


The authors declare that they have no competing financial interests.


1. Giebel B, Punzel M. Lineage development of hematopoietic stem and progenitor cells. Biological chemistry. 2008;389:813–824. [PubMed]
2. Ogawa M. Differentiation and proliferation of hematopoietic stem cells. Blood. 1993;81:2844–2853. [PubMed]
3. Cohen AR, Bjornsson C, Temple S, Banker G, Roysam B. Automatic Summarization of Changes in Biological Image Sequences using Algorithmic Information Theory. IEEE Trans Pattern Anal Mach Intell. 2009;31:1386–1403. [PubMed]
4. Cohen AR, Gomes F, Roysam B, Cayouette M. Computational prediction of neural progenitor cell fates. Nat Methods. 2010;7:213– 218. [PubMed]
5. Al-Kofahi O, et al. Automated cell lineage tracing: a high-throughput method to analyze cell proliferative behavior developed using mouse neural stem cells. Cell Cycle. 2006;5:327– 335. [PubMed]
6. Gonzales R, Woods R, Eddins S. Digital Image Processing Using MATLAB. Gatesmark Publishing; Knoxville, TN: 2009.
7. Papadimitriou CH, Steiglitz K. Combinatorial Optimization: Algorithms and Complexity. Dover Publications; Mineola, NY: 1998.
8. Koller D, Friedman N. Probabilistic Graphical Models:Principles and Techniques. MIT Press; Cambridge, Massachusetts: 2009.
9. Ravin R, et al. Potency and fate specification in CNS stem cell populations in vitro. Cell Stem Cell. 2008;3:670–680. [PubMed]
10. Murray JI, Bao Z, Boyle TJ, Waterston RH. The lineaging of fluorescently-labeled Caenorhabditis elegans embryos with StarryNite and AceTree. Nat Protocols. 2006;1:1468–1476. [PubMed]
11. Winter M, Fang C, Banker G, Roysam B, Cohen A. Microscopic Image Analysis with Applications in Biology. Chicago IL: 2011. Axonal Transport Analysis Using Multitemporal Association Tracking. in press. [PMC free article] [PubMed]
12. Shen Q, et al. The timing of cortical neurogenesis is encoded within lineages of individual progenitor cells. Nat Neurosci. 2006;9:743–751. [PubMed]
13. Gomes FLAF, et al. Reconstruction of rat retinal progenitor cell lineages in vitro reveals a surprising degree of stochasticity in cell fate decisions. Development. 2011;138:227–235. [PubMed]
14. Eilken HM, Nishikawa SI, Schroeder T. Continuous single-cell imaging of blood generation from haemogenic endothelium. Nature. 2009;457:896–900. [PubMed]
15. Li K, et al. Cell population tracking and lineage construction with spatiotemporal context. Medical Image Analysis. 2008;12:546–566. [PMC free article] [PubMed]
16. Chen Y, Ladi E, Herzmark P, Robey E, Roysam B. Automated 5-D analysis of cell migration and interaction in the thymic cortex from time-lapse sequences of 3-D multi-channel multi-photon images. J Immunol Methods. 2009;340:65–80. [PMC free article] [PubMed]
17. Qian X, et al. Timing of CNS cell generation: a programmed sequence of neuron and glial cell production from isolated murine cortical stem cells. Neuron. 2000;28:69–80. [PubMed]
18. Shen Q, et al. Endothelial cells stimulate self-renewal and expand neurogenesis of neural stem cells. Science. 2004;304:1338–1340. [PubMed]