Toward the goal of achieving complete automation and consistency for stem cell culture expansions, we have developed an automated computer vision-based system for continuous monitoring and analysis of in vitro
cell cultures to objectively determine the appropriate time to subculture cells based on confluency measurements (). The main impetus for this work is that in the current state-of-art automated cell culture machines are capable of executing the cell passing procedure with high precision and minimal variability 
, however, they lack the ability to make adaptive decisions based on how fast or slow a culture of cells are growing. Furthermore, in manually-operated systems, estimation of confluency by human operators is highly subjective and dependent on the experience of the operator. The confluency measurement algorithm developed here may additionally serve as a useful training aid for new tissue culture users and help to reduce the subjective nature of confluency estimation.
This system employs phase-contrast microscopy as it is a non-destructive imaging modality that is capable of generating high-contrast images of transparent specimens such as cells 
. Thus, cells can be easily visualized and imaged at high frequency, allowing the user to continuously monitor live cultures without affecting cell viability. Although an image acquisition period of 5 minutes was utilized in these experiments, cells can be imaged at a higher or lower frequency in accordance to experimental requirements, with computer hard disk space and computer algorithm run-time being the only limitations. After image acquisition, the archived data is sent to a server for processing () and 2nd
order polynomial curve fitting is used to predict future confluency (, and ). Although the system is designed to be eventually autonomous when combined with a robotic-handling cell system, a human operator can remotely monitor the current and predicted levels of confluency as well as individual fields of view over the Internet (). 4 hours prior to reaching a predefined threshold for confluency of 0.5, the human operator is alerted by the computer vision system via email and text messaging () to facilitate preparations such as warming media and various other reagents for cell culture. An additional reminder is sent when the predefined threshold for confluency is reached ().
Although a standard computer with a dual core processor and 2 GB Ram is sufficient for implementing the confluency measurement and predictions on a local machine, a server (cloud computing cluster) was utilized so that multiple experiments can be conducted in parallel from different microscope computers or locations. This capability was recently demonstrated for single-cell tracking experiments using image data acquired simultaneously from Tokyo, Japan and Pittsburgh, USA (Intel Developer Forum 2010, California, USA; data not shown). Rsync was chosen for facilitating file transfers because it utilizes a delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination 
. The algorithms employed in this automated computer vision-based system are efficient and can process one set of images (12 phase-contrast images) prior to acquisition of the next set (within 5 minutes), enabling real-time monitoring and analysis of cell cultures. Although a high image acquisition rate is unnecessary for predicting confluency 4 hours ahead of time (Figure S2
), it was used to demonstrate that accumulated photonic energy from a high image rate is non-detrimental to cell growth and that it would be feasible to eventually incorporate it into a real-time cell tracking software to monitor actively migrating and proliferating cells (Intel Developer Forum 2010, California, USA; data not shown), a process requiring high image rates. Furthermore, although less frequent data acquisition is adequate under normally progressing culture conditions (Figure S2
), more frequent acquisition times would enable earlier identification of problems in a given culture, which may facilitate earlier interventions to mediate against possible loss of expensive or unique stem cell culture populations.
Confluency, which is the percentage of the surface area in the cell culture vessel covered by cells, is traditionally used by cell biologists as a convenient indirect way to estimate the number of cells in a cell culture vessel. Thus, measuring confluency of an image area occupied by cells is an appropriate measure for estimating cell growth rates. The confluency algorithm utilized here segments cells on the basis of thresholding on the local intensity value of image pixels, a frequently applied methodology used in image segmentation. However, phase-contrast microscopy images have several characteristic halo and shade-off artifacts that arise as a result of technical limitations in the optical assembly of a phase-contrast microscope 
. To improve segmentation results, a restoration process was applied to remove the halo and shade-off patterns to reconstruct an artifact-free image 
. The output of the confluency algorithm has good correspondence with manually annotated confluency () with a precision of 0.791±0.031 and recall of 0.559±0.043 (). Precision and recall were used to assess the performance of the computer-generated confluency measurements because they are widely employed performance metrics used for pattern recognition algorithms. In general, a precision and recall close to 1.0 indicates good recognition performance. Although some discrepancies between the human- and computer-generated confluency masks were observed, these errors stemmed largely from large, well-spread out cells whose cellular processes were difficult to discern from the image background (iv), resulting in low recall (0.559±0.043) owing to higher false negatives (iv and ). However, the precision (0.791±0.031) of the computer-generated confluency measurements is fairly high and these discrepancies in confluency measurement ultimately did not adversely impact C2C12 cell growth and differentiation (, and ). Given that it is possible to segment cells with ease using computer vision technology, future improvements to the algorithm will incorporate additional measures such as cell density and assessment of cell shape, which may further inform cell culture decisions such as determining the percentage of differentiated cells in culture.
To make confluency predictions, several data fitting models were empirically tested using previously acquired time-lapse phase contrast microscopy images of C2C12 cells and it was determined that that a 2nd
order polynomial model produced the best curve fit (Figure S1
) and that approximately 200 frames were required before prediction becomes reliable and the variance becomes lower than 0.001 (). A 4 hour advance notification was used in conjunction with the prediction model because this was a sufficient time window to allow for planning if personnel were on site, or for a reasonable transport timeframe back to the laboratory, and/or preparation for subculture. Although a 2nd
order polynomial was empirically found to be most suitable for predicting C2C12 cell confluency (Figure S1
) and adequate for expanding C2C12 cells (, and ), this growth model may prove inadequate when culture conditions are altered or when different cell types are used. This is due to the sensitive nature of cells to their environments, which can be severely impacted by variability arising from cell handling, cell passage number, undefined media components such as fetal bovine serum as well as differences in growth and cell spreading rates when considering different cell types. For example, a population of cells that has been recently thawed from liquid nitrogen storage will display a slower rate of growth when compared to the same population after several passages. Mathematical models such as a 2nd
order polynomial do not take perturbations in cell culture conditions into account and will exhibit poor performance when used under such scenarios. We have recently developed a data-driven approach 
to build a confluency prediction model specific for C2C12 cells based on previously acquired data. As long as cell culture conditions are constant, this model is able to predict confluency at least 8 hours in advance with a low error rate 
and this will be incorporated into subsequent work.
This system was successfully used to direct the expansion of C2C12 cells with the confluency threshold set at 0.5 (, and ). C2C12 cells are utilized as a paradigm stem cell population because they are a multipotent cell line that has been previously shown to differentiate into cells of the musculoskeletal system 
. In addition, C2C12 cells are sensitive to the level of confluency and must not be allowed to become confluent, otherwise, cells will spontaneously fuse and deplete the progenitor or stem cell population. Thus, this cell line serves an excellent model for testing our real-time adaptive subculture system. Both human- and computer-directed subculture experiments each required 3 serial passages to theoretically reach a target number of 50 million cells () and showed no significance difference (p
0.308) in average cell yields per dish ().
Following cell expansion, C2C12 cells were differentiated towards osteogenic and myogenic fates to confirm that cells retained their capacity to differentiate into multiple cell types such as osteoblasts and myocytes ( and ). In addition, undifferentiated mononuclear cells were shown to retain their stem cell identity as evidenced by positive staining for the stemness marker, Pax7 ().
It is interesting to note that although the computer-directed cell expansion had lower variability in terms of cell yield compared to the human-directed cell expansion (), it is not drastically different. This result may stem from the limitation that only 12 fields of view were utilized in this experiment. Although attempts were made to ensure that the 12 fields of view were representative of culture conditions in each of the 4 dishes, it is possible that sampling a larger number of fields of view may lower the variability observed from cell yield. In addition, the use of a hemocytometer for manual cell counting may have also contributed to an increase in cell yield variability due to sampling error.
Given that stem cell manufacture for different applications, including large-scale production for multiple patient use and small-scale production for autologous use, requires extensive ex vivo
handling and expansion of cells from various tissue sources 
, a QA/QC system is vital to ensure robust production of cells that are of consistent quality. Although our current system does not incorporate warning alarms to alert users of potential problems associated with cell growth, such problems are easily noticed when overall cell confluency decreases. In such scenarios, individual fields of view can be observed remotely to identify problems (). Future work will incorporate additional algorithms that detect and measure cell behaviors such as mitosis 
, apoptosis and cell fusion to provide a more comprehensive view of cell population behavior for maximizing stem cell growth while minimizing stem cell differentiation. Although the use of a microscope stage incubation chamber limited the number of petri dishes that can be observed at a particular given moment, it provided a representative overview of cell growth and confluency to facilitate manual stem cell production with little to no obvious loss in myogenic and ostegenic potential (, and ). In addition, our system can be integrated with existing commercially available robotic technology for handling cell culture flasks, allowing for the confluency of every individual flask or dish to be monitored provided that the time required for imaging the desired number of cell culture vessels does not exceed the image acquisition rate. In scenarios where a large numbers of cell culture vessels must be monitored, multiple instruments and computers may be employed in parallel to decrease the time required for image acquisition and image processing.
In summary, we have developed an automated computer vision-based system for adaptive subculture of stem cells based on confluency. Using mouse C2C12 cells as a paradigm stem cell population, this study demonstrated that both human- and computer-directed cell expansions had similar performance in terms of the number of serial passages required to reach a target cell yield. Furthermore, both human- and computer-expanded cell populations were capable of differentiating towards osteoblast and myocyte fates, indicating that stem cell capacity was not lost during cell expansion. This capability offers an approach to reproducibly expand cell populations and may have applications in the manufacture of clinically-relevant cells and/or their cell-derived products. Future work on this system will move towards complete automation of cell culture and QA/QC along with improved algorithm accuracy.