Research Focus
One major goal in developmental biology is to understand cell lineages in developing organisms: Where do the cells move during embryogenesis, where and when do they divide, and which cells will later form which organs? To this end, we focus our research in this project on cell tracking - the automatic reconstruction of cell lineages from microscopic image sequences - and develop robust models with high usability for users from life science.
So as to automatically reconstruct the cell lineage of growing organisms - referred to as digital embryo - cell tracking aims at finding associations between cells over time. This is a challenging problem in scientific computing due to the unknown and variable number of cells. In particular, up to ten thousands of cells in each time step need to be detected and tracked over time. Notice that cells may divide and it is a crucial goal to detect all of those cell divisions. We develop machine learning algorithms which can cope with these challenges and are generally applicable for the tracking of arbitrary, potentially dividing, objects. In a parallel line of research, we establish structured learning schemes to automatically track these objects after a learning phase, where the user provides (partially) labeled examples of tracks. The software we develop facilitates high-throughput analysis in a user-friendly way. By staying in close collaboration with our partners from life science we ensure to match the requirements of their research field.
Digital Embryo: Automated Reconstruction of Cell Lineage Trees
We establish an automated pipeline for the analysis of time-lapse images of developing embryos.This workflow has already been applied successfully to Drosophila,Zebrafish,C. elegans, and Arabidopsis.
In a first step, our collaborators use light sheet fluorescence microscopy, e.g. MuVi-SPIM,to image the growing embryos at the single cell level. The acquired data is a 3d+t image sequence with isotropic spatial and high temporal resolution.
We process this data by first segmenting the cell nuclei for each time step separately. For this purpose, we either use our open-source software Ilastik or our segmentation methods described in [Lou et al., 2011] and [Lou & Hamprecht, 2012].
This step is followed by the actual tracking, i.e. finding temporal associations between potential cells. By considering cells in pairs of frames [Lou et al., 2011],our method can accurately track thousands of cells in less than only two minutes. To generalize this approach and to address errors in the segmentation step, we consider the optimization over all time steps at once. To this end, we propose a probabilistic graphical model [Kausler et al., 2012] for tracking, which - due to the temporal context - identifies misdetections from the preceding segmentation step and finally corrects them. Hence, this method provides greater robustness to misdetections and yields consistent cell tracks. Furthermore, by using this tracking-by-assignment approach, our framework facilitates the integration of biological rules into the tracking model. One of those rules could be a minimal cell cycle length between divisions of the same cell, which boosts the performance for the detection of cell divisions. In [Schiegg et al., 2013], we extend this approach to further deal with undersegmented objects, such as merged cells due to segmentation errors. The number of cells comprised by each detected object and the cell assignments between all frames are jointly inferred using a probabilistic graphical model. In a postprocessing step, ellipses are fit to these falsely merged cells to resolve their original identities. To go one step further, in order to make the segmentation and tracking steps benefit most from each other,we propose in [Schiegg et al., 2014] to optimize segmentation and tracking completely jointly in one model. From a sequence of oversegmented images, superpixels/-voxels are assigned both to its corresponding cell (or background) and to the lineage of this cell.The joint optimization allows for maximal synergy effects between these previously independent steps in the pipeline. We provide the Drosophila dataset used in this publication along with our manual cell tracking annotations here.
Finally, the results of our cell tracking pipeline can be visualized, e.g. by coloring each cell at a reference point and marking the same cell as well as their daughter cells in the remaining movie with the same color.
In order to further improve the accuracy of the tracking results, we present in [Schiegg et al., 2015] techniques for guided proof-reading. The local costs in a tracking-by-assignment model are perturbed according to (i) a globally parameterized probability distribution, or (ii) locally determined uncertainty derived from Gaussian process alignification. The variability across the sampled solutions allows to compute uncertainty measures which are efficient to automatically guide the user to potential errors in the result.
References
M. Schiegg, B. Heuer, C. Haubold, S. Wolf, U. Koethe, F. A. Hamprecht.Proof-reading Guidance in Cell Tracking by Sampling from Tracking-by-assignment Models. In: ISBI 2015, in press.[ Technical Report ] M. Schiegg*, P. Hanslovsky*, C. Haubold, U. Koethe, L. Hufnagel, F. A. Hamprecht. Graphical Model for Joint Segmentation and Tracking of Multiple Dividing Cells.Bioinformatics, in press,2014. [* contributed equally] [ PDF | Code | Dataset along with manual cell tracking annotations ] M. Schiegg, P. Hanslovsky, B. X. Kausler, L. Hufnagel, F. A. Hamprecht. Conservation Tracking.In: ICCV 2013, 2928-2935[Technical Report | Code | GUI (Conservation Tracking workflow in ilastik) ] B. X. Kausler, M. Schiegg, B. Andres, M. Lindner, H. Leitte, L. Hufnagel, U. Koethe, F. A. Hamprecht. A Discrete Chain Graph Model for 3d+t Cell Tracking with High Misdetection Robustness.In: ECCV 2012, 144-157. [ 10.1007/978-3-642-33712-3_11 |Technical Report | Code ] X. Lou, F. A. Hamprecht. Learning to Segment Dense Cell Nuclei with Shape Prior.In: CVPR 2012, 1012-1018. [10.1109/CVPR.2012.6247778 | Technical Report X. Lou, F. Kaster, M. Lindner, B. Kausler, U. Köthe, B. Höckendorf, J. Wittbrodt, H. Jänicke, F. A. Hamprecht. DELTR: Digital Embryo Lineage Tree Reconstructor. In: ISBI 2011, 1557-1560 [ 10.1109/ISBI.2011.5872698 | Technical Report ]Structured Learning for Tracking
Inspired by the cell tracking project described above, we follow another line of research to allow for automatic learning of tracks by user-provided annotations. In particular, to avoid the effort of manually tuning the parameters of tracking models, we propose structured learning based methods for multiple-object tracking: Our learning based method in [Lou & Hamprecht, 2011] learns the parameters of an energy function to find a maximal discrimination between the correct association and any other (false) solution. In an intuitive fashion, the expert user provides examples of true object assignments, from which our algorithms automatically learn the feature weights and predict tracks for unlabeled objects. In this way, these approaches facilitate the usage of a huge variety of higher-order features in tracking, which yields a notable boost in tracking performance. An extended version of this paper, augmented with an active learning scheme for cell tracking, is published in [Lou et al., 2014]. Although this approach leads already to significant improvement over methods with manual parameter tuning, in practice, the annotation of entire tracks turns out to be cumbersome, especially for large datasets. To improve usability further, we propose a model which can cope with partially annotated tracks [Lou & Hamprecht, 2012b] due to an extension of these structured learning methods together with optimization speed-ups. With only 40% of the data being labeled, we can even outperform our aforementioned method.
X. Lou, M. Schiegg, F. A. Hamprecht. Active Structured Learning for Cell Tracking: Algorithm, Framework and Usability.In: IEEE Transactions on Medical Imaging [ 10.1109/TMI.2013.2296937 | Technical Report | Code ] X. Lou, F. A. Hamprecht.Structured Learning from Partial Annotations.In: ICML 2012.[Technical Report | Code] X. Lou, F. A. Hamprecht. Structured Learning for Cell Tracking.In: NIPS 2011, 1296-1304.[Technical Report | Code]