Replication Data for: Physics based machine learning for sub-cellular segmentation in living cells

Abstract: Segmenting sub-cellular structures in living cells from fluorescence microscopy images is a ground truth (GT) hard problem. The microscope’s 3-dimensional blurring function, finite optical resolution due to light diffraction, finite pixel resolution, and complex morphological manifestations of the structures, all contribute to GT-hardness. Unsupervised segmentation approaches are quite inaccurate. Manual segmentation relying of heuristics and experience therefore remains the preferred approach. Nonetheless, this process is tedious, with 100s of structures present inside a single cell. Thus, generating analytic across large population of cells or performing advanced artificial intelligence (AI) tasks such as tracking is greatly limited. We bring modeling and deep learning to a nexus for solving this GT-hard problem, improving both the accuracy and speed of sub-cellular segmentation. We introduce the approach of simulation-supervision empowered with physics-based GT. In the simulation-supervision datasets used for training, the physics-based GT resolves the GT hardness while computational modeling of all the relevant physics aspects assists deep learning models to learn to compensate for physics and instrument induced limitations to a great extent. We show extensive results on segmentation of small vesicles and large diversity of mitochondria in diverse independent living and fixed cell datasets, demonstrate the adaptability of the approach across diverse microscopes through transfer learning, and illustrate biologically-relevant applications of automated analytic and motion analysis.