Zebrafish data for whole-embryo lineage reconstruction with linajea

This article provides access to the zebrafish data (160328) for "Automated reconstruction of whole-embryo cell lineages by learning from sparse annotations" (Malin-Mayor et al. 2023, DOI: https://doi.org/10.1038/s41587-022-01427-7).Here we provide the ground truth tracks used to train the deep learning model, the trained networks, and the predicted tracks. Additionally, we provide information on how to access the image data, although it is not uploaded here due to size. Related artifacts include the source code for experiments and methods.Image DataThe image dataset in n5/zarr format (as used in Malin-Mayor et al. 2023) can be accessed at the following Dropbox link: https://www.dropbox.com/scl/fi/1qeac8uwvctq9q1y451lg/160328_zebrafish.tar.gz?rlkey=whs5hyxtacdet0ypwyroigara&dl=0. This image dataset was originally published in "Single-cell reconstruction of emerging population activity in an entire developing circuit" ( Wan, Y. et al. 2019, DOI: https://doi.org/10.1016/j.cell.2019.08.039). The image dataset has two channels, corresponding to two camera views that have been registered but not merged. While originally these channels each were anisotropic with a voxel size ratio of 6:1, we have resampled them to be isotropic by downsampling the dimension with larger resolution by 2 and upsampling the two dimensions with smaller resolution by 3.Ground Truth TracksInside gt_tracks.zip there are a number of files containing different subsets of tracks. Each has the following columns separated by tabs: time, z, y, x, cell_id, parent_id, track_id.tracks_side_1.txt and tracks_side_2.txt are the main files containg manual annotations of individual cells from start to end of video used to train the model. These tracks are sparse, but each cell included had its whole lineage traced as completely as possible from start to end of the video. These tracks were split based on which side of the center line of the embryo the cells were on: discarded.txt contains tracks that crossed the center line and were not used for training or testing.full_frame_divisions.txt is a set of manually annotated division points (points right before the cell divides) that are as complete as possible for target time points 50, 100, 150, 200, 250, 300, and 350 and adjacent frames, and were used for evaluation and not model training.Trained Modelstrained_networks.zip includes both networks trained on the zebrafish dataset. The config files are in the zebrafish_config_files directory, and the other directories are named corresponding to the model_name in each config file and contain the trained model files. There is one model trained/validated on each side of the embryo, and evaluated on the other side, as described in Supplemental Note 1.Predicted Trackspredicted_tracks.zip contains both the TGMM baseline results and the results for the linajea method. Each has the following columns separated by tabs: time, z, y, x, cell_id, parent_id, track_id, [node_score, edge_score] (node and edge score only present for lineajea predictions).The TGMM directory contains the TGMM results provided to us by the authors of the TGMM method. There is a separate result for each input channel.The linajea results are organized similarly to the trained models, with one text file for each side of the embryo. zebrafish_side_1_tracks_071621.txt contains tracks generated by the model trained on side 2 and predicted on side 1, and zebrafish_side_2_tracks_071621.txt contains tracks generated by the model trained on side 1 and predicted on side 2. Predictions were masked via a rough rectangle to only be done on the test "side", to save computation.本文提供了《基于稀疏标注学习的全胚胎细胞谱系自动化重建》（Automated reconstruction of whole-embryo cell lineages by learning from sparse annotations；Malin-Mayor等，2023，DOI：https://doi.org/10.1038/s41587-022-01427-7）一文所使用的斑马鱼数据集（标识号：160328）的获取方式。本数据集包含用于训练深度学习模型的真实标注轨迹、训练完成的神经网络模型，以及预测得到的细胞轨迹。此外，我们还提供了图像数据的获取途径，但由于文件体积过大，未随本数据集一同上传。相关配套资源还包括实验与方法的源代码。 图像数据 采用n5/zarr格式（如Malin-Mayor等2023年研究中所使用）的图像数据集，可通过以下Dropbox链接获取：https://www.dropbox.com/scl/fi/1qeac8uwvctq9q1y451lg/160328_zebrafish.tar.gz?rlkey=whs5hyxtacdet0ypwyroigara&dl=0。该图像数据集最初发表于《完整发育环路中新兴群体活动的单细胞重建》（Single-cell reconstruction of emerging population activity in an entire developing circuit；Wan Y.等，2019，DOI：https://doi.org/10.1016/j.cell.2019.08.039）。该数据集包含两个通道，分别对应两组已配准但未融合的相机拍摄视角。尽管原始数据中每个通道的体素（voxel）均为各向异性，体素尺寸比为6:1，但我们已将其重采样为各向同性：对分辨率较高的维度进行2倍下采样，对分辨率较低的两个维度进行3倍上采样。 真实标注轨迹 在gt_tracks.zip压缩包中，包含多个存储不同轨迹子集的文件，所有文件均采用制表符分隔的列格式，字段依次为：时间（time）、z坐标、y坐标、x坐标、细胞ID（cell_id）、父细胞ID（parent_id）以及轨迹ID（track_id）。 tracks_side_1.txt与tracks_side_2.txt为核心文件，存储了用于模型训练的、从视频起始至结束的人工标注单个细胞轨迹。此类轨迹为稀疏标注，但每一个被标注的细胞均已尽可能完整地追踪了其从视频起始到结束的完整细胞谱系。轨迹根据细胞所处的胚胎中轴线两侧进行划分：discarded.txt文件存储了跨越中轴线的轨迹，此类轨迹未被用于模型训练与测试。 full_frame_divisions.txt文件存储了针对目标时间点50、100、150、200、250、300、350及其相邻帧的人工标注细胞分裂点（细胞分裂前的瞬时位置），该数据集尽可能覆盖完整，仅用于模型评估，未参与模型训练。 训练完成的模型 trained_networks.zip压缩包包含两个基于本斑马鱼数据集训练得到的神经网络模型。配置文件存储于zebrafish_config_files目录中，其余目录的命名与各配置文件中的model_name字段一一对应，目录内存储训练完成的模型文件。如补充说明1所述，本数据集分别在胚胎两侧训练/验证模型，并在对侧进行模型评估。 预测轨迹 predicted_tracks.zip压缩包包含TGMM基线模型结果与linajea方法的预测结果，所有结果均采用制表符分隔的列格式，字段依次为：时间（time）、z坐标、y坐标、x坐标、细胞ID（cell_id）、父细胞ID（parent_id）、轨迹ID（track_id），以及[节点得分（node_score）、边得分（edge_score）]（仅linajea预测结果包含该两项得分）。 TGMM目录存储了由TGMM方法原作者提供的模型结果，每个输入通道对应一份独立的预测结果。 linajea的预测结果组织方式与训练完成的模型类似，胚胎每一侧对应一个文本文件。其中zebrafish_side_1_tracks_071621.txt存储了以侧2数据训练的模型对侧1数据的预测轨迹，zebrafish_side_2_tracks_071621.txt存储了以侧1数据训练的模型对侧2数据的预测轨迹。为节省计算资源，预测过程通过粗略矩形掩码限定仅在测试侧范围内执行。