scMARK an 'MNIST' like benchmark to evaluate and optimize models for unifying scRNA data

Here we present a novel benchmark dataset (scMARK.v2), that consists of 11 published cancer scRNA-seq studies, for which we standardized  cell-type author labels and gene identifiers. scMARK.v2 can be used to ask how well models integrate data from different scRNA studies. We also provide a 12th standardized study (Wu et al 2021) that we held-out for evaluation of alignment of data "never seen" before, and a 13th study of newly generated in-vitro scRNA-seq data from cancer and fibroblast cells.

Data is provided as aData *h5ad files that can be read with Python's library Scanpy.

Studies inclided in scMARK.v2 were downsampled to 10,000 cells per study.

The difference between scMARK.v1 and scMARK.v2, is that in v2, we provide at least two studies for each cancer type and each cell type; whereas in v1 a handfull of cell types were present only in one study.