Designing a single cell RNA sequencing benchmark dataset to compare protocols and analysis methods (9 cell mixture dataset).

Single cell RNA sequencing (scRNA-seq) technology has undergone rapid development in recent years and brings new challenges in data processing and analysis. This has led to an explosion of tailored analysis methods for scRNA-seq to address various biological questions. However, the current lack of gold-standard benchmarking datasets makes it difficult for researchers to evaluate the performance of the many methods available in a systematic manner. Here, we designed and generated a cross-platform benchmark dataset that has in-built truth in various forms and varying levels of biological noise. We used this dataset to compare different protocols and data analysis methods. We found that different protocols have different data quality and ERCC spike-in works independently to endogenous RNA. We found significant differences in the results from the methods compared and we associated the results with data characteristics to identify methods that perform well in different situations. Our dataset and analysis provide a valuable resource for algorithm selection in different biological settings. Overall design: our experiment utilized the 3 human lung adenocarcinoma cell lines H2228, H1975 and HCC827. The experiment included mixtures of RNA and single cells from these cell lines. For the single cell designs, the three cell lines were mixed equally and processed by 10X chromium, Drop-seq and CEL-seq2, referred to as sc_10X, sc_Drop-seq and sc_CEL-seq2 respectively in analysis that follows. For the mixture designs, we used plate-based protocols to mix and dilute samples in 2 different ways. 9 cell mixtures from the 3 cell lines were sorted in different combinations in the cell mixture experiment and data were generated by CEL-seq2, the material after pooling from 384 wells were subsampled in either 1/9 or 1/3 to simulate cells of different sizes, with different PCR product clean up ratios ranging from 0.7 to 0.9, referred to as cellmix1 to cellmix4. For the cell mixture experiment, we also sorted wells with 10 times more cells (90 cells) to provide a pseudo bulk reference for each mixture (referred to as cellmix5). Distinct RNA mixtures which were diluted down to create single cell equivalents (ranging from 3.75, 7.5, 15 to 30 pg per well) were generated using CEL-seq2 and SORT-seq (referred to as RNAmix_CEL-seq2 and RNAmix_Sort-seq. This is the 9 cell mixture dataset.

单细胞RNA测序（single cell RNA sequencing，缩写scRNA-seq）近年来发展迅猛，为数据处理与分析带来了全新的挑战。由此催生了大量针对单细胞RNA测序的定制化分析方法，以应对各类生物学问题。然而当前缺乏金标准基准数据集，使得研究者难以系统性地评估现有众多方法的性能。
在此背景下，我们设计并构建了一款具备多形式内置真值、且包含不同水平生物学噪声的跨平台基准数据集，并利用该数据集对比了不同实验方案与数据分析方法。我们发现，不同实验方案的数据质量存在差异，且ERCC spike-in对照与内源RNA的表达相互独立。我们还观察到，不同分析方法得到的结果存在显著差异，并将结果与数据特征相关联，以识别在不同场景下表现优异的方法。
本数据集与分析结果为不同生物学场景下的算法选择提供了宝贵的参考资源。
总体实验设计：本实验采用了3株人肺腺癌细胞系：H2228、H1975与HCC827。实验包含RNA混合物与来自这些细胞系的单细胞样本两种设计方案。
针对单细胞实验设计，我们将3株细胞系按等比例混合，分别通过10X chromium、Drop-seq与CEL-seq2三种平台进行测序，在后续分析中分别记为sc_10X、sc_Drop-seq与sc_CEL-seq2。
针对RNA混合物实验设计，我们采用基于微孔板的实验流程，以两种不同方式进行样本混合与稀释。在细胞混合物实验中，我们基于3株细胞系的不同组合构建了9种细胞混合物，通过CEL-seq2平台进行测序；将384个孔的混合产物分别按1/9与1/3的比例进行二次取样，以模拟不同大小的细胞，并设置了0.7至0.9不等的PCR产物纯化比例，这些样本在分析中分别记为cellmix1至cellmix4。
在细胞混合物实验中，我们还分选了含90个细胞的孔样本，为每种细胞混合物提供伪批量参考样本（记为cellmix5）。
我们还通过CEL-seq2与SORT-seq平台，制备了稀释至单细胞当量（每孔3.75、7.5、15至30 pg）的不同RNA混合物样本，分别记为RNAmix_CEL-seq2与RNAmix_Sort-seq，即本研究的9细胞混合物数据集。