Count matrices from serverless and conventional single-cell RNA sequencing data processing of 10x Genomics PBMC datasets

Gene-by-cell count matrices produced by the serverless (AWS Lambda) workflow and by the conventional single-VM workflow described in the accompanying manuscript. Input data are the 10x Genomics PBMC 1K v3 and PBMC 10K v3 samples. Contents pbmc1k/conventional/rep1, rep2, rep3 PBMC 1K, single VM, piscem -t 32 pbmc1k/serverless/rep1 PBMC 1K, AWS Lambda workflow pbmc10k/conventional/rep1, rep2, rep3 PBMC 10K, single VM, piscem -t 32 pbmc10k/serverless/rep1 PBMC 10K, AWS Lambda workflow MANIFEST.csv per matrix summary and checksums README.md file layout and comparison procedure Each run directory holds the alevin-fry output triplet: quants_mat.mtx (MatrixMarket counts), quants_mat_rows.txt (cell barcodes) and quants_mat_cols.txt (gene identifiers). Matrix contents PBMC 1K 1146 cells 36601 genes 3042734 non-zero entries 10807344 total UMIs PBMC 10K 11172 cells 36601 genes 30497191 non-zero entries 105622569 total UMIs These values are the same in every run of both workflows. Comparing the matrices SHA-256 of quants_mat.mtx differs between any two runs, including between two repeats of the conventional workflow. piscem maps reads with multiple threads, so the order in which cell barcode rows are written to the matrix is not fixed. The row order changes and the counts do not. Replacing the numeric row and column indices with the barcode and gene names, sorting the resulting triples and hashing that gives a value which does not depend on row order. The exact command is given in README.md. MANIFEST.csv reports that value as sorted_sha256 next to the plain raw_sha256. Within a dataset sorted_sha256 is identical for every run and for both workflows, and raw_sha256 differs for all of them. Compute environment AWS region us-east-2. The conventional workflow and the serverless orchestrator both used an m5dn.8xlarge instance (32 vCPU, 124 GB RAM, Intel Xeon Platinum 8259CL at 2.50 GHz, 25 Gbit network, local NVMe SSD). The mapping Lambda used 10240 MB of memory and 10240 MB of ephemeral storage.