LiteFold/ProteinGym

ProteinGym是一个用于评估蛋白质适应性和突变效应预测器的标准基准套件，由OATML/Marks实验室开发，并于NeurIPS 2023数据集和基准赛道发布。它是比较蛋白质语言模型（如ESM、Tranception、ProGen等）、逆折叠模型和监督适应度预测器在共同标准化任务面上的首选参考，覆盖零样本和监督设置。基准分为四个轨道：DMS substitutions轨道收集了超过200个深度突变扫描实验，涵盖约270万个错义变异，涉及多种蛋白质（酶、病毒蛋白、人类疾病基因、抗体、从头设计）；DMS indels轨道增加了约70个实验，包含约30万个插入或缺失变异；两个临床轨道（substitutions和indels）提供来自ClinVar（致病性与良性）和gnomAD（常见良性对照）的专家注释变异，以便评估适应度模型的临床实用性，而不仅仅是实验相关性。每个实验提供野生型序列、每个变异的适应度分数、预计算的多序列比对（ColabFold和EVE风格对齐）、可用的结构元数据，以及多个难度级别的标准化交叉验证分割（随机、模数、连续）。性能报告使用Spearman、AUC、MCC、NDCG@10%和Top-K召回率来捕捉排名质量和设计相关检索，proteingym.org上的公共排行榜跟踪各轨道约80多个模型。

ProteinGym is the standard benchmark suite for evaluating protein fitness and mutation effect predictors, developed by the OATML / Marks lab and published at NeurIPS 2023 Datasets and Benchmarks. It is the go-to reference for comparing protein language models (ESM, Tranception, ProGen, etc.), inverse folding models, and supervised fitness predictors on a common, standardized task surface, and it covers both zero-shot and supervised settings. The benchmark is split into four tracks. The DMS substitutions track collects more than 200 deep mutational scanning assays covering roughly 2.7 million missense variants across diverse proteins (enzymes, viral proteins, human disease genes, antibodies, de novo designs). The DMS indels track adds around 70 assays with ~300k insertion or deletion variants. Two clinical tracks (substitutions and indels) provide expert-annotated variants from ClinVar (pathogenic vs. benign) and gnomAD (frequent benign controls), so that fitness models can be evaluated for clinical utility, not just experimental correlation. For each assay, ProteinGym ships the wild-type sequence, a per-variant fitness score, precomputed MSAs (ColabFold and EVE-style alignments), structural metadata where available, and standardized cross-validation splits at multiple difficulty levels (random, modulo, contiguous). Performance is reported with Spearman, AUC, MCC, NDCG@10%, and Top-K recall to capture both ranking quality and design-relevant retrieval, and a public leaderboard at proteingym.org tracks roughly 80+ models across tracks.