Rostlab/ProstT5Dataset

---
dataset_info:
features:
- name: input_id_x
sequence: int64
- name: input_id_y
sequence: int64
splits:
- name: test
num_bytes: 1087504
num_examples: 474
- name: valid
num_bytes: 1124160
num_examples: 474
- name: train
num_bytes: 65391887792
num_examples: 17070828
download_size: 810671738
dataset_size: 65394099456
license: mit
task_categories:
- text-generation
tags:
- biology
size_categories:
- 10M<n<100M
---
# Dataset Card for "ProstT5Dataset"

* **Contributors:** Michael Heinzinger and Konstantin Weissenow, Joaquin Gomez Sanchez and Adrian Henkel, Martin Steinegger and Burkhard Rost
* **Licence:** MIT

## Table of Contents
- [Overview](#overview)
- [Dataset Description](#dataset-description)
- [Data Collection and Annotation](#data-collection-and-annotation)
- [Data Splits](#data-splits)
- [Dataset Structure](#dataset-structure)
- [Data Fields](#data-fields)
- [Data Instances](#data-instances)
- [Data Considerations](#data-considerations)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Overview
The ProstT5Dataset is a curated collection of *tokenized* protein sequences and their corresponding structure sequences (3Di).
It is derived from the [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/) and includes various steps of clustering and quality filtering.
To capture 3D information of the sequence, the [3Di structure string representation](https://www.nature.com/articles/s41587-023-01773-0#Sec2) is leveraged. This format
captures the spatial relationship of each residue to its neighbors in 3D space, effectively translating the 3D information of the sequence.
The sequence tokens are generated using the [ProstT5 Tokenizer](https://huggingface.co/Rostlab/ProstT5).

## Data Fields
- **input_id_x** (3Di Tokens): Corresponding tokenized 3Di structure representation sequences derived from the proteins.
- **input_id_y** (Amino Acid Tokens): Tokenized amino acid sequences of proteins.

## Dataset Description

We compare basic protein properties (sequence length, amino acid composition, 3Di-distribution) between our
dataset (training, validation, test sets) and proteins obtained from the [Protein Data Bank (PDB)](https://www.rcsb.org/). Key findings include similar amino acid distributions across datasets,
an overrepresentation of certain 3Di-tokens (d, v, p) and helical structures in AlphaFold2 predictions compared to PDB, and a tendency for shorter protein
lengths in this dataset (average 206-238) relative to PDB proteins (average 255). The analysis also highlights the relationship between
3Di states and secondary structures, with a notable distinction in strand-related tokens between datasets.

## Data Collection and Annotation
The dataset began with the AlphaFold Protein Structure Database , undergoing a two-step clustering process and one step of quality filtering:
1. *First Clustering:* 214M UniprotKB protein sequences were clustered using MMseqs2, resulting in 52M clusters based on pairwise sequence identity.
2. *Second Clustering:* Foldseek further clustered these proteins into 18.8M clusters, expanded to 18.6M proteins by adding diverse members.
3. *Quality Filtering:* Removed proteins with low pLDDT scores, short lengths, and highly repetitive 3Di-strings. The final training split contains 17M proteins.

## Data Splits
Data splits into train, test, and, validation were created by moving whole clusters (after quality filtering - see above), to either of the sets.
For validation and test, we only kept representatives to avoid bias towards large families.
This resulted in 474 proteins for test, 474 proteins for validation and around 17M proteins for training.

## Citation
```
@article{heinzinger2023prostt5,
title={ProstT5: Bilingual language model for protein sequence and structure},
author={Heinzinger, Michael and Weissenow, Konstantin and Sanchez, Joaquin Gomez and Henkel, Adrian and Steinegger, Martin and Rost, Burkhard},
journal={bioRxiv},
pages={2023--07},
year={2023},
publisher={Cold Spring Harbor Laboratory}
}
```

## Tokens to Character Mapping
| Amino Acid Representation | 3DI | Special Tokens |
|---------------------------|-----------|--------------------|
| 3: A | 128: a | 0: \<pad\> |
| 4: L | 129: l | 1: \</s\> |
| 5: G | 130: g | 2: \<unk\> |
| 6: V | 131: v | 148: \<fold2AA\> |
| 7: S | 132: s | 149: \<AA2fold\> |
| 8: R | 133: r | |
| 9: E | 134: e | |
| 10: D | 135: d | |
| 11: T | 136: t | |
| 12: I | 137: i | |
| 13: P | 138: p | |
| 14: K | 139: k | |
| 15: F | 140: f | |
| 16: Q | 141: q | |
| 17: N | 142: n | |
| 18: Y | 143: y | |
| 19: M | 144: m | |
| 20: H | 145: h | |
| 21: W | 146: w | |
| 22: C | 147: c | |
| 23: X | | |
| 24: B | | |
| 25: O | | |
| 26: U | | |
| 27: Z | | |

数据集信息：
特征：
- 名称：input_id_x，类型：int64序列
- 名称：input_id_y，类型：int64序列
划分集：
- 名称：测试集（test），字节数：1087504，样本数：474
- 名称：验证集（valid），字节数：1124160，样本数：474
- 名称：训练集（train），字节数：65391887792，样本数：17070828
下载大小：810671738
数据集总大小：65394099456
开源协议：MIT
任务类别：
- 文本生成
标签：
- 生物学
样本规模类别：
- 1000万 < 样本数 < 1亿
---
# “ProstT5数据集”数据集卡片

* **贡献者：** Michael Heinzinger、Konstantin Weissenow、Joaquin Gomez Sanchez、Adrian Henkel、Martin Steinegger及Burkhard Rost
* **开源协议：** MIT

## 目录
- [概述](#概述)
- [数据集描述](#数据集描述)
- [数据收集与标注](#数据收集与标注)
- [数据划分](#数据划分)
- [数据集结构](#数据集结构)
- [数据字段](#数据字段)
- [数据实例](#数据实例)
- [数据考量](#数据考量)
- [数据集的社会影响](#数据集的社会影响)
- [偏差讨论](#偏差讨论)
- [其他已知局限性](#其他已知局限性)
- [授权信息](#授权信息)
- [引用信息](#引用信息)
- [贡献者](#贡献者)

## 概述
ProstT5数据集是经过精选的分词化蛋白质序列及其对应3Di结构序列的集合。该数据集源自[AlphaFold蛋白质结构数据库（AlphaFold Protein Structure Database）](https://alphafold.ebi.ac.uk/)，并经过多轮聚类与质量过滤处理。为保留序列的三维结构信息，本数据集采用了[3Di结构字符串表示法（3Di structure string representation）](https://www.nature.com/articles/s41587-023-01773-0#Sec2)，该格式能够捕捉每个残基与其三维空间邻近残基之间的空间关联，从而有效编码序列的三维结构信息。序列Token通过[ProstT5分词器（ProstT5 Tokenizer）](https://huggingface.co/Rostlab/ProstT5)生成。

## 数据字段
- **input_id_x**（3Di Tokens）：源自蛋白质的对应分词化3Di结构表示序列。
- **input_id_y**（氨基酸Token）：蛋白质的分词化氨基酸序列。

## 数据集描述

我们将本数据集（训练集、验证集与测试集）的基础蛋白质属性（序列长度、氨基酸组成、3Di分布）与取自[蛋白质数据银行（Protein Data Bank，PDB）](https://www.rcsb.org/)的蛋白质进行了对比分析。主要分析结果包括：不同数据集的氨基酸分布相似；与PDB数据库相比，本数据集的AlphaFold2预测结果中部分3Di Token（d、v、p）与螺旋结构存在过度表征现象；且本数据集的蛋白质长度普遍较短（平均长度206~238），而PDB数据库中的蛋白质平均长度为255。本次分析还阐明了3Di状态与蛋白质二级结构之间的关联，并发现不同数据集之间与折叠链相关的Token存在显著差异。

## 数据收集与标注
本数据集以AlphaFold蛋白质结构数据库为起点，经过两轮聚类与一轮质量过滤处理：
1. **第一轮聚类**：使用MMseqs2对2.14亿条UniprotKB蛋白质序列进行聚类，基于两两序列同一性得到5200万个聚类簇。
2. **第二轮聚类**：使用Foldseek将上述蛋白质进一步聚类为1880万个聚类簇，并通过添加多样化成员将总样本量扩充至1860万条蛋白质序列。
3. **质量过滤**：移除了pLDDT分数较低、长度过短以及3Di字符串高度重复的蛋白质序列。最终训练集包含1700万条蛋白质序列。

## 数据划分
本数据集的训练集、测试集与验证集通过将完整聚类簇（经过质量过滤后，详见上文）分配至不同集合的方式构建。为避免对大型蛋白质家族产生偏差，验证集与测试集仅保留了各聚类簇的代表序列。最终测试集与验证集各包含474条蛋白质序列，训练集包含约1700万条蛋白质序列。

## 引用信息

@article{heinzinger2023prostt5,
title={ProstT5：用于蛋白质序列与结构的双语语言模型},
author={Heinzinger, Michael and Weissenow, Konstantin and Sanchez, Joaquin Gomez and Henkel, Adrian and Steinegger, Martin and Rost, Burkhard},
journal={bioRxiv},
pages={2023--07},
year={2023},
publisher={Cold Spring Harbor Laboratory}
}

## Token与字符映射表
| 氨基酸表示 | 3DI | 特殊Token |
|---------------------------|-----------|--------------------|
| 3: A | 128: a | 0: <pad> |
| 4: L | 129: l | 1: </s> |
| 5: G | 130: g | 2: <unk> |
| 6: V | 131: v | 148: <fold2AA> |
| 7: S | 132: s | 149: <AA2fold> |
| 8: R | 133: r | |
| 9: E | 134: e | |
| 10: D | 135: d | |
| 11: T | 136: t | |
| 12: I | 137: i | |
| 13: P | 138: p | |
| 14: K | 139: k | |
| 15: F | 140: f | |
| 16: Q | 141: q | |
| 17: N | 142: n | |
| 18: Y | 143: y | |
| 19: M | 144: m | |
| 20: H | 145: h | |
| 21: W | 146: w | |
| 22: C | 147: c | |
| 23: X | | |
| 24: B | | |
| 25: O | | |
| 26: U | | |
| 27: Z | | |

原始信息汇总

数据集概述

数据集描述

ProstT5Dataset 是一个经过精心筛选的蛋白质序列及其对应结构序列（3Di）的标记化集合。该数据集源自 AlphaFold Protein Structure Database，并包括多个聚类和质量过滤步骤。为了捕捉序列的3D信息，采用了 3Di结构字符串表示，这种格式有效地翻译了序列的3D信息。序列标记使用 ProstT5 Tokenizer 生成。

数据字段

• input_id_x (3Di Tokens): 从蛋白质中派生的标记化3Di结构表示序列。
• input_id_y (氨基酸 Tokens): 蛋白质的标记化氨基酸序列。

数据收集和标注

数据集从 AlphaFold Protein Structure Database 开始，经过两步聚类和一步质量过滤：

1. 第一步聚类：使用 MMseqs2 对 214M UniprotKB 蛋白质序列进行聚类，基于成对序列相似性得到 52M 个聚类。
2. 第二步聚类：Foldseek 进一步将这些蛋白质聚类成 18.8M 个聚类，并通过添加多样成员扩展到 18.6M 个蛋白质。
3. 质量过滤：移除低 pLDDT 分数、短长度和高度重复的 3Di 字符串的蛋白质。最终训练集包含 17M 个蛋白质。

数据分割

数据被分割为训练集、测试集和验证集，通过将整个聚类（经过质量过滤）移动到任一集合中。为了防止对大型家族的偏见，验证集和测试集仅保留代表性蛋白质。这导致了 474 个蛋白质用于测试，474 个蛋白质用于验证，约 17M 个蛋白质用于训练。

数据集信息

• 特征：
  • input_id_x：序列类型为 int64
  • input_id_y：序列类型为 int64
• 分割：
  • test：字节数为 1087504，示例数为 474
  • valid：字节数为 1124160，示例数为 474
  • train：字节数为 65391887792，示例数为 17070828
• 下载大小：810671738 字节
• 数据集大小：65394099456 字节
• 许可证：MIT
• 任务类别：
  • 文本生成
• 标签：
  • 生物学
• 大小类别：
  • 10M<n<100M

搜集汇总

数据集介绍

构建方式

ProstT5Dataset的构建始于AlphaFold Protein Structure Database，经过两步聚类和一步质量过滤。首先，使用MMseqs2对214M UniprotKB蛋白质序列进行聚类，形成52M个基于成对序列相似性的簇。随后，Foldseek进一步将这些蛋白质聚类为18.8M个簇，并通过添加多样成员扩展至18.6M蛋白质。最后，通过去除低pLDDT评分、短长度和高度重复的3Di字符串的蛋白质，进行质量过滤，最终训练集包含17M蛋白质。

使用方法

ProstT5Dataset主要用于蛋白质序列和结构的双语生成任务。用户可以通过加载数据集中的'input_id_x'和'input_id_y'字段，分别获取3Di结构表示和氨基酸序列的标记化形式。这些数据可用于训练和验证基于Transformer的模型，如ProstT5，以生成或预测蛋白质的结构信息。此外，数据集的MIT许可证允许广泛的研究和应用，促进了生物信息学领域的创新和发展。

背景与挑战

背景概述

在生物信息学领域，蛋白质序列与其三维结构之间的关系一直是研究的核心问题。ProstT5Dataset由Michael Heinzinger等人创建，旨在通过整合AlphaFold Protein Structure Database中的数据，提供一个包含蛋白质序列及其三维结构信息的大规模数据集。该数据集通过两步聚类和质量过滤过程，从214M UniprotKB蛋白质序列中筛选出17M高质量蛋白质，用于训练、验证和测试。ProstT5Dataset的构建不仅有助于推动蛋白质结构预测和功能分析的研究，还为生物信息学领域的模型训练提供了宝贵的资源。

当前挑战

ProstT5Dataset在构建过程中面临多个挑战。首先，从AlphaFold Protein Structure Database中提取和处理大量蛋白质数据需要高效的算法和计算资源。其次，确保数据的质量和一致性，特别是在去除低pLDDT得分、短长度和高度重复的3Di字符串时，需要精细的过滤策略。此外，数据集的划分需避免大族群偏差，确保验证和测试集的代表性。最后，如何有效地将三维结构信息转化为可处理的序列表示，以便于模型训练和预测，也是一个重要的技术难题。

常用场景

经典使用场景

在生物信息学领域，Rostlab/ProstT5Dataset 数据集的经典使用场景主要集中在蛋白质序列与结构的生成和预测任务中。该数据集通过整合 AlphaFold Protein Structure Database 中的蛋白质序列及其对应的 3Di 结构表示，为研究人员提供了一个丰富的资源库。通过训练模型，研究人员可以预测蛋白质的三维结构，从而在药物设计、蛋白质工程等领域中发挥重要作用。

解决学术问题

Rostlab/ProstT5Dataset 数据集解决了生物信息学中长期存在的蛋白质结构预测难题。传统的蛋白质结构预测方法依赖于大量的实验数据，而该数据集通过提供高质量的 3Di 结构表示，使得基于机器学习的预测模型能够更准确地捕捉蛋白质的空间关系。这不仅提高了预测精度，还为蛋白质功能研究提供了新的视角，推动了相关领域的学术进展。

实际应用

在实际应用中，Rostlab/ProstT5Dataset 数据集被广泛用于药物设计和蛋白质工程。通过精确预测蛋白质的三维结构，研究人员可以更有效地设计针对特定蛋白质靶点的药物，从而加速新药的研发过程。此外，该数据集还可用于蛋白质工程，帮助设计具有特定功能的蛋白质，如酶的改造和优化，以满足工业和医疗需求。

数据集最近研究

最新研究方向

在蛋白质结构预测领域，ProstT5Dataset的最新研究方向主要集中在利用深度学习模型，特别是基于Transformer的架构，来提高蛋白质序列和三维结构（3Di）之间的映射精度。这一研究方向的兴起得益于AlphaFold和AlphaFold2在蛋白质结构预测上的突破性进展，这些模型展示了如何有效地将蛋白质序列信息转化为三维结构信息。ProstT5Dataset通过整合AlphaFold Protein Structure Database的数据，并采用先进的3Di结构字符串表示方法，为研究人员提供了一个高质量的数据集，用于训练和验证这些模型。此外，该数据集还支持研究蛋白质序列与功能之间的关系，以及如何通过改进的结构预测来加速药物发现和蛋白质工程。