ProteinNet

该数据集专为训练大型转换器在蛋白质语言任务上而设计，它提供了表达方式，以便分析转换器层间内在维度的变化。此外，该数据集还被用于分析变压器中表达的几何属性，尤其关注训练过程中内在维度轮廓的变化。其所涉及的任务是对蛋白质序列进行自监督训练。

This dataset is specifically designed for training large Transformers on protein language tasks, and it provides suitable representations to enable the analysis of variations in intrinsic dimensionality across Transformer layers. Additionally, this dataset is also utilized to analyze the geometric properties of learned representations in Transformers, with a particular focus on the changes in intrinsic dimensionality profiles during the training process. The core task involved is self-supervised training on protein sequences.