Recurrent Memory Transformer (RMT) Training Dataset
收藏arXiv2025-09-30 收录
下载链接：
https://github.com/booydar/t5-experiments/tree/scaling-report
下载链接
链接失效反馈官方服务：
资源简介：
该数据集旨在评估循环记忆转换模型在训练和测试过程中对不同序列长度的泛化能力。它采用了循序渐进的学习策略，即模型在处理更长的任务之前，先在较短的任务上进行训练，以此提高模型的准确性和稳定性。该数据集的规模可达到2,043,904个标记，任务内容涉及不同序列长度的自然语言理解和生成。
This dataset is designed to evaluate the generalization capability of recurrent memory Transformer models across varying sequence lengths during both training and testing. It employs a curriculum learning strategy, where the model is first trained on shorter-sequence tasks before progressing to longer ones, thereby enhancing the model's accuracy and stability. The dataset has a total size of up to 2,043,904 tokens, with its tasks covering natural language understanding and generation across different sequence lengths.
提供机构：
GitHub repository
