caskcsg/NExtLong-512K-dataset
收藏Hugging Face2025-08-02 更新2025-04-12 收录
下载链接：
https://hf-mirror.com/datasets/caskcsg/NExtLong-512K-dataset
下载链接
链接失效反馈官方服务：
资源简介：
NExtLong数据集是通过负文档扩展方法合成的长上下文数据集，它将文档分解成多个元块，并通过插入来自预训练语料库的硬负干扰项来扩展上下文。这种方法迫使模型区分长距离依赖上下文和干扰内容，从而增强其对长距离依赖的建模能力。
The NExtLong dataset is a set of long-context data synthesized through Negative document Extension method. It decomposes a document into multiple meta-chunks and extends the context by interleaving hard negative distractors retrieved from pretraining corpora. This approach compels the model to discriminate long-range dependent context from distracting content, enhancing its ability to model long-range dependencies.
提供机构：
caskcsg
