Billion Word Benchmark
收藏arXiv2025-09-30 收录
下载链接：
https://code.google.com/archive/p/1-billion-word-language-modeling-benchmark/
下载链接
链接失效反馈官方服务：
资源简介：
该数据集包含了大约8亿个词汇，这些词汇来源于2011年WMT新闻爬取数据。该数据集主要用于检索句子并计算TF-IDF值，以便为给定单词生成上下文词汇。其规模达到了8亿个单词，任务是对上下文词汇的生成。
This dataset contains approximately 800 million words, derived from the 2011 WMT news crawled corpus. It is predominantly employed for sentence retrieval and TF-IDF value calculation to generate contextual vocabulary for given words. With a total scale of 800 million words, its core task is contextual vocabulary generation.
