GigaVerbo
收藏arXiv2025-09-30 收录
下载链接：
https://nkluge-correa.github.io/Tucano/
下载链接
链接失效反馈官方服务：
资源简介：
该数据集名为GigaVerbo，它是由去重的葡萄牙语文本语料库拼接而成，总词量达到2000亿个词元，旨在支持葡萄牙语神经文本生成的发展。该数据集不仅包含了一个高质量的文本数据集，该数据集是利用GPT-4o对文本样本进行评分创建的，而且还经过了筛选，以确保内容的高质量。在规模上，该数据集包含了780GB的文本，超过1.45亿个文档。其任务是对葡萄牙语言的文本生成和质量评估。
This dataset, named GigaVerbo, is compiled from deduplicated Portuguese text corpora, boasting a total of 200 billion tokens, and is designed to support the development of Portuguese neural text generation. It not only comprises a high-quality text dataset created by scoring text samples with GPT-4o, but also has been filtered to guarantee high content quality. In terms of scale, it contains 780 GB of text and over 145 million documents. The targeted tasks of this dataset are Portuguese language text generation and quality assessment.
提供机构：
Openly available datasets and Hugging Face
