CCI2-Data

CCI2-Data 仓库旨在提供高质量的中文安全数据集，以应对相关资源的稀缺问题。它在 CCI 数据集的基础上，通过扩展数据来源和采用更严格的数据清洗方法构建而成。该仓库包含超过1.7亿条样本，总大小为501GB，每条数据包括唯一的ID和对应的文本内容。数据经过关键词过滤、模型分类和去重等处理，保证了内容质量和安全性。用户可以通过标准化数据操作从 BAAI DataHub 和 Hugging Face 下载和使用该数据集，但需遵守相关的使用协议。该数据集适用于文本生成等任务。The CCI2-Data repository aims to provide high-quality secure Chinese datasets to address the scarcity of relevant resources. It is constructed based on the CCI dataset by expanding data sources and adopting more rigorous data cleaning methods. This repository contains over 170 million samples with a total size of 501 GB, and each entry includes a unique ID and its corresponding text content. The data has undergone processing such as keyword filtering, model-based classification, and deduplication to ensure content quality and security. Users can download and use this dataset via standardized data operations on BAAI DataHub and Hugging Face, but must comply with relevant usage agreements. This dataset is applicable to tasks such as text generation.

搜集汇总

背景与挑战

背景概述

CCI2-Data是一个高质量的中文安全数据集，包含超过1.7亿条样本，总大小501GB，旨在解决中文安全数据稀缺问题。它通过严格的数据清洗（如关键词过滤、模型分类和去重）确保内容质量和安全性，适用于文本生成等任务，用户可通过BAAI DataHub和Hugging Face下载使用。