nurkhan5l/tokenized_cc100

该数据集包含了文本处理相关的特征，如预测语言、是否包含哈萨克符号等，以及文本输入的序列表示和掩码信息。它被设计用于训练机器学习模型，尤其是处理文本分类或自然语言处理任务。训练集包含了超过1.96亿个示例，数据集的总大小约为4.9GB。

The dataset includes features related to text processing, such as predicted language, whether it contains Kazakh symbols, etc., as well as the sequential representation and mask information of text input. It is designed for training machine learning models, especially for text classification or natural language processing tasks. The training set contains over 196 million examples, and the total size of the dataset is approximately 4.9GB.