Ba2han/1711-mix_tokenized

Name: Ba2han/1711-mix_tokenized
Creator: Ba2han
Published: 2025-11-17 16:16:09
License: 暂无描述

Hugging Face2025-11-17 更新2025-12-20 收录

下载链接：

https://hf-mirror.com/datasets/Ba2han/1711-mix_tokenized

下载链接

链接失效反馈

官方服务：

资源简介：

# Tokenized version of Ba2han/1711-mix-pt-tr Generated automatically.

提供机构：

Ba2han

Ba2han/tokenized-mix

--- tags: - turkey - english - tokenized --- # Tokenized Mix Dataset Processed using `PleIAs/Baguettotron` tokenizer. **Total Rows:** 63,600,000 **Total Tokens:** 14,517,689,787 ## Dataset Token Co

Hugging Face2025-11-27 更新60

Ba2han/mixed-tokenized_2811

--- dataset_info: features: - name: input_ids list: int32 splits: - name: train_chunk_1 num_bytes: 3274400000 num_examples: 200000 - name: train_chunk_2 num_bytes: 3274400000

Hugging Face2025-11-29 更新60

Ba2han/filtered_tokenized_mix_031125

该数据集包含三个特征字段：长度（length），输入ID（input_ids），注意力掩码（attention_mask）。数据集被划分为一个训练集，包含超过2100万个样本，总大小约为4.88GB。数据集适用于需要处理序列数据，特别是那些使用注意力机制的模型。

Hugging Face2025-11-03 更新40

Ba2han/1711-mix-pt-tr

--- license: apache-2.0 task_categories: - text-generation size_categories: - 10M<n<100M --- # Turkish Language Dataset Mix (1711-mix-pt-tr) This dataset is a curated collection of Turkish language

Hugging Face2025-11-17 更新30

Ba2han/English_mix-tokenized

--- dataset_info: features: - name: input_ids sequence: uint32 config_name: default --- # English Mix Tokenized (uint32) Processed and tokenized dataset mix. ## Settings - **Min Chars**:

Hugging Face2026-03-23 更新10