Ba2han/pt-1501-tokenized-2.2
收藏Hugging Face2026-01-15 更新2026-03-29 收录
下载链接：
https://hf-mirror.com/datasets/Ba2han/pt-1501-tokenized-2.2
下载链接
链接失效反馈官方服务：
资源简介：
---
tags:
- tokenized
- llama-3.2
---
# Tokenized Dataset: pt-1501-tokenized-2
**Base Tokenizer:** `unsloth/Llama-3.2-1B`
**Max Length:** `4000`
**Last Update:** 2026-01-15 18:51:50
## Statistics
| Metric | Count |
| :--- | :--- |
| Total Input Rows | 1,740,000 |
| Deduplicated (Dropped) | 7,137 |
| Final Rows Kept | 1,627,317 |
| **Total Tokens** | **1,190,651,989** |
## Dataset Breakdown
| Dataset Source | Tokens Contributed |
| :--- | :--- |
| HuggingFaceFW/finewiki | 1,190,651,989 |
提供机构：
Ba2han
