Cion-lab/test-1
收藏Hugging Face2026-04-17 更新2026-04-26 收录
下载链接：
https://hf-mirror.com/datasets/Cion-lab/test-1
下载链接
链接失效反馈官方服务：
资源简介：
---
dataset_info:
features:
- name: input_ids
sequence: int32
splits:
- name: train
- name: validation
configs:
- config_name: default
data_files:
- split: train
path: "data/train/*.parquet"
- split: validation
path: "data/validation/*.parquet"
---
# test-1
## Dataset Description
This is a pre-tokenized, packed dataset for Causal Language Model pre-training.
- **Total Target Tokens:** 0.1B
- **Tokenizer:** `TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T`
- **Block Size:** 2048
- **Format:** Packed `input_ids` with no padding.
## Usage
```python
from datasets import load_dataset
dataset = load_dataset('Cion-lab/test-1', streaming=True)
```
提供机构：
Cion-lab
