sinatras/fineweb10B-llama3
收藏官方服务：
资源简介：
--- license: mit task_categories: - text-generation size_categories: - 10M<n<100M --- # FineWeb Dataset tokenized with LLaMA-3.2-1B tokenizer This dataset contains the FineWeb dataset preprocessed and tokenized using the LLaMA-3.2-1B tokenizer. Each .bin file contains: - A header of 256 int32 values - Token data as uint32 values - File format version: 2 Dataset structure: - fineweb_train_*.bin: Training shards - fineweb_val_*.bin: Validation shard
提供机构：
sinatras
