christopher/tokenizers_500k_600k

--- configs: - config_name: default data_files: - split: train path: data/train-* dataset_info: features: - name: _id dtype: string - name: id dtype: string - name: created_at dtype: timestamp[ns, tz=UTC] - name: downloads_all_time dtype: int64 - name: downloads dtype: int64 - name: likes dtype: int64 - name: trending_score dtype: float64 - name: tokenizer_str dtype: string splits: - name: train num_bytes: 335812130805 num_examples: 100000 download_size: 180848915279 dataset_size: 335812130805 ---