Kyle1668/dclm-long-documents-sample-30000-char-limit

--- configs: - config_name: default data_files: - split: train path: data/train-* dataset_info: features: - name: fasttext_score dtype: float64 - name: id dtype: string - name: language dtype: string - name: language_score dtype: float64 - name: text dtype: string - name: url dtype: string - name: nemo_id dtype: string splits: - name: train num_bytes: 49021255809 num_examples: 9031157 download_size: 313065978545 dataset_size: 49021255809 ---