BEE-spoke-data/AutoMathText-top-hf

Name: BEE-spoke-data/AutoMathText-top-hf
Creator: BEE-spoke-data
Published: 2024-02-03 18:32:50
License: 暂无描述

Hugging Face2024-02-03 更新2024-04-21 收录

下载链接：

https://hf-mirror.com/datasets/BEE-spoke-data/AutoMathText-top-hf

下载链接

链接失效反馈

官方服务：

资源简介：

--- license: cc-by-sa-4.0 source_datasets: math-ai/AutoMathText dataset_info: - config_name: code-python-0.70-to-1.00 features: - name: text dtype: string - name: meta struct: - name: hexsha dtype: string - name: size dtype: int64 - name: ext dtype: string - name: lang dtype: string - name: max_stars_repo_path dtype: string - name: max_stars_repo_name dtype: string - name: max_stars_repo_head_hexsha dtype: string - name: max_stars_repo_licenses sequence: string - name: max_stars_count dtype: int64 - name: max_stars_repo_stars_event_min_datetime dtype: string - name: max_stars_repo_stars_event_max_datetime dtype: string - name: max_issues_repo_path dtype: string - name: max_issues_repo_name dtype: string - name: max_issues_repo_head_hexsha dtype: string - name: max_issues_repo_licenses sequence: string - name: max_issues_count dtype: int64 - name: max_issues_repo_issues_event_min_datetime dtype: string - name: max_issues_repo_issues_event_max_datetime dtype: string - name: max_forks_repo_path dtype: string - name: max_forks_repo_name dtype: string - name: max_forks_repo_head_hexsha dtype: string - name: max_forks_repo_licenses sequence: string - name: max_forks_count dtype: int64 - name: max_forks_repo_forks_event_min_datetime dtype: string - name: max_forks_repo_forks_event_max_datetime dtype: string - name: avg_line_length dtype: float64 - name: max_line_length dtype: int64 - name: alphanum_fraction dtype: float64 - name: include dtype: bool - name: reason dtype: string - name: num_tokens dtype: int64 - name: lm_name dtype: string - name: lm_label dtype: string - name: lm_q1_score dtype: float64 - name: lm_q2_score dtype: float64 - name: lm_q1q2_score dtype: float64 splits: - name: train num_bytes: 453790736.455654 num_examples: 111386 download_size: 214040695 dataset_size: 453790736.455654 - config_name: code-python-0.80-to-1.00 features: - name: text dtype: string - name: meta struct: - name: hexsha dtype: string - name: size dtype: int64 - name: ext dtype: string - name: lang dtype: string - name: max_stars_repo_path dtype: string - name: max_stars_repo_name dtype: string - name: max_stars_repo_head_hexsha dtype: string - name: max_stars_repo_licenses sequence: string - name: max_stars_count dtype: int64 - name: max_stars_repo_stars_event_min_datetime dtype: string - name: max_stars_repo_stars_event_max_datetime dtype: string - name: max_issues_repo_path dtype: string - name: max_issues_repo_name dtype: string - name: max_issues_repo_head_hexsha dtype: string - name: max_issues_repo_licenses sequence: string - name: max_issues_count dtype: int64 - name: max_issues_repo_issues_event_min_datetime dtype: string - name: max_issues_repo_issues_event_max_datetime dtype: string - name: max_forks_repo_path dtype: string - name: max_forks_repo_name dtype: string - name: max_forks_repo_head_hexsha dtype: string - name: max_forks_repo_licenses sequence: string - name: max_forks_count dtype: int64 - name: max_forks_repo_forks_event_min_datetime dtype: string - name: max_forks_repo_forks_event_max_datetime dtype: string - name: avg_line_length dtype: float64 - name: max_line_length dtype: int64 - name: alphanum_fraction dtype: float64 - name: include dtype: bool - name: reason dtype: string - name: num_tokens dtype: int64 - name: lm_name dtype: string - name: lm_label dtype: string - name: lm_q1_score dtype: float64 - name: lm_q2_score dtype: float64 - name: lm_q1q2_score dtype: float64 splits: - name: train num_bytes: 63612866.63603604 num_examples: 17947 download_size: 30488390 dataset_size: 63612866.63603604 - config_name: default features: - name: url dtype: string - name: title dtype: string - name: abstract dtype: string - name: text dtype: string - name: meta struct: - name: timestamp dtype: timestamp[s] - name: yymm dtype: string - name: arxiv_id dtype: string - name: language dtype: string - name: url dtype: string - name: abstract dtype: string - name: subjects dtype: string - name: title dtype: string - name: lm_name dtype: string - name: lm_label dtype: string - name: lm_q1_score dtype: float64 - name: lm_q2_score dtype: float64 - name: lm_q1q2_score dtype: float64 splits: - name: train num_bytes: 2373104058.782993 num_examples: 38677 download_size: 1069436618 dataset_size: 2373104058.782993 - config_name: embeddings-text-nomic_text_v1 features: - name: url dtype: string - name: text dtype: string - name: date dtype: timestamp[s] - name: meta struct: - name: domain dtype: string - name: url dtype: string - name: openwebmath_score dtype: float64 - name: openwebmath_perplexity dtype: float64 - name: lm_name dtype: string - name: lm_label dtype: string - name: lm_q1_score dtype: float64 - name: lm_q2_score dtype: float64 - name: lm_q1q2_score dtype: float64 - name: text-embedding sequence: float64 splits: - name: train num_bytes: 466212772 num_examples: 34486 download_size: 303506349 dataset_size: 466212772 - config_name: web-0.70-to-1.00 features: - name: url dtype: string - name: text dtype: string - name: date dtype: timestamp[s] - name: meta struct: - name: domain dtype: string - name: url dtype: string - name: openwebmath_score dtype: float64 - name: openwebmath_perplexity dtype: float64 - name: lm_name dtype: string - name: lm_label dtype: string - name: lm_q1_score dtype: float64 - name: lm_q2_score dtype: float64 - name: lm_q1q2_score dtype: float64 splits: - name: train num_bytes: 2387619496.635066 num_examples: 382396 download_size: 1282263838 dataset_size: 2387619496.635066 - config_name: web-0.80-to-1.00 features: - name: url dtype: string - name: text dtype: string - name: date dtype: timestamp[s] - name: meta struct: - name: domain dtype: string - name: url dtype: string - name: openwebmath_score dtype: float64 - name: openwebmath_perplexity dtype: float64 - name: lm_name dtype: string - name: lm_label dtype: string - name: lm_q1_score dtype: float64 - name: lm_q2_score dtype: float64 - name: lm_q1q2_score dtype: float64 splits: - name: train num_bytes: 244507793.00667277 num_examples: 34486 download_size: 130499757 dataset_size: 244507793.00667277 configs: - config_name: code-python-0.70-to-1.00 data_files: - split: train path: code-python-0.70-to-1.00/train-* - config_name: code-python-0.80-to-1.00 data_files: - split: train path: code-python-0.80-to-1.00/train-* - config_name: default data_files: - split: train path: data/train-* - config_name: embeddings-text-nomic_text_v1 data_files: - split: train path: embeddings-text-nomic_text_v1/train-* - config_name: web-0.70-to-1.00 data_files: - split: train path: web-0.70-to-1.00/train-* - config_name: web-0.80-to-1.00 data_files: - split: train path: web-0.80-to-1.00/train-* --- Top ranking subsets of https://huggingface.co/datasets/math-ai/AutoMathText with additional filters (see notebook in repo root) ## configs ```python from pprint import pprint from datasets import get_dataset_config_names def fetch_and_print_configs(dataset_name): """helper fn to print configs""" configs = get_dataset_config_names(dataset_name) print(f"Available configurations for dataset '{dataset_name}':") pprint(configs, width=80, compact=True) ds_name = "BEE-spoke-data/AutoMathText-top-hf" fetch_and_print_configs(ds_name) ``` <pre>Available configurations for dataset 'BEE-spoke-data/AutoMathText-top-hf': ['code-python-0.70-to-1.00', 'code-python-0.80-to-1.00', 'default', 'web-0.70-to-1.00', 'web-0.80-to-1.00'] </pre> - the `defailt` config is filtered `arxiv-0.80-to-1.00`

提供机构：

BEE-spoke-data

原始信息汇总

数据集概述

数据集名称

AutoMathText

数据集来源

https://hf-mirror.com/datasets/math-ai/AutoMathText

数据集特点

包含顶级排名的子集。
子集通过额外的过滤条件筛选，具体过滤条件参见仓库根目录下的笔记本。

5,000+

优质数据集

54 个

任务类型

进入经典数据集