arpandeepk/generations-olmo-3-1025-7b-simnpo-gentle-checkpoint-64

Name: arpandeepk/generations-olmo-3-1025-7b-simnpo-gentle-checkpoint-64
Creator: arpandeepk
Published: 2026-05-02 05:15:36
License: 暂无描述

Hugging Face2026-05-02 更新2026-05-03 收录

下载链接：

https://hf-mirror.com/datasets/arpandeepk/generations-olmo-3-1025-7b-simnpo-gentle-checkpoint-64

下载链接

链接失效反馈

官方服务：

资源简介：

--- dataset_info: - config_name: arc_challenge features: - name: doc_id dtype: int64 - name: doc struct: - name: answerKey dtype: string - name: choices struct: - name: label list: string - name: text list: string - name: id dtype: string - name: question dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_4 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1903066 num_examples: 1172 download_size: 1728826 dataset_size: 1903066 - config_name: bbh_cot_fewshot_boolean_expressions features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 690851 num_examples: 250 download_size: 680171 dataset_size: 690851 - config_name: bbh_cot_fewshot_causal_judgement features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1236714 num_examples: 187 download_size: 1228486 dataset_size: 1236714 - config_name: bbh_cot_fewshot_date_understanding features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 530841 num_examples: 250 download_size: 518654 dataset_size: 530841 - config_name: bbh_cot_fewshot_disambiguation_qa features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1348754 num_examples: 250 download_size: 1349982 dataset_size: 1348754 - config_name: bbh_cot_fewshot_dyck_languages features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1156939 num_examples: 250 download_size: 1161345 dataset_size: 1156939 - config_name: bbh_cot_fewshot_formal_fallacies features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1802597 num_examples: 250 download_size: 1787979 dataset_size: 1802597 - config_name: bbh_cot_fewshot_geometric_shapes features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1708932 num_examples: 250 download_size: 1694049 dataset_size: 1708932 - config_name: bbh_cot_fewshot_hyperbaton features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1174319 num_examples: 250 download_size: 1172150 dataset_size: 1174319 - config_name: bbh_cot_fewshot_logical_deduction_five_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1225173 num_examples: 250 download_size: 1223045 dataset_size: 1225173 - config_name: bbh_cot_fewshot_logical_deduction_seven_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1363299 num_examples: 250 download_size: 1363564 dataset_size: 1363299 - config_name: bbh_cot_fewshot_logical_deduction_three_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1021088 num_examples: 250 download_size: 1016932 dataset_size: 1021088 - config_name: bbh_cot_fewshot_movie_recommendation features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 827199 num_examples: 250 download_size: 819134 dataset_size: 827199 - config_name: bbh_cot_fewshot_multistep_arithmetic_two features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 922270 num_examples: 250 download_size: 927848 dataset_size: 922270 - config_name: bbh_cot_fewshot_navigate features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 850941 num_examples: 250 download_size: 843729 dataset_size: 850941 - config_name: bbh_cot_fewshot_object_counting features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 566271 num_examples: 250 download_size: 555284 dataset_size: 566271 - config_name: bbh_cot_fewshot_penguins_in_a_table features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 585961 num_examples: 146 download_size: 593488 dataset_size: 585961 - config_name: bbh_cot_fewshot_reasoning_about_colored_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 927904 num_examples: 250 download_size: 922167 dataset_size: 927904 - config_name: bbh_cot_fewshot_ruin_names features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1216455 num_examples: 250 download_size: 1214920 dataset_size: 1216455 - config_name: bbh_cot_fewshot_salient_translation_error_detection features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 2381089 num_examples: 250 download_size: 2368522 dataset_size: 2381089 - config_name: bbh_cot_fewshot_snarks features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 800852 num_examples: 178 download_size: 806672 dataset_size: 800852 - config_name: bbh_cot_fewshot_sports_understanding features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 361338 num_examples: 250 download_size: 346022 dataset_size: 361338 - config_name: bbh_cot_fewshot_temporal_sequences features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1205048 num_examples: 250 download_size: 1203205 dataset_size: 1205048 - config_name: bbh_cot_fewshot_tracking_shuffled_objects_five_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1248180 num_examples: 250 download_size: 1246915 dataset_size: 1248180 - config_name: bbh_cot_fewshot_tracking_shuffled_objects_seven_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1466013 num_examples: 250 download_size: 1468311 dataset_size: 1466013 - config_name: bbh_cot_fewshot_tracking_shuffled_objects_three_objects features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1080037 num_examples: 250 download_size: 1076408 dataset_size: 1080037 - config_name: bbh_cot_fewshot_web_of_lies features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1099776 num_examples: 250 download_size: 1096810 dataset_size: 1099776 - config_name: bbh_cot_fewshot_word_sorting features: - name: doc_id dtype: int64 - name: doc struct: - name: input dtype: string - name: target dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 1085761 num_examples: 250 download_size: 1093041 dataset_size: 1085761 - config_name: cleanslate_qa features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: string - name: content_id dtype: string - name: content_title dtype: string - name: question dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 7945802 num_examples: 12088 download_size: 7055026 dataset_size: 7945802 - config_name: coqa features: - name: doc_id dtype: int64 - name: doc struct: - name: additional_answers struct: - name: '0' struct: - name: input_text list: string - name: span_end list: int64 - name: span_start list: int64 - name: span_text list: string - name: turn_id list: int64 - name: '1' struct: - name: input_text list: string - name: span_end list: int64 - name: span_start list: int64 - name: span_text list: string - name: turn_id list: int64 - name: '2' struct: - name: input_text list: string - name: span_end list: int64 - name: span_start list: int64 - name: span_text list: string - name: turn_id list: int64 - name: answers struct: - name: input_text list: string - name: span_end list: int64 - name: span_start list: int64 - name: span_text list: string - name: turn_id list: int64 - name: id dtype: string - name: questions struct: - name: input_text list: string - name: turn_id list: int64 - name: source dtype: string - name: story dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: float64 - name: score dtype: float64 splits: - name: train num_bytes: 5535046 num_examples: 500 download_size: 5538191 dataset_size: 5535046 - config_name: drop features: - name: doc_id dtype: int64 - name: doc struct: - name: answer struct: - name: date struct: - name: day dtype: string - name: month dtype: string - name: year dtype: string - name: hit_id dtype: string - name: number dtype: string - name: spans list: string - name: worker_id dtype: string - name: answers list: list: string - name: id dtype: string - name: passage dtype: string - name: query_id dtype: string - name: question dtype: string - name: section_id dtype: string - name: validated_answers struct: - name: date list: - name: day dtype: string - name: month dtype: string - name: year dtype: string - name: hit_id list: string - name: number list: string - name: spans list: list: string - name: worker_id list: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 27365664 num_examples: 9536 download_size: 25621851 dataset_size: 27365664 - config_name: gsm8k features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: string - name: question dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 11650132 num_examples: 2638 download_size: 10714895 dataset_size: 11650132 - config_name: hellaswag features: - name: doc_id dtype: int64 - name: doc struct: - name: activity_label dtype: string - name: choices list: string - name: ctx dtype: string - name: ctx_a dtype: string - name: ctx_b dtype: string - name: endings list: string - name: gold dtype: int64 - name: ind dtype: int64 - name: label dtype: string - name: query dtype: string - name: source_id dtype: string - name: split dtype: string - name: split_type dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 39600349 num_examples: 10042 download_size: 38111146 dataset_size: 39600349 - config_name: humaneval_plus features: - name: doc_id dtype: int64 - name: doc struct: - name: canonical_solution dtype: string - name: entry_point dtype: string - name: prompt dtype: string - name: task_id dtype: string - name: test dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: max_gen_toks dtype: int64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: 'null' - name: score dtype: float64 splits: - name: train num_bytes: 22268922 num_examples: 164 download_size: 14259818 dataset_size: 22268922 - config_name: lambada_openai features: - name: doc_id dtype: int64 - name: doc struct: - name: text dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 5115235 num_examples: 5153 download_size: 4753291 dataset_size: 5115235 - config_name: mmlu_abstract_algebra features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 187432 num_examples: 100 download_size: 189151 dataset_size: 187432 - config_name: mmlu_anatomy features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 282193 num_examples: 135 download_size: 280355 dataset_size: 282193 - config_name: mmlu_astronomy features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 366529 num_examples: 152 download_size: 363655 dataset_size: 366529 - config_name: mmlu_business_ethics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 256604 num_examples: 100 download_size: 259968 dataset_size: 256604 - config_name: mmlu_clinical_knowledge features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 557101 num_examples: 265 download_size: 535359 dataset_size: 557101 - config_name: mmlu_college_biology features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 374043 num_examples: 144 download_size: 370933 dataset_size: 374043 - config_name: mmlu_college_chemistry features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 214848 num_examples: 100 download_size: 219167 dataset_size: 214848 - config_name: mmlu_college_computer_science features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 307963 num_examples: 100 download_size: 316795 dataset_size: 307963 - config_name: mmlu_college_mathematics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 215861 num_examples: 100 download_size: 217214 dataset_size: 215861 - config_name: mmlu_college_medicine features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 569102 num_examples: 173 download_size: 564561 dataset_size: 569102 - config_name: mmlu_college_physics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 242967 num_examples: 102 download_size: 245989 dataset_size: 242967 - config_name: mmlu_computer_security features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 226900 num_examples: 100 download_size: 228447 dataset_size: 226900 - config_name: mmlu_conceptual_physics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 419346 num_examples: 235 download_size: 400348 dataset_size: 419346 - config_name: mmlu_econometrics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 333933 num_examples: 114 download_size: 335243 dataset_size: 333933 - config_name: mmlu_electrical_engineering features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 261717 num_examples: 145 download_size: 255763 dataset_size: 261717 - config_name: mmlu_elementary_mathematics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 705180 num_examples: 378 download_size: 661835 dataset_size: 705180 - config_name: mmlu_formal_logic features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 360537 num_examples: 126 download_size: 361682 dataset_size: 360537 - config_name: mmlu_global_facts features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 180845 num_examples: 100 download_size: 182045 dataset_size: 180845 - config_name: mmlu_high_school_biology features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 834802 num_examples: 310 download_size: 806539 dataset_size: 834802 - config_name: mmlu_high_school_chemistry features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 481792 num_examples: 203 download_size: 468068 dataset_size: 481792 - config_name: mmlu_high_school_computer_science features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 319186 num_examples: 100 download_size: 325518 dataset_size: 319186 - config_name: mmlu_high_school_european_history features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1511273 num_examples: 165 download_size: 1524911 dataset_size: 1511273 - config_name: mmlu_high_school_geography features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 394842 num_examples: 198 download_size: 381447 dataset_size: 394842 - config_name: mmlu_high_school_government_and_politics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 523926 num_examples: 193 download_size: 513498 dataset_size: 523926 - config_name: mmlu_high_school_macroeconomics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 962263 num_examples: 390 download_size: 922056 dataset_size: 962263 - config_name: mmlu_high_school_mathematics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 529022 num_examples: 270 download_size: 507520 dataset_size: 529022 - config_name: mmlu_high_school_microeconomics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 606537 num_examples: 238 download_size: 588983 dataset_size: 606537 - config_name: mmlu_high_school_physics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 437164 num_examples: 151 download_size: 439767 dataset_size: 437164 - config_name: mmlu_high_school_psychology features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1308192 num_examples: 545 download_size: 1241046 dataset_size: 1308192 - config_name: mmlu_high_school_statistics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 756158 num_examples: 216 download_size: 743487 dataset_size: 756158 - config_name: mmlu_high_school_us_history features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1675120 num_examples: 204 download_size: 1686269 dataset_size: 1675120 - config_name: mmlu_high_school_world_history features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 2119025 num_examples: 237 download_size: 2123928 dataset_size: 2119025 - config_name: mmlu_human_aging features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 427480 num_examples: 223 download_size: 410371 dataset_size: 427480 - config_name: mmlu_human_sexuality features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 278798 num_examples: 131 download_size: 275956 dataset_size: 278798 - config_name: mmlu_international_law features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 378139 num_examples: 121 download_size: 383787 dataset_size: 378139 - config_name: mmlu_jurisprudence features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 266382 num_examples: 108 download_size: 266261 dataset_size: 266382 - config_name: mmlu_logical_fallacies features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 399385 num_examples: 163 download_size: 394101 dataset_size: 399385 - config_name: mmlu_machine_learning features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 271130 num_examples: 112 download_size: 270974 dataset_size: 271130 - config_name: mmlu_management features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 190383 num_examples: 103 download_size: 191128 dataset_size: 190383 - config_name: mmlu_marketing features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 518883 num_examples: 234 download_size: 502223 dataset_size: 518883 - config_name: mmlu_medical_genetics features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 195066 num_examples: 100 download_size: 198588 dataset_size: 195066 - config_name: mmlu_miscellaneous features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1437634 num_examples: 783 download_size: 1339965 dataset_size: 1437634 - config_name: mmlu_moral_disputes features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 848504 num_examples: 346 download_size: 813606 dataset_size: 848504 - config_name: mmlu_moral_scenarios features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 2678550 num_examples: 895 download_size: 2562797 dataset_size: 2678550 - config_name: mmlu_nutrition features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 728042 num_examples: 306 download_size: 702795 dataset_size: 728042 - config_name: mmlu_philosophy features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 673578 num_examples: 311 download_size: 646185 dataset_size: 673578 - config_name: mmlu_prehistory features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 732458 num_examples: 324 download_size: 702513 dataset_size: 732458 - config_name: mmlu_professional_accounting features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 888790 num_examples: 282 download_size: 865419 dataset_size: 888790 - config_name: mmlu_professional_law features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 10852566 num_examples: 1534 download_size: 10727882 dataset_size: 10852566 - config_name: mmlu_professional_medicine features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1341719 num_examples: 272 download_size: 1337125 dataset_size: 1341719 - config_name: mmlu_professional_psychology features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1706883 num_examples: 612 download_size: 1633272 dataset_size: 1706883 - config_name: mmlu_public_relations features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 243604 num_examples: 110 download_size: 244608 dataset_size: 243604 - config_name: mmlu_security_studies features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 1246763 num_examples: 245 download_size: 1235505 dataset_size: 1246763 - config_name: mmlu_sociology features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 506693 num_examples: 201 download_size: 494024 dataset_size: 506693 - config_name: mmlu_us_foreign_policy features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 233487 num_examples: 100 download_size: 235096 dataset_size: 233487 - config_name: mmlu_virology features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 337931 num_examples: 166 download_size: 331724 dataset_size: 337931 - config_name: mmlu_world_religions features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_2 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_3 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 280300 num_examples: 171 download_size: 270524 dataset_size: 280300 - config_name: triviaqa features: - name: doc_id dtype: int64 - name: doc struct: - name: answer struct: - name: aliases list: string - name: matched_wiki_entity_name dtype: string - name: normalized_aliases list: string - name: normalized_matched_wiki_entity_name dtype: string - name: normalized_value dtype: string - name: type dtype: string - name: value dtype: string - name: entity_pages struct: - name: doc_source list: 'null' - name: filename list: 'null' - name: title list: 'null' - name: wiki_context list: 'null' - name: question dtype: string - name: question_id dtype: string - name: question_source dtype: string - name: search_results struct: - name: description list: 'null' - name: filename list: 'null' - name: rank list: 'null' - name: search_context list: 'null' - name: title list: 'null' - name: url list: 'null' - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 struct: - name: do_sample dtype: bool - name: temperature dtype: float64 - name: until list: string - name: resps list: list: string - name: filtered_resps list: string - name: filter dtype: string - name: metrics list: string - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: bypass dtype: float64 - name: score dtype: float64 splits: - name: train num_bytes: 27494447 num_examples: 17944 download_size: 20771767 dataset_size: 27494447 - config_name: winogrande features: - name: doc_id dtype: int64 - name: doc struct: - name: answer dtype: string - name: option1 dtype: string - name: option2 dtype: string - name: sentence dtype: string - name: target dtype: string - name: arguments struct: - name: gen_args_0 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: gen_args_1 struct: - name: arg_0 dtype: string - name: arg_1 dtype: string - name: resps list: list: list: string - name: filtered_resps list: list: string - name: filter dtype: string - name: metrics list: 'null' - name: doc_hash dtype: string - name: prompt_hash dtype: string - name: target_hash dtype: string - name: score dtype: float64 splits: - name: train num_bytes: 981807 num_examples: 1267 download_size: 884713 dataset_size: 981807 configs: - config_name: arc_challenge data_files: - split: train path: arc_challenge/train-* - config_name: bbh_cot_fewshot_boolean_expressions data_files: - split: train path: bbh_cot_fewshot_boolean_expressions/train-* - config_name: bbh_cot_fewshot_causal_judgement data_files: - split: train path: bbh_cot_fewshot_causal_judgement/train-* - config_name: bbh_cot_fewshot_date_understanding data_files: - split: train path: bbh_cot_fewshot_date_understanding/train-* - config_name: bbh_cot_fewshot_disambiguation_qa data_files: - split: train path: bbh_cot_fewshot_disambiguation_qa/train-* - config_name: bbh_cot_fewshot_dyck_languages data_files: - split: train path: bbh_cot_fewshot_dyck_languages/train-* - config_name: bbh_cot_fewshot_formal_fallacies data_files: - split: train path: bbh_cot_fewshot_formal_fallacies/train-* - config_name: bbh_cot_fewshot_geometric_shapes data_files: - split: train path: bbh_cot_fewshot_geometric_shapes/train-* - config_name: bbh_cot_fewshot_hyperbaton data_files: - split: train path: bbh_cot_fewshot_hyperbaton/train-* - config_name: bbh_cot_fewshot_logical_deduction_five_objects data_files: - split: train path: bbh_cot_fewshot_logical_deduction_five_objects/train-* - config_name: bbh_cot_fewshot_logical_deduction_seven_objects data_files: - split: train path: bbh_cot_fewshot_logical_deduction_seven_objects/train-* - config_name: bbh_cot_fewshot_logical_deduction_three_objects data_files: - split: train path: bbh_cot_fewshot_logical_deduction_three_objects/train-* - config_name: bbh_cot_fewshot_movie_recommendation data_files: - split: train path: bbh_cot_fewshot_movie_recommendation/train-* - config_name: bbh_cot_fewshot_multistep_arithmetic_two data_files: - split: train path: bbh_cot_fewshot_multistep_arithmetic_two/train-* - config_name: bbh_cot_fewshot_navigate data_files: - split: train path: bbh_cot_fewshot_navigate/train-* - config_name: bbh_cot_fewshot_object_counting data_files: - split: train path: bbh_cot_fewshot_object_counting/train-* - config_name: bbh_cot_fewshot_penguins_in_a_table data_files: - split: train path: bbh_cot_fewshot_penguins_in_a_table/train-* - config_name: bbh_cot_fewshot_reasoning_about_colored_objects data_files: - split: train path: bbh_cot_fewshot_reasoning_about_colored_objects/train-* - config_name: bbh_cot_fewshot_ruin_names data_files: - split: train path: bbh_cot_fewshot_ruin_names/train-* - config_name: bbh_cot_fewshot_salient_translation_error_detection data_files: - split: train path: bbh_cot_fewshot_salient_translation_error_detection/train-* - config_name: bbh_cot_fewshot_snarks data_files: - split: train path: bbh_cot_fewshot_snarks/train-* - config_name: bbh_cot_fewshot_sports_understanding data_files: - split: train path: bbh_cot_fewshot_sports_understanding/train-* - config_name: bbh_cot_fewshot_temporal_sequences data_files: - split: train path: bbh_cot_fewshot_temporal_sequences/train-* - config_name: bbh_cot_fewshot_tracking_shuffled_objects_five_objects data_files: - split: train path: bbh_cot_fewshot_tracking_shuffled_objects_five_objects/train-* - config_name: bbh_cot_fewshot_tracking_shuffled_objects_seven_objects data_files: - split: train path: bbh_cot_fewshot_tracking_shuffled_objects_seven_objects/train-* - config_name: bbh_cot_fewshot_tracking_shuffled_objects_three_objects data_files: - split: train path: bbh_cot_fewshot_tracking_shuffled_objects_three_objects/train-* - config_name: bbh_cot_fewshot_web_of_lies data_files: - split: train path: bbh_cot_fewshot_web_of_lies/train-* - config_name: bbh_cot_fewshot_word_sorting data_files: - split: train path: bbh_cot_fewshot_word_sorting/train-* - config_name: cleanslate_qa data_files: - split: train path: cleanslate_qa/train-* - config_name: coqa data_files: - split: train path: coqa/train-* - config_name: drop data_files: - split: train path: drop/train-* - config_name: gsm8k data_files: - split: train path: gsm8k/train-* - config_name: hellaswag data_files: - split: train path: hellaswag/train-* - config_name: humaneval_plus data_files: - split: train path: humaneval_plus/train-* - config_name: lambada_openai data_files: - split: train path: lambada_openai/train-* - config_name: mmlu_abstract_algebra data_files: - split: train path: mmlu_abstract_algebra/train-* - config_name: mmlu_anatomy data_files: - split: train path: mmlu_anatomy/train-* - config_name: mmlu_astronomy data_files: - split: train path: mmlu_astronomy/train-* - config_name: mmlu_business_ethics data_files: - split: train path: mmlu_business_ethics/train-* - config_name: mmlu_clinical_knowledge data_files: - split: train path: mmlu_clinical_knowledge/train-* - config_name: mmlu_college_biology data_files: - split: train path: mmlu_college_biology/train-* - config_name: mmlu_college_chemistry data_files: - split: train path: mmlu_college_chemistry/train-* - config_name: mmlu_college_computer_science data_files: - split: train path: mmlu_college_computer_science/train-* - config_name: mmlu_college_mathematics data_files: - split: train path: mmlu_college_mathematics/train-* - config_name: mmlu_college_medicine data_files: - split: train path: mmlu_college_medicine/train-* - config_name: mmlu_college_physics data_files: - split: train path: mmlu_college_physics/train-* - config_name: mmlu_computer_security data_files: - split: train path: mmlu_computer_security/train-* - config_name: mmlu_conceptual_physics data_files: - split: train path: mmlu_conceptual_physics/train-* - config_name: mmlu_econometrics data_files: - split: train path: mmlu_econometrics/train-* - config_name: mmlu_electrical_engineering data_files: - split: train path: mmlu_electrical_engineering/train-* - config_name: mmlu_elementary_mathematics data_files: - split: train path: mmlu_elementary_mathematics/train-* - config_name: mmlu_formal_logic data_files: - split: train path: mmlu_formal_logic/train-* - config_name: mmlu_global_facts data_files: - split: train path: mmlu_global_facts/train-* - config_name: mmlu_high_school_biology data_files: - split: train path: mmlu_high_school_biology/train-* - config_name: mmlu_high_school_chemistry data_files: - split: train path: mmlu_high_school_chemistry/train-* - config_name: mmlu_high_school_computer_science data_files: - split: train path: mmlu_high_school_computer_science/train-* - config_name: mmlu_high_school_european_history data_files: - split: train path: mmlu_high_school_european_history/train-* - config_name: mmlu_high_school_geography data_files: - split: train path: mmlu_high_school_geography/train-* - config_name: mmlu_high_school_government_and_politics data_files: - split: train path: mmlu_high_school_government_and_politics/train-* - config_name: mmlu_high_school_macroeconomics data_files: - split: train path: mmlu_high_school_macroeconomics/train-* - config_name: mmlu_high_school_mathematics data_files: - split: train path: mmlu_high_school_mathematics/train-* - config_name: mmlu_high_school_microeconomics data_files: - split: train path: mmlu_high_school_microeconomics/train-* - config_name: mmlu_high_school_physics data_files: - split: train path: mmlu_high_school_physics/train-* - config_name: mmlu_high_school_psychology data_files: - split: train path: mmlu_high_school_psychology/train-* - config_name: mmlu_high_school_statistics data_files: - split: train path: mmlu_high_school_statistics/train-* - config_name: mmlu_high_school_us_history data_files: - split: train path: mmlu_high_school_us_history/train-* - config_name: mmlu_high_school_world_history data_files: - split: train path: mmlu_high_school_world_history/train-* - config_name: mmlu_human_aging data_files: - split: train path: mmlu_human_aging/train-* - config_name: mmlu_human_sexuality data_files: - split: train path: mmlu_human_sexuality/train-* - config_name: mmlu_international_law data_files: - split: train path: mmlu_international_law/train-* - config_name: mmlu_jurisprudence data_files: - split: train path: mmlu_jurisprudence/train-* - config_name: mmlu_logical_fallacies data_files: - split: train path: mmlu_logical_fallacies/train-* - config_name: mmlu_machine_learning data_files: - split: train path: mmlu_machine_learning/train-* - config_name: mmlu_management data_files: - split: train path: mmlu_management/train-* - config_name: mmlu_marketing data_files: - split: train path: mmlu_marketing/train-* - config_name: mmlu_medical_genetics data_files: - split: train path: mmlu_medical_genetics/train-* - config_name: mmlu_miscellaneous data_files: - split: train path: mmlu_miscellaneous/train-* - config_name: mmlu_moral_disputes data_files: - split: train path: mmlu_moral_disputes/train-* - config_name: mmlu_moral_scenarios data_files: - split: train path: mmlu_moral_scenarios/train-* - config_name: mmlu_nutrition data_files: - split: train path: mmlu_nutrition/train-* - config_name: mmlu_philosophy data_files: - split: train path: mmlu_philosophy/train-* - config_name: mmlu_prehistory data_files: - split: train path: mmlu_prehistory/train-* - config_name: mmlu_professional_accounting data_files: - split: train path: mmlu_professional_accounting/train-* - config_name: mmlu_professional_law data_files: - split: train path: mmlu_professional_law/train-* - config_name: mmlu_professional_medicine data_files: - split: train path: mmlu_professional_medicine/train-* - config_name: mmlu_professional_psychology data_files: - split: train path: mmlu_professional_psychology/train-* - config_name: mmlu_public_relations data_files: - split: train path: mmlu_public_relations/train-* - config_name: mmlu_security_studies data_files: - split: train path: mmlu_security_studies/train-* - config_name: mmlu_sociology data_files: - split: train path: mmlu_sociology/train-* - config_name: mmlu_us_foreign_policy data_files: - split: train path: mmlu_us_foreign_policy/train-* - config_name: mmlu_virology data_files: - split: train path: mmlu_virology/train-* - config_name: mmlu_world_religions data_files: - split: train path: mmlu_world_religions/train-* - config_name: triviaqa data_files: - split: train path: triviaqa/train-* - config_name: winogrande data_files: - split: train path: winogrande/train-* ---

提供机构：

arpandeepk

搜集汇总

数据集介绍

构建方式

该数据集是基于OLMo-3-1025-7B模型在SimNPO算法框架下，经过柔和偏好优化（Gentle Checkpoint）后于第64个检查点生成的推理结果集合。构建过程涵盖了多个经典评测基准，包括ARC-Challenge、以及BBH系列中涉及布尔表达式、因果判断、日期理解、消歧问答、Dyck语言、形式谬误、几何形状、韵律辨识、逻辑推理（三/五/七对象）、电影推荐、多步算术、导航、物体计数、企鹅表格推理、有色物体推理及名称损坏等多样化的认知任务。每个配置均保留原始文档结构、目标答案、模型生成响应（resps）及过滤后的响应（filtered_resps），并附带详细的生成参数（如温度、采样策略与最大生成长度）与性能评分，构成了一套系统化、结构化的模型输出数据库。

使用方法

使用者可通过HuggingFace Datasets库按配置名称（config_name）加载特定子集，如arc_challenge或bbh_cot_fewshot_boolean_expressions。加载后的数据集以表格形式呈现，包含doc_id、原始文档（doc）、目标答案、模型生成的响应列表（resps）、过滤后响应及评分（score）等字段。研究者可直接利用resps字段与目标进行对比评估，或借助filtered_resps字段分析过滤策略的影响。同时，arguments字段中的生成参数可被提取用于复现实验设置。对于偏好优化研究，可依据score字段筛选高质量与低质量样本对，构建对比学习数据。数据集的官方下载与加载文档均在HuggingFace详情页提供，确保便捷接入与规范化使用。

背景与挑战

背景概述

该数据集由艾伦人工智能研究所（AI2）于2024年创建，旨在评估和提升大语言模型（LLMs）在复杂推理任务上的表现。其核心研究问题聚焦于如何通过SimPO（一种偏好优化算法）结合思维链（Chain-of-Thought）提示，改进模型在多步逻辑推理、常识问答及数学计算等基准测试中的准确性与稳健性。该数据集整合了ARC Challenge、BBH（BIG-Bench Hard）等权威子集，覆盖从科学推理到形式谬误检测的27个细分领域，共计逾万条精心标注的样本，为深入剖析模型推理能力的短板与偏好对齐算法的有效性提供了宝贵的资源，对推动可信赖人工智能的发展具有显著影响力。

当前挑战

该数据集所应对的核心挑战在于大语言模型在复杂、多步骤推理任务中易产生错误或不符合逻辑的“幻觉”现象，且现有的偏好对齐技术常常难以平衡模型在多领域泛化能力与特定任务性能之间的冲突。在构建过程中，挑战之一在于需为每个推理任务（如BBH中的因果判断、逻辑演绎）设计精准的思维链模板与少量示例，以确保生成的响应既符合任务逻辑，又能作为有效的偏好数据。此外，对模型生成的大量响应进行自动过滤与评分，以筛选出高质量的正负样本，同时避免引入标注偏差，亦是技术难点。数据集还需克服子任务规模不均（如ARC Challenge有1172条，而部分BBH子集仅146条）带来的统计效度与模型评估稳定性问题。

常用场景

经典使用场景

该数据集源自OLMo-3-1025-7B模型在SimNPO算法下进行偏好对齐训练过程中，于第64个检查点保存的生成结果。其核心价值在于为研究者提供了一组经过细致筛选与过滤的模型推理样本，涵盖了ARC-Challenge、BBH（如布尔表达式、因果判断、日期理解、几何形状推理等）等多种复杂推理任务。这些样本不仅记录了模型的原始输出，还包含了通过特定过滤策略筛选后的回答，使得该数据集成为评估和提升大语言模型在科学问答、逻辑推理与多步演算上的能力，特别是在探索模型自洽性与回答质量方面，被视为一个极为关键的基准资源。

解决学术问题

在学术研究领域，该数据集致力于解决大语言模型在偏好对齐与推理能力评估中的两个核心难题：一是如何高效地比较不同对齐策略（如SimNPO、DPO等）对模型生成行为的修正效果；二是如何精准量化模型在复杂推理任务上的稳健性与一致性。通过提供包含原始生成、过滤后响应及多维度评分指标的标准化格式，该数据集使研究者能够深入剖析模型在面临困难样本时的错误模式，进而推动无监督或弱监督偏好学习理论的发展，对于理解模型内部推理机制与对齐信号的耦合关系具有奠基性意义。

实际应用

在实际应用层面，该数据集可无缝嵌入大语言模型的产品化开发流程中。例如，在构建面向智能教育、法律咨询或科研辅助的对话系统时，工程师可利用该数据集中的过滤后响应作为高质量示范样本，对模型进行针对性微调或Few-shot提示优化。其结构化字段（如输入文档、生成参数、评分记录）使得自动化数据增强与质量筛选管道得以高效构建，显著降低人工标注成本。同时，该数据集对模型在常识推理、数学运算与逻辑演绎上的表现提供了量化的衡量尺度，为部署高性能、高可靠性的AI助手奠定了数据基石。

数据集最近研究