TAUR-Lab/Taur_CoT_Analysis_Project___google__gemini-1.5-pro-001

--- dataset_info: - config_name: agieval_lsat_ar features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 2098577 num_examples: 230 download_size: 421708 dataset_size: 2098577 - config_name: agieval_lsat_lr features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 5218012 num_examples: 510 download_size: 1999139 dataset_size: 5218012 - config_name: agieval_lsat_rc features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 6038993 num_examples: 269 download_size: 1371301 dataset_size: 6038993 - config_name: arc_challenge features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 1504872 num_examples: 299 download_size: 414100 dataset_size: 1504872 - config_name: arc_easy features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 2709137 num_examples: 570 download_size: 658375 dataset_size: 2709137 - config_name: bbh features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 36025464 num_examples: 6258 download_size: 8191592 dataset_size: 36025464 - config_name: biggen_bench features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 13475179 num_examples: 765 download_size: 5953955 dataset_size: 13475179 - config_name: contexthub_abductive_level1 features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 1769576 num_examples: 360 download_size: 400334 dataset_size: 1769576 - config_name: contexthub_abductive_level2 features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 15311572 num_examples: 2400 download_size: 4308989 dataset_size: 15311572 - config_name: contexthub_deductive_level1 features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 2857723 num_examples: 600 download_size: 669957 dataset_size: 2857723 - config_name: contexthub_deductive_level2 features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 15242219 num_examples: 2400 download_size: 4363387 dataset_size: 15242219 - config_name: csqa features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 5404040 num_examples: 1221 download_size: 1065737 dataset_size: 5404040 - config_name: folio features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 1447466 num_examples: 203 - name: before_fix num_bytes: 1449066 num_examples: 203 download_size: 672462 dataset_size: 2896532 - config_name: folio__round_2_fixes features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 1012515 num_examples: 203 download_size: 231229 dataset_size: 1012515 - config_name: gpqa features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 3198070 num_examples: 448 download_size: 1157312 dataset_size: 3198070 - config_name: gsm8k features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 7745748 num_examples: 1319 - name: before_fix num_bytes: 7745748 num_examples: 1319 download_size: 3425056 dataset_size: 15491496 - config_name: gsm8k__round_2_fixes features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 7505501 num_examples: 1319 download_size: 1677962 dataset_size: 7505501 - config_name: gsm8k_hard features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 7508856 num_examples: 1319 - name: before_fix num_bytes: 7508856 num_examples: 1319 download_size: 3647962 dataset_size: 15017712 - config_name: gsm8k_hard__round_2_fixes features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 7257848 num_examples: 1319 download_size: 1778108 dataset_size: 7257848 - config_name: math features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 26368107 num_examples: 5000 download_size: 6117223 dataset_size: 26368107 - config_name: mm_musr features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 7408761 num_examples: 250 download_size: 3645914 dataset_size: 7408761 - config_name: mmlu features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 82736658 num_examples: 14042 download_size: 27126564 dataset_size: 82736658 - config_name: mmlu_pro features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 84319947 num_examples: 12032 download_size: 32287830 dataset_size: 84319947 - config_name: musique_all features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 45537933 num_examples: 4834 download_size: 20273272 dataset_size: 45537933 - config_name: op_musr features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 6475152 num_examples: 256 download_size: 1413946 dataset_size: 6475152 - config_name: piqa features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 8831335 num_examples: 1838 download_size: 1867992 dataset_size: 8831335 - config_name: siqa features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 8749789 num_examples: 1954 - name: before_fix num_bytes: 8306687 num_examples: 1954 download_size: 3322760 dataset_size: 17056476 - config_name: stratqa features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 11513046 num_examples: 2821 download_size: 1439393 dataset_size: 11513046 - config_name: ta_musr features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 5712472 num_examples: 250 download_size: 2442655 dataset_size: 5712472 - config_name: winogrande features: - name: few_shot_cot_model_response dtype: string - name: few_shot_cot_parsed_model_answer dtype: string - name: few_shot_cot_is_correct dtype: bool - name: few_shot_cot_answer_was_parsed dtype: bool - name: few_shot_cot_errored dtype: bool - name: few_shot_cot_messages dtype: string - name: few_shot_cot_payload dtype: string - name: zero_shot_cot_model_response dtype: string - name: zero_shot_cot_parsed_model_answer dtype: string - name: zero_shot_cot_is_correct dtype: bool - name: zero_shot_cot_answer_was_parsed dtype: bool - name: zero_shot_cot_errored dtype: bool - name: zero_shot_cot_messages dtype: string - name: zero_shot_cot_payload dtype: string - name: few_shot_direct_model_response dtype: string - name: few_shot_direct_parsed_model_answer dtype: string - name: few_shot_direct_is_correct dtype: bool - name: few_shot_direct_answer_was_parsed dtype: bool - name: few_shot_direct_errored dtype: bool - name: few_shot_direct_messages dtype: string - name: few_shot_direct_payload dtype: string - name: zero_shot_direct_model_response dtype: string - name: zero_shot_direct_parsed_model_answer dtype: string - name: zero_shot_direct_is_correct dtype: bool - name: zero_shot_direct_answer_was_parsed dtype: bool - name: zero_shot_direct_errored dtype: bool - name: zero_shot_direct_messages dtype: string - name: zero_shot_direct_payload dtype: string - name: question dtype: string - name: answer dtype: string - name: answerKey dtype: string - name: choices dtype: string - name: additional_information dtype: string splits: - name: latest num_bytes: 5239701 num_examples: 1267 download_size: 860172 dataset_size: 5239701 configs: - config_name: agieval_lsat_ar data_files: - split: latest path: agieval_lsat_ar/latest-* - config_name: agieval_lsat_lr data_files: - split: latest path: agieval_lsat_lr/latest-* - config_name: agieval_lsat_rc data_files: - split: latest path: agieval_lsat_rc/latest-* - config_name: arc_challenge data_files: - split: latest path: arc_challenge/latest-* - config_name: arc_easy data_files: - split: latest path: arc_easy/latest-* - config_name: bbh data_files: - split: latest path: bbh/latest-* - config_name: biggen_bench data_files: - split: latest path: biggen_bench/latest-* - config_name: contexthub_abductive_level1 data_files: - split: latest path: contexthub_abductive_level1/latest-* - config_name: contexthub_abductive_level2 data_files: - split: latest path: contexthub_abductive_level2/latest-* - config_name: contexthub_deductive_level1 data_files: - split: latest path: contexthub_deductive_level1/latest-* - config_name: contexthub_deductive_level2 data_files: - split: latest path: contexthub_deductive_level2/latest-* - config_name: csqa data_files: - split: latest path: csqa/latest-* - config_name: folio data_files: - split: latest path: folio/latest-* - split: before_fix path: folio/before_fix-* - config_name: folio__round_2_fixes data_files: - split: latest path: folio__round_2_fixes/latest-* - config_name: gpqa data_files: - split: latest path: gpqa/latest-* - config_name: gsm8k data_files: - split: latest path: gsm8k/latest-* - split: before_fix path: gsm8k/before_fix-* - config_name: gsm8k__round_2_fixes data_files: - split: latest path: gsm8k__round_2_fixes/latest-* - config_name: gsm8k_hard data_files: - split: latest path: gsm8k_hard/latest-* - split: before_fix path: gsm8k_hard/before_fix-* - config_name: gsm8k_hard__round_2_fixes data_files: - split: latest path: gsm8k_hard__round_2_fixes/latest-* - config_name: math data_files: - split: latest path: math/latest-* - config_name: mm_musr data_files: - split: latest path: mm_musr/latest-* - config_name: mmlu data_files: - split: latest path: mmlu/latest-* - config_name: mmlu_pro data_files: - split: latest path: mmlu_pro/latest-* - config_name: musique_all data_files: - split: latest path: musique_all/latest-* - config_name: op_musr data_files: - split: latest path: op_musr/latest-* - config_name: piqa data_files: - split: latest path: piqa/latest-* - config_name: siqa data_files: - split: latest path: siqa/latest-* - split: before_fix path: siqa/before_fix-* - config_name: stratqa data_files: - split: latest path: stratqa/latest-* - config_name: ta_musr data_files: - split: latest path: ta_musr/latest-* - config_name: winogrande data_files: - split: latest path: winogrande/latest-* ---该数据集为多配置大语言模型通用评测集合，所有子配置均包含以下标准特征字段： 1. 少样本思维链（Chain of Thought, CoT）模型响应：字符串（string）类型 2. 少样本思维链解析后模型答案：字符串类型 3. 少样本思维链预测结果正确性：布尔（bool）类型 4. 少样本思维链答案解析状态：布尔类型 5. 少样本思维链推理错误标记：布尔类型 6. 少样本思维链对话消息：字符串类型 7. 少样本思维链请求载荷：字符串类型 8. 零样本思维链（Chain of Thought, CoT）模型响应：字符串类型 9. 零样本思维链解析后模型答案：字符串类型 10. 零样本思维链预测结果正确性：布尔类型 11. 零样本思维链答案解析状态：布尔类型 12. 零样本思维链推理错误标记：布尔类型 13. 零样本思维链对话消息：字符串类型 14. 零样本思维链请求载荷：字符串类型 15. 少样本直接问答模型响应：字符串类型 16. 少样本直接问答解析后模型答案：字符串类型 17. 少样本直接问答预测结果正确性：布尔类型 18. 少样本直接问答答案解析状态：布尔类型 19. 少样本直接问答推理错误标记：布尔类型 20. 少样本直接问答对话消息：字符串类型 21. 少样本直接问答请求载荷：字符串类型 22. 零样本直接问答模型响应：字符串类型 23. 零样本直接问答解析后模型答案：字符串类型 24. 零样本直接问答预测结果正确性：布尔类型 25. 零样本直接问答答案解析状态：布尔类型 26. 零样本直接问答推理错误标记：布尔类型 27. 零样本直接问答对话消息：字符串类型 28. 零样本直接问答请求载荷：字符串类型 29. 评测问题：字符串类型 30. 标准答案：字符串类型 31. 答案键：字符串类型 32. 候选选项：字符串类型 33. 附加信息：字符串类型 以下为各子配置的拆分与统计信息： 1. **agieval_lsat_ar** - 数据拆分：拆分名称`latest`，字节大小2098577，样本数量230 - 下载总大小：421708，数据集总大小：2098577 2. **agieval_lsat_lr** - 数据拆分：拆分名称`latest`，字节大小5218012，样本数量510 - 下载总大小：1999139，数据集总大小：5218012 3. **agieval_lsat_rc** - 数据拆分：拆分名称`latest`，字节大小6038993，样本数量269 - 下载总大小：1371301，数据集总大小：6038993 4. **arc_challenge** - 数据拆分：拆分名称`latest`，字节大小1504872，样本数量299 - 下载总大小：414100，数据集总大小：1504872 5. **arc_easy** - 数据拆分：拆分名称`latest`，字节大小2709137，样本数量570 - 下载总大小：658375，数据集总大小：2709137 6. **bbh** - 数据拆分：拆分名称`latest`，字节大小36025464，样本数量6258 - 下载总大小：8191592，数据集总大小：36025464 7. **biggen_bench** - 数据拆分：拆分名称`latest`，字节大小13475179，样本数量765 - 下载总大小：5953955，数据集总大小：13475179 8. **contexthub_abductive_level1** - 数据拆分：拆分名称`latest`，字节大小1769576，样本数量360 - 下载总大小：400334，数据集总大小：1769576 9. **contexthub_abductive_level2** - 数据拆分：拆分名称`latest`，字节大小15311572，样本数量2400 - 下载总大小：4308989，数据集总大小：15311572 10. **contexthub_deductive_level1** - 数据拆分：拆分名称`latest`，字节大小2857723，样本数量600 - 下载总大小：669957，数据集总大小：2857723 11. **contexthub_deductive_level2** - 数据拆分：拆分名称`latest`，字节大小15242219，样本数量2400 - 下载总大小：4363387，数据集总大小：15242219 12. **csqa** - 数据拆分：拆分名称`latest`，字节大小5404040，样本数量1221 - 下载总大小：1065737，数据集总大小：5404040 13. **folio** - 数据拆分： - 拆分名称`latest`，字节大小1447466，样本数量203 - 拆分名称`before_fix`，字节大小1449066，样本数量203 - 下载总大小：672462，数据集总大小：2896532 14. **folio__round_2_fixes** - 数据拆分：拆分名称`latest`，字节大小1012515，样本数量203 - 下载总大小：231229，数据集总大小：1012515 15. **gpqa** - 数据拆分：拆分名称`latest`，字节大小3198070，样本数量448 - 下载总大小：1157312，数据集总大小：3198070 16. **gsm8k** - 数据拆分： - 拆分名称`latest`，字节大小7745748，样本数量1319 - 拆分名称`before_fix`，字节大小7745748，样本数量1319 - 下载总大小：3425056，数据集总大小：15491496 17. **gsm8k__round_2_fixes** - 数据拆分：拆分名称`latest`，字节大小7505501，样本数量1319 - 下载总大小：1677962，数据集总大小：7505501 18. **gsm8k_hard** - 数据拆分： - 拆分名称`latest`，字节大小7508856，样本数量1319 - 拆分名称`before_fix`，字节大小7508856，样本数量1319 - 下载总大小：3647962，数据集总大小：15017712 19. **gsm8k_hard__round_2_fixes** - 数据拆分：拆分名称`latest`，字节大小7257848，样本数量1319 - 下载总大小：1778108，数据集总大小：7257848 20. **math** - 数据拆分：拆分名称`latest`，字节大小26368107，样本数量5000 - 下载总大小：6117223，数据集总大小：26368107 21. **mm_musr** - 数据拆分：拆分名称`latest`，字节大小7408761，样本数量250 - 下载总大小：3645914，数据集总大小：7408761 22. **mmlu** - 数据拆分：拆分名称`latest`，字节大小82736658，样本数量14042 - 下载总大小：27126564，数据集总大小：82736658 23. **mmlu_pro** - 数据拆分：拆分名称`latest`，字节大小84319947，样本数量12032 - 下载总大小：32287830，数据集总大小：84319947 24. **musique_all** - 数据拆分：拆分名称`latest`，字节大小45537933，样本数量4834 - 下载总大小：20273272，数据集总大小：45537933 25. **op_musr** - 数据拆分：拆分名称`latest`，字节大小6475152，样本数量256 - 下载总大小：1413946，数据集总大小：6475152 26. **piqa** - 数据拆分：拆分名称`latest`，字节大小8831335，样本数量1838 - 下载总大小：1867992，数据集总大小：8831335 27. **siqa** - 数据拆分： - 拆分名称`latest`，字节大小8749789，样本数量1954 - 拆分名称`before_fix`，字节大小8306687，样本数量1954 - 下载总大小：3322760，数据集总大小：17056476 28. **stratqa** - 数据拆分：拆分名称`latest`，字节大小11513046，样本数量2821 - 下载总大小：1439393，数据集总大小：11513046 29. **ta_musr** - 数据拆分：拆分名称`latest`，字节大小5712472，样本数量250 - 下载总大小：2442655，数据集总大小：5712472 30. **winogrande** - 数据拆分：拆分名称`latest`，字节大小5239701，样本数量1267 - 下载总大小：860172，数据集总大小：5239701 ### 数据集配置文件映射 所有子配置与对应数据文件的映射关系如下： 1. `agieval_lsat_ar`：拆分`latest`对应文件路径`agieval_lsat_ar/latest-*` 2. `agieval_lsat_lr`：拆分`latest`对应文件路径`agieval_lsat_lr/latest-*` 3. `agieval_lsat_rc`：拆分`latest`对应文件路径`agieval_lsat_rc/latest-*` 4. `arc_challenge`：拆分`latest`对应文件路径`arc_challenge/latest-*` 5. `arc_easy`：拆分`latest`对应文件路径`arc_easy/latest-*` 6. `bbh`：拆分`latest`对应文件路径`bbh/latest-*` 7. `biggen_bench`：拆分`latest`对应文件路径`biggen_bench/latest-*` 8. `contexthub_abductive_level1`：拆分`latest`对应文件路径`contexthub_abductive_level1/latest-*` 9. `contexthub_abductive_level2`：拆分`latest`对应文件路径`contexthub_abductive_level2/latest-*` 10. `contexthub_deductive_level1`：拆分`latest`对应文件路径`contexthub_deductive_level1/latest-*` 11. `contexthub_deductive_level2`：拆分`latest`对应文件路径`contexthub_deductive_level2/latest-*` 12. `csqa`：拆分`latest`对应文件路径`csqa/latest-*` 13. `folio`：拆分`latest`对应文件路径`folio/latest-*`，拆分`before_fix`对应文件路径`folio/before_fix-*` 14. `folio__round_2_fixes`：拆分`latest`对应文件路径`folio__round_2_fixes/latest-*` 15. `gpqa`：拆分`latest`对应文件路径`gpqa/latest-*` 16. `gsm8k`：拆分`latest`对应文件路径`gsm8k/latest-*`，拆分`before_fix`对应文件路径`gsm8k/before_fix-*` 17. `gsm8k__round_2_fixes`：拆分`latest`对应文件路径`gsm8k__round_2_fixes/latest-*` 18. `gsm8k_hard`：拆分`latest`对应文件路径`gsm8k_hard/latest-*`，拆分`before_fix`对应文件路径`gsm8k_hard/before_fix-*` 19. `gsm8k_hard__round_2_fixes`：拆分`latest`对应文件路径`gsm8k_hard__round_2_fixes/latest-*` 20. `math`：拆分`latest`对应文件路径`math/latest-*` 21. `mm_musr`：拆分`latest`对应文件路径`mm_musr/latest-*` 22. `mmlu`：拆分`latest`对应文件路径`mmlu/latest-*` 23. `mmlu_pro`：拆分`latest`对应文件路径`mmlu_pro/latest-*` 24. `musique_all`：拆分`latest`对应文件路径`musique_all/latest-*` 25. `op_musr`：拆分`latest`对应文件路径`op_musr/latest-*` 26. `piqa`：拆分`latest`对应文件路径`piqa/latest-*` 27. `siqa`：拆分`latest`对应文件路径`siqa/latest-*`，拆分`before_fix`对应文件路径`siqa/before_fix-*` 28. `stratqa`：拆分`latest`对应文件路径`stratqa/latest-*` 29. `ta_musr`：拆分`latest`对应文件路径`ta_musr/latest-*` 30. `winogrande`：拆分`latest`对应文件路径`winogrande/latest-*`