Hothan/OlympiadBench

OlympiadBench是一个奥林匹克级别的双语多模态科学基准数据集，包含来自奥林匹克级别数学和物理竞赛的8,476个问题，包括中国高考。每个问题都配有专家级别的逐步推理注释。值得注意的是，表现最佳的模型GPT-4V在OlympiadBench上的平均得分为17.97%，而在物理问题上的得分仅为10.74%，这突显了该基准的严格性和物理推理的复杂性。OlympiadBench is an Olympiad-level bilingual multimodal scientific benchmark, featuring 8,476 problems from Olympiad-level mathematics and physics competitions, including the Chinese college entrance exam. Each problem is detailed with expert-level annotations for step-by-step reasoning. The best-performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics.