OpenEvals/SimpleQA

SimpleQA是由OpenAI开发的一个事实性基准测试数据集，用于评估语言模型在回答简洁、事实性问题时的准确性。数据集包含4,326个问题，涵盖科学、技术、娱乐等多个主题。每个问题设计为有单一、明确的答案，便于评分和评估。数据集具有高正确性、多样性和挑战性，适合研究人员使用。数据集结构包括问题、答案和元数据字段，元数据包含主题、答案类型和支持URL。数据集分为测试集（4,321个问题）和少样本集（5个示例问题）。

SimpleQA is a factuality benchmark developed by OpenAI to evaluate the factual accuracy of language models when answering concise, fact-seeking questions. The dataset comprises 4,326 questions spanning diverse topics including science, technology, entertainment, and more. Each question is designed to have a single, indisputable answer, ensuring straightforward grading and assessment. The dataset features high correctness, diversity, and is challenging for frontier models, making it a practical tool for researchers. The dataset structure includes problem, answer, and metadata fields, with metadata containing topic, answer type, and supporting URLs. The dataset is split into a test set (4,321 questions) and a few-shot set (5 example questions).