xbench/DeepSearch

xbench是一个持续更新、无污染的真实世界特定领域AI评估框架，旨在通过两个互补的赛道测量AI系统的智能前沿和实际应用效用。它包括AGI跟踪赛道，用于衡量模型的核心能力，如推理、工具使用和记忆；以及与领域专家共同设计的职业对齐赛道，基于工作流程、环境和商业KPIs。本文开源了ScienceQA和DeepSearch两个AGI Tracking基准的源数据和评估代码。xbench is an evergreen, contamination-free, real-world, domain-specific AI evaluation framework designed to measure both the intelligence frontier and real-world utility of AI systems through two complementary tracks. It includes the AGI Tracking track for measuring core model capabilities like reasoning, tool-use, and memory, and the Profession Aligned track, co-designed with domain experts, based on workflows, environments, and business KPIs. The source data and evaluation code for two AGI Tracking benchmarks, ScienceQA and DeepSearch, are open-sourced in this paper.

搜集汇总

背景与挑战

背景概述

xbench/DeepSearch是一个用于评估AI模型在搜索和信息检索场景中工具使用能力的中文数据集，采用csv格式，大小小于1K，包含多个模型的评估结果和准确率数据。