meituan-longcat/VitaBench

VitaBench是一个评估agent在多样化交互任务中的表现的挑战性基准，这些任务基于现实世界设置。它包括来自食品配送、店内消费和在线旅游服务的日常应用，为agent提供了至今为止最复杂的生活服务模拟环境，包含66个工具。通过消除领域特定策略的框架，它支持这些场景和工具的灵活组合，产生了100个跨场景任务和300个单场景任务。每个任务都源自多个真实用户请求，并要求agent在多轮对话中跨时间和空间维度进行推理，使用复杂的工具集，主动澄清模糊的指示，并跟踪不断变化的患者意图。

VitaBench is a challenging benchmark that evaluates agents on versatile interactive tasks grounded in real-world settings. It includes daily applications from food delivery, in-store consumption, and online travel services, providing agents with the most complex life-serving simulation environment to date, comprising 66 tools. Through a framework that eliminates domain-specific policies, it supports the flexible composition of these scenarios and tools, yielding 100 cross-scenario tasks and 300 single-scenario tasks. Each task is derived from multiple real user requests and requires agents to reason across temporal and spatial dimensions, utilize complex tool sets, proactively clarify ambiguous instructions, and track shifting user intent throughout multi-turn conversations.