Arc-Intelligence/arc-crm-benchmark

Arc CRM Benchmark是一个面向评估LLM代理在状态修改工作流中的性能的合成CRM环境数据集。它包含了1200个涵盖不同CRM工作流和复杂性的多轮对话。数据集采用JSONL格式，为每个对话提供了唯一的标识符、工作流类别、复杂度等级、对话轮次、初始实体状态、期望的最终状态等信息。旨在通过持续学习框架测量代理的性能、可靠性和适应性。

The Arc CRM Benchmark is a production-realistic synthetic CRM environment dataset designed for evaluating the performance of LLM agents on state-modifying workflows. It includes 1,200 multi-turn conversations covering various CRM workflows and complexity levels. The dataset is in JSONL format, providing each conversation with a unique identifier, workflow category, complexity level, conversation turns, initial entity states, expected final states, and more. It aims to measure agent performance, reliability, and adaptability through continual learning frameworks.