SGD(Schema-Guided Dialogue)

Schema-Guided Dialogue (SGD) 数据集包含超过 20k 个带注释的多域、人类和虚拟助手之间的面向任务的对话。这些对话涉及与跨越 20 个领域的服务和 API 的交互，例如银行、活动、媒体、日历、旅行和天气。对于大多数这些领域，数据集包含多个不同的 API，其中许多具有重叠的功能但不同的接口，这反映了常见的现实世界场景。广泛的可用注释可用于意图预测、槽填充、对话状态跟踪、策略模仿学习、语言生成和用户模拟学习，以及开发大型虚拟助手的其他任务。此外，该数据集包含评估集中看不见的域和服务，以量化零样本或少样本设置中的性能。
Schema-Guided Dialogue - eXtended (SGD-X) 是衡量对话系统对模式中语言变化的鲁棒性的基准。 SGD-X 扩展了 SGD 数据集，为每个模式提供了 5 个众包变体，其中变体在语义上相似但风格多样。在 SGD 上训练的模型在 SGD-X 上进行评估，以衡量它们在存在大量语言风格的现实世界环境中的泛化能力。

The Schema-Guided Dialogue (SGD) dataset contains over 20k annotated multi-domain task-oriented dialogues between humans and virtual assistants. These dialogues involve interactions with services and APIs spanning 20 domains, such as banking, events, media, calendars, travel, and weather. For most of these domains, the dataset includes multiple distinct APIs, many of which have overlapping functionalities but different interfaces, reflecting common real-world scenarios. A wide range of annotations are available for tasks including intent prediction, slot filling, dialogue state tracking, policy imitation learning, language generation, user simulation learning, and other tasks for developing large virtual assistants. Additionally, the dataset contains unseen domains and services in its evaluation split to quantify performance in zero-shot or few-shot settings.
Schema-Guided Dialogue - eXtended (SGD-X) is a benchmark for measuring the robustness of dialogue systems to linguistic variations in schemas. SGD-X extends the SGD dataset by providing 5 crowdsourced variants for each schema, where the variants are semantically similar but stylistically diverse. Models trained on SGD are evaluated on SGD-X to gauge their generalization ability in real-world environments with abundant linguistic styles.

搜集汇总

背景与挑战

背景概述

SGD数据集包含超过2万个多领域对话，涉及银行、旅行等20个领域，支持对话状态跟踪等任务。其扩展版本SGD-X通过语言变体评估模型的泛化能力，适用于零样本或少样本设置。