edmdias/multilingual-task-oriented-dialog

这是一个面向任务的跨语言对话数据集，包含英语、西班牙语和泰语三种语言。数据集被分为训练集、测试集和评估集，每种语言都有相应的数据部分。数据采用PYTEXT parquet格式存储，包括意图标签、槽位标注、未分词的语句、语言信息以及内部多语言分词器的token跨度。为了平衡英语与其他语言的数据量，还提供了西班牙语和泰语的抽样数据。

This is a multilingual task-oriented dialog dataset including English, Spanish, and Thai. The dataset is divided into training, testing, and evaluation sets for each language. Data is stored in the PYTEXT parquet format, which includes intent labels, slot annotations, untokenized utterances, language information, and token spans from an in-house multilingual tokenizer. Upsampled Spanish and Thai data are provided to balance the dataset size with English.