PPTC-R

PPTC-R数据集由北京大学和微软亚洲研究院共同创建，旨在评估大型语言模型在完成复杂PPT任务时的鲁棒性。数据集包含279个多轮对话会话，每个会话涉及创建新幻灯片和编辑现有PPT模板任务。通过模拟用户指令在句子、语义和多语言层面的对抗性扰动，以及软件版本变化对API可用性的影响，数据集设计了多种鲁棒性测试场景。该数据集的应用领域主要集中在提高语言模型在实际用户场景中的任务完成性能，特别是在面对多重挑战时的表现。

The PPTC-R dataset was co-created by Peking University and Microsoft Research Asia, aiming to evaluate the robustness of large language models (LLMs) when performing complex PowerPoint (PPT) generation and editing tasks. The dataset consists of 279 multi-turn dialogue sessions, each involving tasks of creating new slides and editing existing PPT templates. It designs various robustness test scenarios by simulating adversarial perturbations of user instructions at the syntactic, semantic and multilingual levels, as well as the impact of software version changes on API availability. The main application focus of this dataset is to improve the task completion performance of language models in real-world user scenarios, especially their performance when facing multiple challenges.