KorAdvMRSTestData

KorAdvMRSTestData是由韩国的Kakao Enterprise和SK Telecom创建的一个对抗性数据集，旨在评估开放域韩语多轮响应选择模型的弱点。数据集包含2220个测试案例，每个案例都根据不同的错误类型进行分类，如重复、否定、时态、主客体、词汇矛盾、疑问词和话题。创建过程中，五位注释者生成了200个对话会话，每个会话包含两个正确响应和一个或多个错误响应。数据集的应用领域主要集中在提高响应选择模型的鲁棒性，特别是在对抗性和真实环境中的表现。

KorAdvMRSTestData is an adversarial dataset created by Kakao Enterprise and SK Telecom of South Korea, which aims to evaluate the vulnerabilities of open-domain Korean multi-turn response selection models. It contains 2220 test cases, each categorized by different error types including repetition, negation, tense, subject-object, lexical contradiction, interrogative words, and topic. During the dataset's development, five annotators generated 200 dialogue sessions, each containing two correct responses and one or more erroneous responses. The primary application of this dataset is to improve the robustness of response selection models, particularly their performance in adversarial and real-world environments.