tartuNLP/smugri-mt-bench

SMUGRI-MT-Bench是一个芬兰-乌戈尔语版的多轮对话和指令遵循能力评估数据集，包含4种极低资源语言：爱沙尼亚语、利沃尼亚语、科米语和沃罗语。该数据集由80个单轮和多云问题组成，涵盖数学、推理、写作和一般知识四个主题。这些问题由这些语言的流利使用者从LMSYS-Chat-1M数据集中挑选并手动翻译。此外，数据集还包括每个语言20个有害提示，用于研究极低资源场景下语言模型的潜在漏洞和偏见。

SMUGRI-MT-Bench is a Finno-Ugric version of MT-Bench for evaluating the multi-turn conversation and instruction-following capabilities of LLMs, covering 4 extremely low-resource languages: Estonian, Livonian, Komi, and Võro. The dataset consists of 80 single and multi-turn questions across four topics: math, reasoning, writing, and general knowledge. These questions are selected and manually translated from the LMSYS-Chat-1M dataset into Estonian, Võro, Komi, and Livonian by fluent speakers. Additionally, the dataset includes 20 harmful prompts for each language to facilitate research on potential vulnerabilities and biases in language models within extremely low-resource scenarios.