DUMB

DUMB是由格罗宁根大学创建的荷兰语模型基准数据集，包含九个不同资源级别的任务，旨在评估荷兰语模型的性能。数据集涵盖了从低资源到高资源的多种任务，包括先前未在荷兰语中可用的四个任务。DUMB不仅提供了一个全面的评估平台，还引入了相对误差减少（RER）作为比较模型性能的新指标。数据集内容丰富，涉及词级、词对级、句子对级和文档级任务，旨在全面测试模型的语言理解能力。此外，DUMB还提供了一个公共排行榜，以便于跟踪和比较不同模型的性能。

DUMB is a Dutch language model benchmark dataset developed by the University of Groningen. It encompasses nine tasks across varying resource levels, designed to evaluate the performance of Dutch language models. The dataset covers a full spectrum of tasks ranging from low-resource to high-resource scenarios, including four tasks that were previously unavailable for Dutch language. Beyond offering a comprehensive evaluation platform, DUMB also introduces Relative Error Reduction (RER) as a novel metric for comparing model performance. Featuring rich content covering word-level, word-pair, sentence-pair and document-level tasks, it aims to comprehensively test the language understanding capabilities of models. Additionally, DUMB provides a public leaderboard to enable convenient tracking and performance comparison across different models.