projecte-aina/veritasQA

VeritasQA是一个用于评估语言模型真实性的上下文和时间无关的问答基准，旨在在零样本设置下评估大型语言模型的真实性。该数据集包含353个问题-答案对，涉及加泰罗尼亚语、西班牙语、加利西亚语和英语。数据集的结构包括唯一ID、问题、正确答案、最佳答案和错误答案等字段。数据集的创建过程包括对原始TruthfulQA实例的修订、翻译和新实例的创建。该数据集由巴塞罗那超级计算中心语言技术部门开发，作为Projecte AINA和Desarrollo Modelos ALIA项目的一部分。

VeritasQA is a multilingual truthfulness benchmark dataset for evaluating the truthfulness of Large Language Models. The dataset is designed to be context- and time-independent, focusing on common misconceptions and falsehoods. It includes 353 question-answer pairs in four languages: Catalan, Spanish, Galician, and English. The dataset is intended for use in zero-shot settings for tasks such as language modeling, multiple-choice QA, and open-domain QA. The creation process involved revising the TruthfulQA benchmark, translating it into multiple languages, and ensuring that the content is free from context-specific and time-sensitive information. The dataset is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.