Siye01/IrrQA

我们提供了针对不同大型语言模型（如ChatGPT、GPT-4、Gemini和Llama-2-7B）的IrrQA数据集，包括PopQA和EntityQuestions的数据。数据集的特征包括问题ID、主题实体名称、关系类型、对象实体名称、实体ID、实体别名、Wikidata URI、问题文本、可能的答案列表、LLM提供的答案、LLM的参数化记忆支持证据、无关信息、部分相关信息以及相关信息等。数据集的分割包括不同模型和数据集类型的组合，如Irrelevant_PQA_chatgpt、Irrelevant_PQA_gpt4等。

The IrrQA dataset is designed to evaluate the robustness of various Large Language Models (LLMs) such as ChatGPT, GPT-4, Gemini, and Llama-2-7B to irrelevant information. The dataset includes data for both PopQA and EntityQuestions, focusing on irrelevant information that could potentially skew the responses of these models. The dataset features include identifiers, entity names, relationships, URIs, questions, possible answers, and various types of related and unrelated information. The dataset is divided into multiple splits based on the type of question (PopQA or EntityQuestions) and the model used. Each split contains a number of examples and bytes of data. The dataset is intended to help researchers understand how easily LLMs can be influenced by irrelevant inputs.