jalling/LFAI_RAG_qa_v1

LFAI_RAG_qa_v1数据集旨在为LeapfrogAI的RAG（检索增强生成）评估提供基础。该数据集包含36个问题/答案/上下文条目，专门用于LLM-as-a-judge评估。数据来源于多个公开文档，并通过DeepEval的Synthesizer生成，经过人工筛选和修改以提高质量。数据集可能存在GPT-4o模型的偏见，并且随着新模型的发布，其独特性可能会降低。

The LFAI_RAG_qa_v1 dataset aims to serve as the foundation for RAG-focused question and answer evaluations for LeapfrogAI. This dataset contains 36 question/answer/context entries designed specifically for LLM-as-a-judge enabled RAG evaluations. Each entry includes a question (input), an expected answer, and a document context that contains or informs the expected answer. The dataset sources include multiple PDF documents, which can be downloaded from the provided links. The dataset creation process involved using DeepEvals synthesizer and refining the data by removing poorly formatted or overly simplistic questions, question/answer pairs that did not make sense in context, and modifying questions to reduce verbosity and increase factual accuracy. The dataset was generated using the GPT-4o model, carrying the biases of the model and the human annotator who refined it. The intention behind the dataset creation was to use source documentation unlikely to be in the training data of any current models, but this may change as new models are released.