SuzuJ/RAG-Evaluation-Dataset-JA

Allganize RAG Leaderboard数据集是一个用于评估日语检索增强生成（RAG）系统性能的基准数据集，涵盖五个行业领域：金融、信息通信、制造、公共和流通零售。数据集包含每个领域的PDF文档（共65个文档，总页数约200-300页）、每个领域60个问题及其对应答案，并标注了答案来源的上下文类型（段落、表格或图像）。该数据集旨在帮助企业评估RAG系统在特定行业和文档类型下的表现，支持自动性能评估方法，使用LLM进行答案相似性和正确性评分。数据集还包括不同RAG解决方案（如Alli、LangChain、OpenAI Assistant、Cohere）的性能比较结果。

The Allganize RAG Leaderboard dataset is a benchmark dataset for evaluating the performance of Japanese Retrieval-Augmented Generation (RAG) systems across five industry domains: finance, information and communication, manufacturing, public sector, and retail. It includes PDF documents for each domain (65 documents in total, with approximately 200-300 pages), 60 questions and corresponding answers per domain, and annotations for the context type (paragraph, table, or image) from which answers are derived. The dataset is designed to help businesses assess RAG systems in specific industries and document types, supporting automated performance evaluation using LLMs for answer similarity and correctness scoring. It also includes performance comparisons of various RAG solutions such as Alli, LangChain, OpenAI Assistant, and Cohere.