CGIAR/RAG-Chunk-Analysis

该数据集包含了来自农业文档的实际用户查询中检索到的片段的人工评估。每个片段都被标记为相关或不相关，并且相关和不相关的部分在单独的列中提及。数据集由多个XLS文件组成，每个XLS文件有多个工作表，对应于价值链的内容。查询来自于farmer.chat原型机器人的实际用户问题。对于每个用户查询和响应，使用了哪些片段以及元数据都被提供。四位非农业领域的人工评估者被要求评审问题、相应的响应，然后评估相应的片段是否相关或不相关，并标记相关部分。为了最大化覆盖范围，对话日志来自于不同的地理区域：肯尼亚以及印度的中央邦、拉贾斯坦邦和北方邦。数据集还涵盖了不同的价值链，时间范围为2023年11月至2024年1月。

The dataset contains human evaluation of retrieved chunks from agriculture documents for actual user queries. Each chunk is marked as relevant and irrelevant, with the relevant and irrelevant portions of the chunks mentioned in separate columns. The dataset consists of multiple XLS files, each with multiple sheets corresponding to the content for the value chain. The queries are taken from actual user questions on farmer.chat prototype bots. For each user query and response, the chunks used and the metadata are provided. Four human evaluators, not from the agriculture domain, were asked to review questions, corresponding responses, and evaluate the chunks for relevance or irrelevance and mark the relevant portion. The conversation logs were taken from different geographies: Kenya and Madhya Pradesh, Rajasthan, Uttar Pradesh (India) to maximize coverage. The dataset also covers different value chains and spans from November 2023 to January 2024.