freshstack/queries-oct-2024-unfiltered

FreshStack是一个全面框架，用于构建专注于搜索小众和最新主题的具有挑战性的信息检索/阅读理解评估数据集。这个数据集（2024年10月）包含了5个小众主题的查询、碎片、答案和碎片级别的相关性判断，这些主题聚焦于软件工程和机器学习。查询和答案（已接受）来自Stack Overflow，GPT-4o生成碎片并标注每个碎片与给定文档列表之间的相关性。

FreshStack is a holistic framework for constructing challenging IR/RAG evaluation datasets that focus on search across niche and recent topics. This dataset (October 2024) contains the query, nuggets, answers, and nugget-level relevance judgments of five niche topics focused on software engineering and machine learning. The queries and answers (accepted) are taken from Stack Overflow, GPT-4o generates the nuggets and labels the relevance between each nugget and a given document list.