freshstack/corpus-oct-2024

FreshStack是一个全面的框架，用于构建专注于搜索小众和最新话题的挑战性信息检索和检索增强生成（IR/RAG）评估数据集。该数据集包含5个小众话题的查询、碎片、答案以及碎片级别的相关度判断，这些数据是从Stack Overflow上获取的查询和答案，并由GPT-4o生成碎片和相关度标签。此外，该数据集还包含了GitHub上关于这些小众话题的文档集合。

FreshStack is a holistic framework for constructing challenging IR/RAG evaluation datasets that focus on search across niche and recent topics. The dataset contains queries, nuggets, answers, and nugget-level relevance judgments for five niche topics, sourced from Stack Overflow and generated by GPT-4o for relevance between each nugget and a given document list. Additionally, the dataset includes a corpus of GitHub chunked documents related to these niche topics.