SakanaAI/ChouBun
收藏Hugging Face2024-10-18 更新2025-04-08 收录
下载链接：
https://hf-mirror.com/datasets/SakanaAI/ChouBun
下载链接
链接失效反馈官方服务：
资源简介：
ChouBun是一个评估LLM在长上下文任务中性能的基准，包含从多个网站和由GPT-4变种和Claude-3.5-Sonnet生成的合成问答对。当前版本包含两种任务类型——提取式问答和抽象式摘要——以及以下四个任务：wiki_qa是基于日本维基百科的提取式问答任务，edinet_qa是基于EDINET安全报告的提取式问答任务，corp_sec_qa是基于三家公司网站安全报告的提取式问答任务，corp_sec_sum是基于corp_sec_qa数据的抽象式摘要任务。
ChouBun is a benchmark for assessing LLMs performance in long-context tasks, including documents from multiple websites and synthetic question-answer pairs generated by GPT-4 variants and Claude-3.5-Sonnet. The current version contains two task categories — extractive QA and abstractive summarization — and four tasks: wiki_qa is an extractive QA task based on Japanese Wikipedia, edinet_qa is based on EDINET security reports, corp_sec_qa is based on security reports from three corporation websites, and corp_sec_sum is an abstractive summarization task based on the data of corp_sec_qa.
提供机构：
SakanaAI
