ELITR-Bench

ELITR-Bench是一个专为评估长上下文语言模型在会议助手场景中的性能而设计的数据集。该数据集基于现有的ELITR语料库，增加了271个手工制作的问题及其标准答案，特别关注处理来自自动语音识别的长而嘈杂的转录稿的挑战。数据集的应用领域是评估和改进长上下文语言模型在处理会议记录等长文档时的能力，旨在解决模型在理解和回答基于长文档的问题时的性能问题。

ELITR-Bench is a specialized dataset designed to evaluate the performance of long-context language models in meeting assistant scenarios. Built upon the existing ELITR corpus, this dataset adds 271 handcrafted questions and their corresponding standard answers, with particular focus on the challenges of processing long, noisy transcripts produced by automatic speech recognition (ASR). The dataset is intended for evaluating and improving the capabilities of long-context language models when handling long documents like meeting minutes, with the goal of addressing the performance issues that models encounter when understanding and answering questions based on lengthy documents.