Ada-LEval

Ada-LEval是一个用于评估大型语言模型（LLMs）长上下文理解能力的长度可调节基准。该数据集包含两个挑战性子集：TSort和BestAnswer，能够更可靠地评估LLMs的长上下文能力。这些基准支持对测试案例长度的精细调整，并能轻松生成高达128k令牌的文本样本。Ada-LEval旨在解决现有长文本评估基准的局限性，特别是在超长上下文设置（100k+令牌）方面的不足，通过提供可控的测试案例和全面的文本理解需求，推动了对LLMs在处理长文本方面的能力和限制的深入理解。

Ada-LEval is a length-adjustable benchmark dedicated to evaluating the long-context understanding capabilities of large language models (LLMs). This dataset comprises two challenging subsets, TSort and BestAnswer, which facilitate more reliable assessment of LLMs' long-context performance. These benchmarks support fine-grained tuning of test case lengths, and can readily generate text samples up to 128k tokens. Ada-LEval aims to address the limitations of existing long-text evaluation benchmarks, especially the shortcomings in ultra-long context settings (100k+ tokens). By providing controllable test cases and integrating comprehensive text understanding requirements, it advances in-depth understanding of the capabilities and limitations of LLMs in processing long-form texts.