FollowBench

FollowBench是一个专为大型语言模型设计的多级细粒度约束跟随基准，由香港科技大学(广州)和华为诺亚方舟实验室合作创建。该数据集包含820个精心挑选的指令，涵盖超过50个NLP任务，旨在评估模型遵循复杂指令的能力。数据集通过引入多级机制，逐步增加单一约束到初始指令，以精确估计模型在不同难度下的跟随能力。FollowBench的应用领域包括评估和提升大型语言模型在实际应用中的指令跟随能力，解决模型在遵循复杂约束时的挑战。

FollowBench is a multi-level and fine-grained constraint-following benchmark specifically developed for large language models, jointly created by The Hong Kong University of Science and Technology (Guangzhou) and Huawei Noah's Ark Lab. This dataset includes 820 carefully selected instructions covering over 50 NLP tasks, targeting the evaluation of models' capability to follow complex instructions. By introducing a multi-level mechanism that gradually adds individual constraints to the initial instructions, FollowBench enables precise estimation of models' constraint-following performance across different difficulty levels. The application scope of FollowBench covers evaluating and improving the instruction-following abilities of large language models in real-world scenarios, as well as addressing the challenges that models encounter when adhering to complex constraints.

搜集汇总

背景与挑战

背景概述

FollowBench是一个多级细粒度约束跟随基准数据集，包含820个指令覆盖50多个NLP任务，用于评估大型语言模型在不同难度下遵循复杂指令的能力。