legalbench_small

LegalBench Small 是 LegalBench 数据集的子采样版本，旨在为大型推理模型提供成本高效的评估基准。该数据集包含 159 个法律任务的代表性子集，每个任务提供 20 个样本（共 3,180 个样本），以零样本提示形式呈现，并包含真实答案。每个样本包含四个字段：任务名称、输入提示、答案和评估方法。评估方法包括多种类型，如输出包含答案字符串、列表项完全匹配等。数据集构建过程中，从原始任务中随机抽取样本，并转换为零样本格式，排除了少数不适合的任务。该数据集适用于法律推理能力的评估，覆盖了广泛的法律领域和任务类型。

LegalBench Small is a subsampled variant of the LegalBench dataset, designed as a cost-effective evaluation benchmark for large reasoning models. This dataset comprises a representative subset of 159 legal tasks, with 20 samples per task (totaling 3,180 samples) formatted in zero-shot prompt style and accompanied by ground-truth answers. Each sample contains four fields: task name, input prompt, answer, and evaluation method. Evaluation methods cover various types, such as checking whether the output contains the target answer string, exact match for list items, and so on. During dataset construction, samples were randomly selected from the original tasks, converted to zero-shot format, and a small number of unsuitable tasks were excluded. This dataset is suitable for evaluating legal reasoning capabilities, covering a wide range of legal domains and task types.