CURRICULUM

CURRICULUM是一个广泛覆盖语言现象的自然语言理解评估基准，由罗斯-胡尔曼理工学院的Zeming Chen和Qiyue Gao创建。该数据集包含36种主要语言现象的集合，旨在诊断语言模型对不同类型语言现象的推理能力。数据集通过多种诊断测试评估模型性能，如零样本、接种、假设仅和跨分布测试。CURRICULUM的应用领域包括分析现有模型和数据集的局限性，以及推动未来在数据集设计、模型架构和学习目标方面的研究。

CURRICULUM is a natural language understanding evaluation benchmark covering a wide range of linguistic phenomena, developed by Zeming Chen and Qiyue Gao from Rose-Hulman Institute of Technology. This dataset includes a curated collection of 36 core linguistic phenomena, intended to diagnose the reasoning abilities of language models across diverse categories of linguistic phenomena. The benchmark evaluates model performance through multiple diagnostic test paradigms, including zero-shot testing, inoculation testing, hypothesis-only testing, and cross-distribution testing. Applications of CURRICULUM cover analyzing the limitations of existing models and datasets, as well as advancing future research in dataset design, model architectures, and learning objectives.