CLiMP

CLiMP是由科罗拉多大学博尔德分校创建的中文语言模型评估基准，包含16个语法对比下的1,000个最小对（MPs），总计16,000个条目。该数据集半自动生成，覆盖9个主要的中文语言现象，用于评估中文语言模型对语法知识的掌握。数据集通过两轮人工验证，确保了95.8%的人类一致性。CLiMP的应用领域包括语言模型的语法评估，特别是解决中文语言模型在处理复杂语法结构和长距离依赖时的挑战。

CLiMP is a Chinese language model evaluation benchmark developed by the University of Colorado Boulder. It comprises 1,000 minimal pairs (MPs) across 16 grammatical contrasts, amounting to a total of 16,000 entries. The dataset is semi-automatically generated and covers 9 major Chinese linguistic phenomena, designed to assess the grammatical knowledge proficiency of Chinese language models. It has undergone two rounds of manual validation, achieving a human inter-annotator agreement of 95.8%. Application scenarios of CLiMP include grammatical evaluation of language models, particularly addressing the challenges faced by Chinese language models when processing complex grammatical structures and long-distance dependencies.