CMMLU

CMMLU是一个综合性的中文评估基准，专门用于评估语言模型在中文语境下的知识和推理能力。CMMLU涵盖了从基础学科到高级专业水平的67个主题。它包括：需要计算和推理的自然科学，需要知识的人文科学和社会科学,以及需要生活常识的中国驾驶规则等。此外，CMMLU中的许多任务具有中国特定的答案，可能在其他地区或语言中并不普遍适用。因此是一个完全中国化的中文测试基准。

CMMLU is a comprehensive Chinese evaluation benchmark specifically designed to evaluate the knowledge and reasoning abilities of language models within Chinese contexts. CMMLU encompasses 67 topics spanning from foundational disciplines to advanced professional levels. It covers natural sciences that require computation and reasoning, humanities and social sciences that demand specialized knowledge, as well as content such as Chinese traffic regulations related to daily common sense. Furthermore, numerous tasks in CMMLU feature China-specific answers, which may not be universally applicable across other regions or languages. Consequently, it constitutes a fully Sinicized Chinese test benchmark.