CP-Bench

CP-Bench是一个包含101个不同组合问题描述及其可运行CP模型的新颖基准数据集，专门为评估基于LLM的CP建模而设计。数据集涵盖了来自CP社区的不同组合问题类别，包括自包含的问题描述，并针对自动化评估LLM生成的模型进行了结构化。CP-Bench旨在帮助研究人员评估LLM在转换组合问题描述为可执行约束模型方面的建模能力，并解决现有评估数据集在多样性和复杂性方面的不足。

CP-Bench is a novel benchmark dataset comprising 101 distinct combinatorial problem descriptions and their runnable Constraint Programming (CP) models, specifically designed for evaluating LLM-based CP modeling. The dataset encompasses diverse combinatorial problem categories from the CP community, includes self-contained problem descriptions, and is structured to facilitate automated evaluation of models generated by large language models (LLMs). CP-Bench aims to assist researchers in assessing the capability of LLMs to translate combinatorial problem descriptions into executable constraint models, and addresses the shortcomings of existing evaluation datasets in terms of diversity and complexity.