DCP-Bench-Open

Here is the GitHub repository: https://github.com/DCP-Bench/DCP-Bench-OpenDCP-Bench-Open is a collaborative benchmark of Discrete Combinatorial Problems, with only involving integer and Boolean decision variables. Many problem formulations come from the Constraint Programming (CP) community, as well as the Integer Programming community (no continuous variables), Pseudo-Boolean and Satisfiability communities.This benchmark has two primary goals:1. **To provide a centralized repository** of discrete combinatorial optimisation and satisfaction problems, including clear natural language descriptions, corresponding data instances and ground-truth constraint models.2. **To serve as an evaluation framework** for evaluating generative AI systems (e.g. LLMs) in their ability to generate executable constraint models from natural language descriptions.In the repository, the ground-truth models are (currently) using the CPMpy library. The evaluation framework can (currently) evaluate generated models in CPMpy, MiniZinc or Or-Tools CP-SAT and more can be added with limited effort.This benchmark is an open source project that welcomes additional problems, data instances and evaluation tooling from interested developers. For reproducability, always use a specific 'Release' in your research (see below). This project started as an extension of the original CP-Bench (https://huggingface.co/datasets/kostis-init/CP-Bench) published at ECAI 2025.