Code, benchmarks and experiment data for the KR 2025 paper "Domain-Independent Instance Generation for Classical Planning"

Included Files benchmarks-inst-gen.zip contains the planning domains augmented with axioms that characterize the legal tasks, as well as the tasks that were generated for the learning and planning phase of the experiment. The augmented domains also have a separate DOI on Zenodo and their living version can be found on GitHub. instance-generator.zip contains the instance generator code as a python package. logs-instance-generation.zip contains the log files from the instance generation, including all generated instances (not only those selected for the subsequent planner experiments). Folders *-inst-gen-eval contain processed data for the generation phase: html files with aggregated results, as well as 'properties' files, which contain a JSON directory with combined data of all runs of the corresponding experiment. In essence, the properties file is the union over all properties files generated for each individual run. There are separate files for the childsnack domain because we had a separate extra run after some late bugfixes in the formalization of the domain. logs-learning.zip contains the logs of the learning phase (exported as by the IPC with too large logs being truncated) and the learned domain knowledge. Folder learning-eval contains processed data for the planners' learning phase, analogously to the *-eval folders in logs-instance-generation.zip. logs-planning.zip contains the logs of the planning phase (exported as by the IPC with too large logs being truncated). Folder planning-eval contains processed data for the planning phase, analogously to the *-eval folders in logs-instance-generation.zip. scripts.zip contains the scripts for conducting the experiments. Reproducing the data Instance Generation Create a folder structure where the instance generator, benchmarks-inst-gen and scripts are next to each other. Create a Python virtual environment and install the requirements from scripts/instance-generation/requirements.txt as well as the instance generator (cf. README in instance-generator.zip). v1-inst-gen.py in scripts/instance-generation/ runs the experiment. Afterwards organize-generated-instances.py data/v1-inst-gen in scripts/instance-generation/ puts the generated instances into benchmarks-inst-gen (a folder training and testing in each benchmark folder) and outputs a summary. Script count-instances.py creates from this summary a latex table on the generated instances of different hardness. Planner experiments You first need to build the planners from the learning IPC: Check out the repositories (e.g. repo5) from https://github.com/ipc2023-learning In the repositories, some Apptainer files refer to Ubuntu kinetic, which reached its end of life. To nevertheless use them, add line "sed -i 's|http://\(.*\).ubuntu.com/ubuntu|http://old-releases.ubuntu.com/ubuntu|g' /etc/apt/sources.list" at the beginning of the %POST section (before apt-get update). There can be multiple such occurrences in a file. Use script build-all-images.sh to build in each repository all images. The resulting sif files should be placed in a folder images next to the scripts folder. Learning Clone https://github.com/jendrikseipp/runsolver, build it and include the runsolver binary in $PATH. Clone https://github.com/ipc2023-learning/benchmarks.git next to https://github.com/roeger/ipc23-learning-scripts In addition you need (next to these directories): the directory images with the IPC planners (cf. Build the IPC planners) the directory with the generated instances (benchmarks-inst-gen.zip). the scripts directory (scripts.zip) Create a Python virtual environment in scripts/experiments, activate it and install the requirements. Some planners are a bit brittle wrt newer environments. In scripts/experiments/learning/v1-learning.py you find the versions of the relevant modules that were loaded on the grid. Make sure that at the creation of the environment, you have the same modules loaded as specified in this file. The learning experiment is in learning/v1-learning.py. It requires an environment variable $TMP that points to a writable directory on the system (e.g. /tmp). On our grid, the environment variable is provided by Slurm. After running this experiment, there is a folder "logs" next to the different repositories that contains the learning log files and the domain knowledge for the planners. The detailed experiment data can be found in scripts/experiments/learning/data/v1-learning/ (assuming that you started the experiment from the 'learning' directory). Processed and aggregated data is located in v1-learning-eval, next to it. Note that we had a late change of the formalization of the childsnack domain. For this reason, you see a separate experiment in the logs where we re-ran the learning phase for this domain. If you want to reproduce the experiments, v1-learning.py will be enough. Planning After the learning phase, the planning can be started as follows (in the same virtual environment): The planning experiment is in planning/v1-planning.py. It requires an environment variable $TMP that points to a writable directory on the system (e.g. /tmp). The detailed experiment data can be found in scripts/experiments/plannig/data/v1-planning/ (assuming that you started the experiment from the 'planning' directory). Processed and aggregated data is located in v1-planning-eval, next to it.