Spec2ArtifactBench: A Benchmark for Specification-to-Artifact Synthesis in Data-Product Generation

Spec2ArtifactBench is a benchmark for evaluating specification-to-artifact synthesis in data-product generation. The benchmark contains 64 synthetic data-product specifications organized into four transformation groups: direct, derived, join/conditional, and nested/multi-source transformations. Each benchmark case includes source-schema information, source-to-target mappings, artifact constraints, input data, reference SQL and YAML artifacts, expected execution outputs, and execution metadata. The benchmark was constructed programmatically to provide controlled transformation coverage and deterministic reference resources for reproducible evaluation. Spec2ArtifactBench supports the evaluation of systems that generate coordinated implementation artifacts from structured data-product specifications. It enables assessment across specification fidelity, artifact validity, execution, behavioral correctness, and cross-artifact consistency. This record corresponds to Spec2ArtifactBench version 1.0.0. Project repository: https://github.com/Monsleyanish/specification-to-artifact-synthesis