SEAL Release V1: Semantically Altered Coding Tasks and Model Evaluations for LLM Code Generation

This dataset is the release artifact for SEAL (Semantic Alteration), a benchmark construction framework for evaluating whether large language models can adapt to coding tasks whose semantics have been deliberately changed. The release contains 471 seed tasks from HumanEval+, MBPP+, and LiveBench, and 11,545 semantically altered coding tasks aligned with the main model evaluation. SEAL creates these tasks by modifying reference-solution semantics through seven operators, including numeric constant changes, operator flips, API substitutions, task composition, chaining, conditional execution, and loop-based composition. Each task includes a prompt, reference solution, entry point, and assertion-based tests. The artifact also includes deterministic model outputs and scoring results for six contemporary LLMs: GPT-5.4, Claude Opus 4.6, Claude Sonnet 4.6, Gemini 3.1 Pro, GLM-5.1, and MiniMax-M2.7. In total, the release contains 77,748 raw model outputs, regenerated-test scoring results, original-test scoring results for selected operators, and aggregate statistics used to analyze pass rates, performance drop ratios, memory inertia, and code-delta effects. This artifact is released under Apache License 2.0. It includes derived benchmark content from HumanEval/EvalPlus, MBPP, and LiveBench; users should also respect the licenses and attribution requirements of the original sources.