IPC+AL (Active Learning Enhanced Interactive Prompt Clarification (IPC)) Experiment

This document serves as the companion reproducibility package for the manuscript\emph{IPC+AL: Active Learning--Governed Interactive Prompt Clarification for Safety-Critical IEC~61131-3 PLC Code Generation}.It provides (i) a complete, implementation-level specification of the IPC+AL protocol and (ii) a transparent replication workflow for benchmarking IPC+AL against IPC and the baseline prompting suite (RaR, APE, AP, ReAct, MTPR) on external PLC datasets. To enable independent reproduction, the package consolidates the full prompt cards, fixed protocol obligations, and artifact schemas used in the study, including the baseline prompts, the IPC protocol, and the IPC+AL extension. Complete, executable prompt instances are provided for both evaluation settings: the Batch Mixing Process dataset and the Robot Pick-and-Place dataset. By keeping the protocol components fixed and substituting only task instances, external researchers can instantiate IPC and IPC+AL on arbitrary industrial use cases and alternative PLC benchmarks. The documentation further specifies end-to-end procedures for evaluation and governance, including LLM-in-the-Loop (LITL) validation with interchangeable validator committees (e.g., ChatGPT, Copilot Pro, Gemini or user-specified LLMs), and Human-in-the-Loop (HITL) expert assessment. Inter-rater reliability (IRR) is formalized at the gate level using Cohen’s $\kappa$ (pairwise agreement) and Krippendorff’s $\alpha$ (multi-rater agreement), with IRR computed \emph{across tasks} ($N$, e.g., 25 industrial use cases) rather than across prompt methods ($M$), preserving the rater--item structure required for statistically valid agreement estimates. Overall, this package is intended to function as a standalone replication manual: by fixing the protocol and acceptance gates while varying only task instances and validator choices, external reviewers and researchers can regenerate all intermediate artifacts, recompute the reported statistics, and verify that IPC+AL consistently overcomes IPC and the baseline family under controlled, auditable conditions.本文件为论文《IPC+AL：面向安全关键型国际电工委员会(IEC) 61131-3标准可编程逻辑控制器(Programmable Logic Controller, PLC)代码生成的主动学习驱动交互式提示澄清》的配套可复现研究包。本研究包涵盖两项核心内容：(i) IPC+AL协议完整的实现级规范；(ii) 用于将IPC+AL与IPC以及基准提示套件（RaR、APE、AP、ReAct、MTPR）在外部PLC数据集上开展基准测试的透明复现工作流。 为支持独立复现，本研究包整合了本研究中使用的全部提示卡片、固定协议规约与工件架构，涵盖基准提示、IPC协议及IPC+AL扩展模块。针对两种评估设置——批量混合流程数据集与机器人拾取放置数据集，均提供了完整可执行的提示实例。通过固定协议组件、仅替换任务实例，外部研究者可将IPC与IPC+AL适配至任意工业用例与替代PLC基准测试集。 本文档还详述了评估与管控的端到端流程，包括可替换验证委员会（如ChatGPT、Copilot Pro、Gemini或用户指定的大语言模型(LLM/Large Language Model)）的大语言模型在环(Large Language Model-in-the-Loop, LITL)验证，以及人类在环(Human-in-the-Loop, HITL)专家评估。评分者间信度(Inter-rater reliability, IRR)通过科恩κ系数（Cohen’s κ，两两一致性指标）与克里彭多夫α系数（Krippendorff’s α，多评分者一致性指标）在关卡层面进行形式化定义，且IRR的计算基于任务间维度（样本量N，例如25个工业用例）而非提示方法间维度（样本量M），保留了统计上有效一致性估计所需的评分者-条目结构。 总体而言，本研究包旨在作为独立复现手册：通过固定协议与验收关卡，仅调整任务实例与验证器选择，外部评审人员与研究者可复现所有中间工件、重新计算报告的统计数据，并验证在可控、可审计的条件下，IPC+AL始终优于IPC及其基准套件。