遇见数据集

Reproducibility Artifact for ""Text2NGAC: a multi-LLM pipeline for NGAC policy extraction with mutation-based verification"

收藏
Zenodo2026-08-13 更新2026-08-20 收录
官方服务:

资源简介:

Reproducibility artifact for the manuscript "Text2NGAC: a multi-LLM pipeline for NGAC policy extraction with mutation-based verification".The artifact contains: - The four annotated NGAC corpora used in the paper (iTrust Text2Policy, iTrust ACRE-n, CyberChair, IBM Course Registration; 725 access-control policy sentences in total). - The complete pipeline source: Phase 0 proactive semantic disambiguation, Phase A six-stage multi-LLM extraction, Phase B five-strategy automated verification, all four datasets covered end-to-end. - The mutation-testing harness used to evaluate Phase B's effectiveness against extractor errors (eight controlled fault- injection operators, 918 mutants across the four datasets). - The three reproduced baselines: Abdelgawad et al. (2023) spaCy-based extractor, Lawal et al. (2024) Code4Policy prompting baseline, and a fine-tuned BERT-RE relation-extraction baseline. - The Qwen-2.5-7B + QLoRA adapters at two training scales (50-sentence design-time benchmark and 580-sentence stratified split). - Statistical-analysis scripts: bootstrap and Wilson confidence intervals, McNemar's paired-difference tests with Bonferroni correction, Mann-Kendall trend tests. - The submitted manuscript PDF. Reproducibility is split into two tiers: - Tier 1 (Phase B verification, mutation harness, R2 trigger-rate analysis, Cypher false-positive analysis, test-suite coverage) is bit-reproducible from a fixed random seed on a single CPU at zero API cost. - Tier 2 (Phase 0 + Phase A LLM stages) is reproducible up to vendor-side stochasticity at temperature 0.1 (≤1.5% sentence- level cross-run disagreement on iTrust ACRE-n with the pinned model versions reported in the manuscript). See README.md and REPRODUCIBILITY.md inside the archive for a full reproduction guide.

提供机构:
Zenodo
创建时间:
2026-08-13
二维码
社区交流群
二维码
科研交流群
商业服务