遇见数据集

ALS-WMS Research Experiment (n=20): Empirical Validation of Atomic Logic Sheet Effectiveness in LLM-based Code Generation

收藏
Zenodo2026-05-25 更新2026-05-26 收录
官方服务:

资源简介:

This dataset contains the complete experimental records of an n=20 study validating the effectiveness of the Atomic Logic Sheet (ALS) in LLM-based code generation for Warehouse Management System (WMS) development. Three input conditions (Group A: Requirements only; Group B: + Natural-language design; Group C: + Atomic Logic Sheet) are compared across 720 inference calls (3 groups × 20 runs × 12 tasks) using Claude Sonnet 4.5 (claude-sonnet-4-5-20250929). Three metrics are reported: Logic Compliance Rate (LCR), Conflict Detection Rate (CDR), and Schema Deviation Count (SDC). The ALS condition achieves perfect conflict detection (100.0 % CDR with zero standard deviation across all 20 runs), reduces schema deviations by 89.5 % vs. the requirements-only baseline, and shows a ceiling effect on LCR (93.4–96.9 %), demonstrating the unique value of formal rule notation over natural-language specifications in LLM-based code generation. The dataset includes all input prompts, raw LLM outputs, generated code snapshots, evaluation scripts, evaluation CSV results, statistical analyses, and cost analyses.

提供机构:
Zenodo
创建时间:
2026-05-24
二维码
社区交流群
二维码
科研交流群
商业服务