遇见数据集

Replication Package for "When Postconditions Help or Mislead: An Empirical Study of LLM Code Review"

收藏
Zenodo2026-11-04 更新2026-11-05 收录
官方服务:

资源简介:

Replication package for “When Postconditions Help or Mislead: An Empirical Study of LLM Code Review.” It contains the review items (210 Java methods from PostcondBench, each in its original form and as three mutants, under every review condition), the raw reviews of all three reviewers (Claude Opus 5.5, Gemini 4 Argon, and GPT-6.1 Sol), including repeat runs, the LLM-judge verdicts used for comparison, the scoring and analysis scripts, scripts that regenerate every table and figure in the paper, and the execution check of postcondition-citing false alarms. The analysis scripts use Python 3.10+ with the standard library only. See README.md for step-by-step instructions. Some analyses require the separately available PostcondBench repository. The items are derived from PostcondBench (Gehao Zhang and Juan Zhai, Findings of ACL 2026; https://github.com/zhang-ge-hao/postcond-bench) and the open-source projects it draws upon. Reused source code remains subject to the respective upstream project licenses. Please cite PostcondBench when using these items.

提供机构:
Zenodo
创建时间:
2026-11-03
二维码
社区交流群
二维码
科研交流群
商业服务