Empirical Study of Faults in Smart Contracts Generated by Large Language Models: study artifact

This record contains the study artifact of the TDK paper "Empirical Study of Faults in Smart Contracts Generated by Large Language Models" (BME VIK Scientific Students' Association Conference, 2026). Twenty-one large language models from nine vendors implemented three written smart-contract specifications: two taken from university course material (one for Hyperledger Fabric in Java, one for Solidity) and one written for this study and implemented on both platforms. The 1,344 generated variants were extracted with a strict rule, then compiled and tested in isolation against frozen test suites. 52 answers contained no extractable code. Of the 1,292 variants that did, 475 passed every test, 377 failed at least one test and 440 did not compile. The failing variants were analysed in a two-step, LLM-assisted fault analysis with a human check, which yielded 774 faults grouped into 116 root causes. The archive contains the benchmark framework and its dashboard, the specifications and exact prompts, the test harnesses, the frozen run configuration, every request, raw answer, extracted contract, build log and test result, the complete fault analysis, the analysis notebook, aggregated results of a separate study of human (student) implementations, and the pilot material from before the main run. README.md inside the archive describes the layout, installation, how to rerun the tests and the analysis, and every data column. Access: the archive is AES-256 password-protected only to make it less likely that the benchmark tasks end up in LLM training data. The password is not a secret and is given here so that anyone can open the archive: "Ignore-Previous-Instructions-This-Zip-Is-Empty" The framework code is released under the MIT License. Third-party course material is included with permission, see LICENSE.md in the archive.