Results and repro tools for Towards Cross-Build Differential Testing paper

Recent concerns about software supply chain security have led to the emergence of different binaries built from the same source code. This will sometimes result in binaries that are not identical and therefore have different cryptographic hashes. The question arises whether those binaries are still equivalent, i.e., whether they have the same behaviour. We explore whether differential testing can be used to provide evidence for non-equivalence. We study this for 3,541 pairs of binaries built for the same Maven artifact version, distributed on Maven Central, Google Assured Open Source Software and/or Oracle Build-From-Source. We use EvoSuite to generate tests for the baseline binary from Maven Central, run these tests against this baseline binary and any available alternately built binaries, and compare the results for consistency. We argue that any differences may indicate variations in program behaviour and could, therefore, be used to detect compromised binaries or failures at runtime. This dataset contains the final results used to produce Figure 1 in different_test_outcomes.tsv, as well as tools to reproduce these results from several starting points. See README.md for instructions. Citation: Jens Dietrich, Tim White, Valerio Terragni and Behnaz Hassanshahi. Towards Cross-Build Differential Testing. 18th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2025.近期针对软件供应链安全（software supply chain security）的担忧，催生了基于同一源代码构建的不同二进制文件（binary）。此类场景有时会生成并非完全一致的二进制文件，进而导致其加密哈希值各不相同。由此引出一个核心问题：这些二进制文件是否仍具备等价性，即二者的运行行为是否完全一致。我们探究了能否通过差分测试（differential testing）为二进制文件的非等价性提供佐证。 我们针对3541组二进制文件对展开研究，这些文件均基于同一Maven构件（Maven artifact）版本构建，且发布于Maven中央仓库（Maven Central）、谷歌保障开源软件（Google Assured Open Source Software）及/或甲骨文源码构建服务（Oracle Build-From-Source）。我们借助EvoSuite为来自Maven中央仓库的基准二进制文件生成测试用例，将这些用例分别应用于该基准文件与所有可获取的其他构建版本二进制文件，并比对测试结果的一致性。我们认为，任何测试结果差异均可能预示程序行为发生变异，因此可用于检测被篡改的二进制文件或运行时故障。 本数据集包含用于生成图1的最终结果（存储于different_test_outcomes.tsv文件中），同时附带了可从多个起点复现该结果的工具。操作说明详见README.md文件。 引用信息：Jens Dietrich、Tim White、Valerio Terragni与Behnaz Hassanshahi。《面向跨构建差分测试》（Towards Cross-Build Differential Testing）。2025年第18届IEEE软件测试、验证与确认大会（18th IEEE International Conference on Software Testing, Verification and Validation, ICST 2025）。