Oracles for the Equivalence of Java Bytecode

Incidents like log4shell and SolarWinds have led to an increased focus on software supply chain security. A particular concern is the detection and prevention of compromised builds. A common approach is to independently re-build projects, and compare the results. This leads to the availability of different binaries built from the same sources, and raises the question of how to compare the respective binaries (to confirm the integrity of builds, to detect compromised builds, etc). It is however not clear how to do this: naive bitwise comparison is often too strict, and establishing the behavioural equivalence of two binaries is undecidable. A pragmatic step towards a solution is to provision a benchmark that can be used to test and train equivalence relations. We present such a benchmark for Java bytecode, consisting of \input{generated/total-oracle-record-count}pairs of binaries (compiled Java classes) labelled as to whether these classes are equivalent or not. We refer to these pairs as equivalence and non-equivalence oracles, respectively. We derive equivalence oracles from building 56 projects and project versions using 32 dockerised build environments (with different compilers, compiler versions and configurations). Non-equivalence oracles are derived from three different sources: (1) proven breaking API changes, (2) semantic code changes synthesised by means of bytecode mutations, and (3) code changes extracted from vulnerability patches. A detailed description of the dataset can be found in: Jens Dietrich, Tim White, Mohammad Mahdi Abdollahpou, Elliott Wen and Behnaz Hassanshahi: BenEq -- A Benchmark of Compiled Java Programs to Assess Alternative Builds. Proceedings of the ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses (SCORED '24).诸如Log4Shell（log4shell）与SolarWinds（SolarWinds）等安全事件，促使全球学界与产业界对软件供应链安全的关注度显著提升。其中尤为受关注的核心问题，是受入侵构建（compromised builds）的检测与防范。当前主流的应对思路为对项目执行独立重构建，并比对最终构建结果。由此会产生由同一源代码编译得到多份不同二进制文件（binary）的场景，进而引出如何有效比对这些二进制文件的核心问题——例如验证构建完整性、识别受入侵构建等。但目前尚无通用可行的解决方案：单纯的逐位比对往往过于严苛，而证明两份二进制文件的行为等价性则属于不可判定问题。 迈向该问题解决方案的务实之举，是构建可用于测试与训练等价关系的基准数据集。本文提出一款针对Java字节码（Java bytecode）的基准数据集，包含共input{generated/total-oracle-record-count}组二进制文件（即编译后的Java类文件），并为每组标注其类文件是否等价。我们将此类标注对分别称为等价预言集（equivalence oracles）与非等价预言集（non-equivalence oracles）。 等价预言集通过32个容器化构建环境（搭载不同编译器、编译器版本与配置参数）编译56个项目及其版本生成。非等价预言集则源自三类不同来源：(1) 已证实会造成破坏性影响的API变更；(2) 通过字节码突变生成的语义代码变更；(3) 从漏洞修复补丁中提取的代码变更。 本数据集的详细说明可参阅以下文献：Jens Dietrich、Tim White、Mohammad Mahdi Abdollahpou、Elliott Wen与Behnaz Hassanshahi：《BenEq——用于评估备选构建方案的编译型Java程序基准数据集》，收录于ACM软件供应链攻击研究与生态系统防御研讨会（SCORED '24）会议论文集。