LocalMode Bench: analysis artifact and data bundles for the cross-runtime in-browser LLM and embedding inference methodology

What this record is This is the frozen, reproducible analysis kit behind the LocalMode Bench methodology (protocol localmode-bench/5). It holds the data, the software that collected it, the pipeline that regenerates every figure and number, and the manuscript sources. This is version 1. All results come from one pinned dataset commit, feae71c. The full hash is in the identifiers list at the end. Files analysis-artifact.tar.gz (6.3 MB): the pipeline, the LaTeX sources, the analysis exports, and ARTIFACT.md with the hash of every input and output. analysis-artifact.filelist.txt: the tarball's table of contents, with a SHA-256 for each file. LocalMode-Bench.bundle (62 MB): the dataset repository as a git bundle. It has every run file, the controls, the quarantine, the native baselines and the errata. CC0. LocalMode.bundle (12 MB): the monorepo. It has the browser runner, the submission service and the analysis CLI. MIT. Quick start Clone the two repositories from the bundles: git clone LocalMode-Bench.bundle LocalMode-Bench-feae71c git -C LocalMode-Bench-feae71c checkout --detach feae71c4f7dd4add068f4587194b41c4a718b7a5 git clone LocalMode.bundle LocalMode git -C LocalMode checkout --detach 10b3d01e9cc7fd45236941a08443423c0300060a Then regenerate the outputs in artifact/figures: python3.14 -m venv .venv .venv/bin/pip install -r requirements.txt .venv/bin/python build_all.py shasum -a 256 captions.tex out/* You need Python 3.14, Tectonic 0.17.0, git, Node.js 22.13 or later with pnpm 10, and network access on the first run. The build first checks the dataset against the pin. It then writes 125 output files, or 116 without the control runs. make VENUE=mlsys ANON=0 in artifact/draft rebuilds the manuscript. The full procedure is in artifact/figures/README.md. What is not included Model weights, browsers and devices are not included. The run files hold every measurement. Checking integrity Compare every output hash with ARTIFACT.md section 4 (pipeline version 50). Check the extracted tarball with grep -v '^symlink' analysis-artifact.filelist.txt | shasum -a 256 -c. The SHA-256 of the four files are listed below. Licences Dataset bundle and artifact/data/: CC0 1.0. Monorepo bundle, artifact/figures/, ARTIFACT.md, history-map.md and the file list: MIT. artifact/draft/ except draft/style/: all rights reserved, provided for regenerating and checking the outputs. draft/style/: the MLSys 2025 style kit's own terms. History note On 2026-10-14 the dataset's public history was rewritten to remove a computer name a participant typed into a free-text answer. No file content changed. Two earlier citations map 9898aaf to feae71c and c530be8 to 680cfbb. artifact/history-map.md carries the map and the check. Links Dataset: https://github.com/LocalMode-AI/LocalMode-Bench Monorepo: https://github.com/LocalMode-AI/LocalMode Leaderboard: https://localmode.ai/bench Methodology: https://localmode.ai/bench/methodology The manuscript is linked under Related works once its arXiv identifier exists. Identifiers and checksums analysis-artifact.tar.gz: 6,330,493 bytes, SHA-256 1bba6fbff9cd017ef013f82e007b3b52e952dd53f86f1905afaf29563958cc30 analysis-artifact.filelist.txt: 14,767 bytes, SHA-256 290552d86dc9546dd394c409e1c928494d1e343104eea644efba30984edf3081 LocalMode-Bench.bundle: 62,454,783 bytes, SHA-256 007c53bfb24970c945641a9b5c744e8e86a68105f8eef668c749a832fe250fc0 LocalMode.bundle: 12,131,278 bytes, SHA-256 a19ef151598952d4b0a1d7d0e36aa0fe79ac530fb3829bbc3e09b2249a974a51 Dataset bundle head: main at e3c44550e68c24719046b0ce33c318e0046c1b44 (2026-10-26), the controls commit with 112 control runs. Monorepo bundle head: main at 85fd9f241d3018f0a00c4402420e60b9b83890c9 (2026-10-24), @localmode/bench 0.9.14. Dataset pin: feae71c4f7dd4add068f4587194b41c4a718b7a5, 488 run files, 411 under protocol version 5. Monorepo commit the pipeline depicts: 10b3d01e9cc7fd45236941a08443423c0300060a. Errata commit: 23c441f. It excludes 12 cells of 11 runs from aggregates. Nothing is re-scored. Analysis exports written by @localmode/bench 0.9.5 at fb9c27e; earlier lab exports by 0.9.3 at 1a6d125.