What Do Collusion Probes Read? Code and data

Code, data, and pre-registration for the paper "What Do Collusion Probes Read? Three Pitfalls in Evaluating Multi-Agent Collusion Monitors" (Mainul Hasan Mohsin and Mohammad Shahidur Rahman). The benchmark contains simulated committee and code-review discussions between LLM agents (Qwen3-14B, Qwen3-32B, Mistral-Small-24B), including agents who receive a collusion instruction and refuse it and agents who choose freely under the same prompt. The code archive holds the episode generator, activation extraction, probes and group statistics, all baselines and analyses, the scripts that turn the result files into every number in the paper, and every committed version of the pre-registration with an OpenTimestamps proof. The data archive holds all episodes (Parquet), stance labels, all result files with the cross-fitted per-run monitor scores, and the anonymised labels of the human stance check. Version 1.2 is the release that accompanies the paper. It contains corrected data: a first version named the wrong option in some colluder briefs; the fix was pre-registered (amendment 7), and every affected episode was regenerated. Version 1.2 also reruns the no-brief concealed-message analyses with the retrained probes, which version 1.1 had missed. Code: MIT licence. Data: CC BY 4.0. The NARCBench scenarios used by the committee task are MIT-licensed (Rose et al., 2026).