Two-Context Mode Split: Substitution vs Synthesis under a Token-Cap Sweep on a Production Arabic Sales Router

Measurement record: the two model calls of a production Arabic e-commerce salesrouter — a stateless routing call ("substitution") and a contextual reply call("synthesis") — swept across four completion-token caps (400 / 800 / 1600 /4096) at N=20 per cell. v1.0.0: 160/160 calls clean, zero failures,gpt-4o-mini, T=0.2. v1.1.0 adds the instruction-removed attribution arm: 160further calls (80 control + 80 with the prompt's only explicit lengthinstruction deleted from the frozen payload — a byte-verified, single-span15-byte change), 160/160 clean, zero retries, same model, temperature and grid. The prompts are the ones the router actually ships: imported from the runningapplication's prompt builders at execution time, never retyped, and the fullyresolved message payloads are published in this record. The entire run thereforereplays against any OpenAI-compatible endpoint with fetch and nothing else — noapplication source, no database, no framework. This is the production-runtime leg of a three-stack research chain on thesubstitution-vs-synthesis split, joining a sovereign Ollama fixture leg(Converse, Triava Labs, 26b MoE) and a cognitive-middleware leg (Seo,Hashevolution, JAMES at e4b). What this leg adds is the thing a fixture sweepcannot: the same two call shapes measured through a production runtime — persona,store policies, a sales-mode block, a dialect guide, grounding rules and anegotiation contract wrapped around them. HEADLINE RESULTS Confidence grading: ★★★ = re-derivable from the published raw data alone;★★ = mechanism-supported but not ablated; ★ = single-shape observation, do notgeneralize beyond it. ★★★ Substitution is exactly flat. 49 completion tokens with min = median = maxat every cap; one distinct output hash across all 80 calls — byte-identical overa 10× cap range; 0/80 truncated; 1.2% of the largest cap tested. ★★★ Synthesis does not track the cap here. Medians 127.5 / 130.5 / 118.5 /122.5 across the four caps — a 12-token, non-monotonic movement — whilewithin-cell ranges run 48–63 tokens, i.e. run-to-run noise is 4–5× thebetween-cap spread. 80 distinct output hashes out of 80 calls; 0/80 truncated;median output is 3% of the largest cap. ★★★ (v1.1.0) The attribution is measured, not guessed. Deleting the shippedprompt's "2-4 sentences." instruction moves per-cap medians from 124–128.5 to133.5–144.5 tokens (pooled 127 → 138.5, 1.09×) — a shift smaller than therun-to-run variation inside any single cell — with 0/160 truncations and a174-token maximum against a 4096 cap. Neither the cap nor the instruction isthe binding constraint; the model's natural answer length for the task is. Ahosted model with no reasoning trace floors near its visible answer. Thesame-day control arm reproduced the v1.0.0 numbers, ruling out provider driftunder the comparison. ★★★ Nothing truncates anywhere: finish_reason is "stop" on 320/320 callsacross both versions. The caps the application actually ships (160 routing /200 response) were never exceeded by any completion — maxima observed: 49 and159 (v1.0.0), and 174 (v1.1.0, instruction removed — 87% of the shippedresponse cap, i.e. the instruction is part of the shipped cap's effectivemargin). ★ Unlooked-for deployment finding: all benign synthesis replies came backcontaining Arabic-Indic digits ٠-٩, with no digit anywhere in the customermessage and ASCII-printed prices in the retrieved context — 80/80 in v1.0.0,and 160/160 across both v1.1.0 arms, including with the length instructionabsent. Every reply quoted a catalog price; an ASCII-only price extractor (theJavaScript \d idiom, which matches [0-9] only) returned an empty list for allof them. The trigger is the deployment's language configuration, not thecustomer. Measured on one turn shape and one store configuration — a total forthis shape, not a traffic-wide rate. WHAT IS IN THE DEPOSIT - analysis.md — the full result: both call shapes, the v1.1.0 attribution arm, the deployment-side findings, declared protocol deviations, threats to validity, open items.- two-context-sweep-2026-08-12T12-27-16-764Z.json — v1.0.0 raw data, all 160 calls: per-call token counts, latency, finish_reason, output hash, full reply text, plus the frozen fixture and resolved prompts.- instruction-off-control-replay-2026-08-19T11-59-57-828Z.json and instruction-off-no-instruction-replay-2026-08-19T12-04-06-633Z.json — v1.1.0 raw data, 80+80 calls (in the repository these live under instruction-off/).- prompts.json — the frozen, fully resolved prompt payloads for both shapes.- instruction-off-variant-prompts.json — the v1.1.0 variant payload: identical bytes minus the single length-instruction span.- instruction-off-make-variant.mjs — builds the variant from prompts.json and mechanically verifies the diff is exactly that one span.- instruction-off-README.md — the arm's design, results and verdict.- replay.mjs — standalone replay driver, stdlib + fetch, zero dependencies; runs the full grid or any single cell against any OpenAI-compatible endpoint; PROMPTS=path selects an alternate frozen payload (new in v1.1.0).- check-numerals.mjs — re-derives the numeral-system counts from the raw data alone; no network, no application source.- two-context-sweep.ts — the original driver, kept as provenance: it imports the prompt builders from the running application rather than copying them, which is why the published prompts cannot have drifted from what ships. It does not run outside that application; use replay.mjs.- README.md, LICENSE. METHOD IN BRIEF OpenAI-compatible /chat/completions; gpt-4o-mini; temperature 0.2 pinned tomatch the two prior legs (the application itself passes no temperature — thebyte-level determinism above is a T=0.2 result, not a claim about thedeployment); v1.0.0: 2 call shapes × 4 caps × N=20 = 160 calls; v1.1.0: thesynthesis shape only, control and instruction-removed payloads, 4 caps × N=20each = 160 calls; frozen fictional store fixture and frozen customer turn sothe record reproduces without any database; zero-failure rule — a cell thatstill fails after retries aborts the run and writes nothing, because a datasetwith holes must never be labelled clean. Across all 320 calls no call needed aretry; attempt counts are recorded per call in the raw JSON. WHAT THIS RECORD DOES NOT DO - It does not claim the workload gradient is absent in general — only that it does not appear on this leg's fixed-workload production reply. The gradient belongs to natural-stop synthesis with varying workload, measured on the middleware leg.- It does not claim the visible-answer-floor attribution generalizes beyond the measured pair: one hosted no-trace model, one production prompt, one frozen fixture. The reasoning half of the separating cell (reasoning mode on/off) is the middleware leg's measurement, not this record's.- It does not claim byte-determinism at production temperature. What is expected to survive temperature is the flat token count, not byte equality.- It does not cover a second model family. The within-stack family split is the named open item, not a shipped result.- It does not measure the negotiating reply shape (pricing-authorization and focus-lock blocks attached) — a real production shape left for a later record.- It does not test the shipped cap values (160/200) directly; the shipped-cap conclusion is derived from where the uncapped completions landed.- It does not state traffic-wide rates. One fixture message per call shape; N=20 measures run-to-run variance at fixed input, not variance across customers. The numeral observation is a total for this turn shape.- It does not study adversarial inputs. The numeral finding arrived unlooked-for on benign traffic; adversarial treatment of that failure class belongs to a separate record.- It is not a software library. The runnable artifact is a replay script; the original driver is provenance. RELATION TO THE CHAIN Sovereign-stack leg: Converse (Triava Labs), DOI 10.5281/zenodo.20570701(github.com/triavalabs/gemma4-26b-mode-split) — substitution deterministic andflat at 26b MoE. Middleware leg: Seo (Hashevolution), DOI10.5281/zenodo.20363998 — v0.3.1, archiving the seven-tier natural-stopgradient closed in PRs #461/#463; the earlier two-mode / three-workload splitis PR #440; cross-stack numbers from the Converse leg are in Issue #448. Thethree-way finding itself is to be archived in a forthcoming joint three-authorZenodo record (Afana / Converse / Seo); this deposit is the production-runtimeper-stack input to it. REPLAY OPENAI_API_KEY=sk-... node replay.mjsOPENAI_API_KEY=sk-... SHAPE=synthesis PROMPTS=instruction-off/variant-prompts.json node replay.mjsOPENAI_BASE_URL=http://localhost:11434/v1 SWEEP_MODEL=gemma3:12b OPENAI_API_KEY=x node replay.mjs