The Discarded 99.99%: Internal Structure of the Probability Field Before Collapse in Large Language Models.

This paper documents the internal structure of the discarded probability mass in large language models. At each generation step, a model with vocabulary size 100,000 computes probabilities for 100,000 candidate tokens and retains one. The remaining 99.999% is discarded at collapse. This paper demonstrates that the discarded probability mass is not random noise — it contains organized relational structure including amplification, cancellation, alliance, and isolation dynamics between candidate tokens prior to collapse. These dynamics are mathematically defined, causally influential, and entirely invisible to standard interpretability frameworks. A behavioral probing methodology is introduced that extracts this internal structure without requiring direct weight access. The findings represent an unmapped information layer in large language model processing that current interpretability research does not account for.