Measuring, Modelling and Exploiting the Memory Hierarchy of TinyML Inference on a Cached Microcontroller

Where model data live — internal SRAM, external PSRAM or memory-mapped flash — dominates the latency of TinyML inference on cached microcontrollers, yet developers place data by rule of thumb and the cache models used to reason about it summarise the cache as a capacity. We study this on the ESP32-S3 with three boards and eleven held-out models under a pre-registered protocol. First, we measure the data cache directly: configured as 64 KB, 8-way, 32-byte lines, it behaves as 512 sets × 4 ways with FIFO replacement, and its ROM line lock removes one way from half of the sets for every 8 KB locked. Line locking therefore yields a calibrated, within-binary capacity axis that is reproducible across boards. Second, a pre-registered capacity predictor fails on held-out layer families (classification accuracy 55%, mean absolute error 17.6 percentage points), because the optimized kernels keep their hot working buffers in the tensor arena, where they conflict with activations set by set. A simulator that combines the measured cache with line-level access traces generated from the kernel source code, real addresses and write-backs predicts held-out real models with frozen predictions: 94.2% classification, Spearman ρ 0.84 and an error of 1.40 points per layer, 0.44 points per model; its fitted miss cost (≈101 cycles) equals the measured PSRAM miss latency. Third, the same simulator chooses which weights to keep in a limited SRAM budget: across 24 model–budget cells and 317 measured placements, its choice ranks at the 94th percentile of measured placements and matches the best measured one in 13 cells. We also show that comparing two builds biases memory effects by up to one point through data layout alone. Code, data and pre-registrations are released.