Token-as-a-Utility: Hardware Measurements, Calibrated-Model Outputs and Scripts
收藏资源简介:
Data and scripts for the paper "Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services". The deposit contains (1) hardware measurements of LLM inference on three NVIDIA RTX 5000 Ada GPUs with vLLM 0.11.0 (experiments E1 to E8: decode and prefill rooflines, per-request overhead, cold-load and sleep/wake swap times, guaranteed-class TPOT attainment under leases, prefix-cache discount, quality-class check on MMLU-Pro, and trace-driven energy runs), with GPU board energy from NVML; (2) the calibrated multi-tenant model and analysis scripts with their outputs at the nominal and measured coefficient vintages, including the sensitivity sweep; and (3) raw token counts across languages and tokenisers with the model configuration files used for the architecture constants. README.md maps each file to the paper and lists the environment settings needed to reproduce every run. Third-party inputs (BurstGPT, Azure LLM Inference Trace 2023, MMLU-Pro) are not redistributed; the included scripts rebuild them from their public sources.



