A comprehensive benchmark dataset capturing power consumption, performance metrics, and timing traces for Large Language Models (LLMs) running on TensorRT-LLM inference engine using NVIDIA H100 GPUs.
"Large-language model (LLM) inference is a rapidly growing class of computer workload, with over 100~GW of compute capacity expected to come online in the next 5 years. The most popular chips used for
A comprehensive benchmark dataset capturing power consumption, performance metrics, and timing traces for Large Language Models (LLMs) running on TensorRT-LLM inference engine using NVIDIA H100 GPUs.
