GenAI Benchmarking Dataset for Edge Devices

The benchmarking dataset, GenAI on the Edge, contains performance metrics from evaluating Large Language Models (LLMs) on edge devices, utilizing a distributed testbed of Raspberry Pi devices orchestrated by Kubernetes (K3s). It includes performance data collected from multiple runs of prompt-based evaluations with various LLMs, leveraging Prometheus and the Llama.cpp framework. The dataset captures key metrics such as resource utilization, token generation rates/throughput, and detailed inference timing for stages such as Sample, Prefill, and Decode. These metrics enable benchmarking of LLM performance in constrained, distributed environments, offering valuable insights into the feasibility and efficiency of deploying generative AI on edge computing devices.