PerfCount: A Performance Counter Dataset for CPU Prediction Problems

Hardware performance counters have been widely used as inputs to manage and predict CPU and computer system behavior. In particular, machine learning (ML)-based methods have emerged as a promising direction to leverage performance counter data for design- and run-time optimization. However, developing heuristics or training and testing ML models requires collecting a large amount of reliable hardware counter data across different target platforms and applications. The lack of publicly available datasets in this space has hindered reproducibility and wider exploration of system prediction methods. This paper presents PerfCount, a public dataset of hardware performance counter traces collected from heterogeneous and homogeneous edge, desktop, and server systems. PerfCount is accompanied by an automated framework for hardware counter data collection, curation, and evaluation to facilitate replication and extension. PerfCount is intended to support expansions with new platforms, workloads, and models to enable future system prediction research. We evaluate PerfCount on a range of case studies demonstrating an average of less than 4% MAPE for cross-frequency prediction, below 10% for workload forecasting tasks on SPEC, and 12–28% MAPE for cross-system counter prediction. In the process, we also evaluate the predictability of different workloads and platforms to highlight future prediction opportunities and challenges.