AIClusterKPI: A Multivariate KPI Dataset for Cloud-Native Anomaly Detection

Overview This dataset provides a comprehensive collection of multivariate Key Performance Indicator (KPI) traces covering a continuous period of 16 days with a temporal resolution of one minute. It is specifically designed for time-series anomaly detection models in cloud-native environments. Dataset Composition The 31 features are categorized into two main groups: Node-Level Metrics (28 dimensions): For each of the 7 monitored nodes, four key performance indicators (KPIs) are captured: CPU Utilization: Calculated as a percentage (%). Available Memory: Measured in Gigabytes (GB). Disk I/O Wait Time: Representing I/O pressure in seconds per second (s/s). Network Inbound Traffic: Measured in Megabytes per second (MB/s). Cluster-Level Metrics (3 dimensions): These provide a global view of the cluster health: Pending Pods: The total count of pods awaiting scheduling. API Server Latency: The 99th percentile (P99) response duration in seconds. Container Restarts: The aggregate rate of change in container restart counts over a 15-minute window. Annotations & Ground Truth The dataset is fully annotated with precise temporal intervals. We distinguish between two types of system events: Anomalous Periods (Anomalies): Realistic faults such as gradual memory leakages, CPU synchronization stutters, and network latency jitter. These are labeled as is_anomaly = 1. Benign Events (Noise): Normal but intensive system activities, such as Prometheus metric scraping and log rotation, which often trigger false positives in traditional detectors. These are labeled as is_anomaly = 0. Data Quality & Artifacts Researchers should note that the dataset contains certain measurement noise and monitoring artifacts. Specifically, occasional negative values in CPU utilization may appear; these are not physical system events but arise from counter resets or differential calculation errors (e.g., Prometheus rate() artifacts) during high-load periods. File Structure The AIClusterKPI (ACK) dataset is partitioned into training and testing subsets to facilitate reproducible machine learning experiments. The repository consists of the following four files: groundtruth.csv: The master annotation file containing precise start and end timestamps for all recorded events. It includes metadata for each entry, such as the target node, event type, and a detailed description distinguishing between real anomalies and benign system noise. train1.csv & train2.csv: These files comprise the primary training set, featuring 31-dimensional multivariate KPI traces. They provide the long-term operational baseline required for models to learn the "normal" behavioral patterns of a cloud-native AI cluster under varying workloads. test.csv: The dedicated evaluation dataset used to verify model performance. It contains specifically reserved intervals of both anomalous faults and benign artifacts (e.g., measurement noise, log rotation), allowing for a robust assessment of a detector's precision and false-positive rate.