CODEF Dynamic Cloud–Edge Resource Demand Dataset from Kubernetes Experiments and Stress Testing

This dataset contains real-world worker-level CPU and memory utilisation time-series collected from controlled cloud–edge experiments on Kubernetes-based infrastructures. The data were generated using the CODEF experimentation framework across multiple Kubernetes distributions (Vanilla Kubernetes, K3s) and Container Network Interfaces (Flannel, Calico) under dynamic and heterogeneous workload conditions.Each time series captures the resource usage (CPU and memory) of a worker node executing benchmarking experiments. The experiments include pod-scaling scenarios, compute-intensive workloads generated with stress-ng and the CNCF kube-burner tool, and multi-phase workload transitions, reflecting realistic, fluctuating service loads of benchmarking microservices. The dataset is designed to incorporate heterogeneity across software stacks and workload mixes, reflecting variability rather than a single homogeneous setup.Metrics are sampled every 30 seconds and recorded continuously to evaluate the impact of pod counts and stress levels on resource use. In summary, the dataset comprises a total of 250,432 timestamped samples spanning 280.23 hours (~12 days) across three experimental scenarios conducted between May and September 2025.Three distinct benchmark scenarios are included:Experiment I — Mixed Workload: A batch of 1–100 application pods is launched to measure creation-to-readiness latency at several K8s-cluster scales. In parallel, controlled CPU pressure is generated with stress-ng and kube-burner, while the stress-pod count, target CPU utilisation (in MB), and experiment duration are varied across different ranges.Experiment II — Two-Stage Mixed Workload: The procedure of Experiment I is repeated but divided into two distinct phases: the first half reproduces the original workload unchanged, while the second half increases/decreases the pod count and adjusts the stress parameters. This mid-run shift produces a contrasting load pattern that tests the models' ability to adapt and detect workload transitions.Experiment III — Dynamic Workload Profiles: Three workload profiles with realistic temporal patterns are introduced across 2 diverse K8s flavors (K8s, K3s) and CNI plugins (Flannel, Calico) to capture experimental tradeoffs of Cloud–Edge infrastructures: (i) stable-low, representing lightweight background services (e.g., monitoring, data collectors); (ii) brief-high, applying short bursts from interactive queries or request spikes (e.g., web services); and (iii) long-medium, reflecting sustained server activity. These scenarios are alternated and repeated by users with pod scaling and stress pods, as in the previous experiments.Use Cases and Intended AudienceThis dataset is primarily intended for:Training and evaluating Machine Learning (ML) algorithms for cloud–edge resource demand prediction and anomaly detection.Benchmarking resource provisioning and auto-scaling strategies under realistic workload dynamics.Research on ML-driven network and service management in heterogeneous Kubernetes environments.Development of edge-cloud orchestration policies under resource constraints.Competition and challenge settings evaluating ML approaches for cloud–edge resource management (e.g., resource demand prediction, anomaly detection, service provisioning).Data Origin and Generation MethodologyOrigin: Real-world telemetry data collected from live Kubernetes clusters managed by the CODEF framework.Not based on any pre-existing dataset.Generation process:Kubernetes clusters were provisioned using CODEF across heterogeneous infrastructure nodes (cloud VMs, edge devices including Raspberry Pi).Two Kubernetes distributions were employed: Vanilla Kubernetes (kubeadm) and K3s (lightweight edge distribution).Two Container Network Interfaces were configured: Calico and Flannel.Workloads were generated via pod-scaling experiments (batches of 1–100 application pods) combined with compute-intensive stress-ng and kube-burner workloads, with varying stress-pod counts, target CPU ranges, and experiment durations.CPU and memory utilisation metrics were collected from Grafana dashboards connected to the cluster monitoring stack (node-exporter / Prometheus) and exported as CSV files at 5-second intervals.Each experiment was repeated multiple times to capture variability across runs.Refer to: https://gitlab.eclipse.org/eclipse-research-labs/codeco-project/datasets/edge-cloud-infrastructure/-/tree/main/HEU-101092696-CODECO-CODEF?ref_type=heads