SWEAT-Classifiers

SWEAT-Classifiers A comprehensive classification framework implementing 16 models for evaluating workload classification under single-infrastructure and cross-infrastructure settings. Pipeline Overview The SWEAT-Classifiers framework follows a structured end-to-end pipeline for workload classification and cross-infrastructure evaluation. Raw Infrastructure Traces │ ▼ Data Preprocessing (unit normalization, CPU stats, resampling) │ ▼ Window Segmentation (window=60s, stride=30s) │ ├───────────────┬────────────────────┐ ▼ ▼ ▼ Group K-Fold Cross-Infrastructure Config Files (Infra-A only) Splits (Infra-A → B/C/D/E) (m_config.py) │ │ ▼ ▼ Model Training Model Evaluation (Grid Search) (Generalization Test) │ │ └───────┬───────┘ ▼ Results & Logs (ALL_RUNS_SUMMARY.csv, logs/, results/) Experimental Validation Framework Single-Infrastructure Configuration We establish performance baselines on Infra-A using 5-fold Group K-Fold cross-validation to prevent data leakage. Standard K-Fold splitting risks leakage in time-series data because windows extracted from the same workload file are highly correlated. If model training includes segments from a session while testing includes other segments from the same session, the model may memorize session-specific patterns rather than learn generalizable workload behavior. Group K-Fold solution: All windows from the same file are grouped into a single fold. Entire files are assigned exclusively to either training or validation in each iteration, ensuring evaluation on truly unseen workload sessions. Hyperparameter optimization: For each classifier, we conduct grid-search optimization over critical parameters using the fixed 5 folds. We evaluate all parameter combinations, record performance across all folds, and select the optimal configuration for each model based on cross-validation accuracy. Cross-Infrastructure Validation We evaluate generalization by training exclusively on Infra-A and testing on unseen infrastructures (Infra-B, C, D, E) individually. This framework tests whether models learn infrastructure-invariant workload signatures rather than overfitting to environment-specific hardware characteristics. The resulting cross-infrastructure performance matrix reveals the portability of classification approaches across heterogeneous environments. Directory Structure SWEAT-classifiers/ ├── config/ │ ├── dataset_manifest.csv # List of all dataset files │ ├── kfolds_5folds.json # Group K-Fold assignments │ └── m_config.py # Central configuration (hyperparams, paths, settings) │ ├── kfold-data/ # Single-infrastructure K-Fold segmented data │ ├── fold_1/ {train/, test/} │ ├── fold_2/ {train/, test/} │ ├── fold_3/ {train/, test/} │ ├── fold_4/ {train/, test/} │ └── fold_5/ {train/, test/} │ ├── cross-infras-validation-data/ # Cross-infrastructure segmented data │ ├── fold_1/ {train/, test/} # Train: Infra-A, Test: Infra-B │ ├── fold_2/ {train/, test/} # Train: Infra-A, Test: Infra-C │ ├── fold_3/ {train/, test/} # Train: Infra-A, Test: Infra-E │ └── fold_4/ {train/, test/} # Train: Infra-A, Test: Infra-D │ ├── src/ │ ├── common_function.py # Shared utility functions │ ├── kfold.py # Generate Group K-Fold splits │ ├── segmentation.py # Segment data into windows (single-infra) │ ├── segmentation-cross-infras.py # Segment cross-infrastructure validation data │ └── run_[modelname]_grid.py # Grid-search for each model │ ├── models/ # Model implementations │ ├── FCN.py │ ├── MLP.py │ ├── SVM.py │ └── ... (complete list below) │ ├── modelsAuthorsCode/ # Original author implementations (where reused) │ └── [model author implementations] │ ├── logs/ # Execution logs from grid-search experiments │ └── [Model_Name]_grid_[Date]-[Time]/ │ ├── results/ # Grid-search results and metrics │ ├── [Model_Name]_grid_[Date]-[Time]/ │ │ ├── ALL_RUNS_SUMMARY.csv # Ranked summary of all hyperparameter configs │ │ └── [individual run results] │ └── cross-infras/ # Cross-infrastructure evaluation results │├── requirements.txt └── README.md # This file Optimal Hyperparameter Configurations The following table summarizes the optimal hyperparameter settings identified through grid-search optimization on Infra-A, used for cross-infrastructure evaluation. These configurations maximize single-infrastructure accuracy and are evaluated for cross-infrastructure generalization. Model Optimal Configuration kNN-DTW k=10, weights=distance, window=3 SVM kernel=rbf, C=100, gamma=scale SVM-DTW kernel=linear, C=100, n_refs=20 Random Forest n_estimators=500, min_samples_split=2, min_samples_leaf=1, max_depth=None, max_features=sqrt MLP epochs=50, batch_size=32, learning_rate=0.1 WEASEL-MUSE window_size=[10,20,30], word_size=6, n_bins=2, chi2_threshold=2, C=5.0 XGBoost n_estimators=400, learning_rate=0.1, max_depth=10, subsample=0.8, colsample_bytree=0.8 LightGBM n_estimators=400, learning_rate=0.05, num_leaves=31, max_depth=None MiniRocket n_kernels=10000 MultiRocket n_kernels=5000 FCN epochs=100, learning_rate=0.001, batch_size=128 ResNet epochs=50, learning_rate=0.001, batch_size=64 InceptionTime epochs=1500, batch_size=64, nb_filters=32, depth=6, kernel_size=41 TodyNet learning_rate=1e-4, kernel_size="11,3,3" MLSTM-FCN epochs=250, learning_rate=0.001, batch_size=128, hidden_size=8 MALSTM-FCN epochs=250, learning_rate=0.001, batch_size=128, hidden_size=8 Key Findings: Single-Infrastructure: Attention-augmented hybrids (MALSTM-FCN: 0.9970) and gradient boosting (XGBoost: 0.9956) achieve top accuracy Cross-Infrastructure: Classical methods (MLP: 0.7323, SVM-DTW: 0.7258) demonstrate superior generalization Trade-off: High single-infrastructure accuracy does not guarantee cross-infrastructure robustness Data Segmentation Strategy Window Configuration Window size: 60 seconds (fixed duration to capture full workload behavior) Stride: 30 seconds (50% overlap for data augmentation while balancing redundancy) This ensures models are evaluated on entirely new workload sessions, not just different time windows from seen sessions. Reproducibility Deterministic Data Processing Group K-Fold assignments are reused across all model evaluations Window segmentation is deterministic and pre-generated; no random resampling during training Evaluation Metrics Primary metrics: Accuracy: Overall classification correctness Training Time: Total time to train model (seconds) Inference Latency: Time to classify a single 60-second window (milliseconds) Secondary metrics: Macro-averaged Precision, Recall, F1-Score (to account for workload class balance) Per-infrastructure accuracy breakdown Requirements All dependencies required to run the SWEAT-Classifiers framework are listed in: requirements.txt Core Dependencies Python 3.14.2 and 3.13.13 Keras TensorFlow 2.21.0 PyTorch 2.11.0 NumPy 2.3.5 pandas 3.0.0 SciPy scikit-learn 1.8.0 Time-Series & Feature Extraction sktime tslearn 0.8.1 pyts pycatch22 Gradient Boosting Models LightGBM 4.6.0 XGBoost 3.2.0 Usage Examples Single-Infrastructure Evaluation # Generate Group K-Fold splits (prevents data leakage from time-series correlation) python src/kfold.py # Segment data into fixed windows (window=60s, stride=30s) python src/segmentation.py # Run grid-search hyperparameter optimization on Infra-A python src/run_fcn_grid.py # Example: FCN python src/run_resnet_grid.py # Example: ResNet # ... (repeat for each model) Results and optimal hyperparameters are saved to results/[Model_Name]_grid_[%Y%m%d-%H%M%S]/ALL_RUNS_SUMMARY.csv Cross-Infrastructure Evaluation # Segment cross-infrastructure validation data python src/segmentation-cross-infras.py # Evaluate best-performing configurations on unseen infrastructures # (Training on Infra-A, testing on Infra-B, C, D, E separately) # Optimal hyperparameters are set as defaults in each model script python models/FCN.py.py --result_dir results/cross-infras/ python models/resnet.py --result_dir results/cross-infras/ # ... (repeat for each model) Results are saved to results/cross-infras/ Changelog Version 1.0 (May 2026) Initial public release Last Updated: May 2026Current Version: 1.0SWEAT-Classifiers 一套综合性分类框架，集成16种模型，用于评估单基础设施与跨基础设施场景下的工作负载分类性能。 ## 流水线概览 SWEAT-Classifiers框架遵循结构化的端到端流水线，用于工作负载分类与跨基础设施评估： 原始基础设施追踪（Raw Infrastructure Traces） │ ▼ 数据预处理（单位归一化、CPU统计量重采样） │ ▼ 窗口分割（窗口大小=60s，步长=30s） │ ├───────────────┬────────────────────┐ ▼ ▼ ▼ 分组K折交叉验证（仅Infra-A） 跨基础设施拆分（Infra-A → B/C/D/E） 配置文件（m_config.py） │ │ ▼ ▼ 模型训练（网格搜索） 模型评估（泛化测试） │ │ └───────┬───────┘ ▼ 结果与日志（ALL_RUNS_SUMMARY.csv、logs/、results/） ## 实验验证框架 ### 单基础设施配置 我们在Infra-A上采用5折分组K折交叉验证（Group K-Fold Cross-Validation）建立性能基准，以避免数据泄露（data leakage）。标准K折拆分在时间序列数据中存在数据泄露风险：同一工作负载文件提取的窗口具有高度相关性，若训练集包含某会话的片段而测试集包含同一会话的其他片段，模型可能会记住会话特定模式，而非学习到可泛化的工作负载行为。 分组K折交叉验证的解决方案为：将同一文件的所有窗口划分为单个折，每次迭代中整个文件仅分配至训练集或验证集，确保模型在真正未见过的工作负载会话上完成评估。 超参数优化：针对每个分类器，我们使用固定的5折划分进行网格搜索优化，遍历所有关键参数组合，记录各折的性能表现，并基于交叉验证准确率为每个模型选择最优配置。 ### 跨基础设施验证 我们通过仅在Infra-A上训练、分别在未见过的基础设施（Infra-B、C、D、E）上测试的方式评估模型泛化能力。该框架用于测试模型是否学习到与基础设施无关的工作负载特征，而非过拟合至特定环境的硬件特性。最终得到的跨基础设施性能矩阵可揭示分类方法在异构环境下的可移植性。 ## 目录结构 SWEAT-classifiers/ ├── config/ │ ├── dataset_manifest.csv # 所有数据集文件列表 │ ├── kfolds_5folds.json # 分组K折分配方案 │ └── m_config.py # 中央配置文件（含超参数、路径、设置） │ ├── kfold-data/ # 单基础设施K折分割后的数据 │ ├── fold_1/ {train/, test/} │ ├── fold_2/ {train/, test/} │ ├── fold_3/ {train/, test/} │ ├── fold_4/ {train/, test/} │ └── fold_5/ {train/, test/} │ ├── cross-infras-validation-data/ # 跨基础设施验证数据集 │ ├── fold_1/ {train/, test/} # 训练集：Infra-A，测试集：Infra-B │ ├── fold_2/ {train/, test/} # 训练集：Infra-A，测试集：Infra-C │ ├── fold_3/ {train/, test/} # 训练集：Infra-A，测试集：Infra-E │ └── fold_4/ {train/, test/} # 训练集：Infra-A，测试集：Infra-D │ ├── src/ │ ├── common_function.py # 通用工具函数 │ ├── kfold.py # 生成分组K折拆分 │ ├── segmentation.py # 将数据分割为固定窗口（单基础设施） │ ├── segmentation-cross-infras.py # 分割跨基础设施验证数据 │ └── run_[modelname]_grid.py # 针对各模型的网格搜索 │ ├── models/ # 模型实现代码 │ ├── FCN.py │ ├── MLP.py │ ├── SVM.py │ └── ...（完整列表见下文） │ ├── modelsAuthorsCode/ # 复用的原作者实现代码 │ └── [模型原作者实现代码] │ ├── logs/ # 网格搜索实验的执行日志 │ └── [Model_Name]_grid_[Date]-[Time]/ │ ├── results/ # 网格搜索结果与评估指标 │ ├── [Model_Name]_grid_[Date]-[Time]/ │ │ ├── ALL_RUNS_SUMMARY.csv # 所有超参数配置的排序汇总 │ │ └── [单次运行结果] │ └── cross-infras/ # 跨基础设施评估结果 ├── requirements.txt └── README.md # 本文档 ## 最优超参数配置 下表汇总了通过在Infra-A上的网格搜索优化得到的最优超参数设置，将用于跨基础设施评估。这些配置最大化了单基础设施分类准确率，并用于评估跨基础设施泛化能力。 | 模型 | 最优配置 | |--------------|--------------------------------------------------------------------------| | kNN-DTW | k=10, weights=distance, window=3 | | SVM | kernel=rbf, C=100, gamma=scale | | SVM-DTW | kernel=linear, C=100, n_refs=20 | | Random Forest| n_estimators=500, min_samples_split=2, min_samples_leaf=1, max_depth=None, max_features=sqrt | | MLP | epochs=50, batch_size=32, learning_rate=0.1 | | WEASEL-MUSE | window_size=[10,20,30], word_size=6, n_bins=2, chi2_threshold=2, C=5.0 | | XGBoost | n_estimators=400, learning_rate=0.1, max_depth=10, subsample=0.8, colsample_bytree=0.8 | | LightGBM | n_estimators=400, learning_rate=0.05, num_leaves=31, max_depth=None | | MiniRocket | n_kernels=10000 | | MultiRocket | n_kernels=5000 | | FCN | epochs=100, learning_rate=0.001, batch_size=128 | | ResNet | epochs=50, learning_rate=0.001, batch_size=64 | | InceptionTime| epochs=1500, batch_size=64, nb_filters=32, depth=6, kernel_size=41 | | TodyNet | learning_rate=1e-4, kernel_size="11,3,3" | | MLSTM-FCN | epochs=250, learning_rate=0.001, batch_size=128, hidden_size=8 | | MALSTM-FCN | epochs=250, learning_rate=0.001, batch_size=128, hidden_size=8 | ## 核心发现 1. **单基础设施场景**：带注意力机制的混合模型（MALSTM-FCN：0.9970）与梯度提升模型（XGBoost：0.9956）取得了最高的分类准确率。 2. **跨基础设施场景**：经典机器学习方法（MLP：0.7323、SVM-DTW：0.7258）展现出更优的泛化性能。 3. **权衡关系**：高单基础设施准确率并不等价于强跨基础设施鲁棒性。 ## 数据分割策略 ### 窗口配置 - 窗口大小：60秒（固定时长，以捕获完整的工作负载行为） - 步长：30秒（50%重叠以实现数据增强，同时平衡数据冗余） 该配置确保模型在全新的工作负载会话上完成评估，而非仅针对已见过会话的不同时间窗口。 ## 可复现性 ### 确定性数据处理 - 所有模型评估均复用同一分组K折分配方案 - 窗口分割为确定性操作并预先生成，训练过程中无随机重采样 ## 评估指标 ### 主指标 1. 准确率：整体分类正确性 2. 训练时间：模型训练总时长（单位：秒） 3. 推理延迟：分类单个60秒窗口的耗时（单位：毫秒） ### 次指标 1. 宏平均精确率、召回率、F1分数（用于考量工作负载类别平衡情况） 2. 各基础设施准确率细分结果 ## 依赖要求 运行SWEAT-Classifiers框架所需的所有依赖均列于`requirements.txt`文件中： ### 核心依赖 - Python 3.14.2 与 3.13.13 - Keras - TensorFlow 2.21.0 - PyTorch 2.11.0 - NumPy 2.3.5 - pandas 3.0.0 - SciPy - scikit-learn 1.8.0 ### 时间序列与特征提取 - sktime - tslearn 0.8.1 - pyts - pycatch22 ### 梯度提升模型 - LightGBM 4.6.0 - XGBoost 3.2.0 ## 使用示例 ### 单基础设施评估 bash # 生成分组K折拆分（规避时间序列相关性引发的数据泄露） python src/kfold.py # 将数据分割为固定窗口（窗口=60s，步长=30s） python src/segmentation.py # 在Infra-A上运行网格搜索超参数优化 python src/run_fcn_grid.py # 示例：FCN模型 python src/run_resnet_grid.py # 示例：ResNet模型 # ... 为每个模型重复上述步骤 结果与最优超参数将保存至`"results/[Model_Name]_grid_[%Y%m%d-%H%M%S]/ALL_RUNS_SUMMARY.csv"` ### 跨基础设施评估 bash # 分割跨基础设施验证数据 python src/segmentation-cross-infras.py # 在未见过的基础设施上评估最优配置 # （训练集：Infra-A，测试集分别为Infra-B、C、D、E） # 最优超参数已在各模型脚本中设为默认值 python models/FCN.py --result_dir results/cross-infras/ python models/resnet.py --result_dir results/cross-infras/ # ... 为每个模型重复上述步骤 结果将保存至`results/cross-infras/` ## 更新日志 ### 版本1.0（2026年5月） 初始公开版本 最后更新：2026年5月 | 当前版本：1.0