SWEAT-Classifiers

SWEAT-Classifiers A comprehensive classification framework implementing 16 models for evaluating workload classification under single-infrastructure and cross-infrastructure settings. Pipeline Overview The SWEAT-Classifiers framework follows a structured end-to-end pipeline for workload classification and cross-infrastructure evaluation. Raw Infrastructure Traces │ ▼ Data Preprocessing (unit normalization, CPU stats, resampling) │ ▼ Window Segmentation (window=60s, stride=30s) │ ├───────────────┬────────────────────┐ ▼ ▼ ▼ Group K-Fold Cross-Infrastructure Config Files (Infra-A only) Splits (Infra-A → B/C/D/E) (m_config.py) │ │ ▼ ▼ Model Training Model Evaluation (Grid Search) (Generalization Test) │ │ └───────┬───────┘ ▼ Results & Logs (ALL_RUNS_SUMMARY.csv, logs/, results/) Experimental Validation Framework Single-Infrastructure Configuration We establish performance baselines on Infra-A using 5-fold Group K-Fold cross-validation to prevent data leakage. Standard K-Fold splitting risks leakage in time-series data because windows extracted from the same workload file are highly correlated. If model training includes segments from a session while testing includes other segments from the same session, the model may memorize session-specific patterns rather than learn generalizable workload behavior. Group K-Fold solution: All windows from the same file are grouped into a single fold. Entire files are assigned exclusively to either training or validation in each iteration, ensuring evaluation on truly unseen workload sessions. Hyperparameter optimization: For each classifier, we conduct grid-search optimization over critical parameters using the fixed 5 folds. We evaluate all parameter combinations, record performance across all folds, and select the optimal configuration for each model based on cross-validation accuracy. Cross-Infrastructure Validation We evaluate generalization by training exclusively on Infra-A and testing on unseen infrastructures (Infra-B, C, D, E) individually. This framework tests whether models learn infrastructure-invariant workload signatures rather than overfitting to environment-specific hardware characteristics. The resulting cross-infrastructure performance matrix reveals the portability of classification approaches across heterogeneous environments. Directory Structure SWEAT-classifiers/ ├── config/ │ ├── dataset_manifest.csv # List of all dataset files │ ├── kfolds_5folds.json # Group K-Fold assignments │ └── m_config.py # Central configuration (hyperparams, paths, settings) │ ├── kfold-data/ # Single-infrastructure K-Fold segmented data │ ├── fold_1/ {train/, test/} │ ├── fold_2/ {train/, test/} │ ├── fold_3/ {train/, test/} │ ├── fold_4/ {train/, test/} │ └── fold_5/ {train/, test/} │ ├── cross-infras-validation-data/ # Cross-infrastructure segmented data │ ├── fold_1/ {train/, test/} # Train: Infra-A, Test: Infra-B │ ├── fold_2/ {train/, test/} # Train: Infra-A, Test: Infra-C │ ├── fold_3/ {train/, test/} # Train: Infra-A, Test: Infra-E │ └── fold_4/ {train/, test/} # Train: Infra-A, Test: Infra-D │ ├── src/ │ ├── common_function.py # Shared utility functions │ ├── kfold.py # Generate Group K-Fold splits │ ├── segmentation.py # Segment data into windows (single-infra) │ ├── segmentation-cross-infras.py # Segment cross-infrastructure validation data │ └── run_[modelname]_grid.py # Grid-search for each model │ ├── models/ # Model implementations │ ├── FCN.py │ ├── MLP.py │ ├── SVM.py │ └── ... (complete list below) │ ├── modelsAuthorsCode/ # Original author implementations (where reused) │ └── [model author implementations] │ ├── logs/ # Execution logs from grid-search experiments │ └── [Model_Name]_grid_[Date]-[Time]/ │ ├── results/ # Grid-search results and metrics │ ├── [Model_Name]_grid_[Date]-[Time]/ │ │ ├── ALL_RUNS_SUMMARY.csv # Ranked summary of all hyperparameter configs │ │ └── [individual run results] │ └── cross-infras/ # Cross-infrastructure evaluation results │├── requirements.txt └── README.md # This file Optimal Hyperparameter Configurations The following table summarizes the optimal hyperparameter settings identified through grid-search optimization on Infra-A, used for cross-infrastructure evaluation. These configurations maximize single-infrastructure accuracy and are evaluated for cross-infrastructure generalization. Model Optimal Configuration kNN-DTW k=10, weights=distance, window=3 SVM kernel=rbf, C=100, gamma=scale SVM-DTW kernel=linear, C=100, n_refs=20 Random Forest n_estimators=500, min_samples_split=2, min_samples_leaf=1, max_depth=None, max_features=sqrt MLP epochs=50, batch_size=32, learning_rate=0.1 WEASEL-MUSE window_size=[10,20,30], word_size=6, n_bins=2, chi2_threshold=2, C=5.0 XGBoost n_estimators=400, learning_rate=0.1, max_depth=10, subsample=0.8, colsample_bytree=0.8 LightGBM n_estimators=400, learning_rate=0.05, num_leaves=31, max_depth=None MiniRocket n_kernels=10000 MultiRocket n_kernels=5000 FCN epochs=100, learning_rate=0.001, batch_size=128 ResNet epochs=50, learning_rate=0.001, batch_size=64 InceptionTime epochs=1500, batch_size=64, nb_filters=32, depth=6, kernel_size=41 TodyNet learning_rate=1e-4, kernel_size="11,3,3" MLSTM-FCN epochs=250, learning_rate=0.001, batch_size=128, hidden_size=8 MALSTM-FCN epochs=250, learning_rate=0.001, batch_size=128, hidden_size=8 Key Findings: Single-Infrastructure: Attention-augmented hybrids (MALSTM-FCN: 0.9970) and gradient boosting (XGBoost: 0.9956) achieve top accuracy Cross-Infrastructure: Classical methods (MLP: 0.7323, SVM-DTW: 0.7258) demonstrate superior generalization Trade-off: High single-infrastructure accuracy does not guarantee cross-infrastructure robustness Data Segmentation Strategy Window Configuration Window size: 60 seconds (fixed duration to capture full workload behavior) Stride: 30 seconds (50% overlap for data augmentation while balancing redundancy) This ensures models are evaluated on entirely new workload sessions, not just different time windows from seen sessions. Reproducibility Deterministic Data Processing Group K-Fold assignments are reused across all model evaluations Window segmentation is deterministic and pre-generated; no random resampling during training Evaluation Metrics Primary metrics: Accuracy: Overall classification correctness Training Time: Total time to train model (seconds) Inference Latency: Time to classify a single 60-second window (milliseconds) Secondary metrics: Macro-averaged Precision, Recall, F1-Score (to account for workload class balance) Per-infrastructure accuracy breakdown Requirements All dependencies required to run the SWEAT-Classifiers framework are listed in: requirements.txt Core Dependencies Python 3.14.2 and 3.13.13 Keras TensorFlow 2.21.0 PyTorch 2.11.0 NumPy 2.3.5 pandas 3.0.0 SciPy scikit-learn 1.8.0 Time-Series & Feature Extraction sktime tslearn 0.8.1 pyts pycatch22 Gradient Boosting Models LightGBM 4.6.0 XGBoost 3.2.0 Usage Examples Single-Infrastructure Evaluation # Generate Group K-Fold splits (prevents data leakage from time-series correlation) python src/kfold.py # Segment data into fixed windows (window=60s, stride=30s) python src/segmentation.py # Run grid-search hyperparameter optimization on Infra-A python src/run_fcn_grid.py # Example: FCN python src/run_resnet_grid.py # Example: ResNet # ... (repeat for each model) Results and optimal hyperparameters are saved to results/[Model_Name]_grid_[%Y%m%d-%H%M%S]/ALL_RUNS_SUMMARY.csv Cross-Infrastructure Evaluation # Segment cross-infrastructure validation data python src/segmentation-cross-infras.py # Evaluate best-performing configurations on unseen infrastructures # (Training on Infra-A, testing on Infra-B, C, D, E separately) # Optimal hyperparameters are set as defaults in each model script python models/FCN.py.py --result_dir results/cross-infras/ python models/resnet.py --result_dir results/cross-infras/ # ... (repeat for each model) Results are saved to results/cross-infras/ Changelog Version 1.0 (May 2026) Initial public release Last Updated: May 2026Current Version: 1.0