Literature on Cloud Capacity Planning

This release captures the state-of-the-art (as of 2020) in cloud capacity planning literature and provides a set of complementary scripts to analyze this literature. The dataset which is central to this release (publications.yaml) maps 57 cloud capacity planning approaches as published in literature to the taxonomy on cloud capacity planning which the authors of this release have proposed. The approaches were gathered with a systematic literature survey process, aggregating multiple common sources and executing a set of automated and manual filtering steps. Taxonomy The taxonomy and the process used to derive it is described in detail in the MSc Thesis of Georgios Andreadis at Delft University of Technology (to be published end of August 2020), on cloud capacity planning. We describe the taxonomy here to provide context to the raw data. The taxonomy divides the process underlying capacity planning systems into the following categories: System Model Workloads Resources Model Inputs Forecast Model Modeling Strategy Model Structure Decision Support Role Type of Advice Advice Method For each of these categories, the taxonomy prescribes a set of possible classes (possible instantiations of the category). We list these for each category, below, preceded by its abbrevation as appearing in the dataset: System Model Workloads VM: Virtual Machines DB: Databases S: Streaming Workloads BD: Big Data Frameworks WS: Web Service B: Batch Jobs Resources C: Compute Hardware S: Storage Hardware N: Network Hardware E: Energy Hardware (Storage and Supply) H: Heat Control Hardware V: Virtualized Resources (VM, containers, etc.) Model Inputs H: Historical Data RS: Resource Specifications B: (Micro)Benchmarks or Systematic Performance Tests S: SLAs P: Pricing Data LC: Lease Contracts HP: Human Personnel-related Factors Forecast Model Modeling Strategy A: Analytical S: Simulation E: Real-world Experimentation Model Structure U: Unconditional Extrapolation W: What-if Scenarios Decision Support Role F: Forecast A: Adaptation Advice Type of Advice N: Number of Resources T: Type of Resources L: Locality of Resources Advice Method H: Heuristic R: Regression L: Local Search SS: Stochastic Search SP: Stochastic Programming NN: Neural Network GT: Game Theory GA: Genetic Algorithm NLP: (Non)Linear Programming File Structure This release is structured as follows: publications.yaml: This is the dataset of mappings of publications to the taxonomy. Each item in the array represents a publication, with a set of true-false classifications per category for each class. The id field of each publication identifies the publication (first-author and publication year). The summary field of each publication summarizes the publication in a short sentence. The classification field contains a set of true-false classifications per category for each class. The notes field is an optional field containing any additional notes kept by the author of this dataset on their classification, in the case where doubts arose during the classification process. taxonomy.py: Script which parses the YAML dataset into different CSV views per category, to facilitate meta-analysis. Also prints out a full (long-table) representation of the mappings. taxonomy_analysis.py: Jupyter notebook which contains several meta-analysis processing steps, including trend, cluster, and correlation analysis. README.md: A file containing this description.本数据集发布包收录了截至2020年的云容量规划（cloud capacity planning）领域前沿研究成果，并配套提供了一组用于分析该类文献的辅助脚本。本发布包的核心数据集为`publications.yaml`，它将文献中收录的57种云容量规划方法，映射至本发布包作者提出的云容量规划分类体系中。上述研究方法通过系统性文献调研流程收集所得，整合了多个主流数据源，并经过了一系列自动化与人工筛选步骤。 ### 分类体系 本分类体系及其构建流程，详见代尔夫特理工大学（Delft University of Technology）Georgios Andreadis将于2020年8月底发表的理学硕士学位论文（MSc Thesis），研究方向为云容量规划。为便于理解原始数据，下文将对该分类体系进行简要介绍。该分类体系将容量规划系统的核心流程划分为以下类别：系统模型、工作负载、资源、模型输入、预测模型、建模策略、模型结构、决策支持角色、建议类型、建议方法。针对每一类别，分类体系均预设了一组可选类别（即该类别的具体实例）。下文将按类别逐一列出各可选项，并标注数据集内使用的对应缩写： #### 工作负载（Workloads） VM：虚拟机（Virtual Machines） DB：数据库（Databases） S：流处理工作负载（Streaming Workloads） BD：大数据框架（Big Data Frameworks） WS：Web服务（Web Service） B：批处理作业（Batch Jobs） #### 资源（Resources） C：计算硬件（Compute Hardware） S：存储硬件（Storage Hardware） N：网络硬件（Network Hardware） E：能源硬件（Storage and Supply，即能源存储与供给硬件） H：热控硬件（Heat Control Hardware） V：虚拟化资源（Virtualized Resources，含虚拟机、容器等） #### 模型输入（Model Inputs） H：历史数据（Historical Data） RS：资源规格（Resource Specifications） B：（微）基准测试或系统性性能测试（(Micro)Benchmarks or Systematic Performance Tests） S：服务水平协议（SLAs） P：定价数据（Pricing Data） LC：租赁合约（Lease Contracts） HP：与人力相关的因素（Human Personnel-related Factors） #### 预测模型与建模策略（Forecast Model & Modeling Strategy） A：分析方法（Analytical） S：仿真方法（Simulation） E：真实实验法（Real-world Experimentation） #### 模型结构（Model Structure） U：无条件外推（Unconditional Extrapolation） W：假设场景分析法（What-if Scenarios） #### 决策支持角色（Decision Support Role） F：预测（Forecast） A：自适应（Adaptation） #### 建议类型（Type of Advice） N：资源数量（Number of Resources） T：资源类型（Type of Resources） L：资源布局（Locality of Resources） #### 建议方法（Advice Method） H：启发式方法（Heuristic） R：回归方法（Regression） L：局部搜索法（Local Search） SS：随机搜索法（Stochastic Search） SP：随机规划法（Stochastic Programming） NN：神经网络（Neural Network） GT：博弈论（Game Theory） GA：遗传算法（Genetic Algorithm） NLP：（非）线性规划（(Non)Linear Programming） ### 文件结构 本发布包的文件结构如下： 1. `publications.yaml`：本数据集的核心文件，实现了文献与分类体系的映射。数组中的每个元素代表一篇文献，包含针对每个类别下各子类别的布尔型分类标记。每篇文献的`id`字段用于标识该文献，格式为第一作者+发表年份；`summary`字段以短句概括该文献的核心内容；`classification`字段包含各分类类别下各子类别的布尔分类结果；`notes`为可选字段，用于记录本数据集作者在分类过程中产生的额外备注，例如分类时存在疑虑的相关内容。 2. `taxonomy.py`：用于将YAML格式的数据集解析为各分类类别的CSV视图，以支持元分析；同时可输出映射关系的完整长表形式。 3. `taxonomy_analysis.py`：Jupyter Notebook脚本，包含多种元分析处理流程，涵盖趋势分析、聚类分析与相关性分析。 4. `README.md`：包含本数据集描述的说明文件。