M100 dataset 1: from 20-03 to 20-12
收藏资源简介:
This entry is a part of a larger data set collected from the most recent Tier-0 supercomputer hosted at CINECA (Marconi100, https://www.hpc.cineca.it/hardware/marconi100). The data covers the entirety of the system, ranging from the computing nodes (980+ computing nodes) internal information such as core loads, temperatures, frequencies, memory write/read operations, CPU power consumption, fan speed, GPU usage details, etc., to the system-wide information, including the liquid cooling infrastructure, the air conditioning system, the power supply units, workload manager statistics, and job-related information, system status alerts, and weather forecast.<br> It comprises hundreds of metrics measured on each computing node, in addition to hundreds of other metrics gathered from sensors monitored along all system components.<br> The whole data set is stored as a collection of Zenodo entries; this particular entry corresponds to the period: 20-03, 20-12. The dataset is stored as a partitioned Parquet dataset, with this partitioning hierarchy: year_month ("YY-MM"), plugin, metric. The data is distributed as tarball files, each corresponding to one month of data (first-level partitioning, year_month).<br> The collected data is generated by a monitoring infrastructure working on unstructured data (to improve efficiency and scalability); however, this data has been organized in a structured manner to facilitate its fruition. The simplest way to understand how the access the data is to refer to the companion software modules released together with the dataset itself, which can be found at: https://gitlab.com/ecs-lab/exadata.
本条目隶属于从意大利国家高性能计算中心(CINECA)最新部署的Tier-0超级计算机Marconi100(https://www.hpc.cineca.it/hardware/marconi100)所采集的大型数据集。该数据集覆盖该超级计算机全栈系统,涵盖计算节点(超980台)内部信息与全系统级信息:前者包括核心负载、温度、主频、内存读写操作、CPU功耗、风扇转速、GPU使用详情等;后者则涵盖液冷基础设施、空调系统、电源单元、工作负载管理器统计数据、作业相关信息、系统状态告警以及天气预报数据。除部署于全系统各组件的传感器所采集的数百项指标外,每个计算节点还会上报数百项自身监测指标。整个数据集以多个Zenodo条目形式存储;本条目对应数据周期为20-03至20-12。该数据集采用分区Parquet格式存储,分区层级为:年-月(YY-MM)、插件(plugin)、指标(metric)。数据以tarball压缩包形式分发,每个压缩包对应一个月的数据(一级分区维度为年-月)。所采集的数据最初由面向非结构化数据设计的监控基础设施生成,旨在提升效率与可扩展性,但已被整理为结构化格式以方便使用。若需了解数据访问方法,可参阅随数据集同步发布的配套软件模块,相关资源可通过以下链接获取:https://gitlab.com/ecs-lab/exadata。



