five

M100 dataset 6: 22-03

收藏
Zenodo2023-05-22 更新2026-05-26 收录
下载链接:
https://zenodo.org/record/7590061
下载链接
链接失效反馈
官方服务:
资源简介:
This entry is a part of a larger data set collected from the most recent Tier-0 supercomputer hosted at CINECA (Marconi100, https://www.hpc.cineca.it/hardware/marconi100). The data covers the entirety of the system, ranging from the computing nodes (980+ computing nodes) internal information such as core loads, temperatures, frequencies, memory write/read operations, CPU power consumption, fan speed, GPU usage details, etc., to the system-wide information, including the liquid cooling infrastructure, the air conditioning system, the power supply units, workload manager statistics, and job-related information, system status alerts, and weather forecast.<br> It comprises hundreds of metrics measured on each computing node, in addition to hundreds of other metrics gathered from sensors monitored along all system components.<br> The whole data set is stored as a collection of Zenodo entries; this particular entry corresponds to the period: 22-03. The dataset is stored as a partitioned Parquet dataset, with this partitioning hierarchy: year_month ("YY-MM"), plugin, metric. The data is distributed as tarball files, each corresponding to one month of data (first-level partitioning, year_month).<br> The collected data is generated by a monitoring infrastructure working on unstructured data (to improve efficiency and scalability); however, this data has been organized in a structured manner to facilitate its fruition. The simplest way to understand how the access the data is to refer to the companion software modules released together with the dataset itself, which can be found at: https://gitlab.com/ecs-lab/exadata.

本条目隶属于一套从意大利国家高性能计算中心(CINECA)托管的最新Tier-0级超级计算机Marconi100(详见https://www.hpc.cineca.it/hardware/marconi100)采集的大型数据集。该数据集覆盖该超级计算机全系统范畴:从计算节点(共计980余台)的内部运行参数,如核心负载、温度、主频、内存读写操作、CPU功耗、风扇转速、GPU使用详情等,到全系统级别的监测数据,包含液冷基础设施、空调系统、电源单元、作业调度管理器统计信息、作业相关数据、系统状态告警以及气象预报数据。 本数据集不仅包含从全系统各组件传感器采集的数百项监测指标,还涵盖每台计算节点上测得的数百项运行指标。 整套数据集以多个Zenodo条目形式存储,本条目对应22-03时段的数据。该数据集采用分区Parquet格式存储,分区层级为:年_月(格式为"YY-MM")、插件(plugin)、指标(metric)。数据以tarball压缩包形式分发,每个压缩包对应一个月的数据集(一级分区为年_月)。 本数据集的原始采集数据由一套面向非结构化数据的监控基础设施生成(该设计旨在提升效率与可扩展性);但为便于数据使用,已将其整理为结构化格式。了解数据访问方式的最简途径是参考随数据集一同发布的配套软件模块,相关代码可在https://gitlab.com/ecs-lab/exadata获取。
提供机构:
Zenodo
创建时间:
2023-02-02
二维码
社区交流群
二维码
科研交流群
商业服务