遇见数据集

Write performance with different numbers of OSTs for BeeGFS in PlaFRIM

收藏
Zenodo2026-04-29 更新2026-05-26 收录
官方服务:

资源简介:

This file contains performance measured with the IOR benchmarking tool when writing to the BeeGFS parallel file system following different strategies and using different numbers of OSTs. This data set was used for experiments reported in [1]. Note: The second version of this repository, uploaded in April 2026, includes the "part2" file, which extends the data set to include more access patterns. The full dataset (both parts) was used in [3]. For information, the part 1 was published in January 2024. All experiments were conducted on the PlaFRIM platform (https://www.plafrim.fr/) between June and December 2023 (part 1) and between 2024 and 2025 (part 2), using the Bora cluster. This I/O infrastructure has been described in [2] (the higher-speed network was used for these experiments). IOR version 4.1.0+dev was used, with the POSIX API (-a). The file is a .csv in text format. The relevant columns are: "nodes" is the number of compute nodes and "procs" is the total number of processes. procs/nodes gives hence the number of processes per node. "filestrategy" is either shared-file (where a single file is accessed by all processes) or file-per-proc (where each process has its own file, created by adding the -F IOR option). "spatiality" may be contig (each process has a contiguous portion of the file), strided (1D-strided access pattern, created using the -s option from IOR, only possible for "shared-file" cases), or random (obtained with the -z IOR option). "reqsize" is the size of each request (IOR option: -t). K and M correspond to KiB and MiB, respectively. "totaldata" is the total amount of data accessed in the experiment (the amount accessed per process, for "contig" IOR option -b, will therefore be totaldata/procs). "ost_number" is the number of BeeGFS OSTs used. That was configured on a per-directory basis by the system administrators. Multiple repetitions of each configuration were executed, the "repetition" column's only use is to differentiate between them. However, they were executed in random order (so the actual number in "repetition" means nothing). "time" is reported in seconds and corresponds to the total time (including open and close) reported by IOR. [1] Alexis Bandet, Francieli Boito, Guillaume Pallez. Scheduling distributed I/O resources in HPC systems. pre-print, 2024. https://inria.hal.science/hal-04394004 [2] Francieli Boito, Guillaume Pallez, Luan Teylo. The role of storage target allocation in applications' I/O performance with BeeGFS. CLUSTER 2022 - IEEE International Conference on Cluster Computing, Sep 2022, Heidelberg, Germany. https://inria.hal.science/hal-03753813 [3] Francieli Boito, Luan Teylo, Mihail Popov, Laora Aimi, Alexis Bandet, Laércio Lima Pilla, Guillaume Pallez, TOTO : Transparent I/O Tuning for HPC Applications, ACM International Conference on Supercomputing (ICS), July 2026, Belfast, UK.

本数据集包含使用IOR基准测试工具,在采用不同策略、使用不同数量对象存储目标(Object Storage Target,OST)时,向BeeGFS并行文件系统写入数据的性能测试结果。本数据集被用于文献[1]中报道的实验。 注:本仓库的第二版于2026年4月上传,包含了名为"part2"的数据文件,该文件扩展了原数据集,新增了更多访问模式。完整数据集(包含两部分)被用于文献[3]中的实验。补充说明:第一部分数据集于2024年1月发布。 所有实验均在PlaFRIM平台(https://www.plafrim.fr/)上开展:第一部分实验于2023年6月至12月间完成,第二部分实验于2024年至2025年间完成,均使用Bora集群。该I/O基础设施的详细说明可参见文献[2](本次实验使用了其中的高速网络)。 本次实验使用的IOR版本为4.1.0+dev,采用POSIX API(命令行参数:-a)。 本数据集为文本格式的.csv文件,各关键列含义如下: - "nodes":计算节点的数量;"procs":总进程数,"procs/nodes"即为每个节点上的进程数。 - "filestrategy":文件访问策略,可选值为"shared-file"(单文件被所有进程共同访问)或"file-per-proc"(每个进程拥有独立文件,通过IOR的-F参数创建)。 - "spatiality":访问空间模式,可选值包括"contig"(每个进程访问文件的连续区域)、"strided"(一维跨步访问模式,通过IOR的-s参数实现,仅适用于"shared-file"场景)或"random"(随机访问模式,通过IOR的-z参数实现)。 - "reqsize":单次I/O请求的大小(对应IOR的-t参数),其中K和M分别代表KiB和MiB。 - "totaldata":本次实验访问的总数据量(对于"contig"模式,通过IOR的-b参数设置的单进程访问量为"totaldata/procs")。 - "ost_number":使用的BeeGFS OST数量,由系统管理员基于目录维度进行配置。 所有实验配置均进行了多次重复测试,"repetition"列仅用于区分不同的重复实验,且实验执行顺序为随机(因此"repetition"列的数值本身无实际意义)。 - "time":以秒为单位的总耗时(包含文件打开和关闭阶段),由IOR工具上报。 [1] Alexis Bandet, Francieli Boito, Guillaume Pallez. 高性能计算系统中的分布式I/O资源调度. 预印本, 2024. https://inria.hal.science/hal-04394004 [2] Francieli Boito, Guillaume Pallez, Luan Teylo. BeeGFS环境下存储目标分配对应用程序I/O性能的影响. 2022年IEEE集群计算国际会议(CLUSTER 2022), 2022年9月, 德国海德堡. https://inria.hal.science/hal-03753813 [3] Francieli Boito, Luan Teylo, Mihail Popov, Laora Aimi, Alexis Bandet, Laércio Lima Pilla, Guillaume Pallez. TOTO:面向高性能计算应用的透明I/O调优方案. 2026年ACM国际超级计算大会(ICS), 2026年7月, 英国贝尔法斯特.

提供机构:
Zenodo
创建时间:
2024-01-16
二维码
社区交流群
二维码
科研交流群
商业服务