SynthSoM: A synthetic intelligent multi-modal sensing-communication dataset for Synesthesia of Machines (SoM)

Given the importance of datasets for sensing-communication integration research, a novel simulation platform for constructing communication and multi-modal sensory dataset is developed. The developed platform integrates three high-precision software, i.e., AirSim, WaveFarer, and Wireless InSite, and further achieves in-depth integration and precise alignment of them. Based on the developed platform, a new synthetic intelligent multi-modal sensing-communication dataset for Synesthesia of Machines (SoM), named SynthSoM, is proposed. The SynthSoM dataset contains various air-ground multi-link cooperative scenarios with comprehensive conditions, including multiple weather conditions, times of the day, intelligent agent densities, frequency bands, and antenna types. The SynthSoM dataset encompasses multiple data modalities, including radio-frequency (RF) channel large-scale and small-scale fading data, RF millimeter wave (mmWave) radar sensory data, and non-RF sensory data, e.g., RGB images, depth maps, and light detection and ranging (LiDAR) point clouds. The quality of SynthSoM dataset is validated via statistics-based qualitative inspection and evaluation metrics through machine learning (ML) via real-world measurements. The SynthSoM dataset is open-sourced and provides consistent data for cross-comparing SoM-related algorithms.

鉴于数据集在感知与通信融合研究中的重要性，本研究开发了一款用于构建通信与多模态感知数据集的新型仿真平台。该平台集成了AirSim、WaveFarer与Wireless InSite三款高精度软件，并实现了三者的深度融合与精准对齐。
基于该平台，本研究提出了一款面向机器通感（Synesthesia of Machines, SoM）的新型合成智能多模态感知-通信数据集，命名为SynthSoM。SynthSoM数据集涵盖各类空地多链路协同场景，场景条件覆盖全面，包含多种天气状况、时段、智能体密度、频段与天线类型。
该数据集包含多种数据模态，具体涵盖射频（radio-frequency, RF）信道大尺度与小尺度衰落数据、射频毫米波（millimeter wave, mmWave）雷达感知数据，以及非射频感知数据，例如RGB图像、深度图与激光雷达（light detection and ranging, LiDAR）点云。
本研究通过基于统计的定性检验、基于机器学习（machine learning, ML）的评估指标，结合真实场景测量数据，验证了SynthSoM数据集的质量。
该数据集已开源，可为SoM相关算法的横向对比研究提供统一规范的数据支持。