UR-MAT: A Multimodal, Material-Aware Synthetic Dataset of Urban Scenarios

🏙️ URMAT: URban MATerials Dataset URMAT (Urban Materials Dataset) is a large-scale, multimodal synthetic dataset designed for training and benchmarking material-aware semantic segmentation, scene understanding, and electromagnetic wave simulation tasks in complex urban environments. The dataset provides pixel-wise annotated images, depth maps, segmentation masks, physical material metadata, and aligned 3D point clouds, all derived from realistic 3D reconstructions of urban scenes including Trastevere, CityLife, Louvre, Canary Wharf, Bryggen, Siemensstadt, and Eixample. 🧱 Key Features 14 material classes: Brick, Glass, Steel, Tiles, Limestone, Plaster, Concrete, Wood, Cobblestone, Slate, Asphalt, Plastic, Gravel, Unknown. Multimodal data: RGB, depth, material masks, mesh segmentation Physically annotated metadata: includes permittivity, reflectance, attenuation 8 diverse European city districts, georeferenced and stylistically accurate Precomputed point clouds for 3D analysis or downstream simulation Compatible with Unreal Engine, PyTorch, and MATLAB pipelines 📁 Dataset Structure At the root of the dataset: *_mapping/ folders: mapping files, mesh metadata, camera poses *_pointclouds/ folders: colored 3D point clouds with material labels train/, val/, test/: standard splits for training and evaluation Inside each split (train/, val/, test/): Folder Name Description rgb/ RGB images rendered from Unreal Engine depth_png/ Depth maps as grayscale .png (normalized for visualization) depth_npy/ Raw depth arrays saved as .npy segmentation_material_png/ Color-encoded material segmentation masks for visualization segmentation_material_npy/ Material masks in .npy format (integer IDs per pixel, for training) segmentation_mesh/ Optional masks identifying the mesh origin of each pixel metadata/ JSON metadata with material type and physical properties per mesh 📦 Recommended Use Cases Material-aware semantic segmentation Scene-level reasoning for 3D reconstruction Ray tracing and wireless signal propagation simulation Urban AI and Smart City research Synthetic-to-real generalization studies 📜 Citation If you use URMAT v2 in your research, please cite the paper: UR-MAT: A Multimodal, Material-Aware Synthetic Dataset of Urban Scenarios@inproceedings{10.1145/3746027.3758314,author = {Russo, Debora and Mazzocca, Nicola and Vittorini, Valeria},title = {UR-MAT: A Multimodal, Material-Aware Synthetic Dataset of Urban Scenarios},year = {2025},isbn = {9798400720352},publisher = {Association for Computing Machinery},address = {New York, NY, USA},url = {https://doi.org/10.1145/3746027.3758314},doi = {10.1145/3746027.3758314},pages = {13164–13169},numpages = {6},location = {Dublin, Ireland},series = {MM '25}} Debora Russo, Nicola Mazzocca, and Valeria Vittorini. 2025. UR-MAT: A Multimodal, Material-Aware Synthetic Dataset of Urban Scenarios. In Proceedings of the 33rd ACM International Conference on Multimedia (MM '25). Association for Computing Machinery, New York, NY, USA, 13164–13169. https://doi.org/10.1145/3746027.3758314