A Dataset of Synthetic Images of Outdoor Scenes Taken from Sidewalks, for Temporal Semantic Segmentation Applications

This dataset has been generated using the CARLA simulator (release 0.9.11), an open-source 3D simulator for experiments in autonomous vehicle, based on the Unreal Engine game engine. It comes with pre-made city environment maps. CARLA is distributed with several integrated maps as well as parameters to increase the variety in the dataset. In the release that we have used, there are 13 semantic segmentation classes: None, Building, Fence, Other, Pedestrian, Pole, Lane-marking, Road, Sidewalk, Vegetation, Vehicle, Wall, and Traffic sign. The "None" category corresponds to textures that are not part of an object, such as lawns which are not part of "Vegetation", or sky. In the “Other” category are found objects that are not included in the other classes like plant and flower pots. For smart mobility applications, the “Sidewalks” and “Road” classes are of particular importance to find the way forward, as well as “Buildings” and “Poles” for obstacle avoidance. Sequences are made of 4 images. The dataset is composed of 46436 frames (11609 sequences) partitioned in 41024 frames (10256 sequences) for train, 2696 frames (674 sequences) for validation, and 2716 for test (679 sequences). The size of the images is 800 x 600 (resp. width x height). Additionaly, we have generated another smaller dataset with images taken from 2 different viewpoints: one located on the road and the other located on the sidewalk. The number of frames for train/validation/test is respectively 7288 (1822 sequences) partitioned in 6344 (1687 sequences) for train, 416 frames (104 sequences) for validation, and 424 for test (106 sequences). This smaller dataset is aimed at showing the importance of the viewpoint in the result of semantic segmentation. This can be done by cross-validation: learning on images taken from a viewpoint located on the road and test on images with a viewpoint located on the sidewalk, and vice versa.本数据集基于CARLA仿真器（CARLA simulator，版本0.9.11）生成，该仿真器是依托虚幻引擎（Unreal Engine）打造的开源自动驾驶三维实验仿真平台，内置多款预设城市环境地图。官方分发的CARLA版本附带多款集成地图与可调参数，可有效提升数据集的多样性。 本次使用的CARLA版本共包含13个语义分割类别：无（None）、建筑物（Building）、围栏（Fence）、其他（Other）、行人（Pedestrian）、杆状物（Pole）、车道标线（Lane-marking）、道路（Road）、人行道（Sidewalk）、植被（Vegetation）、车辆（Vehicle）、墙体（Wall）以及交通标志（Traffic sign）。其中“无”类别对应不属于任何物体的纹理，例如未归类为“植被”的草坪，或是天空；“其他”类别则涵盖未被其他类别纳入的物体，例如盆栽植物与花卉。 针对智能出行应用场景，“人行道”与“道路”类别对规划通行路径至关重要，而“建筑物”与“杆状物”则可用于避障规划。 数据样本以序列为单位，每个序列包含4张图像。主数据集共计46436帧（对应11609个序列），具体划分为：训练集41024帧（10256个序列）、验证集2696帧（674个序列）、测试集2716帧（679个序列）。图像分辨率统一为800×600（宽×高）。 此外，我们还构建了一个规模更小的辅助数据集，其图像采集自两种不同视角：一种设于道路上方，另一种设于人行道上方。该小型数据集总帧数为7288帧（对应1822个序列），划分比例为：训练集6344帧（1687个序列）、验证集416帧（104个序列）、测试集424帧（106个序列）。 该小型数据集旨在验证视角差异对语义分割任务结果的影响，可通过交叉验证的方式开展实验：例如在道路视角采集的图像上训练模型，再在人行道视角的图像上进行测试，反之亦然。