ConVeX: A Multi-agent Context-aware Dataset for Collaborative Perception in Autonomous Driving

Collaborative perception, using vehicle-to-everything (V2X) communication to sharesensor data between connected vehicles and between the latter and the network infrastructure, hasemerged as a prominent solution to extend the view of single autonomous vehicles. The effectivenessof this paradigm, however, may be hindered by the presence of adverse weather conditionsand changes in lighting, often affecting real-world scenarios. Thus, assessing the robustness ofcollaborative perception to environmental contingencies is still an open issue. Importantly, althoughsome large-scale datasets for collaborative perception, comprising realistic and simulated data, arenow publicly available, most of them lack diversity in terms of environmental conditions in theautonomous driving scenarios they collect, making it difficult for researchers to assess how suchconditions may affect perception performance. We thus introduce ConVeX, an extensive multi-agentsynthetic dataset for collaborative perception that reproduces different realistic driving scenarios(urban, rural, highway), road layouts, and weather and lighting conditions. Remarkably, ConVexincludes multi-modal data (images from RGB cameras, LiDAR points, and GPS coordinates)collected by different vehicles, and includes ground-truth annotations for object detection.