DE, Extended Sensors, EDM-enabled extended sensors with surround view generation

Use Case Category: Extended Sensors
User Story: EDM-enabled extended sensors with surround view generation
Location: German (DE) trial site

According to 3GPP TS 22.186 R16, Extended Sensors “enable the exchange of raw or processed data gathered through local sensors or live video data among vehicles, RSUs, devices of pedestrians and V2X application servers. The vehicles can enhance the perception of their environment beyond what their own sensors can detect and have a more holistic view of the local situation”.

User Story: EDM-enabled extended sensors with surround view generation

The objective of this user story is to share LDM data and raw sensor data for real-time prediction and planning tasks made possible by 5G technology. More precisely, the use case deals with a situation when the perception obtained by the on-board sensors is not enough and needs to be enhanced by sensor data from other traffic participants.

The user story contains several connected vehicles equipped with sensors as well as roadside infrastructure comprising sensors and edge computing infrastructure (eRSU). The vehicles and the eRSU using their respective sensor data build their individual situational awareness, identifying objects, lane markings or the road condition to support their prediction and planning functions. However, each individual vehicle’s sensors as well as roadside sensors are limited in the perception in different ways. The sensors view could be obstructed by objects, limited by weather conditions, or not covering a specific area. To mitigate the lack of environment information, vehicles share extracts ROIs (regions of interest) from their LDMs and/or sensor raw data and the eRSU shares its Edged Dynamic Map (EDM).

In the proposed setup, the eRSU assisted map update is valid within the coverage area of the eRSU. Cars not within the coverage area are relying on updates of their respective eRSU and their neighbouring vehicles. To assist cars moving from one coverage area to another, the future eRSU’s EDM will be provided. Neighbouring eRSUs exchange their EDMs to provide the vehicles with map information when approaching a new coverage area.

The storyline of the use case goes like this. There are two connected autonomous cars driving on the same lane. The two cars are sending relevant LDM data to the corresponding eRSU where the EDM is updated. Suddenly, there is an unexpected event that makes the first car brake and start a lane changing manoeuvre. The event can be for instance a vehicle that stops and blocks the lane. This sudden action is propagated to the rest of the cars that perceive that something is happening. The two connected and automated cars request the EDM to the eRSU under their coverage and they fuse it with their LDM to analyse the situation. They determine that a lane changing manoeuvre is necessary as the lane is blocked some meters ahead. Using the collected information, they start planning and executing the manoeuvre. The EDM contains only processed lightweight data of traffic participants (mainly position, heading, size and speed) that is sufficient for rapid risk estimation and decision making but it is not enough to create a 360º surround view. To favour a quick decision the eRSU provide to each vehicle a filtered EDM with the relevant items for the ROI of the vehicle, filtering any irrelevant data for the vehicle’s path. The rear vehicle has its field of view severely restricted and determines that a surround view generation would help keeping the driver in the loop and decreasing the risk of the lane changing manoeuvre. The leading vehicle has better visibility and the do not require a surround view. Consulting the EDM, the rear vehicle selects to which vehicles it needs to request raw sensor data to enhance its field of view. The vehicle generates a 360º surround view by fusing onboard sensors (cameras and Lidar) and data (video and Lidar’s 3D cloud) coming from the selected traffic participants. This is done by direct Vehicle to Vehicle (V2V) communication. According to the processing capacity of data source and destination and the network performance between V2V communication participants, the data origin vehicle generates data streams with an appropriate resolution and bitrate. Furthermore, the eRSU provides tokens to be used to perform secure data transfer between the vehicles.