CVPR2023-3D-Occupancy

世界上第一个用于自动驾驶场景感知的 3D 占用基准。 了解包括背景材料和前景物体在内的 3D 环境对于自动驾驶非常重要。 在传统的 3D 物体检测任务中，前景物体由 3D 边界框表示。 但是物体的几何形状比较复杂，不能用简单的3D盒子来表示，并且缺乏对背景的感知。 此任务的目标是预测场景的 3D 占用。 在此任务中，我们提供了一个基于 nuScenes 数据集的大规模占用基准。 基准是 3D 空间的体素化表示，并且在此任务中联合估计 3D 空间中体素的占用状态和语义。 该任务的复杂性在于在给定环视图像的情况下对 3D 空间进行密集预测。 给定来自多个摄像机的图像，目标是预测场景中每个体素网格的当前占用状态和语义。 体素状态被预测为空闲或占用。 如果体素被占用，则还需要预测其语义类别。 此外，我们还为每一帧提供了一个二进制观察/未观察掩码。 观察到的体素定义为当前相机观察中不可见的网格，在评估阶段将其忽略。

This is the world's first 3D occupancy benchmark for autonomous driving scene perception. Understanding the 3D environment, including both background elements and foreground objects, is critical for autonomous driving. In traditional 3D object detection tasks, foreground objects are represented by 3D bounding boxes. However, complex object geometries cannot be accurately represented by simple 3D boxes, and traditional methods also lack the ability to perceive the background environment. The goal of this task is to predict the 3D occupancy of the scene. For this task, we introduce a large-scale occupancy benchmark built upon the nuScenes dataset. The benchmark is a voxelized representation of 3D space, and the task jointly estimates the occupancy status and semantic category of each voxel in the 3D space. The complexity of this task lies in performing dense prediction of the 3D space given surround-view camera images. Given images from multiple cameras, the objective is to predict the current occupancy status and semantic category for every voxel grid in the scene. The voxel state is predicted as either free or occupied. If a voxel is occupied, its semantic category must also be predicted. Additionally, a binary observed/unobserved mask is provided for each frame. Observed voxels are defined as grid cells that are not visible in the current camera views, and these voxels will be ignored during the evaluation phase.

搜集汇总

背景与挑战

背景概述

CVPR2023-3D-Occupancy是首个自动驾驶场景感知的3D占用基准数据集，基于nuScenes数据集构建，专注于通过多摄像机图像预测3D空间中体素的占用状态和语义类别。该数据集提供了体素化的3D空间表示和观察掩码，适用于自动驾驶中的密集3D环境感知任务。