MVSEC 多车立体声事件摄像机数据集

多车立体声事件摄像机数据集是为针对基于事件的摄像机开发新颖的3D感知算法而设计的数据集合。立体事件数据是从汽车，摩托车，六翼飞机和手持数据中收集的，并与激光雷达，IMU，运动捕捉和GPS融合在一起，以提供地面真实姿态和深度图像。此外，我们还提供了基于标准立体框架相机对的图像，可与传统技术进行比较。 基于事件的摄像机是一种新的异步感应方式，可以测量图像强度的变化。当像素上的对数强度更改为设置的阈值以上时，相机会立即返回更改的像素位置，以及具有微秒精度的时间戳和更改的方向（向上或向下）。这允许以极低的等待时间进行感测。此外，这些相机具有极高的动态范围和低功耗。 在白天和黑夜的环境下，从室内和室外环境中的四种不同车辆收集数据。所有六轴飞行器序列均具有来自室内Vicon区域和室外Qualisys区域的运动捕获地面真相，而其他序列具有通过将激光雷达信息与IMU和GPS融合而生成的地面真相。序列的完整列表可以在下面找到： 大量不同的传感器和模式，并牢固地安装在一对立体事件摄像机上，以生成准确的地面真实信息，并为研究模式之间的传感器融合提供途径。 对于事件，将两个实验DAVIS 346B摄像机以立体声（X轴对齐）配置安装。每个相机的分辨率均为346x260像素，配备4mm镜头和大约70度的垂直视场。摄像机时钟由左摄像机生成的硬件触发器同步，并发送到右摄像机。除事件外，摄像机还分别生成IMU和基于帧的图像测量值。 此外，DAVIS摄像机还安装了带IMU（VI传感器）的Velodyne激光雷达和立体框摄像机。如果可用，还可以使用室内（Vicon，左）或室外（Qualisys，右）运动捕获系统捕获地面真相姿势。 完整的传感器特性可以在下面找到： 对于大多数序列，机载传感器的融合可提供准确的姿态和深度。 Please cite the following papers when using this work in an academic publication: For the main dataset, please cite: Zhu, A. Z., Thakur, D., Ozaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018). The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception. IEEE Robotics and Automation Letters, 3(3), 2032-2039. An arXiv preprint is also available: Zhu, A. Z., Thakur, D., Ozaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018). The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception. arXiv preprint arXiv:1801.10202. For the ground truth optical flow, please cite: Zhu, A. Z., Yuan, L., Chaney, K., Daniilidis, K. (2018). EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras Robotics: Science and Systems 2018. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

The Multi-Vehicle Stereo Event Camera Dataset is a collection designed for developing novel 3D perception algorithms for event-based cameras.
Stereo event data is collected from cars, motorcycles, hexacopters, and handheld setups, and fused with LiDAR, IMU, motion capture, and GPS to provide ground-truth poses and depth images. Additionally, images from a standard stereo frame camera pair are provided for comparison with traditional vision techniques.
Event-based cameras are a novel asynchronous sensing modality that measures changes in image intensity. When the logarithmic intensity at a pixel exceeds a set threshold, the camera immediately returns the changed pixel's position, a timestamp with microsecond precision, and the direction of the change (increase or decrease). This enables sensing with extremely low latency. Furthermore, these cameras feature an extremely high dynamic range and low power consumption.
Data is collected in both daytime and nighttime environments across indoor and outdoor settings from four different vehicle types. All hexacopter sequences have motion capture ground truth from indoor Vicon areas and outdoor Qualisys areas, while other sequences have ground truth generated by fusing LiDAR data with IMU and GPS. The full list of sequences can be found below:
A wide range of diverse sensors and modalities are rigidly mounted on a pair of stereo event cameras to generate accurate ground-truth information and provide a platform for researching sensor fusion across modalities.
For event data collection, two prototype DAVIS 346B cameras are configured in a stereo setup (X-axis aligned). Each camera has a resolution of 346×260 pixels, is equipped with a 4mm lens, and has a vertical field of view of approximately 70 degrees. The camera clocks are synchronized via a hardware trigger generated by the left camera and sent to the right camera. In addition to event data, the cameras also generate IMU and frame-based image measurements separately.
Additionally, a Velodyne LiDAR with an integrated IMU (VI sensor) and a stereo frame camera are mounted alongside the DAVIS cameras. Ground-truth poses can also be captured using indoor (Vicon, left) or outdoor (Qualisys, right) motion capture systems when available.
The full specifications of the sensors can be found below:
For most sequences, the fusion of on-board sensors provides accurate poses and depth information.
When using this work in an academic publication, please cite the following papers: For the main dataset, please cite: Zhu, A. Z., Thakur, D., Ozaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018). The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception. IEEE Robotics and Automation Letters, 3(3), 2032-2039. An arXiv preprint is also available: Zhu, A. Z., Thakur, D., Ozaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018). The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception. arXiv preprint arXiv:1801.10202. For the ground truth optical flow, please cite: Zhu, A. Z., Yuan, L., Chaney, K., Daniilidis, K. (2018). EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras Robotics: Science and Systems 2018. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

搜集汇总

背景与挑战

背景概述

MVSEC多车立体声事件摄像机数据集是一个专为基于事件的摄像机设计的3D感知算法开发数据集，包含从多种车辆收集的立体事件数据，并与多种传感器数据融合以提供地面真实信息。数据集还提供了标准立体框架相机对的图像，支持与传统技术的比较研究。