BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

数据集是推动计算机视觉发展的核心动力，但现有的驾驶场景数据集在视觉内容与支持自动驾驶多任务学习的任务类型上均较为匮乏。研究者通常只能在单个数据集上开展少量问题的研究，而真实世界的计算机视觉应用则需要处理不同复杂度的各类任务。本研究构建了BDD100K——目前规模最大的驾驶视频数据集，包含10万条视频与10项任务，用于评估自动驾驶场景下图像识别算法的前沿进展。该数据集具备地理、环境与天气维度的多样性，有助于训练能够更好适配未知场景的模型，降低模型在新环境下出现意外错误的概率。基于这一多样化数据集，本研究构建了面向异构多任务学习的基准测试平台，并探索如何协同完成多项任务。实验结果表明，现有模型若要完成此类异构任务，需要采用针对性的训练策略。BDD100K为该重要研究方向的后续工作打开了新的大门。