Traffic Scenarios Event Caption (TSEC) Dataset

General video captioning datasets are not suitable for challenges of fast moving camera, highly overlapping vocabulary and caption for camera-carrier in driving scenes. To address this issue, we develop the Traffic Scenarios Event Caption (TSEC) Dataset to describe the key events of ego vehicle, road environment and other traffic participants. To acquire diverse types of traffic scenarios, we select the videos that we take with our on-board camera and other public dataset videos. We also download from BiliBili and Youtube to get the traffic accident videos. Because each video of these may contain many traffic events, we spilt the videos into distinct clips and make each sub-clip associated with 1 to 3 key events. We follow the rules when processing the video clips: 1) In order to increase the diversity, different weathers, different periods, different road conditions and different vehicle conditions need to be selected. 2) Due to the complexity of traffic situations at intersections, a number of intersection scenarios are chosen. 3) The clip needs to contain the entire event. Finally, we obtain 8,000 videos with 32,000 event caption.通用视频字幕数据集难以适配驾驶场景下的三类核心挑战：相机快速运动、词汇高度重叠，以及面向搭载相机的交通载体的字幕生成需求。为解决这一问题，我们构建了交通场景事件字幕（Traffic Scenarios Event Caption，简称TSEC）数据集，用于描述自车（ego vehicle）、道路环境及其他交通参与者的关键事件。为获取多样化的交通场景样本，我们采集了自研车载相机拍摄的视频，并整合了其他公开数据集的视频资源；同时从哔哩哔哩（BiliBili）与YouTube平台下载交通事故相关视频以扩充样本库。由于单条原始视频可能包含多个交通事件，我们将其切分为独立的子片段，并为每个子片段关联1至3个关键事件。在处理视频子片段时，我们遵循以下规则：1）为提升样本多样性，需覆盖不同天气、时段、道路状况与车辆工况；2）鉴于交叉口交通场景的复杂性，选取大量交叉口场景样本；3）子片段需完整包含对应交通事件。最终，我们共获得8000条视频及32000条事件字幕标注。

搜集汇总

背景与挑战

背景概述

TSEC数据集是一个专门为自动驾驶研究设计的交通场景事件描述数据集，包含8,000个视频和32,000个事件标注，涵盖多种天气、时间和道路条件。该数据集提供原始视频、语义分割视频和CSV标注文件，用于描述自车、道路环境和其他交通参与者的关键事件，旨在支持交通场景理解的研究。