Traffic Scenarios Event Caption (TSEC) Dataset

General video captioning datasets are not suitable for challenges of fast moving camera, highly overlapping vocabulary and caption for camera-carrier in driving scenes. To address this issue, we develop the Traffic Scenarios Event Caption (TSEC) Dataset to describe the key events of ego vehicle, road environment and other traffic participants. To acquire diverse types of traffic scenarios, we select the videos that we take with our on-board camera and other public dataset videos. We also download from BiliBili and Youtube to get the traffic accident videos. Because each video of these may contain many traffic events, we spilt the videos into distinct clips and make each sub-clip associated with 1 to 3 key events. We follow the rules when processing the video clips: 1) In order to increase the diversity, different weathers, different periods, different road conditions and different vehicle conditions need to be selected. 2) Due to the complexity of traffic situations at intersections, a number of intersection scenarios are chosen. 3) The clip needs to contain the entire event. Finally, we obtain 8,000 videos with 32,000 event caption.