VibV A fusion dataset based on vibration-vision for safety enhancement in self-driving tasks from another perspective

This data set is provided by the School of Vehicles and Transportation, Tsinghua University. The recorded lane line video and vibration signal simultaneously from 17:00 to 19:00 in scenes such as turning intersections and speed bumps for two months on roads near Shigezhuang Bridge East, Changping District, Beijing, China. The data set is divided into two parts: vibration data and vision data. Visual data is recorded in the form of video frames with a total size of 8.6G.<b>Vibration data.</b>The vibration data contains raw vibration data collected from 39 experiments. The data collected each time is stored in a subfolder, with a total of 39 subfolders. The subfolder’s name contains the information of the acquisition date and time. There are two files in .mat format in each sub-folder, which are the raw vibration data and the time series and variables related to the preliminary processing of the vibration signal.<b>Vision data.</b>The video frames are classified according to the annotation results. The first category is classified according to whether the wheels touch the lane line, and the second category is classified according to whether the collection time is before sunset or after sunset, that is, based on brightness. The two categories are stored in two folders respectively.The touch lane linecategory. This file contains three subfolders, subfolder "yes" contains the image data touched line, subfolder "no" contains the image data without touched line, and "list" cantains the annotation file. The "Yes" and "no" folders each contain 39 folders with the same name as the vibration data. Each folder contains at least 4 consecutive video frames to facilitate subsequent work using consecutive frames as input. Each folder contains image corresponding to vibration signals and their binarized image. The subfolder "list" contains three annotation files named "train_gt.txt", "val_gt.txt" and "test_gt.txt", which represent the test set, training set and validation set respectively. The annotation file includes the status of line touching, storage path, and name.The brightness category. This folder contains three subfolders. The sub-folder "before_sunset_18_49" and the sub-folder "after_sunset_18_49" respectively save the video frames from the left view, right view and front view before and after sunset according to the local sunset time of 18:49 as the dividing line. Manually segment and label the lane lines of each image, and then convert the image to binary. The subfolder "label" contains the final label file "label.csv". The first column is the image serial number, and the second to seventh columns are the image storage paths of the left, right and front views and the image paths after lane line annotation of the left, right and front views respectively. The eighth column is a 4-bit non-mutually exclusive one-hot code describing the label annotation form of the data set. Its meaning is described in detail in the "Method" section above. The ninth column is the collection time. Good light conditions before sunset are marked as 1, and poor light conditions after sunset are marked as 0.