NTU Pedestrian Dataset

**************** NTU Pedestrian Dataset *******************<br>Attached files contain our data collected inside Nanyang Technological University Campus for pedestrian intention prediction. The dataset is particularly designed to capture spontaneous vehicle influences on pedestrian crossing/not-crossing intention.<br>We utilize this dataset in our journal paper "Context Model for Pedestrian Intention Prediction using Factored Latent-Dynamic Conditional Random Fields" accepted by IEEE Transactions on Intelligent Transportation Systems.<br>The dataset consists of 35 crossing and 35 stopping* (not-crossing) scenarios. The image sequences are in 'Image_sequences' folder.<br>'stopping_instants.csv' and 'crossing_instants.csv' files provide the stopping and crossing instants respectively, utilized for labeling the data and providing ground-truth for evaluation. Camera1 and Camera2 images are synchronized. Two cameras were used to capture the whole scene of interest.<br>We provide pedestrian and vehicle bounding boxes obtained from [1]. The occlusions and mis-detections are linearly interpolated. All necessary detections are stored in 'Object_detector_pedestrians_vehicles' folder. Each column within the csv files ('car_bndbox_..') corresponds to a unique tracked car within each image sequence. Each of the pedestrian csv files ('ped_bndbox_..') contains only one column, as we consider each pedestrian in the scene separately.<br>Additional details:* [xmin xmax ymin ymax] = [left right top down] (for the bounding boxes)* Dataset frequency: 15 fps.* Camera parameters (in pixels): f = 1135, principal point = (960, 540).<br><br>Additionally, we provide semantic segmentation output [2] and our depth parameters. As the data were collected in two phases, there are two files in each folder, highlighting the sequences in each phase.<br>Crossing sequences 1-28 and stopping sequences 1-24 were collected in Phase 1, while crossing sequences 29-35 and stopping sequences 25-35 were collected in Phase 2.<br>We obtained the optical flow from [3]. Our model (FLDCRF) codes are available here: https://github.com/satyajitneogiju/FLDCRF-for-sequence-labeling<br><br>If you use our dataset in your research, please cite our paper(s):1. S. Neogi, M. Hoy, K. Dang, H. Yu, J. Dauwels, "Context Model for Pedestrian Intention Prediction using Factored Latent-Dynamic Conditional Random Fields". Accepted by IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2019.<br><br>2. "S. Neogi, M. Hoy, W. Chaoqun, J. Dauwels, 'Context Based Pedestrian Intention Prediction Using Factored Latent Dynamic Conditional Random Fields', IEEE SSCI-2017."<br><br>Please email us if you have any questions:<br>1. Satyajit Neogi, PhD Student, Nanyang Technological University @ [e-mail]2. Justin Dauwels, Associate Professor, Nanyang Technological University @ [e-mail]<br><br>Our other group members include:<br>3. Dr. Michael Hoy, @ [e-mail]4. Dr. Kang Dang, @ [e-mail]5. Ms. Lakshmi Prasanna Kachireddy,6. Mr. Mok Bo Chuan Lance, and7. Mr. Xu Yan <br><br>References:<br>1. S. Ren, K. He, R. Girshick, J. Sun, ``Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NIPS 2015.2. A. Kendall, V. Badrinarayanan, R. Cipolla, ``Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding", BMVC 2017.3. C. Liu. ``Beyond Pixels: Exploring New Representations and Applications for Motion Analysis". Doctoral Thesis. Massachusetts Institute of Technology. May 2009.<br><br><br>* Please note, we had to remove sequence Stopping-33 for privacy reasons.<br>FUNDINGSTE-NTU NRF corporate lab@university scheme**************** 南洋理工大学行人数据集（NTU Pedestrian Dataset）******************* 本附件包含我们在南洋理工大学校园内采集的行人意图预测数据集。该数据集专为捕捉自发行驶车辆对行人过街/不过街意图的影响而设计。 我们将该数据集应用于发表于《IEEE智能交通系统汇刊》（IEEE Transactions on Intelligent Transportation Systems）的期刊论文《基于因子分解隐态动态条件随机场的行人意图预测上下文模型》。 本数据集包含35组过街场景与35组停步（不过街）场景。图像序列存储于'Image_sequences'文件夹中。 'stopping_instants.csv'与'crossing_instants.csv'文件分别提供了停步时刻与过街时刻，用于数据标注与评估所需的真值标签。Camera1与Camera2采集的图像已完成同步，两台相机用于完整覆盖目标场景。 我们提供了从参考文献[1]中获取的行人和车辆边界框。对于遮挡与误检测情况，我们采用线性插值进行补全。所有必要的检测结果存储于'Object_detector_pedestrians_vehicles'文件夹中。各CSV文件（如'car_bndbox_..'）中的每一列对应一个图像序列中唯一的跟踪车辆。行人CSV文件（如'ped_bndbox_..'）仅包含一列，因为我们对场景中的每个行人单独进行处理。 补充说明： * 边界框标注格式为[xmin xmax ymin ymax] = [左 右 上 下] * 数据集帧率：15 fps（帧每秒） * 相机参数（像素单位）：焦距f=1135，主点坐标为(960, 540) 此外，我们还提供了参考文献[2]中的语义分割输出结果以及深度参数。由于数据分两个阶段采集，每个文件夹中包含两个文件，分别对应各阶段的序列。 第一阶段采集了过街序列1-28与停步序列1-24，第二阶段采集了过街序列29-35与停步序列25-35。 我们从参考文献[3]中获取了光流数据。我们的FLDCRF模型代码可通过以下链接获取：https://github.com/satyajitneogiju/FLDCRF-for-sequence-labeling 若您在研究中使用本数据集，请引用以下论文： 1. S. Neogi、M. Hoy、K. Dang、H. Yu、J. Dauwels，《基于因子分解隐态动态条件随机场的行人意图预测上下文模型》，发表于《IEEE智能交通系统汇刊》（IEEE Transactions on Intelligent Transportation Systems, T-ITS），2019年。 2. S. Neogi、M. Hoy、W. Chaoqun、J. Dauwels，《基于上下文的行人意图预测：采用因子分解隐态动态条件随机场》，IEEE SSCI-2017会议。 如有任何疑问，请通过以下方式联系我们： 1. 南洋理工大学博士生Satyajit Neogi，邮箱：[e-mail] 2. 南洋理工大学副教授Justin Dauwels，邮箱：[e-mail] 本研究团队其他成员包括： 3. Michael Hoy博士，邮箱：[e-mail] 4. Kang Dang博士，邮箱：[e-mail] 5. Lakshmi Prasanna Kachireddy女士 6. Mok Bo Chuan Lance先生 7. Xu Yan先生 参考文献： 1. S. Ren、K. He、R. Girshick、J. Sun，《Faster R-CNN：基于区域提议网络的实时目标检测》，NIPS 2015。 2. A. Kendall、V. Badrinarayanan、R. Cipolla，《贝叶斯SegNet：面向场景理解的深度卷积编码器-解码器架构中的模型不确定性》，BMVC 2017。 3. C. Liu，《超越像素：探索运动分析的新型表征与应用》，麻省理工学院博士论文，2009年5月。 * 请注意，出于隐私保护原因，我们移除了停步序列Stopping-33。 资助信息： TE-NTU 新加坡国家研究基金会（NRF）大学合作实验室计划