NTU Pedestrian Dataset

**************** NTU Pedestrian Dataset *******************
Attached files contain our data collected inside Nanyang Technological University Campus for pedestrian intention prediction. The dataset is particularly designed to capture spontaneous vehicle influences on pedestrian crossing/not-crossing intention.
We utilize this dataset in our journal paper "Context Model for Pedestrian Intention Prediction using Factored Latent-Dynamic Conditional Random Fields" accepted by IEEE Transactions on Intelligent Transportation Systems.
The dataset consists of 35 crossing and 35 stopping* (not-crossing) scenarios. The image sequences are in 'Image_sequences' folder.
'stopping_instants.csv' and 'crossing_instants.csv' files provide the stopping and crossing instants respectively, utilized for labeling the data and providing ground-truth for evaluation. Camera1 and Camera2 images are synchronized. Two cameras were used to capture the whole scene of interest.
We provide pedestrian and vehicle bounding boxes obtained from [1]. The occlusions and mis-detections are linearly interpolated. All necessary detections are stored in 'Object_detector_pedestrians_vehicles' folder. Each column within the csv files ('car_bndbox_..') corresponds to a unique tracked car within each image sequence. Each of the pedestrian csv files ('ped_bndbox_..') contains only one column, as we consider each pedestrian in the scene separately.
Additional details:* [xmin xmax ymin ymax] = [left right top down] (for the bounding boxes)* Dataset frequency: 15 fps.* Camera parameters (in pixels): f = 1135, principal point = (960, 540).

Additionally, we provide semantic segmentation output [2] and our depth parameters. As the data were collected in two phases, there are two files in each folder, highlighting the sequences in each phase.
Crossing sequences 1-28 and stopping sequences 1-24 were collected in Phase 1, while crossing sequences 29-35 and stopping sequences 25-35 were collected in Phase 2.
We obtained the optical flow from [3]. Our model (FLDCRF) codes are available here: https://github.com/satyajitneogiju/FLDCRF-for-sequence-labeling

If you use our dataset in your research, please cite our paper(s):1. S. Neogi, M. Hoy, K. Dang, H. Yu, J. Dauwels, "Context Model for Pedestrian Intention Prediction using Factored Latent-Dynamic Conditional Random Fields". Accepted by IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2019.

2. "S. Neogi, M. Hoy, W. Chaoqun, J. Dauwels, 'Context Based Pedestrian Intention Prediction Using Factored Latent Dynamic Conditional Random Fields', IEEE SSCI-2017."

Please email us if you have any questions:
1. Satyajit Neogi, PhD Student, Nanyang Technological University @ [e-mail]2. Justin Dauwels, Associate Professor, Nanyang Technological University @ [e-mail]

Our other group members include:
3. Dr. Michael Hoy, @ [e-mail]4. Dr. Kang Dang, @ [e-mail]5. Ms. Lakshmi Prasanna Kachireddy,6. Mr. Mok Bo Chuan Lance, and7. Mr. Xu Yan

References:
1. S. Ren, K. He, R. Girshick, J. Sun, ``Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NIPS 2015.2. A. Kendall, V. Badrinarayanan, R. Cipolla, ``Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding", BMVC 2017.3. C. Liu. ``Beyond Pixels: Exploring New Representations and Applications for Motion Analysis". Doctoral Thesis. Massachusetts Institute of Technology. May 2009.

* Please note, we had to remove sequence Stopping-33 for privacy reasons.
FUNDINGSTE-NTU NRF corporate lab@university scheme