Event-based vision datasets for pedestrian detection in urban scenarios

Datasets supporting the Spiking Perception and processing for Intelligent Detection of pEdestrians on urban Roads (SPIDER) Project. The codebase is available on Gihub at https://github.com/th-nuernberg/spider.In Project SPIDER, we propose a novel solution for urban road surveillance using event-based neuromorphic cameras (i.e. Dynamic Vision Sensors – DVS), neural algorithms, and embedded neuromorphic computing platforms (i.e. Brainchip Akida). The solution is described by rapid detection and identification of abnormal activities, typically describing roadside pedestrian and bicyclist dynamics.In order to train the detection system, we decided to use DVS camera events. To also obtain ground truth information we added a traditional CMOS camera on the mount with a preset offset capturing the same field of view. The CMOS frames were only used as support in the labeling process and were not included in the training of the system, the testing, or the evaluation, respectively. We have chosen two locations with different properties of the road, the number of pedestrians and bicyclists, and overall different traffic properties. 
The SPIDER project is an award-winning edge solution for the TinyML Vision Zero San Jose Hackathon. The project placed 2nd among 29 teams in the world in the final. The final pitch is available at https://www.youtube.com/watch?v=ZhBCtfalcOk&t=2872s

Dataset archive content:

Dataset location 1
• 4 lanes (4 per direction) wide street
• Location: https://goo.gl/maps/JaYGwaTaBHj5H6SL9
• 50 kmh (urban) speed limit
• Near the university campus with Pedestrians (people walking) and bicyclists (people biking, scooting, rolling, etc.)
• Ideal Operating Environment 
Dataset location 2
• 8 lanes (4 per direction) wide street
• Location: https://goo.gl/maps/jar6AjysZiM2LP5S7
• 50 kmh (urban) speed limit
• Near the main train stations of the city and a location with Pedestrians (people walking, running, or jogging), and cyclists (people biking, scooting, rolling, etc.) 
Dataset location 3
• 6 lanes (3 per direction) wide street on the bridge
• Location: https://goo.gl/maps/SEEsmpgmLPcD8fG7A
• 50 kmh (urban) speed limit
• Near ring street of Munich and a location with Pedestrians (people walking, running, or jogging) and bicyclists (people biking, scooting, rolling, etc.)
• Night-time data acquisition