Participant Study Dataset in Virtual Reality Integrated CARLA Simulator

Ensuring the safety of autonomous vehicles in dynamic environments hinges on accurately modeling interactions with vulnerable road users (VRUs). Traditional testing methods often fall short in replicating realistic human behavior, while collecting real-world data in safety-critical situations poses significant ethical and practical challenges. To address this, we present a human-in-the-loop simulation framework that seamlessly integrates virtual reality (VR) with the CARLA driving simulator. This setup allows for the study of pedestrian-vehicle interactions within a controlled yet immersive environment. By leveraging VR-based motion tracking, our approach captures natural pedestrian movements and decision-making processes, thereby enhancing the realism of scenario-based testing. Using this VR-integrated CARLA simulator, we conducted a user study involving 17 participants equipped with Meta Quest Pro headsets. Participants were instructed to cross a virtual crosswalk in front of them within the CARLA Town01 environment, while an ego vehicle approached. We designed four distinct scenario variants by manipulating the ego vehicle’s speed and the pedestrian’s initial visibility of the vehicle: 1st Run: Pedestrian sees the car — ego vehicle approaches at 30 km/h 2nd Run: Pedestrian sees the car — ego vehicle approaches at 50 km/h 3rd Run: Pedestrian does not see the car — ego vehicle approaches at 30 km/h 4th Run: Pedestrian does not see the car — ego vehicle approaches at 50 km/h Each scenario could be repeated multiple times. Data collected for each run includes scene images from the ego vehicle’s perspective, alongside a CSV file containing: Timestamp (datetime) Pedestrian avatar’s position in CARLA Town01 (posx, posy, posz) Participant’s head rotation (pitch, yaw, roll) Metadata about the participants profile (sex, age, height, eye glasses)