Benchmarking data on worker reactions to triggering events

1. Real-world Benchmarking Data The objective of this task was to determine if Virtual Reality-based captured behavioral data on responses to notifications are similar to what is expected in real-world settings. For this purpose, a real-world bench mark experiment was designed to capture participant response times to wearable watch alarms triggered upon simulated traffic near the mobile work zone on the experiment site in an urban setting. The proposed scope of data collection of the real-world study included the external environmental factors (e.g., site accessibility, weather). The key parameters of research are defined as reaction time to received alarms and the heart rate measures. Table 1 provides the list of parameters that were controlled and measured during the experiments. Table 1. Key parameters measured and tracked during real-world experiments Variable name Descriptions Key parameters captured Reaction time The time that one takes from getting the haptic or sound alarm from a wearable alarm device, herein referring to the apple watch, to the point when the participant gives a response by stopping the alarm by pressing on the screen of the smartwatch Inter-beat interval (IBI, heart rate) The time interval between individual beats of the heart; the data is measured by using E4 application provided by Empatica External factors tracked Ambient noise The level of ambient noise in the area is a factor potentially influencing participants’ reactions and is considered in the experiment design Temperature Daytime temperature recorded at each experiment Number of pedestrians on site Number of participants counted during the time of the experiment to record on the varying factors in the external environment in real-world settings In the experiment, each participant was asked to participate in the experiment three times. In each trial, data was recorded separately for each alarm sent to smartwatch from the administrator at triggering events (precisely, every time the remote-controlled toy car reaches the line 30 ft apart from the designated work area). Each alarm signal at each trial was recorded for all 31 participants to the experiment. Timestamps are automatically recorded in server in the events recorded in the format of Table 2: Table 2. Format of raw data stored in the server, starting in December 2022. Timestamp From Event 0 2022-12-08 13:37:53.101391 VR Received car approaching alert, mode=3, id=1000 1 2022-12-08 15:53:05.098288 Watch Start Simulation 2 2022-12-08 15:53:07.437488 VR Received car approaching alert, mode=4, id=1004 3 2022-12-08 15:53:13.064067 Watch Stop Simulation 4 2022-12-08 15:53:13.163635 Watch Stop Simulation ... 2417 2023-03-03 16:17:46.166644 Watch 1398 2418 2023-03-03 16:18:00.004425 Watch 1398 2419 2023-03-03 16:18.01.272071 Watch 1398 2420 2023-03-03 16:18:07.359187 Watch Stop Simulation 2421 2023-03-03 16:18:07.388183 Watch Stop Simulation Some intervals used different timestamps as benchmarks to calibrate on the vehicle speed and user response time to the alarm signals, which include the following cases: 1) At the beginning of each trial, vehicle travels 70 ft from start point to the 30 ft apart point, when the first alarm is signaled; given this travel distance, the travel time of the first trip the toy vehicle makes is calculated by subtracting tn_alarm1_sent from tn_start. 2) Similarly, user response times to all alarms are recorded by subtracting the timestamps when the alarm is received by participant from when the alarm is sent from the server. (tn_alarmn_sent - tn_alarmn_received) 2. Supplementary Data Ambient noise level data were collected using a noise meter, allowing to save noise level by seconds to multiple seconds (i.e., 5, 10, 30, 60 seconds). All noise data recorded were recorded in the interval of one second using the meter. The collected data was processed to match the certain timestamps collected for user response time data collected in the experiment to allow comparisons and correlation analysis to be performed later on, which include the following: 1) worker response; 2) sending of alarm signals; 3) start and stop of experiments. All data points were later modified using the rolling mean function of pandas python module to replace the missing data points by moving average method.