Raw-Data for ConfLab: A Rich Multimodal Multisensor Dataset of Free-Standing Social Interactions In-the-Wild

This file contains raw data for cameras and wearables of the ConfLab dataset. <br> <strong>./cameras </strong> contains the overhead video recordings for 9 cameras (cam2-10) in MP4 files. These cameras cover the whole interaction floor, with camera 2 capturing the bottom of the scene layout, and camera 10 capturing top of the scene layout. Note that cam5 ran out of battery before the other cameras and thus the recordings are cut short. However, cam4 and 6 contain significant overlap with cam 5, to reconstruct any information needed. <br> Note that the annotations are made and provided in 2 minute segments. The annotated portions of the video include the last 3min38sec of x2xxx.MP4 video files, and the first 12 min of x3xxx.MP4 files for cameras (2,4,6,8,10), with "x" being the placeholder character in the mp4 file names. If one wishes to separate the video into 2 min segments as we did, the "video-splitting.sh" script is provided. <br> ./camera-calibration contains the camera instrinsic files obtained from https://github.com/idiap/multicamera-calibration. Camera extrinsic parameters can be calculated using the existing intrinsic parameters and the instructions in the multicamera-calibration repo. The coordinates in the image are provided by the crosses marked on the floor, which are visible in the video recordings. The crosses are 1m apart (=100cm). <br> <strong>./wearables</strong> subdirectory includes the IMU, proximity and audio data from each participant at the Conflab event (48 in total). In the directory numbered by participant ID, the following data are included: 1. raw audio file 2. proximity (bluetooth) pings (RSSI) file (raw and csv) and a visualization 3. Tri-axial accelerometer data (raw and csv) and a visualization 4. Tri-axial gyroscope data (raw and csv) and a visualization 5. Tri-axial magnetometer data (raw and csv) and a visualization 6. Game rotation vector (raw and csv), recorded in quaternions. <br> All files are timestamped. The sampling frequencies are: - audio: 1250 Hz - rest: around 50Hz. However, the sample rate is not fixed and instead the timestamps should be used. <br> For rotation, the game rotation vector's output frequency is limited by the actual sampling frequency of the magnetometer. For more information, please refer to https://invensense.tdk.com/wp-content/uploads/2016/06/DS-000189-ICM-20948-v1.3.pdf <br> Audio files in this folder are in raw binary form. The following can be used to convert them to WAV files (1250Hz): <br> ffmpeg -f s16le -ar 1250 -ac 1 -i /path/to/audio/file <br> <strong>Synchronization of cameras and werables data</strong> Raw videos contain timecode information which matches the timestamps of the data in the "wearables" folder. The starting timecode of a video can be read as: ffprobe -hide_banner -show_streams -i /path/to/video <br> <strong>./audio</strong> ./sync: contains wav files per each subject ./sync_files: auxiliary csv files used to sync the audio. Can be used to improve the synchronization. The code used for syncing the audio can be found here: https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/audio

本文件包含ConfLab数据集（ConfLab dataset）中摄像头与可穿戴设备的原始数据。<br><strong>./cameras</strong> 目录存放9台摄像头（cam2至cam10）的顶视录像，格式为MP4。该系列摄像头覆盖了整个交互场地，其中cam2拍摄场景布局的底部，cam10拍摄场景布局的顶部。需注意，cam5的电量早于其他摄像头耗尽，因此录制内容被提前截断，但cam4与cam6与cam5存在大量重叠区域，可用于重建所需的任何缺失信息。<br>需说明的是，标注内容以2分钟片段的形式制作并提供。对于摄像头2、4、6、8、10的录像，标注片段分别为x2xxx.MP4文件的最后3分38秒，以及x3xxx.MP4文件的前12分钟，其中“x”为MP4文件名中的占位符。若需按照我们的方式将视频拆分为2分钟片段，可使用提供的“video-splitting.sh”脚本。<br><strong>./camera-calibration</strong> 目录存放从https://github.com/idiap/multicamera-calibration 获取的摄像头内参文件。可利用已获取的内参文件，结合multicamera-calibration仓库中的说明计算摄像头外参。图像中的坐标由地面上的十字标记提供，该标记在录像中清晰可见，相邻十字间距为1米（即100厘米）。<br><strong>./wearables</strong> 子目录包含Conflab活动中48名参与者各自的惯性测量单元（IMU）、近距离通信及音频数据。在以参与者ID命名的目录中，包含以下数据：1. 原始音频文件；2. 近距离（蓝牙）探测信号（接收信号强度指示，RSSI）文件（原始格式与CSV格式）及可视化文件；3. 三轴加速度计数据（原始格式与CSV格式）及可视化文件；4. 三轴陀螺仪数据（原始格式与CSV格式）及可视化文件；5. 三轴磁力计数据（原始格式与CSV格式）及可视化文件；6. 游戏旋转矢量数据（原始格式与CSV格式），以四元数形式记录。<br>所有文件均带有时间戳。采样频率如下：- 音频：1250 Hz；- 其余数据：约50 Hz。但采样率并非固定值，因此应优先使用时间戳进行时序对齐。对于旋转矢量数据，游戏旋转矢量的输出频率受限于磁力计的实际采样频率。如需更多信息，请参考https://invensense.tdk.com/wp-content/uploads/2016/06/DS-000189-ICM-20948-v1.3.pdf。<br>本目录中的音频文件为原始二进制格式，可通过以下命令将其转换为1250Hz采样率的WAV文件：<br>ffmpeg -f s16le -ar 1250 -ac 1 -i /path/to/audio/file<br><strong>摄像头与可穿戴设备数据的同步</strong> 原始录像包含与“wearables”目录中数据时间戳匹配的时间码信息。可通过以下命令读取视频的起始时间码：<br>ffprobe -hide_banner -show_streams -i /path/to/video<br><strong>./audio</strong> ./sync：包含每位受试者的WAV格式音频文件。./sync_files：用于辅助音频同步的CSV文件，可用于优化同步效果。用于音频同步的代码可在以下仓库获取：https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/audio