Raw-Data for ConfLab: A Rich Multimodal Multisensor Dataset of Free-Standing Social Interactions In-the-Wild

This file contains raw data for cameras and wearables of the ConfLab dataset. <br> <strong>./cameras </strong> contains the overhead video recordings for 9 cameras (cam2-10) in MP4 files. These cameras cover the whole interaction floor, with camera 2 capturing the bottom of the scene layout, and camera 10 capturing top of the scene layout. Note that cam5 ran out of battery before the other cameras and thus the recordings are cut short. However, cam4 and 6 contain significant overlap with cam 5, to reconstruct any information needed. <br> Note that the annotations are made and provided in 2 minute segments. The annotated portions of the video include the last 3min38sec of x2xxx.MP4 video files, and the first 12 min of x3xxx.MP4 files for cameras (2,4,6,8,10), with "x" being the placeholder character in the mp4 file names. If one wishes to separate the video into 2 min segments as we did, the "video-splitting.sh" script is provided. <br> ./camera-calibration contains the camera instrinsic files obtained from https://github.com/idiap/multicamera-calibration. Camera extrinsic parameters can be calculated using the existing intrinsic parameters and the instructions in the multicamera-calibration repo. The coordinates in the image are provided by the crosses marked on the floor, which are visible in the video recordings. The crosses are 1m apart (=100cm). <br> <strong>./wearables</strong> subdirectory includes the IMU, proximity and audio data from each participant at the Conflab event (48 in total). In the directory numbered by participant ID, the following data are included: 1. raw audio file 2. proximity (bluetooth) pings (RSSI) file (raw and csv) and a visualization 3. Tri-axial accelerometer data (raw and csv) and a visualization 4. Tri-axial gyroscope data (raw and csv) and a visualization 5. Tri-axial magnetometer data (raw and csv) and a visualization 6. Game rotation vector (raw and csv), recorded in quaternions. <br> All files are timestamped. The sampling frequencies are: - audio: 1250 Hz - rest: around 50Hz. However, the sample rate is not fixed and instead the timestamps should be used. <br> For rotation, the game rotation vector's output frequency is limited by the actual sampling frequency of the magnetometer. For more information, please refer to https://invensense.tdk.com/wp-content/uploads/2016/06/DS-000189-ICM-20948-v1.3.pdf <br> Audio files in this folder are in raw binary form. The following can be used to convert them to WAV files (1250Hz): <br> ffmpeg -f s16le -ar 1250 -ac 1 -i /path/to/audio/file <br> <strong>Synchronization of cameras and werables data</strong> Raw videos contain timecode information which matches the timestamps of the data in the "wearables" folder. The starting timecode of a video can be read as: ffprobe -hide_banner -show_streams -i /path/to/video <br> <strong>./audio</strong> ./sync: contains wav files per each subject ./sync_files: auxiliary csv files used to sync the audio. Can be used to improve the synchronization. The code used for syncing the audio can be found here: https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/audio

本文件涵盖ConfLab数据集（ConfLab Dataset）中摄像设备与可穿戴设备的原始数据。<br><strong>./cameras</strong> 目录内存储了9台摄像机（cam2至cam10）的顶视录像，均为MP4格式文件。所有摄像机覆盖完整的交互场地：cam2拍摄场景布局的底部区域，cam10则拍摄场景顶部区域。需注意，cam5较其余摄像机提前耗尽电量，因此其录像存在提前中断的情况。不过cam4与cam6与cam5存在大量重叠区域，可用于补全所需的任何信息。<br>需说明，标注内容以2分钟片段的形式制作并提供。针对cam2、4、6、8、10这五台摄像机，其标注片段分别为x2xxx.MP4文件的最后3分38秒，以及x3xxx.MP4文件的前12分钟，其中“x”为MP4文件名中的占位符。若需如本数据集所用方式将视频切割为2分钟片段，可使用提供的`video-splitting.sh`脚本。<br><strong>./camera-calibration</strong> 目录存储了从https://github.com/idiap/multicamera-calibration 获取的摄像机内参文件。可结合已获取的内参文件与该多摄像机标定仓库中的说明，计算得到摄像机外参。图像中的坐标由场地地面上的标记十字提供，这些十字在录像中清晰可见，且相邻十字间距为1米（即100厘米）。<br><strong>./wearables</strong> 子目录包含了Conflab活动中总计48名参与者的惯性测量单元（IMU, Inertial Measurement Unit）、蓝牙邻近信号与音频数据。在以参与者ID命名的子目录中，包含以下数据：1. 原始音频文件；2. 邻近（蓝牙）信号Ping（接收信号强度指示RSSI）文件（含原始数据与CSV格式文件）及可视化结果；3. 三轴加速度计数据（含原始数据与CSV格式文件）及可视化结果；4. 三轴陀螺仪数据（含原始数据与CSV格式文件）及可视化结果；5. 三轴磁力计数据（含原始数据与CSV格式文件）及可视化结果；6. 以四元数形式记录的游戏旋转向量数据（含原始数据与CSV格式文件）。<br>所有文件均带有时间戳。采样频率如下：- 音频：1250 Hz；- 其余类型数据：约50 Hz。需注意，采样率并非固定值，因此应以文件自带的时间戳为准。<br>针对旋转相关数据，游戏旋转向量的输出频率受磁力计实际采样频率限制。如需更多信息，可参考文档：https://invensense.tdk.com/wp-content/uploads/2016/06/DS-000189-ICM-20948-v1.3.pdf<br>本目录下的音频文件均为原始二进制格式，可通过以下命令将其转换为1250Hz采样率的WAV文件：`ffmpeg -f s16le -ar 1250 -ac 1 -i /path/to/audio/file`<br><strong>摄像设备与可穿戴设备数据同步</strong> 原始录像中包含时间码信息，可与“wearables”目录内数据的时间戳匹配。可通过以下命令读取视频的起始时间码：`ffprobe -hide_banner -show_streams -i /path/to/video`<br><strong>./audio</strong> 目录下包含：<br>./sync：存储每名受试者的WAV格式音频文件；<br>./sync_files：用于音频同步的辅助CSV格式文件，可用于优化同步效果。音频同步所用代码可从以下仓库获取：https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/audio