Raw Data for ConfLab: A Data Collection Concept, Dataset, and Benchmark for Machine Analysis of Free-Standing Social Interactions in the Wild

This file contains raw data for cameras and wearables of the ConfLab dataset. <br> <strong>./cameras </strong> contains the overhead video recordings for 9 cameras (cam2-10) in MP4 files. These cameras cover the whole interaction floor, with camera 2 capturing the bottom of the scene layout, and camera 10 capturing top of the scene layout. Note that cam5 ran out of battery before the other cameras and thus the recordings are cut short. However, cam4 and 6 contain significant overlap with cam 5, to reconstruct any information needed. <br> Note that the annotations are made and provided in 2 minute segments. The annotated portions of the video include the last 3min38sec of x2xxx.MP4 video files, and the first 12 min of x3xxx.MP4 files for cameras (2,4,6,8,10), with "x" being the placeholder character in the mp4 file names. If one wishes to separate the video into 2 min segments as we did, the "video-splitting.sh" script is provided. <br> ./camera-calibration contains the camera instrinsic files obtained from https://github.com/idiap/multicamera-calibration. Camera extrinsic parameters can be calculated using the existing intrinsic parameters and the instructions in the multicamera-calibration repo. The coordinates in the image are provided by the crosses marked on the floor, which are visible in the video recordings. The crosses are 1m apart (=100cm). <br> <strong>./wearables</strong> subdirectory includes the IMU, proximity and audio data from each participant at the Conflab event (48 in total). In the directory numbered by participant ID, the following data are included: 1. raw audio file 2. proximity (bluetooth) pings (RSSI) file (raw and csv) and a visualization 3. Tri-axial accelerometer data (raw and csv) and a visualization 4. Tri-axial gyroscope data (raw and csv) and a visualization 5. Tri-axial magnetometer data (raw and csv) and a visualization 6. Game rotation vector (raw and csv), recorded in quaternions. <br> All files are timestamped. The sampling frequencies are: - audio: 1250 Hz - rest: around 50Hz. However, the sample rate is not fixed and instead the timestamps should be used. <br> For rotation, the game rotation vector's output frequency is limited by the actual sampling frequency of the magnetometer. For more information, please refer to https://invensense.tdk.com/wp-content/uploads/2016/06/DS-000189-ICM-20948-v1.3.pdf <br> Audio files in this folder are in raw binary form. The following can be used to convert them to WAV files (1250Hz): <br> ffmpeg -f s16le -ar 1250 -ac 1 -i /path/to/audio/file <br> <strong>Synchronization of cameras and werables data</strong> Raw videos contain timecode information which matches the timestamps of the data in the "wearables" folder. The starting timecode of a video can be read as: ffprobe -hide_banner -show_streams -i /path/to/video <br> <strong>./audio</strong> ./sync: contains wav files per each subject ./sync_files: auxiliary csv files used to sync the audio. Can be used to improve the synchronization. The code used for syncing the audio can be found here: https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/audio

本文件包含ConfLab数据集（ConfLab Dataset）中相机与可穿戴设备的原始数据。<br> <strong>./cameras</strong> 目录存放9台相机（cam2至cam10）的顶视录像，格式为MP4文件。所有相机覆盖完整的交互场地：cam2采集场景布局的底部区域，cam10采集场景布局的顶部区域。需注意，cam5较其余相机提前耗尽电量，因此录像提前中断。不过cam4与cam6的录像与cam5存在大量重叠区域，可用于重构所需的全部信息。<br> 需注意，标注数据以2分钟为片段进行制作与提供。针对cam2、4、6、8、10的录像，x2xxx.MP4文件的标注片段为其最后3分38秒内容，x3xxx.MP4文件的标注片段为其前12分钟内容，其中"x"为MP4文件名中的占位符。若需如本数据集所示将录像分割为2分钟片段，可使用提供的"video-splitting.sh"脚本。<br> ./camera-calibration 目录存放从https://github.com/idiap/multicamera-calibration 获取的相机内参文件。可利用已获取的内参文件，结合multicamera-calibration仓库中的说明计算相机外参。图像中的坐标由地面上的十字标记提供，该标记可在录像中被观测到。相邻十字标记间距为1米（即100厘米）。<br> <strong>./wearables</strong> 子目录包含ConfLab活动中全部48名参与者的惯性测量单元（Inertial Measurement Unit, IMU）、近距离蓝牙信号与音频数据。在以参与者ID命名的子目录中，包含以下数据：1. 原始音频文件；2. 近距离蓝牙信号（接收信号强度指示，Received Signal Strength Indication, RSSI）文件（含原始数据与CSV格式文件）及可视化结果；3. 三轴加速度计数据（含原始数据与CSV格式文件）及可视化结果；4. 三轴陀螺仪数据（含原始数据与CSV格式文件）及可视化结果；5. 三轴磁力计数据（含原始数据与CSV格式文件）及可视化结果；6. 以四元数格式记录的游戏旋转矢量数据（含原始数据与CSV格式文件）。<br> 所有文件均带有时间戳。采样频率如下：- 音频：1250 Hz；- 其余类型数据：约50 Hz。但采样率并非固定值，建议以时间戳为准进行数据处理。<br> 针对旋转数据，游戏旋转矢量的输出频率受限于磁力计的实际采样频率。如需了解更多细节，请参考https://invensense.tdk.com/wp-content/uploads/2016/06/DS-000189-ICM-20948-v1.3.pdf。<br> 本目录下的音频文件为原始二进制格式，可使用以下命令将其转换为1250Hz的WAV文件：<br> ffmpeg -f s16le -ar 1250 -ac 1 -i /path/to/audio/file<br> <strong>相机与可穿戴设备数据同步</strong> 原始录像内嵌的时间码信息与"wearables"目录下的数据时间戳完全匹配。可通过以下命令读取录像的起始时间码：ffprobe -hide_banner -show_streams -i /path/to/video<br> <strong>./audio</strong> 目录包含以下内容：<br> ./sync：存放每名受试者的WAV格式音频文件；<br> ./sync_files：用于音频同步的辅助CSV文件，可用于优化同步效果。音频同步所用的代码可从以下仓库获取：https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/audio