HauntedLamp

A Multi-camera Dataset: HauntedLamp by LISA ULB and ETRI The HauntedLamp sequence is provided by Laurie Van Bogaert², Andrew Karam*, Armand Losfeld², Gun Bang³, Jinho Lee³, Gauthier Lafruit², Mehrdad Teratani². *student at EPB (Ecole Polytechnique de Bruxelles), ULB (Université Libre de Bruxelles), Belgium. ²members of the LISA department, EPB (Ecole Polytechnique de Bruxelles), ULB (Université Libre de Bruxelles), Belgium. ³members of Electronics and Telecommunications Research Institute, ETRI, Korea Production Laboratory of Image Synthesis and Analysis, LISA department, Ecole Polytechnique de Bruxelles, Universite Libre de Bruxelles, Belgium. Terms of use Any kind of publication or report using this dataset should refer to the following reference:Laurie Van Bogaert, Andrew Karam, Armand Losfeld, Gun Bang, Jinho Lee, Mehrdad Teratani, Gauthier Lafruit. "[INVR] Proposition of a Multi-camera Dataset: HauntedLamp", ISO/IEC JTC 1/SC 29/WG 4, m64774, October 2023, Hannover, Germany. Content The HauntedLamp dataset has been captured using 12 (3x4) synchronized Black-Magic cameras on a structure that is an arc of a circle as seen in Figure 1. The part of the circle arc occupied by the camera is of length 80cm (width) and 50 cm (height) with a rayon of length 2 meters. The acquisition is done using a DELTA-12G-e-h 2i1c acquisition card, an ESE ES 208A clock amplifier, and 12 Blackmagic Micro Studio 4K cameras. The scene represents an office environment with a person, a lamp, and moving objects. In the sequence, the person is cleaning a blackboard, stops and sits down, and then takes a yellow lamp to protect himself from the haunted lamp approaching him. The dataset has been acquired and calibrated in the scope of a master thesis at the Polytechnical School of Brussels, Université Libre de Bruxelles, Belgium [Karam2023]. raw.zip `Raw.zip` contains twelve synchronized raw views with 456 frames acquired at a speed of 30 frames per second (YUV422-8bits-packed). corrected.zip `Corrected.zip` contains twelve undistorted and color-corrected videos of 300 frames (format: YUV420-8bits/yuv420p, 1920x1080 pixels). In addition, the folder contains the `cams_MultiCam_1920x1080.json` file with the camera parameters (intrinsic and extrinsic). This file uses the OMAF coordinates system (Camera position: X: forwards, Y: left, Z: up, Rotation: yaw, pitch, roll). ngpTest.zip The `ngpTest.zip` folder contains an instant-ngp [NGP] model (base2_35000.ingp) trained for 35000 steps with the frame 276 of the camera array, as well as the images and `transforms.json` file used. Inside this folder, the `screenshotTransforms/screen_transforms_p_30.json` file contains a path with 30 intermediate positions between each of the 12 input cameras. The video of this path is present in the `PreviewVideos.zip` folder. previewVideos.zip This folder contains two mp4 videos. The first video is the sequence from camera 6 (`1_cam6.mp4`). The second video (`2_instant_ngp_camera_path_f276.mp4`) navigates in between the cameras and is synthesized with instant-ngp (the configuration file is the `screenshotTransforms/screen_transforms_p_30.json` file in the `ngpTest.zip` folder). License CC BY-NC-SA References [Karam2023] Andrew Karam. (2023) "Room-size Multicamera System Setup for VR Applications" (Master's thesis, Université libre de Bruxelles, Brussels, Belgium) [NGP] Müller, T., Evans, A., Schied, C., & Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4), 1-15.多相机数据集：HauntedLamp 出品方：布鲁塞尔自由大学（Université Libre de Bruxelles, ULB）LISA实验室与韩国电子通信研究院（Electronics and Telecommunications Research Institute, ETRI） 本数据集的HauntedLamp序列由Laurie Van Bogaert²、Andrew Karam*、Armand Losfeld²、Gun Bang³、Jinho Lee³、Gauthier Lafruit²、Mehrdad Teratani² 提供。 * 比利时布鲁塞尔理工学院（Ecole Polytechnique de Bruxelles, EPB）、布鲁塞尔自由大学（ULB）在读学生。 ² 比利时布鲁塞尔自由大学（Université Libre de Bruxelles, ULB）布鲁塞尔理工学院（Ecole Polytechnique de Bruxelles, EPB）LISA实验室成员。 ³ 韩国电子通信研究院（Electronics and Telecommunications Research Institute, ETRI）研究员。 数据集制作单位：比利时布鲁塞尔自由大学（Université Libre de Bruxelles, ULB）布鲁塞尔理工学院LISA图像合成与分析实验室。 使用条款 任何使用本数据集进行出版物撰写或报告创作的工作，均需引用以下参考文献： Laurie Van Bogaert, Andrew Karam, Armand Losfeld, Gun Bang, Jinho Lee, Mehrdad Teratani, Gauthier Lafruit. "[INVR] Proposition of a Multi-camera Dataset: HauntedLamp", ISO/IEC JTC 1/SC 29/WG 4, m64774, 2023年10月，德国汉诺威。 数据集内容 HauntedLamp数据集采用12台（3×4阵列）同步Blackmagic相机进行采集，相机安装于圆弧状支架上，具体布局见图1。相机阵列所覆盖的圆弧段尺寸为：宽度80cm，高度50cm，圆弧半径为2米。本次采集使用DELTA-12G-e-h 2i1c采集卡、ESE ES 208A时钟放大器及12台Blackmagic Micro Studio 4K相机完成。 采集场景为办公室环境，包含人物、台灯及移动物体。序列内容为：人物正在擦拭黑板，随后停下并落座，接着拿起黄色台灯以抵御向其逼近的闹鬼台灯。本数据集的采集与标定工作基于比利时布鲁塞尔自由大学布鲁塞尔理工学院的一篇硕士论文完成[Karam2023]。 ### 压缩包说明 1. raw.zip `Raw.zip` 包含12路同步原始视图，共456帧，采集帧率为30fps，格式为YUV422-8bits-packed。 2. corrected.zip `Corrected.zip` 包含12路经畸变校正与色彩校正的视频，共300帧，格式为YUV420-8bits/yuv420p，分辨率为1920×1080像素。此外，该文件夹内还包含`cams_MultiCam_1920x1080.json`相机参数文件，存储了所有相机的内参与外参。该文件采用OMAF坐标系（相机位置：X轴向前，Y轴向左，Z轴向上；旋转参数：偏航、俯仰、滚转）。 3. ngpTest.zip `ngpTest.zip` 文件夹包含一个基于instant-ngp[NGP]训练得到的模型（base2_35000.ingp），训练步数为35000，训练数据源为该相机阵列的第276帧。此外还包含训练所用的图像数据与`transforms.json`配置文件。该文件夹内的`screenshotTransforms/screen_transforms_p_30.json`文件定义了一条路径，包含12台输入相机之间的30个中间位姿。该路径对应的漫游视频存储于`PreviewVideos.zip`文件夹中。 4. previewVideos.zip 该文件夹包含两段MP4视频：第一段为相机6的拍摄序列`1_cam6.mp4`；第二段`2_instant_ngp_camera_path_f276.mp4`为基于相机阵列漫游路径合成的视频，由instant-ngp生成（配置文件为`ngpTest.zip`中的`screenshotTransforms/screen_transforms_p_30.json`）。 授权协议 CC BY-NC-SA（知识共享署名-非商业性使用-相同方式共享） 参考文献 [Karam2023] Andrew Karam. (2023) "Room-size Multicamera System Setup for VR Applications" (硕士论文, 布鲁塞尔自由大学, 比利时布鲁塞尔) [NGP] Müller, T., Evans, A., Schied, C., & Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4), 1-15.