HauntedLamp

A Multi-camera Dataset: HauntedLamp by LISA ULB and ETRI The HauntedLamp sequence is provided by Laurie Van Bogaert², Andrew Karam*, Armand Losfeld², Gun Bang³, Jinho Lee³, Gauthier Lafruit², Mehrdad Teratani². *student at EPB (Ecole Polytechnique de Bruxelles), ULB (Université Libre de Bruxelles), Belgium. ²members of the LISA department, EPB (Ecole Polytechnique de Bruxelles), ULB (Université Libre de Bruxelles), Belgium. ³members of Electronics and Telecommunications Research Institute, ETRI, Korea Production Laboratory of Image Synthesis and Analysis, LISA department, Ecole Polytechnique de Bruxelles, Universite Libre de Bruxelles, Belgium. Terms of use Any kind of publication or report using this dataset should refer to the following reference:Laurie Van Bogaert, Andrew Karam, Armand Losfeld, Gun Bang, Jinho Lee, Mehrdad Teratani, Gauthier Lafruit. "[INVR] Proposition of a Multi-camera Dataset: HauntedLamp", ISO/IEC JTC 1/SC 29/WG 4, m64774, October 2023, Hannover, Germany. Content The HauntedLamp dataset has been captured using 12 (3x4) synchronized Black-Magic cameras on a structure that is an arc of a circle as seen in Figure 1. The part of the circle arc occupied by the camera is of length 80cm (width) and 50 cm (height) with a rayon of length 2 meters. The acquisition is done using a DELTA-12G-e-h 2i1c acquisition card, an ESE ES 208A clock amplifier, and 12 Blackmagic Micro Studio 4K cameras. The scene represents an office environment with a person, a lamp, and moving objects. In the sequence, the person is cleaning a blackboard, stops and sits down, and then takes a yellow lamp to protect himself from the haunted lamp approaching him. The dataset has been acquired and calibrated in the scope of a master thesis at the Polytechnical School of Brussels, Université Libre de Bruxelles, Belgium [Karam2023]. raw.zip `Raw.zip` contains twelve synchronized raw views with 456 frames acquired at a speed of 30 frames per second (YUV422-8bits-packed). corrected.zip `Corrected.zip` contains twelve undistorted and color-corrected videos of 300 frames (format: YUV420-8bits/yuv420p, 1920x1080 pixels). In addition, the folder contains the `cams_MultiCam_1920x1080.json` file with the camera parameters (intrinsic and extrinsic). This file uses the OMAF coordinates system (Camera position: X: forwards, Y: left, Z: up, Rotation: yaw, pitch, roll). ngpTest.zip The `ngpTest.zip` folder contains an instant-ngp [NGP] model (base2_35000.ingp) trained for 35000 steps with the frame 276 of the camera array, as well as the images and `transforms.json` file used. Inside this folder, the `screenshotTransforms/screen_transforms_p_30.json` file contains a path with 30 intermediate positions between each of the 12 input cameras. The video of this path is present in the `PreviewVideos.zip` folder. previewVideos.zip This folder contains two mp4 videos. The first video is the sequence from camera 6 (`1_cam6.mp4`). The second video (`2_instant_ngp_camera_path_f276.mp4`) navigates in between the cameras and is synthesized with instant-ngp (the configuration file is the `screenshotTransforms/screen_transforms_p_30.json` file in the `ngpTest.zip` folder). License CC BY-NC-SA References [Karam2023] Andrew Karam. (2023) "Room-size Multicamera System Setup for VR Applications" (Master's thesis, Université libre de Bruxelles, Brussels, Belgium) [NGP] Müller, T., Evans, A., Schied, C., & Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4), 1-15.# 多相机数据集：HauntedLamp 本数据集由比利时布鲁塞尔自由大学（Université Libre de Bruxelles, ULB）LISA实验室与韩国电子通信研究院（Electronics and Telecommunications Research Institute, ETRI）联合推出。 HauntedLamp序列由Laurie Van Bogaert²、Andrew Karam*、Armand Losfeld²、Gun Bang³、Jinho Lee³、Gauthier Lafruit²、Mehrdad Teratani²共同提供。 * 作者为比利时布鲁塞尔理工学院（Ecole Polytechnique de Bruxelles）、布鲁塞尔自由大学（ULB）EPB系在读学生。 ² 隶属于比利时布鲁塞尔自由大学布鲁塞尔理工学院EPB系LISA实验室。 ³ 隶属于韩国电子通信研究院（ETRI）。 ## 制作单位 比利时布鲁塞尔自由大学布鲁塞尔理工学院LISA实验室图像合成与分析实验室。 ## 使用条款 任何使用本数据集进行发表或撰写报告的工作，均需引用以下参考文献： Laurie Van Bogaert, Andrew Karam, Armand Losfeld, Gun Bang, Jinho Lee, Mehrdad Teratani, Gauthier Lafruit. "[INVR] 多相机数据集HauntedLamp的构建方案", ISO/IEC JTC 1/SC 29/WG 4, m64774, 2023年10月, 德国汉诺威。 ## 数据集内容 HauntedLamp数据集通过安装在圆弧支架上的12台（3×4排列）同步Blackmagic Micro Studio 4K相机采集得到，相机布置形式如图1所示。相机所在圆弧段的尺寸为：宽度80cm、高度50cm，圆弧半径为2米。本次采集采用DELTA-12G-e-h 2i1c采集卡、ESE ES 208A时钟放大器以及12台Blackmagic Micro Studio 4K相机完成。 本数据集的拍摄场景为办公室环境，包含一名人物、一盏台灯以及若干移动物体。序列内容为：人物正在擦拭黑板，随后停下并坐下，拿起一盏黄色台灯以抵御逼近的“闹鬼台灯”。本数据集是比利时布鲁塞尔自由大学布鲁塞尔理工学院一篇硕士论文的研究成果[Karam2023]，已完成采集与标定工作。 ### 数据文件说明 1. `raw.zip` 该压缩包包含12路同步原始视图，共456帧图像，采集帧率为30帧/秒，格式为YUV422-8bits-packed。 2. `corrected.zip` 该压缩包包含12路经过畸变校正与色彩校正的视频，共300帧，格式为YUV420-8bits/yuv420p，分辨率为1920×1080像素。此外，该文件夹中还包含`cams_MultiCam_1920x1080.json`文件，存储了相机的内参与外参参数。该文件采用OMAF坐标系（相机位置：X轴向前，Y轴向左，Z轴向上；旋转参数：偏航、俯仰、滚转）。 3. `ngpTest.zip` 该文件夹包含一个基于instant-ngp（Instant Neural Graphics Primitives）训练得到的模型（base2_35000.ingp），训练步数为35000步，训练输入为相机阵列的第276帧图像，同时包含所用的图像文件与`transforms.json`配置文件。该文件夹内的`screenshotTransforms/screen_transforms_p_30.json`文件定义了一条路径，包含12台输入相机之间的30个中间位姿。该路径的渲染视频存放在`PreviewVideos.zip`压缩包中。 4. `previewVideos.zip` 该压缩包包含两段MP4视频：第一段为来自6号相机的序列视频`1_cam6.mp4`；第二段`2_instant_ngp_camera_path_f276.mp4`为在相机阵列之间漫游的合成视频，由instant-ngp渲染生成，所用配置文件为`ngpTest.zip`文件夹内的`screenshotTransforms/screen_transforms_p_30.json`。 ## 许可证 CC BY-NC-SA（知识共享署名-非商业性使用-相同方式共享） ## 参考文献 [Karam2023] Andrew Karam. (2023) "面向虚拟现实应用的房间级多相机系统搭建" (硕士论文, 布鲁塞尔自由大学, 比利时布鲁塞尔) [NGP] Müller, T., Evans, A., Schied, C., & Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4), 1-15.