LiDAR and thermal data for camera pose estimation using the depth-map correspondence algorithm

Folder and file structure: lidar_roi.ply : ~360 MB mesh file which is a sub-part of the whole Orlova Chuka scan collected in [1] yyyy-mm-dd total of ~17 GB. All video data including raw data, exported video, digitised xy points and calibration results 2018-08-19 2018-08-17 2018-08-14 2018-07-28 2018-07-25 2018-07-21 Thermal camera YYYY-MM-DD folder substructure: Each of the yyyy-mm-dd dates is one recording session. Each session folder has the following structure: avi_files (present on some nights) cave_photos: (present on some nights) mic_and_wall_points tmc_files: (present on some nights) The TMC files is a proprietary format to store thermal camera video data (TeAx GmbH, Germany). on 2018-08-17, only P0000000 is provided as it doesnt' have humans blocking the scene. Each frame can be exported to csv using the ThermoViewer tool, downloadable at: https://thermalcapture.com/thermoviewer-download/ video_calibration: results and associated data to get DLT coefficients estimated using the easyWand [2] workflow. image : csv file with pixel values of the images used for annotations mics : 2D point locations of mics placed on the cave walls other_cave_surface : other points on the cave surface that were pointed at calibration_output: results from easyWand runs. Choose the highest round number yyyy-mm-dd_roundX_<wandscore>_cam1Tforms.mat (undistortion files) yyyy-mm-dd_roundX_<wandscore>_cam2Tforms.mat yyyy-mm-dd_roundX_<wandscore>_cam3Tforms.mat yyyy-mm-dd_roundX_<wandscore>_dltCoefs.csv (each column is one camera's DLT coefficients) yyyy-mm-dd_roundX_<wandscore>_easyWandData.mat (easyWand session file) gravity: (mostly there) video and xy points for a falling object to align the calbiration to gravity. Output from DLTdv7 clicking session. wand: video and xy points of the 'wand' calibration object. Output from DLTdv7 clicking session. camera_intrinsic.txt or thermalcam_camprofiles_profile.txt : the camera instrinsics alignment_results: ~887 MB zipped folder. dmcp_experiments: the results of DMCP alignment round_01 : ignore this folder round_03 : ignore this folder round_05: here yyyy-mm-dd is short for all other nights. Each yyyy-mm-dd folder has multiple csv files. The 'transform.csv' is the most relevant file, as it holds the transformation matrix to move 3D points from camera triangulations into the LiDAR coordinate system. 2018-07-21--cam0 2018-07-21--cam1 2018-07-21--cam2 yyyy-mm-dd--cam0 yyyy-mm-dd--cam1 yyyy-mm-dd--cam2 ... ... ... 2018-08-19--cam0 2018-08-19--cam1 2018-08-19--cam2 CITATION: If you use this dataset for your research please cite this Zenodo dataset and the accompanying paper. This uploaded dataset is part of the Ushichka dataset [3]. The audio-video system was designed by Holger R. Goerlitz. The LiDAR data was collected by Asparuh Kamburov. Video data collected by Thejasvi Beleyur. References [1] : Kamburov, A., Goerlitz, H. R., Beleyur, T 2018, Geospatial modelling inside the "Orlova Chuka" cave in Bulgaria, non-peer reviewed conference contribution, XXVIII International Symposium on Modern Technologies and Professional Practise in Geodesy and related fields [2]: Theriault, D. H., Fuller, N. W., Jackson, B. E., Bluhm, E., Evangelista, D., Wu, Z., M., Betke & Hedrick, T. L. (2014). A protocol and calibration method for accurate multi-camera field videography. Journal of Experimental Biology, 217(11), 1843-1848. [3]: Beleyur Thejasvi, 2021. Theoretical and empirical investigations of echolocation in bat groups, PhD dissertation, University of Konstanz (http://nbn-resolving.de/urn:nbn:de:bsz:352-2-q41u3qlu1em03)

文件夹与文件结构：`lidar_roi.ply`：约360 MB的网格文件，为文献[1]中采集的Orlova Chuka扫描数据的子部分，总数据集规模约17 GB。所有视频数据包含原始数据、导出视频、数字化xy坐标点与标定结果，对应采集日期为2018-08-19、2018-08-17、2018-08-14、2018-07-28、2018-07-25、2018-07-21。

热成像相机文件夹子结构：每个`yyyy-mm-dd`文件夹对应一次录制会话，各会话文件夹包含以下子目录/文件：
- `avi_files`：部分夜间录制会话包含该目录
- `cave_photos`：部分夜间录制会话包含该目录
- `mic_and_wall_points`
- `tmc_files`：部分夜间录制会话包含该目录

其中TMC格式为德国TeAx GmbH开发的专有格式，用于存储热成像相机视频数据。

2018-08-17的会话仅提供`P0000000`片段，因该片段无人类遮挡场景。所有视频帧可通过`ThermoViewer`工具导出为CSV格式，工具下载链接为：https://thermalcapture.com/thermoviewer-download/

`video_calibration`：用于通过`easyWand`[2]工作流估算DLT系数的标定结果与关联数据集：
- `image`：存储用于标注的图像像素值的CSV文件
- `mics`：洞穴壁上部署的麦克风的二维点坐标
- `other_cave_surface`：标定过程中采集的洞穴表面其余测点坐标
- `calibration_output`：`easyWand`运行输出的标定结果，需选择轮次最高的文件，具体包括：
- `yyyy-mm-dd_roundX_<wandscore>_cam1Tforms.mat`：畸变校正文件
- `yyyy-mm-dd_roundX_<wandscore>_cam2Tforms.mat`
- `yyyy-mm-dd_roundX_<wandscore>_cam3Tforms.mat`
- `yyyy-mm-dd_roundX_<wandscore>_dltCoefs.csv`：每列对应一台相机的DLT系数
- `yyyy-mm-dd_roundX_<wandscore>_easyWandData.mat`：`easyWand`会话文件
- `gravity`：多数情况下包含该目录，存储用于将标定结果对齐至重力坐标系的落体视频与xy坐标点数据，为`DLTdv7`点击会话的输出结果
- `wand`：存储用于标定的"标杆"校准对象的视频与xy坐标点数据，为`DLTdv7`点击会话的输出结果
- `camera_intrinsic.txt` 或 `thermalcam_camprofiles_profile.txt`：相机内参文件

`alignment_results`：约887 MB的压缩文件夹，其中`dmcp_experiments`为DMCP对齐的实验结果：
- `round_01`：需忽略该目录
- `round_03`：需忽略该目录
- `round_05`：有效结果目录，其中`yyyy-mm-dd`指代其余所有日期，每个`yyyy-mm-dd`文件夹包含多个CSV文件，其中`transform.csv`为最关键的文件，其存储了将三维点从相机三角测量坐标系转换至激光雷达（LiDAR）坐标系的变换矩阵。
子目录命名示例包括：`2018-07-21--cam0`、`2018-07-21--cam1`、`2018-07-21--cam2`，直至`2018-08-19--cam0`、`2018-08-19--cam1`、`2018-08-19--cam2`等。

## 引用说明
若将本数据集用于学术研究，请引用该Zenodo数据集及配套发表论文。本上传数据集属于`Ushichka`数据集[3]。音视频系统由Holger R. Goerlitz设计，激光雷达（LiDAR）数据由Asparuh Kamburov采集，视频数据由Thejasvi Beleyur采集。

## 参考文献
[1] Kamburov, A., Goerlitz, H. R., Beleyur, T. 2018, 保加利亚Orlova Chuka洞穴内部的地理空间建模，未经过同行评审的会议成果，第28届国际现代技术与大地测量及相关领域专业实践研讨会
[2] Theriault, D. H., Fuller, N. W., Jackson, B. E., Bluhm, E., Evangelista, D., Wu, Z., Betke, M. & Hedrick, T. L. (2014). 高精度多相机野外视频拍摄的协议与标定方法，《实验生物学杂志》，217(11), 1843-1848
[3] Beleyur Thejasvi, 2021. 蝙蝠集群回声定位的理论与实证研究，博士学位论文，康斯坦茨大学（http://nbn-resolving.de/urn:nbn:de:bsz:352-2-q41u3qlu1em03）