Cross-modal LiDAR-camera joint calibration for heading and mining face in coal mines

Objective In underground coal mines, it is challenging to extract the features of the heading and mining face stably under insufficient illumination, significant dust interference, and sparse point clouds. This severely undermines the accuracy and robustness of camera-LiDAR joint calibration based on underground unstructured scene characteristics.MethodsTargeting complex operating conditions, including fully mechanized mining face and tunneling roadways, this study developed a LiDAR-camera joint calibration method based on cross-depth feature coupling. In this method, an improved random sample consensus (RANSAC) algorithm for multi-plane fitting was designed for feature extraction. By integrating normal vector-based pre-clustering and an adaptive iteration mechanism, the improved RANSAC algorithm can efficiently extract geometric structures such as hydraulic support roof beams and tunnel boring machine (TBM) casings. Meanwhile, a cross-depth edge fusion strategy was proposed, which utilized curvature discontinuities and planar intersection features collaboratively to enhance the integrity and robustness of edge structures. Regarding the calibration framework, a two-stage registration strategy consisting of coarse and fine registration was developed. Among these, the coarse registration allowed for the rapid estimation of the initial extrinsic parameters through axial cyclic perturbations. In contrast, the fine registration adopted joint point-line constraints within Lie group space and nonlinear optimization through iteration. This registration strategy was aimed at ensuring high-precision alignment even under coal dust interference and complex operating conditions. Results and ConclusionsThe proposed method was validated on the Gazebo simulation platform and in actual underground scenarios. The results indicate that under noise-free conditions, the proposed method yielded rotational errors of less than 0.2°, translational errors of below 0.02 m, and mean reprojection errors of 3.5 px or less. Furthermore, this method remained highly stable even in environments with high noise levels. Under the scenarios of heading face and fully mechanized mining face, the proposed method yielded mean reprojection errors of 2.89 px and 3.03 px, respectively, significantly outperforming traditional methods. Therefore, this method, requiring no artificial calibration targets, exhibits superior environmental adaptability and stability, meeting the requirements for high-precision calibration of multimodal perception systems in complex underground environments in coal mines.