RGB-Depth (KITTI), RGB-Polarimetric, RGB-Infrared (M3FD)
收藏资源简介：
RGBX-DiffusionDet是一种多模态对象检测框架，它扩展了DiffusionDet模型，以融合异构2D数据（X）与RGB图像。为了实现跨模态交互，设计了一种在卷积块注意力模块（DCR-CBAM）中的动态通道减少，这通过动态突出显著的通道特征来促进子网络之间的交叉对话。此外，提出了动态多级聚合块（DMLAB），以通过自适应多尺度融合来细化空间特征表示。最后，引入了新的正则化损失，以强制执行通道显著性和空间选择性，从而产生紧凑且具有区分度的特征嵌入。该框架在RGB-Depth (KITTI)、RGB-Polarimetric和RGB-Infrared (M3FD)数据集上进行了广泛实验，证明了该方法相对于基线RGB-only DiffusionDet的一致性优势。RGBX-DiffusionDet作为一个灵活的多模态对象检测方法，为将不同的2D传感模式集成到基于扩散的检测流程中提供了新的见解。
RGBX-DiffusionDet is a multimodal object detection framework that extends the DiffusionDet model to fuse heterogeneous 2D data (X) with RGB images. To enable cross-modal interaction, a dynamic channel reduction mechanism within the convolutional block attention module (DCR-CBAM) is designed, which facilitates cross-talk between sub-networks by dynamically highlighting salient channel features. Furthermore, a dynamic multi-level aggregation block (DMLAB) is proposed to refine spatial feature representations via adaptive multi-scale fusion. Finally, a novel regularization loss is introduced to enforce channel saliency and spatial selectivity, yielding compact and discriminative feature embeddings. Extensive experiments are conducted on RGB-Depth (KITTI), RGB-Polarimetric, and RGB-Infrared (M3FD) datasets, which demonstrate the consistent superiority of the proposed method over the baseline RGB-only DiffusionDet. As a flexible multimodal object detection approach, RGBX-DiffusionDet provides new insights into integrating diverse 2D sensing modalities into diffusion-based detection pipelines.
