Results of module ablation on the validation set.
收藏资源简介:
Traditional knowledge graphs of water conservancy project risks have supported risk decision-making. However, they are constrained by limited data modalities and low accuracy in information extraction. A multimodal water conservancy project risk knowledge graph is proposed in this study, along with a synergistic strategy involving multimodal large language models Risk decision-making generation is facilitated through a multi-agent agentic retrieval-augmented generation framework. To enhance visual recognition, a DenseNet-based image classification model is improved by incorporating single-head self-attention and coordinate attention mechanisms. For textual data, risk entities such as locations, components, and events are extracted using a BERT-BiLSTM-CRF architecture. These extracted entities serve as the foundation for constructing the multimodal knowledge graph. To support generation, a multi-agent agentic retrieval-augmented generation mechanism is introduced. This mechanism enhances the reliability and interpretability of risk decision-making outputs. In experiments, the enhanced DenseNet model outperforms the original baseline in both precision and recall for image recognition tasks. In risk decision-making tasks, the proposed approach—combining a multimodal knowledge graph with a multi-agent agentic retrieval-augmented generation method—achieves strong performance on BERTScore and ROUGE-L metrics. This work presents a novel perspective for leveraging multimodal knowledge graphs in water conservancy project risk management.
传统水利工程风险知识图谱已为风险决策提供支撑,但受限于数据模态单一、信息提取精度偏低的缺陷。本研究提出一种多模态水利工程风险知识图谱,以及融合多模态大语言模型的协同策略;通过多智能体检索增强生成框架,助力风险决策的生成。为提升视觉识别性能,本研究通过引入单头自注意力与坐标注意力机制,对基于密集卷积网络(DenseNet)的图像分类模型进行优化。针对文本数据,本研究采用BERT-BiLSTM-CRF架构提取位置、构件、事件等风险实体,所提取的实体为多模态知识图谱的构建奠定基础。为支撑决策生成环节,本研究引入多智能体检索增强生成机制,该机制可提升风险决策输出结果的可靠性与可解释性。实验结果表明,优化后的密集卷积网络模型在图像识别任务的精确率与召回率两项指标上均优于原始基线模型。在风险决策任务中,所提出的融合多模态知识图谱与多智能体检索增强生成方法的方案,在BERTScore与ROUGE-L指标上均表现优异。本研究为多模态知识图谱在水利工程风险管理中的应用提供了全新视角。




