Geological map information extraction combining knowledge guidance and few-shot learning

Geological maps serve as crucial data sources for reflecting stratigraphy, rock masses, structural features, and other geological information. Extracting domain-specific knowledge from such maps is of significant importance for geological surveys, resource exploration, and mineral prospectivity prediction. However, the inherent complexity of geological structures, the diversity of legend symbols, and the presence of image noise introduce considerable challenges to information extraction. Traditional approaches to geological map information extraction often rely heavily on data quality and availability, while existing large language and vision models also exhibit limitations when directly applied to this domain. To address these challenges, this study focuses on advancing from local extraction to global extraction and proposes a method termed GMIKF (Geological Map Information extraction combining Knowledge guidance and Few-shot learning). The method incorporates external domain knowledge into the GPT-4o model to provide knowledge-guided instruction, while integrating a local visual enhancement mechanism to design visual prompts for geological legends. By combining few-shot prompt learning, the model transitions from legend-level information extraction to comprehensive main-map interpretation. Furthermore, a feedback mechanism is introduced to ensure the accuracy of extraction for each geological map within the dataset. Experimental results demonstrate that, on the geological map dataset constructed in this study, the proposed GMIKF method achieves an extraction accuracy of 93.54%, significantly surpassing the performance of directly applying large models to geological map information extraction (67.42%), yielding an improvement of approximately 26%. These results validate the effectiveness of GMIKF in tackling geological map information extraction tasks.