Grading System

Extracting entity connectivity from texts is important for uncovering how places relate within real-world discourse. While structured data is informative, textual data captures rich contextual and semantic knowledge enabling us to identify hidden networks of interdependence and thematic connections among geographic entities. Entity connectivity is not just complementary to information retrieval, but rather essential in various activities including event analysis, spatial decision support systems, urban studies, and knowledge graph development. This research presents a grading system for evaluating connectivity between geospatial entities, such as places, events, and geopolitical entities extracted from texts. The methodology proposes two versions of the grading system: one based on co-occurrences and semantic similarity, and a second one that additionally incorporates the geodesic distance feature. The two systems are evaluated and compared using six machine learning algorithms: Random Forest, Gradient Boosting, Multi-Layer Perceptron (MLP), K-Nearest Neighbors (KNN), Decision Tree, and Support Vector Machine (SVM). The performance of the algorithms is analyzed by measuring accuracy, precision, recall, F1-score, and R². The results show that the system without geodesic distance performs better on general texts, indicating that the addition of geographic features can introduce noise in cases where geographic relevance is not immediately important. In terms of accuracy, Decision Tree was the best performing algorithm for the system without geodesic distance, whereas K-Nearest Neighbors (KNN) was the best performing model for the system with geodesic distance, which indicates that different feature sets may benefit from different learning strategies. You can find attached the data used for training and the trained classifiers for both the 2 versions of the grading system.

从文本中提取实体连通性（entity connectivity），对于揭示现实世界话语中各类场所的关联逻辑至关重要。尽管结构化数据具备信息价值，但文本数据蕴含丰富的语境与语义知识，可助力我们挖掘地理实体间隐藏的依存网络与主题关联。实体连通性不仅是信息检索的补充手段，更在事件分析、空间决策支持系统、城市研究以及知识图谱构建等诸多场景中不可或缺。本研究提出一套评分体系，用于评估从文本中提取的场所、事件、地缘政治实体等地理空间实体间的关联程度。该方法论设计了两套评分方案：其一基于共现关系与语义相似度，其二则额外融入测地距离特征。研究采用随机森林（Random Forest）、梯度提升树（Gradient Boosting）、多层感知机（Multi-Layer Perceptron，MLP）、K近邻（K-Nearest Neighbors，KNN）、决策树（Decision Tree）以及支持向量机（Support Vector Machine，SVM）共六种机器学习算法，对两套体系开展评估与对比。通过准确率、精确率、召回率、F1值以及决定系数R²分析各算法的性能表现。结果显示，未引入测地距离的评分体系在通用文本场景下表现更优，表明当地理相关性并非核心考量因素时，加入地理特征反而可能引入噪声。在准确率维度上，未加入测地距离的体系最优算法为决策树，而加入测地距离的体系最优模型则为K近邻（KNN），这说明不同的特征集可通过适配不同的学习策略以获得更佳性能。本研究附带了两套评分体系所用的训练数据与训练完成的分类器。