Grading System

Extracting entity connectivity from texts is important for uncovering how places relate within real-world discourse. While structured data is informative, textual data captures rich contextual and semantic knowledge enabling us to identify hidden networks of interdependence and thematic connections among geographic entities. Entity connectivity is not just complementary to information retrieval, but rather essential in various activities including event analysis, spatial decision support systems, urban studies, and knowledge graph development. This research presents a grading system for evaluating connectivity between geospatial entities, such as places, events, and geopolitical entities extracted from texts. The methodology proposes two versions of the grading system: one based on co-occurrences and semantic similarity, and a second one that additionally incorporates the geodesic distance feature. The two systems are evaluated and compared using six machine learning algorithms: Random Forest, Gradient Boosting, Multi-Layer Perceptron (MLP), K-Nearest Neighbors (KNN), Decision Tree, and Support Vector Machine (SVM). The performance of the algorithms is analyzed by measuring accuracy, precision, recall, F1-score, and R². The results show that the system without geodesic distance performs better on general texts, indicating that the addition of geographic features can introduce noise in cases where geographic relevance is not immediately important. In terms of accuracy, Decision Tree was the best performing algorithm for the system without geodesic distance, whereas K-Nearest Neighbors (KNN) was the best performing model for the system with geodesic distance, which indicates that different feature sets may benefit from different learning strategies. You can find attached the data used for training and the trained classifiers for both the 2 versions of the grading system.