A Reproducible POI Recommendation Framework: Works Mapping and Benchmark Evaluation

In the last years, most works have been concerned with proposing new methods and strategies to improve the points-of-interest (POI) recommendation. However, despite all their advances, there is not a uniform methodology to evaluate these proposed methods by a fair evaluation with metrics that cover different aspects besides accuracy. In this sense, our previous paper (Werneck et al., 2021) have studied the most recent works and proposed a specific framework to simulate and evaluate POI recommendations. Thus, here, we present a reproducibility framework of our previous experiments and results. This framework is composed of: (1) a package to perform analysis over a systematic mapping of the POI recommendation literature; and (2) an extensible benchmark to support research aiming to properly evaluate POI recommendations proposals. While the first contains all collected data and insightful views on current advances and opened challenges, the second addresses these problems by considering different datasets, metrics, and the strongest baselines in the literature. Moreover, this work also demonstrates all processes required to instantiate its framework and reproduces the results of the previous paper, leading us to the same conclusions.近年来，绝大多数相关研究均致力于提出新方法与策略，以优化兴趣点（Point-of-Interest, POI）推荐任务。然而，尽管现有研究已取得诸多进展，但目前尚未形成统一的评估方法论——无法通过覆盖精度以外多维度的指标，对所提出的方法开展公平公正的评估。有鉴于此，我们在先前的研究工作（Werneck等人，2021）中对当前最新的相关研究进行了系统性梳理，并提出了一套用于模拟与评估POI推荐方法的专用框架。因此，本文在此呈现我们此前实验与结果的可复现性框架。该框架包含两部分：(1) 一套用于对POI推荐领域文献开展系统性映射分析的工具包；(2) 一个可扩展的基准测试集，用于支持旨在合理评估POI推荐方法的相关研究。其中，第一部分涵盖了所有已收集的研究数据，以及对当前研究进展与未解决挑战的深度分析视角；第二部分则通过纳入不同数据集、评估指标与领域内主流基线模型，针对性地解决了上述评估不统一的问题。此外，本文还详细展示了实例化该框架所需的全部流程，并复现了此前论文中的实验结果，最终得到了与先前研究一致的结论。