GitHub Repository Deduplication

这是一个包含 1060 万个 GitHub 项目的数据集，这些项目是其他项目的副本，并将每条记录与项目的最终父项链接起来。最终父母来自六个指标的排名。相关项目被计算为一个 1820 万个节点和 1200 万个边去噪图的连通分量，这些边是通过将边指向最终父节点而创建的。该图是通过过滤掉 30 多个精心挑选和 230 万个模式匹配的聚类项目而创建的。通过反复可视化不相关的重要项目之间的最短路径距离来识别引入不需要的聚集的项目。

This is a dataset comprising 10.6 million GitHub forked projects, where each record is linked to the project's ultimate parent. The ultimate parents are derived from rankings across six metrics. Relevant projects are computed as connected components of a denoised graph containing 18.2 million nodes and 12 million edges, with edges constructed by directing links toward the ultimate parent nodes. This graph is built by filtering out over 30 carefully selected and 2.3 million pattern-matched clustered projects. Projects that introduce unwanted clustering are identified by repeatedly visualizing the shortest path distances between significant but unrelated projects.

搜集汇总

背景与挑战

背景概述

该数据集包含1060万个GitHub项目副本，每个项目通过六个指标排名与最终父项链接，形成由1820万个节点和1200万条边构成的去噪图。图的构建通过过滤模式匹配的聚类项目，并利用最短路径距离可视化来移除不需要的聚集项目。