whatphiliptrains/wikipos

WikiPos 是一个经过处理的维基媒体维基百科数据集，包含通过降维技术生成的 2D 空间坐标。数据集包含维基百科文章的原始文本内容加上使用 UMAP 和 t-SNE 算法从句子嵌入生成的 x,y 坐标。它允许对维基百科内容进行空间可视化和探索，在 2D 坐标空间中分析文章之间的语义关系。数据集以多种配置形式提供，基于不同的嵌入模型和降维技术，并包含一个包含 1,000,000 个样本的训练分割。数据集是从维基媒体维基百科数据集（20231101.en 压缩包）派生出来的，包括处理流程、技术细节和使用数据时的注意事项。

WikiPos is a processed version of the Wikimedia Wikipedia dataset that includes 2D spatial coordinates generated through dimensionality reduction techniques. The dataset contains Wikipedia articles with their original text content plus x,y coordinates derived from sentence embeddings using UMAP and t-SNE algorithms. It allows for spatial visualization and exploration of Wikipedia content, analyzing semantic relationships between articles in a 2D coordinate space. The dataset is available in multiple configurations based on different embedding models and dimensionality reduction techniques, and includes a single train split with 1,000,000 samples. The dataset is derived from the Wikimedia Wikipedia dataset (20231101.en dump) and includes details about the processing pipeline, technical details, and considerations for using the data.