Upstash/wikipedia-2024-06-bge-m3

该数据集包含了2024年6月维基百科的11种最流行语言的嵌入向量，这些嵌入向量是通过BGE-M3模型生成的。数据集中的维基百科文章被分割成段落，并在嵌入前添加了相应的文章标题。此外，少于100个字符的段落被排除在外。数据集总共包含约1.44亿个向量嵌入。

This dataset contains embeddings from the June 2024 Wikipedia dump for the 11 most popular languages. The embeddings are generated with the multilingual BGE-M3 model. The dataset consists of Wikipedia articles split into paragraphs, and embedded with the aforementioned model. To enhance search quality, the paragraphs are prefixed with their respective article titles before embedding. Additionally, paragraphs containing fewer than 100 characters, which tend to have low information density, are excluded from the dataset. The dataset contains approximately 144 million vector embeddings in total.