4Xsample/wikipedia-ca-2026-bge-m3

这是一个基于加泰罗尼亚语维基百科的向量化数据集，发布于2026年5月，使用BGE-M3模型生成1024维嵌入向量。数据集包含清洗后的文本内容，去除了MediaWiki伪影、残留括号和技术标签，以增强语义价值。文章被分割成约1000字符的块，并带有100字符的重叠，以保持上下文。所有向量已归一化，使得内积可直接用作余弦相似度度量。数据集附有FAISS索引（使用SQ8量化），以减小存储大小（约2.4GB），便于在资源受限的工作站上使用。适用于检索增强生成（RAG）、文本检索和问答任务。数据集结构包括唯一ID、标题、文本、URL、向量、源版本标签和清洗时间戳。

This is a vectorized dataset based on Catalan Wikipedia, released in May 2026, using the BGE-M3 model to generate 1024-dimensional embeddings. It contains cleaned text content with removal of MediaWiki artifacts, residual brackets, and technical tags to enhance semantic value. Articles are split into chunks of approximately 1000 characters with a 100-character overlap to preserve context. All vectors are normalized, allowing inner product to be used as a direct measure of cosine similarity. The dataset includes a FAISS index (using SQ8 quantization) to reduce storage size (approx. 2.4GB), making it accessible on resource-constrained workstations. It is suitable for retrieval-augmented generation (RAG), text retrieval, and question-answering tasks. The dataset structure includes unique ID, title, text, URL, vector, source version tag, and cleaning timestamp.