jasonrichdarmawan/nllb-200-6M-sample-embedding
收藏Hugging Face2025-09-23 更新2025-10-25 收录
下载链接：
https://hf-mirror.com/datasets/jasonrichdarmawan/nllb-200-6M-sample-embedding
下载链接
链接失效反馈官方服务：
资源简介：
该数据集包含了文本对及其相关信息，每个样本包括两个文本（text1和text2）、对应文本的语言（lang1和lang2）、两个文本的激光分数（laser_score）、两个文本的嵌入向量（embedding1和embedding2）以及两个文本的blaser相似度（blaser_sim）。数据集通过blaser_sim >= 3.5进行过滤，并使用SONAR的TextToEmbeddingModelPipeline添加了新的嵌入向量列。适用于训练以嵌入向量为输入的模型，如稀疏自动编码器。
The dataset consists of text pairs and their related information, each sample includes two texts (text1 and text2), the corresponding languages of the texts (lang1 and lang2), laser scores for the two texts (laser_score), embedding vectors for the two texts (embedding1 and embedding2), and the blaser similarity (blaser_sim) between the two texts. The dataset is filtered by blaser_sim >= 3.5 and new embedding vector columns are added using SONARs TextToEmbeddingModelPipeline. It is suitable for training models with embeddings as input, such as Sparse Autoencoders.
提供机构：
jasonrichdarmawan
