sentence-transformers/msmarco-msmarco-distilbert-base-tas-b

MS MARCO with hard negatives from msmarco-distilbert-base-tas-b 数据集是一个基于真实用户搜索查询的大规模信息检索语料库，这些查询来自 Bing 搜索引擎。该数据集包含多种三元组（查询、正面、负面）子集，用于训练 Sentence Transformer 模型。这些子集在每对查询-正面样本的负面样本数量以及数据存储为字符串或ID方面有所不同。该数据集旨在通过提供具有挑战性的负面样本来提高信息检索系统的性能。

MS MARCO with hard negatives from msmarco-distilbert-base-tas-b is a large-scale information retrieval corpus based on real user search queries from the Bing search engine. This dataset contains multiple triplet (query, positive, negative) subsets for training Sentence Transformer models. These subsets differ in the number of negative samples per query-positive sample pair and the storage format of the data, i.e., whether it is saved as strings or IDs. The dataset aims to improve the performance of information retrieval systems by providing challenging negative samples.

原始信息汇总

数据集概述

基本信息

• 语言: 英语
• 多语言性: 单语种
• 数据集大小: 10M < n < 100M
• 任务类别: 特征提取、句子相似度
• 标签: sentence-transformers
• 数据集名称: MS MARCO with hard negatives from msmarco-distilbert-base-tas-b

数据集配置

triplet 配置

• 特征:
  • query: 字符串
  • positive: 字符串
  • negative: 字符串
• 分割:
  • train: 365876392 字节, 502939 样本
• 下载大小: 240229198 字节
• 数据集大小: 365876392 字节

triplet-50 配置

• 特征:
  • query: 字符串
  • positive: 字符串
  • negative_1 至 negative_50: 字符串
• 分割:
  • train: 5257441890 字节, 288020 样本
• 下载大小: 3444062582 字节
• 数据集大小: 5257441890 字节

triplet-50-ids 配置

• 特征:
  • query: 整数
  • positive: 整数
  • negative_1 至 negative_50: 整数
• 分割:
  • train: 119816320 字节, 288020 样本
• 下载大小: 102036560 字节
• 数据集大小: 119816320 字节

triplet-all 配置

• 特征:
  • query: 字符串
  • positive: 字符串
  • negative: 字符串
• 分割:
  • train: 19792167373 字节, 26391481 样本
• 下载大小: 4702308008 字节
• 数据集大小: 19792167373 字节

triplet-all-ids 配置

• 特征:
  • query: 整数
  • positive: 整数
  • negative: 整数
• 分割:
  • train: 633395544 字节, 26391481 样本
• 下载大小: 192368748 字节
• 数据集大小: 633395544 字节

triplet-hard 配置

• 特征:
  • query: 字符串
  • positive: 字符串
  • negative: 字符串
• 分割:
  • train: 11788992227 字节, 15981882 样本
• 下载大小: 3119903058 字节
• 数据集大小: 11788992227 字节

triplet-hard-ids 配置

• 特征:
  • query: 整数
  • positive: 整数
  • negative: 整数
• 分割:
  • train: 383565168 字节, 15981882 样本
• 下载大小: 121870475 字节
• 数据集大小: 383565168 字节

triplet-ids 配置

• 特征:
  • query: 整数
  • positive: 整数
  • negative: 整数
• 分割:
  • train: 12070536 字节, 502939 样本
• 下载大小: 10133540 字节
• 数据集大小: 12070536 字节

数据集子集

triplet 子集

• 列: "query", "positive", "negative"
• 列类型: str, str, str
• 示例: python { "query": "what are the liberal arts?", "positive": liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects., "negative": The New York State Education Department requires 60 Liberal Arts credits in a Bachelor of Science program and 90 Liberal Arts credits in a Bachelor of Arts program. In the list of course descriptions, courses which are liberal arts for all students are identified by (Liberal Arts) after the course number. }

triplet-ids 子集

• 列: "query", "positive", "negative"
• 列类型: int, int, int
• 示例: python { "query": 571018, "positive": 7349777, "negative": 6948601 }

triplet-all 子集

• 列: "query", "positive", "negative"
• 列类型: str, str, str
• 示例: python { "query": "what are the liberal arts?", "positive": liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects., "negative": The New York State Education Department requires 60 Liberal Arts credits in a Bachelor of Science program and 90 Liberal Arts credits in a Bachelor of Arts program. In the list of course descriptions, courses which are liberal arts for all students are identified by (Liberal Arts) after the course number. }

triplet-all-ids 子集

• 列: "query", "positive", "negative"
• 列类型: int, int, int
• 示例: python { "query": 571018, "positive": 7349777, "negative": 6948601 }

triplet-hard 子集

• 列: "query", "positive", "negative"
• 列类型: str, str, str
• 示例: python { "query": "what are the liberal arts?", "positive": liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects., "negative": The New York State Education Department requires 60 Liberal Arts credits in a Bachelor of Science program and 90 Liberal Arts credits in a Bachelor of Arts program. In the list of course descriptions, courses which are liberal arts for all students are identified by (Liberal Arts) after the course number. }

triplet-hard-ids 子集

• 列: "query", "positive", "negative"
• 列类型: int, int, int
• 示例: python { "query": 571018, "positive": 7349777, "negative": 6948601 }

triplet-50 子集

• 列: "query", "positive", negative_1, negative_2, negative_3, negative_4, negative_5, negative_6, negative_7, negative_8, negative_9, negative_10, negative_11, negative_12, negative_13, negative_14, negative_15, negative_16, negative_17, negative_18, negative_19, negative_20, negative_21, negative_22, negative_23, negative_24, negative_25, negative_26, negative_27, negative_28, negative_29, negative_30, negative_31, negative_32, negative_33, negative_34, negative_35, negative_36, negative_37, negative_38, negative_39, negative_40, negative_41, negative_42, negative_43, negative_44, negative_45, negative_46, negative_47, negative_48, negative_49, negative_50
• 列类型: str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str
• 示例: python { "query": "what are the liberal arts?", "positive": "liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects.", "negative_1": "The New York State Education Department requires 60 Liberal Arts credits in a Bachelor of Science program and 90 Liberal Arts credits in a Bachelor of Arts program. In the list of course descriptions, courses which are liberal arts for all students are identified by (Liberal Arts) after the course number.", "negative_2": "What Does it Mean to Study Liberal Arts? A liberal arts major offers a broad overview of the arts, sciences, and humanities. Within the context of a liberal arts degree, you can study modern languages, music, English, anthropology, history, womens studies, psychology, math, political science or many other disciplines.", "negative_3": "What Is Liberal Studies? Liberal studies, also known as liberal arts, comprises a broad exploration of social sciences, natural sciences, humanities, and the arts. If you are interested in a wide-ranging education in humanities, communication, and thinking, read on to find out about the educational and career possibilities in liberal studies.", "negative_4": "You can choose from an array of liberal arts majors. Most of these are offered in the liberal arts departments of colleges that belong to universities and at smaller colleges that are designated as liberal arts institutions.", "negative_5": "Majors. You can choose from an array of liberal arts majors. Most of these are offered in the liberal arts departments of colleges that belong to universities and at smaller colleges that are designated as liberal arts institutions.", "negative_6": "liberal arts. plural noun. Definition of liberal arts for English Language Learners. : areas of study (such as history, language, and literature) that are intended to give you general knowledge rather than to develop specific skills needed for a profession. Nglish: Translation of liberal arts for Spanish speakers Britannica.com: Encyclopedia article about liberal arts.", "negative_7": "Because they award less than 50% of their degrees in engineering, and the rest in liberal arts (sciences). Baccalaureate colleges are a type of Liberal Arts colleges, But offering lesser number of degrees compared to LAC. Its the other way round. A liberal arts college focuses on liberal arts, e.g. sciences, literature, history, sociology, etc. They might offer a few professional degrees (most frequently engineering) as well, but typically the professional majors are well integrated into the liberal arts framework as well.", "negative_8": "A liberal arts college is a four-year institution that focuses on the study of liberal arts. Liberal arts colleges are geared more toward the acquisition of knowledge and less toward specific professions. [MORE: The Path to Higher Education] Graduate school.", "negative_9": "1 BA = Bachelor of Arts degree BS = Bachelor of Science degree. 2 I think the question requires more of an explanation than what the terms BA and BS translate to. 3 B.A. (Bachelor of Arts) A bachelor of arts (B.A.) degree is what is generally called a liberal arts degree. I think the question requires more of an explanation than what the terms BA and BS translate to. 2 B.A. (Bachelor of Arts) A bachelor of arts (B.A.) degree is what is generally called a liberal arts degree.", "negative_10": "West Hills College LemooreAssociate of Arts (A.A.), Liberal Arts and Sciences/Liberal StudiesAssociate of Arts (A.A.), Liberal Arts and Sciences/Liberal Studies. -Student Government President for two years. -Valedictorian. -Alpha Gamma Sigma (Alpha Chi chapter) President/College Relations Liaison.", "negative_11": "You can pursue associate degree in academic area such as business administration, law, arts, engineering, paralegal studies, liberal arts, computer science, and more. Q: What are online associate programs?", "negative_12": "liberal arts definition The areas of learning that cultivate general intellectual ability rather than technical or professional skills. Liberal arts is often used as a synonym for humanities, because literature, languages, history, an" }