BidirLM/BidirLM-Contrastive

该数据集是一个多语言文本数据集，支持英语、中文、阿拉伯语、保加利亚语、加泰罗尼亚语、捷克语、丹麦语、德语、希腊语、西班牙语、爱沙尼亚语、波斯语、芬兰语、法语、希伯来语、克罗地亚语、匈牙利语、亚美尼亚语、印度尼西亚语、冰岛语、意大利语、日语、格鲁吉亚语、哈萨克语、韩语、立陶宛语、拉脱维亚语、马其顿语、马来语、马耳他语、荷兰语、挪威博克马尔语、波兰语、葡萄牙语、罗马尼亚语、俄语、斯洛伐克语、斯洛文尼亚语、阿尔巴尼亚语、塞尔维亚语、瑞典语、泰语、土耳其语、乌克兰语、越南语、南非荷兰语、阿塞拜疆语、白俄罗斯语、波斯尼亚语、威尔士语、巴斯克语、爱尔兰语、加利西亚语等多种语言。它包含多个配置，如情感分类、GooAQ、MAmmoTH2、MIRACL、MSMARCO、NFCorpus、自然问题、PAQ、SQuAD、合成分类数据、TriviaQA以及针对特定语言的数据集（例如中文的AFQMC、AdvertiseGen、CAIL2019-SCM、CHEF、CINLID、ChatMed咨询数据集、ChineseSTS、DRCD、LCSTS、Multi-CPR、QBQTC、RefGPT、SimCLUE、T2Ranking、THUCNews、UMETRIP-QA、WebCPM、atec、bq、cCOVID-News、cMedQA-V2.0、cmnli、cmrc2018、csl、dureader、law-gpt、lawzhidao、lima-chinese、llm检索变体、llm_sts变体、mmarco-chinese、nli_zh、ocnli、retrieval_data_llm_infgrad、webqa、xnli_zh，以及英语和其他语言的类似数据集）。数据集设计用于文本嵌入、对比学习、检索、句子相似性和文本分类任务，规模在1000万到1亿条数据之间，以Apache 2.0许可证发布。

This dataset is a multilingual text dataset supporting languages including English, Chinese, Arabic, Bulgarian, Catalan, Czech, Danish, German, Greek, Spanish, Estonian, Persian, Finnish, French, Hebrew, Croatian, Hungarian, Armenian, Indonesian, Icelandic, Italian, Japanese, Georgian, Kazakh, Korean, Lithuanian, Latvian, Macedonian, Malay, Maltese, Dutch, Norwegian Bokmål, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Albanian, Serbian, Swedish, Thai, Turkish, Ukrainian, Vietnamese, Afrikaans, Azerbaijani, Belarusian, Bosnian, Welsh, Basque, Irish, and Galician. It includes multiple configurations such as EmotionClassification, GooAQ, MAmmoTH2, MIRACL, MSMARCO, NFCorpus, NaturalQuestions, PAQ, SQuAD, SyntheticClassificationData, TriviaQA, and language-specific datasets (e.g., Chinese AFQMC, AdvertiseGen, CAIL2019-SCM, CHEF, CINLID, ChatMed Consult Dataset, ChineseSTS, DRCD, LCSTS, Multi-CPR, QBQTC, RefGPT, SimCLUE, T2Ranking, THUCNews, UMETRIP-QA, WebCPM, atec, bq, cCOVID-News, cMedQA-V2.0, cmnli, cmrc2018, csl, dureader, law-gpt, lawzhidao, lima-chinese, llm retrieval variants, llm_sts variants, mmarco-chinese, nli_zh, ocnli, retrieval_data_llm_infgrad, webqa, xnli_zh, and similar datasets for English and other languages). The dataset is designed for text embedding, contrastive learning, retrieval, sentence similarity, and text classification tasks, with a size between 10 million and 100 million entries, and is released under the Apache 2.0 license.