BidirLM/BidirLM-Omni-Contrastive

BidirLM-Omni-Contrastive是一个用于训练BidirLM-Omni模型的多模态对比学习数据集集合。该数据集总共有1,800,000个样本，平衡分布在多种模态组合中，包括文本-文本（1,200,000个样本）、音频-文本（300,000个样本，来自两个子集）和图像-文本（300,000个样本，来自三个子集）。它旨在通过对比学习的方式，训练能够理解和生成跨模态表示的双向编码器。数据集由多个子数据集组成，分别从不同的来源收集，如LAION Audio、LibriSpeech、ColPali、NatCap和MSCOCO等，并通过统一的代码框架进行加载、混洗和采样，以构建用于模型训练的数据混合。

BidirLM-Omni-Contrastive is a collection of omnimodal contrastive learning datasets used to train the BidirLM-Omni model. The dataset consists of a total of 1,800,000 samples, balanced across multiple modality pairs, including text-text (1,200,000 samples), audio-text (300,000 samples from two subsets), and image-text (300,000 samples from three subsets). It is designed to train bidirectional encoders capable of understanding and generating cross-modal representations through contrastive learning. The dataset is composed of several sub-datasets sourced from various repositories, such as LAION Audio, LibriSpeech, ColPali, NatCap, and MSCOCO, and is integrated with a unified code framework for loading, shuffling, and sampling to construct the data mixture for model training.