BAAI/IndustryCorpus2_finance_economics

行业模型在推动企业智能化转型和创新发展中发挥着至关重要的作用。高质量的行业数据是提升大模型性能和实现行业应用落地的关键。然而，目前用于行业模型训练的数据集普遍存在数据量少、质量低、专业性不足等问题。

Industry models play a crucial role in promoting enterprises' intelligent transformation and innovative development. High-quality industry data is the key to enhancing the performance of large language models (LLMs) and realizing the practical deployment of industry-specific applications. However, the datasets currently used for training industry models generally suffer from issues such as limited data volume, low quality, and insufficient professional specialization.

搜集汇总

背景与挑战

背景概述

该数据集是智源研究院IndustryCorpus2.0中的金融行业预训练数据集，旨在解决行业模型训练中数据量少、质量低的问题。它通过引入高质量数据源、实施严格的质量筛选与分层（分为high、middle、low三个层次），并基于36K规模的中英文采样数据构建，以提升大模型的训练效率和性能。