库帕思金融大模型评测数据集（2024版）

金融大模型评测数据集（2024版），对标《金融大模型应用测评指南》（T/SAIAS 019—2024），涵盖金融行业核心领域，数据来自金融机构行业实践，是金融领域大模型应用成效评测的重要抓手。
评测数据集比照最高水平、最好标准，具有规模大、结构优、价值对齐等特点，符合金融领域对知识鲜活度、多样性和高密度的整体要求。
聚焦“模型基础能力”，围绕计算能力、逻辑推理等6个维度，设计评测数据22000余句对。
聚焦“金融安全与价值对齐能力”，围绕信息内容、社会秩序等13个维度，设计评测数据2000余句对。
聚焦“金融风险控制能力”，围绕合规、市场、操作等5类金融风险，设计评测数据1000余句对。
聚焦“金融业务辅助拓展能力”，围绕舆情分析、智能投研等3项业务场景，设计评测数据12000余句对。
聚焦“金融专业认知能力”，围绕金融专业知识、IPO图表等7种知识类型，设计评测数据7000余句对。
金融大模型评测数据集定期更新、动态迭代，1250条样例集已在Open Data Lab完成开源。

Financial LLM Evaluation Dataset (2024 Edition) is aligned with the Guidelines for Application Evaluation of Financial Large Language Models (T/SAIAS 019—2024). It covers core domains of the financial industry, with data sourced from industry practices of financial institutions, serving as a crucial benchmark for evaluating the effectiveness of LLM applications in the financial sector.
This evaluation dataset aligns with the highest industry standards, featuring large scale, optimized structure, and value alignment, meeting the overall requirements of the financial sector for the freshness, diversity, and high density of knowledge.
Focusing on "Model Basic Capabilities" with 6 dimensions including computing power, logical reasoning, etc., the dataset contains over 22,000 evaluation sentence pairs.
Focusing on "Financial Security and Value Alignment Capabilities" with 13 dimensions including information content, social order, etc., the dataset contains over 2,000 evaluation sentence pairs.
Focusing on "Financial Risk Control Capabilities" covering 5 types of financial risks including compliance, market, operational risks, etc., the dataset contains over 1,000 evaluation sentence pairs.
Focusing on "Financial Business Assistance and Expansion Capabilities" with 3 business scenarios including public opinion analysis, intelligent investment research, etc., the dataset contains over 12,000 evaluation sentence pairs.
Focusing on "Financial Professional Cognitive Capabilities" covering 7 knowledge types including financial professional knowledge, IPO charts, etc., the dataset contains over 7,000 evaluation sentence pairs.
The Financial LLM Evaluation Dataset is updated regularly and dynamically iterated, with a sample set of 1,250 entries open-sourced at the Open Data Lab.