代码大语言模型多任务评估基准CodeFuseEval

codefuse-ai系列以及开源大模型（如：MFTCoder\CodeLlama\CodeGeex系列）代码能力评测基准，可用于评估模型在代码生成、跨语言代码翻译、代码优化等多类任务的代码能力，旨在构建贴近企业实际应用大模型代码能力的评测基准。

This code capability evaluation benchmark covers the CodeFuse-AI series and open-source large language models such as MFTCoder, CodeLlama, and CodeGeex series. It can be used to evaluate the code capabilities of models across various tasks including code generation, cross-language code translation, and code optimization, and aims to build a code capability evaluation benchmark for large language models that closely aligns with real-world enterprise applications.

搜集汇总

背景与挑战

背景概述

CodeFuseEval是一个用于评估代码大语言模型的多任务基准数据集，专注于代码翻译、代码补全和文本到代码等任务。该数据集基于CC-BY-NC-SA-4.0许可证发布，由codefuse-ai维护，截至2024年9月已获得超过2,000次下载。