KRAMABENCH
收藏资源简介:
KRAMABENCH是一个由104个手动整理的真实世界数据科学管道组成的基准,涵盖了来自24个数据源的1700个数据文件,涉及6个不同领域。这些管道测试了AI系统在数据处理方面的端到端能力,包括数据发现、数据清洗和准备、高效处理、统计推理以及根据高级任务编排数据处理步骤。数据集用于评估AI模型在设计和执行复杂数据科学管道方面的能力,旨在推动自主数据科学代理在现实世界应用中的发展。
KRAMABENCH is a benchmark consisting of 104 manually curated real-world data science pipelines. It encompasses 1700 data files from 24 data sources and spans 6 distinct domains. These pipelines evaluate the end-to-end data processing capabilities of AI systems, including data discovery, data cleaning and preparation, efficient data processing, statistical inference, and the orchestration of data processing workflows based on high-level tasks. The dataset is designed to assess the ability of AI models to design and execute complex data science pipelines, with the goal of advancing the development of autonomous data science agents for real-world applications.




