kaust-generative-ai/maestro-mas-benchmark

maestro-mas-benchmark数据集源自MAESTRO，这是一个框架无关的评估套件，用于基于LLM的多智能体系统（MAS）。它提供了MAS行为的系统级视图，旨在跨多种场景对MAS的性能和行为进行基准测试、观察和分析。数据集目前包含12种不同MAS系统的数据，涵盖多个应用领域，并使用多种MAS框架实现。数据以两个Parquet文件形式提供：traces.parquet（包含功能级跟踪数据，如时间、属性和智能体通信元数据）和metrics.parquet（包含硬件指标记录，如CPU和内存使用情况）。通过共享的run_id可以合并跟踪和指标数据进行分析。

maestro-mas-benchmark is a dataset derived from MAESTRO, a framework-agnostic evaluation suite for LLM-based multi-agent systems (MAS). It provides a systems-level view of MAS behavior and is designed to benchmark, observe, and analyze MAS performance and behavior across diverse scenarios. The dataset currently includes data for 12 different MAS systems spanning multiple application domains and implemented using several MAS frameworks. The data is provided as two consolidated Parquet files: traces.parquet (function-level trace data, including timings, attributes, and agent communication metadata) and metrics.parquet (hardware metric records, including CPU and memory usage). Trace and metric data can be merged for analysis by aligning them using the shared run_id.