starpacker52/imaging-101
收藏资源简介：
Imaging-101是一个用于评估LLM编码代理在科学计算成像问题上性能的基准数据集。它包含57个经过专家验证的计算成像任务，覆盖天文学、生物学、物理学、化学与材料、地球科学和医学六个领域。数据集提供了观测数据、地面真实数据和元数据，以及用于评估的预计算输入/输出对。每个任务都遵循标准化的结构，包括数据目录和评估固定装置目录。数据集没有训练/测试分割，因为它旨在评估LLM代理从头开始实现完整成像管道的能力。
Imaging-101 is a benchmark dataset designed to evaluate the performance of LLM coding agents on scientific computational imaging problems. It includes 57 expert-verified computational imaging tasks across six domains: Astronomy, Biology, Physics, Chemistry & Materials, Earth Science, and Medicine. The dataset provides observation data, ground truth data, and metadata, as well as pre-computed input/output pairs for evaluation. Each task follows a standardized structure, including a data directory and an evaluation fixtures directory. There is no train/test split as the benchmark evaluates the ability of an LLM agent to implement a complete imaging pipeline from scratch.
