baidu/ERIA-1K-Benchmark

ERIA-1K（ERNIE-Image-Aes-1K）是一个开源的人类标注图像美学基准数据集，旨在反映真实世界的图像分布，并提供更面向实际部署的评估协议。该数据集包含1,000张图像，涵盖6个类别，其比例经过校准以近似真实世界分布。标注采用成对瑞士系统锦标赛方法，由来自美术、设计和摄影背景的专业标注者进行，生成了从1到10的等级标签。该基准旨在暴露现有美学预测模型的系统偏见，并提供一个稳定和可复现的排名评估。数据格式包括图像文件夹和一个包含真实美学等级分数的JSON文件，用于计算SRCC和PLCC评估指标。ERIA-1K (ERNIE-Image-Aes-1K) is an open-source human-annotated image aesthetics benchmark dataset. It is designed to reflect real-world image distributions and offer evaluation protocols more tailored for practical deployment. This dataset comprises 1,000 images across 6 categories, with their proportions calibrated to approximate real-world distributions. Annotations were completed using the paired Swiss-system tournament methodology, conducted by professional annotators with backgrounds in fine art, design, and photography, producing rating labels ranging from 1 to 10. This benchmark aims to uncover the systemic biases of current aesthetics prediction models and deliver a stable and reproducible ranking evaluation. The data format consists of image folders and a JSON file containing ground-truth aesthetic rating scores, which is utilized to compute the SRCC and PLCC evaluation metrics.