ZhuOnR/ScreenSpot-Pro

ScreenSpot-Pro是一个新颖的基准数据集，旨在评估多模态大语言模型（MLLMs）在高分辨率专业环境中的图形用户界面（GUI）定位能力。与之前专注于通用和简单任务的基准不同，ScreenSpot-Pro捕捉了专业软件应用带来的独特挑战，包括更高的屏幕分辨率、更小的相对目标尺寸和复杂的界面。该数据集包含1,581张真实的高分辨率屏幕截图，涵盖5个行业（开发、创意、CAD、科学和办公）和3个操作系统的23个专业应用。每个截图都标注了目标UI元素的精确边界框和相应的自然语言指令。ScreenSpot-Pro is a novel benchmark designed to evaluate the GUI grounding capabilities of multimodal large language models (MLLMs) in high-resolution professional environments. Unlike previous benchmarks that focus on general and easy tasks, ScreenSpot-Pro captures the unique challenges posed by professional software applications, including higher screen resolutions, smaller relative target sizes, and complex interfaces. The benchmark comprises 1,581 authentic high-resolution screenshots spanning 23 professional applications across 5 industries (Development, Creative, CAD, Scientific, and Office) and 3 operating systems, each annotated with precise bounding boxes for target UI elements and corresponding natural language instructions.