SpokenVisIT

该数据集是一个用于评估视觉基础语音交互能力的基准，它基于现实世界的视觉问答基准VisIT构建而成。在评估过程中，我们采用GPT模型（gpt-4o版本）来对回应进行评分，所有的语音评估被划分为语音转文本和语音转语音两种设置。该任务旨在推动视觉基础下的语音交互技术发展。

This dataset serves as a benchmark for evaluating visual-grounded speech interaction capabilities, built upon the real-world visual question answering benchmark VisIT. During the evaluation workflow, we utilized the GPT model (specifically the gpt-4o variant) to score generated responses, with all speech evaluation tasks categorized into two settings: speech-to-text and speech-to-speech. This task is designed to drive the advancement of visual-grounded speech interaction technologies.