SPARTQA (SPAtial Reasoning on Textual Question Answering)

SpartQA是自然语言文本空间推理的文本问题回答基准，其中包含先前数据集未涵盖的更现实的空间现象，并且对于最先进的语言模型 (LM) 具有挑战性。
SPARTQA建立在NLVR的图像上，该图像包含更多具有更丰富空间结构的对象。SPARTQA的故事更自然，句子更多，每个句子的空间关系更丰富，问题需要更深层次的推理，有四种类型: 查找关系 (FR) 、查找块 (FB) 、选择对象 (CO) 和是/否 (YN)，这允许对模型的功能进行更细粒度的分析。
https://aclanthology.org/2021.naacl-main.364/

SpartQA is a textual question answering benchmark for spatial reasoning over natural language texts, which encompasses more realistic spatial phenomena not covered by prior datasets and poses significant challenges for state-of-the-art language models (LMs).
SPARTQA is built upon images from NLVR, which feature more objects with richer spatial structures. The stories in SPARTQA are more natural, with a greater number of sentences and more abundant spatial relations in each individual sentence. The questions demand deeper reasoning and are categorized into four types: Find Relation (FR), Find Block (FB), Choose Object (CO), and Yes/No (YN), which allows for more fine-grained analysis of model capabilities.
https://aclanthology.org/2021.naacl-main.364/

搜集汇总

背景与挑战

背景概述

SPARTQA是一个用于自然语言文本空间推理的问答基准数据集，基于NLVR图像构建，包含更丰富的空间结构和对象，问题涵盖四种类型以支持模型功能细粒度分析。该数据集旨在涵盖现实空间现象，对先进语言模型具有挑战性。