Investigating the Dimensions of Spatial Language: A Preliminary Study

In order to collect rich data on spatial prepositions, we have created a framework for gathering user annotations in 3D virtual environments. The framework is built using the Blender 3D modelling software and game engine. The framework initially provides two distinct tasks with which our preliminary study was conducted — a Selection Task and a Description Task. In both tasks, participants are given a first person view of a scene which they can navigate using the mouse and keyboard. In the Selection Task participants are given a preposition on screen and asked to select all figure-ground pairs in the scene which fit the preposition. Once they have selected all pairs they believe to be admissible they are shown another preposition and asked to repeat the process. In our preliminary study, we limited this task to ‘in’, ‘inside’, ‘on’, ‘on top of’, ‘against’, ‘over’, ‘under’, ‘above’ and ‘below’. In the Description Task objects are highlighted and participants are able to type in a spatial description of the object. In our preliminary study we asked participants to give descriptions of the object locations using a definite description, in the format figure + preposition + ground e.g. ‘the guitar by the bookshelf’. We also allowed the use of multiple prepositions if the participant deemed it necessary e.g. ‘the cup on the table near the lamp’. To make the annotations more relevant to our research we asked participants to only use the prepositions in the Selection Task plus ‘to the right of’, ‘to the left of’, ‘in front of’, ‘behind’, ‘near’, ‘next to’, ‘at’. The current dataset is from a session we hosted in one of our computing labs where we invited participants from across the university campus to take part. We asked participants to complete either the Selection Task or Description Task in one of our scenes. Participants were given a brief introduction to the study and instructions on how to complete the given task along with explanations of key terminology. All participants were asked to create annotations for ten minutes. 23 native English speakers participated, of which 13 performed the Selection Task and 10 performed the Description Task.为收集空间介词相关的丰富数据，我们搭建了一套可在3D虚拟环境中采集用户标注的研究框架。该框架基于Blender 3D建模软件与游戏引擎开发。本框架最初设置两项独立任务以开展预实验——选择任务与描述任务。两项任务中，参与者均可通过鼠标与键盘操控视角，获得场景的第一人称浏览体验。在选择任务中，屏幕将向参与者展示一个空间介词，要求其选中场景中所有符合该介词的图形-背景对（figure-ground pairs）。待参与者确认所有符合要求的配对后，系统将展示下一个介词，重复上述流程。在本次预实验中，我们限定使用的介词为"in"、"inside"、"on"、"on top of"、"against"、"over"、"under"、"above"与"below"。在描述任务中，目标物体将被高亮标记，参与者可输入该物体的空间描述文本。本次预实验要求参与者以限定性描述格式给出物体位置，格式为「图形+介词+背景」，例如"the guitar by the bookshelf"。若有必要，我们同时允许参与者使用多个介词组合，例如"the cup on the table near the lamp"。为使标注结果更贴合研究目标，我们要求参与者仅使用选择任务中限定的介词，以及"to the right of"、"to the left of"、"in front of"、"behind"、"near"、"next to"与"at"。本次公开的数据集源自我们在计算机实验室举办的一场数据采集活动，我们邀请了校园内的各院系参与者参与实验。参与者需在给定场景中完成选择任务或描述任务其中一项。实验开始前，我们向参与者简要介绍了研究背景，讲解了任务完成方法与关键术语的含义。所有参与者均需进行十分钟的标注工作。本次实验共有23名以英语为母语的参与者，其中13人完成了选择任务，10人完成了描述任务。