Data of an exploratory user study comparing interaction techniques in different Virtual Reality scenarios

<strong>Motivation</strong> This data was collected during a user study where multiple interaction techniques were compared in different Virtual Reality scenarios. Twelve interaction techniques were evaluated in selection and manipulation tasks. Not all evaluated techniques are capable of both selection and manipulation. Therefore, we tested eight techniques in the manipulation sub-study and ten in the selection sub-study. The twelve tested interaction techniques were Bimanual Fishing Reel, Expand, Flashlight, Go-Go + PRISM, Head Based Selection, IntenSelect, Scaled HOMER, Scaled HOMER + Scale, Simple Virtual Hand, Spindle, Scaled Scrolling World in Miniature + Scale, and Crank Handle (Controller). For details on the techniques, we refer to S3DIT (10.5281/zenodo.4292312), a web application for finding suitable interaction techniques. The Hardware Setup consisted of an HTC Vive Pro with the corresponding controller and an Alienware 17 R4 (Intel i7-7700HQ, NVIDIA GTX 1070, 16 GB Ram). Unity was used to implement the test environment (10.5281/zenodo.4172112). <strong>Tasks</strong> In the selection task, a red ball must be selected (see <em>images/task_selection_1.png</em>), whereupon a structure of one or more balls appears, with the target colored green (see <em>images/ task_selection_2.png</em>). Regardless of the interaction technique used, the ball hit by the selection tool is highlighted with a yellow border to increase the comparability of the techniques. The time measurement started as soon as the red ball was selected and stopped as soon as the correct ball was selected. In the manipulation sub-study, the task was to position, rotate and scale a cube with six different colored areas in such a way that it fits as closely as possible into a transparent representation of the cube (see <em>images/task_manipulation.png</em>). The position may differ 10 cm, the rotation 20 degrees, and the size of the cube 10 cm from the values of the transparent cube. The solution of the task must be actively initiated via the grip button of the Vive Controller. If an attempt was made to solve the task although the cube was not yet within the tolerances, nothing happened. After tests in a preliminary study, the cube was extended with two bars to provide additional spatial indicators. Besides, there were cones of different colors centrally under the objects, which should help to better estimate the distance between the objects. The time measurement started as soon as the object was gripped and stopped as soon as the solution was accepted. Regardless of the task type, the test persons had 30 seconds to complete the task successfully. If this was not possible, this counted as a failed attempt, the current task structure disappeared and the next task was displayed. If a task was solved correctly, a small animation with stars was triggered, accompanied by an acoustic signal and the next task was displayed. In both task types, the objects were created relative to the position of the head, so that each test person could view the objects from the same perspective. For each combination of independent variables (without technique), two tasks were randomly generated for manipulation and three for selection and presented in random order. This resulted in 81 tasks for selection and 90 tasks for manipulation. For further details on the tasks used, we refer to the repository of the testbed application (10.5281/zenodo.4172112). The tasks are run through randomly, with some restrictions, as not every technique supports every task. Some techniques only allow interaction at short distances or do not support the scaling of objects. For this reason, dummy tasks have been generated to replace tasks that are not supported by the techniques. These always come at the end to allow better comparability for the supported tasks, because then the dummy tasks do not act as training. Thus, in selection and manipulation, short distance tasks came first for each technique, and in manipulation, tasks that allow positioning and rotation came first. Due to the many techniques that were compared, no Within-Subject Design was possible, and not every technique was tested by every test person. Also, a Within-Subject Design was not possible, because a much higher number of test persons would have been necessary. For these reasons, an intermediate design was chosen. Since the tasks could be carried out more quickly during selection, five techniques per test subject were tested here and three techniques per test subject during manipulation. The techniques were randomly assigned but evenly distributed. <strong>Dependent and independent variables</strong> The independent variables for the selection task were distance (0.6 m, 3 m, and 6 m), object size (15 cm, 10 cm, and 5 cm), and object density (single, 10, or 5 cm between objects). For the manipulation task, the independent variables were distance (0.6 m, 3 m, and 6 m), task type (positioning, rotation, scaling, positioning + rotation or positioning + rotation + scaling), and degree of manipulation (low, medium or high). The degree of manipulation influences the number of axes on which the object must be moved and to which extent the manipulation had to be performed. As none of the techniques allows asymmetric scaling, the enlargement or reduction of objects always took place on all axes. Speed and precision were recorded as objective measurements. The speed is determined by the time required to perform the task. In the case of selection, the precision results from the number of failures, i.e. how often the wrong object or no object was selected until the correct object was selected. In the case of manipulation, the difference to the target object for the position, rotation, and size of the object is recorded individually to determine the achieved precision. The System Usability Scale was used to determine usability. Besides, custom questions were asked to capture the additional factors naturalness, fun, precision, speed, and motion sickness. The NASA Task Load Index (TLX) was used to record the subjective effort involved in performing the tasks. The test persons could make additional comments via a comment field. After all techniques had been tested by a test person, they were also asked to sort the techniques according to their own preferences. It is important to note that the test subjects did not always necessarily use techniques that could perform the same tasks. When evaluating the techniques, it was therefore important to independently weigh up various factors such as usability and expressiveness of the techniques. <strong>Test Procedure</strong> Initially, the test person filled in a personal questionnaire (<em>data/demographic_data_[eng|ger].csv</em>), asking for demographic data (age, handiness, gender, wearer of glasses and occupational field) and the level of experience with certain devices and input methods (computer, mouse, touch screen, body recognition, finger recognition, eye tracking and 3D controller), as well as certain applications (3D applications, 3D computer games and VR/AR applications). A further document, which had to be read by the test persons on site, described the task to be performed in detail and explained the necessary buttons of the controllers. If the test person had no further questions, he was led into the test area and the VR headset was put on. Even if a technique was tested that works with one hand, the test person was given both controllers, which also had to be held in the hands throughout the entire process. This was followed by a short introduction to the virtual environment. Then the training phase of the first technique began. Here, depending on the complexity of the technique, 1-4 explanatory texts were read out, which were created using Google Text-to-Speech. The speed at which the explanatory texts followed each other could be controlled by the study director. The test subjects then had five minutes to familiarize themselves with the technique in previously generated sample tasks. These sample tasks were similar to the actual tasks. However, the test persons could also say that they wanted to start with the tasks before the time was up. Then the task phase began. For the completed tasks, the required time and failed attempts or the achieved precision was recorded (<em>data/measurements_manipulation_.csv</em> and <em>data/measurements_selection_.csv</em>). Once all the tasks were completed, the test person removed the headset and had to answer the System Usability Scale and the custom questions on a PC (<em>data/system_usability_scale_and_custom_questions_[eng|ger].csv</em>). The questionnaires were implemented using Google Forms. The NASA Task Load Index was filled out in paper form (<em>data/nasa_tlx_[eng|ger].csv</em>). The test person was offered a break and then he continued with the next technique. After all techniques were tested the final questionnaire for ranking the techniques was filled in (<em>data/ranking_[manipulation|selection]_[eng|ger].csv</em>). <strong>Files</strong> The study was done in German. The questions and results can be found in the files ending with “ger”. The English translation can be found in the files ending with “eng”. The files containing the results of the questionnaires include the asked questions and don’t need further explanations. In the following, only the columns of the file <em>measurements_manipulation</em> and <em>measurements_selection</em> are explained: <strong>UserId</strong>: A number identifying the test person. The same number in different files identifies the same test person. <strong>InteractionTechnique</strong>: Name of the interaction technique used. <strong>TaskID</strong>: Internal id of the task. <strong>Type</strong>: Tasks that needed to be executed. <strong>TaskObject</strong>: The used task object. For the selection task, this indicates the size of the object. <strong>Distance</strong>: The distance between the test person and the task object(s). <strong>NumberOfObjects</strong>: The number of task objects. <strong>MinDensity</strong>: The minimum distance between two task objects. <strong>NeededDoFs</strong>: The number of axes on which the object needs to be manipulated (1=x, 2=x, and y, 3= x,y, and z). <strong>ManipulationAmount</strong>: The amount the object needs to be changed in position (1 = 1/3 m, 2 = 2/3 m, 3 = 1 m), rotation (1 = 45-90°, 2 = 90-135°, 3 = 135-180°) or scale (1 = factor 1.25-1.5, 2 = factor 1.5-1.75, 3 = factor 1.75-2). <strong>Success</strong>: Depicts whether the test person was able to successfully finish the task within the time limit. <strong>Time</strong>: The needed time. <strong>Misses</strong>: Number of wrong selections until the correct object was selected <strong>PositionDifference</strong>: The position difference between the task object and the target object at the end of the task. <strong>RotationDifference</strong>: The rotation difference between the task object and the target object at the end of the task. <strong>ScaleDifference</strong>: The scale difference between the task object and the target object at the end of the task.<strong>研究动机</strong> 本数据集源自一项用户研究，该研究在不同虚拟现实（Virtual Reality，VR）场景中对比了多种交互技术。研究共评估了12种交互技术在选择与操作任务中的表现。并非所有被评估的技术均可同时支持选择与操作两类任务，因此操作子研究中共测试8种技术，选择子研究中共测试10种技术。本次测试的12种交互技术包括：双手钓鱼卷轴（Bimanual Fishing Reel）、扩展式（Expand）、手电筒式（Flashlight）、Go-Go + PRISM、头部基选择（Head Based Selection）、IntenSelect、缩放式HOMER（Scaled HOMER）、缩放式HOMER + 缩放（Scaled HOMER + Scale）、简易虚拟手（Simple Virtual Hand）、纺锤式（Spindle）、缩放式微型滚动世界（Scaled Scrolling World in Miniature） + 缩放，以及曲柄手柄控制器（Crank Handle (Controller)）。如需了解这些技术的详细信息，可参考S3DIT（10.5281/zenodo.4292312）——一款用于筛选适配交互技术的Web应用。本次实验的硬件配置包括：搭载对应控制器的HTC Vive Pro头显，以及Alienware 17 R4笔记本电脑（配置为Intel i7-7700HQ处理器、NVIDIA GTX 1070显卡、16GB内存）。实验测试环境基于Unity引擎开发（10.5281/zenodo.4172112）。 <strong>实验任务</strong> 在选择任务中，被试需先选中一个红色小球（详见<em>images/task_selection_1.png</em>），随后会出现由一个或多个小球组成的结构，其中目标小球为绿色（详见<em>images/task_selection_2.png</em>）。无论使用何种交互技术，被选择工具命中的小球都会以黄色边框高亮显示，以提升不同技术间的可比性。计时将在红色小球被选中时启动，并在正确目标小球被选中时停止。在操作子研究中，实验任务为：将一个带有六个不同颜色面的立方体进行位移、旋转与缩放操作，使其尽可能贴合透明的目标立方体模型（详见<em>images/task_manipulation.png</em>）。允许的误差范围为：位移偏差不超过10cm，旋转偏差不超过20°，立方体尺寸偏差不超过10cm。任务完成需通过Vive控制器的握持按钮主动触发。若未达到误差范围就尝试提交任务，则系统无响应。经预实验测试后，我们为立方体增设了两根辅助杆以提供额外空间参考；同时在物体下方中心位置设置了不同颜色的圆锥体，辅助被试估算物体间的距离。计时将在物体被握持时启动，并在任务提交被确认时停止。无论任务类型如何，每位被试均有30秒时间成功完成当前任务，若超时未完成则计为失败，当前任务界面会消失并跳转至下一个任务。若任务成功完成，则会触发带有星型特效的动画，并伴随音效提示，随后跳转至下一个任务。两类任务中的所有物体均基于被试的头部位置生成，确保每位被试均能以相同视角观察物体。针对每组自变量组合（不包含交互技术），操作任务将随机生成2个，选择任务随机生成3个，并以随机顺序呈现。最终共生成81个选择任务与90个操作任务。如需了解实验任务的详细信息，可参考测试平台应用的代码仓库（10.5281/zenodo.4172112）。任务会以随机顺序执行，但存在部分限制：并非所有技术都支持所有任务类型，部分技术仅支持近距离交互，或不支持物体缩放操作。因此我们生成了占位任务以替代当前技术不支持的任务，这类占位任务始终置于任务序列末尾，避免其起到训练作用，从而保证有效任务间的可比性。因此，在选择与操作任务中，所有技术均先执行近距离任务；在操作任务中，支持位移与旋转的任务会优先呈现。由于本次实验对比的技术数量较多，无法采用被试内设计（Within-Subject Design），并非每位被试都会测试所有技术。同时，采用被试内设计也需要更多的被试样本，因此我们选择了折中实验设计。由于选择任务的完成速度更快，每位被试需测试5种交互技术；而操作任务每位被试测试3种技术。交互技术的分配为随机，但整体分布均匀。 <strong>因变量与自变量</strong> 选择任务的自变量包括：距离（0.6m、3m、6m）、物体尺寸（15cm、10cm、5cm）以及物体密度（单个物体、物体间距10cm、物体间距5cm）。操作任务的自变量包括：距离（0.6m、3m、6m）、任务类型（位移、旋转、缩放、位移+旋转，或位移+旋转+缩放）以及操作程度（低、中、高）。操作程度会影响物体所需调整的轴数，以及操作的幅度。由于所有技术均不支持非对称缩放，因此物体的放大或缩小始终在所有坐标轴上同步进行。我们记录了两项客观指标：完成速度与操作精度。完成速度由任务的耗时决定。对于选择任务，精度通过失败次数计算，即选中正确目标前，误选其他物体或未选中任何物体的次数。对于操作任务，我们会分别记录物体位移、旋转与尺寸与目标值的偏差，以此计算最终的操作精度。我们采用系统可用性量表（System Usability Scale, SUS）评估实验的易用性；同时通过自定义问卷收集自然度、趣味性、精度、速度与晕动症这几项额外维度的评价。NASA任务负荷指数（NASA Task Load Index, TLX）用于记录被试完成任务时的主观认知负荷。被试可通过备注栏补充额外反馈。在一位被试完成所有技术的测试后，我们会请其根据自身偏好对所有交互技术进行排序。需要注意的是，被试所测试的技术未必均支持当前任务类型，因此在评估技术时，需综合考量易用性、表现力等多维度因素，独立进行权衡。 <strong>实验流程</strong> 实验初始阶段，被试需填写个人基本问卷（<em>data/demographic_data_[eng|ger].csv</em>），问卷内容包括人口统计学信息（年龄、利手、性别、是否佩戴眼镜、职业领域），以及对各类设备与输入方式（电脑、鼠标、触摸屏、身体识别、手指识别、眼动追踪与3D控制器）和特定应用（3D应用、3D电脑游戏与VR/AR应用）的使用经验等级。此外，被试需现场阅读一份说明文档，详细介绍实验任务内容与控制器各按钮的功能。若被试无疑问，则引导其进入实验区域并佩戴VR头显。即便测试的技术仅支持单手握持，被试仍会获得两个控制器，并需在整个实验过程中双手握持。随后，实验人员会简要介绍虚拟实验环境。接下来开始第一个交互技术的训练阶段：根据技术的复杂程度，实验人员会通过Google文本转语音工具播放1-4段讲解文本，讲解文本的播放速度可由实验负责人控制。被试有5分钟时间，通过预先生成的样例任务熟悉当前交互技术，样例任务与正式任务结构相似；但若被试认为已熟悉技术，也可提前结束训练进入正式任务阶段。正式任务阶段随即开始。被试完成的任务数据（包括耗时、失败次数与操作精度）将被记录至<em>data/measurements_manipulation_.csv</em>与<em>data/measurements_selection_.csv</em>文件中。所有任务完成后，被试摘下头显，需在PC上填写系统可用性量表与自定义问卷（<em>data/system_usability_scale_and_custom_questions_[eng|ger].csv</em>），问卷基于Google Forms开发。NASA任务负荷指数问卷采用纸质版填写（<em>data/nasa_tlx_[eng|ger].csv</em>）。随后被试可休息片刻，再开始下一个交互技术的测试。当一位被试完成所有交互技术的测试后，需填写最终的技术排序问卷（<em>data/ranking_[manipulation|selection]_[eng|ger].csv</em>）。 <strong>数据文件说明</strong> 本研究以德语开展，问卷题目与实验结果可在后缀为"ger"的文件中查看；英文翻译版本则位于后缀为"eng"的文件中。包含问卷结果的文件已内置原题，无需额外说明。下文仅对<em>measurements_manipulation</em>与<em>measurements_selection</em>两个数据文件的列字段进行说明： <strong>UserId</strong>：被试标识编号，不同文件中相同编号代表同一位被试。 <strong>InteractionTechnique</strong>：本次测试使用的交互技术名称。 <strong>TaskID</strong>：实验任务的内部编号。 <strong>Type</strong>：需执行的任务类型。 <strong>TaskObject</strong>：实验使用的任务物体，对于选择任务，该字段表示物体的尺寸。 <strong>Distance</strong>：被试与任务物体之间的距离。 <strong>NumberOfObjects</strong>：任务中包含的物体数量。 <strong>MinDensity</strong>：两个任务物体之间的最小间距。 <strong>NeededDoFs</strong>：物体操作所需的轴数（1代表仅X轴，2代表X轴与Y轴，3代表X、Y、Z三轴）。 <strong>ManipulationAmount</strong>：物体的操作幅度，其中位移幅度分为3档：1=1/3m、2=2/3m、3=1m；旋转幅度分为3档：1=45°~90°、2=90°~135°、3=135°~180°；缩放幅度分为3档：1=缩放因子1.25~1.5、2=缩放因子1.5~1.75、3=缩放因子1.75~2。 <strong>Success</strong>：标识被试是否在时限内成功完成任务。 <strong>Time</strong>：任务完成所需的耗时。 <strong>Misses</strong>：选中正确目标前，误选物体或未选中物体的总次数。 <strong>PositionDifference</strong>：任务结束时，操作后物体与目标物体的位移偏差。 <strong>RotationDifference</strong>：任务结束时，操作后物体与目标物体的旋转偏差。 <strong>ScaleDifference</strong>：任务结束时，操作后物体与目标物体的尺寸缩放偏差。