Supplementary data for the paper 'AR-based telepresence in a robotic manipulation task: An experimental evaluation'

The current study investigated a spectrum of control methods in human-robot interaction, ranging from direct control to telepresence with a virtual representation of the robot arm. Twenty-four participants used a setup comprising a Franka Emika Panda robot arm, Varjo XR-3 head-mounted display, and Leap Motion Controller. Participants performed a box-and-block task with the bare hand (A), and under five gesture-controlled robotic operation methods: direct sight (B), sight via video-feedthrough (C), in a 3D telepresence environment with (D) and without (E) virtual representation of the robot arm, and using a 2D video feed (F). The number of grabbing attempts did not differ significantly between conditions, but local operation (B & C) yielded more transferred blocks than teleoperation (D–F). Teleoperation using a 3D presentation was advantageous compared to using a 2D video feed, as demonstrated by lower peak forces and smaller range in gripper heights compared to the 2D video feed, findings supported by analyses of head movement activity. Finally, the bare hand yielded the best performance and subjective ratings. In summary, teleoperation using a 3D presentation provided a smoother interaction than a 2D video feed. However, direct human interaction remains a benchmark yet to surpass.

本研究针对人机交互领域的多类控制方法展开探究，覆盖从直接控制到搭载机械臂虚拟化身的遥临场感（telepresence）交互等多种范式。24名受试者参与了本次实验，实验平台包含Franka Emika Panda机械臂、Varjo XR-3头戴式显示器以及Leap Motion控制器。受试者分别完成裸手操作下的箱块任务（标记为A），以及五种手势控制的机械臂操作任务：直接目视操控（B）、通过视频透视的目视操控（C）、带机械臂虚拟化身的3D遥临场环境操控（D）、不带机械臂虚拟化身的3D遥临场环境操控（E），以及2D视频流操控（F）。不同实验条件下的抓取尝试次数无显著差异，但本地操作组（B与C）的方块转移数量显著多于遥操作组（D至F）。相较于2D视频流操控，采用3D呈现的遥操作具备显著优势：其夹持器的峰值作用力更低、高度调节范围更小，该结论通过头部运动活动分析得到验证。最后，裸手操作的任务表现与主观评分均为最优。综上，采用3D呈现的遥操作交互流畅度优于2D视频流操控，但人类直接交互仍是尚未被超越的性能基准。