Zelda Smart Home Prototype

This is the avatar and animation set created for the Auslan Communication Technologies Pipeline project. It is the core of the demo presented in Zelenskaya et al., 2023, however the virtual smart home included in the demo was removed in this prototype, as it was created using third party assets. This prototype contains: 1) An Unreal Engine project with the Zelda avatar as it was used in the demo; 2) Models and scripts for recognising emblematic gestures inspired by Auslan signs, disambiguating from context, assembling a command, executing the command, and generating an appropriate response from the avatar; 3) Instructions on setting up SplitCam to use a single webcam as input in two locations; 4) Instructions on connecting MediaPipe to Unreal Engine to run the prototype; 5) The required version of Python (3.9); 6) The Unreal Engine project we used to record Auslan signs through motion capture; and 7) Several backup versions of the avatar, and tracking, and HTTP communication, from previous iterations of the prototype.

本数据集为澳大利亚手语（Auslan）通信技术管线项目创制的虚拟形象与动画套件。其为Zelenskaya等人2023年展示的演示原型的核心内容，但由于演示中包含的虚拟智能家居采用第三方素材制作，因此本原型移除了该模块。本原型包含以下内容：1）搭载演示所用塞尔达（Zelda）虚拟形象的虚幻引擎（Unreal Engine）项目；2）用于识别受澳大利亚手语启发的标志性手势、结合上下文进行歧义消解、组合指令、执行指令并为虚拟形象生成适配回应的模型与脚本；3）关于配置SplitCam以实现单摄像头同时向两处位置提供输入的操作指南；4）关于将MediaPipe接入虚幻引擎以运行本原型的配置说明；5）本原型所需的Python（3.9）版本环境；6）我们用于通过动作捕捉录制澳大利亚手语的虚幻引擎项目；7）本原型此前迭代版本中留存的虚拟形象、追踪模块与HTTP通信功能的多个备份版本。