On the Applicability of Brain Reading for Predictive Human-Machine Interfaces in Robotics

The ability of today's robots to autonomously support humans in their daily activities is still limited. To improve this, predictive human-machine interfaces (HMIs) can be applied to better support future interaction between human and machine. To infer upcoming context-based behavior relevant brain states of the human have to be detected. This is achieved by brain reading (BR), a passive approach for single trial EEG analysis that makes use of supervised machine learning (ML) methods. In this work we propose that BR is able to detect concrete states of the interacting human. To support this, we show that BR detects patterns in the electroencephalogram (EEG) that can be related to event-related activity in the EEG like the P300, which are indicators of concrete states or brain processes like target recognition processes. Further, we improve the robustness and applicability of BR in application-oriented scenarios by identifying and combining most relevant training data for single trial classification and by applying classifier transfer. We show that training and testing, i.e., application of the classifier, can be carried out on different classes, if the samples of both classes miss a relevant pattern. Classifier transfer is important for the usage of BR in application scenarios, where only small amounts of training examples are available. Finally, we demonstrate a dual BR application in an experimental setup that requires similar behavior as performed during the teleoperation of a robotic arm. Here, target recognition processes and movement preparation processes are detected simultaneously. In summary, our findings contribute to the development of robust and stable predictive HMIs that enable the simultaneous support of different interaction behaviors.当前机器人自主辅助人类完成日常事务的能力仍存在局限。为改善这一状况，可采用预测性人机交互界面（human-machine interfaces, HMIs）以更好地支撑未来的人机交互活动。若要预判基于当前情境的后续行为，需先检测人类的相关脑状态。这一目标可通过脑读取技术（brain reading, BR）实现：该技术是一种采用监督机器学习（machine learning, ML）方法的单试次脑电图（electroencephalogram, EEG）分析被动方案。本研究提出，脑读取技术可检测交互过程中人类的具体脑状态。为验证这一论点，我们证实脑读取技术可检测脑电图中的特征模式，这些模式可与脑电图中的事件相关脑活动（如P300电位）相关联，而这类活动正是具体脑状态或脑加工过程（如目标识别加工过程）的指示标志。此外，本研究通过筛选并整合适用于单试次分类的最相关训练数据，并采用分类器迁移技术，提升了脑读取技术在面向实际应用场景中的鲁棒性与适用性。我们证实，若两类样本均缺失某一相关特征模式，则可在不同类别间完成分类器的训练与测试（即分类器的实际应用）。分类器迁移技术对于在训练样本稀缺的实际应用场景中使用脑读取技术至关重要。最后，本研究在一项实验范式中展示了脑读取技术的双重应用场景，该实验范式所需的行为与机械臂遥操作过程中的行为相似。在此场景中，目标识别加工过程与运动准备加工过程可被同时检测到。综上，本研究结果有助于开发鲁棒性与稳定性俱佳的预测性人机交互界面，使其能够同时支撑多种交互行为。