Brain-in-the-Loop Learning for Intelligent Vehicle Decision-Making

<br>The inflexible human-autonomy relationship within autonomous driving scenarios still has not realized deep intelligent synergy, therefore unable to provide adaptive and context-sensitive decision-making and sometimes leading to violation of human preferences or even hazards. In this paper, we utilize functional near-infrared spectroscopy (fNIRS) signals as real-time human risk-perception feedback to establish a brain-in-the-loop (BiTL) trained artificial intelligence algorithm for decision-making. The proposed algorithm uses the result of driving risk reasoning as one input of reinforcement learning combining fNIRS-based risk and driving safety field model-based risk, realizing integrating human brain activity into the reinforcement learning scheme, then overcoming the disadvantage of machine-oriented intelligence that could violate human intentions. To achieve policy learning within limited BiTL training periods, we add two modification features to the proposed algorithm based on TD3. The experiment involving twenty participants has been conducted, and the results show that in continuously high-risk driving scenarios, compared to traditional reinforcement learning algorithms without human participation, the proposed algorithm can maintain a cautious driving policy and avoid potential collisions, validated with both proximal surrogate indicators and success rates. This repository contains the experimental dataset and Python code to reproduce the experimental results used in our research on 'Brain-in-the-Loop Learning for Intelligent Vehicle Decision-Making'. Both human subject studies, control groups and ablation studies data are included in this repository. Detailed description of file organization, data structures, requirements could be found in the README.md document.

自动驾驶场景下僵化的人机自主关系尚未实现深度智能协同，既无法提供自适应且感知上下文的决策能力，有时还会违背人类偏好乃至引发安全隐患。针对该问题，本文采用功能近红外光谱（functional near-infrared spectroscopy, fNIRS）信号作为实时人类风险感知反馈，构建了脑在回路（brain-in-the-loop, BiTL）训练的人工智能决策算法。所提算法将驾驶风险推理结果作为强化学习（reinforcement learning）的输入之一，结合基于fNIRS的风险评估与基于驾驶安全场模型的风险评估，实现将人类脑活动融入强化学习框架，从而克服传统面向机器的智能易违背人类意图的缺陷。为在有限的BiTL训练周期内实现策略学习，本文基于TD3算法新增了两项改进特性。本文开展了包含20名参与者的实验，结果表明，相较于传统无人类参与的强化学习算法，所提算法在持续高风险驾驶场景中能够保持谨慎的驾驶策略并规避潜在碰撞，该结论通过近端替代指标与成功率两项指标均得到了验证。本仓库包含了本研究《面向智能车辆决策的脑在回路学习》（Brain-in-the-Loop Learning for Intelligent Vehicle Decision-Making）所用的实验数据集与复现实验结果的Python代码。仓库中涵盖了人类被试研究、对照组实验与消融实验的全部数据。文件组织、数据结构与依赖要求的详细说明可参见README.md文档。