Obtaining emergent behaviors for swarm robotics singling with deep reinforcement learning

Isolating (singling) an individual from a group can be essential for protection, rescue or capture tasks. In this paper a system with multiple shepherds who must coordinate the sheep to achieve a specific singleness is proposed. We present a realistically modeled system that will be finally tested in a real robotic system. We want to encourage the adaptability of the system and provide different solutions by promoting the emergence of the swarm. In this line we will focus on the use of reinforcement learning, avoiding a manual design of the behavior in order to not restrict the resulting behaviors and to facilitate their adaptation. A detailed MDP model will be specified as well as the keys to reduce its dimensionality and facilitate its training. We will check the results of the obtained singling policy with respect to a greedy policy and focus on evaluating different behavioral strategies that can solve the problem in different ways. In addition, one of the obtained policies will be analyzed in detail to check both its robustness and its scalability with respect to the number of shepherds and sheep. This policy will be finally tested on a physical robotic swarm.

将单个个体从群体中分离（即单独隔离），在防护、营救或捕获任务中至关重要。本文提出一种由多台牧羊机器人组成的系统，该系统需协调被牧羊群体以实现特定的个体分离目标。本文构建了具备真实物理建模的系统，最终将在实体机器人平台上开展测试验证。我们旨在通过促进集群智能的涌现，提升系统的适应性，并提供多样化的解决方案。基于此研究思路，本文将聚焦于强化学习（reinforcement learning）的应用，避免手动设计个体行为，以此避免限制最终涌现的行为模式，并便于其适配不同场景。本文将详细指定马尔可夫决策过程（Markov Decision Process, MDP）模型，并阐述降低模型维度、简化训练流程的关键方法。我们将对比贪婪策略与所习得的分离策略的效果，并重点评估可通过不同路径解决该问题的各类行为策略。此外，本文将对其中一项习得策略展开详细分析，验证其在牧羊机器人与被牧羊个体数量变化时的鲁棒性与可扩展性。该策略最终将在实体机器人群体平台上进行实测验证。