Dynamic robotic tracking of underwater targets using reinforcement learning

To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new approaches and techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study is to investigate the use of deep reinforcement learning as a tool for ran..., , Deep Reinforcement Learning methods for Underwater Target Tracking
This is a set of tools developed to train an agent (and multiple agents) to find the optimal path to localize and track a target (and multiple targets).
The deep Reinforcement Learning (RL) algorithms implemented are:

Deep Deterministic Policy Gradient (DDPG)
Twin-Delayed DDPG (TD3)
Soft Actor-Critic (SAC)

The environment to train the agents is based on the OpenAI Particle.
The main objective is to find the optimal path that an autonomous vehicle (e.g. autonomous underwater vehicles (AUV) or autonomous surface vehicles (ASV)) should follow in order to localize and track an underwater target using range-only and single-beacon algorithms. The target estimation algorithms implemented are based on:

Least Squares (LS)
Particle Filter (PF)

More information at this Github repository: https://github.com/imasmitja/RLforUTracking

为充分发挥自主水下机器人的潜能、扩充海洋观测能力，亟需研发全新的研究方法与技术手段。多机编队的自主水下机器人可通过新型成像配置，或追踪佩戴标记的海洋动物以研究其行为，助力复杂海洋系统与海洋生物的相关研究。此类研究成果可为海洋群落保护政策的制定提供参考与支撑。当前，关于海洋动物通过主动移动实现种群连通性的研究，仍是深海种群分布相关知识体系中的重大空白。在原位观测海洋生物过程中，水下目标追踪是一项核心挑战，同时亟需研发可在监测任务中稳健应对动态变化环境的技术方法。针对水下目标定位的最优传感器布局与路径规划分析技术，在该场景下尚无成熟可行的解决方案。本研究旨在探索深度强化学习（Deep Reinforcement Learning, DRL）作为技术工具的应用潜力……，即面向水下目标跟踪的深度强化学习方法。
本数据集为一套用于训练智能体（单智能体及多智能体）以寻得最优航行路径，从而实现水下目标（单目标及多目标）定位与跟踪的工具集。本项目所实现的深度强化学习（Deep Reinforcement Learning, RL）算法包括：深度确定性策略梯度（Deep Deterministic Policy Gradient, DDPG）、双延迟深度确定性策略梯度（Twin-Delayed DDPG, TD3）以及软演员评论家（Soft Actor-Critic, SAC）。
用于训练智能体的仿真环境基于OpenAI Particle环境。本项目的核心目标是，为自主航行器（例如自主水下航行器（Autonomous Underwater Vehicle, AUV）或自主水面航行器（Autonomous Surface Vehicle, ASV））寻得最优航行路径，使其可通过仅测距与单信标算法完成水下目标的定位与跟踪。所采用的目标估计算法基于最小二乘法（Least Squares, LS）与粒子滤波（Particle Filter, PF）。
更多详细信息可访问该GitHub仓库：https://github.com/imasmitja/RLforUTracking