Reinforcement Learning for Multiple Gravity Assist Trajectory Optimization

This study presents a reinforcement learning (RL) approach to trajectory optimization. Traditional methods for preliminary trajectory design with multiple gravity assists typically rely on global optimization techniques such as genetic algorithms, particle swarm optimization, and differential evolution. While effective, these methods optimize all design variables simultaneously and hence may, for instance, discard solutions in which only one variable is not optimal. In contrast, RL, formulated within the Markov Decision Process (MDP) framework, allows each transfer arc to be optimized independently, with the next state determined solely by the current state and action. Building on this principle, we propose a RL-based formulation that defines states, rewards, and actions according to astrodynamics, and we implement Proximal Policy Optimization (PPO) to refine individual trajectory segments rather than minimizing the total cost of the entire trajectory at once. The effectiveness of this approach is demonstrated through the successful computation of the Cassini-like trajectory, illustrating both its potential benefits and the challenges that remain for future applications.