Receding-horizon direct policy optimization with Koopman embedding for trajectory tracking of fixed-wing UAVs

Trajectory tracking is critical for fixed-wing unmanned aerial vehicles (UAVs) to address operational requirements in complex environments. However, deriving computationally efficient optimal control solutions for such a task remains challenging due to the complexity and uncertainty of UAV dynamics. This paper proposes a receding-horizon direct policy optimization (RHDPO) approach with Koopman embedding for trajectory tracking of fixed-wing UAVs. First, a data-driven Koopman model of fixed-wing UAVs is built, which establishes a globally linearized representation of their dynamics. Building on this, an RHDPO algorithm is proposed to achieve efficient control policies. By incorporating a receding-horizon mechanism, our method solves infinite-horizon control problems via direct policy optimization within finite prediction horizons. Unlike conventional model predictive control (MPC), it generates an explicit closed-loop control policy via online multi-step policy learning without relying on numerical solvers. The closed-loop policy can be successively optimized in a receding-horizon manner, enabling rapid and efficient learning. Moreover, unlike actor-critic architectures in reinforcement learning, our approach directly optimizes control policies with respect to the cost function over prediction horizons, eliminating auxiliary value function approximation and accounting for state propagation effects within the prediction horizon. Simulation results demonstrate that our approach shows a significant improvement in tracking performance over model predictive control and receding-horizon dual heuristic programming.