Deep Q-learning-Based AGV Scheduling Method for Automated Container Terminals with Horizontal Layout Design

With the increasing demand for port efficiency and automation, horizontal layout design has emerged as a cost-effective transition mode from traditional to automated container terminals (ACTs). However, the complex topological structure of horizontal layouts poses significant challenges to AGV scheduling, including longer transportation distances and task dependencies. This paper addresses the real-time AGV scheduling problem in horizontal-layout ACTs by proposing a novel Deep Reinforcement Learning with Advantage and Value Network (DRL-AVN) method. The DRL-AVN architecture integrates a self-attention-based feature extraction network to capture spatial-temporal relationships between tasks, an advantage network for policy optimization, and a value network for state evaluation. This framework enables efficient decision-making under conditions while minimizing makespan. A series of numerical experiments are conducted are to validate the performance of the proposed method. The results indicate that the proposed DRL-AVN approach exhibits remarkable generalization capability across varying task scales, demonstrating particular efficacy in large-scale terminal operations.