Deep reinforcement learning-based composite path planning with key path points

This article proposes a robot path planning method that combines Voronoi diagrams and deep reinforcement learning. In terms of global planning, a “safe zone” is constructed through Voronoi diagrams, and path points are optimized using the A* algorithm and a critical path point extraction algorithm to provide objectives for local planning. During the local planning period, we propose the SW-RDQN (stage-key & weight-balanced replay Rainbow deep Q-network) algorithm, which improves navigation adaptability in complex dynamic environments through three key designs. First, a multi-stage path segmentation strategy is employed to partition and prune the global path, extracting and preserving critical key-point information. Second, a reward function is designed by integrating predicted trajectory landing points with an environmental potential field, guiding the agent to learn safer and more effective obstacle-avoidance behaviors. Third, we adopt a prioritized experience replay scheme with dynamically weighted sampling based on rewards and TD errors. By applying offsetting and logarithmic normalization, the scheme increases the sampling probability of critical experiences, thereby improving sample efficiency. The state input integrates a convolutional layer and a multi-layer perceptron to extract information, takes several scene frames, sends them to a long short-term memory model, and finally makes decisions using a dense layer, thereby improving the robot's perception and response capabilities in dynamic environments. This method effectively improves the path planning accuracy and robustness of robots in dynamic environments.