The average completion time of each method.

The H-beam riveting and welding work cell is an automated unit used for processing H-beams. By coordinating the gripping and welding robots, the work cell achieves processes such as riveting and welding stiffener plates, transforming the H-beam into a stiffened H-beam. In the context of intelligent manufacturing, there is still significant potential for improving the productivity of riveting and welding tasks in existing H-beam riveting and welding work cells. In response to the multi-agent system of the H-beam riveting and welding work cell, a recurrent multi-agent proximal policy optimization algorithm (rMAPPO) is proposed to address the multi-agent scheduling problem in the H-beam processing. The algorithm employs recurrent neural networks to capture and process historical information. Action masking is used to filter out invalid states and actions, while a shared reward mechanism is adopted to balance cooperation efficiency among agents. Additionally, value function normalization and adaptive learning rate strategies are applied to accelerate convergence. This paper first analyzes the H-beam processing flow and appropriately simplifies it, develops a reinforcement learning environment for multi-agent scheduling, and applies the rMAPPO algorithm to make scheduling decisions. The effectiveness of the proposed method is then verified on both the physical work cell for riveting and welding and its digital twin platform, and it is compared with other baseline multi-agent reinforcement learning methods (MAPPO, MADDPG, and MASAC). Experimental results show that, compared with other baseline methods, the rMAPPO-based agent scheduling method can reduce robot waiting times more effectively, demonstrate greater adaptability in handling different riveting and welding tasks, and significantly enhance the manufacturing efficiency of stiffened H-beam.

H型钢铆焊工作单元是用于H型钢加工的自动化单元。通过协同夹持机器人与焊接机器人，该工作单元可完成加强筋板铆焊等工序，将常规H型钢加工为加强型H型钢。在智能制造背景下，现有H型钢铆焊工作单元的铆焊作业生产效率仍存在较大提升空间。针对H型钢铆焊工作单元的多智能体系统，本文提出循环多智能体近端策略优化算法（recurrent Multi-Agent Proximal Policy Optimization, rMAPPO）以解决H型钢加工中的多智能体调度问题。该算法采用循环神经网络捕获并处理历史信息，通过动作掩码筛除无效状态与动作，并采用共享奖励机制平衡智能体间的协作效率。此外，还引入价值函数归一化与自适应学习率策略以加速算法收敛。本文首先分析H型钢加工流程并进行合理简化，构建面向多智能体调度的强化学习环境，应用rMAPPO算法开展调度决策。随后，在实体铆焊工作单元及其数字孪生平台上验证了所提方法的有效性，并与其他多智能体强化学习基线方法（MAPPO、MADDPG及MASAC）进行对比。实验结果表明，相较于其他基线方法，基于rMAPPO的智能体调度方法可更有效地缩短机器人等待时长，在处理不同铆焊任务时具备更强的适应性，且可显著提升加强型H型钢的制造效率。