Numerical framework for multi-stage control of surface polishing

We consider the automation of polishing process for manufactured components, which is typically an iterative, multi-stage process that depends heavily on the practitioner’s expertise and visual inspection to guide decisions on polishing pad changes and fine-tuning of control parameters. We use a model-free, on-policy actor-critic reinforcement learning (RL) algorithm to determine the choice of pad, downforce, rotational speed, polishing duration for each stage, and the total number of polishing / inspection stages. The reward structure includes both dense and sparse components, optimizing for surface finish quality with minimal energy consumption, while also considering the cost implications of each polishing step and minimizing the risk of scratches and defects. The surface dynamics is captured by a stochastic model, serving as a digital twin, that incorporates material removal and redistribution during polishing of 3D printed materials. We pose the dynamic decision-making problem as a constrained Markov Decision Process (MDP) with the primary constraint to reach a desired surface roughness level. A key challenge addressed is the lack of real-time micro-level feedback during polishing, necessitating a multi-stage process. Between stages, the RL controller can only measure the surface roughness value, and adjust the control variables accordingly. The parameters of the digital twin are chosen to reflect the material properties of 3D-printed Ti-6Al-4V dental implants. Results suggest that higher downforce and lower rotational speed minimize energy consumption, in addition to emphasizing the critical role of the number of polishing steps and choice of pad in achieving precise endpoint criteria.