Fig. 4 | Transition of autonomous microrobots from simulation to physical environment.

a. A plot of MBRL demonstrates training of microrobots with 32 actions in an experimental racetrack square channel. The y-axis indicates the reward function, with positive values signalling successful navigation towards the target and negative values indicating crashes. The x-axis displays the number of training steps. The reward drops significantly at around 300k steps due to overfitting to a specific corner of the channel. b. (Top) Image sequences illustrate the experimental training of microrobots at 100k steps of graph a. At this early stage, microrobots frequently fail and reach the target only 20% of the time. (Bottom) The image sequence shows that by 350k training steps, the microrobots demonstrate improved efficiency, reaching the target faster (in seconds) 75% of the time. A red box marks the microrobot when it is far from the target, while green indicates proximity to the target. The target position is denoted by a circle with a black plus symbol. c. A plot of RL performance illustrates the transition of a pre-trained model from simulation to the physical environment. The score rapidly increases during the simulation phase (blue), drops after the transition to experiments (marked by a red vertical dashed line), and then increases as the model adapts to the physical environment (green). d. A micrograph with superimposed images of the microrobots' navigation steps within an artificial vasculature, with the start and target points marked by red hollow circles. Arrows in the top left corner indicate all possible movement directions of the microrobots. e, f, g. Image sequence demonstrating a microrobot navigating an artificial vasculature, starting from the initial position and sequentially reaching three predefined targets:,and (. h. The heatmap depicts the relationship between speed and position within an artificial vasculature channel. The color gradient indicates the speed at each position, with darker colors indicating higher speeds and lighter shades denoting slower velocities.