Variable resistance and magnetic-pneumatic multi-modal fusion tactile perception

To overcome the limitation that single-modal tactile sensing is insufficient for achieving diversified and multi-dimensional fine tactile perception, we propose a multi-modal tactile perception method that integrates variable resistance, magnetic field, and pneumatic pressure sensing information. First, a variable resistance unit is formed by impregnating a cylindrical sponge with a conductive carbon powder -polysiloxane solution. For magnetic sensing, a permanent magnet is embedded in the outer silicone layer, and the resulting field information and its orientation is measured using a Hall-effect sensor. Pneumatic sensing is realized by employing an external pressure sensor to monitor pressure changes within the encapsulated silicone cavity. Subsequently, the three-modal sensor devices are integrated to construct a variable resistance-magnetic-pneumatic multi-modal tactile sensor. Finally, a support vector machine (SVM) algorithm is utilized to process magnetic field information for orientation perception, and a neural network-based fusion model integrates variable resistance, magnetic field, and pneumatic pressure information to estimate the contact force under different orientations. The results show that for orientation recognition, the recognition accuracy of magnetic field modality is 93.0%, and the multi-modal fusion recognition accuracy is 97.1%. For force estimation, the root mean square error (RMSE) of variable resistance modality is 1.475 53 N, and that of the pneumatic modality is 3.747 95 N, while that of the multi-modal fusion is 0.744 60 N. Therefore, the variable resistance-magnetic-pneumatic multi-modal fusion tactile sensing method enables accurate regional force perception and is suitable for tactile perception in robotic manipulation tasks.