Tactile, Audio, and Visual Dataset During Bare Finger Interaction with Textured Surfaces

This paper presents a comprehensive multi-modal dataset capturing concurrent haptic, audio, and visual signals recorded during bare finger interactions with various textured surfaces. The dataset includes stereoscopic images of textures, fingertip position, speed, applied load, emitted sound, and friction-induced vibrations, providing an unprecedented insight into the complex dynamics underlying human tactile perception. Our approach utilizes a real human finger (while most previous studies relied on rigid sensorized probes), enabling the naturalistic acquisition of haptic data and addressing a significant gap in resources for studies of human tactile exploration, perceptual mechanisms, and artificial tactile perception. Through carefully designed data collection protocols, encompassing both controlled and free exploration scenarios, this dataset lays the groundwork for understanding human multi-sensory integration and supports the development of algorithms for texture recognition based on multi-modal inputs. A preliminary analysis demonstrates the dataset's potential, as classifiers trained on different combinations of data modalities show promising accuracy in surface identification, highlighting its value for advancing research in multi-sensory perception and the development of human-machine interfaces.