Multimodal Deception Detection Dataset with Personality Profiling across Indonesian Ethnic Groups

Multimodal Deception Detection Dataset with Personality Profiling across Indonesian Ethnic Groups This dataset presents a comprehensive multimodal approach to deception detection, integrating cultural diversity and personality profiling across Indonesian ethnic groups. The dataset comprises 196 participants stratified across six major Indonesian ethnicities (Javanese, Sundanese, Madurese, Bugis-Makassar, Batak, and Minangkabau) and four DISC personality types (Dominance, Influence, Steadiness, Conscientiousness). Key Features Multimodal Data: Text (Indonesian & English transcripts with linguistic features), Audio (94 MFCC and spectral features + 17 pause features), and Visual (468 facial + 10 iris + 33 pose landmarks per frame) Cultural Diversity: Six Indonesian ethnic groups with documented metadata enabling cross-cultural behavioral analysis Personality Profiling: DISC personality assessment for individual difference analysis Controlled Design: Testimonial narratives about witnessed theft incidents with verified ground truth labels High Quality: Advanced preprocessing with automated quality validation (SNR: 79.8±3.1 dB, landmark detection: 90.1-100%) Data Collection 196 participants distributed across six ethnic groups: Javanese (n=30) Sundanese (n=25) Madurese (n=43) Bugis-Makassar (n=36) Batak (n=31) Minangkabau (n=31) 1,568 video recordings (8 per participant: 4 truthful, 4 deceptive) Personality distribution: Dominance (n=28), Influence (n=32), Steadiness (n=71), Conscientiousness (n=65) Multi-regional collection across 6 Indonesian cities (Makassar, Medan, Padang, Yogyakarta, Bangkalan/Sampang, Bandung) Standardized CCTV footage stimuli from Indonesian news broadcasts Professional recording equipment (iPhone 13: 4K at 60fps, TNW wireless lavalier microphone: 16kHz) Applications Multimodal deception detection algorithm development Cross-cultural behavioral pattern analysis Personality-aware detection systems Forensic and security applications Cultural psychology research Fairness and bias mitigation studies Data Format Released Files (Total: ~18.5 GB): Preprocessed audio: 1,568 WAV files (16 kHz mono, 16-bit PCM) - 324.2 MB compressed Text datasets: Indonesian transcripts (16 columns) and English translations (11 columns) - 312.7 KB Audio features: 104 acoustic features (MFCC, spectral, pause patterns) - 3.3 MB Visual landmarks: 647,871 frames with 1,536 landmark coordinates - 18.0 GB Metadata: Feature descriptions and documentation - 8.4 KB Note: Raw videos are NOT publicly released to protect participant privacy. Only derived features and preprocessed audio are available. Technical Validation Audio quality: SNR 79.8±3.1 dB (threshold: >20 dB) Landmark detection rates: Face 90.1%, Iris 90.1%, Pose 100.0% Transcription reliability: Cohen's κ=0.82 [0.78, 0.86] (10% sample, two independent raters) Class balance: Perfect 50/50 split (784 truthful, 784 deceptive) Missing values: <0.05% across all modalities Baseline performance: 58-69% accuracy across modalities (demonstrating data usability) Ethics Ethical approval: University of Surabaya Institutional Ethics Committee (Protocol No. 756/KE/XII/2025) Informed consent: All participants provided explicit consent for public release of derived features and preprocessed audio Data anonymization: Alphanumeric codes, no personally identifiable information Privacy protection: Raw videos retained securely, only derived features released publicly Citation If you use this dataset, please cite: Rahayu, Y. D., Fatichah, C., Yuniarti, A., Probowati, Y., & Kusnanti, E. A. (2025). Multimodal Deception Detection Dataset with Personality Profiling across Indonesian Ethnic Groups [Data set]. Zenodo. https://doi.org/10.5281/zenodo.17421590 Funding This research was supported by the Research and Innovation Incentive Program (RIIM) from Indonesia Endowment Fund for Education (LPDP) and Indonesian National Research and Innovation Agency (BRIN), grant numbers 16/IV/KS/02/2025 and 821/PKS/ITS/2025.