MuSe-Wilder: Multimodal Continuous Emotions in-the-Wild (MuSe2021)

MuSe-Wilder of the 2nd Multimodal Sentiment in-the-Wild Challenge! Predicting the level of emotional dimensions (valence, arousal) in a time-continuous manner from audio-video-text data. This package includes only MuSe-Wilder features (all partitions) and annotations of the training and development set (test scoring via the MuSe website). More: https://www.muse-challenge.org/muse2021 General: The purpose of the Multimodal Sentiment Analysis in Real-life media Challenge and Workshop (MuSe) is to bring together communities from different disciplines. We introduce the novel dataset MuSe-CAR that covers the range of aforementioned desiderata. MuSe-CAR is a large (>36h), multimodal dataset which has been gathered in-the-wild with the intention of further understanding Multimodal Sentiment Analysis in-the-wild, e.g., the emotional engagement that takes place during product reviews (i.e., automobile reviews) where a sentiment is linked to a topic or entity. We have designed MuSe-CAR to be of high voice and video quality, as informative video social media content, as well as everyday recording devices have improved in recent years. This enables robust learning, even with a high degree of novel, in-the-wild characteristics, for example as related to: i) Video: Shot size (a mix of close-up, medium, and long shots), face-angle (side, eye, low, high), camera motion (free, free but stable, and free but unstable, switch, e.g., zoom, fixed), reviewer visibility (full body, half-body, face only, and hands only), highly varying backgrounds, and people interacting with objects (car parts). ii) Audio: Ambient noises (car noises, music), narrator and host diarisation, diverse microphone types, and speaker locations. iii) Text: Colloquialisms, and domain-specific terms.第二届野外多模态情感分析挑战赛（Multimodal Sentiment in-the-Wild Challenge）的MuSe-Wilder赛道！任务为基于音视频-文本（audio-video-text）数据，以时间连续的方式预测情感维度——效价（valence）与唤醒度（arousal）的水平。本套件仅包含MuSe-Wilder的全部划分特征以及训练集与开发集的标注，测试集评分需通过MuSe官网提交。更多信息请访问：https://www.muse-challenge.org/muse2021 【赛事概况】现实媒体多模态情感分析挑战赛暨研讨会（Multimodal Sentiment Analysis in Real-life media Challenge and Workshop，简称MuSe）旨在汇聚不同学科领域的研究社群。本次赛事推出全新数据集MuSe-CAR，以覆盖前述各项研究诉求。MuSe-CAR是总时长超36小时的大规模多模态数据集，采集自真实野外（in-the-wild）场景，旨在深化对野外多模态情感分析的研究——例如汽车评测类视频中的情感交互：此时情感与特定主题或实体（汽车相关）绑定。 近年来，日常录制设备与社交视频内容的音视频质量大幅提升，基于此我们打造了高保真音视频画质的MuSe-CAR数据集，以支持鲁棒性模型训练，即便面对大量具备真实野外特性的复杂场景，例如： i) 视频维度：镜头景别（特写、中景、远景混合）、人脸拍摄角度（侧面、眼部特写、低角度、高角度）、摄像机运动模式（自由移动、自由且稳定、自由但抖动、变焦等切换操作、固定机位）、评测者出镜范围（全身、半身、仅面部、仅手部）、高度多变的背景环境，以及评测者与实体（汽车部件）的交互行为； ii) 音频维度：环境噪声（汽车杂音、背景音乐）、旁白与主持人的说话人分割（diarization）、多样化的麦克风类型与拾音位置； iii) 文本维度：口语化表达与领域专属术语。