Russian multimodal conversational data

This multimodal corpus is a collection of video and speech recordings of face-to-face dialogues of Russian immigrants aged from 20-30 years old. At the moment 7 dialogue conversations of 40-60 minutes were recorded at the University of Cologne in November 2022, in total ca 4,5 hours of speech. The participants were native Russian speakers who live in Germany no longer than 5 years. Mostly pairs of not acquainted participants were selected, except one dyad (they were a married couple at the time of recordings). The participants were allowed to talk about any topic they wanted. The language material was captured in optimal conditions using frontal view HD camera with green background. This corpus contains data from 10 participants. All of them were university students. The video recordings will be accompanied with ELAN transcriptions and gesture annotations with a focus on discourse organization and conversational interaction and with particular attention to manual and nonmanual gestures (such as head nods, head shake, eye gaze, eye blinks, mouth gesture, nose wrinkles, face expressions, eyebrow and lip movement). The data has been elicited for the purposes of conducting the project "Gestures or Signs? Comparing Manual and Non-manual Constructions Sharing the Same Form in Co-speech Gesture and Sign Language: A Corpus-driven Approach" funded by DFG under the priority programme ViCom. As the project is aimed at comparing constructions of bodily movements that take the same form in languages belonging to different modalities, the annotations focus on alignments between speech and (non)manual gesture.

本多模态语料库收录了20至30岁俄罗斯移民的面对面对话视频与语音录音。2022年11月于科隆大学录制了7段时长40至60分钟的对话，总语音时长约4.5小时。参与人员均为以俄语为母语、在德居住时长不超过5年的移民。参与者多为互不相识的配对组，仅一组例外：录制时二人为已婚夫妻。参与者可自由选择对话主题。语料采集于最优环境中，采用带绿幕的正面高清摄像机录制。本语料库共收录10名参与者的数据，全部为在校大学生。视频录制文件将配套ELAN转录文本与手势标注，标注重点涵盖话语组织与会话互动，尤其关注手动与非手动手势，包括点头、摇头、视线移动、眨眼、嘴部动作、皱鼻、面部表情、眉毛与唇部动作等。本语料库数据为"手势还是符号？对比共语手势与手语中形式一致的手动与非手动结构：语料库驱动研究方法"项目采集所得，该项目由德国研究基金会（Deutsche Forschungsgemeinschaft, DFG）在ViCom优先计划资助下开展。鉴于本项目旨在对比不同模态语言中形式一致的身体动作结构，标注工作将聚焦于语音与（非）手动手势之间的对齐匹配。