PTVD: A Large-Scale Plot-Oriented Multimodal Dataset Based on Television Dramas

Art forms such as movies and television (TV) dramas are reflections of the real world, which have attracted much attention from the multimodal learning community recently. However, existing corpora in this domain share three limitations: i. annotated in a scene-oriented fashion, they ignore the coherence within plots; ii. their text lacks empathy and seldom mentions situational context; iii. their video clips fail to cover long-form relationship due to short duration. To address these fundamental issues, using 1,106 TV drama episodes and 24,875 informative plot-focused sentences written by professionals, with the help of 449 human annotators, we constructed PTVD, the first plot-oriented multimodal dataset in the cinema domain.

电影与电视剧（TV）这类艺术形式是现实世界的具象映照，近来备受多模态学习（multimodal learning）领域的广泛关注。然而当前该领域的现有语料库存在三大局限：其一，其标注采用场景导向模式，忽略了剧情内部的连贯性；其二，其文本缺乏共情性，且极少提及情境上下文；其三，受限于时长过短，其视频片段无法覆盖长时序的人物关系。为解决这些根本性问题，研究团队依托1106集电视剧剧集与24875条由专业人士撰写的、聚焦剧情的详实语句，在449名人类标注者的协助下，构建了影视领域首个以剧情为导向的多模态数据集PTVD。