NYK-MS
收藏资源简介：
NYK-MS数据集是由北京大学创建的一个多模态隐喻和讽刺理解基准，包含1,583个隐喻样本和1,578个讽刺样本，总计3,161个样本。数据集来源于《纽约客》漫画标题比赛，由专业画家创作的漫画和读者提交的标题组成。数据集经过多轮注释以提高一致性和质量，并使用GPT-4V辅助注释。该数据集旨在解决多模态隐喻和讽刺理解任务，包括分类、词检测和解释等7个任务，适用于自然语言处理和计算机视觉领域的研究。
The NYK-MS Dataset is a multimodal metaphor and irony understanding benchmark created by Peking University. It includes 1,583 metaphor samples and 1,578 irony samples, totaling 3,161 samples in all. The dataset is sourced from The New Yorker Cartoon Caption Contest, which comprises original cartoons drawn by professional cartoonists and captions submitted by readers. It has undergone multiple rounds of annotation to improve consistency and quality, with GPT-4V employed to assist in the annotation process. This dataset targets multimodal metaphor and irony understanding tasks, covering seven tasks such as classification, word detection and explanation, and is applicable to research in the fields of natural language processing and computer vision.
数据集概述
数据集名称
Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest
数据集来源
The New Yorker Caption Contest
数据集描述
该数据集包含来自《纽约客》漫画标题比赛的幽默理解基准。数据集包括多个任务，如匹配、排序和解释生成。
数据集任务
- 匹配任务：选择与图像最匹配的标题。
- 排序任务：对多个标题进行排序，选择最合适的标题。
- 解释生成任务：生成对幽默标题的解释。
数据集结构
- 匹配任务：包含多个标题选项和对应的图像描述，标签指示正确答案。
- 排序任务：包含两个标题选项和对应的图像描述，标签指示正确答案。
- 解释生成任务：包含标题和对应的解释。
数据集示例
匹配任务示例
json { "caption_choices": [ "Tell me about your childhood very quickly.", "Believe me . . . its whats UNDER the ground thats most interesting.", "Stop me if youve heard this one.", "I have trouble saying no.", "Yes, I see the train but I think we can beat it." ], "contest_number": 49, "entities": [ "https://en.wikipedia.org/wiki/Rule_of_three_(writing)", "https://en.wikipedia.org/wiki/Bar_joke", "https://en.wikipedia.org/wiki/Religious_institute" ], "from_description": "scene: a bar description: Two priests and a rabbi are walking into a bar, as the bartender and another patron look on. The bartender talks on the phone while looking skeptically at the incoming crew. uncanny: The scene depicts a very stereotypical bar joke that would be unlikely to be encountered in real life; the skepticism of the bartender suggests that he is aware he is seeing this trope, and is explaining it to someone on the phone. entities: Rule_of_three_(writing), Bar_joke, Religious_institute. choices A: Tell me about your childhood very quickly. B: Believe me . . . its whats UNDER the ground thats most interesting. C: Stop me if youve heard this one. D: I have trouble saying no. E: Yes, I see the train but I think we can beat it.", "image": "<PIL.JpegImagePlugin.JpegImageFile image mode=L size=323x231 at 0x7F34F283E9D0>", "image_description": "Two priests and a rabbi are walking into a bar, as the bartender and another patron look on. The bartender talks on the phone while looking skeptically at the incoming crew.", "image_location": "a bar", "image_uncanny_description": "The scene depicts a very stereotypical bar joke that would be unlikely to be encountered in real life; the skepticism of the bartender suggests that he is aware he is seeing this trope, and is explaining it to someone on the phone.", "instance_id": "21125bb8787b4e7e82aa3b0a1cba1571", "label": "C", "n_tokens_label": 1, "questions": [ "What is the bartender saying on the phone in response to the living, breathing, stereotypical bar joke that is unfolding?" ] }
排序任务示例
json { "choices": { "A": "Looks to be a herniated disco.", "B": "Everyone, wish upon a star!" }, "image": "fc79106cf3660f5b81cdbeed0f968d98.jpg", "instance_id": "cba6d1ce5711ad56c31e5577f3207ac3" }
解释生成任务示例
json { "caption": "Please! I have a wife and two thousand kids!", "contest_number": 509, "explanation": "A play on the common plea people use in dire situations: I have a wife and two kids; this is stated to try to have people take mercy and not kill someone. But here, the victim of the bear is a fish about to be eaten, and fish tend to have many more than two kids, so the phrase is updated with the fish-version of it: two thousand kids.", "n_expl_toks": 70 }
数据集下载
引用信息
如果使用该数据集，请引用以下文献：
@inproceedings{hessel2023androids, title={Do Androids Laugh at Electric Sheep? {Humor} ``Understanding Benchmarks from {The New Yorker Caption Contest}}, author={Hessel, Jack and Marasovi{c}, Ana and Hwang, Jena D. and Lee, Lillian and Da, Jeff and Zellers, Rowan and Mankoff, Robert and Choi, Yejin}, booktitle={Proceedings of the ACL}, year={2023} }
- 1NYK-MS: A Well-annotated Multi-modal Metaphor and Sarcasm Understanding Benchmark on Cartoon-Caption Dataset北京大学 · 2024年
