lmqg/qg_annotation

这是一个由不同模型生成的注释问题集，用于在《生成语言模型用于段落级问题生成：统一基准与评估，EMNLP 2022主会议》中测量自动指标与人类之间的相关性。

This is an annotated question corpus generated by diverse models, designed to measure the correlation between automatic evaluation metrics and human judgments in the EMNLP 2022 main conference paper *Generative Language Models for Paragraph-level Question Generation: Unified Benchmark and Evaluation*.

原始信息汇总

数据集概述

基本信息

• 许可证: cc-by-4.0
• 数据集名称: QG Annotation
• 语言: 英语 (en)
• 多语言性: 单语种
• 数据集大小: 小于1K

数据集描述

• 摘要: 该数据集包含由不同模型生成的已标注问题，用于衡量自动指标与人类评估之间的相关性，主要用于研究“Generative Language Models for Paragraph-Level Question Generation: A Unified Benchmark and Evaluation, EMNLP 2022 main conference”。

数据集结构

• 示例结构: python { "correctness": 1.8, "grammaticality": 3.0, "understandability": 2.4, "prediction": "What trade did the Ming dynasty have a shortage of?", "Bleu_4": 0.4961682999359617, "METEOR": 0.3572683356086923, "ROUGE_L": 0.7272727272727273, "BERTScore": 0.9142221808433532, "MoverScore": 0.6782580808848975, "reference_raw": "What important trade did the Ming Dynasty have with Tibet?", "answer_raw": "horse trade", "paragraph_raw": "Some scholars note that Tibetan leaders during the Ming frequently engaged in civil war and conducted their own foreign diplomacy with neighboring states such as Nepal. Some scholars underscore the commercial aspect of the Ming-Tibetan relationship, noting the Ming dynastys shortage of horses for warfare and thus the importance of the horse trade with Tibet. Others argue that the significant religious nature of the relationship of the Ming court with Tibetan lamas is underrepresented in modern scholarship. In hopes of reviving the unique relationship of the earlier Mongol leader Kublai Khan (r. 1260u20131294) and his spiritual superior Drogu00f6n Chu00f6gyal Phagpa (1235u20131280) of the Sakya school of Tibetan Buddhism, the Yongle Emperor (r. 1402u20131424) made a concerted effort to build a secular and religious alliance with Deshin Shekpa (1384u20131415), the Karmapa of the Karma Kagyu school. However, the Yongle Emperors attempts were unsuccessful.", "sentence_raw": "Some scholars underscore the commercial aspect of the Ming-Tibetan relationship, noting the Ming dynastys shortage of horses for warfare and thus the importance of the horse trade with Tibet.", "reference_norm": "what important trade did the ming dynasty have with tibet ?", "model": "T5 Large" }

引用信息

@inproceedings{ushio-etal-2022-generative, title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration", author = "Ushio, Asahi and Alva-Manchego, Fernando and Camacho-Collados, Jose", booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing", month = dec, year = "2022", address = "Abu Dhabi, U.A.E.", publisher = "Association for Computational Linguistics", }