Natural Language Inference Dataset for Software Engineering

This repository introduces a specialized NLI dataset designed to optimize the performance of the language models targeted for tackling NLP tasks related to the software engineering domain. We consider different texts from the software engineering domain and manually curated the entailment relationships among different sentences. The source of the sentence pairs includes PROMISE dataset, PURE dataset, user guides of different softwares, articles on operating systems (e.g. official documentation of Windows, Mac), databases (e.g. official documentation of MongoDB, Oracle), cyber security (MITRE documentation) and software product descriptions including AWS documentation. Our created text entailment dataset contains 10k sentence pairs manually labeled for balanced classification with the labels Entailment, Contradiction, and Neutral.本仓库介绍了一款专用自然语言推理（Natural Language Inference，简称NLI）数据集，旨在优化面向软件工程领域自然语言处理（Natural Language Processing，简称NLP）任务的语言模型性能。我们采集软件工程领域的多样化文本，并手动梳理并标注了不同语句间的蕴含关系。该数据集的语句对来源涵盖PROMISE数据集、PURE数据集、各类软件用户手册、操作系统相关文章（如Windows、Mac官方文档）、数据库官方文档（如MongoDB、Oracle官方文档）、网络安全领域资料（如MITRE官方文档），以及包含AWS文档在内的软件产品说明文本。本次构建的文本蕴含数据集共包含1万条手动标注语句对，用于开展均衡分类任务，标注标签涵盖蕴含（Entailment）、矛盾（Contradiction）与中立（Neutral）三类。