Natural Language Inference Dataset for Software Engineering

This repository introduces a specialized NLI dataset designed to optimize the performance of the language models targeted for tackling NLP tasks related to the software engineering domain. We consider different texts from the software engineering domain and manually curated the entailment relationships among different sentences. The source of the sentence pairs includes PROMISE dataset, PURE dataset, user guides of different softwares, articles on operating systems (e.g. official documentation of Windows, Mac), databases (e.g. official documentation of MongoDB, Oracle), cyber security (MITRE documentation) and software product descriptions including AWS documentation. Our created text entailment dataset contains 10k sentence pairs manually labeled for balanced classification with the labels Entailment, Contradiction, and Neutral.