中文成语语义推理数据集(CINLID)
收藏资源简介:
计算词/词组、句子、段落和文档之间的语义相似性(STS,Semantic similarity of text)在自然语言处理和计算语言学中起着重要作用,是一个非常重要的任务。 语义相似性中的条目(词、短语、句等)之间的距离概念是基于其意义或语义内容的相似性,而不是词汇学的相似性。基于字面含义的“望文生义”在做NLP相关任务时很容易出现问题,如传统测度文本相似度的方法经常会把“目不识丁”和“目中无人”的相似度算得很高。但它们的语义明显是不相关的;又比如“孤芳自赏”和“师心自用”都有“自负、骄傲”的含义,语义相似度较高,但一般传统的文本相似度计算方法很难将它们的内在语义相似性准确的捕捉到。 为了得到良好的语句表示,我们需要一个能编码基础语义关系的语料,而且字面重叠的情况要少,让机器学习的难度更大些,以便学到更多有用的语义信息。因此,我们基于同一关系、包含关系、重叠关系、分离关系这4种基本的语义类别构建了中文成语语义推理数据集(Chinese Idioms Natural Language Inference Dataset)。
Semantic Textual Similarity (STS), which calculates the semantic similarity between words, phrases, sentences, paragraphs and documents, plays a critical role in natural language processing (NLP) and computational linguistics, and is a highly significant research task. The concept of distance between items (words, phrases, sentences, etc.) in semantic similarity is based on the similarity of their meaning or semantic content, rather than lexicographical similarity. Taking literal meanings out of context, a common pitfall in NLP-related tasks, often leads to erroneous results. For instance, traditional text similarity measurement methods frequently assign a high similarity score to "mù bù shí dīng (illiterate)" and "mù zhōng wú rén (arrogant and disdainful of others)", while their actual semantic meanings are completely unrelated. Another example is "gū fāng zì shǎng (self-admiring and conceited)" and "shī xīn zì yòng (self-opinionated and stubborn)", both of which carry the connotation of "being conceited and proud", thus exhibiting high semantic similarity. However, most conventional text similarity calculation methods struggle to accurately capture their underlying semantic similarity. To acquire high-quality sentence representations, a corpus that can encode basic semantic relations is required, with minimal literal overlap to increase the difficulty of machine learning, thereby enabling models to learn more valuable semantic information. Accordingly, we developed the Chinese Idioms Natural Language Inference Dataset based on four fundamental semantic categories: identity relation, containment relation, overlap relation, and separation relation.




