Polarity Shifter Resources

This repository was created as part of Marc Schulder's doctoral thesis Sentiment Polarity Shifters: Creating Lexical Resources through Manual Annotation and Bootstrapped Machine Learning The collection of polarity shifter resources presented herein is also connected to a number of publications: Schulder et al. (IJCNLP 2017): Lexicon of English Verbal Shifters (bootstrapped, lemma-level) and sentiment verb phrase dataset. doi: 10.5281/zenodo.3364812 Schulder et al. (LREC 2018): Lexicon of English Verbal Shifters (manual, sense-level). doi: 10.5281/zenodo.3365288 Schulder et al. (COLING 2018): Lexicon of German Verbal Shifters (bootstrapped, lemma-level). doi: 10.5281/zenodo.3365370 Schulder et al. (LREC 2020): Lexicon of Polarity Shifting Directions (supervised classification, lemma-level). doi: 10.5281/zenodo.3545947 Schulder et al. (JNLE 2020): General Lexicon of English Shifters (bootstrapped, lemma-level). doi: 10.5281/zenodo.3365601 Data The repository contains the following resources: A general lexicon of English polarity shifters, covering verbs, adjectives and nouns. Provides lemma labels for shifters and for which polarities they can affect. A lexicon of English verbal shifters. Provides word sense labels for shifters and their shifting scopes. A lexicon of German verbal shifters. Provides lemma labels for shifters. A set of verb phrases annotated for shifting polarities. 1. English Shifter Lexicon (Lemma) A lexicon of 9145 English words, annotated for whether they are polarity shifters and which polarities they affect. The lexicon is based on the vocabulary of WordNet v3.1 (Miller et al., 1990). It contains 2631 shifters and 6514 non-shifters. File: shifters.english.all.lemma.txt The lexicon is a comma-separated value (CSV) table. Each line follows the format POS,LEMMA,SHIFTER_LABEL,DIRECTION_LABEL,SOURCE. POS: The part of speech of the word (verb, noun, adj) LEMMA: The lemma representation of the word in question. Multiword expressions are separated by an underscore (WORD_WORD). SHIFTER_LABEL: Whether the word is a polarity shifter (SHIFTER) or a non-shifter (NONSHIFTER). DIRECTION_LABEL: Whether the shifter affects only positive polarities (AFFECTS_POSITIVE), only negative polarities. (AFFECTS_NEGATIVE) or can shift in both directions (AFFECTS_BOTH). Non-shifters are all labeled (NONE). SOURCE: Whether the word was part of the gold standard. (GOLD_STANDARD) or was bootstrapped (BOOTSTRAPPED). Note that while bootstrapped shifter labels are verified by a human annotator, their direction label is automatically classified without verification. 2. English Verbal Shifter Lexicon (Word Sense) A lexicon of word senses of English verbs, annotated for whether they are polarity shifters and their shifting scope. The lexicon covers all verbs of WordNet v3.1 (Miller et al., 1990) that are single word or particle verbs. Polarity shifter and scope labels are given for each lemma-synset pair (i.e. each word sense of a lemma). The data is presented in the following forms: A complete lexicon of all verbal shifters and their shifting scopes. Two auxiliary lists containing simplified information: A list of all lemmas with shifter labels A list of all word senses with shifter labels All files are in CSV (comma-separated value) format. 2.1. Complete Lexicon The main lexicon lists all verbal shifters and their shifting scopes. Verbal shifters are modeled as lemma-sense pairs with one or more shifting scopes. The lexicon lists all lemma-sense pairs that are verbal shifters. Any lemma-sense pair not listed is not a verbal shifter. When a lemma-sense pair has more than one possible scope, a separate entry is made for each scope. File name: shifters.english.verb.sense.csv Each line contains a single lemma-sense-scope triple, using the format LEMMA,SYNSET,SCOPE. LEMMA: The lemma representation of the verb in question. Multiword expressions are separated by an underscore (WORD_WORD). SYNSET: The numeric identifier of the synset, commonly referred to as offset or database location. It consists of 8 digits, including leading zeroes (e.g. 00334568). SCOPE: The scope of the shifting: subj: The verbal shifter affects its subject. dobj: The verbal shifter affects its direct object. pobj_*: The verbal shifter affects objects within a prepositional phrase. The preposition in question is included in the annotation. For example a from-preposition scope receives the label pobj_from and a a for-preposition receives pobj_for. comp: The verbal shifter affects a clausal complement, such as infinitive clauses or gerunds. 2.2. List of Lemmas List of all verb lemmas and whether they are shifters in at least one of their word senses. Does not provide shifter scope information. Many verbal shifter lemmas only cause shifting in some of their word senses. This list is therefore considerably more coarse-grained than the main lexicon. It is intended as a convenience measure for quick experimentation. File name: shifters.english.verb.sense.lemmas_only.csv Each line follows the format LEMMA,SHIFTER_LABEL. LEMMA: The lemma representation of the verb in question. Multiword expressions are separated by an underscore (WORD_WORD). SHIFTER_LABEL: Whether the verb is a polarity shifter (SHIFTER) or a non-shifter (NONSHIFTER). 2.3. List of Synsets List of all synsets and whether their lemmas are shifters in this specific word sense. Does not provide shifter scope information. Shifting is shared among lemmas of the same word sense. This list, therefore, provides (almost) the same granularity for the shifter label as the main lexicon. However, in a few exceptions, synsets contained words with subtly different senses that did not all cause shifting. These senses are considered shifters in this list, analogous to the generalization in the list of lemmas. File name: shifters.english.verb.sense.synsets_only.csv Each line follows the format SYNSET,SHIFTER_LABEL. SYNSET: The numeric identifier of the synset, commonly referred to as offset or database location. It consists of 8 digits, including leading zeroes (e.g. 00334568). SHIFTER_LABEL: Whether the verb is a polarity shifter (SHIFTER) or a non-shifter (NONSHIFTER). 3. German Verbal Shifter Lexicon (Lemma) A lexicon of 2595 German verbs, annotated for whether they are polarity shifters and which polarities they affect. The lexicon is based on the vocabulary of GermaNet (Hamp and Feldweg, 1997). It contains 677 shifters and 1918 non-shifters. File: shifters.german.verb.lemma.txt The lexicon is a comma-separated value (CSV) table. Each line follows the format LEMMA,SHIFTER_LABEL,SOURCE. LEMMA: The lemma representation of the verb in question. Multiword expressions are separated by an underscore (WORD_WORD). SHIFTER_LABEL: Whether the verb is a polarity shifter (SHIFTER) or a non-shifter (NONSHIFTER). SOURCE: Whether the word was part of the gold standard. (GOLD_STANDARD) or was bootstrapped (BOOTSTRAPPED). In either case the verbs were verified by a human annotator. 4. Sentiment Verb Phrases A set of verb phrases, annotated for the polarity of the verb phrase and the polarity of a polar noun that it contains. Can be used to evaluate whether a polarity classifier correctly recognizes polarity shifting. The file starts with 400 phrases containing shifter verbs, followed by 2231 phrases containing non-shifter verbs. File: sentiment_phrases.txt Every item consists of: The sentence from which the VP and the polar noun were extracted. The VP, polar noun and the verb heading the VP. Constituency parse for the VP. Gold labels for VP and polar noun by a human annotator. Predicted labels for VP and polar noun by RNTN tagger (Socher et al., 2013) and LEX_gold approach. Items are separated by a line of asterisks (*)本仓库是马克·舒尔德（Marc Schulder）博士论文《情感极性转移词：通过人工标注与自举机器学习构建词汇资源》（Sentiment Polarity Shifters: Creating Lexical Resources through Manual Annotation and Bootstrapped Machine Learning）的配套成果。 本文呈现的极性转移词（polarity shifter）资源集合亦关联多篇学术出版物： 1. Schulder等（IJCNLP 2017）：英语动词转移词词典（自举式，词元（lemma）级）与情感动词短语数据集。DOI：10.5281/zenodo.3364812 2. Schulder等（LREC 2018）：英语动词转移词词典（人工标注，词义（word sense）级）。DOI：10.5281/zenodo.3365288 3. Schulder等（COLING 2018）：德语动词转移词词典（自举式，词元级）。DOI：10.5281/zenodo.3365370 4. Schulder等（LREC 2020）：极性转移方向词典（监督分类，词元级）。DOI：10.5281/zenodo.3545947 5. Schulder等（JNLE 2020）：通用英语转移词词典（自举式，词元级）。DOI：10.5281/zenodo.3365601 ## 数据资源 本仓库包含以下词汇资源： - 通用英语极性转移词词典：涵盖动词、形容词与名词，为转移词提供词元标注及其可作用的极性类型。 - 英语动词转移词词典：为转移词提供词义标注及其转移作用域。 - 德语动词转移词词典：为转移词提供词元标注。 - 一组标注了极性转移属性的动词短语数据集。 --- ### 1. 英语转移词词典（词元级） 本词典包含9145个英语词汇，基于WordNet v3.1（Miller等，1990）的词汇表构建，标注了每个词汇是否为极性转移词及其作用的极性类型，其中包含2631个转移词与6514个非转移词。 文件：shifters.english.all.lemma.txt 本词典采用逗号分隔值（CSV）表格格式，每行遵循如下格式：POS,LEMMA,SHIFTER_LABEL,DIRECTION_LABEL,SOURCE - POS：词汇的词性，可选值为动词、名词、形容词 - LEMMA：目标词汇的词元形式，多词表达式以下划线分隔（如WORD_WORD） - SHIFTER_LABEL：标注该词汇是否为极性转移词（SHIFTER）或非转移词（NONSHIFTER） - DIRECTION_LABEL：标注转移词仅作用于正极性（AFFECTS_POSITIVE）、仅作用于负极性（AFFECTS_NEGATIVE），或可双向转移（AFFECTS_BOTH）；非转移词统一标注为NONE - SOURCE：标注该词汇是否属于金标准数据集（GOLD_STANDARD）或通过自举方法构建（BOOTSTRAPPED）。需注意，自举得到的转移词标签已通过人工标注验证，但其极性方向标签仅为自动分类结果，未经过人工验证。 --- ### 2. 英语动词转移词词典（词义级） 本词典针对英语动词的词义进行标注，标注其是否为极性转移词及其转移作用域，覆盖WordNet v3.1（Miller等，1990）中所有单字词动词与短语动词。每个词元-同义词集（synset）对（即词元的每个词义）均配有极性转移词与作用域标签。 数据以下列形式呈现： - 完整的动词转移词及其作用域词典 - 两份辅助列表，包含简化后的信息： - 所有标注了转移词标签的词元列表 - 所有标注了转移词标签的词义列表 所有文件均采用逗号分隔值（CSV）格式。 #### 2.1 完整词典 主词典列出所有动词转移词及其转移作用域。动词转移词以词元-词义对形式建模，每个词元-词义对可对应一个或多个转移作用域。本词典仅列出属于极性转移词的词元-词义对，未列出的词元-词义对均不属于动词转移词。若一个词元-词义对存在多个可能的作用域，则为每个作用域单独生成一条条目。 文件名：shifters.english.verb.sense.csv 每行包含单个词元-词义-作用域三元组，格式为：LEMMA,SYNSET,SCOPE - LEMMA：目标动词的词元形式，多词表达式以下划线分隔（如WORD_WORD） - SYNSET：同义词集的数字标识符，通常称为偏移量（offset）或数据库位置，由8位数字组成，包含前导零（例如00334568） - SCOPE：转移作用域，可选值如下： - subj：动词转移词作用于其主语 - dobj：动词转移词作用于其直接宾语 - pobj_*：动词转移词作用于介词短语内的宾语，标注中包含对应的介词。例如，作用于from介词宾语的标签为pobj_from，作用于for介词宾语的标签为pobj_for - comp：动词转移词作用于从句补语，如不定式从句或动名词短语 #### 2.2 词元列表 该列表包含所有动词词元及其是否在至少一个词义中属于极性转移词，不提供转移作用域信息。由于许多动词词元仅在部分词义中具备极性转移属性，本列表相比主词典粒度更粗，仅用于快速实验的便捷工具。 文件名：shifters.english.verb.sense.lemmas_only.csv 每行遵循如下格式：LEMMA,SHIFTER_LABEL - LEMMA：目标动词的词元形式，多词表达式以下划线分隔（如WORD_WORD） - SHIFTER_LABEL：标注该动词是否为极性转移词（SHIFTER）或非转移词（NONSHIFTER） #### 2.3 词义列表 该列表包含所有同义词集及其词元在该特定词义下是否为极性转移词，不提供转移作用域信息。同一词义下的词元共享转移属性，因此本列表的转移词标签粒度与主词典几乎一致。但存在少数例外情况：部分同义词集包含语义略有差异的词汇，并非所有词汇都具备转移属性，但本列表仍将这些词义统一标注为转移词，与词元列表的泛化规则保持一致。 文件名：shifters.english.verb.sense.synsets_only.csv 每行遵循如下格式：SYNSET,SHIFTER_LABEL - SYNSET：同义词集的数字标识符，通常称为偏移量（offset）或数据库位置，由8位数字组成，包含前导零（例如00334568） - SHIFTER_LABEL：标注该动词是否为极性转移词（SHIFTER）或非转移词（NONSHIFTER） --- ### 3. 德语动词转移词词典（词元级） 本词典包含2595个德语动词，基于GermaNet（Hamp与Feldweg，1997）的词汇表构建，标注了每个动词是否为极性转移词及其作用的极性类型，其中包含677个转移词与1918个非转移词。 文件：shifters.german.verb.lemma.txt 本词典采用逗号分隔值（CSV）表格格式，每行遵循如下格式：LEMMA,SHIFTER_LABEL,SOURCE - LEMMA：目标动词的词元形式，多词表达式以下划线分隔（如WORD_WORD） - SHIFTER_LABEL：标注该动词是否为极性转移词（SHIFTER）或非转移词（NONSHIFTER） - SOURCE：标注该词汇是否属于金标准数据集（GOLD_STANDARD）或通过自举方法构建（BOOTSTRAPPED）。无论属于哪种来源，所有动词均经过人工标注验证。 --- ### 4. 情感动词短语数据集 该数据集包含一组动词短语，标注了动词短语本身的极性以及其中包含的极性名词的极性，可用于评估极性分类器是否能正确识别极性转移属性。文件前400个短语包含转移词动词，后续2231个短语包含非转移词动词。 文件：sentiment_phrases.txt 每个条目包含以下内容： - 提取出动词短语（VP）与极性名词的源句子 - 该动词短语、极性名词以及作为短语核心的动词 - 该动词短语的成分句法分析树 - 人工标注的动词短语与极性名词的金标准标签 - 由递归神经网络句法分析器（RNTN tagger，Socher等，2013）与LEX_gold方法得到的动词短语与极性名词的预测标签 条目之间以一行星号（*）分隔。