Joint Entity and Event Extraction with Generative Adversarial Imitation Learning

Six figures and three tables of this paper. Figure 1 is our framework that includes a reward estimator based on a generative adversarial network (GAN) to issue dynamic rewards with regard to the labels (actions) committed by event extractor (agent). The reward estimator is trained upon the difference between the labels from ground truth (expert) and extractor (agent). If the extractor repeatedly misses Execute label for “death”, the penalty (negative reward values) is strengthened; if the extractor makes surprising mistakes: label “death” as Person or label Person “Masih” as Place role in Sentence event, the penalty is also strong. For cases where extractor is correct, simpler cases such as Sentence on “death” will take a smaller gain while difficult cases Execute on “death” will be awarded with larger reward values. Figure 2 is a pipeline from input sentence to sequence labels mentioned in Section 4.1. Q-Table and values for each current step is calculated using the unidirectional LSTM based on context embeddings of current and previous tokens as well as Q-Tables and values from previous steps. Context embeddings are calculated using Bi-LSTM from local token embeddings. Pre-trained embeddings based on Bi-LSTM such as ELMo are also good candidates for context embeddings. Figure 3 presents an illustrative example of updating the Q-values with Equation (4), with fixed rewards r = ±5 for correct/wrong labels and discount factor l = 0.01. Score for a wrong label is penalized while correct one is reinforced. Figure 4 is the extractor that combines context embeddings of the trigger and entity, as well as a one-hot vector that represents entity type and Bi-LSTM output of sub-sentence between the trigger and argument. The column “trend” denotes the changes of P(atr,ar|str,ar) after policy gradient optimization in Equation (10). Figure 5 presents an illustrative example of the GAN structure in sequence labeling scenario (argument role labeling scenario has the identical frameworks except vector dimensions). As introduced in Section 5, the “real data” in the original GAN is replaced by feature/state representation (Equation (1), or Equation (6) for argument role labeling scenario) and ground-truth labels (expert actions) in our framework, while the “generator data” consists of features and extractor’s attempt labels (agent actions). The discriminator serves as the reward estimator and a linear transformation is utilized to extend the D’s original output of probability range [0,1]. Figure 6 depicts change of rewards with regard to event type labels on the trigger “death” mentioned in Figure 1. Table 1 shows entity extraction performance. Table 2 presents the performance comparison with state-of-the-art frameworks with system predicted entities. Table 3 is comparison (F1) with state-of-the-art frameworks on ground-truth (gold) entity as argument candidates.

本文共包含六幅图表及三张表格。图一展示了本研究的框架结构，其中包含一个基于生成对抗网络（GAN）的奖励估算器，用于根据事件提取器（智能体）所执行的行为（标签）动态地分配奖励。该奖励估算器基于来自真实标签（专家）与提取器（智能体）标签之间的差异进行训练。若提取器频繁遗漏“执行”标签对“死亡”的标注，则会对惩罚（负奖励值）进行强化；若提取器出现令人惊讶的错误：将“死亡”标签为“人物”或将“人物”标签“Masih”标注为“地点”在句子事件中，惩罚也将同样严厉。对于提取器正确标注的情况，如对“死亡”的简单句子标注，将获得较小的增益，而复杂的“死亡”执行标注则将获得更大的奖励值。图二展示了从输入句子到第4.1节中提到的序列标签的流程。每个当前步骤的Q-表和值通过基于当前和先前标记的上下文嵌入以及先前步骤的Q-表和值的单向LSTM进行计算。上下文嵌入通过局部标记嵌入的Bi-LSTM进行计算。基于Bi-LSTM的预训练嵌入，如ELMo，也是上下文嵌入的良好候选。图三以方程（4）更新Q值为例，展示了固定奖励r = ±5（正确/错误标签）和折扣因子l = 0.01的情况。错误标签的得分会受到惩罚，而正确的标签则得到强化。图四展示了结合触发词和实体上下文嵌入，以及表示实体类型的one-hot向量和触发词与论元之间的子句的Bi-LSTM输出的提取器。列“趋势”表示在方程（10）中的策略梯度优化后P(atr,ar|str,ar)的变化。图五展示了序列标注场景（论元角色标注场景框架相同，仅向量维度不同）中GAN结构的示例。如第5节所述，在原始GAN中的“真实数据”在我们的框架中由特征/状态表示（方程（1）或论元角色标注场景的方程（6））和真实标签（专家行为）替换，而“生成数据”由特征和提取器的尝试标签（智能体行为）组成。判别器充当奖励估算器，并利用线性变换扩展D的原输出概率范围[0,1]。图六描绘了与图一中提到的触发词“死亡”相关的事件类型标签的奖励变化。表1展示了实体提取性能。表2呈现了与最先进框架的系统预测实体的性能比较。表3展示了基于真实标签（金标签）作为论元候选的框架（F1）比较。