Joint Entity and Event Extraction with Generative Adversarial Imitation Learning

Six figures and three tables of this paper. Figure 1 is our framework that includes a reward estimator based on a generative adversarial network (GAN) to issue dynamic rewards with regard to the labels (actions) committed by event extractor (agent). The reward estimator is trained upon the difference between the labels from ground truth (expert) and extractor (agent). If the extractor repeatedly misses Execute label for “death”, the penalty (negative reward values) is strengthened; if the extractor makes surprising mistakes: label “death” as Person or label Person “Masih” as Place role in Sentence event, the penalty is also strong. For cases where extractor is correct, simpler cases such as Sentence on “death” will take a smaller gain while difficult cases Execute on “death” will be awarded with larger reward values. Figure 2 is a pipeline from input sentence to sequence labels mentioned in Section 4.1. Q-Table and values for each current step is calculated using the unidirectional LSTM based on context embeddings of current and previous tokens as well as Q-Tables and values from previous steps. Context embeddings are calculated using Bi-LSTM from local token embeddings. Pre-trained embeddings based on Bi-LSTM such as ELMo are also good candidates for context embeddings. Figure 3 presents an illustrative example of updating the Q-values with Equation (4), with fixed rewards r = ±5 for correct/wrong labels and discount factor l = 0.01. Score for a wrong label is penalized while correct one is reinforced. Figure 4 is the extractor that combines context embeddings of the trigger and entity, as well as a one-hot vector that represents entity type and Bi-LSTM output of sub-sentence between the trigger and argument. The column “trend” denotes the changes of P(atr,ar|str,ar) after policy gradient optimization in Equation (10). Figure 5 presents an illustrative example of the GAN structure in sequence labeling scenario (argument role labeling scenario has the identical frameworks except vector dimensions). As introduced in Section 5, the “real data” in the original GAN is replaced by feature/state representation (Equation (1), or Equation (6) for argument role labeling scenario) and ground-truth labels (expert actions) in our framework, while the “generator data” consists of features and extractor’s attempt labels (agent actions). The discriminator serves as the reward estimator and a linear transformation is utilized to extend the D’s original output of probability range [0,1]. Figure 6 depicts change of rewards with regard to event type labels on the trigger “death” mentioned in Figure 1. Table 1 shows entity extraction performance. Table 2 presents the performance comparison with state-of-the-art frameworks with system predicted entities. Table 3 is comparison (F1) with state-of-the-art frameworks on ground-truth (gold) entity as argument candidates.