Double attention recurrent convolution neural network for answer selection

Answer selection is one of the key steps in many Question Answering (QA) applications. In this paper, a new deep model with two kinds of attention is proposed for answer selection: the Double Attention Recurrent Convolution Neural Network (DARCNN). Double attention means self-attention and cross-attention. The design inspiration of this model came from the Transformer in the domain of machine translation. Self-attention can directly calculate dependencies between words regardless of the distance. However, self-attention ignores the distinction between its surrounding words and other words. Thus, we design a decay self-attention that prioritizes local words in a sentence. In addition, cross-attention is established to achieve interaction between question and candidate answer. With the outputs of self-attention and decay self-attention, we can get two kinds of interactive information via cross-attention. Finally, using the feature vectors of the question and answer, elementwise multiplica...