Position-aware Attention for Enhancing the Machine Comprehension

Attention mechanism has been regarded as a popular approach for machine comprehension (MC) in recent years. Many existing approaches calculate the attention weight by integrating the question and context representation. However, these approaches ignore the impact of position information, which is demonstrated to be effective in information retrieval (IR). Combined with position information, we also notice that when doing reading comprehension, people usually pay more attention to the passage words which appear close to question words. In this paper, inspired by the two reasons above, we investigate the effect of position information for MC. Particularly, we propose an improved positional attention based MC model, which integrates the position-aware attention to weight the context words. Focusing on the substantial syntactic dataset of MC task, we also use the paraphrase database to expanse the question words. The evaluation on the benchmark TriviaQA data proves the effectiveness of our method.