KB-Transformer: Incorporating Knowledge into End-to-End Task-Oriented Dialog Systems

Existing end-to-end task-oriented dialogue systems are lack of proper support of knowledge base, and training based on RNN seq2seq models is time-consuming. In this paper, we propose a novel parallel computing framework(KB-Transformer) to incorporate KB which can be trained faster than RNN seq2seq models. A key contribution is that we propose multi-head key-value memory network for the first time, comprehensively encoding semantic information of KB and incorporating KB into task-oriented dialog systems from multiple dimensions and sub-spaces. KB-Transformer tries to combine memory network with transformer, implementing a parallel dialogue framework based entirely on attention mechanism. As a result, we show that KB-Transformer can be trained faster and attain better performance than RNN seq2seq models on two different task-oriented dialog datasets.