Training Neural Machine Translation Models by Using an Entire Comparable Corpus

Neural machine translation (NMT) is often trained by using parallel sentences automatically extracted from comparable corpus where two language sentences correspond at a document level. Since few sentences correspond to each other in the comparable corpus, the other large portion of sentences are discarded without being extracted as the parallel sentences. Instead, this paper proposes to utilize an entire comparable corpus for training NMT. In order to use the comparable corpus as an additional training data, existing data augmentation methods from monolingual corpora are employed. In addition, a novel data augmentation method specifically designed for a comparable corpus, called Iterative Back-Translation with Comparability (IBTwC), is proposed. Our experimental evaluation was conducted on both low and high resource settings. For the low resource setting, it showed that the proposed IBTwC outperformed the conventional data augmentation methods. For the large resource setting, it revealed that the additional use of the comparable corpus did improve the MT performance, which resulted in the new state-of-the-art of the NTCIR-10 PatentMT task.