An Unsupervised Approach to Bilingual Lexicon Induction with Comparable Corpora

The steady growth of globalization leads to an increasing requirement for translations among different languages and an urgent need to keep pace with the challenge of cross-lingual international exchanges in multiple fields. Bilingual lexica are important language resources that provide valuable information for semantic equivalence of words across Languages, which has been a research hotspot. However, under resource-poor scenarios, it is difficult to find sufficient cross-lingual knowledge for model training, which requires the exploration of unsupervised learning methods. Unsupervised BLI aims at obtaining the bilingual translation dictionary from monolingual corpora of source and target languages without any cross-lingual knowledge, which can bring huge benefits to NLP tasks in resource-poor fields. Based on previous research, this thesis proposes a neural bilingual lexicon induction (NBLI) method with comparable corpora. This unsupervised method jointly learns word embeddings of source and target languages in the same vector space based on a bidirectional Long Short-Term Memory (BiLSTM) network, and then matches translation pairs through the Cross-domain Similarity Local Scaling (CSLS) retrieval to implement the induction task. It performs well in the general domain and improves the induction performance in the optoelectronic domain between English and Chinese, which breaks the limitation of the pure linear transformation model based on isomorphic assumption in resource-poor scenarios and solves the hubness problem in the bilingual vector space. Furthermore, it requires only comparable corpora in the model training process, and is applicable to various language pairs and specific fields.