Improving distributed representation by feature selection of Wikipedia

Distributed representation plays an important role in many application of Natural Language Processing (NLP). Today, Word2Vec model has been getting an attention against the backdrop of the easy access to enormous language data from the Internet such as Wikipedia. For the effective use of Word2Vec, we have to concern not only about the improvement of the method itself but also about the process of making training data. In this paper, we demonstrate that adequate selection of training data can make a great improvement of the performance of Word2Vec compared to existing research. We also confirmed that Wikipedia dump data is not a good source of training data as is.