Do They All Look the Same? Deciphering Chinese, Japanese and Koreans by Fine-Grained Deep Learning

We study to what extend Chinese, Japanese and Korean faces can be classified and which facial attributes offer the most important cues. First, we propose a novel way of ob- taining large numbers of facial images with nationality la- bels. Then we train state-of-the-art neural networks with these labeled images. We are able to achieve an accuracy of 75.03% in the classification task, with chances being 33.33% and human accuracy 49% . Further, we train mul- tiple facial attribute classifiers to identify the most distinc- tive features for each group. We find that Chinese, Japanese and Koreans do exhibit substantial differences in certain at- tributes, such as bangs, smiling, and bushy eyebrows. Along the way, we uncover several gender-related cross-country patterns as well. Our work, which complements existing APIs such as Microsoft Cognitive Services and Face++, could find potential applications in tourism, e-commerce, social media marketing, criminal justice and even counter- terrorism.