Prototype-Based Discriminative Feature Learning for Kinship Verification

In this paper, we propose a new prototype-based discriminative feature learning (PDFL) method for kinship verification. Unlike most previous kinship verification methods which employ low-level hand-crafted descriptors such as local binary pattern and Gabor features for face representation, this paper aims to learn discriminative mid-level features to better characterize the kin relation of face images for kinship verification. To achieve this, we construct a set of face samples with unlabeled kin relation from the labeled face in the wild dataset as the reference set. Then, each sample in the training face kinship dataset is represented as a mid-level feature vector, where each entry is the corresponding decision value from one support vector machine hyperplane. Subsequently, we formulate an optimization function by minimizing the intraclass samples (with a kin relation) and maximizing the neighboring interclass samples (without a kin relation) with the mid-level features. To better use multiple low-level features for mid-level feature learning, we further propose a multiview PDFL method to learn multiple mid-level features to improve the verification performance. Experimental results on four publicly available kinship datasets show the superior performance of the proposed methods over both the state-of-the-art kinship verification methods and human ability in our kinship verification task.

I. Introduction

Recent advances in psychology and cognitive sciences [2], [6], [7], [24] have revealed that human face is an important cue for kin similarity measure as children usually look like their parents more than other adults because children and their parents are biologically related and have overlapped genetics. Inspired by this finding, there have been some seminal attempts on kinship verification via human faces, and computer vision researchers have developed several advanced computational models to verify human kinship relations via facial image analysis [15]–[18], [31], [42], [45], [49], [50], [52], [56], [57]. While there are many potential applications for kinship verification such as missing children searching and social media mining, it is still challenging to develop a robust kinship verification system for real applications because there are usually large variations on pose, illumination, expression, and aging on facial images, especially when face images are captured in unconstrained environments. While the past five years have witnessed encouraging progress in this area [15], [16], [18], [26], [42], [45], [49], [50], [52], [56], [57], the problem of kinship verification still remains unsolved because it is extremely challenging to extract kin-related features from human ages, especially when face images are captured in the wild.