Fusion of Visual and Infrared Images for Face Recognition

This paper proposes a robust face recognition method by fusing images acquired from visual and infrared (IR) sensors. Although current 2D image face recognition systems have reached a certain level of maturity, the performance of these systems has been limited by external conditions such as pose, expression and illumination. To alleviate some of these problems, infrared sensor based methods have been suggested. These methods showed very good performance when used in illumination variation environments. However, one of the main drawbacks of using IR sensors for recognition is that they are very sensitive to ambient temperatures. To solve this problem, we suggest a face recognition method that fuses both visual and IR techniques. We developed a visual and IR face image database with photographs taken under a wide range of imaging conditions. With this database, we are able to produce full combinations of comparative experiments in the field of face recognition. Our experiments showed that when using both modalities, results are far better in terms of recognition performance than when using only one modality. The overall average performance was observed to have improved under all imaging conditions

1. INTRODUCTION

Although current 2D image face recognition systems have reached a certain level of maturity, the performance of these systems has been limited by external conditions such as pose, expression and illumination. Especially, the influence of varying ambient illumination on the subject is known to be one of the major confounding factors for visual face recognition [1]. To alleviate these conditions, 3D sensor-based or infra-red (IR) sensor-based methods have recently received significant attention [2] [3] [4]. The IR-based method particularly showed very good performance in an illumination variation environment [2]. Also, the IR-based method has been proved to be very robust in terms of expression variation [3]. On the other hand, a weakness of the IR-based method is that variations in environmental conditions like temperature or wind can cause similar problems to changes in illumination do for visual images.