Eyes Attribute Editing Assisted by Dual-Coordinate System and Multi-Probability Fusion Prediction

The eye area, which can effectively convey social information and behavioral intentions, contains most of the information in facial images. However, the occlusion of glasses makes it difficult to accurately obtain the eye information in some facial images. Although the existing attribute editing method can remove glasses from face images, there are existing drawbacks lay in incomplete eye information and insufficient disentanglement after the glasses were removed (e.g. the background and hair color also changed when the glasses were removed), unable to get real and natural eye editing results. To address this problem, we propose an eye attribute editing model assisted by dual-coordinate system and multi-probability fusion prediction. The former one is designed to enhance the position awareness of inpainting to directly focus on the eyes area while the later one could integrate both local and global information of the target area. Specifically, the first stage of our method is to remove the glasses, and then use multi-probability fusion prediction method to inpaint the eye area to obtain a reasonably complete eye editing result supported by dual-coordinate system. On the one hand, we designed a dual-coordinate positioning module that uses a cascaded classifier and a dual-coordinate system to locate the human eye height area mask, which effectively solves the problem of inconsistent eye position height in facial images and determines the editing area for inpainting after the glasses are removed. On the other hand, we employed a multi-probability fusion prediction algorithm to supplement and enhance the ocular information, which can combine a distance-based local probability and an adaptive global probability, resulting in improved editing results. Quantitative and qualitative evaluations show that our method outperforms the state-of-the-art methods.