Cross Layer Weakly Supervised Data Augmentation Network for Offline Signature Verification

Current Offline Signature Verification(OSV) methods based on deep learning predominantly rely on deep semantic information for discrimination, while often neglecting shallow visual features. This neglect leads to a lack of thorough exploration of the signature’s inherent characteristics, thereby limiting the extraction of stable features from limited data. Consequently, these methods perform suboptimally in tasks involving cross-modal and consistency verification between electronic and traditional paper-based signatures. In response to the above issues, we propose a Cross Layer Weakly Supervised Data Augmentation Network (CWSA), inspired by the process of humans analyzing similarities between two things. CWSA utilizes the features from each layer of the CNN for discrimination, and combines weakly supervised learning with attention-guided to map the spatial distribution of clues. Then, based on this distribution, we generate a series of augmentation images with varying granularities during the training process. This approach is designed to expand the training set and foster a greater initiative in the model’s learning process. Finally, we propose a multi-step iterative training strategy to implement the previously discussed methods. This strategy enables mutual learning between the deep and shallow layers of the network model, ensuring comprehensive utilization of features across all levels to boost the accuracy of verification. We tested the proposed method on datasets in different languages and modalities: CEDAR, BHSig260, MSDS, ChiSig. The experimental results indicate the effectiveness of CWSA in various OSV tasks.