DFT-CAM: Discrete Fourier Transform Driven Class Activation Map

Class activation map methods (CAMs) serve one of the key roles in explainable artificial intelligence (XAI) and are recently being applied to weakly-supervised object localization, showing great potential in many applications. However, the current CAM methods still have room for improvement regarding the weakly-supervised object localization task. In this paper, we proposed DFT-CAM, a novel CAM method that combines discrete Fourier transform based feature encoding with an orthogonality-based feature selection scheme. DFT-CAM doesn’t require any training, better captures semantic information, aggregates only the most representative convolutional features, and improves the downstream tasks such as weakly-supervised object localization. The experimental results show promising performance compared to other CAM methods across different deep learning classification networks.

1. INTRODUCTION

Deep learning has revolutionized computer vision tasks. Deep image classification networks such as CNN [9], VGG [17], residual network [6] and their variants; image segmentation networks such as fully convolutional network [10], U-Net [13], DeepLab [3] and their variants show promising results in many applications of image analysis and computer vision. Two significant problems with deep learning are: (1) black box models that lack explainability and interpretability; and (2) need for large amounts of annotated data. Explainability is particularly critical for high stakes fields such as medicine.