Loading [MathJax]/extensions/MathZoom.js
Analyzing Pre-Trained Neural Network Behavior with Layer Activation Optimization | IEEE Conference Publication | IEEE Xplore

Analyzing Pre-Trained Neural Network Behavior with Layer Activation Optimization


Abstract:

Image classification and object recognition with neural networks could have applications in aesthetically-focused branches of the humanities, such as landscape architectu...Show More

Abstract:

Image classification and object recognition with neural networks could have applications in aesthetically-focused branches of the humanities, such as landscape architecture. However, such methods require either the assembly of a massive, domain specific labeled data set or use of network weights initialized on another data set, a technique known as transfer learning. Transfer learning research has established that a pre-trained convolutional neural network (CNN) can achieve high accuracy on new image recognition tasks with relatively few training images. In practice, pre-trained tends to mean pre-trained on ImageNet, the standard dataset for computer vision research. Experiments have shown that the dataset on which a pre-trained model was originally optimized can quantitatively bias it. The goal of this project was to design an experiment to qualitatively analyze how the dataset used to initialize a pre-trained classification system affects its behavior at progressive network layers using feature visualization strategies. We initialized two ResNet-18 CNNs with weights pre-trained on ImageNet and the Places365 dataset, respectively, and fine-tuned them for a new classification task on a landscape image dataset which we collected. Using class activation optimization methods taken from the deep visualization literature, we compared the network filters at several hidden layers and the final output layers. The class activation optimization results show that even at early stages in the networks, their neurons exhibit notably different behavior. Accordingly, we show both that feature visualization techniques can be used to qualitatively study the effect of original training data on transfer learning and, consequently, that the homogeneous use of ImageNet in computer vision experiments may have notable implications for model behavior.
Date of Conference: 24-24 April 2020
Date Added to IEEE Xplore: 02 June 2020
ISBN Information:
Conference Location: Charlottesville, VA, USA

I. Introduction

When facing an image classification problem with only a small dataset of training images, using a pre-trained convolutional neural network (CNN) to initialize the feature weights before fine-tuning on the data becomes an attractive option. The motivation for the project came from this scenario, where a group working to digitize a seminal work on landscape architecture approached the authors to build a machine learning model to classify landscape images using the approximately 600 images in the book as training data. Working with a highly specialized dataset led to concerns about the easily available pre-trained models (trained on ImageNet) and their efficacy on the dataset. An alternate dataset (Places365) was sought to see how models trained on it compared in their final layers after being fine-tuned on our data. Feature visualization arose as a way to explore how the layers of the CNN were activated as a result of training. Feature visualization is a field concerned with creating semantically intelligible representations of a neural network’s learned weights. Past work has applied feature visualization techniques to analyze the transfer learning process. Here we applied the same techniques to understand how the training data itself affects that process. CNNs appear to learn increasingly task-specific representations at each successive layer. Therefore, we hypothesized that the two networks would behave most similarly at the shallowest layers and then diverge relative to depth. In the next section we discuss the research that informs this project.

References

References is not available for this document.