Robust Hyperspectral Object Tracking by Exploiting Background-Aware Spectral Information With Band Selection Network

Deep color trackers mainly use pretrained convolutional neural networks (CNNs) for classification and regression, but it is difficult to discriminate targets in complex backgrounds for its limited spectral information. Compared with color video, hyperspectral videos provide better discriminative ability due to the abundant material-based information. However, it is hard to train a robust deep model for hyperspectral videos. The key issues are that there exists much redundant information in hyperspectral videos and the training samples are inadequate. In this letter, a new background-aware hyperspectral tracking (BAHT) method is designed for hyperspectral tracking task. Our method first designs a background-aware band selection module to preserve bands that can better recognize a target from backgrounds. Then the selected bands are input to the backbone networks, which are pretrained on color videos, to describe the appearances of targets with deep semantic features. Experiments on the hyperspectral video tracking dataset illustrate the good performance of BAHT tracker compared with popular color and hyperspectral trackers.

I. Introduction

Visual tracking is a widespread technique in computer vision and it can serve for high-level applications [1], [2], [3]. For single-object tracking task, it demands to predict the states of a given object in successive video sequences [4]. At present, color trackers achieve good performance due to the rapid development on correlation filters (CFs) and Siamese network, but features extracted from color videos are limited in presenting the intrinsic information of targets, and it results in failing in complex backgrounds. Different from color videos, there are abundant spectral information without losing spatial structures in hyperspectral videos, which can effectively improve the discriminative ability of extracted features [5], [6].