Universal Fine-Grained Visual Categorization by Concept Guided Learning

Existing fine-grained visual categorization (FGVC) methods assume that the fine-grained semantics rest in the informative parts of an image. This assumption works well on favorable front-view object-centric images, but can face great challenges in many real-world scenarios, such as scene-centric images (e.g., street view) and adverse viewpoint (e.g., object re-identification, remote sensing). In such scenarios, the mis-/over- feature activation is likely to confuse the part selection and degrade the fine-grained representation. In this paper, we are motivated to design a universal FGVC framework for real-world scenarios. More precisely, we propose a concept guided learning (CGL), which models concepts of a certain fine-grained category as a combination of inherited concepts from its subordinate coarse-grained category and discriminative concepts from its own. The discriminative concepts is utilized to guide the fine-grained representation learning. Specifically, three key steps are designed, namely, concept mining, concept fusion, and concept constraint. On the other hand, to bridge the FGVC dataset gap under scene-centric and adverse viewpoint scenarios, a Fine-grained Land-cover Categorization Dataset (FGLCD) with 59,994 fine-grained samples is proposed. Extensive experiments show the proposed CGL: 1) has a competitive performance on conventional FGVC; 2) achieves state-of-the-art performance on fine-grained aerial scenes & scene-centric street scenes; 3) good generalization on object re-identification and fine-grained aerial object detection. The dataset and source code will be available at https://github.com/BiQiWHU/CGL.

I. Introduction

Fine-grained visual recognition (FGVC) stands as a fundamental task in image understanding. Its principal objective lies in the discrimination of similar objects, species, brands and comparable entities, assigning them to highly specific categories that are subsets of broader, coarse-grained classifications [1]. A prevalent paradigm adopted by existing FGVC methodologies is part-driven approach. These methods, whether explicitly or implicitly, are geared towards the identification and utilization of informative components, often referred to as parts, to aptly represent a fine-grained category [2], [3], [4], [5]. In this context, parts typically allude to localized regions or patches within an image [6], [7], [8]. The part-driven approaches have yielded promising results, especially when applied to object-centric images, such as diverse dog species or various aircraft types, which are typically constrained and exhibit favorable front-view perspectives.