Symmetric Fine-Tuning for Improving Few-Shot Object Detection

Object detection plays a crucial role in automated image analysis by identifying and localizing objects within an image. One-stage Deep Learning (DL)-based object detectors have achieved impressive results, primarily due to large-scale datasets available for training them. However, these approaches rely heavily on abundant labeled data, posing challenges when only a few samples per class are available. To this end, few-shot object detection approaches have been proposed. Among them, fine-tuning the final detection head while keeping the feature extractor/backbone frozen is a commonly used approach for few-shot object detection. This approach effectively utilizes pre-existing knowledge encoded in the backbone, using a small number of samples to learn new object categories. However, in this paper, we argue that fine-tuning only the last layers may limit accuracy and lead to overfitting if the initial layers of the detection head are not adapted for the new task. The data processing inequality, which states that information lost in early network layers cannot be recovered in subsequent ones, supports this argument. To address this issue, we propose a symmetric fine-tuning method that involves both the first and last layers of the detection head, aiming to maintain a fixed trainable parameter budget while strategically selecting parameters for fine-tuning. Experimental results demonstrate the effectiveness and efficiency of this approach and open up several interesting future research directions.