Controllable Base Class Synthesis with Generative Diffusion Model for Few-Shot Object Detection

Few-shot object detection (FSOD) models detect new objects with scarce training samples on novel data using pre-trained models on large-scale base data. Recent FSOD works use a stable diffusion model to generate new synthetic objects for novel classes. However, their synthetic objects lack diversity due to fixed sizes, shapes, or uncontrollable occlusions. Besides, most previous FSOD methods forget to effectively exploit base classes, which cannot leverage data diversity to create distinguished features and enhance the generalization.A recent method [1] exploits and shows potential results while it provides more base classes. The simple method only uses fake classes, which leads to the ambiguity between the fake and real base classes and then reduces the performance when the model gets more data. To solve the above issues, we introduce HBC using Fine-grained Data generation using Diffusion (FDD) and Three-Stage Training (TST) to create and leverage large-scale data. Specifically, FDD constructs hierarchical information from the base and extended real classes in ImageNet to find new fine-grained classes. The new objects of fine-grained classes that are added to the base image using a controllable diffusion model have different sizes, shapes, and positions based on the base bounding boxes. The method creates real fine-grained classes with rich synthetic samples and minimizes the ambiguity between these classes and the original base classes. Then, TST uses the generated data to enhance the distinguished ability of features and improve the detector without negatively affecting the performance of base classes. In the end, we provide comprehensive results to demonstrate the effectiveness of our proposed method in FSOD settings. Our method improves averaged 4.5% and 2.8% nAP50 compared to the baseline and previous work, respectively.