PMHO: Point-Supervised Oriented Object Detection Based on Segmentation-Driven Proposal Generation

Oriented object detection has gained increasing attention due to its ability to detect objects with arbitrary orientations in the field of remote sensing (RS) images. However, the laborious task of annotating oriented bounding boxes (OBBs) presents significant challenges for training a fully supervised arbitrary-oriented object detector. Some existing approaches apply annotated horizontal bounding boxes (HBBs) as weakly supervised signals, in which even HBB annotations require significant human efforts. In this article, we propose a point-to-mask-to-HBB-to-OBB (PMHO) method that achieves weakly supervised oriented object detection by requiring only single-point object annotations. Specifically, we first take the input images and the given annotated points as prompts to obtain the initial segmentation masks, and then present a neighboring mask combination scheme to address the over-segmentation issue and a point-centric mask selection strategy to filtrate the related masks. Based on positive and negative proposal bags transferred from the mask regions, the pseudo-HBB generation network which consists of a classification branch for classification and an instance branch for the localization of individual proposals, aims to generate pseudo HBBs for each object. For further refinement of pseudo HBBs, we present a pseudo-HBB filtering strategy with K-means clustering on features extracted by a CNN model pretrained on a large-scale offline RS dataset. To train the oriented object detector, an inscribed ellipse constraint is proposed to measure the regression loss between predicted OBBs and the given pseudo HBBs. Extensive experiments including OBB detection, pseudo-HBB generation, and ablation studies are conducted on public DIOR and DIOR-R datasets, which demonstrates that our method achieves state-of-the-art performance.