Soft Weighted Ordinal Classification for Monocular Height Estimation in Remote Sensing Image

Estimating height information from a single remote sensing image is a critical component for 3D perception. Recent methods formulate it as a dense height prediction task based on regression loss functions. However, the regression accuracy is limited by the infinite continuous solution space. In this paper, we propose the soft weighted ordinal (SWO) classification loss for height prediction model to convert the regression problem with infinite continuous values into the classification problem with finite discrete values. which greatly improves the accuracy of high estimation. Specifically, we first define the discrete height rule and introduce the distance penalty metric to transform the continuous ground truth height value to the soft probability distributions. This is then used as supervised information to optimize the pixel-wise classification model. Finally, we utilize soft weighted summation to generate continuous height values in the inference phase. The proposed SWO classification loss can be used directly with existing dense prediction structures whose performance can be strengthened by direct replacement of the loss functions. Comprehensive experiments on the IS-PRS Vaihingen dataset show that the proposed method has achieved promising results.

1. Introduction

Monocular height estimation aims to assign each pixel to an accurate height value in a single remote sensing image, which plays a critical role in many applications, e.g., urban man-agement, land planning, and disaster monitoring. A stream of existing research regard height estimation as a regression task [1], [2]. However, the regression problem, as a non-convex optimization, makes it difficult for the model to realize the accurate prediction from the input image to the ground truth values in the case of the infinite continuous solution space, which often leads to sub-optimization. Fig. 1.

An example patch of 3d reconstructed remote sensing image. The height of the object varies greatly.