SAR-Based Terrain Classification Using Weakly Supervised Hierarchical Markov Aspect Models

We introduce the hierarchical Markov aspect model (HMAM), a computationally efficient graphical model for densely labeling large remote sensing images with their underlying terrain classes. HMAM resolves local ambiguities efficiently by combining the benefits of quadtree representations and aspect models—the former incorporate multiscale visual features and hierarchical smoothing to provide improved local label consistency, while the latter sharpen the labelings by focusing them on the classes that are most relevant for the broader local image context. The full HMAM model takes a grid of local hierarchical Markov quadtrees over image patches and augments it by incorporating a probabilistic latent semantic analysis aspect model over a larger local image tile at each level of the quadtree forest. Bag-of-word visual features are extracted for each level and patch, and given these, the parent–child transition probabilities from the quadtree and the label probabilities from the tile-level aspect models, an efficient forwards–backwards inference pass allows local posteriors for the class labels to be obtained for each patch. Variational expectation-maximization is then used to train the complete model from either pixel-level or tile-keyword-level labelings. Experiments on a complete TerraSAR-X synthetic aperture radar terrain map with pixel-level ground truth show that HMAM is both accurate and efficient, providing significantly better results than comparable single-scale aspect models with only a modest increase in training and test complexity. Keyword-level training greatly reduces the cost of providing training data with little loss of accuracy relative to pixel-level training.

I. Introduction

The Last decade has witnessed an explosion in the number and throughput of airborne and spaceborne terrain sensors using modalities such as the synthetic aperture radar (SAR) [1]. Overwhelming quantities of high-resolution satellite imagery are now available to support accurate earth observations and topographic measurements. Even with modern computers, it is a daunting task to densely label such images with the underlying terrain-type classes. There are three main reasons for this. 1)

Complex and ambiguous image appearance: Within a single terrain class, objects of different materials or layouts or observed from different perspectives often produce markedly different images. Sensor artifacts such as SAR “speckle” make the interpretation even more difficult, as does the fact that, locally, small regions of imagery are often highly ambiguous. For example, a homogeneous dark region in a SAR image may be calm water, a road surface, or a radar shadow, Fig. 1.

2)

The need for high throughput: To process the huge quantities of data that are available, very efficient visual features and classifiers are needed. Stringent accuracy requirements and the incorporation of local context to mitigate aperture effects both tend to increase the computational complexity.

3)

The scarcity of labeled training data: System performance is critically dependent on the amount and accuracy of the available training data. Producing suitable human-supplied annotations can be prohibitively expensive, dangerous, or even impossible. This is especially true when training requires detailed pixel-level labelings.

Fig. 1.

Ambiguity in SAR images. (a) Two patches of similar appearance and the corresponding intensity histograms. (b) Images containing the patches. One patch is the radar shadow of a building, the other is water.