Loading [a11y]/accessibility-menu.js
Geometry-Guided Street-View Panorama Synthesis From Satellite Imagery | IEEE Journals & Magazine | IEEE Xplore

Geometry-Guided Street-View Panorama Synthesis From Satellite Imagery


Abstract:

This paper presents a new approach for synthesizing a novel street-view panorama given a satellite image, as if captured from the geographical location at the center of t...Show More

Abstract:

This paper presents a new approach for synthesizing a novel street-view panorama given a satellite image, as if captured from the geographical location at the center of the satellite image. Existing works approach this as an image generation problem, adopting generative adversarial networks to implicitly learn the cross-view transformations, but ignore the geometric constraints. In this paper, we make the geometric correspondences between the satellite and street-view images explicit so as to facilitate the transfer of information between domains. Specifically, we observe that when a 3D point is visible in both views, and the height of the point relative to the camera is known, there is a deterministic mapping between the projected points in the images. Motivated by this, we develop a novel satellite to street-view projection (S2SP) module which learns the height map and projects the satellite image to the ground-level viewpoint, explicitly connecting corresponding pixels. With these projected satellite images as input, we next employ a generator to synthesize realistic street-view panoramas that are geometrically consistent with the satellite images. Our S2SP module is differentiable and the whole framework is trained in an end-to-end manner. Extensive experimental results on two cross-view benchmark datasets demonstrate that our method generates more accurate and consistent images than existing approaches.
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence ( Volume: 44, Issue: 12, 01 December 2022)
Page(s): 10009 - 10022
Date of Publication: 07 January 2022

ISSN Information:

PubMed ID: 34995180

Funding Agency:


1 Introduction

Given a satellite image, such as Fig. 1a, what would one see when standing at the location of the image center? In this example, one would reason that there is a fork in the road, a tree inside the fork, and grass elsewhere. This satellite to street-view image synthesis task aims to generate an omni-directional street-view panorama captured at a location corresponding to the center of the given satellite image. Our goal in this work is to synthesize a street-view panorama with scene structures that are as geometrically consistent with the satellite image as possible, while preserving visual similarity with the ground-truth panorama.

References

References is not available for this document.