Geographic Optimal Transport for Heterogeneous Data: Fusing Remote Sensing and Social Media

The fusion of heterogeneous remote sensing and social media data can fill the gaps in satellite image collections and improve the spatiotemporal resolution of the available data sets. As a result, it is being gradually adopted in multimodal data analytics. Generally, the fusion of heterogeneous geographic data faces the following issues: 1) the probability density functions may differ from different data sources and 2) the geolocations may not be well aligned. The former one can be generally solved by performing an alignment of representations in the source and target domains using, for instance, domain adaptation. The latter issue is seldom considered in the fusion of heterogeneous geographic data. In this article, we present a new method called geographic optimal transport (GOT), which aims at aligning representations and geolocations in a simultaneous fashion. A flood event that took place in 2013 in Boulder, CO, USA, is taken as a case study to evaluate our GOT method. Here, we consider two remote sensing features derived from water indicators, i.e., the normalized difference vegetation index (NDVI) and the normalized difference water index (NDWI), for the fusion of Landsat 8 imagery and Twitter data. A comparison between our newly developed GOT and the traditional optimal transport (OT) is performed. Experimental results demonstrate that the proposed GOT can accurately align spatially biased georeferenced tweets to the flood phenomena, leading to the conclusion that GOT can effectively fuse heterogeneous remote sensing and social media data.