Riemannian Diffusion Adaptation over Graphs with Application to Online Distributed PCA

Distributed adaptation and learning recently gained considerable attention in solving optimization problems with streaming data collected by multiple agents over a graph. This work focuses on such problems where the solutions lie on a Riemannian manifold. This research topic is of particular interest for many applications, e.g., principal component analysis (PCA). Although several incremental and consensus algorithms have been proposed, there is a lack of methods designed for general Riemannian manifolds with efficient diffusion strategies. In this paper, we devise two Riemannian diffusion adaptation strategies, namely, adaptation-then-combination (ATC) and combination-then-adaptation (CTA), for decentralized Riemannian optimization over graphs. In the adaptation step, a Riemannian stochastic gradient descent method (SGD) is used to estimate the local solution at each node. In the combination step, the local estimates at the different nodes are combined by computing the weighted Fréchet mean over the neighborhood of each node. We apply our algorithms to online distributed PCA and compare them to both non-cooperative and centralized solutions.