The Distributed Karhunen–Loève Transform

The Karhunen-Loeve transform (KLT) is a key element of many signal processing and communication tasks. Many recent applications involve distributed signal processing, where it is not generally possible to apply the KLT to the entire signal; rather, the KLT must be approximated in a distributed fashion. This paper investigates such distributed approaches to the KLT, where several distributed terminals observe disjoint subsets of a random vector. We introduce several versions of the distributed KLT. First, a local KLT is introduced, which is the optimal solution for a given terminal, assuming all else is fixed. This local KLT is different and in general improves upon the marginal KLT which simply ignores other terminals. Both optimal approximation and compression using this local KLT are derived. Two important special cases are studied in detail, namely, the partial observation KLT which has access to a subset of variables, but aims at reconstructing them all, and the conditional KLT which has access to side information at the decoder. We focus on the jointly Gaussian case, with known correlation structure, and on approximation and compression problems. Then, the distributed KLT is addressed by considering local KLTs in turn at the various terminals, leading to an iterative algorithm which is locally convergent, sometimes reaching a global optimum, depending on the overall correlation structure. For compression, it is shown that the classical distributed source coding techniques admit a natural transform coding interpretation, the transform being the distributed KLT. Examples throughout illustrate the performance of the proposed distributed KLT. This distributed transform has potential applications in sensor networks, distributed image databases, hyper-spectral imagery, and data fusion

I. Introduction

The approximation or compression of an observed signal is a central and widely studied problem in signal processing and communication. The Karhunen–Loève transform (KLT), also referred to as principal component analysis (PCA), [3]–[5], has always played a pivotal role in this context. Assume, for instance, that the observed signal is a random vector with covariance matrix and that the statistics of the source are known. Then to solve the approximation problem, one can apply the KLT to to obtain uncorrelated components and the optimal linear least squares -order approximation of the source is given by the components corresponding to the largest eigenvalues of . In the case of compression, the uncorrelated components can be compressed independently and more rate can be allocated to the components related to the largest eigenvalues of , according to a principle that is sometimes referred to as “reverse water-filling,” see, e.g., [6, p. 349]. This compression process is widely known as transform coding and, if the input source is a jointly Gaussian source, it is possible to show that it is optimal [7]. For an excellent review on transform coding and a discussion of the optimality of the KLT in this context, we refer to the exposition in [8]. Fig. 1.

The distributed KLT problem: Distributed compression of multiple correlated vector sources. Encoder provides a description of its observation. This paper investigates the case where is a -dimensional approximation of the observation, and the case where is a bit stream of bits per observation.