Interactive Encoding and Decoding for One Way Learning: Near Lossless Recovery With Side Information at the Decoder

A source coding paradigm called interactive encoding and decoding (IED) is considered for a source network where a finite alphabet source X is to be encoded, and another finite alphabet source Y correlated with X is available only to the decoder as a helper. The optimal performance achievable asymptotically (OPAA) by IED is investigated, where the performance is measured as the average number of bits per symbol exchanged by the encoder and decoder until the decoder learns X with high probability. First, it is shown that for any stationary (X, Y), the OPAA by IED is given by the conditional entropy rate H(X|Y) of X given Y. This is in contrast with noninteractive Slepian-Wolf (SW) coding, where the OPAA is shown in general to be strictly greater than H(X|Y) when (X, Y) is not ergodic. Second, for a memoryless source pair (X, Y), it is shown that IED approaches H(X|Y) faster than SW coding does. Finally, it is demonstrated that one can convert any classical universal data compression algorithm with side information to a universal IED algorithm for the class ¿ of all stationary ergodic source pairs. In contrast, universal SW coding algorithms for the class ¿ do not exist.

I. Introduction

In the traditional source coding with one source, one encoder, and one decoder, the encoder acts much like a master, sending an instruction, i.e., a codeword, to the decoder, and the decoder acts like a slave, simply executing the instruction received from the encoder. The master-slave paradigm of encoding and decoding has also been largely adopted in distributed source coding [23], [2], and is now commonly referred to as Slepian-Wolf (SW) coding in the case of distributed source coding. For example, in their seminal work [23], Slepian and Wolf showed that for the source network shown in Fig. 1, where the source is to be encoded and transmitted to the decoder, and the source correlated with is available only to the decoder as a helper, if is memoryless, it is still sufficient for the encoder to send bits per symbol to the decoder, which, upon receiving these bits, can recover the source without any essential loss with the help of . This result was later extended in [2] to the case of stationary ergodic sources .