Optimizing Parametrized Information Bottleneck Compression Mappings with Genetic Algorithms

Preserving relevant mutual information under compression is the fundamental challenge of the information bottleneck method and has numerous applications in machine learning and in communications. The literature describes very successful applications of this concept in quantized detection and channel decoding schemes. The resulting receiver algorithms only use simple lookup tables and process quantization indices, but can achieve performance close to that of conventional high-precision systems. In some applications, however, it is desirable to design a parametrized compression rule instead of a possibly huge lookup table. Genetic algorithms are very powerful generic optimization algorithms which are inspired from the natural evolution of the species. In this paper, we show that genetic algorithms can be used to optimize parametrized compression mappings that aim for maximum preservation of relevant information, especially in cases where standard optimization methods cannot be applied straightforwardly. We exemplarily investigate the receiver-sided channel output quantization as an important application in communications to illustrate the notable performance and the flexibility of the proposed concept.