Complexity Reduction of Neural Networks for Nonlinear Fourier Transform in Optical Transmission Systems

The nonlinear Fourier transform (NFT) method was proposed a decade ago as a tool for the mitigation of nonlinear impairments in fiber-optic communication systems. Within the NFT-based transmission methods, we use the parameters of the so-called nonlinear modes of non-decaying signals as data carriers. Naturally, to make the NFT transmission more practical, we need an efficient low-complexity method for performing the NFT processing of our signals at the transmitter and receiver. Utilizing neural network models has recently become a popular direction in signal processing, including studies related to the application of neural networks for NFT computations. Complexity reduction (model compression) is an important step for the practical implementation of neural networks in signal-processing applications. In this work, we develop a weight clustering compression technique applied to the convolutional neural network (CNN) performing a direct NFT operation. This CNN is used to retrieve the phases of non-decaying nonlinear solutions (nonlinear modes) to the nonlinear Shrödinger equation. We found that satisfactory model performance is retained even when the number of clusters falls to four. When utilized at the receiver of a communication system, our compressed model demonstrates the data restoration quality below the 7% forward error correction (FEC) threshold with the number of clusters starting from six where compression of the model reaches 99%.