Recurrent Neural Network Based Joint Equalization and Decoding Method for Trellis Coded Modulated Optical Communication System

In this paper, a joint equalization and decoding method based on recurrent neural networks (RNNs) is proposed for trellis coded modulation (TCM) systems. For traditional methods based on concatenated equalizers and Viterbi decoder, the information may be lost between the equalization and decoding. However, our proposed joint method can obtain the information bits directly from the received symbols and the error also can be corrected in iterations. In two-dimensional eight-level pulse amplitude modulation (2D-PAM8) links, we experimentally demonstrate that, compared with the traditional method based on Volterra filter and Viterbi decoder, the proposed joint method based on RNNs with similar complexity can achieve bit-error rate (BER) performance improvements of 1 dB, 1 dB, and 2 dB in 87.5 Gb/s, 100 Gb/s, and 112.5 Gb/s transmissions at 7% hard-decision forward-error-correction (HD-FEC) limit, respectively. For the 10 km standard single-mode fiber (SSMF) link at 87.5 Gb/s, the traditional method cannot get the BER below the 7% HD-FEC limit whereas the joint RNNs method with similar complexity can get the BER below the 7% HD-FEC limit. As the nonlinearity increases, the proposed joint method can achieve greater improvement of BER performance.