Inter-Channel Fiber Nonlinearity Mitigation in High Baud-Rate Optical Communication Systems

Kerr nonlinearity in long distance fiber-optic link imposes a fundamental limitation to the capacity of wavelength division multiplexed (WDM) optical communication systems. Solving the inverse-propagating nonlinear Schrödinger equation (NLSE) is in principle a fundamental methodology to mitigate fiber nonlinear impairments. However, solving multi-channel coupled NLSE is too complexed to be implemented commercially. It is of great importance to compensate for nonlinear distortions in high baud-rate WDM systems based on the information of one single target channel. In this article, we experimentally study fiber nonlinearity mitigation in an 8 × 512 Gb/s (64Gbaud) polarization division multiplexed (PDM) 16-ary quadrature amplitude modulation (16-QAM) Nyquist-WDM system with 960 km standard single mode fiber (SSMF) loop transmission. The digital back-propagation (DBP) algorithm based on the target channel is applied, which can compensate for intra-channel nonlinearity. The inter-channel nonlinearity of cross phase modulation (XPM) induces nonlinear polarization scattering and nonlinear phase noise, which are partially correlated over adjacent pulses in high baud-rate systems. We propose an advanced nonlinear polarization crosstalk canceller (NPCC) with a novel decision feedback (DF) update strategy that is called the DF-NPCC, which incorporates the iterative process of the least mean square (LMS) algorithm and can thus mitigate the correlated nonlinear polarization crosstalk and nonlinear phase noise. Through nonlinearity mitigation, the DBP together with the DF-NPCC can achieve a 0.77 dB Q
²
factor promotion. The DF-NPCC can suppress the remaining nonlinear distortions after the single channel DBP, which corresponds to a 0.36 dB Q
²
factor improvement due to inter-channel nonlinear mitigation. Our work indicates XPM distortions can be mitigated with low complexity digital signal processing such as the DF-NPCC based on the single channel information, which provides an attractive solution to compensate inter-channel fiber nonlinearity in high baud-rate Nyquist-WDM systems.