Impact and Mitigation of Polarization- or Mode-Dependent Gain in Ultra-Long-Haul Systems

We show by analysis and simulation of a multisection model that polarization- or mode-dependent gain from single-mode semiconductor optical amplifiers or multimode erbium-doped fiber amplifiers causes the channel capacity with minimum-mean-square error detection to become substantially lower than that of optimal maximum likelihood detection. However, the capacity loss can be mitigated with techniques such as successive interference cancellation (SIC) that retain reasonable complexity. We reaffirm that frequency diversity enables the outage capacity to approach the average capacity and evaluate the performance of frequency-domain SIC equalizers for single-carrier transmission on frequency-selective optical fiber channels operating at 64 Gbaud up to 15,000 km.