Frequency-Domain Optimization of Signal Waveform Shaper in High-Speed Optical Communication Systems Considering ADC Quantization Noise

This work presents an approximate differentiable model for fiber channels that is suitable for short-distance optical communication systems with moderate nonlinear effects. The study investigates the impact of quantization and dispersion on the signal by utilizing a differentiable quantization function. A deep complex neural network was constructed using the PyTorch platform, and the optimization of the signal waveform shaper was performed. The optimized waveform shape successfully learned the characteristic features of quantitative noise under dispersion. At the end of the paper, the joint influence of the quantization process and the dispersion process is studied, and the statistical characteristics of the quantization noise are analyzed. The neural network structure proposed in this paper is of great significance for studying the interaction between quantitative effects and dispersion effects.