Waveform Design for High-Order QAM Faster-Than-Nyquist Transmission in the Presence of Phase Noise

The state-of-the-art radio-frequency (RF) devices limit the deployment of extremely high-order quadrature amplitude modulation (QAM) formats (e.g., 16384-QAM) to meet the high-capacity demand on microwave backhaul links. This paper turns to faster-than-Nyquist (FTN) transmission using lower-order constellations and lower-cost RF devices as a solution to the demand. To realize low-complexity interference cancellation, we pre-equalize the FTN-induced inter-symbol interference at the transmitter by using Tomlinson-Harashima precoding (THP), while at the receiver suppressing the phase noise (PHN) generated by the RF local oscillators with pilot symbol assisted approaches. However, the THP may distort the pilots, which degrades the performance of PHN compensation. To resolve this problem, we propose two pilot designs that are distortion-free to precisely estimate the PHN samples. Moreover, we derive a closed-form expression of the symbol detection signal-to-noise ratio (SNR), in terms of the THP-FTN waveform parameters. With the SNR expression, a waveform optimization procedure is developed to maximize the SNR and enhance the achievable FTN capacity. The proposed scheme is validated in the simulated platform of 4096-QAM microwave link. The results demonstrate that the FTN signaling achieves the system capacity equivalent to that of the 16384-QAM Nyquist signaling with an SNR gain of 5.8 dB.