Bit Error Probability of MQAM in the Presence of Phase Noise

In Gray coded rectangular and square quadrature amplitude modulation (QAM), for each bit, the bit decision boundaries are distributed only along one dimension. Consequently, the exact calculation of bit error probability (BEP) of rectangular and square QAM in the additive white Gaussian noise (AWGN) channel results in simple one-dimensional (1-D) error/Q-functions. This is while the symbol decision boundaries involve two dimensions. This paper presents exact and general expressions for the BEP of Gray coded M-ary rectangular and square QAM in the presence of phase noise in the AWGN as well as flat-fading channels. In doing so, we first derive exact expressions for the BEP of MQAM in the presence of a fixed phase error. The expression will be a weighted sum of 1-D complementary error functions for the AWGN channel. For the flat-fading channels, these formulas consist of only single integrals or simple functions. We then integrate the formulas across noise distribution, to derive the average BEP expression of Gray coded MQAM in the presence of random phase error. Moreover, we provide an exact expression for the bit error floor (BEF) of MQAM in the presence of phase noise in the AWGN and fading channels. The exact expressions include numerous terms for higher order MQAM. Considering the most dominant terms of the exact expressions, we also propose simpler expressions. Simulation results validate the accuracy of the proposed exact expressions for arbitrary levels of input SNR. Moreover, the results show very good performance for the approximate expressions.