I. Introduction
Lattice reduction plays an important role in numerous fields of mathematics, computer science [1]–[4], and cryptology [5], [6]. Recently, lattice reduction turned out to be extremely useful for detection and precoding in wireless multiple-input multiple-output (MIMO) systems. for lattice type modulation, the optimal maximum-likelihood (ML) decoding can be modeled as the closest vector problem (CVP), which can be solved by the sphere decoding algorithms [7]–[11]. However, the complexity of the sphere decoding algorithms increases exponentially with the number of transmit antennas [7], [8], [12]. It has been found that lattice reduction, used as an efficient preprocessor, has the potential to achieve high performance for low-complexity sub-optimal decoding algorithms such as zero-forcing (ZF) decoding and successive interference cancellation (SIC) decoding [13]–[17]. The basic idea is to view the channel matrix as a lattice basis (generator) matrix, and lattice reduction can improve the orthogonality defect of the basis matrix. Then the detection/precoding problem is solved based on the reduced basis to improve performance and complexity of a low-complexity decoding algorithm. See [18] for an introduction to lattice reduction and a survey of its applications in wireless communications.