Introduction
Cognitive Radio (CR) is a revolutionary technology which aims to tackle the spectral congestion problem. The basic function of the cognitive engine (CE) is to “sense” available opportunities or spectrum holes and “allocate” the unlicensed users or secondary user (SU) to the spectrum hole. Generalized Frequency Division Multiplexing (GFDM) is a recently proposed non-orthogonal multicarrier waveform which is a potential candidate for 5G wireless technology. Its benefits are flexible carrier aggregation and low out-of-band (OOB) radiation. However the signal suffers from self-intercarrier interference (self ICI) among adjacent subcarriers in the same time-slot [1]. This would definitely cause harmful interference to licensed users (LUs) operating within the spectral band spanned by the GFDM subcarrier. Deployment of GFDM in a multiple antenna based cognitive transceiver can be attractive as the scheme promises higher spectral efficiency in addition to flexible non-contiguous aggregation of spectrum sub-bands. But due to the self-ICI problem along with inter-antenna interference (IAI), MIMO-GFDM leads to inferior bit error rate (BER) performance compared to MIMO-OFDM. Spatial modulation (SM) is an attractive technique which avoids inter-antenna synchronization (IAS) and inter-antenna interference (IAI) among the transmit antennas in a particular time slot [2]. The technique applied to GFDM avoids both self-ICI and IAI and protects the licensed user transmission in the spectrum bands [3]. However, SU transmission can be severely hampered by the interference arising out of licensed user transmissions. The reason is, within a geographical area, SU transmission and LU transmission can take place simultaneously if the maximum tolerable interference threshold (interference temperature) to the LU is satisfied by the SU [4].