CycleGAN based Speech Enhancement Using Time Frequency Masking

The process of separating one or more desired speech signals from a collection of mixed speech signals or noise is known as audio source separation. The proposed work aims to restore clear audio by reducing noise in mixed audio. These technologies, which are now among the most researched in the field of audio signal processing, are essential to the accomplishment of successful noise removal. Initial source separation is accomplished through time frequency filtering and conditional adversarial networks are used to enhance the quality of the separated sound. A pitch tracking method is used to determine the pitch tracks that are present in the mixed speech and binary masks are constructed that correspond to each pitch track and its harmonics. Using conditional adversarial networks built on CycleGAN and working on the spectrogram images, the noise is removed and speech source is separated. Cycle GAN is used to train the masked audio signal spectrograms and the clean ground truth spectrograms. By using the inverse Short-Time Fourier Transform, the reconstructed spectrogram is transformed back into speech signals. The model's effectiveness is evaluated objectively and the performance of the proposed model is comparable to that of the baseline models.