Single-Channel Speech Enhancement Based on Multi-Scale Gradients Generative Adversarial Networks

Recently, Generative Adversarial Network (GAN) has been gradually applied in the field of speech enhancement and has achieved good performance. However, the traditional GAN-based enhancement methods have such problems as unsatisfactory quality of generated speech, instability of model training, etc. In this work, a speech enhancement model based on Multi-Scale Gradients Generative Adversarial Networks is proposed, and the problem existed in tradition GAN is solved by allowing gradient flow to flow at multiple dimensions from discriminator to generator. Specifically, in the proposed model, a multi-output generator is used to generate enhanced speech in different dimensions and several sub-discriminators are adopted to distinguish the real speech and the generated speech from different dimensions. In addition, different weight coefficients are set for each sub-discriminator to optimize the generation ability of generator. The proposed model enhances the speech signal from multiple dimensions, so that the generated speech signal is closer to the real signal, and then improves the overall speech enhancement effect. The experimental results on the TIMIT show that the proposed model not only improves the performance of speech enhancement but also enhances the stability of the model training.