A Novel Adversarial Training Scheme for Deep Neural Network based Speech Enhancement

In this work, we propose a novel representation-learning technique for Deep Learning-based Speech Enhancement algorithms inspired by Domain-Adversarial training. A gradient reversal layer and an additional network are employed, only at training time, to explicitly enforce a representation that is orthogonal to the additive noise in the input signal. We show that such learning scheme, which can be applied easily to most mask-based Deep Neural Network Speech Enhancement approaches, is able to improve the denoising performance when used in conjunction with scale-invariant signal-to-distortion ratio loss and allows to reach state-of-the-art performance with no computational overhead at run-time. In particular, on the commonly used VoiceBank-DEMAND benchmarking dataset, we improve signal-to-distortion ratio and signal-to-noise ratio over the nonadversarial model and CSIG, COVL and CBAK over other, state-of-the art, adversarial training techniques.

I. Introduction

Recently, monaural Speech Enhancement (SE) has seen a significant leap in performance thanks mainly to the adoption of supervised-learning techniques based on Deep Neural Networks (DNNs). In fact, DNN models, have been shown to be able to significantly improve intelligibility measures like the Short-Time Objective Intelligibility measure (STOI) [1], a feat which was not possible with non-supervised classical approaches unless very constrained situations were considered [2]. In the supervised approach, the model learns a direct mapping between noisy input features and output clean features by minimizing a loss function between the ground truth clean example and the output of the model.