Artificial Bandwidth Extension Using a Conditional Generative Adversarial Network with Discriminative Training

The aim of artificial bandwidth extension is to recreate wideband speech (0 - 8 kHz) from a narrowband speech signal (0 - 4 kHz). State-of-the-art approaches use neural networks for this task. As a loss function during training, they employ the mean squared error between true and estimated wideband spectra. This, however, comes with the drawback of over-smoothing, which expresses itself in strongly underestimated dynamics of the upper frequency band. We previously proposed to tackle this problem by discriminative training, i.e., a modification of the loss function that is designed to improve the separation between fricatives and vowels. Other authors instead took a generative adversarial network (GAN) approach. This was motivated by the fact that GANs demonstrated big reductions of over-smoothing in speech synthesis. In this work, we combine these two approaches. In particular, we show that conditional GANs improve the speech quality by a CMOS score of 0.28 compared to GANs while the combined approach yields an improvement of 0.84.

1. INTRODUCTION

The aim of artificial bandwidth extension (BWE) is to improve the quality of narrowband (NB) telephony speech (0 - 4 kHz) by artificially extending the signal to wideband (WB), i.e., a bandwidth of 0 - 8 kHz. This is achieved by estimating the missing upper band (UB) between 4 and 8 kHz based on the NB spectrum. In most approaches, the signal is first transformed to the frequency domain and then decomposed into a spectral envelope and an excitation signal. Both these parts are then extended separately, which simplifies the estimation problem. The extension of the excitation signal is typically achieved with rather simple methods like spectral shifting. This is perfectly sufficient, as the introduced degradation of the speech quality is rather low [1], [2]. In early BWE approaches, the estimation of WB features was often performed with codebooks, Gaussian mixture models (GMM), or hidden Markov models (HMM).