Acoustic Model Adaption Using x-vectors for Improved Automatic Speech Recognition

Traditionally, i-vectors have been used for acoustic model adaptation and have been shown to capture the speaker and channel information quite robustly. Although x-vectors are known to be more robust for speaker verification, they have not, yet, been used successfully for large-scale ASR adaptation. This paper investigates the use of x-vectors for large vocabulary hybrid acoustic model adaptation. We keep our study limited to one x-vector per utterance due to the high computational cost of x-vector extraction. With ASR experiments conducted on Librispeech and TED-LIUM datasets using x-vector extractor trained on Voxceleb, we demonstrate three important findings. First, we show that only x-vectors concatenated with acoustic features, are not sufficient for improving performance compared to i-vectors. Second, a Word Error Rate improvement of 5.4% and 4.2% is observed in Librispeech and TED-LIUM databases respectively, when both x-vectors and i-vectors are concatenated with acoustic features and used for adaptation. Third, in order to enable streaming ASR, we propose an enrollment-test mechanism where a target speaker is enrolled and the enrollment x-vector is appended with the online i-vectors during inference. We show that such concatenation of x-vectors and i-vectors for adaptation improves the performance of speech recognition systems in both offline and streaming mode.