The Xmuspeech System for Audio-Visual Target Speaker Extraction in Misp 2023 Challenge

This paper presents the XMUSPEECH system for the Multimodal Information based Speech Processing (MISP) 2023 Challenge. The primary difficulty in this competition lies in effectively suppressing noise in mixture while maintaining the integrity of the target speaker’s speech signal. To achieve this, we introduce the AV-MCCMGAN, a multi-modal multichannel speech enhancement model. This model employs the raw 6-channels audio as a supplement to the guided source separation (GSS) signal and utilizes the visual clues of the target speaker to further eliminate noise. Ultimately, our system achieves the character error rate (CER) values of 23.3% and 33.41% on the Dev and Eval set, ranking the third.