End-to-End Tibetan-Chinese Speech Translation Based on Multi-task and Multi-level Pre-training

End-to-end speech translation is a technology capable of translating speech in the source language directly into text in the target language, and it can mitigate the propagation errors that cascade system can introduce. Nevertheless, compared to relatively mature speech recognition and machine translation, the present challenge of end-to-end speech translation is the lack of parallel corpus data for low-resource minority languages. This creates obstacles in building efficient speech translation models. In this paper, we propose a model for end-to-end Tibetan-Chinese speech translation based on the multi-task and multi-level pre-training strategy to address the issue of scarce Tibetan-Chinese parallel corpus data. Our model extends the Transformer speech translation model with a speech recognition decoder and constructs a multi-task learning model that shares the encoder. Moreover, we utilize high-resource ST training data and ASR training data and adopt a multi-level pre-training strategy during the training process to improve the generalization of the Tibetan-Chinese speech translation model. Experimental results demonstrate that our proposed method effectively enhances the ability of Tibetan-Chinese speech translation in low-resource environments, outperforming both the end-to-end baseline models and simple multi-task learning models.