MultiLingualSync: A Novel Method for Generating Lip-Synced Videos in Multiple Languages

We present a Multi Lingual Sync model for generating lip-synced videos in multiple languages. The model consists of Lingua Speak for translation and Wav2Lip for lip synchronization. The workflow involves extracting audio from a video, translating it using Lingua Speak, converting the translated text to speech, and then using Wav2Lip to generate synchronized lip movements. Wav2Lip utilizes a Generator with an identity encoder, speech encoder, and face decoder, along with a pre-trained lip-sync discriminator called SyncNet. The proposed model enables the creation of accurate and efficient multilingual videos with synchronized speech and video.