eCMU: An Efficient Phase-aware Framework for Music Source Separation with Conformer

In this paper, we attempt to build an affordable model to solve the music source separation (MSS) task in the spectral domain with limited computing resources. Our model optimizes the estimated complex spectrogram for each source in an end-to-end manner thanks to a differentiable Multi-channel Wiener Filter (MWF) and Multi-domain loss function. Therefore, the phase reconstruction for the estimated source is more optimal than using the noisy phase of the mixture or only applying MWF as a post-processing technique. Furthermore, motivated by the outperforming results of Convolution-augmented transformers (Conformers) in Automatic speech recognition (ASR) and Speech enhancement (SE), we apply Conformers blocks for the MSS problem to utilize their ability to capture both local and global feature dependencies on time and frequency axis. Our experiments on the MusDB18-HQ dataset show an improving result on the average SDR of about +0.21 dB, especially on vocals and other instruments while keeping the model size which is smaller than half of our baseline.