Source Separation by Skip-Connection Based on Transformer Architecture in the Time-Frequency Domain

There are many end-to-end speech source separation models based on the information of the input audio signal at the time domain. The information on the frequency domain plays an important role in audio processing. In this study, we modified the dual-path transformer network (DPT-Net) with additional information on the time-frequency distribution. To the U-Net, we added the skip connections between the encoder and the decoder acting on the time-frequency distribution. In the experiment, the modification produced a better result than other methods of similar size.