Audio Source Count Estimation using Deep Learning

Modern technology applications which respond according to human voice commands need to accurately identify the number of concurrent speakers present in a single-channel mixture. In daily communication, several speakers speak simultaneously resulting in the generation of overlapping speech signals. To separate the speakers using unsupervised learning, estimating the maximum number of speakers present in a mixed-signal is essential. In several speech-based applications like speaker diarisation, blind source separation, and crowd monitoring, estimating the source count is the initial step. We propose a unifying model based on the intensity variation of the frequencies in the time domain. For designing an architecture that makes the model independent of the speaker's dialect, identity and verbal message use of Short-Time Fourier Transform (STFT) as a feature is helpful. The sparsity in the spectrogram varies depending upon the number of concurrent speakers. This feature is fed to the supervised multi-class Convolutional Neural Network (CNN) which predicts the number of speakers present in the audio segments of 1s each for the entire audio file of a specified duration. The predicted results are verified with Voice Activity Detection (VAD) plots of individual speakers to ensure that the ground truth holds good.