Joint Drum Transcription and Metrical Analysis Based on Periodicity-Aware Multi-Task Learning

This paper describes a multi-task learning method that detects the onset times of drums and the beat and downbeat times from a music signal. Since the drum part typically consists of repetitive patterns synchronized with the metrical structure in popular music, drum transcription and metrical analysis would benefit each other. The basic approach is to train a deep neural network (DNN) with a branching architecture that extracts latent features common to both tasks from a music signal and then feeds them to task-specific networks separately. The estimated frame-level posterior probabilities of drum onsets, beats, and downbeats, however, often have weak and inconsistent periodic structures. To solve this problem, we propose a regularized training method that encourages the three probability sequences to be highly auto- and cross-correlated. Specifically, our method aims to minimize the entropy of the auto-spectrum computed from each probability sequence and that of the cross-spectrum computed from each of the three sequence pairs, because a smaller entropy indicates a stronger element-wise periodicity or pairwise consistency. The experiment showed the mutual benefit of drum transcription and metrical analysis and the effectiveness of the periodicity- and consistency-based regularizations.