Enhancing Vocal Melody Extraction With Multilevel Contexts

This paper proposes a convolutional recurrent neural network model for vocal melody extraction. It is deeper and wider than existing models in terms of the following aspects. First, it considers multilevel contexts that are a natural reflection of the process followed by humans in recognizing melody, whereas existing models usually consider only one level of contexts. Second, its overall receptive field used to capture temporal-frequency patterns is larger than most existing models. This aspect is also critical for humans to recognize melody. Third, it can enforce the dependency between neighboring melody pitches and thus avoid abrupt change in the extracted melody pitches. The proposed model outperforms existing models on multiple datasets. Additionally, it can achieve the highest performance with least training data.