From Pixels to Waveforms: Evaluating Pre-trained Image Models for Few-Shot Audio Classification

Within machine learning literature, few-shot learning has emerged as a crucial tool in many use-cases, enabling adaptation to new tasks quickly with limited supervision. While the majority of this research has concentrated on the image domain, recent efforts have extended its scope to others. A prevalent strategy in addressing few-shot learning involves harnessing pre-trained models—whether supervised or otherwise—as feature extractors. These models, paired with a lightweight trainable head, demonstrate remarkable efficacy, often achieving near or state-of-the-art results. However, this method encounters challenges in domains like audio, where the number of available and regularly published pre-trained models is comparatively low. Given that audio signals can be represented as spectrograms, that are akin to traditional imagery, a fundamental question arises: Can off-the-shelf pre-trained image models prove beneficial for few-shot audio classification? Additionally, can insights from the image domain guide model and approach selection in the audio domain? Our investigation yields diverse insights, showcasing the effectiveness of both supervised and self-supervised image-pre-trained models for few-shot audio classification. Alongside this, we identify strong relationships between the common Cross-Domain few-shot imagery learning settings and few-shot audio performance.