vBeats: A Framework for Converting Gestures into Bass Strokes from Indian Tabla using Smartphones

Human gestures while playing instruments are highly critical. A small incorrect gesture can result in undesirable noise, disturbing the overall synchronization of the orchestra. Certainly, the most obvious way of honing the skills in playing instruments can be accomplished with dedicated practice, albeit the learner must always possess the instrument. In this paper, we propose an initial version of the framework vBeats that allows learners to use the COTS smartphones to practice sophisticated percussion-based instruments like tabla without physically having it. Preliminary results on an in-house dataset, with offline inferencing, show that vBeats can be used to generate similar waveforms; however, it may need accurate fine-tuning, which we discuss as a part of future work.

I. Introduction

For ages, music has been a major source of entertainment, and technological advancement has contributed heavily to how humans produce and perceive music. Interestingly, in the last few years, especially during the pandemic, many people across the globe explored the idea of learning how to play different musical instruments mostly to spend their leisure time and to reduce stress [1]. However, one of the primary necessities that limit this self-learning is the requirement of the intended instrument. Subsequently, different solutions have emerged, ranging from simple applications that provide a simulated software version of the instruments like guitars [2] to sophisticated systems like [3], [4], which can act as a proxy when the actual instrument is unavailable.