Improving a real-time music alignment algorithm for opera performances

Music alignment is the process of matching two versions of a musical performance. We present an adaptation of a real-time music alignment algorithm to integrate within opera.guru, a mobile software solution to enhance the experience of attending an opera performance. Our algorithm is a proof-of-concept and is based on On-line Time Warping, a derivative of the well-studied Dynamic Time Warping, and offers effective improvements. Among those is a novel technique of cost matrix pre-processing called the Axes method. The enhanced solution was tested in simulated settings to measure the alignment precision of different methods. These tests showed good results in terms of computational performance and alignment error rate. However, our test dataset is of a limited scope and more thorough verification is required. Finally, we conducted user experience tests to study the applicability of our solution within the mobile solution in a real-world setting. According to these user tests, the alignment miscalculations proved to be mostly unnoticeable to the audience.