Building AR Virtual String Quartet for Live Concert

In this paper, we proposed the system for building virtual string quartets. This system is combined with three parts, which is animation control, sound synthesis, and central synchronization system. In the animation control part, we first design animation and use Unity for animation playback software. Next, the animation is controlled by the CSV file which is generated in the labeling stage. In the sound synthesis part, we analyze the music score and turn the expressive musical performance such as the velocity, duration, and dynamic of each note into the MIDI file. We develop a bowed-string instrument VSTi and use Reaper for sound playback software. Finally, we build the central synchronization system for synchronizing the animation with the sound in the real-time virtual music concert.