Score Genaration for Taiko no Tatsujin Based on Machine Learning

Taiko no Tatsujin is one of the most popular rhythm-based games. Players simulate playing a drum in time with music. In order for players to play songs, an additional percussion part need to be composed based on the audio an existing song, the additional percussion part is not exactly the original percussion part of the song, but it is related to the melody, rhythm and other feathers of the song. This paper introduces two ways to generate the score of the additional percussion part that follow beat and melody of songs. One is that we split the raw audio into many measures, record some certain features of each measure, and build a database for measures and a database for features whose relationship is one on one. Using KNN model, we find every predicted measure for a target song which becomes the score in a whole. In another way, we pre-process the raw audio to extract MFCCs feature and utilize onset detection using CNN to classify notes at any time. Through these approaches, the score can be created in 5 seconds while its approach 86% similarity of the human written score.