Investigate the Effects of Background Music on Visual Cognitive Tasks Using Multimodal Learning Analytics

Music is a popular form of entertainment and has become common practice to adjust cognition, affect, and motivation. Regarding the effects of background music on learning tasks, results are inconclusive in the literature. Recent advancement of wearable devices and computing analytics supports automated detection of multimodal physiological signals, such as eye movements, neural responses, and heart rates in a real-time fashion, which can facilitate tracking learners’ changes of affect, attention, and cognition while they study with the accompaniment of background music. However, most existing studies focus only on behavioral levels, and few employed signals at physiological levels to investigate the impact of background music on learning. To fill in the research gap, this doctoral project designs two types of visual cognitive tasks, that is, reading comprehension task and art appreciation task. It aims to integrate multimodal data (e.g., eye movements, electroencephalogram (EEG), and peripheral physiological signals) to probe the effect of background music on the tasks. Its findings will extend our knowledge on the interactions among learners’ performance, emotion, and engagement at both physiological and behavioral levels in multi-channel learning settings, and contribute to a goal of recommending suitable background music for self-learning.