Effects of Delayed Visual Feedback on Brain Activity: A Near-Infrared Spectroscopy Study

Human-machine interface (HMI) devices, such as touch screens and rotary commander controls used in car navigation systems, have become multifunctional and complicated to operate, requiring better usability. Nevertheless, response latency (time lag between operation input and display output) has a negative impact on the usability of these devices because it reduces the operator's sense of agency (SoA), -the feeling that they are the ones causing the action. SoA and stress information can be obtained by measuring physiological indicators, such as brain activity. This study was designed to investigate the effects of response latency on the operator's brain activity state through near-infrared spectroscopy, which has been used to measure the brain activity state based on changes in the concentration of oxygenated hemoglobin
$(\Delta\text{Oxy}-\text{Hb})$
. In this study, 20 subjects performed a task in which they manipulated a cursor on a screen with an HMI commander rotation, which was used as an in-vehicle device. The temporal response latency (0, 50, 100, and 150 ms) between manipulating the commander and display response was introduced, and the subjects' brain activity, subjective evaluation, and performance efficiency were measured. The results showed that
$\Delta \text{Oxy}-\text{Hb}$
at the right end of the prefrontal cortex (PFC) increased significantly under 150 ms latency in relation to those under other conditions, whereas the performance efficiency did not decrease. Moreover, a significant difference in the
$\Delta\text{Oxy}-\text{Hb}$
of the PFC was observed between the latency recognition and non-recognition groups. Thus, we conclude that a 150 ms latency activates the PFC owing to SoA loss and the perception of stress and brain activity differs depending on the recognition of the latency.