Sound Source Tracking by incorporating Target Motion estimated by Visual Trackers

It is one of the fundamental features for intelligent robots to track plural targets around them to recognize the environment, and various sensors including visual and acoustic devices have been utilized. Sound source tracking is one of such trackers required by the systems with microphones, but sound signals may include pauses when the system is not able to perceive the signal from the source, or multiple sound sources may locate very close to each other which makes it impossible for the system to distinguish sources only by acoustic information. In order to overcome this difficulty, this paper proposes to utilize visual trackers to estimate the motion of sources and to utilize them to track dynamic acoustic targets. The proposed algorithm incorporates Kalman filter that takes the motion vector from the visual trackers based on Kernelized Correlation Filters with estimating the uncertainty of the association between audio and visual features. Experiments to track dynamic acoustic sources validate the proposed method, which is also shown in this paper.