Smart headphones: enhancing auditory awareness through robust speech detection and source localization

We describe a method for enhancing auditory awareness by selectively passing speech sounds in the environment to the user. We develop a robust far-field speech detection algorithm for noisy environments and a source localization algorithm for flexible arrays. We then combine these methods to give a user control over the spatial regions from which speech will be passed through. Using this technique, we have implemented a "smart headphones" system in which a user can be listening to music over headphones and hear speech from specified directions mixed in. We show our preliminary results on the algorithms and describe initial user feedback about the system.