A Neural Network Based Approach for Combining Ultrasound and MRI Data of 2-D Dynamic Records of Human Speech

We aim at exploring the geometrical information that can be extracted from dynamic two-dimensional audiovisual records created by magnetic resonance imaging (MRI) and ultrasound (US) techniques during human speech. We connect US and MRI data by machine learning using tongue contours fitted automatically to the MRI and US images. We create different system configurations depending on the type and number of the input and output parameters of the network and the number of the hidden layers and neurons. We perform qualitative and quantitative analyses for all settings. The main benefit of this approach is to better understand the role of the geometric parameters of the vocal tract in speech production and to create a possible way to harmonize MRI and US sources.