Developmental Word Grounding Through a Growing Neural Network With a Humanoid Robot

This paper presents an unsupervised approach of integrating speech and visual information without using any prepared data. The approach enables a humanoid robot, Incremental Knowledge Robot 1 (IKR1), to learn word meanings. The approach is different from most existing approaches in that the robot learns online from audio-visual input, rather than from stationary data provided in advance. In addition, the robot is capable of learning incrementally, which is considered to be indispensable to lifelong learning. A noise-robust self-organized growing neural network is developed to represent the topological structure of unsupervised online data. We are also developing an active-learning mechanism, called "desire for knowledge", to let the robot select the object for which it possesses the least information for subsequent learning. Experimental results show that the approach raises the efficiency of the learning process. Based on audio and visual data, they construct a mental model for the robot, which forms a basis for constructing IKR1's inner world and builds a bridge connecting the learned concepts with current and past scenes

I. Introduction

As human beings, we discriminate various concepts through formation of “iconic representations” of them. Judgments of their resemblance or difference are based on similarity and difference comparisons of these iconic representations [1]. We also interpret their meaning through language. Therefore, in a sense, we ground the meanings of language to its perceptual context. For a robot to be more like a human, it must understand the sound patterns of words and understand their meanings. It must ground language in its world as mediated by its perceptual, motor, and cognitive capacities. Under such a scenario, it must analyze the current scene along with the associated utterance, integrate the extracted information, and then finally acquire their meanings.