Mining Logic Patterns from Visual Data

Logic reasoning, a fundamental ability of human intelligence, is an important topic in artificial intelligence. Often, these reasoning patterns need to beforehand be provided by some domain experts, which has resulted in an interesting research topic: can reasoning patterns be directly learned from given data in which meaning of any symbols is not known in advance? In order to study this issue, in this study, we design four visual logic data set (named Fashion-Logic) using the Fashion-MNIST data, in which first each image is embedded with n special figure symbols each of which is a picture from the Fashion-MNIST data set, then a relation (the relations: Bitwise And, Bitwise Or, Addition and Subtraction are used in this paper) is added to any two images to generate one output image. And these resulting images are called the logical images. Our aim is to obtain a learning model that can directly mine and reason the logic relation between two input images and one output image, without presetting any reasoning patterns (i.e. a learning model does not know the meaning of the figure symbols embedded in the images and the relation between the input images and the output image in advance). This novel task, mining the logic patterns from the Fashion-Logic data set, is called a fashion logic task in this paper. In this work, we test the performances of many typical neural network models on Fashion-Logic data set, and produce fruitful results which indicates that it is feasible to directly mine the logic reasoning patterns using a learning strategy from given data. Last, we test and find that the ability of models is very poor to mine the abstract logic patterns on the more complex data sets called Fashion-LogicV2.