Pseudoinvertible Neural Networks

This work introduces novel architecture components and training procedures to create augmented neural networks with the ability to process data bidirectionally via an end-to-end approximate inverse. We develop pseudoinvertible neural network (PsI-NN) layers which function as drop-in replacements for corresponding convolutional and fully connected layers; by using these, existing architectures gain a pseudoinverse function which, with training, approximately reverses the forward function. For cases where learning both a task and its inverse are necessary or desirable, we show that PsI-NN enabled models match or exceed the quality of results generated by systems that use two separate models while drastically reducing system parameter count. We demonstrate this on two tasks: unpaired image translation and semisupervised classification. In both, PsI-NN greatly reduces parameter count without any loss of output quality; in semisupervised image classification, PsI-NN improves classification performance beyond the baseline autoencoder method. Our approach creates pseudoinvertible versions of existing architectures, circumventing the stringent constraints required to create a true inverse while allowing a single neural network to learn a task in two directions simultaneously.

I. Introduction

In TRAINING neural network systems, obtaining the inverse of a multilayer operation is often either desirable or necessary, as evidenced by the widespread use of approximate inverse networks whose objective includes the reconstruction of the input from the output such as in autoencoders [6] and unpaired image translation [3]. These solutions require a second model with mirrored tensor dimensions to be trained on the opposite task of the first network. While this approach has proven functional in practice, it usually doubles the number of trainable parameters in the system and requires the mirrored model to be trained to approximately invert the function represented by the first model, which itself is already an approximation of the true mapping function. This approach has been necessary to obtain an approximation to a bijective mapping between two data distributions because multilayer neural networks are not invertible functions. In this article, we introduce pseudoinvertible neural networks (PsI-NNs), a class of neural networks with the capability to process data both forwards and backwards, which through training enables them to approximate a bijective mapping function with a single model.