Self, Semi and Fully Supervised Learning for Autoencoders using Ternary Classification

Autoencoders are widely used in anomaly detection, due to their ability to learn from unlabeled data. They are commonly trained in a self-supervised way on HEALTHY examples, and they are tested on a combination of HEALTHY and FAULTY data. According to research literature, autoencoders can be trained in a fully supervised fashion, to obtain greater reconstruction errors when the inputs are FAULTY. However, it is hard to binary classify data coming from equipment that has been in operation for a long time and which may be worn down. In addition, the current health status of the equipment may be unknown. Moreover, historical data is not usually labeled, and mostly maintenance interventions are recorded. To alleviate this problem, a third label is introduced, UNKNOWN, which enables the autoencoder to learn the structure of HEALTHY and FAULTY data from the correspondingly labeled data points. The structure is used in reconstructing the UNKNOWN inputs. This can increase the performance of autoencoders in a wide range of anomaly detection use cases, including for industrial machines which have been in operation for a long time. Classical loss functions, such as MSE and L2-norm, are modified through the addition of new hyperparameters. This enables the loss functions to consider label information. The new methodology enables the training of an autoencoder in any combination of self, semi and fully supervised fashions. The drawback of these new label-aware loss functions is the extra hyperparameters which should be optimized.