Neural Architecture Search Applied to Autoencoders Trained on Noisy Industrial Data: Single and Multi-Objective Investigations

Neural Architecture Search (NAS) has emerged as a powerful technique for automatically designing neural network architectures, offering the potential to discover models with superior performance and efficiency compared to manually designed architectures. In this study, we explore the application of NAS to autoencoder architectures, aiming to find models that provide the best trade-off between model complexity and reconstruction accuracy. Initially, we manually designed and evaluated several autoencoder architectures based on intuition, seeking to identify a baseline for comparison. Subsequently, we employed a Differential Evolution (DE) approach to perform NAS, searching for architectures that minimize the number of parameters, the number of nodes in the Latent Layer, and the Mean Absolute Percentage Error (MAPE) simultaneously. Our results demonstrate that NAS can identify models with improved objective values compared to manually designed architectures. Furthermore, we developed a Non-dominated Sorting Differential Evolution (NSDE) approach tailored for multi-objective optimization in NAS. We applied NSDE to the same set of objectives and observed its ability to identify diverse and high-quality architectures.

I. Introduction

Training neural networks involves the intricate process of optimizing the model's parameters to minimize a predefined loss function, typically accomplished through gradient descent-based algorithms like backpropagation [1]. While this method has delivered impressive results, the manual design of the optimal neural network architecture remains a formidable challenge. Manually crafted architectures may fail to fully explore the vast design space, leading to suboptimal solutions.