Data Mode Related Interpretable Transformer Network for Predictive Modeling and Key Sample Analysis in Industrial Processes

Accurate prediction of quality variables that are difficult to measure is crucial for industrial process control and optimization. However, the fluctuations in raw material quality and production conditions may cause industrial process data to be distributed in multiple working conditions. The data under the same working condition show similar characteristics, which are often defined as one data mode. Hence, the overall process data exhibit multimode characteristics, which brings great challenges in developing a uniform prediction model. Besides, the noninterpretability of the existing data-driven prediction models brings great resistance to their practical application. To address these issues, this article proposes a novel data mode related interpretable transformer network (DMRI-Former) for predictive modeling and key sample analysis in industrial processes. In DMRI-Former, a novel data mode related interpretable self-attention mechanism is designed to enhance the homomode perceptual ability of each individual mode while also capturing cross-mode features of different modes. Moreover, the key samples under different modes can be discovered using DMRI-Former, which further improves the interpretability of the modeling process. Finally, the superiority of the proposed DMRI-Former is verified in two real-world industrial processes compared to other state-of-the-art methods.

I. Introduction

Under the background of carbon peaking and carbon neutrality, industrial processes urgently seek intellectual transformation and upgrading, with real-time monitoring, control, and optimization of processes being among the most important tasks [1], [2]. Usually, real-time measurement of key quality variables is the most effective reflection of the industrial manufacturing state. Unfortunately, due to the limitations of measurement techniques and the industrial environment, most of these quality variables cannot be measured in time [3]. This leads to large time delays in industrial process control and optimization [4]. In this context, the soft sensor techniques for predicting difficult-to-measure quality variables using easy-to-measure process variables emerge as time requires [5], [6].