Learning-based Image Analytics in User-AI Agent Interactions for Cyber-enabled Manufacturing

Manual trial-and-error methods are employed for image parameter selection decisions in the processes that control cyber-enabled scientific instruments. Particularly in materials manufacturing use cases, where image analytics can be iterative, time-consuming and prone to errors, there is a need to enhance existing processes by using agents featuring learning algorithms that recommend image analytics parameters for users. In this paper, we present an analysis focused on identifying the optimal learning algorithm for an AI agent guiding real-time image processing tasks in carbon nanotube (CNT) manufacturing, which involves enhancing instrument control of cyber-enabled scanning electron microscope (SEM) instrument image scanning setup parameters such as Zoom, Focus, and Contrast. Specifically, we demonstrate the use of Reinforcement Learning (RL) and Imitation Learning (IL) based agents within a Remote Instrumentation Science Environment (RISE), a modularized system capable of utilizing multiple learning algorithms for guiding image analytics tasks. Further, we conduct a comparative performance analysis of RL and IL using 236CNT images captured in SEM material synthesis experiments. The experiments include configuring scanning parameters for CNT images generated by SEM, implementing CNT image segmentation, and assessing the effectiveness of RL and IL agents in identifying scanning parameters to enhance image quality. The objective is to predict Zoom, Focus, and Contrast parameters using limited labeled data for offline training, guiding dynamic adjustments in SEM settings. Our findings reveal that the IL agent outperforms the RL agent under dynamic conditions in characterizing CNT image parameters - specifically, Zoom, Focus, and Contrast - by evaluating image segmentation metrics.

I. Introduction

In data-intensive scientific workflows, particularly in the materials development field, as is the case of carbon nanotube (CNT) images characterization using a scanning electron microscope (SEM) [1], researchers deal with intricate manual tasks that involve instrument calibration, data acquisition and management, and complex data analysis. Additionally, the collected datasets involve multi-dimensional parameters [2], [3], requiring specialized analytical tools for meaningful insights. Conventional manual approaches are notably laborious and result in time-consuming and error-prone processes. Image characterization plays a crucial role in the field of CNT synthesis research. Researchers rely on these results to determine, in real-time, whether the CNT growth process is progressing as expected. This enables them to take prompt actions, such as updating SEM settings, to steer the experiment in the desired direction. Implementing an automated process to assist users in analyzing CNT images promptly will enhance their decision-making abilities, ultimately improving the outcomes of the experiment.