Elevating Handwritten Mathematical Expression Recognition: Unveiling 2D Structural Insights Through Weak Supervision

Handwritten Mathematical Expression Recognition (HMER) poses a formidable challenge due to its intricate two-dimensional structure, variations in handwriting styles, and other complexities. The intricate nature of the problem offers numerous potential applications. The purpose of this paper is to suggest an architecture that utilizes weak supervision for extracting structural information from images, thereby enhancing the capabilities of HMER. We begin by defining a two-dimensional structural relationship by analyzing LaTeX syntax. Subsequently, we employ a decoder that utilizes Gated Recurrent Units (GRUs) to perceive the structural layout of handwritten expressions. Under weak supervision, Convolutional Neural Networks (CNNs) and a fully connected network work together to process the structure of mathematical expressions written by hand in two dimensions. Experimental results vividly demonstrate the superiority of this approach over alternative models, both in terms of structural and overall recognition accuracy. The model's recognition accuracy for the CROHME 2014 and 2016 datasets for online handwritten mathematical expressions is 55.13
^%
and 52.44
^%
, respectively. Additionally, the model achieves a structural recognition accuracy of 76.9% and 73.4%.