Attention on Attention as a part of decoder for the vehicle license plate recognition

The attention mechanism is getting widely used in modern research papers to solve computer vision problems by showing great results, even optical character recognition tasks have multiple attention-based solutions. In this research paper, I would like to know how is actually attention-based architectures are applicable to the vehicle license plate recognition problem, and compare them with the recurrent neural networks-based approach. I implement two different decoder blocks to decode extracted features from CNN layers, the first decoder is based on the bidirectional gated recurrent unit (BiGRU), and the second decoder is based on the attention-on-attention (AoA) technique. Connectionist temporal classification (CTC) [1] is applied for sequence labeling. The proposed methods were trained on the RodoSol-ALPR dataset [2] for 1000 iterations. Experiments show that the architecture with the AoA technique is less accurate and challenging to train than the BiGRU method.