Asynchronously Parallel Decoding For Automatic Speech Recognition Services

We proposed a new automatic speech recognition (ASR) service architecture that is extendable to medium-scale ASR service and more flexible than the previous architecture. Improvement aims to substitute the distributed processing approach with an asynchronous parallel thread for decoding multiple voice streams. We replace our TCP-based communication protocol with a remote procedure call developed by Google (gRPC) that makes our ASR service become a developer-friendly, less overhead connection. Besides, the API gateway is employed to reinforce the ASR services by multiple servers so that we can increase our new ASR service to a larger scale. The experimental result shows that our new architecture performs faster than the previous architecture in terms of real-time factor.