Exploring the Viability of Sequence-to-Sequence Models for Automatic Speech Recognition in Peer-to-Peer Learning Environments

Sequence-to-sequence (Seq2Seq) models have become a cornerstone in natural language processing (NLP) and se-quence generation tasks, with applications ranging from machine translation to automatic speech recognition (ASR). While these models are well-studied in centralized and federated learning contexts, their application in peer-to-peer (P2P) deep learning remains underexplored. This study investigates the feasibility of deploying Seq2Seq models for ASR tasks in P2P environments, where agents, such as edge devices, train local models using private datasets and exchange these models directly with peers. Using the UserLibri and LJ Speech datasets, we simulate P2P learning with different communication strategies, assessing model performance using the Word Error Rate (WER) metric. Our findings reveal that while P2P learning requires more iterations to achieve convergence compared to centralized training, Seq2Seq models can be effectively utilized in such decentralized settings. The experiments highlight the critical role of local data availabil-ity in model performance, suggesting that the quantity of local data significantly impacts WER.