Human Behavior Modeling in Speech Transcribing Process via Pretrained Speech Recognition Models

The process of transcribing speech plays a critical role in developing automatic speech recognition (ASR) systems. During this process, human transcribers listen to spoken utterances and convert them into accurately written texts. However, our initial investigations using telephone conversation datasets have shown that approximately 40% of the recorded utterances are challenging to transcribe and often disregarded by transcribers due to various factors, such as unintelligibility. As a result, transcribers waste significant time and effort listening to such utterances, significantly slowing down the overall transcribing speed. In this paper, we explore the potential of pretrained speech recognition models to model transcribers’ behavior in the speech transcribing process. By leveraging the knowledge and insights encoded in these models, we aim to filter out problematic utterances that transcribers typically disregard before they reach human transcribers, thus optimizing their time and effort. Through experiments on practical telephone datasets, our approach utilizing pretrained speech recognition models successfully reduces unwanted utterances by 50% while preserving the high quality of the collected dataset.