Improved Features and Models for Detecting Edit Disfluencies in Transcribing Spontaneous Mandarin Speech

Detection of edit disfluencies is key to transcribing spontaneous utterances. In this paper, we present improved features and models to detect edit disfluencies and enhance transcription of spontaneous Mandarin speech using hypothesized disfluency interruption points (IPs) and edit word detection. A comprehensive set of prosodic features that takes into account the special characteristics of edit disfluencies in Mandarin is developed, and an improved model combining decision trees and maximum entropy is proposed to detect IPs. This model is further adapted to desired prosodic conditions by latent prosodic modeling, a probabilistic framework for analyzing speech prosody in terms of a set of latent prosodic states. These techniques contribute to higher recognition accuracy (by rescoring with the hypothesized IPs) and better edit word detection (using conditional random fields defined on Chinese characters) in the final transcription, as verified by experiments on a spontaneous Mandarin speech corpus.

I. Introduction

There has been significant progress in spoken language processing in recent years. A main goal for developing such technologies is the construction of effective human-centric information systems, a required feature of which is that the user communicates with the system using natural language. This necessitates a hands-free speech interface that can recognize and understand human language in the form of spontaneous utterances. Also, information is increasingly available in the form of multimedia, such as broadcast programs, meeting, and lecture recordings, in which speech signals are very often produced spontaneously. Processing and extracting information from spontaneous utterances in such multimedia content thus is another highly desired key feature of human-centric information systems.