Frame-Level Matching Scheme Using Posteriorgram Probability Distance of Spoken Data to Improve Search Accuracy of Spoken Term Detection

Active research has been conducted of Spoken Term Detection (STD), which uses text queries (search words) to seek specific scenes in large amounts of data, including audio data. In subword-to-subword and state-to-state matching using results of automatic speech recognizer (ASR) for speech data, if a query includes an out-of-vocabulary (OOV) word that does not exist in the lexicon of ASR, retrieval accuracy deterioration occurs. Frame-level matching performs more detailed matching at the frame level using a maximum likelihood series that arranges the most probable states for each frame of the posterior probability vector, which is the output of a Deep Neural Network (DNN) used as an acoustic model for ASR, whereas subword-to-subword or state-to-state uses preconstructed acoustic distances between subwords or states. For query by example (QbE), which uses spoken queries, high retrieval accuracy is obtained using the posterior probability vector of speech data. As described herein, we propose a new frame-level matching scheme for a text query using the posterior probabilities of speech data instead of the preconstructed acoustic distance between subwords of states. Thereby, the retrieval accuracy is improved. Machine learning models of two types with different numbers of states are introduced to improve the retrieval accuracy further. Because each model shows different retrieval results, integration of these models is expected to be mutually complementary, thereby improving the retrieval accuracy. Experiments were conducted using public test collections of the SpokenDoc task from NTCIR-10 and NTCIR-12 workshops as benchmarks. Results indicate that the proposed method outperforms the best accuracy obtained at the workshop by 26.95 points for NTCIR-10 and 4.44 points for NTCIR-12. These results demonstrate the effectiveness of the proposed method.