Fine-Tuned training method for semantic text similarity measurement using SBERT, Bi-LSTM and Attention Network

The Semantic Textual Similarity (STS) problem measures the degree to which two text fragments are similar. In QA systems, it is a crucial tool for extracting useful information from documents. In STS challenges, optimizing lexical gaps between phrases has been a crucial issue. Text similarity may be affected by the distance between major parts of the sentence and the absence of context in the text. Numerous studies have developed answers to this issue. In recent years, however, Sentence-BERT has become widespread in semantic textual similarity tasks. Siamese BERT-Networks generate embeddings that are more context-aware and semantically intact. In addition, BiDirectional LSTM provides a deeper grasp of the input and captures long-range dependencies, whereas the Attention mechanism highlights the most important information aspects from the data. Due to the success of SBERT and Bi-LSTM in exploiting lexical gaps and semantics, we introduce SBERT-Bi-LSTM-Attention (SBiLA), a training model that combines SBERT, Bi-LSTM, and a simple attention strategy. The model improves similarity by addressing the lexical gap between short and long texts and reduces training loss to 0.052, MSE to 0.042, and RMSE to 0.206 on the SQuAD 2.0 and WikiHow datasets.

I. Introduction

Semantic Text Similarity (STS) measuring task is a field of computer science that employs statistical statistics to infer the significance of text data. It plays a crucial role in numerous natural language processing applications, including question-answering systems, chatbots, gene clustering, text summarization, plagiarism detection, and social network information retrieval [1]. Daily, a large number of internet users seek information from websites and make comments and replies to search requests, generating an enormous volume of text data. Such data is utilized by QA systems to determine the best feasible context-based response to inquiries. Due to the fact that many studies disregard contextual relationships, QA systems lose semantic information and may predict irrelevant outcomes. We extract questions and responses from two distinct contexts. First, we evaluate and extract important information from the question string, then we search for relevant information in the answer dataset, and finally we forecast the best potential answer.