Performance Evaluation of Vector Embeddings with Retrieval-Augmented Generation

Vector embeddings form the basis of sophisticated language models. These language models were developed with the advent of developments in natural language processing (NLP) and aid in a variety of downstream tasks. Contextually relevant responses are improved using a combination of generation-based models and information retrieval, which is combined in the Retrieval-Augmented Generation (RAG) framework. The state-of-the-art research focuses on the RAG framework. Performance evaluation of vector embeddings in context with the RAG framework for data querying from documents is presented in this paper. The research encompasses a comparative analysis of various vector embeddings and their average, weighted average ensemble, evaluating their effectiveness in easing information retrieval and subsequent generation activities. The investigations focus on the impact of alternative embedding approaches on the overall performance of context generation across the NCERT books dataset using a systematic evaluation. The capabilities of ChatGPT and Llama2 are employed for evaluating the performance of embedding models. NCERT books form the underlying database, and LLM models are used to rank the contexts derived from the database. The optimized prompt is utilized to achieve ranking of the results. The same LLM is also utilized to generate the response for all the embedding models employing generated contexts. The variety in vector embedding approaches is exhibited by the experimental results.