GAT-SQL: An Advanced Prompt Engineering Approach for Effective Text-to-SQL Interactions

In natural language processing, recent advancements in large language models (LLMs) have significantly impacted the text-to-SQL task, particularly in single-question interactions. However, multi-turn question interactions present unique challenges not fully addressed by current LLMs like GPT-3.5-turbo and GPT-4.5-turbo. In this paper, we perform a comprehensive and systematic analysis on a multi-turn interaction dataset known as SParC to compare various existing prompt engineering methods, including prompt representations and in-context learning methods. Following this, we present GAT-SQL, a novel prompt engineering approach based on three techniques: GAT representation, GAT reviser, and GAT verifier. Comparing our GAT representation and GAT verifier techniques to the previous methods of prompt engineering, they were very successful for the zero-shot experiments. GAT representations improve performance by an average of 2.9% EX and 3.8% IX across all existing question representation methods. While GAT verifier results in a greater improvement in accuracy by an average of 3.6% EX and 4% IX. Furthermore, with regard to in-context learning experiments, the GAT reviser achieved 77.4% EX and 59.9% IX, outperforming the best state-of-the-art model by 3.4% EX and 6.4% IX. As a further demonstration of the effectiveness of GAT-SQL, we tested it on another dataset of multi-turn interactions named CoSQL. GAT reviser achieved a new benchmark in the CoSQL competition, achieving 74.5% EX and 50.2% IX, higher than the closest baseline by 6.3% EX and 9.7% IX.