DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models

DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models

2 Feb 2024 | Mohammadreza Pourreza, Davood Rafiei
This paper introduces DTS-SQL, a two-stage fine-tuning approach for text-to-SQL tasks that decomposes the task into schema-linking and SQL generation. The method uses two smaller large language models (LLMs) with 7 billion parameters each to improve execution accuracy by 3-7% compared to conventional single-step fine-tuning. The approach is evaluated on two large cross-domain datasets, Spider and Spider-SYN, and two 7B LLMs, DeepSeek and Mistral. Results show that the method achieves performance comparable to methods using GPT-4 with few-shot learning and well-designed prompts. The method also outperforms previous open-source methods on the Spider development set and achieves results comparable to state-of-the-art open-source methods on the Spider test set. The paper discusses the methodology, including supervised fine-tuning for Text-to-SQL, and presents results on various metrics, including exact set match accuracy and execution accuracy. The approach is shown to be effective in improving the performance of small open-source models, enabling them to rival larger proprietary models. The paper also discusses limitations, including the need for further research into schema-linking, and emphasizes the importance of ethical considerations in research. The authors provide all necessary code and models to replicate the results in their GitHub repository.This paper introduces DTS-SQL, a two-stage fine-tuning approach for text-to-SQL tasks that decomposes the task into schema-linking and SQL generation. The method uses two smaller large language models (LLMs) with 7 billion parameters each to improve execution accuracy by 3-7% compared to conventional single-step fine-tuning. The approach is evaluated on two large cross-domain datasets, Spider and Spider-SYN, and two 7B LLMs, DeepSeek and Mistral. Results show that the method achieves performance comparable to methods using GPT-4 with few-shot learning and well-designed prompts. The method also outperforms previous open-source methods on the Spider development set and achieves results comparable to state-of-the-art open-source methods on the Spider test set. The paper discusses the methodology, including supervised fine-tuning for Text-to-SQL, and presents results on various metrics, including exact set match accuracy and execution accuracy. The approach is shown to be effective in improving the performance of small open-source models, enabling them to rival larger proprietary models. The paper also discusses limitations, including the need for further research into schema-linking, and emphasizes the importance of ethical considerations in research. The authors provide all necessary code and models to replicate the results in their GitHub repository.
Reach us at info@study.space
[slides] DTS-SQL%3A Decomposed Text-to-SQL with Small Large Language Models | StudySpace