15 Apr 2024 | Md Mahadi Hasan Nahid, Davood Rafiei
TabSQLify is a novel method that enhances the reasoning capabilities of large language models (LLMs) by decomposing tables into smaller, relevant sub-tables. This approach leverages text-to-SQL generation to extract essential information for answering questions or verifying statements, reducing the input context length and improving scalability for large-scale table reasoning tasks. The method involves two key steps: (1) generating SQL queries from natural language questions or statements and executing them on the original tables to obtain sub-tables containing only essential information, and (2) using LLMs with the sub-table and the question or claim to generate the answer.
TabSQLify was evaluated on four challenging datasets: WikiTQ, FeTaQA, TabFact, and WikiSQL. On WikiTQ, it achieved an accuracy of 64.7%, surpassing other LLM-based baselines. On TabFact, it achieved an accuracy of 79.5%, outperforming existing methods. The method significantly reduces the input length, making it more scalable and efficient for large-scale table reasoning applications.
The approach also demonstrates robustness and scalability, as shown in experiments where the model performed well even when tables were truncated to meet token limits. TabSQLify reduces the size of tables significantly, making it more efficient for LLMs to process large tables without compromising performance. The method provides an intermediate representation (SQL queries and sub-tables) that is more interpretable and explainable for tracing and verification purposes.
TabSQLify's ability to decompose tables into smaller, relevant sub-tables allows for more focused reasoning and reduces the burden on LLMs in table encoding and reasoning. The method is particularly effective for handling large tables that exceed the maximum allowable context window of LLMs. The results show that TabSQLify achieves high accuracy on various table reasoning tasks, demonstrating its effectiveness in improving the reasoning capabilities of LLMs through table decomposition.TabSQLify is a novel method that enhances the reasoning capabilities of large language models (LLMs) by decomposing tables into smaller, relevant sub-tables. This approach leverages text-to-SQL generation to extract essential information for answering questions or verifying statements, reducing the input context length and improving scalability for large-scale table reasoning tasks. The method involves two key steps: (1) generating SQL queries from natural language questions or statements and executing them on the original tables to obtain sub-tables containing only essential information, and (2) using LLMs with the sub-table and the question or claim to generate the answer.
TabSQLify was evaluated on four challenging datasets: WikiTQ, FeTaQA, TabFact, and WikiSQL. On WikiTQ, it achieved an accuracy of 64.7%, surpassing other LLM-based baselines. On TabFact, it achieved an accuracy of 79.5%, outperforming existing methods. The method significantly reduces the input length, making it more scalable and efficient for large-scale table reasoning applications.
The approach also demonstrates robustness and scalability, as shown in experiments where the model performed well even when tables were truncated to meet token limits. TabSQLify reduces the size of tables significantly, making it more efficient for LLMs to process large tables without compromising performance. The method provides an intermediate representation (SQL queries and sub-tables) that is more interpretable and explainable for tracing and verification purposes.
TabSQLify's ability to decompose tables into smaller, relevant sub-tables allows for more focused reasoning and reduces the burden on LLMs in table encoding and reasoning. The method is particularly effective for handling large tables that exceed the maximum allowable context window of LLMs. The results show that TabSQLify achieves high accuracy on various table reasoning tasks, demonstrating its effectiveness in improving the reasoning capabilities of LLMs through table decomposition.