#llmsql
LLMSQL&Arabic WikiTableQA&Payment-SQL:SQL Dataset(202510) This month, the NL2SQL field saw the emergence of several noteworthy new datasets and benchmarks. Moving beyond the previous focu...

Origin | Interest | Match
LLMSQL&Arabic WikiTableQA&Payment-SQL:SQL Dataset(202510)
This month, the NL2SQL field saw the emergence of several noteworthy new datasets and benchmarks....
dev.to
November 27, 2025 at 9:58 AM
LLMSQL refreshes WikiSQL with clean natural‑language questions and plain‑text SQL queries, fixing case and syntax errors. Gemma 3 and OpenAI o4‑mini achieved the highest exact‑match scores. https://getnews.me/llmsql-a-refreshed-benchmark-for-text-to-sql-in-the-llm-era/ #llmsql #texttosql #llm
October 6, 2025 at 4:31 AM
LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL
Converting natural language questions into SQL queries (Text-to-SQL) enables non-expert users to interact with relational databases and has long been a central task for natural language interfaces to data. While the WikiSQL dataset played a key role in early NL2SQL research, its usage has declined due to structural and annotation issues, including case sensitivity inconsistencies, data type mismatches, syntax errors, and unanswered questions. We present LLMSQL, a systematic revision and transformation of WikiSQL designed for the LLM era. We classify these errors and implement automated methods for cleaning and re-annotation. To assess the impact of these improvements, we evaluated multiple large language models (LLMs), including Gemma 3, LLaMA 3.2, Mistral 7B, gpt-oss 20B, Phi-3.5 Mini, Qwen 2.5, OpenAI o4-mini, DeepSeek R1 and others. Rather than serving as an update, LLMSQL is introduced as an LLM-ready benchmark: unlike the original WikiSQL, tailored for pointer-network models selecting tokens from input, LLMSQL provides clean natural language questions and full SQL queries as plain text, enabling straightforward generation and evaluation for modern natural language-to-SQL models.
arxiv.org
October 6, 2025 at 4:18 AM
Dzmitry Pihulski, Karol Charchut, Viktoria Novogrodskaia, Jan Koco\'n
LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL
https://arxiv.org/abs/2510.02350
October 6, 2025 at 7:59 AM
Dzmitry Pihulski, Karol Charchut, Viktoria Novogrodskaia, Jan Koco\'n: LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL https://arxiv.org/abs/2510.02350 https://arxiv.org/pdf/2510.02350 https://arxiv.org/html/2510.02350
October 6, 2025 at 6:30 AM