Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing
| Autor: | Peter Shaw, Kenton Lee, Alane Suhr, Ming-Wei Chang |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
SQL
Parsing Database Computer science Generalization Database schema 020207 software engineering 02 engineering and technology computer.file_format Variation (game tree) computer.software_genre Set (abstract data type) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Executable computer Natural language computer.programming_language |
| Zdroj: | ACL |
| DOI: | 10.18653/v1/2020.acl-main.742 |
| Popis: | We study the task of cross-database semantic parsing (XSP), where a system that maps natural language utterances to executable SQL queries is evaluated on databases unseen during training. Recently, several datasets, including Spider, were proposed to support development of XSP systems. We propose a challenging evaluation setup for cross-database semantic parsing, focusing on variation across database schemas and in-domain language use. We re-purpose eight semantic parsing datasets that have been well-studied in the setting where in-domain training data is available, and instead use them as additional evaluation data for XSP systems instead. We build a system that performs well on Spider, and find that it struggles to generalize to our re-purposed set. Our setup uncovers several generalization challenges for cross-database semantic parsing, demonstrating the need to use and develop diverse training and evaluation datasets. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|