Static analysis for optimizing big data queries
| Autor: | Mike Barnett, Madanlal Musuvathi, Zvonimir Pavlinovic, Edgardo Zoppi, Diego Garbervetsky, Todd Mytkowicz |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
Database
business.industry Programming language Computer science Relational database Big data 020207 software engineering 02 engineering and technology Static analysis computer.software_genre Query optimization Query language Scripting language 020204 information systems 0202 electrical engineering electronic engineering information engineering Table (database) business computer Scope (computer science) |
| Zdroj: | ESEC/SIGSOFT FSE |
| Popis: | Query languages for big data analysis provide user extensibility through a mechanism of user-defined operators (UDOs). These operators allow programmers to write proprietary functionalities on top of a relational query skeleton. However, achieving effective query optimization for such languages is extremely challenging since the optimizer needs to understand data dependencies induced by UDOs. SCOPE, the query language from Microsoft, allows for hand coded declarations of UDO data dependencies. Unfortunately, most programmers avoid using this facility since writing and maintaining the declarations is tedious and error-prone. In this work, we designed and implemented two sound and robust static analyses for computing UDO data dependencies. The analyses can detect what columns of an input table are never used or pass-through a UDO unchanged. This information can be used to significantly improve execution of SCOPE scripts. We evaluate our analyses on thousands of real-world queries and show we can catch many unused and pass-through columns automatically without relying on any manually provided declarations. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|