Scaling-up in-memory datalog processing
| Autor: | Zuyu Zhang, Zhiwei Fan, Aws Albarghouthi, Paraschos Koutris, Jianqiao Zhu, Jignesh M. Patel |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Theoretical computer science Computer science General Engineering Databases (cs.DB) 02 engineering and technology computer.software_genre Relational system Datalog Program analysis Computer Science - Databases Margin (machine learning) 020204 information systems 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Baseline (configuration management) computer Scaling computer.programming_language Data integration |
| Zdroj: | Proceedings of the VLDB Endowment. 12:695-708 |
| ISSN: | 2150-8097 |
| Popis: | Recursive query processing has experienced a recent resurgence, as a result of its use in many modern application domains, including data integration, graph analytics, security, program analysis, networking and decision making. Due to the large volumes of data being processed, several research efforts across multiple communities have explored how to scale up recursive queries, typically expressed in Datalog. Our experience with these tools indicate that their performance does not translate across domains---e.g., a tool designed for large-scale graph analytics does not exhibit the same performance on program-analysis tasks, and vice versa. Starting from the above observation, we make the following two contributions. First, we perform a detailed experimental evaluation comparing a number of state-of-the-art Datalog systems on a wide spectrum of graph analytics and program-analysis tasks, and summarize the pros and cons of existing techniques. Second, we design and implement our own general-purpose Datalog engine, called RecStep, on top of a parallel single-node relational system. We outline the techniques we applied on RecStep, as well as the contribution of each technique to the overall performance. Using RecStep as a baseline, we demonstrate that it generally out-performs state-of-the-art parallel Datalog engines on complex and large-scale Datalog evaluation, by a 4-6X margin. An additional insight from our work is that it is possible to build a high-performance Datalog system on top of a relational engine, an idea that has been dismissed in past work. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|