Shasta
| Autor: | Jeffrey D. Ullman, Jun Xu, Chad Whipkey, Nitin Gangahar, Stephan Ellner, Alejandro Estrella-Balderrama, Bart Samwel, Gokul Nath Babu Manoharan, Himani Apte, Karl Schnaitter, Stephan Gudmundson, Larysa Aharkava, Ben Handy, Divyakant Agrawal, Apurv Gupta, Sridatta Chegu, Shivakumar Venkataraman |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
SQL
Database Computer science Materialized view 020206 networking & telecommunications 02 engineering and technology computer.software_genre Query language User-defined function Data warehouse Relational database management system 020204 information systems Middleware (distributed applications) Schema (psychology) Scalability 0202 electrical engineering electronic engineering information engineering Cache computer Transaction data computer.programming_language |
| Zdroj: | SIGMOD Conference |
| Popis: | We describe Shasta, a middleware system built at Google to support interactive reporting in complex user-facing applications related to Google's Internet advertising business. Shasta targets applications with challenging requirements: First, user query latencies must be low. Second, underlying transactional data stores have complex "read-unfriendly" schemas, placing significant transformation logic between stored data and the read-only views that Shasta exposes to its clients. This transformation logic must be expressed in a way that scales to large and agile engineering teams. Finally, Shasta targets applications with strong data freshness requirements, making it challenging to precompute query results using common techniques such as ETL pipelines or materialized views. Instead, online queries must go all the way from primary storage to user-facing views, resulting in complex queries joining 50 or more tables. Designed as a layer on top of Google's F1 RDBMS and Mesa data warehouse, Shasta combines language and system techniques to meet these requirements. To help with expressing complex view specifications, we developed a query language called RVL, with support for modularized view templates that can be dynamically compiled into SQL. To execute these SQL queries with low latency at scale, we leveraged and extended F1's distributed query engine with facilities such as safe execution of C++ and Java UDFs. To reduce latency and increase read parallelism, we extended F1 storage with a distributed read-only in-memory cache. The system we describe is in production at Google, powering critical applications used by advertisers and internal sales teams. Shasta has significantly improved system scalability and software engineering efficiency compared to the middleware solutions it replaced. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|