Introducing Schema Inference as a Scalable SQL Function [Extended Version]

Autor:	Dani, Calvin, Jahangiri, Shiva, Hütter, Thomas
Rok vydání:	2024
Předmět:	Computer Science - Databases
Druh dokumentu:	Working Paper
Popis:	This paper introduces a novel approach to schema inference as an on-demand function integrated directly within a DBMS, targeting NoSQL databases where schema flexibility can create challenges. Unlike previous methods relying on external frameworks like Apache Spark, our solution enables schema inference as a SQL function, allowing users to infer schemas natively within the DBMS. Implemented in Apache AsterixDB, it performs schema discovery in two phases, local inference and global schema merging, leveraging internal resources for improved performance. Experiments with real world datasets show up to a two orders of magnitude performance boost over external methods, enhancing usability and scalability. Comment: Extended version of EDBT 2025 submission
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2411.13278 Zobrazit plný text záznamu View this record from Arxiv