An optimized distributed OLAP system for big data
| Autor: | Wenhao Chen, Haoxiang Wang, Xingming Zhang, Qidi Lin |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
Distributed database
Database business.industry Computer science Online analytical processing Big data 020206 networking & telecommunications 02 engineering and technology computer.software_genre Data type Metadata Data acquisition Data visualization 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Cache business computer |
| Zdroj: | 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA). |
| DOI: | 10.1109/ciapp.2017.8167056 |
| Popis: | To solve the problems of heterogeneous data types and large amount of calculation in making decision for big data, an optimized distributed OLAP system for big data is proposed in this paper. The system provides data acquisition for different data sources, and supports two types of OLAP engines, Impala and Kylin. First of all, the architecture of the system is proposed, consisting of four modules, data acquisition, data storage, OLAP analysis and data visualization, and the specific implementation of each module is descripted in great detail. Then the optimization of the system is put forward, which is automatic metadata configuration and the cache for OLAP query. Finally, the performance test of the system is conduct to demonstrate that the efficiency of the system is significantly better than the traditional solution. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|