Designing and Implementing Data Warehouse for Agricultural Big Data
| Autor: | Ngo, Vuong M., Le-Khac, Nhien-An, Kechadi, M-Tahar |
|---|---|
| Rok vydání: | 2019 |
| Předmět: | |
| Zdroj: | BigData 2019 |
| Druh dokumentu: | Working Paper |
| Popis: | In recent years, precision agriculture that uses modern information and communication technologies is becoming very popular. Raw and semi-processed agricultural data are usually collected through various sources, such as: Internet of Thing (IoT), sensors, satellites, weather stations, robots, farm equipment, farmers and agribusinesses, etc. Besides, agricultural datasets are very large, complex, unstructured, heterogeneous, non-standardized, and inconsistent. Hence, the agricultural data mining is considered as Big Data application in terms of volume, variety, velocity and veracity. It is a key foundation to establishing a crop intelligence platform, which will enable resource efficient agronomy decision making and recommendations. In this paper, we designed and implemented a continental level agricultural data warehouse by combining Hive, MongoDB and Cassandra. Our data warehouse capabilities: (1) flexible schema; (2) data integration from real agricultural multi datasets; (3) data science and business intelligent support; (4) high performance; (5) high storage; (6) security; (7) governance and monitoring; (8) replication and recovery; (9) consistency, availability and partition tolerant; (10) distributed and cloud deployment. We also evaluate the performance of our data warehouse. Comment: Business intelligent, data warehouse, constellation schema, Big Data, precision agriculture |
| Databáze: | arXiv |
| Externí odkaz: |
|